Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks
Date
2023
Authors
Trajanov, Dimitar
Lazarev, Georgi
Chitkushev, Lubomir T.
Vodenska, Irena
Version
Published version
OA Version
Citation
D. Trajanov, G. Lazarev, L. Chitkushev, I. Vodenska. 2023. "Comparing the Performance of ChatGPT and State-of-The-Art Climate NLP Models on Climate-related Text Classification Tasks" Proceedings of the 4th International Conference on Environmental Design (ICED2023).
Abstract
Recently, there has been a surge in general-purpose language models, with ChatGPT being the most advanced model to date. These models are primarily used for generating text in response to user prompts on various topics. It needs to be validated how accurate and relevant the generated text from ChatGPT is on the specific topics, as it is designed for general conversation and not for context-specific purposes. This study explores how ChatGPT, as a general-purpose model, performs in the context of a real-world challenge such as climate change compared to ClimateBert, a state-of-the-art language model specifically trained on climate-related data from various sources, including texts, news, and papers. ClimateBert is fine-tuned on five different NLP classification tasks, making it a valuable benchmark for comparison with the ChatGPT on various NLP tasks. The main results show that for climate-specific NLP tasks, ClimateBert outperforms ChatGPT.
Description
License
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http ://creativecommons.org/licenses/by/4.0/).