Investigation into Essays Generated by ChatGPT: Strengths and Weaknesses

Abstract

The use of artificial intelligence (AI) in various fields, especially language education has been on the rise in recent years. One such application is the generation of text through chatbots that mimic human conversation. Both teachers and students can employ AI in teaching and learning languages in general and writing in particular. In this study, we evaluate the essays written by ChatGPT, an AI language model developed by OpenAI. Four criteria are picked from IELTS writing band score descriptors, namely Task-responding, Coherence and Cohesion, Lexical Resource and Grammatical Range and Accuracy. The results showed that while ChatGPT essays had some positive aspects, such as coherence/cohesion and correct grammar, they lacked lexical variety and diverse grammatical structures. Furthermore, the essays generated by ChatGPT fell short of originality and critical thinking. Therefore, we recommend that ChatGPT essays should be used as a tool for assistance and not as a replacement for human writing. Following the analysis, some recommendations are given to both teachers and students.