Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions (Journal Article)
- USA :American Chemical Society ,May 2023
- 1905–1916p.
- American Chemical Society, Volume 100, Issue 5 .
***______________***
Abstract-
The artificial intelligence chatbot ChatGPT was used to answer questions from final exams administered in two general chemistry courses, including questions with closed-response format and with open-response format. For closed-response questions, ChatGPT was very capable at identifying the concept even when the question included a great deal of chemical symbolism. However, ChatGPT’s success at solving problems was only 44%, a value well below the class average of 69%. On open-response questions, ChatGPT’s responses displayed strong language processing ability with higher performance on questions that could be solved with more generalizable information compared to questions that required specific skills, especially when those topics or skills were primarily found in lecture. Incorrect responses and flawed explanations were often logically sound and would be persuasive to a novice. The chatbot is currently ill-equipped to provide reliable answers or explanations to students for many representative exam questions, but a potential use is to create assignments in which students analyze and improve ChatGPT’s responses.
0021-9584
RIEBPL Library
Testing/Assessment Problem Solving/Decision Making General Public High School/Introductory Chemistry First-Year Undergraduate/General Internet/Web-Based Learning Misconceptions/Discrepant Events