TY  - BOOK
AU  -  Ted M. Clark
TI  - Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions:    (Journal Article)
T2  - American Chemical Society, Volume 100, Issue 5
SN  - 0021-9584
U1  - 540.7 
PY  - 2023///
CY  - USA
PB  - :American Chemical Society
KW  - Testing/Assessment
KW  - Problem Solving/Decision Making
KW  - General Public
KW  - High School/Introductory Chemistry
KW  - First-Year Undergraduate/General
KW  - Internet/Web-Based Learning
KW  - Misconceptions/Discrepant Events
N1  - ***______{For Hard Copy, Please visit Library.}________***
N2  - Abstract-

The artificial intelligence chatbot ChatGPT was used to answer questions from final exams administered in two general chemistry courses, including questions with closed-response format and with open-response format. For closed-response questions, ChatGPT was very capable at identifying the concept even when the question included a great deal of chemical symbolism. However, ChatGPT’s success at solving problems was only 44%, a value well below the class average of 69%. On open-response questions, ChatGPT’s responses displayed strong language processing ability with higher performance on questions that could be solved with more generalizable information compared to questions that required specific skills, especially when those topics or skills were primarily found in lecture. Incorrect responses and flawed explanations were often logically sound and would be persuasive to a novice. The chatbot is currently ill-equipped to provide reliable answers or explanations to students for many representative exam questions, but a potential use is to create assignments in which students analyze and improve ChatGPT’s responses.


UR  -  https://doi.org/10.1021/acs.jchemed.3c00027
ER  -