Did a Fourth Grader Write This? Or the New Chatbot?
Don’t be surprised if you can’t always tell. Neither could two teachers, a professor, nor even the renowned children's author Judy Blume.
“I’m just gonna say it’s a student and prepare for my soul to be crushed.”
Did a Fourth Grader Write This? Or the New Chatbot?
Don’t be surprised if you can’t always tell. Neither could a fourth-grade teacher — or Judy Blume.
It’s hard to fully grasp the enormous potential of ChatGPT, a new artificial intelligence chatbot released last month. The bot doesn’t just search and summarize information that already exists. It creates new content, tailored to your request, often with a startling degree of nuance, humor and creativity. Most of us have never seen anything like it outside of science fiction.
To better understand what ChatGPT can do, we decided to see if people could tell the difference between the bot’s writing and a child’s.
We used real essay prompts from the National Assessment of Educational Progress (the standardized test from the Department of Education, known as the nation’s report card). We asked the bot to produce essays based on those prompts — sometimes with a little coaching, and always telling it to write like a student of the appropriate age. We put what it wrote side by side with sample answers written by real children.
We asked some experts on children’s writing to take our variation on the Turing test, live on a call with us. They were a fourth-grade teacher; a professional writing tutor; a Stanford education professor; and Judy Blume, the beloved children’s author. None of them could tell every time whether a child or a bot wrote the essay. See how you do.