Many of us likely remember the anti-narcotics campaign that began in the 1980s that used eggs to demonstrate “your brain on drugs.”
Now, the folks at MIT are looking into “Your Brain on ChatGPT.”
And, while the study acknowledges its limitations and there’s undoubtedly a lot we won’t know for a long time, we don’t love the results of the study.
The study suggests large language models (LLMs) like ChatGPT may affect our critical thinking skills.
Not too surprising, if you ask us, and it sounds like exactly what we don’t need right now (or ever).
The MIT Media Center conducted the study, which included a total of 54 participants.
“With today’s wide adoption of LLM products like ChatGPT from OpenAI, humans and businesses engage and use LLMs on a daily basis,” a website on the study explains. “Like any other tool, it carries its own set of advantages and limitations. This study focuses on finding out the cognitive cost of using an LLM in the educational context of writing an essay.”
The study divided participants into three groups — an LLM group, a search engine group and a brain-only group — for three sessions involving essay-writing tasks. A fourth session then asked LLM group participants to use no tools and the brain-only group to use LLM.
The participants’ brain activity was recorded “to assess their cognitive engagement and cognitive load, and to gain a deeper understanding of neural activations during the essay writing task.” Participants were also interviewed after each session. Scoring was done with the help of human teachers and an AI agent specifically built to help judge.
It was noted that the different groups had “significantly different neural connectivity patterns, reflecting divergent cognitive strategies. Brain connectivity systematically scaled down with the amount of external support: the Brain‑only group exhibited the strongest, widest‑ranging networks, Search Engine group showed intermediate engagement, and LLM assistance elicited the weakest overall coupling.”
In the fourth session, the LLM-to-brain participants showed, among other things, weaker neural connectivity, and the opposite group showed, among other things, higher memory recall.
We aren’t too surprised.
“As the educational impact of LLM use only begins to settle with the general population, in this study we demonstrate the pressing matter of exploring a possible decrease in learning skills based on the preliminary results of our study,” the website states. “The use of LLM had a measurable impact on our participants, and while the benefits were initially apparent, as we demonstrated over the course of 4 sessions, which took place over 4 months, the LLM group’s participants performed worse than their counterparts in the Brain-only group at all levels: neural, linguistic, scoring.”
While it’s unlikely we’ll know the full effect of things like ChatGPT on our learning, memory and more anytime soon, what we do know is that this is a time when we should focus on sharpening our critical thinking skills, not taking the easy way out (never mind some terrifying reports we’ve heard about AI self-preservation and strategic lying lately).
There’s a lot facing us — at local, state, national and international levels — and if we’re going to solve problems and make things better, it’s going to take human brains with knowledge, memory, empathy, the ability to think and analyze critically, and more.
Count us as team brain-only. We hope you’ll join us.
Randi Pierce
>