The Pulse
GPT-5 Help Cut Persistence on Follow-Up Tasks
Three randomized experiments involving 1,222 people found that brief access to GPT-5 improved performance while available but was followed by weaker results on unaided tasks. The tests measured responses immediately after assistance ended,
AI.info Team · Written with AI ·
Across three randomized experiments with 1,222 participants, brief AI assistance improved performance while available but was followed by weaker unaided results and more giving up on some tasks. The research, presented at the Conference on Language Modeling and reported by UC Berkeley on October 9, tested what happened when an assistant disappeared after roughly 10 to 15 minutes of use.
Three experiments, two kinds of task
Researchers Grace Liu, Brian Christian, Tsvetomira Dumbalska, Michiel A. Bakker and Rachit Dubey recruited participants in the United States through Prolific. They studied fraction problems in two experiments and SAT-style reading comprehension in a third. Participants assigned to the AI condition used a GPT-5 assistant in a ChatGPT-style sidebar; researchers then removed it before the final problems.
The paper’s authors write, “Across a variety of tasks, including mathematical reasoning and reading comprehension, we find that although AI assistance improves performance in the short-term, people perform significantly worse without AI and are more likely to give up.” In the first fraction experiment, the AI group solved 57% of the final three problems correctly, compared with 73% in the control group. Participants in the AI group skipped 20% of those problems, against 11% in the control group.
The second math test produced a mixed result
A larger replication recruited 667 people and added a pretest to address differences between groups in the first experiment. Among participants retained for analysis, the AI group solved 71% of the final fraction problems correctly, compared with 77% for controls. That performance gap was statistically significant, but the groups’ skip rates—10% and 7%—did not differ significantly.
The researchers also compared outcomes by how participants said they used the assistant. In that analysis, people who reported asking for direct answers showed lower later solve rates and higher skip rates than those who asked for hints or clarification. The authors caution that this comparison was based on reported usage rather than randomized assignment, so it cannot establish that asking for direct answers caused the difference.
Reading results broaden the finding—and its limits
In the third experiment, 201 people worked on reading-comprehension questions, with the AI removed before the last three. In the retained sample, participants who had AI access solved 76% of those final questions correctly, compared with 89% of controls, and skipped 8% rather than 1%. The study counted answers submitted in under five seconds as skipped, an important detail in interpreting its persistence measure.
Brian Christian, a research fellow with UC Berkeley’s Center for Human-Compatible AI, said, “We need to work toward a future in which human abilities are, as much as possible, augmented rather than supplanted.” The team argues that assistants designed to coach users, rather than supply complete answers, may better support independent skill; its hint-use analysis offers a possible lead, not proof that this approach prevents the effect.
Immediate results are not evidence of lasting decline
The experiments measured performance immediately after AI access ended. They did not test whether the changes lasted hours or days, or whether repeated everyday use accumulates into a lasting loss of ability. The authors also note that their assistant was set up to provide complete answers on request, making it a particular kind of AI use rather than a test of every way people work with ChatGPT.
The findings point to a specific trade-off in these short tasks: AI can help people get answers during a session, while leaving them less successful when asked to continue alone. Whether tutor-style assistance can preserve the immediate help without that immediate follow-up drop remains untested.