A good piece of writing is finished. Has something been understood?
At a desk with AI, these two things can part company more easily than before. The writing flows, and its argument seems clear even to the person reading it. Then they close the screen and try to explain. Only a connective arrives: “In other words…” Nothing follows.
The reverse happens too. A conversation with AI helps someone over a step in an explanation they could not manage alone. The next day, they can think without borrowing the same explanation. From the outside, both people have used AI to finish an assignment.
Measuring beyond the deliverable
Zara Contractor and Germán Reyes step away from the finished assignment to follow the learner. They randomly assigned 211 university students to conditions allowing or forbidding AI, gave them a fixed time to learn about an unfamiliar technology and write an essay, and tested their knowledge immediately and about a week later. The assessments were meant to be completed without assistance. [1]
The delay matters. It lets us ask not only how quickly work proceeds while AI is helping, but what someone takes back to a desk where AI is absent.
On the immediate knowledge test, the AI-allowed group's accuracy was 7.2 percentage points higher. A week later, the difference was 3.9 points. The latter had a p-value of .068, which does not meet the conventional 5% significance threshold. The later essay's overall quality also had a positive estimate without meeting that threshold. A lasting learning effect should not be treated as settled.
The slightly untidy numbers are less a defect than an outline of what is currently visible. Experiments do not obligingly line up behind either “AI helps people learn” or “Delegating to AI prevents learning.”
What did the convenience make room for?
The study also examines how AI is used: asking for explanations to support understanding, or delegating the production of text. But those styles of use were not randomly assigned. A difference between them does not by itself prove that one style caused learning.
People already good at framing questions may also have asked AI for explanations. People hoping to finish quickly may have asked it to write. Separating the way a tool is used from the circumstances of its user requires another layer of research.
Still, there is a question to take into everyday life. It is not so much the experiment's conclusion as something left by reading it. What do we put in the space where AI has taken over a task?
Read another source. Explain the subject in our own words. Ask again about something we did not understand. Or close the assignment and move to the next job. The same amount of saved time can have different destinations. It does not come with an automatic transfer to a learning account.
Nor is the answer to preserve difficulty for its own sake. Spending half an hour stuck on awkward wording is not the same burden as testing an idea. Removable inconvenience and worthwhile difficulty cannot be separated by asking only what is convenient.
Putting AI aside is not a punishment
This experiment has qualifications. It was a short task at one university, with follow-up about a week later. On the immediate test, students who had been allowed AI were also more likely to violate the assessment rules. The observed score difference cannot all be treated as pure learning. The paper is still a preprint.
Yet the shift in where to place the assessment is clear.
At work too, it may help to separate the quality of a deliverable from the capability remaining in its maker. If today's goal is to finish a presentation, assess the joint output. If the goal also includes tomorrow's ability to change assumptions, find mistakes, and explain the work to someone else, assess that separately.
This is not a proposal to give everyone an exam without AI. Work is not always solitary, and continuing to use tools has value in itself. But if we claim to develop people, we should not relabel a tool's output as their growth. A brief explanation or a question with different conditions might help distinguish them.
Even asking, “How would you revise that answer?” moves a little closer to the texture of understanding and away from admiring a finished product. The point someone cannot explain need not be a verdict on their ability. It can become the next place to think with AI.
Imagine handing a presentation to a colleague. A polished document helps its recipient. But if one condition changes, which part should change with it? The beauty of the document does not tell us whether its maker can answer. Being able to explain also means being able to return to the choices behind the finished work.
Recording every exchange with AI does not prove those choices have stayed with the person. A log is a route; being able to walk it is something else. Conversely, someone may forget the precise route while still knowing what to check. Treating learning as the amount remembered is too narrow as well.
If we add an assessment, then, we might look beyond reproducing a correct answer to the questions someone asks when facing a slightly different problem. This is not something the present study fully measures. It is a form of understanding worth investigating next. Instead of making the moment the tool is removed the finish line, we might also look at how the person relates to it when they use it again.
Sometimes, put aside a tool that is usually beside us. A little distance can reveal what the time together has done.
Not what remains on the screen, but what remains in the person who closed it. That question is worth keeping alongside convenience.