Do smarter AIs make
a smarter organization?

Put five exceptionally capable people in a meeting room. Surely they will arrive at a better answer than one person alone.

Some of us might pause at that claim. What do those five people know? What are they deciding? Who brings the discussion together? Did this issue need a meeting at all? There is a surprisingly wide territory between assembling capable people and creating conditions in which their capabilities are useful.

With AI, it is tempting to hurry across that territory. If one can research a topic, several can divide the work. Have another AI check the result and errors should fall. Unlike people, they do not need to coordinate calendars. No meeting room needs booking.

But does the cost of coordination disappear too?

“Towards a Science of Scaling Agent Systems,” by researchers at Google Research, MIT and other institutions, recasts this as a question that can be tested. Under what conditions do multiple AIs outperform a single one—and when do they do worse?

What makes the study interesting is that it does not collect only success stories about working together. Cases in which forming a team makes the work worse sit inside the same experiment.

The researchers compared a single agent with four kinds of teams that communicated differently: agents working independently before pooling results; a central coordinator assigning work; agents communicating directly with one another; and a hybrid of the last two. Across three model families and six task groups, they evaluated 260 configurations. Tools and computational budgets were controlled to compare the effects of organization.

Matching computational budgets matters. Comparing five people working all day with one person working for an hour would not measure the power of a team. For AI, too, the words spent consulting and checking are not free. Within a fixed budget, resources spent on coordination cannot be spent elsewhere.

Work that can be divided. Work that must stay connected.

The results reversed depending on the task.

On financial-research tasks, the centrally coordinated configuration raised average performance from 34.9% for a single agent to 63.1%. On PlanCraft, a planning task requiring agents to respect sequences and prerequisites, all four team configurations tested performed worse than a single agent.

Division of labor helps in one case; coordination becomes a burden in the other. The paper examines this difference in terms of how easily a task can be separated and the overhead required to coordinate it.

Imagine researching a company. Investigating revenue and investigating costs can, to some extent, proceed in parallel. Bring the findings together at the end, and separate research streams can inform a common judgment.

Other work changes what is possible next according to what happened in the previous step. Every division of responsibility then requires a handoff of the current state. What has been completed? What is still available? Which conditions have been met? A handoff is not an accessory outside the work. It becomes part of the work itself.

These examples suggest one more question before deciding to divide a difficult task. Is it difficult because of the breadth of research required, or because it demands moving forward without losing track of intermediate states? Equally difficult problems can call for different kinds of help.

Of course, this experiment cannot give us an ideal organization chart. Its subjects are AI systems, not workplaces where humans and AI collaborate. Results might change with larger computational budgets or settings tailored more carefully to each model and task. Two task groups contained only 20 questions each, so individual wins and losses carry uncertainty. The authors acknowledge those limits.

What the capability list leaves out.

Even so, the question it leaves with the reader is substantial.

When choosing an AI, we first want to know what it can do. Can it read long documents? Write code? Calculate accurately? All are necessary questions. But once several are asked to work together, problems emerge that do not appear on a capability list.

How far should each think independently, and when should it consult others? What intermediate work should it record? If answers differ, who decides, and on what evidence?

Set out that way, the agenda suddenly looks familiar—even though we thought we were discussing the latest technology. That familiarity need not rush us to conclude that human organizations and AI teams are the same. We can stay with the resemblance of the questions for a while.

What is interesting is that there are things to consider even while waiting for a smarter AI to solve everything. In what units should the capabilities available now work? Where should their results come together? What should they avoid consulting one another about?

Picture those five capable people in the meeting room again. The question is no longer only who is smartest.

What are these five people here to think through together?