One AI Agent Is Good. Thirty-Two Are Nearly 20x Better.

A research team at Replit just ran an experiment that changes how software teams should think about AI development tools. They asked a simple question: does throwing more AI agents at a problem actually help, or does it create the kind of coordination chaos Fred Brooks warned about in 1975?

The answer, it turns out, depends entirely on the model and how it communicates.

Using a benchmark called Optimization-Bench, Replit tested how well AI agents could speed up real-world code performance. A single Claude Opus 5.5 agent improved NumPy workload performance by 3.7 times in a two-hour window. That is already impressive. When they scaled to 32 Opus agents working as a coordinated swarm, that number jumped to 18.9 times. Same time window, same goal, five times the single-agent result.

That is not a linear improvement. That is a qualitative leap.

Here is what made the difference: communication structure. The swarms that worked best used a shared task board. Agents posted results, claimed work, and updated their status without flooding each other with direct messages. The swarms that failed routed everything through a central coordinator, whose inbox became a bottleneck as more agents were added. The more agents piled on, the worse it got.

Replit also tested other models. GPT-6.2 Astra performed best as a solo agent. Unlike Claude Opus, its swarms degraded as they grew larger. Different models have different strengths when it comes to working in teams, just like different people do.

The shape of the work mattered too. When the codebase had genuinely independent problems, each additional agent contributed something new. When tasks were tightly coupled, agents competed for the same code, stepped on each other’s changes, and wasted time repeating failed attempts. The lesson is old: badly structured work produces poor outcomes regardless of how many smart contributors are on it.

There is a real business insight buried in these benchmarks. AI agents are not just smarter autocomplete tools anymore. They are starting to function like engineering teams, with meaningful differences in how they coordinate and what happens when you scale them up. A development firm might soon pair each engineer with an AI swarm that multiplies output by an order of magnitude. Or deploy swarms directly on tasks that never needed a human in the loop at all.

A single agent can accomplish in two hours what used to take days. Thirty-two of them can go five times further in the same window.

Replit has released Optimization-Bench publicly so other organizations can test their own agent configurations against real workloads. That kind of transparency is valuable. The question for any technology company is not whether AI agents will play a role in software development. That question is settled. The question is how to structure the work so the swarm can actually deliver.

Want to explore how AI agent automation could benefit your business? Let’s talk.

One AI Agent Is Good. Thirty-Two Are Nearly 20x Better.

Leave a Reply

Your email address will not be published. Required fields are marked *