Eight AI agents beat one, but not by talking, MAC Bench finds

A new Bespoke Labs benchmark finds that teams of frontier agents gain from extra hands, while messaging each other adds little and costs a lot.

Illustration of eight robots at a shared workbench, each holding an identical model rocket, with tangled dashed lines and speech bubbles between them. Image: AgenticFeed illustration

Teams of eight frontier AI agents score higher than a single agent on long, multi-step tasks, but the gain comes from having more agents rather than from the agents working together, according to a new benchmark released Wednesday by Bespoke Labs.

The startup calls the test Multi Agent Coordination Bench, or MAC Bench. It contains twelve tasks in simulated environments, each built so that one agent's work depends on its teammates. The tasks range from air traffic control in Dubai to designing a Mars sample-return mission, restoring a power grid after an outage and surviving a real-time zombie game.

"We find that current frontier agents coordinate poorly," Bespoke Labs wrote in its announcement on X.

How the test works

Bespoke Labs evaluated Anthropic's Opus 5.5 and OpenAI's GPT-6.1 Sol, along with Google's Gemini 4 Argon, each running in its developer's own harness. Every run lasted four hours of wall-clock time regardless of team size, so a team of eight had eight times the agent-hours of a single agent but no extra time.

Each model ran every task alone, in a team of eight that could message on a shared board, and in a team of eight with messaging switched off. Comparing the two teams separates the effect of coordination from the effect of adding agents.

A hidden, deterministic grader scored each run between 0 and 1.

What the numbers show

Averaged across the three models, a team of eight scored 0.42 and a single agent 0.35, according to the report. Eight agents with no way to talk scored 0.44. Bespoke Labs puts the change from coordination at minus 0.018, with a 95% interval of minus 0.038 to 0.003, which it says is not statistically significant.

Results differed by model. Opus 5.5 and GPT-6.1 Sol scored about the same alone, but in teams of eight Opus 5.5 reached 0.53 while GPT-6.1 Sol reached 0.41. The report says only Opus 5.5 teams gained anything from messaging.

The authors describe Opus 5.5 as a model that listens to teammates and GPT-6.1 Sol as one that broadcasts without listening back. Gemini 4 Argon tended to work alone. In 9 of 16 runs with eight Gemini 4 Argon agents, the logs showed no stable division of work.

Talking was also expensive. Even in teams of two, 22% of tool calls went to the message board, and runs with more messages per agent did not reliably score higher, the report says.

Cost and speed

A team of eight cost 8.8 times as much as a single agent at list model prices. A single agent delivered the most score per dollar in every task and model pairing.

Teams were faster, but not in proportion to their size. In 30 of 36 task and model pairings, a team of eight matched the single agent's final score within four hours, a median of 2.2 times sooner.

Common failures

The authors had Claude Sonnet 5.5 read the logs of 139 team runs for 18 kinds of coordination failure, with Claude Opus 5.5 checking each episode. They caution that the shares are rough.

Duplicated work was the most common problem, appearing in 64% of team runs. In one Mars mission run, three Gemini 4 Argon agents pushed separate fixes for the same flight software bug within four minutes, and two of the fixes were discarded after merge conflicts.

Why it matters

Labs have started to scale agent count as a way to spend more compute on hard problems. The report points to OpenAI deploying roughly 10,000 agents on a mathematics problem in September and to Anthropic's account of agent teams losing track of a project's state while formalizing Fermat's Last Theorem. Bespoke Labs argues that existing benchmarks either use short multi-agent tasks or long tasks written for one agent, and so do not measure a coordination gap.

Bespoke Labs says it plans to add tasks in which agents have conflicting goals and to open source the harness that runs the benchmark. The results come from Bespoke Labs' own evaluation and have not been independently replicated.

Source: Bespoke Labs, “Multi Agent Coordination Bench”

Additional sources: Bespoke Labs on X, “MAC Bench announcement”

Get the AgenticFeed newsletter

A free, regular roundup of the agent stories that matter. No spam, and you can unsubscribe at any time.

Want to reach readers following AI agents? Advertise with AgenticFeed

Next event · 19 Nov, 2026

MCP Dev Summit Nairobi

The Linux Foundation’s Nov. 19 and 20 summit brings Model Context Protocol developers to Kenya, with keynotes from OpenAI, GitHub and the Gates Foundation.

The AgenticFeed newsletter

A free, regular roundup of the agent stories that matter. No spam.