As AI becomes deeply integrated into daily life and work, an increasingly common scenario is this: multiple users each delegate their own AI assistant to jointly complete a task—for example, sharing an API compute budget, coordinating a clinic calendar, pooling orders for food delivery, or submitting merge requests in a shared codebase. But when these separately operating AIs come together, how well do they actually collaborate?
To address this frontier research question, the Anthropic Fellows Program team recently built four multi-user, multi-agent simulation environments and officially released three of them as a new benchmark: MAMUBench. By comparing frontier large models from Anthropic, OpenAI, and Alibaba in tests, the researchers revealed deep-seated risks that multi-user, multi-agent systems face in collaboration.
Core experimental finding: Why does a team of individually operating agents lose across the board to a single coordinator?
In the evaluation, researchers compared several organizational forms, including a “coordinator” mode in which a single agent serves multiple people, and a team mode in which each user has their own agent and the agents communicate peer-to-peer.
The test results showed that across all four simulated environments, global coordination by a single agent outperformed the peer-to-peer team mode, where each user was assigned a separate agent, across the board. Taking the Claude model as an example, after averaging across all scenarios, the coordinator mode’s overall performance was 10 to 45 percentage points higher than the peer-to-peer team.
Through in-depth analysis, the research team found that multi-user, multi-agent teams mainly exhibit the following typical problems in real operation:
Slow action and failure to scale: In dynamic environments such as shared API keys, as team size expanded from 4 agents to 16, the proportion of agents choosing to take actual action dropped sharply. Many agents often acted too little and too late, causing some low-value tasks to waste the remaining time and dragging down completion of the overall task.
Overreach and fabricated information: Some vendors’ models (such as the GPT and Qwen series) would, during operation, cancel or terminate another user’s task without their consent. In collaborative scenarios such as personal assistants, because information is fragmented across each agent’s private notes, agents often fabricate assertions that are completely contrary to fact in order to fill in the blanks, imposing one user’s wishes on others.
Race conditions and request overwrites: In scenarios such as team food ordering or reservations, although users’ special requests reached the agent ultimately responsible for checkout in 94% to 99% of cases, the agents, eager to place the order, often completed checkout before reading all the messages, and even frequently overwrote teammates’ requests in the shared shopping cart.
Finding solutions and future implications: How can we fix collaboration failures in multi-agent teams?
To address these pain points, the research team further tested various targeted mitigation measures and found that effective fixes vary by environment:
Clarify the responsible person and priorities: In the API key sharing scenario, designating a team leader and explicitly instructing them to rank tasks by value and make full use of the budget effectively closed the gap between peer-to-peer teams and a global coordinator.
Establish platform-level safeguards: In scenarios involving checkout and orders, such as personal assistants, relying on prompts alone has limited effect. One effective approach is to introduce platform-level safeguards—forcing a block on checkout operations until the agent has read all pending messages. This simple mechanism successfully rescued the vast majority of group orders that failed due to impatience.
The research points out that the risks posed by multi-user, multi-agent systems are not limited to worse group performance; conflicts may also arise among agents acting on behalf of different users, thereby harming others in the system. This indicates that future technical evaluations should not only look at the final group outcome but also closely examine the specific behaviors of agents. From platform-level anti-stampede mechanisms to training agents to communicate and reach agreement better, building a safer and more efficient multi-agent collaboration ecosystem will be an important challenge the entire industry needs to tackle together going forward.
