Recently, Google Research, Google DeepMind, and MIT jointly released a study that challenges the traditional view that "more agents mean better results." The research team conducted 180 controlled experiments to explore the performance of multi-agent systems in different tasks. The results showed significant fluctuations in performance, with some tasks improving by up to 81%, while others declined by as much as 70%.

The study shows that the type of task has a significant impact on the performance of multi-agent systems. In some parallel tasks, such as financial analysis, using a centralized multi-agent coordination can significantly improve performance. Different agents can independently analyze sales trends, cost structures, and market data, then aggregate the results, ultimately achieving an 81% performance improvement.

However, in some tasks that require sequential execution, such as planning tasks in Minecraft, the multi-agent setup actually reduces performance by 39% to 70%. This is because each production action affects the inventory state that subsequent actions depend on. The sequential dependencies between tasks may lead to loss or fragmentation of information when it is passed between agents.

The study found that there are three main factors affecting the performance of multi-agent systems: First, the more tools involved in the task, such as web search or coding, the more likely it is to be negatively affected by the multi-agent system. Second, when the success rate of a single agent exceeds 45%, adding more agents usually leads to diminishing returns or even negative returns, as the coordination costs offset the gains. Finally, errors accumulate faster in a multi-agent environment. Without information sharing, errors occur 17 times faster than in a single-agent environment.

Key Points:

🌟 Multi-agent systems perform well in parallel tasks but significantly reduce efficiency in sequential tasks.

🤖 When the success rate of a single agent exceeds 45%, using a multi-agent system may not be cost-effective.

🔧 The more tools involved in the task, the higher the coordination costs of the multi-agent system, which affects performance.