AI agents – systems capable of reasoning, planning, and taking action – are becoming a common paradigm for real-world AI applications. From coding assistant For personal health trainers, the industry is shifting from single-shot question answers to continuous, multi-step conversations. While researchers have used long-established metrics to optimize the accuracy of traditional machine learning models, agents introduce a new layer of complexity. Unlike discrete predictions, agents must navigate continuous, multi-step interactions where a single error can cascade throughout the workflow. This shift forces us to look beyond standard accuracy and ask: How do we actually design these systems for optimal performance?
Practitioners often rely on assumptions, such as the assumption that “More agents are better“, assuming that adding particular agents will consistently improve outcomes. For example, “you just need more agents“It is reported that the performance of LLM varies with the number of agents Collaborative Scaling Research found that multi-agent cooperation “…often surpasses each individual through collective reasoning.”
In our new paper, “Toward a Science of Scaling Agent Systems“, we challenge this assumption. Through a large-scale controlled evaluation of 180 agent configurations, we derive the first quantitative scaling principles for agent systems, showing that the “more agents” approach often reaches a limit, and can even degrade performance if not aligned with the specific properties of the task.
