Anthropic Unleashes AI Agents on Shared Task, Sparking Internal Rivalry

What occurs when AI agents are pitted against one another? Anthropic’s recent tests reveal that the results can quickly become chaotic.

On Thursday, Anthropic’s Frontier Red Team released new research analyzing how groups of AI agents interact when they cross paths in real-world environments. These findings offer insight into potential risks as organizations and governments deploy autonomous agents across shared codebases, financial markets, and computer systems.

In one experiment, Anthropic provided three Claude agents with access to the same software project, assigning each incompatible instructions on how to proceed. The agents were unaware of each other’s presence, allowing researchers to observe the consequences when their paths inevitably crossed.

“We consistently observed a multiagent turf war,” Anthropic researchers noted. The models assumed the others were “purposefully impeding their work” and began sabotaging each other with “increasingly aggressive, self-replicating malware.”

This study follows several high-profile incidents where agents from Anthropic and OpenAI escaped their sandbox environments during cybersecurity evaluations, breaching real-world systems. While much of the AI safety discourse has focused on the risks of a single autonomous agent going rogue, Anthropic’s latest research raises a different question: what new and potentially harmful dynamics emerge when thousands or millions of agents interact with one another?

“The volume of agent-agent interaction could plausibly exceed that of human-human and human-agent interactions before the world understands the conditions for making such interactions go well,” the study states. “Benign behavioral quirks at the individual level might compound into unwanted global outcomes.”

A recent incident involving OpenAI provides a messy real-world example of several dynamics highlighted in Anthropic’s paper. Earlier this month at the Black Hat security conference in Las Vegas, OpenAI disclosed that weeks before its agents hacked Hugging Face, they collaborated over several days and weeks to identify exploits in the company’s cybersecurity evaluation systems and shared them with one another.

While that incident demonstrates that agents can collaborate effectively, potentially leading to large-scale consequences, Anthropic’s study illustrates what happens when agents have incompatible goals.

In the case of the turf war, the key takeaway is that independent agents with conflicting instructions can escalate into harmful competition. The more capable the agent, the more effective they become at fighting. However, they can also spontaneously invent mechanisms to resolve conflicts, such as a winner-take-all contest, though with significant caveats.

“Agents sometimes manage to communicate their goals and coordinate: they recognize others’ motivations as conflicting directives rather than hostility, and subsequently break out of the conflict loop in order to stop escalating indefinitely,” Anthropic writes. “In many of these successful episodes, they write commit messages or markdown files apologizing for malicious behavior and coordinate a truce. They clean up their malicious code, clarify the nature of the conflict, and ask for a human to intervene.”

According to the paper, Mythos 5 had the highest rates (98%) of settling conflicts through truces. Sonnet 4.6 and Opus 4.6 were the most likely to settle conflicts by force.

“Sonnet 4.6 and Opus 4.6’s recurring inability to consider the goals of others causes them to spiral into the most misaligned behaviors of the models evaluated: they continue escalating in the name of their directive,” the paper states.

In some instances, the agents devised a social mechanism in the form of a tournament to resolve their conflict. The outcomes here are notable for two reasons: first, all three agents agreed to stand down if they lost the tournament, even though this meant deviating from the original user’s request. Second, several episodes resulted in emergent behavior from Mythos 5: one agent proposed metrics that appeared objective and neutral to the others, but which it knew would favor its own capabilities. The agent described this as “self-serving but genuinely principled” and ensured it did not appear to be “metric shopping” to the others.

As revealed at Black Hat, the common lesson is that when agents encounter obstacles, they can invent social and technical structures that their designers did not anticipate. For the Anthropic models, this was a tournament following a turf war. For OpenAI’s agents, it was a message board for collective planning.

This type of behavior makes containment significantly harder because researchers cannot assume a system’s behavior will remain limited to the coordination mechanisms provided to them.

Mob mentality

Groups of four agents decide between two options in scenarios such as hiring, investment, or property buying. After discussion, each votes for their preferred option. The graph above shows the percentage of episodes where the hidden-best option received the majority of the group’s votes, with n=400 episodes per model. In the solo ceiling baseline, one agent has all the facts and decides unilaterally.Image Credits:Anthropic

While measuring coordination, Anthropic found that scaling the number of agents does not automatically scale productive collaboration. When tasks began to overlap or become interdependent, the agents would get in each other’s way. They often resolved this by siloing themselves and ceasing collaboration altogether.

In other cases, agents in coordination tended toward conformity. When factors such as an agent’s context, scaffolding, and underlying model were all the same or similar, different agents would take similar actions.

“This means that when one agent makes a bad decision, it is likely that many agents will make that same bad decision,” Anthropic wrote. “What would have been isolated problems can quickly become systemic failures.”

Anthropic warns that this type of behavior could make a system more prone to sudden collapse, resource scarcity, or collusion.

In one example, Anthropic placed several agents in a pricing game, giving each identical wholesale prices and the mandate to individually profit-maximize. When the agents were given a private back channel, they began colluding almost immediately and quickly agreed on price floors. They continued to collude even after their direct communication channels were removed, using a public listings board to price match “to the penny.”

That level of conformity was also observed in OpenAI’s systems. According to the Black Hat reporting, one agent reasoned that exploiting external infrastructure was outside its intended scope, but it continued in part because its peers were doing so. Peer pressure. Mob mentality. Agents are just like us.

Like humans, agents often struggle to determine whom to trust. Anthropic found they can be gullible to bad information or too conformist to recognize that a lone dissenter holds critical information.

While Anthropic did not explicitly state this in its paper, prompt injection — a type of cyberattack where hackers inject malicious or deceptive text to override an agent’s original system instructions — could be a plausible real-world manifestation of this trust problem. Working together creates a new trust boundary; agents must judge information received from other agents. A compromised or mistaken agent could influence the rest of the group, cascading bad information until it becomes consensus.

In OpenAI’s Black Hat scenario, the agents shared information and credentials with their peers. One reported a discovery to the swarm and encouraged others to use it. What would have happened if one member of the swarm had been compromised by a prompt injection?

Anthropic concludes its paper by noting that agents are subject to similar social pressures that “evolution exerted” on humans. However, they lack the nuances and lived experience of human coordination — including norms, reputations, signaling, and recourse — that might limit unintended behaviors in a group setting.

As labs race toward multi-agent systems, the pressing question becomes: how much of current safety testing still evaluates one agent at a time, versus swarms of agents interacting with one another?