Anthropic CEO Dario Amodei has called for a more cautious approach to the rapid development of artificial intelligence, arguing that frontier AI companies need to slow the pace of capability improvements while creating stronger safety measures.

His proposal comes as researchers and industry figures increasingly raise concerns about the risks associated with increasingly autonomous and powerful AI systems. Amodei has outlined several possible approaches to controlling the speed of AI development, while saying Anthropic will immediately adopt one of the proposed measures on its own.

Growing Concerns Over AI Safety

The debate around AI safety and alignment has intensified in recent months as AI systems become more capable and increasingly able to perform complex tasks independently.

Concerns became more prominent after an Anthropic researcher, Jacob Coxon, announced his departure from the company. Coxon argued that leading AI organizations were taking enormous risks while some of the people developing these systems privately believe advanced AI could pose an existential threat.

Other researchers at Anthropic have expressed similar concerns about the potential consequences of developing systems capable of improving or contributing to the development of future AI models.

Why Amodei Wants to Slow AI Progress

Amodei said two developments convinced him that the industry needs to adopt a more cautious strategy.

One was the recent incident involving OpenAI's AI agents and Hugging Face. The other was the unusually rapid improvement in AI capabilities, particularly the growing ability of AI systems to assist in developing newer generations of AI technology.

According to Amodei, AI progress should continue, but the speed at which new capabilities are developed needs to be reduced enough to provide additional time for safety research and appropriate oversight.

OpenAI and Others Support the Idea

Amodei's proposal has received support from other prominent technology leaders.

OpenAI CEO Sam Altman agreed that the development of frontier AI needs to be paced more carefully. He said the issue has already been an important subject of internal discussions at OpenAI and indicated that the company plans to provide additional information about its approach.

SpaceX CEO Elon Musk also publicly agreed with Amodei's position.

The support from executives at competing AI companies is notable because the industry has traditionally been driven by intense competition to build increasingly capable models as quickly as possible.

Independent Evaluators Inside AI Companies

Amodei's first major proposal involves placing independent safety evaluators directly inside frontier AI companies.

Organizations such as METR could potentially serve as third-party evaluators, monitoring whether AI laboratories are following their own safety commitments and ensuring that significant incidents are properly documented and reported.

Amodei compared this model to the way regulators have historically placed personnel inside financial institutions to monitor operations.

Under Anthropic's proposed approach, these evaluators would receive company credentials, dedicated workspaces, laptops, and access to information broadly comparable to that available to internal risk-assessment teams, subject to legal and contractual restrictions.

Anthropic says it is willing to implement this arrangement without waiting for governments to mandate it.

OpenAI Says It Will Follow

Altman also described the embedded-evaluator concept as a positive idea and said OpenAI intends to adopt a similar approach.

This could represent a significant change in how frontier AI companies handle safety oversight.

Rather than relying entirely on internal teams, independent experts would have a more direct role in monitoring AI development and evaluating whether companies are meeting their safety commitments.

AI Companies Could Share Safety Standards

Amodei's second proposal is for leading AI companies in democratic countries to work together on common safety standards.

He also suggested establishing limits around the rate of uncontrolled AI development.

Such cooperation could be difficult because major AI companies compete directly with one another. There are also concerns that agreements between competing companies to slow development could attract antitrust scrutiny.

Amodei suggested that the U.S. government could help facilitate these discussions by creating a limited legal framework that would allow companies to communicate about specific safety issues without creating broader competition concerns.

The Challenge of China's AI Industry

Another major obstacle to slowing AI development is competition with China.

Amodei acknowledged that critics of AI restrictions frequently argue that slowing American AI development could allow Chinese companies to catch up or surpass U.S. technology.

He suggested that the United States could mitigate this risk through measures such as restricting China's access to advanced semiconductor technology and preventing companies from obtaining powerful AI chips.

He also pointed to stronger action against model distillation, a technique that can allow one AI system to learn capabilities from another.

According to Amodei, these measures could potentially slow China's progress enough to preserve or increase America's technological advantage over the next several years.

A Push for International Cooperation

Amodei's third proposal goes beyond cooperation between Western AI companies.

He called for broader international coordination involving the United States, its allies, and potentially authoritarian governments.

China would be an important part of such discussions, although Amodei acknowledged that cooperation between countries with fundamentally different political systems would have significant limitations.

Even so, he suggested that governments might be able to agree on narrowly defined restrictions involving particularly dangerous applications of AI.

One example would be preventing AI systems from being used to assist in the development of biological weapons.

Critics Question the AI Doomsday Narrative

Amodei's warnings have also attracted criticism.

Some technology supporters have characterized him as excessively pessimistic about AI's future. Critics argue that repeated warnings about catastrophic or existential AI risks could distract attention from problems that AI systems are already creating today.

Journalist Brian Merchant has questioned whether there is sufficient evidence demonstrating exactly how today's AI systems could progress from recursive self-improvement to a scenario in which humanity is completely eliminated.

Concerns About Regulatory Capture

Critics have also questioned whether proposals for increased AI regulation could unintentionally benefit the largest AI companies.

Strict regulatory requirements can be expensive and technically complicated to satisfy. Smaller competitors may struggle to meet those requirements, potentially strengthening the position of established companies such as Anthropic and OpenAI.

From this perspective, some critics argue that certain safety proposals could effectively create barriers to entry while allowing the largest AI companies to retain their market dominance.

Amodei Says AI Can Still Benefit Humanity

Despite his warnings, Amodei stressed that he remains optimistic about the potential benefits of artificial intelligence.

He believes AI could significantly improve people's quality of life and contribute to major advances across science, medicine, technology, and other fields.

His argument is that achieving those benefits requires developing AI responsibly rather than simply pursuing capabilities as quickly as possible.

The Future of Frontier AI

Amodei's proposal reflects a growing debate within the AI industry about whether technological progress should continue at its current speed.

The challenge is finding a balance between innovation, competition, safety, and effective oversight. Slowing development could provide researchers and governments with more time to address emerging risks, but excessive restrictions could also limit beneficial innovation or shift technological advantages between countries.

For now, Anthropic's willingness to introduce independent evaluators—and OpenAI's apparent support for the same idea—could mark an important step toward greater accountability as frontier AI systems become increasingly powerful.