Image Courtesy: Getty Images

Leaders from some of the world’s most powerful AI companies are increasingly agreeing that the development of advanced AI systems may need additional safeguards, following warnings that future systems could pose catastrophic risks to humanity.

Anthropic CEO Dario Amodei called for a measured approach to developing frontier AI, arguing that the industry should use any additional time to improve safety. OpenAI CEO Sam Altman, Elon Musk and Google DeepMind founder Demis Hassabis have also expressed support for greater caution. The debate has intensified following warnings from former Anthropic engineer Jacob Coxon.

Former President Barack Obama said governments need to be proactive in regulating AI, arguing that society cannot simply reverse technological progress but can still determine how the technology is developed and deployed.

Others are far more skeptical of calls for a slowdown. Investor Michael Burry argued that restrictions could benefit established AI companies by making it harder for smaller competitors to keep pace. Trump administration technology adviser David Sacks similarly questioned whether Anthropic and OpenAI were motivated by safety or by efforts to gain regulatory advantages.

The open-source community has raised similar concerns. Hugging Face CEO Clément Delangue called for greater transparency and proposed involving his company’s “open alignment initiative” in independent AI evaluations. Investor Jason Calacanis argued that safety rules could instead weaken open-source competitors while protecting companies with the deepest pockets.

Some politicians are advocating much stronger measures. Sen. Bernie Sanders has proposed temporarily pausing advanced AI development and banning the development and deployment of artificial superintelligence. He has also called for the United States and China to negotiate an international agreement on advanced AI.

Other technology leaders favor safeguards without slowing innovation. Microsoft’s Satya Nadella supports independent evaluation but says AI governance should include academia, governments and other parts of the ecosystem. Box CEO Aaron Levie similarly described AI safety as essential while warning against measures that unnecessarily reduce competition.

AI researcher Gary Marcus remains skeptical that today’s systems justify an industry-wide slowdown. He argues that current AI agents are often unreliable and that the immediate priority should be removing unsafe products rather than halting development.

The emerging consensus, therefore, is less about stopping AI than determining who should oversee its development and how. Independent testing, transparency and safeguards are gaining support, but disagreements remain over regulation, open-source models and whether existential risks are imminent or highly speculative.