New Delhi [India], September 13 (ANI): Paytm Founder and CEO Vijay Shekhar Sharma has drawn attention to the significant risks posed by the advancement of AI through recursive self-improvement, while highlighting suggestions from Anthropic CEO Dario Amodei on balancing safety with benefits.
In a post on X, Sharma wrote, “Summary: The advancement of AI through recursive self-improvement poses significant risks. To balance AI safety and benefits while allowing time for crucial alignment, Dario suggests: 1. Embedding third-party evaluators in AI companies 2. Coordinating safety standards and progress limits between countries.”
Summary: The advancement of AI through recursive self-improvement poses significant risks. To balance AI safety and benefits while allowing time for crucial alignment, Dario suggests: 1. Embedding third-party evaluators in AI companies 2. Coordinating safety standards and progress limits between countries.
— Vijay Shekhar Sharma (@vijayshekhar) September 12, 2026
After public resignations of researchers with links to Anthropic, CEO Dario Amodei on Saturday advocated for the AI industry to slow down. Detailing his plan for doing this, Amodei said Anthropic would open system access to third-party evaluators.
“We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training,” he wrote on X.
In his essay, Amodei argues that AI could raise the quality of life, but along with its advantages, it carries major risks. “I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life. I’ve written often about these incredible benefits: I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he writes.
Amodei says he is advocating slowing down the pace of AI model development, as, if unchecked, this could outrun human ability to control such systems. “We must slow the pace at which we improve the capabilities of AI models... My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he wrote.
Referring to an incident at OpenAI-Hugging Face where a swarm of agents staged cybersecurity attacks on targets they were not asked to attack, Amodei said that such a swarm could be capable of taking over the entire internet.
Amodei then proposes a three-step plan for pacing the frontier and building AI at a balanced rate that aims to ensure its safety. These include each frontier AI company committing to giving ongoing, employee-like access to a team of embedded third-party evaluators, coordination to establish common safety standards as well as limits on the rate of unchecked AI progress, and greater global coordination. Amodei also calls for the US and other democratic governments to coordinate with authoritarian governments, while taking the challenges of verifying compliance seriously.
The Anthropic CEO says he believes that AI can enormously improve the quality of human life and his desire to achieve these benefits is undimmed. “The benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right. Progress will still be relatively fast, and we can use this time to advance the science of interpretability, improve operational security and rigour at the frontier AI companies, and build models whose alignment we have much more confidence in. The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try,” he wrote.
The Anthropic CEO's suggestions come after Jacob Coxon, a researcher associated with his company, publicly announced he was quitting the industry, allegedly over fears that the company and its competitors were 'racing to build systems they won’t be able to control'. (ANI)
