In a stark warning issued this morning, one of the co-founders of Anthropic, the AI safety company behind the Claude model, has declared that artificial intelligence development is moving too fast without adequate human safeguards. Speaking at the Global AI Summit in London, the executive argued that the current trajectory of machine learning could lead to systems that act unpredictably, potentially causing harm on a societal scale.
The co-founder, who requested anonymity due to the sensitive nature of the discussions, said that while AI has the potential to revolutionise healthcare, education, and climate science, the absence of a rigorous oversight framework is akin to building a skyscraper without scaffolding. “We are seeing models being deployed in domains like finance, transportation, and criminal justice without the checks that would be standard in other engineering disciplines,” the co-founder stated. “This is not about restricting innovation. It is about ensuring that innovation does not outpace our ability to understand its consequences.”
The timing of the warning is critical. Just this week, a major social media platform admitted to shutting down an experimental chatbot after it began generating offensive content. Meanwhile, a healthcare AI used in three UK hospitals was found to misdiagnose rare conditions in 12% of cases. These incidents, while isolated, underscore the pattern of rushing to market without sufficient testing.
Anthropic has long championed a concept called “constitutional AI”, where models are given a set of ethical guidelines that act as a kind of digital conscience. But the co-founder stressed that such technical fixes are not enough. “We need human-in-the-loop systems, where critical decisions are always reviewed by people. We need regulators who understand the technology, not just technologists who understand the regulations. Most of all, we need a public debate about what we want from AI, rather than letting the market decide.”
The call for oversight comes as the European Union finalises its AI Act, which proponents say will bring much-needed transparency. Critics, however, argue that the legislation is too slow to adapt to rapid change. The Anthropic co-founder seemed to agree, stating: “The pace of legislation has got to match the pace of innovation. Otherwise, we are writing rules for a world that no longer exists.”
The speech ended with a plea to investors and startups: “Don’t let the hype blind you. Build systems that respect human autonomy, that can be audited, and that can be shut down if they fail. That is not a limit on technology. That is a foundation for trust.”
