The co-founder of British-backed AI safety company Anthropic has issued a stark warning: artificial intelligence must not advance without strict human oversight. Speaking at a London tech summit, the executive cautioned that unbridled AI development could lead to catastrophic outcomes, from economic disruption to loss of democratic control.
“We are at an inflection point,” the co-founder said. “The technology is moving faster than our ability to govern it. If we don’t embed human values and control mechanisms now, we risk creating systems that operate beyond our comprehension and intervention.”
Anthropic, which received £500 million in backing from UK investors, has positioned itself as a responsible alternative to Big Tech’s race for AI supremacy. The company’s flagship model, Claude, is designed with a “constitutional AI” approach, aligning outputs with ethical guidelines set by human trainers.
But the co-founder’s warning goes beyond product design. He called for international treaties similar to those governing nuclear weapons, arguing that AI poses existential risks comparable to pandemics or climate change. “We wouldn’t let a pharmaceutical company release a drug without clinical trials,” he noted. “Why would we let AI systems deploy without rigorous safety testing?”
Critics, however, point out that Anthropic’s own technology is already being used in sensitive sectors like healthcare and finance. A London-based data scientist told The Insider: “They warn about control, yet their models are quietly influencing medical diagnoses. The control they seek might already be slipping.”
The debate highlights a fundamental tension within the tech community. On one side are accelerationists who believe AI progress is inevitable and must be embraced; on the other are precautionists demanding moratoriums on development until safety protocols catch up. Anthropic’s co-founder falls squarely in the latter camp, though he acknowledges that slowing down is nearly impossible given market pressures.
“The profit motive creates a terrible incentive to cut corners,” he admitted. “But we must resist. The alternative is a world where algorithms make life-and-death decisions without empathy or accountability. That’s not a future I want to live in.”
The UK government has taken note. The Department for Science, Innovation and Technology announced a £100 million AI safety taskforce just last week. But without binding regulation, critics argue that such measures are toothless.
As quantum computing begins to supercharge AI capabilities, the window for intervention narrows. “We have perhaps five years to get this right,” the co-founder concluded. “After that, the genie may be truly out of the bottle.”
