Jack Clark, co-founder of the AI safety start-up Anthropic, has issued a stark warning that artificial intelligence systems must never be allowed to operate beyond meaningful human oversight. Speaking at a technology ethics conference in London, Clark argued that as AI capabilities accelerate, the risk of losing control grows exponentially.
Clark, whose company focuses on building 'constitutional AI' that adheres to strict safety protocols, emphasised that the current trajectory of AI development is perilously close to creating systems we cannot fully trust. 'We are building intelligence that could outpace our ability to understand or correct it,' he said. 'That is not a future we should accept.'
Anthropic has been at the forefront of developing techniques to align AI behaviour with human values. Their research includes systems that can explain their reasoning and resist being 'jailbroken' by malicious inputs. Yet Clark warns that even these measures may be insufficient without global regulatory frameworks.
'We need a governance infrastructure that ensures any deployed AI remains under human control at all times,' he said. 'Not just in safety tests, but in real-world deployment. The consequences of a single misaligned AI decision could be catastrophic.'
The warning comes as tech giants race to deploy generative AI in products, from chatbots to autonomous agents. Clark criticised the 'move fast and break things' mentality that still permeates Silicon Valley. 'We are not building social media apps anymore,' he said. 'We are building entities that think. The margin for error is zero.'
His remarks echo concerns from other AI safety researchers, who argue that without robust oversight, we risk creating systems that optimise for unintended goals. The term 'alignment problem' has become central to these debates: ensuring that AI systems do what we want, not just what we say.
Clark proposed a three-point plan: rigorous pre-deployment testing, continuous monitoring during operation, and 'kill switches' that cannot be overridden by the AI itself. However, he acknowledged that such measures must be balanced with innovation. 'We are not Luddites. We believe AI can solve huge problems, from climate change to disease. But we must proceed with humility and caution.'
The speech has sparked debate among industry leaders. Some argue that slowing AI development cedes advantage to less scrupulous nations. Others, like Clark, counter that safety is a competitive advantage. 'The country or company that earns trust will win in the long run,' he said.
For now, the Anthropic co-founder's message is clear: human control is non-negotiable. In a technology world obsessed with what AI can do, he asks a more urgent question: what should it never be allowed to do?
