As the UK government prepares to host a landmark global summit on artificial intelligence safety, one of the sector’s leading voices has issued a stark warning: AI must never be allowed to develop beyond human control. Dario Amodei, co-founder and CEO of Anthropic, the company behind the Claude chatbot, told the BBC that current trajectories of development could lead to systems that are ‘potentially very dangerous’ if left unchecked.
Speaking ahead of the AI Safety Summit scheduled for November 1-2 at Bletchley Park, Amodei said the industry has a ‘very high responsibility’ to ensure that powerful AI systems remain aligned with human values. ‘If we’re not careful, we could lose control of the very technology we’re building,’ he said. Amodei’s comments come as the summit’s agenda is finalised, with a focus on ‘frontier’ models – those with capabilities that could pose significant risks.
The summit, announced by Prime Minister Rishi Sunak, is expected to bring together international governments, leading AI companies, and civil society groups. According to draft documents seen by the Guardian, the agenda includes sessions on 'existential risk', 'safety frameworks', and 'international governance'. The UK aims to position itself as a global hub for AI safety regulation, following the launch of its Foundation Model Taskforce earlier this year.
Amodei’s Anthropic has been at the forefront of responsible AI development, advocating for techniques like ‘constitutional AI’ to embed safety constraints directly into models. But he stressed that technical solutions alone are not enough: ‘We need robust oversight, international coordination, and a shared commitment to putting humans at the centre of this technology.’
Critics, however, argue that the summit’s focus on hypothetical future risks may distract from immediate harms such as bias, surveillance, and job displacement. ‘The existential threat narrative can be a convenient smokescreen for companies to avoid accountability for today’s problems,’ said Dr. Eleanor Walsh, a digital ethics researcher at the University of Oxford.
Amodei acknowledged these concerns but insisted that both short and long-term risks must be addressed. ‘We can walk and chew gum at the same time,’ he said. ‘The challenges we face are urgent – from algorithmic discrimination to the potential for autonomous weapons. But we also need to prepare for a world where AI becomes as powerful as some forecast.’
The summit’s agenda reflects this dual focus. Day one will cover ‘current harms and opportunities’, while day two will delve into ‘future risks and governance’. Notably, the agenda includes a session on the ‘democratisation of AI’, a topic pushed by developing nations who fear being left behind.
For the UK, the summit is a political gamble. Sunak has faced criticism from both sides: some say he is overhyping apocalyptic scenarios, while others accuse him of kowtowing to tech giants. ‘This is a moment to show leadership, not just on the world stage but on shaping an industry that will define the 21st century,’ said a Downing Street spokesperson.
As the countdown to November begins, the stakes could not be higher. Amodei’s warning echoes a growing consensus among AI pioneers: the window for safe development is closing. ‘We have a narrow band of time to get this right,’ he said. ‘If we fail, it may be too late to put the genie back in the bottle.’
The summit is expected to produce a non-binding international declaration on AI safety, but sceptics question its teeth. ‘Words are cheap. What matters is whether governments will actually regulate,’ said Walsh. For now, all eyes are on Bletchley Park – the wartime codebreaking site that now hosts a different kind of code race.
