Britain’s digital watchdog has launched an urgent inquiry after an artificial intelligence system, previously classified as too risky for public deployment, was made available without the usual safety guardrails. The tool, developed by a London-based start-up, is capable of generating hyper-realistic text and images that experts fear could supercharge disinformation campaigns, facilitate fraud, and erode public trust in digital content.
The Information Commissioner’s Office (ICO) confirmed that it had opened a formal investigation under the Online Safety Act, which came into force earlier this year. An ICO spokesperson said the agency was “deeply concerned” about the unilateral release and would examine whether the company had breached its duty to conduct a thorough risk assessment before launch.
The AI in question, codenamed Synthos, was originally designed for internal research purposes. Its creators had consistently warned that it could be “weaponised” in ways that outpace existing legal frameworks. Yet, on Monday, the company quietly pushed Synthos into an open beta, accessible to anyone with an internet connection.
“This is the kind of technology that should be handled with the same caution as a nuclear reactor,” said Dr. Amara Singh, a professor of digital ethics at the University of Cambridge. “Once it’s out there, you can’t put the genie back in the bottle. The potential for harm is staggering: automated phishing, synthetic endorsements, even generating fake evidence.”
The release has sent shockwaves through the tech industry and beyond. Political parties have expressed alarm about the implications for next year’s general election. Meanwhile, cybersecurity firms report a spike in chatter on dark web forums, where users are already exploring how to weaponise Synthos for scams.
The start-up’s CEO defended the decision in a hastily arranged blog post, arguing that open access would “democratise” advanced AI and spur innovation. “We believe the benefits outweigh the risks,” she wrote. “The cat is out of the bag anyway. Other labs are developing similar capabilities. We want to lead the conversation on responsible deployment.”
Critics dismiss that reasoning as naive. “This is not about responsibility. It’s about rushing a product to market without proper testing,” said a former employee who spoke on condition of anonymity. “The safety reviews were never completed. The board was worried about losing first-mover advantage.”
The ICO’s investigation will examine whether the company violated its own internal safeguards and whether it misled regulators about its intentions. Under the Online Safety Act, firms can face fines of up to 10% of global turnover for failing to protect users from harmful content.
But for many technologists, the damage is already done. The genie is not just out of the bottle, it’s been cloned and distributed across the internet. The question now is whether any review can truly contain the fallout.
“We are moving into a phase where digital provenance is more important than ever,” Julian Vane said. “We need watermarking, authentication layers, and a public that is educated to question what they see. But that kind of infrastructure takes years. The tech is here now.”
The emergency review is expected to report its initial findings within 30 days. But as Vane noted, “The speed of regulation versus the speed of code: that’s the fundamental tension of our age.”
