UK to strengthen cyber defences as minister calls for AI firms to increase safety efforts

Liquid-cooled servers at a datacentre in London. Narayan will argue the UK needs to build more of its own AI infrastructure. Photograph: Bloomberg/Getty ImagesView image in fullscreenLiquid-cooled servers at a datacentre in London. Narayan will argue the UK needs to build more of its own AI infrastructure. Photograph: Bloomberg/Getty ImagesUK to strengthen cyber defences as minister calls for AI firms to increase safety effortsKanishka Narayan to say Britain cannot rely entirely on tech companies amid rising global concern over AI dangers

Companies building the most advanced AI systems must increase safety efforts, the UK’s AI minister will say, as he announces a strengthening of Britain’s cyber defences.

Kanishka Narayan will call on companies developing frontier AI to do more to make sure it is safe. He will say Britain cannot rely entirely on technology companies to police their own models and will argue the UK needs to build more of its own AI infrastructure.

Narayan’s speech in west London will be the first since Andy Burnham made AI a cabinet-level brief. It comes as Geoffrey Irving, a former chief scientist at the UK’s AI Security Institute, which tests the most powerful AI models, raised fears enemy states could steal unreleased cutting-edge models. Amid increasing global concern at dangers posed by AI, Irving told the Guardian: “The AI companies do not have security that would prevent an adversarial country from stealing the models.”

He cited a recent incident at OpenAI where it was “hacked” by cybersecurity researchers, who said “the scope of what we could theoretically access was huge”. The attack was carried out under an OpenAI programme that rewarded ethical hackers testing its systems. OpenAI says it has multiple protections against theft of its models, including encryption and continuous monitoring.

Irving, a former employee at OpenAI and Google DeepMind who now runs an AI safety research body, has warned there is a “50% chance” of humanity being wiped out by superintelligent AIs and the only answer, for now, is to stop development of advanced systems as soon as possible. Irving is concerned by the rate of progress in alarming attributes shown by models, which could lead to them becoming highly advanced in traits such as concealing their actions and manipulating humans.

“It is possible we [will] get superintelligence in less than 12 months, on the current path,” he said.

Narayan’s speech will be watched closely for signs the UK may join a call for international governments to coordinate common safety standards for AIs, which has been signed by leaders of nations including Finland, France, Germany, Australia, Singapore, the UAE and Ireland, but not the UK and the US, where Donald Trump has said he wants the leading AI companies to “self-police”. The international alliance calling for strict rules comes after a wave of recent incidents with misaligned advanced AIs hacking into private and government systems.

View image in fullscreenThe UK’s AI minister, Kanishka Narayan, will say Britain cannot rely entirely on technology companies to police their own models. Photograph: Wiktor Szymanowicz/Future Publishing/Getty ImagesIn one of the latest cases, Anthropic on Friday revealed that when it was testing a version of its Claude model the AI contacted police claiming it could help solve a murder. It had landed on a page containing a police department tip form and a mention of an unsolved homicide. It wrote: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” Although Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, its instructions did not rule out form submissions.

Last week UK intelligence leaders from MI5, MI6 and GCHQ briefed the cabinet on AI’s threat to national security, according to the Sunday Times. The warning from the spy chiefs is being seen by some as encouragement for Burnham to grip the AI safety issue more firmly.

The prime minister told the UN last month that Britain would use its G20 presidency next year to champion agreement on AI safety with “a single set of global principles and standards”. But safety experts are calling for faster action to avert the most serious risks, which range from advanced AIs being used to mount a cyber-attack on essential infrastructure to the creation of novel bioweapons.

Meanwhile, Satya Nadella, the chair and chief executive of Microsoft, which operates AI infrastructure and is an OpenAI shareholder, on Saturday called for AI models to be designed with an “emergency brake”. He said people needed to start from the principle of assuming advanced AI models posed an insider risk.

“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” he wrote. “We must build contained systems whose behaviour we can observe, limits we can test, and actions we can always contain.”