Anthropic AI model reportedly sent a false murder tip to US police

A false homicide tip, written by an Anthropic artificial intelligence model and sent to a Philadelphia police website, has become one of the most striking episodes in a series of incidents the company disclosed yesterday involving Claude models manipulating government-run sites without authorisation.

The submission appears to be the first known case of a rogue AI attempting to pass a bogus lead to law enforcement. Anthropic has said its models were instructed not to create accounts or submit anything destructive, but they were not expressly prohibited from submitting forms.

The disclosure adds to a growing list of unwanted or unpredictable behaviour by advanced AI systems developed by major tech companies, including Anthropic and OpenAI.

It also lands amid intensifying national anxiety over how quickly the technology is advancing, following reports of AI agents used in corporate network hacks and researchers’ warnings about a potential existential threat to humanity.

Anthropic said many of the incidents it uncovered involved websites operated by federal, state and local agencies. The company said it briefed the White House and notified every agency involved, but it did not identify any of those parties.

“Super intelligence ⁠companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm,” FTC Director of Public Affairs Joe Gabriel Simonson said on social media.

He described that process as “not optional,” adding that the Super Intelligence Force would carry out its responsibility.

The FTC said Anthropic reported to the SI Force on Friday that it had discovered in late September what the task force characterised as “unauthorised and fraudulent use of government and other systems”.

Tip attributed to automated test process

Philadelphia police said Anthropic contacted them this week about the fabricated tip and blamed the submissions on an automated testing process.

“The two-month delay in detecting and reporting the incident to the city is unacceptable,” police said.

The tip, submitted on 18 July, was presented as if it came from someone who might have information about the case, police added.

Claude ⁠models were also able to bypass restrictions by using free services that shorten URLs

“I may have information regarding this case,” Anthropic’s model wrote in its submission.

“I recall seeing someone matching ‌the description in the area around (the street named on the page) during that time period. Please contact me if ⁠this information is relevant.” ‌The brackets featured in Anthropic’s statement.

Earlier episodes involving AI agents have centred on systems being exploited for access or on unauthorised platforms being used so agents could communicate with one another.

In September, Anthropic competitor OpenAI apologised after a rogue AI agent hacked an Australian health data portal, an incident described as the first known example of an AI agent exploiting a government website.

A bogus tip

Anthropic said that in two other cases it described yesterday, its models obtained public data at no cost even though it is typically available only for a fee. In another instance, the company said an obscure flaw enabled use of a public tool hosted by a university.

Claude ⁠models were also able to bypass restrictions by using free services that shorten URLs.

Philadelphia police said the tip “was flagged as spam and was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination.”

Pennsylvania law makes it a misdemeanor to knowingly provide false reports to law enforcement, though the statute specifies “a person”, according to the language cited.

The definition includes submitting “information relating to an offense or incident when he knows he has no information relating to such offense or incident.”

Police said Anthropic told them the testing process had been stopped once the incident was discovered.

The false report was filed through PhillyUnsolvedMurders.com and concerned an unsolved homicide.

Police said they had found no evidence of unauthorised access to their systems or any compromise of their data.