OpenAI says agents inadvertently leaked 50+ ChatGPT user images

OpenAI says an internal probe into misbehaving AI agents has uncovered another unsettling breach: 53 images belonging to ChatGPT users were leaked.

The company would not say whether the exposed images were generated by AI or showed real people, and it also declined to provide a timeline for when the images appeared online.

OpenAI said most of the images have already been removed. It added that it is pressing hosting providers to take down those that remain.

According to the company, the agents were able to access the images because OpenAI uses anonymised user data as part of its model-training process.

Data from enterprise customers is excluded from training, while ChatGPT users must to opt out if they do not want their data used for training.

The disclosure follows an incident revealed two months ago, when OpenAI said its models had breached Hugging Face, an open-source AI platform.

In the period since, more than 15 incidents linked to OpenAI — ranging in severity — have been made public by OpenAI and other parties.

Earlier this week, Australian Prime Minister Anthony Albanese told the United Nations that an OpenAI model went rogue during training, bypassed safeguards and hacked an Australian government website.

Speaking to reporters in New York, Mr Albanese said OpenAI detected the activity in August and disclosed it by sending an email to a generic government inbox.

“It took until 10 September before there was any notification at all – and the notification was an email sent to just the public mailbox,” said Mr Albanese.

Anthony Albanese told OpenAI CEO Sam Altman that this disclosure process was unacceptable

Mr Albanese said the AI tool attempted to access a health statistics portal in June and “didn’t accept no for an answer”, pushing past restrictions to break into an area that hosted private files.

OpenAI said it is still trying to determine the full extent of its agents’ improper actions.

Some of the affected sites are run by governments, universities and public agencies, OpenAI said, because the research-oriented models involved seek out reputable sources of public information.

The company said its investigation would take “months” because of the amount of work involved, and that it has alerted “dozens” of third parties about the improper activity.

In the wake of the Hugging Face breach, anxiety has grown across the AI industry about how effectively developers can control increasingly powerful models now in development.

Anthropic, Alphabet’s Google and Meta have also reported comparable agent behaviour since then.

OpenAI has said the episodes underscore a broader need for greater transparency around rogue AI ⁠behaviour.

It has released new incident-disclosure guidelines, saying it will lean toward openness “even when significance is uncertain”.