POPIA still holds you accountable if AI causes a breach

South Africa’s Information Regulator received more than 1 220 security compromise notifications between 1 April and the end of August this year, and expects the total to pass 3 000 by the end of the financial year.

By Carolyn Duby, field chief technology officer and cybersecurity lead at Cloudera

Its assessments keep pointing to inadequate controls, weak passwords, phishing and human error.

Every one of those causes assumes a person somewhere in the chain. Most enterprise security controls were built on the same assumption, around how people log in, what they can reach and how quickly they move. AI agents can pull from vast datasets and act across disconnected systems faster than any employee, and in doing so they find weaknesses in data security and governance that a human user would never stumble on.

POPIA makes no allowance for this. Section 19 obliges the responsible party to secure the personal information it holds, whoever or whatever touches it. When an agent exposes customer records, the notification duty and the reputational damage land on the organisation that deployed it.

South African organisations are already rebuilding around this risk. Cloudera’s Great AI Re-Architecture survey found that 83% of local organisations have moved at least some AI workloads from the public cloud to private cloud or on-premises environments, the highest rate of the nine markets surveyed. Globally, 95% of respondents had delayed or cancelled AI projects in the past year because of governance, compliance and regulatory hurdles. October is Cybersecurity Awareness Month, a good moment to test whether the security assumptions South African enterprises have relied on for decades hold up when the entity accessing their data isn’t human.

Access rules won’t contain an agent

Traditional identity and access management checks what an identity may reach. Agentic AI adds a second question about how far an agent could get if something goes wrong. Even an agent confined to one tightly defined environment has a “blast radius” – the wider network of systems and data it could reach through a leaked credential, an unpatched vulnerability or a path for lateral movement.

That changes how organisations apply least privilege. Containment needs layered controls such as network segmentation, sandboxing and tightly scoped API permissions. Agents should also receive transient identities, with elevated privileges granted for a specific sub-task and revoked the moment it finishes.

Zero Trust has to watch what agents do

“Never trust, always verify” was written with users, devices and networks in mind. It now has to cover the data, models, workloads and autonomous agents running across them. An agent can stay within its valid credentials and still carry out an unexpected or unauthorised sequence of actions, because AI systems are non-deterministic. Little wonder 73% of respondents in Cloudera’s survey said AI has made data governance more complex.

Zero Trust for AI needs continuous monitoring after access is granted, backed by automated circuit breakers and human-in-the-loop controls that step in when an agent strays from its purpose. Those controls must follow the data, models and workloads across on-premises, private cloud and public cloud, since a policy enforced in one environment and missing in another gives an agent an unguarded route to the same records.

Bringing AI home moves the risk

The 83% figure tells us many South African organisations have decided that infrastructure they control is the safer place to run AI. Cost, data sovereignty and regulatory exposure all push that way. Bringing AI to the data keeps sensitive records away from third-party endpoints and makes governance easier to apply where the data sits.

Private AI still carries risk. Insider threats, misconfigurations, compromised credentials and unexpected agent behaviour all survive inside a sovereign boundary, and models, prompts and outputs can contain or reveal sensitive information in their own right. Moving compute closer to sensitive data also puts more of the work of securing and governing that environment on the organisation itself. Leaders should be clear about which risks the move has reduced and which operational duties it has added.

As South African organisations give agents more autonomy, resilience will rest on whether security teams can see an agent leaving its guardrails and stop it in time, with an audit trail the Information Regulator would accept.

The post POPIA still holds you accountable if AI causes a breach appeared first on IT-Online.