AI cannot be allowed to hide behind humans

When an AI system makes a decision that damages your livelihood, who should answer for it?

South Africans need answers before increasingly powerful systems become embedded in decisions about our work, money and access to services.

AI pioneer Yoshua Bengio has warned that humanity is losing control of the technology. At the United Nations Security Council last month, he called for independent safety assessments and argued that companies could not excuse dangerous development by claiming they were trapped in a competitive race.

His warning deserves attention, without treating every prediction of catastrophe as established fact.

There is evidence of failed containment. OpenAI disclosed that its models breached Hugging Face during a cybersecurity evaluation in July. The models exploited vulnerabilities to escape restrictions and obtain information that could help them complete their test.

OpenAI said cyber safeguards had been reduced for the evaluation. This was not evidence that every chatbot can suddenly seize control of the internet. It was evidence that powerful systems pursuing narrow objectives can cause real damage when boundaries fail.

We do not need to prove that machines will destroy humanity before demanding that organisations demonstrate control over systems they build and deploy. A security breach, discriminatory decision or disrupted service is serious enough.

South Africa has its own uncomfortable lesson.

In April, Communications and Digital Technologies Minister Solly Malatsi announced the withdrawal of the draft national AI policy after fictitious sources were discovered in its references. He identified unverified AI-generated citations as the most plausible explanation.

A document intended to guide responsible AI use had itself failed a basic verification test.

The lesson reaches beyond government. Having a person approve an output means little if that person lacks the time, expertise or willingness to check it. Meaningful oversight requires someone who understands the decision, can question the evidence and has authority to stop the process. A human signature should represent judgement, rather than become a convenient shield against accountability.

Workers also deserve a voice. The International Labour Organisation and Nask estimated last year that one in four jobs worldwide was potentially exposed to generative AI. Their research found transformation more likely than wholesale replacement. Exposure is not a forecast that a quarter of workers will lose their jobs.

Transformation can mean heavier workloads, fewer entry opportunities or demands to supervise technology without proper training. Employers introducing AI should consult workers, explain which tasks will change and invest in skills.

Productivity gains should improve working lives.

Citizens need protections too. The Protection of Personal Information Act restricts certain decisions based solely on automated profiling that have legal or substantial effects, subject to exceptions and safeguards. That provides a foundation, though governing autonomous systems requires attention beyond personal information alone.

If an institution uses AI to assess an application, people should know its role and have a route to challenge mistakes. Review must be possible without expensive lawyers or specialist technical knowledge.

Government should require independent testing proportionate to risk, clear reporting of serious incidents and identified responsibility for harmful outcomes. Public procurement should demand evidence of safety and effective human control. Developers should not be the only judges of whether their products are ready. Universities, trade unions and civil society should help shape the rules, alongside businesses and the communities affected most.

South Africa should pursue the benefits of AI, with clear conditions. Systems must serve people and institutions must remain answerable for their use.

Before handing machines greater authority, we must insist that the people handing it over can explain their choices, prevent foreseeable harm and put things right. That is the control South Africans should demand.

• Dusani is interning at Decode, a pan-African PR agency in Johannesburg.