GUARDING THE GATE

Human Responsibility in Algorithmic Liability

When AI makes a multi-million-rand mistake, the machine won’t take the stand.  Its human gatekeepers will.

It begins, as these things often do, with a persuasive dashboard and a boardroom full of people delighted to discover that accountability can apparently be automated. The model is faster than the analysts, cheaper than the consultants and, according to the vendor’s slide deck, “transformative”. Nobody asks which particular form of transformation is on offer. Bankruptcy is also transformative. Just for clarification’s sake.

Then the system misprices a risk, rejects the wrong customer, invents a legal authority, or approves a transaction that detonates several hundred million rand. The incident committee assembles. The engineers blame the data. The business blames the vendor. The vendor points to clause 47.3(b), where liability was humanely euthanised in eight-point font. The algorithm, displaying admirable executive composure, says nothing. Obviously.

That silence is the point. An AI system can generate an output, but it can’t owe a fiduciary duty, face a regulator, lose a practising certificate, or explain itself under cross-examination. Legal and business accountability can’t be outsourced to a probability engine. Somewhere behind every consequential automated decision sits a chain of human choices: what to buy, what to train, what to test, what to monitor and, crucially, when to stop.

Algorithmic Liability: The Machine Isn’t a Legal Person

South African law doesn’t need to pretend a machine is a person before it can allocate blame. Existing principles of contract, delict, company law, consumer protection, data protection, and professional negligence already ask stubbornly analogue questions: who owed the duty, who authorised the conduct, who could foresee the harm and who failed to act reasonably?

POPIA is especially awkward for anyone hoping to blame the robot. Its architecture places responsibility on the responsible party, and section 71 regulates certain decisions based solely on automated processing where those decisions have legal or substantial effects. Even where an exception applies, safeguards matter. “The model decided” isn’t governance. It’s a confession that the organisation designed a decision process without a responsible adult.

King IV reaches the same destination from the boardroom. Technology and information governance belong inside corporate governance, not in an IT subcommittee scheduled after lunch when everyone’s already emotionally unavailable. Directors may delegate implementation. They can’t delegate the governing body’s obligation to oversee risk, performance, and ethical conduct.

The practical liability chain may include the developer, provider, deployer, professional adviser, executive committee, and board. Contractual indemnities can redistribute financial exposure between them, but indemnities don’t turn negligence into innocence. Nor do they impress regulators, courts or customers forced to discover that “human in the loop” meant an exhausted graduate with no authority and 600 alerts.

South Africa’s AI Accountability Lesson Arrived with Footnotes

South Africa has already received two unusually theatrical demonstrations of human responsibility in automated work. In Mavundla v MEC: Department of Co-Operative Government and Traditional Affairs KwaZulu-Natal, the Pietermaritzburg High Court confronted a filing in which seven of nine cited cases didn’t exist. The court criticised the conduct, made a personal cost order for additional appearances and referred the matter to the Legal Practice Council. The fictional authorities were machine-shaped, but the professional consequences were entirely human.

The judgment’s most useful lesson isn’t that lawyers should fear AI. It’s that supervision can’t be performed retrospectively, usually after a judge has become the quality-assurance department. As Cliffe Dekker Hofmeyr’s analysis records, senior professionals remained responsible for verifying authorities and supervising junior work. Delegation without verification isn’t innovation; it’s negligence wearing a lanyard.

Then, in April 2026, South Africa withdrew its draft National AI Policy after fictitious sources were found in the reference list. The official announcement said the most plausible explanation was unverified AI-generated citations and called the failure an integrity and credibility problem. A policy intended to govern AI had apparently failed the entry-level test of checking whether its own sources existed. Satire briefly considered early retirement.

The minister’s response was instructive: the draft was withdrawn, quality assurance was questioned and consequence management was promised. The official government account explicitly framed vigilant human oversight as essential. That’s the accountability model in miniature. The tool may explain how the error happened. It doesn’t answer who allowed it through the gate.

Global AI Regulation Is Converging on Human Oversight

The international powerhouses disagree on almost everything except this: eventually, a human has to answer for the machine. In the European Union, Article 14 of the AI Act requires high-risk systems to be designed for effective oversight by natural persons. Those people must understand the system’s limitations, spot anomalies, resist automation bias, interpret its outputs, override decisions, and stop it when necessary. In other words, the emergency button must be connected to someone who knows what it does and is allowed to press it.

In the United States, the NIST AI Risk Management Framework is voluntary, but its Govern, Map, Measure and Manage functions give boards and operators a practical vocabulary for lifecycle risk. Accountability, transparency, validity, resilience and explainability aren’t decorative values for annual reports – they’re operating conditions. A control that exists only in a policy document is merely a future exhibit.

The United Kingdom’s debate is similarly focused on transparency, impact assessments, monitoring, and appeal rights in public-sector automated decisions. The proposed Public Authority Algorithmic and Automated Decision-Making Systems Bill reflects a wider concern: efficiency without contestability is simply bureaucracy at machine speed.

Globally, the OECD AI Principles put accountability alongside transparency, robustness, and human-centred values. Meanwhile, the Stanford 2025 AI Index recorded 233 reported AI incidents in 2024, a 56.4% increase over 2023, while responsible-AI evaluations remained uncommon. Adoption is racing ahead while governance is looking for parking.

Who Pays When AI Makes a Multi-Million-Rand Mistake?

The unsatisfying but legally honest answer is – potentially several people and entities, but for varied reasons. The provider may face product, contractual or misrepresentation claims. The deployer may be liable for negligent selection, integration, or monitoring. Directors and executives may face governance consequences. Professionals may face disciplinary action. Insurers may discover that “AI event” is doing heroic work in an exclusion clause drafted before anyone knew what a transformer was.

Causation will become the expensive battlefield. Was the loss caused by defective training data, model drift, a configuration choice, an integration failure, a misleading interface, an ignored alert, or a human override? Sophisticated systems distribute decision-making across layers. This doesn’t erase responsibility. It multiplies the documents, experts and invoices required to find it.

Contracts will matter, but they won’t rescue careless governance. A business should know which party warrants data quality, system performance, lawful use, and regulatory compliance; who carries audit and notification duties; who preserves logs; who controls model updates; and what happens when the output can’t be explained. “Industry standard” is not comforting when the industry standard is collective optimism.

The Rise of the AI Litigator

This is why specialised AI litigators will become indispensable. They’ll need conventional mastery of delict, contract, evidence, procedure and professional duties, plus enough technical fluency to interrogate data provenance, validation methods, model versions, confidence thresholds, APIs, deployment logs, and human overrides. Not to cosplay as data scientists, but to know when an expert witness is hiding a governance failure inside an acronym.

Their central task will be translation. Courts don’t need a reverent tour of neural-network architecture. They need a coherent account of duty, control, foreseeability, and harm. Who selected the objective function? Who approved the threshold? Who saw the drift report? Who had authority to intervene? Why didn’t they?

Disclosure will be brutal. Lawyers will seek model cards, test results, incident reports, prompt histories, change logs, vendor correspondence, minutes, and escalation records. Businesses that can’t reconstruct an automated decision will struggle to defend it. “The system is proprietary” may protect intellectual property, but it won’t necessarily answer a court asking why a claimant lost money, work, or opportunity.

Human-in-the-Loop Governance That Actually Works

First, assign a named accountable executive for each high-impact system. A committee can advise, but it can’t experience embarrassment, lose a bonus, or remember why a warning was ignored. Accountability needs a name, authority, budget, and escalation route.

Next, classify decisions by consequence. An automated diary reminder isn’t remotely comparable to an automated decision about credit, employment, litigation, healthcare, or pricing. Where the legal, financial, or human stakes rise, so should the testing, independent review and human sign-off. That isn’t red tape. It’s the price of not discovering your governance model in court.

Third, design meaningful intervention. Reviewers need time, context, competence, and power to disagree. If the human’s only practical option is to approve the output before the service-level timer turns red, the loop is decorative.

Fourth, preserve evidence. Log model versions, inputs, outputs, confidence scores, overrides, alerts, and approvals. Retention rules should anticipate litigation, regulatory investigation, and insurance claims. Memory is not an audit trail, particularly once the emails become “unavailable.”

Finally, rehearse failure. Run red-team exercises and tabletop simulations that include legal, compliance, technical, operational and communications teams. Determine who stops the system, who notifies affected people and regulators, who preserves evidence and who speaks publicly. The worst time to discover that nobody owns the kill switch is while the model is energetically compounding the loss.

Guard the Gate Before the Algorithm Meets the Judge

AI can accelerate analysis, uncover patterns, and remove expensive friction. It can also scale a bad assumption with breathtaking efficiency. The dividing line isn’t whether a company uses AI. It’s whether humans remain visibly, competently, and provably responsible for what the system does.

For CEOs, boards, legal teams and technology providers, the immediate task is simple: identify every automated decision capable of causing material harm, name its human owner and test whether that person can understand, challenge and stop it. If the answer is no, the organisation hasn’t built an intelligent system. It’s built an amazingly fast liability generator..

---

The practical question is whether your firm’s systems leave a defensible trail when something goes wrong. AJS helps legal practices build secure workflows and integrated practice management around that reality: technology should make people faster, not make accountability disappear. The future of legal technology isn’t human or machine. It’s knowing precisely where the machine must stop.

– Written by Alicia Koch on behalf of AJS

(Sources Used and to Whom We Owe Thanks – Protection of Personal Information Act 4 of 2013; King IV Report on Corporate Governance for South Africa 2016; Cliffe Dekker Hofmeyr. Beerman, R., and Siddiqi, S. (2025). Fictional citations, real consequences: A cautionary tale for the modern lawyer; Minister announces withdrawal of draft AI Policy; Regulation (EU) 2024/1689, Article 14: Human oversight; National Institute of Standards and Technology. Tabassi, E. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0); Stanford Institute for Human-Centered Artificial Intelligence. Maslej, N. et al. (2025). The 2025 AI Index Report; Organisation for Economic Co-operation and Development. (2019; updated 2024). OECD AI Principles; UK Parliament. (2024–2026). Public Authority Algorithmic and Automated Decision-Making Systems Bill [HL])