JUDGMENT CALLS

Why AI Can’t Compute Legal Ethics

The boardroom has acquired a new oracle. It doesn’t wear robes, pause for effect or charge in six-minute units. It produces an answer before anyone’s coffee has cooled, preferably in a dashboard with reassuring green ticks. Naturally, this has been mistaken for wisdom.

AI is exceptionally useful at finding patterns, comparing clauses, triaging documents and identifying familiar risks. Then the ground shifts. Legal ethics begins where familiarity ends, amid terms such as “reasonable”, “fair”, “good faith” and “public interest”. These aren’t drafting defects awaiting a software patch. They’re deliberate spaces in which facts, values and consequences must be weighed. Research on open texture in law describes these terms as an obstacle to automatic processing because their meaning isn’t exhausted by the words on the page.

AI Legal Ethics and the Seductive Myth of the Correct Answer

Most AI systems generate outputs from patterns in training data and instructions. That makes them powerful retrospective engines. It doesn’t make them moral agents. They can estimate what has usually happened after a set of facts. They can’t decide what ought to happen when the available options are lawful, ugly and commercially attractive – the corporate equivalent of finding a wallet and consulting the quarterly forecast.

Legal reasoning isn’t a slot machine into which precedent is fed until a judgment falls out. A precedent may be distinguished, extended or challenged because the social, technological or commercial context has changed. Sometimes the groundbreaking argument is simply that the old rule no longer fits the new reality. A model optimised to reproduce probable language will usually prefer the well-trodden path. That preference is useful until the path stops where the business problem begins.

This doesn’t mean AI can’t support legal work. It means fluency mustn’t be confused with authority. Automation bias – the tendency to over-rely on automated recommendations – is a recognised risk in human-AI collaboration, particularly in high-stakes settings. The machine’s great rhetorical advantage is that it never looks nervous. Confidence arrives pre-installed. But accountability remains an optional extra.

South Africa received a spectacular demonstration earlier this year. The draft national AI policy was withdrawn after fictitious sources were found in its reference list. The Minister said the most plausible explanation was that AI-generated citations had been included without verification and called the lapse proof of why vigilant human oversight matters. It’s difficult to improve on the satire of an AI policy hallucinating its own evidence, except perhaps by asking the hallucination to chair the inquiry.

POPIA, GDPR and the Grey Areas Automated Compliance Misses

Regulatory systems reward careful reading and punish casual equivalence. The GDPR protects natural persons and restricts certain decisions based solely on automated processing. POPIA, by contrast, defines a data subject to include a juristic person and expressly addresses automated decision-making. A compliance system trained around a European template can therefore deliver a serene green light while a South African regulator reaches for the paperwork.

The problem isn’t that the statutes are unknowable. It’s that similarity isn’t identity, and context can’t be reduced to keyword overlap. Territorial scope, lawful grounds, sector rules, legitimate interests, proportionality, constitutional values and the practical effect on a person or business all matter. A system can retrieve these factors. It can’t assume responsibility for balancing them.

European case law makes the point sharply. In OQ v Land Hessen (SCHUFA Holding), Case C-634/21, the Court of Justice of the European Union held that generating a creditworthiness probability score can amount to automated individual decision-making under Article 22 of the GDPR where a lender relies strongly on it when deciding whether to grant credit. In other words, an algorithmic score doesn’t escape scrutiny merely because a human formally makes the final decision. In a later case concerning access to profiling information, the Court insisted on meaningful information about the logic involved – not a shrug, a trade-secret incantation and a brochure featuring diverse people smiling at laptops.

Algorithmic Bias: When Yesterday’s Injustice Becomes Today’s Workflow

Historical data carries history’s bruises. Hiring, lending, insurance and fraud systems can reproduce unequal outcomes while presenting them as neutral mathematics. Research on algorithmic bias points to opacity, fragmented regulation and enforcement gaps. Code doesn’t become objective merely because nobody remembers who encoded the assumptions.

That distinction matters for legal ethics. A rule may be applied consistently and still produce an indefensible result. A screening model may never mention a protected characteristic yet rely on proxies that reproduce exclusion. The legal question isn’t only whether the model followed its specification. It’s whether the specification, data and outcome can survive scrutiny under equality, privacy, labour and administrative-law principles. Because “the computer did it” remains a poor pleading strategy.

The greater danger is institutional. Once a model is embedded in a workflow, its output becomes the default. Challenging it costs time, demands confidence and creates a paper trail. Accepting it is frictionless. Thus, an advisory tool quietly becomes a decision-maker, while everyone continues to insist that a human remains “in the loop”. The human is indeed in the loop, usually clicking “approve”.

Board AI Governance: Reputation Isn’t a Spreadsheet Cell

Boards don’t live in a purely legal universe. They must weigh revenue, operational resilience, stakeholder trust, employee impact and the possibility that tomorrow’s headline will translate today’s efficiency initiative into plain English. This is where human counsel earns the part of the fee that can’t be replaced by autocomplete.

Current governance evidence isn’t comforting. The Thomson Reuters Foundation’s 2025 research covered 2,972 companies: nearly 90% hadn’t publicly committed to a named AI governance framework, and only 13% had a policy ensuring human oversight. The American Arbitration Association’s 2026 survey of 500 senior legal and executive leaders found that 87% reported some form of AI governance, although only 22% believed it worked effectively. Technology teams were involved in 80% of cases. Legal and compliance teams appeared in just 35%. Apparently, the lawyers are still being invited after the smoke alarm has developed an opinion.

South African governance expectations have also moved on. King V superseded King IV and applies to financial years beginning on or after 1 January 2026. It retains the core insistence on ethical and effective leadership while responding to a governance landscape reshaped by technology and regulation. Technology governance isn’t an IT appendix. It belongs inside strategy, risk, assurance and accountability.

Reputational risk is particularly resistant to computation. A model can estimate sentiment, media volume and likely customer churn. It can’t know which technically lawful decision will become a symbol of corporate contempt, nor whether an apology will sound sincere after the internal memo appears in discovery. Reputation is the public’s accumulated judgment about character. Character doesn’t fit neatly into a dropdown menu. Unfortunately.

Why Groundbreaking Business Strategy Still Needs Human Lawyers

Groundbreaking business moves are awkward for probabilistic systems because the relevant precedent may not exist. A new platform, ownership model or data use may sit between regulatory categories. The easy answer is “no” because yesterday provides no comforting example. The reckless answer is “yes” because no rule expressly forbids it. Good counsel occupies the unpleasant middle: testing purpose, constitutional values, enforcement posture, stakeholder harm and the organisation’s capacity to defend its position in public as well as in court.

Lawyers sometimes have to argue for the spirit of a law over a wooden reading of its text. That isn’t licence to ignore legislation whenever profit looks lonely. It’s disciplined interpretation: identifying the purpose of the framework, explaining why an existing category misfits the facts, and constructing a position that a regulator or court can evaluate. Machines can marshal authorities for that argument. They can’t feel the weight of asking a board to become the test case.

 

A Practical Human-in-the-Loop Test for Legal AI Decisions

Before a material AI-assisted decision reaches the board, counsel should test the recommendation from five directions. What facts and assumptions produced it? Which legal uncertainties were quietly converted into fixed rules? Who can override the output – and when did that last happen? What harm could arise even if the decision is lawful? Finally, would the organisation defend the decision, using the same words, to a regulator, employee, customer and journalist?

If those questions don’t have clear answers, the problem isn’t insufficient automation. It’s insufficient judgment. AI should widen the field of vision, not narrow responsibility to whatever the model happened to notice. The system may draft the options, retrieve the law and model the exposure. The board and its counsel must still decide what kind of organisation is making the choice.

---

The Competitive Advantage Is Knowing When Not to Obey the Machine

The future of legal technology won’t be won by pretending uncertainty has been abolished. It’ll be won by designing systems that expose uncertainty, preserve challenge and record the reasons for human decisions. The valuable legal adviser won’t compete with AI on document retrieval. They’ll know when precedent is a guide, when it’s a trap and when a business is about to confuse regulatory silence with moral permission.

AI can calculate probabilities. It can’t carry professional duties, absorb reputational shame or explain itself under oath with a straight face. For the decisions that define an organisation’s character, human judgment isn’t an inefficient legacy feature. It’s the part of the system that can be held responsible. In a world addicted to frictionless answers, that responsibility is becoming more valuable.

– Written by Alicia Koch on behalf of AJS

(Sources Used and to Whom We Owe Thanks: Al-Abdulkarim, L., Atkinson, K. and Bench-Capon, T. (2024), “The challenge of open-texture in law”, Artificial Intelligence and Law, Springer Nature; Al-Abdulkarim, L. and colleagues (2025), “Identifying open-texture in regulations using LLMs”, Artificial Intelligence and Law, Springer Nature; Republic of South Africa (2013), Protection of Personal Information Act 4 of 2013, Department of Justice and Constitutional Development; European Parliament and Council of the European Union (2016), Regulation (EU) 2016/679: General Data Protection Regulation, EUR-Lex; Court of Justice of the European Union (2023), OQ v Land Hessen, Case C-634/21 (SCHUFA Holding), EUR-Lex judgment; Court of Justice of the European Union (2025), CK v Dun & Bradstreet Austria GmbH, Case C-203/22, Court press release and judgment summary; South African Government News Agency (2026), “Minister announces withdrawal of draft AI Policy”, SAnews; Government of South Africa (2026), Withdrawal of the Draft South Africa National Artificial Intelligence Policy, Government Gazette; Institute of Directors in South Africa (2025), King V Report on Corporate Governance for South Africa, Official King V page; Lendvai, G. F. and Gosztonyi, G. (2025), “Algorithmic Bias as a Core Legal Dilemma in the Age of Artificial Intelligence: Conceptual Basis and the Current State of Regulation”, Laws, 14(3), 41, MDPI; Thomson Reuters Foundation (2025), Responsible AI in Practice: 2025 Global Insights from the AI Company Data Initiative, Corporate AI Governance Report 2025; AAA-ICDR Institute (2026), From Principles to Practice: A Benchmark Study in AI Governance, American Arbitration Association survey findings; and Romeo, G. and Conti, D. (2025), “Exploring automation bias in human–AI collaboration: a review and implications for explainable AI”, AI & Society, Springer Nature).