THE ALGORITHMIC SEANCE

Reconstructing an AI Decision After the Machine Has Spoken

When AI’s Perfect Memory Becomes Conveniently Paranormal…

September took us into the Friction Economy: that expensive little collision between hyper-automation and the stubbornly unautomated human. October takes the argument somewhere colder. Because knowing where the risk lives is one thing. Inviting it into the business, giving it a login, and then pretending not to recognise it 18 months later is quite another. By then, somebody may have to explain why an algorithm refused a customer, flagged an employee, priced a deal, or quietly recommended a dismissal.

The model may have been retrained, the data pipeline changed and the prompt library “improved” by somebody who has since left to disrupt another industry. The vendor may have released 9 versions, 3 patches and a soothing blog post featuring the word “trust” rather more often than the evidence warrants. The approving executive may recall only that the dashboard was green. Unfortunately, litigation, regulation and internal investigations aren’t terribly interested in the system’s current personality. They want to know what this system did to this person, on that day, using those inputs, under that policy. Preferably before everyone involved develops selective amnesia.

That’s the algorithmic séance – lawyers, auditors and technologists gathered around a dead decision, asking the machine to knock twice if a human ever checked it. There’ll be candles, confidence and, inevitably, a PowerPoint titled “Responsible AI Journey”. There may even be biscuits. What there won’t be, unless somebody planned for it, is a reliable account of what actually happened. My position is simple – if a business chooses to rely on an automated decision, it must also be able to reconstruct that decision. Otherwise, the efficiency was private, but the evidentiary risk becomes everyone else’s problem.

AI Model Drift Turns Yesterday’s Decision Into Today’s Ghost

Machine-learning systems aren’t filing cabinets. They change because their data changes, their environments change and their owners quite reasonably want better performance. IBM’s explanation of model drift describes how performance can degrade when data, or the relationship between inputs and outputs, shifts over time. Monitoring and retraining may improve tomorrow’s result. They may also make yesterday’s result impossible to reproduce unless the organisation preserved the old model, configuration, and context.

Generative AI adds further ectoplasm. A response may depend on a system prompt, user prompt, retrieved documents, access permissions, model version, temperature setting, safety layer, and vendor-side change that the customer never saw. Preserve only the final output and you’ve kept the corpse but misplaced the crime scene.

This isn’t an argument for freezing every system in amber. Continuous optimisation is commercially sensible. So is replacing defective models. The problem is confusing improvement with erasure. A company can run a current system and retain a defensible historical snapshot. It merely has to decide that evidence matters before somebody in a robe, regulator’s office or hostile board meeting makes the same discovery for it.

AI Audit Trails Are Evidence, Not Decorative Governance

The regulatory direction isn’t subtle. Article 12 of the EU AI Act requires high-risk AI systems to permit automatic recording of events over their lifetime, with logging capabilities appropriate to the system’s purpose. The AI Risk Management Framework from NIST (the US agency that develops technology standards and guidance) centres governance, mapping, measurement, and management across the AI lifecycle, while ISO/IEC 42001 supplies a management-system structure for responsible AI, including traceability, transparency, and continual improvement. Different instruments, different acronyms, same deeply unromantic message – if accountability matters, keep the records. Preferably somewhere more dependable than “Colin’s” inbox.

For South African businesses, section 71 of the Protection of Personal Information Act 4 of 2013 (POPIA) is where the haunting becomes local. It’s especially relevant when a decision with legal consequences or substantial effects is based solely on the automated processing of a person’s profile. The safeguards include an opportunity to make representations and access to sufficient information about the underlying logic. In other words, “the vendor says it’s proprietary” isn’t a legal explanation. It’s a procurement confession wearing Louboutin’s and hoping nobody asks who signed the purchase order.

In England and Wales, the courts are becoming less enchanted by AI’s “magical” aura. The 2025 guidance for judicial office holders stresses personal responsibility for material produced in one’s name, warns about hallucinations and bias, and ties AI use to the integrity of justice. That guidance addresses judicial users in a specific jurisdiction, but its instinct travels well – humans don’t become optional merely because software speaks in complete sentences.

How To Reconstruct an AI Decision Without Summoning the IT Department

A defensible record begins with a decision inventory. Identify which automated outputs can affect rights, money, employment, access, safety, reputation, or regulatory exposure. Not every autocomplete suggestion deserves a digital mausoleum. A credit decision probably does. So does an investigation alert that can trigger disciplinary action, or a legal tool that classifies privileged documents.

For each material decision, preserve the date and time, the model and vendor version, the system and user prompts, the relevant source material, the input data or a verifiable reference to it, the output, any available confidence score, the applicable policy, the human reviewer, any override or approval, and the action that followed. That sounds tedious. Because it is. So is discovery. Add access controls, integrity checks, and a retention schedule, because a log that anyone can quietly edit isn’t an audit trail. It’s collaborative fiction with metadata.

Then preserve the surrounding system. Version the model cards, validation results, risk assessments, operating thresholds, and change approvals. Record when drift was detected and why retraining occurred. Capture the human process too – who was authorised to rely on the output, what training they received and when they were expected to intervene. “Human in the loop” is meaningless if the human’s sole function was to click Accept before lunch, whilst doing their nails.

Finally, test reconstruction. Select an old decision and ask an independent team to recreate the material steps from the retained record. If they can’t, the organisation doesn’t have an audit trail. It has an eternally optimistic folder structure. Tabletop exercises should include legal, compliance, information security, records management, procurement, and the business owner. Yes, that’s a meeting. Apparently even AI can’t save us from those.

Vendor Contracts Must Preserve the AI Evidence Chain

Many corporate AI systems are rented, embedded, or layered across vendors. The business may own the decision but not the model, logs, or release history. That asymmetry should be negotiated, not discovered during disclosure. Contracts should specify logging, retention, export formats, version notices, incident cooperation, access to technical personnel, evidentiary support, subcontractor controls and what happens when the service ends.

Beware the clause promising that logs will be retained according to the vendor’s “standard policy”. Standard for whom? For how long? In which jurisdiction? Can the customer place a legal hold? Can a regulator receive intelligible records rather than 40 million lines of machine telemetry and a sympathetic shrug? The cloud isn’t a jurisdiction-free spiritual realm. It’s somebody else’s computer, running somebody else’s update schedule, under terms drafted by somebody else’s lawyers.

Legal teams should also resist buying “explainability” as if it were a feature beside dark mode. An explanation must be useful to its audience and connected to the actual decision. A generic list of influential factors may help a data scientist tell an affected person almost nothing. The record must translate technical causation into legally and commercially meaningful reasons, without revealing more personal, confidential, or security-sensitive information than necessary.

Should Missing AI Records Trigger an Adverse Inference?

Now for the unpleasant question. If a business can’t recreate the AI decision it relied on, should a court or regulator be entitled to draw an adverse inference from the missing record?

A blanket presumption would be crude. Systems fail, records corrupt and proportionality matters. It could also encourage defensive data hoarding, with privacy and security consequences of its own. But allowing the party that deployed the system, controlled the evidence, and benefited from the decision to plead technological amnesia is much worse. It rewards opacity and transfers the evidentiary disadvantage to the person least able to cure it.

A more defensible answer is an adverse inference, calibrated to materiality, control, and the organisation’s legal duties. If a business was reasonably expected to preserve the decision record, failed to do so and can’t produce a credible reconstruction, a court or regulator should be entitled to infer that the missing evidence wouldn’t have helped it. That isn’t hostility to technology. It’s hostility to a party improving its case by losing the evidence it controlled. We already understand that burning the map shouldn’t make the traveller more believable. Machine learning doesn’t require us to become superstitious about the principle.

AI Governance Begins Before the Litigation Invitation Arrives

Boards don’t need to understand every parameter. As per usual. They do however need to ask whether material AI decisions are identifiable, reviewable, and reproducible. And whether retention rules align with legal exposure, whether vendors can supply the necessary record, and whether senior accountability survives outsourcing. The answer can’t be “the data team has a dashboard”. Dashboards are lovely. So are lava lamps (they are really). Neither is governance.

The commercial advantage is larger than compliance. A business that can reconstruct decisions can resolve complaints faster, defend good outcomes, identify recurring harm, negotiate stronger vendor terms, and improve systems without deleting institutional memory. Proper records don’t make innovation slower. They stop innovation becoming archaeology.

That’s the afterlife of automated power. The decision leaves the screen, but it doesn’t leave the organisation. It hangs around as an obligation, a cost, a precedent and, on a particularly bad Tuesday, an exhibit. When the ghost returns, the frightening words won’t be “the algorithm decided”. They’ll be “we can’t show you how” – usually followed by a silence in which several careers reconsider their options.

---

Build Defensible AI Records With AJS

AJS helps legal teams turn governance from a haunted policy document into a working record – connected matters, approvals, documents, workflows, and reporting that people can actually find when the lights flicker. If your organisation’s AI decision trail depends on inbox archaeology, vendor goodwill and one employee who “knows where everything is”, you don’t have a system. You have folklore. Truth. Get in touch with AJS to build legal operations that preserve the evidence before the séance begins – while everyone still remembers the password.

– Written by Alicia Koch on behalf of AJS

(Sources Used and Those to Whom We Owe Thanks – European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence, particularly Article 12 on record-keeping for high-risk AI systems; National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023, and Generative Artificial Intelligence Profile, 2024; International Organization for Standardization. ISO/IEC 42001:2023 – Artificial intelligence management systems; Republic of South Africa, Department of Justice and Constitutional Development. Protection of Personal Information Act 4 of 2013, particularly section 71 on automated decision-making; Courts and Tribunals Judiciary of England and Wales. Artificial Intelligence (AI) – Judicial Guidance, October 2025 and IBM. What Is Model Drift?, 16 July 2024)