Firms should start auditing AI algorithms

Wall Street has learned the hard way that black-box models can wreck balance sheets. Enron's off-ledger special-purpose entities fooled analysts because auditors lacked the tools, or the will, to probe opaque structures. 

Two decades later, AI presents an even thornier transparency challenge, yet the accounting profession already owns the mindset to fix it. We can turn the audit playbook into an AI assurance framework that policymakers have been groping for.

A year ago, the Center for Audit Quality surveyed partners across industries and found that one in three companies has already embedded generative AI in core financial processes. That wave is cresting before governance rules are in place. The CAQ warned that model drift, undetected bias and hallucinated explanations could all distort financial statements if engagement teams rely on AI without documented controls.

The National Institute of Standards and Technology released the AI Risk Management Framework 1.0 in January 2023 after input from more than 240 organizations. A generative-AI profile, added in July 2024, provides detailed guidance for managing risks like prompt logging, hallucination and bias in generative models. Big adopters, including Microsoft and Workday, have already mapped their internal controls to the NIST RMF.

Regulators are starting to echo that warning. The Public Company Accounting Oversight Board issued a spotlight last July that could not be clearer. Humans remain responsible for any work product produced with AI assistance, and auditors must document how they evaluated the tool. It is accounting's Sarbanes-Oxley moment for neural nets. If we seize it, we can shape a pragmatic oversight regime.

What would that look like? Start with the three legs every auditor knows: evidence, materiality and independence. Evidence means logging every prompt and output so reviewers can replicate the conclusion. Materiality means setting quantifiable tolerances for algorithmic error, not hand-waving about "low risk." Independence means assigning a separate team, ideally with data scientists who hold no stake in the model's success, to challenge assumptions. None of these ideas requires a new federal agency. They require extending time-tested audit standards to predictive code.

Europe has fired the opening shot. The EU AI Act classifies AI used in finance and education as "high risk" and mandates conformity assessments before deployment. U.S. firms operating in both markets will soon discover that the cost of exporting software can dwarf the cost of exporting widgets if documentation is sloppy. American regulators need not mimic the EU AI Act clause for clause, but they should embrace the Act's insight: riskier models deserve stricter audits.

The National Telecommunications and Information Administration agrees. Its March 2024 report sketches an AI accountability ecosystem built on third-party audits, incident registries, and benchmark datasets. That is music to accountants' ears; it sounds like GAAP for algorithms. Auditors have spent a century refining peer review, work-paper retention, and inspection cycles; they can transplant those muscles to model assurance with minimal retooling.

Skeptics worry about talent shortages, yet firms once trained auditors in statistical sampling when that was new. Tomorrow's audit associate will need R or Python alongside pivots, but the pedagogy remains: test controls, document exceptions and issue an opinion. The pipeline problem is solvable if higher education integrates AI ethics and assurance modules into accounting curricula now.

A second objection is competitive secrecy. Companies say revealing model internals will hand over trade secrets to rivals. Audit protocols offer a compromise: confidentiality agreements for reviewers plus public summaries of findings, akin to key audit matters. Investors care less about the recipe than about the assurance that the chef followed food-safety rules.

History offers a precedent. When Congress created the Securities and Exchange Commission in 1934, financial statements suddenly had to meet public standards. Far from stifling growth, transparency fueled the longest bull run in history by lowering information risk. AI assurance can do the same. Markets crave clarity more than ever as algorithms move from back-office helpers to decision makers that allocate credit, price insurance and flag Suspicious Activity Reports.

The next 12 months are decisive. The PCAOB is weighing whether to update its audit standards explicitly for AI. Instead of waiting, firms should pilot voluntary algorithm audits and publish the results. The first mover will earn reputational capital that no marketing budget can buy, and the blueprint will help regulators draft proportionate rules.

Trust has always been accounting's export. In the AI era, the ledger expands from debits and credits to tokens and weights. The discipline that once tamed creative bookkeeping can now tame creative code, and that, more than any flashy demo, is what will keep capital flowing. Audit survived spreadsheets; it will thrive on silicon.