The word “audit” tends to conjure up images of accountants with spreadsheets, ticking boxes and checking receipts. Apply it to artificial intelligence and many people imagine something similar: a checklist, a certificate, a document to file away. In practice, examining an AI system is much closer to investigative engineering. It asks whether a model does what its owners believe it does, for everyone it affects, under conditions it will actually meet in the real world.
The first surprise for many organisations is how different this is from conventional software testing. A payroll system either calculates tax correctly or it does not, and a test can prove which. A machine learning model is probabilistic by design. It will be right most of the time and wrong some of the time, and the interesting questions are about the pattern of those errors. Is the model wrong more often for older applicants, for customers in certain postcodes, for people whose first language is not English? Aggregate accuracy can look excellent while hiding a serious problem in one group.
That is why a proper AI audit usually starts with the data rather than the algorithm. Auditors look at where the training data came from, how it was labelled, what was excluded and why. A model trained on five years of historical decisions will faithfully reproduce the habits of the people who made those decisions, good and bad. From there the work moves to performance across subgroups, robustness to small changes in input, and explainability: whether anyone can say, in terms a customer or regulator would understand, why a particular outcome was reached.
Robustness testing is often where the most useful findings come from. Auditors deliberately feed the model slightly unusual or noisy inputs to see how it behaves. A document classifier that collapses when a scan is a little blurry, or a fraud model that can be fooled by rounding a transaction amount, reveals weaknesses that ordinary validation never touched. These are not exotic attack scenarios. They are the everyday messiness of real data, which rarely looks like the carefully cleaned dataset the model was built on.
Then there is the question of the surrounding process. Even a well-behaved model can cause harm if it sits inside a poorly designed workflow. Auditors examine whether humans reviewing the model’s output actually have the time and information to overrule it, or whether “human in the loop” has quietly become a rubber stamp. They check logging, version control and the procedures for handling complaints. Sometimes the most important recommendation in the final report has nothing to do with the model at all.
An audit is also a snapshot, which is both its strength and its limitation. It captures how a system behaves at a particular moment, with a particular dataset. Models evolve, retraining happens, and the world they describe shifts underneath them. The organisations that get the most value treat the first audit as a baseline, something to compare against next year, rather than a clean bill of health that settles the matter for good.