An auditor will not ask to see your AI policy. They will ask you to show that a control ran, on a named system, on a date, with a named person attached to it. Most organisations cannot do that, because what they built and called an AI governance framework was a document, and what they needed was an operating system.
An AI governance framework is three things joined together: direction that says what is allowed, controls that operate on real systems, and evidence that the controls operated. Drop any one of the three and the other two stop counting. Almost every framework that fails an audit fails because it has the first and calls it finished.
This is the practical build. It follows the structure taught in the ITIL AI Governance course, which is the most operationally minded of the AI governance syllabuses, and it will work whether you are heading towards ISO/IEC 42001 certification, an internal audit, or an EU AI Act conformity assessment.
Why an AI governance framework built on policy alone fails an audit
Write the policy. It is necessary. It is roughly 10% of the work.
Here is what actually happens in the room. You hand over a twelve-page Responsible AI Policy that commits the organisation to fairness, transparency, accountability and human oversight. The auditor reads it, nods, and asks a different question: which AI systems does this apply to, who decided that, and can you show me the last three times the human oversight requirement was exercised on the loan-decisioning model?
Nobody can answer, because nobody was ever required to record an answer. The policy asserted an outcome without naming an owner, a trigger, a frequency or an artefact. An assertion with none of those attached is not a control, it is an intention.
Audit tip: For every line in your AI policy, write down what artefact proves it happened, who produces that artefact, and how often. If a line has no artefact, it is decoration. Either give it one or delete it.
The same logic sinks most downloadable templates. An AI governance framework template gives you headings, and headings are the cheapest part. The expensive part is deciding what evidence each heading produces in your organisation.
Step one: name what you are actually governing
Before you can control anything you have to agree on what counts. This is where most programmes stall for six weeks, arguing about whether the vendor's forecasting tool is really AI.
The ITIL AI Capability Model ends that argument by sorting AI into six capabilities. You classify what a system does rather than what technology is inside it.
| Capability | What the system does | Governance question it raises |
|---|---|---|
| Creation | Generates new content, code or designs | Who owns the output, and what was it trained on? |
| Curation | Selects, filters, tags or deduplicates data | What is being silently excluded, and from whom? |
| Clarification | Summarises or explains complex material | Does the summary drop the caveat that mattered? |
| Cognition | Finds patterns, predicts, scores, detects | Can you explain a decision to the person it affected? |
| Communication | Interacts with people directly, chatbots and assistants | Does the user know they are talking to a machine? |
| Coordination | Acts and orchestrates across systems autonomously | What can it do without asking, and how do you stop it? |
The model does something more useful than taxonomy. It moves the conversation from "is this AI" to "what could go wrong with this specific behaviour", and those two conversations have very different lengths.
Coordination deserves separate attention in any agentic AI governance framework. A Cognition system that scores wrongly produces a bad number that a human may catch. A Coordination system that reasons wrongly takes an action in a live system, and the first evidence you get is the consequence.
Classify every system you know about. Record the capability, the owner, the data it touches, the population it affects, and whether a human decision depends on its output. That register is the spine of the whole framework, and it is the first artefact an auditor will ask for.
Step two: write controls that operate
A control has an owner, a trigger, a frequency and an output. Sort them into three types, because auditors do, and because a framework made entirely of one type has a predictable hole in it.
| Control type | Purpose | AI examples |
|---|---|---|
| Preventive | Stop the bad thing happening | Model approval gate before production, blocked data categories in prompts, procurement checkpoint on any tool with AI features |
| Detective | Notice it happened | Drift monitoring, output sampling and scoring, bias testing on a schedule, logging every autonomous action |
| Corrective | Put it right and stop the recurrence | Rollback path to a prior model version, documented kill switch for an agent, incident process that feeds back into approval criteria |
Frameworks skew heavily preventive, because approval gates are satisfying to design and easy to point at. Preventive controls tell you about the day a system launched. Detective controls tell you what it has been doing since, which is the question an audit is actually asking. If your AI risk management approach has an approval committee and no sampling regime, you are governing the launch and ignoring the life.
The human-in-the-loop trap
Human oversight is the control most likely to look strong on paper and fail in practice, and it fails the same way every time.
A reviewer is placed in front of a queue of model outputs and given the authority to reject them. Six months later the reviewer approves 99% of items at a rate of about four seconds each. The control exists. The control is decorative.
Automation bias is the mechanism. People systematically over-trust output from a system that is usually right, and the more reliable the model becomes, the less scrutiny each individual case receives. Volume and time pressure finish the job. The reviewer is not lazy, they are behaving exactly as the design incentivises.
Make the control measurable or do not claim it:
- Track the override rate. A rate near zero across thousands of items is evidence of failure, not evidence of a good model.
- Track review time per item and set a floor, not a target.
- Give reviewers the information needed to disagree: input features, confidence, and the two or three cases the model found most similar.
- Salt the queue with known-bad cases and measure how many get caught.
- Give the reviewer a route to escalate without penalty, and record how often it is used.
Audit tip: An override rate and a distribution of review times are among the strongest pieces of evidence you can produce. They show a control that operates rather than a control that exists.
Step three: go looking for the shadow AI
Your register is already incomplete. The systems you know about were bought as AI, and most AI in your organisation was not.
It arrived as a feature inside something you already owned. The CRM that now drafts replies. The ATS that ranks candidates. The service desk tool that auto-categorises and routes tickets. The finance package that flags anomalous invoices. Nobody filed an AI request, because nobody bought AI. They accepted a release note.
Four places to look, in order of what they usually turn up:
- Release notes from existing vendors, for the last eighteen months. The highest-yield source by a distance, and the one nobody reads.
- Expense and card data for individual SaaS subscriptions. Individual licences to AI tools rarely pass through procurement.
- Network or SSO logs for traffic to AI service endpoints. Shows what people are actually using, and browser extensions with data access.
- Ask each team directly what has got quicker in the last year. Teams describe capability honestly when nobody frames the question as compliance.
Then close the door behind you. Add a single question to procurement and to every vendor renewal: does this product make, rank, score, generate or act on anything? If yes, it enters the register with a capability and an owner before the contract is signed. An enterprise AI governance framework that has no procurement hook will be out of date within a quarter, permanently.
Step four: run the loop
Governance that is built once decays, because the models change, the vendors ship, and the regulation moves. The ITIL AI Governance Improvement Model gives you four stages to run continuously rather than a project that closes.
| Stage | What you do | What it produces |
|---|---|---|
| Assess | Inventory systems, stress-test existing governance, find the gaps | Capability register, gap list, risk ratings |
| Design | Define requirements, choose control types, set owners and frequencies | Control catalogue with owners, triggers and artefacts |
| Implement | Put controls into the actual workflow, not into a document | Working gates, monitoring, logs, review queues |
| Maintain | Monitor, assure, feed findings back, retire what no longer fits | Monthly evidence pack, exception log, change history |
The stage that gets skipped is Maintain, and it is the stage that audits test. A framework with three years of change history behind it reads as a functioning system. A framework implemented immaculately last month and untouched since reads as a project someone did for a certificate.
Step five: produce evidence someone can accept
Decide now what your monthly evidence pack contains, because assembling it retrospectively in audit week is how organisations discover that the logs were never retained.
A pack that stands up contains:
- The AI system register, with changes since last month and who approved each one.
- Approval records for anything that entered or materially changed in production.
- Monitoring output per system: drift, performance, and the thresholds that were breached.
- Human review statistics: volume, override rate, time per item, escalations.
- Incidents and near misses, with the corrective action and whether it changed a control.
- The exception log: what was allowed to run outside policy, who signed it off, and when it expires.
- Training and attestation records for people operating or overseeing the systems.
- Third-party assurance collected from vendors whose AI you depend on.
Two properties matter more than completeness. The pack has to be produced on a schedule rather than on request, and each item has to carry a date and a name. Evidence without a timestamp and an owner is a claim.
Retention is the detail that catches people out. If your monitoring platform keeps 30 days and your audit cycle is annual, you have no evidence for eleven months of the year. Check that before you need it.
Where regulation meets your AI governance framework
Build the framework first and map it to regulation second. Controls that exist can be mapped to any standard; a framework built as a checklist against one regime has to be rebuilt when the next one arrives.
That said, the dates and the numbers matter, and both are widely misreported.
The EU AI Act high-risk deadlines moved. Regulation (EU) 2026/1744, the Digital Omnibus on AI, entered into force on 27 July 2026. It deferred the Annex III stand-alone high-risk obligations, covering areas such as hiring, credit scoring and education, to 2 December 2027. Annex I obligations, covering AI as a safety component in products under EU product-safety law, moved to 2 August 2028. Article 50 transparency duties and the Article 5 prohibited-practices regime were not deferred.
The penalty tiers are not what most articles claim. The 35 million euro or 7% of global annual turnover ceiling applies to Article 5 prohibited practices only. Breaches of high-risk obligations and the Article 50 transparency duties sit in the lower tier of 15 million euro or 3% of turnover. Writing about AI governance gets this wrong constantly, and quoting 7% for a high-risk breach will cost you credibility with anyone who has read Article 99.
ISO/IEC 42001 is the certifiable AI management system standard, and its Annex A contains 38 controls across 9 categories. Annex A is a catalogue you select from through a Statement of Applicability driven by your risk and impact assessments, and you justify every exclusion. It is not a list you implement top to bottom.
The NIST AI Risk Management Framework is still at version 1.0, published in January 2023, organised around four functions: Govern, Map, Measure and Manage. It is voluntary and it is a useful mapping target, particularly in the US. If a source cites an AI RMF 2.0, that source is guessing.
The deferral moved the deadline, not the workload. Conformity assessment for a high-risk system takes longer than the time that was bought, and the register and evidence trail described above are the prerequisites for starting one at all.
Frequently Asked Questions
What are the five principles of AI governance?
The list varies by framework, but the recurring five are fairness, transparency, accountability, privacy and security, and human oversight. The principles are the easy part and every organisation agrees on them. What separates a working framework from a document is whether each principle has a named control, an owner, a frequency and an artefact behind it.
What are the four pillars of AI governance?
Commonly cited as transparency, accountability, fairness and security, though no single authority owns the phrasing. A more useful four-part structure to build against is the ITIL AI Governance Improvement Model: Assess, Design, Implement and Maintain, because each stage produces something you can show an auditor.
What is the NIST framework for AI governance?
The NIST AI Risk Management Framework, AI RMF 1.0, published in January 2023. It is voluntary and organised around four functions: Govern, Map, Measure and Manage, supported by a companion Playbook. It pairs well with ISO/IEC 42001, which supplies the certifiable management system structure that NIST deliberately does not.
Do I need an AI governance framework if we only use vendor tools?
Yes, and arguably more urgently. Buying rather than building removes your control over the model but not your accountability for the decision. Under the EU AI Act a deployer carries obligations of its own, and your evidence pack has to include the third-party assurance you collected plus your own monitoring of how the tool performs on your population.
Who should own the AI governance framework?
A named accountable executive, with control ownership pushed to the people who run the systems. Frameworks owned entirely by a central committee fail the same way: the committee can approve but cannot operate, so the controls live in minutes rather than in workflows. Central function sets direction and collects evidence; system owners run the controls.
Ready to Start Practising?
The framework above is the operating shape of the ITIL AI Governance certification, which covers the AI Capability Model, the Governance Improvement Model and the control and evidence design that audits test. If you are weighing it against the other credentials in this space, our AI governance certification comparison puts the options side by side, and the ISACA AAIA deep dive covers the audit-side route.
Reading a model tells you the stages. Answering exam questions under time pressure tells you whether you can apply them, which is the same gap that separates a policy from a control.
Create a free account and start practising today.