Approval is a moment. Operation is a period.
A typical AI governance process concentrates its effort at one point: the review before launch. Risk assessments are completed, test results are attached, someone signs. Then the system goes live and the governance record stops changing, while the system itself keeps changing. Models are updated by the provider, prompts are tuned, retrieval sources grow, the user base shifts and people find uses nobody planned for.
The NIST AI RMF is explicit that managing AI risk continues for as long as a system is in use. ISO/IEC 42005, the impact assessment standard published in 2025, also treats impact assessment as something repeated at different lifecycle stages rather than done once.
What actually needs watching
Monitoring an AI system is not the same as monitoring uptime. The signals that matter are usually about behaviour and use:
- Output quality on a sample reviewed by people who know the domain, on a fixed schedule.
- Differences in outcomes between groups of users, where the system influences decisions about people.
- Override and escalation rates. A sudden drop in overrides can mean reviewers have stopped checking, not that the system improved.
- Complaints, corrections and support tickets that mention the AI feature.
- Upstream changes: new model versions, changed system prompts, new data sources.
Triggers, not just dashboards
A dashboard nobody acts on is decoration. For each signal, decide in advance what level triggers a review, who is told, and what they are allowed to do: pause the feature, roll back a model version, restrict it to a smaller group, or accept the change and record why. This is the part most teams skip, and it is where governance either works or does not.
Plan the incident before you need it
When an AI feature produces something harmful in production, the first hour is usually spent working out who owns it and whether anyone can switch it off. A short incident runbook, agreed before launch, avoids that: how to disable the feature, who decides, how affected users are identified and contacted, what is preserved for investigation, and when a regulator or customer must be told.
If your team has launch approval nailed but no clear plan for running an AI system safely afterwards, we cover that in the AI Opportunity and Governance Sprint.