AI Governance: A Starting Framework for Your Organization
A practical starting point for governing AI systems before regulation forces the issue.
The short version
- Start with an inventory. Most organizations cannot list the AI systems already in use, and policy written without that list governs nothing.
- Risk-tier use cases, because the oversight a hiring model needs is not the oversight a meeting summarizer needs.
- Vendor-procured AI is the largest blind spot, since it enters through purchasing rather than engineering.
- The NIST AI Risk Management Framework organizes AI risk into Govern, Map, Measure, and Manage, mirroring the structure of NIST CSF.
What is AI governance?
AI governance is the set of policies, approval workflows, and monitoring practices that determine how an organization builds, buys, and operates AI systems. It answers four questions: what AI are we using, who approved it, what could go wrong, and who is accountable when it does. Organizations tend to reach for policy first, which is the wrong end of the problem.
Start with an inventory, not a policy
Most organizations can't answer a basic question: what AI systems are actually in use, and who owns them? Before writing governance policy, build an inventory of models and AI-enabled tools in production and in development, including ones procured through vendors rather than built in-house. The vendor-procured category is usually the larger one and the less visible one, because it arrives through purchasing rather than through engineering, and often through individual team budgets rather than central procurement.
How should I risk-tier AI use cases?
- High risk: decisions affecting individuals, such as hiring, credit, or healthcare, and autonomous action with limited oversight
- Medium risk: internal decision support with human review
- Low risk: productivity tools with no material decision authority
What does a starting governance structure include?
- An inventory and risk-tiering process for AI systems
- An approval workflow for new AI use cases, scaled to risk tier
- Data provenance and usage policy for training and inference data
- Human oversight requirements defined per tier, not as a blanket rule
- Ongoing monitoring for model drift and unexpected outcomes
- A defined route for people affected by an AI-influenced decision to contest it
How does this align with the NIST AI Risk Management Framework?
The NIST AI RMF organizes AI risk management into four functions, Govern, Map, Measure, and Manage, that mirror the structure of NIST CSF 2.0. Organizations already familiar with CSF have a natural head start applying the same thinking to AI risk. Govern sets strategy and accountability, Map establishes context and identifies risks, Measure assesses and tracks them, and Manage allocates response. The framework is voluntary and not certifiable.
What about AI in vendor products?
AI features increasingly arrive inside software you already bought, enabled by default in a release you did not review. Treat this as a vendor risk question as much as an AI question: add AI-specific items to your vendor assessment covering what data the feature processes, whether that data trains vendor models, where inference happens, and what contractual commitments exist about model changes. The practical control is a procurement gate, because once a feature is live across the organization, turning it off is a much harder conversation.
Need hands-on help with this?
See how our AI Governance & Risk Management engagement works.
Ready to put this into practice?
Reading the guide is the easy part. If you want practitioners who have run these implementations to scope the work with you, start with a free consultation.