Most organisations answer "Is our AI work paying off?" with licence counts or monthly logins. These numbers show the tool has been switched on, but not whether employees use it correctly, safely and in ways that produce business results.
This article offers a practical three-level model for measuring AI adoption.
Why measuring matters so much
In McKinsey's March 2025 survey, of the 12 practices tested for adopting and scaling generative AI, tracking well-defined KPIs for generative AI solutions had the most impact on the bottom line. According to the same survey, fewer than one in five respondents track these KPIs.
The cost of not measuring is simple: a programme that can't prove its value struggles in the next budget cycle.
Level 1: Usage
Question: Who uses the approved tools, how often and for what work?
| Indicator | How to collect it |
|---|---|
| Active user rate (weekly / monthly) | Admin dashboards and usage reports of the tools |
| Usage by unit | The same reports broken down by unit |
| Frequency of use | Weekly sessions per user |
| Types of work | Short survey or champion feedback |
| Unapproved tool use | Anonymous survey and IT observations |
Note: Large differences between units often relate to manager support or the presence of a champion. They show where intervention is needed.
Level 2: Proficiency
Question: Do employees know and follow safe-use rules?
| Indicator | How to collect it |
|---|---|
| Safe-use knowledge | Short knowledge test (the same test at the start and end of the programme) |
| Verification habit | Scenario questions: "AI gave you a number. What do you do?" |
| Confidence | Agreement with "I know how to use AI in my work" |
| Training completion | Training records |
| Data incidents or suspicious use reports | Information security records |
To get started, use our free AI Literacy Quiz or training needs analysis survey.
Level 3: Business results
Question: What changed in the workflows where AI is used?
Producing a single organisation-wide "AI return" figure is hard and often contested. A more reliable route is before-and-after measurement in selected pilot workflows:
| Example workflow | Measurable result |
|---|---|
| Replying to customer emails | Average response time, share of repeated questions |
| Monthly management report | Preparation time, number of correction rounds |
| Job ads and candidate pre-screening | Time to prepare an ad |
| Preparing proposals | Time to prepare, feedback on proposal quality |
| Searching technical documents | Time to find information |
Tips: - Measure before the pilot starts; estimating "how long it used to take" afterwards isn't reliable. - Measure quality as well as time saved; a process that gets faster but makes more mistakes isn't a gain. - Present results with concrete examples alongside the numbers.
Measurement schedule
| When | What |
|---|---|
| Start | Baseline on all three levels, "before" measurement in pilot workflows |
| Monthly | Usage indicators, champion feedback |
| Quarterly | Repeat proficiency test, "after" measurement in pilot workflows, management report |
| Yearly | Repeat maturity assessment, update the roadmap |
Using the four levels of the Kirkpatrick model (reaction, learning, behaviour, results) to measure training impact fits this framework too; see how to measure the impact of AI training for details.
What belongs in the management report?
A regular one-page report is enough:
- Usage: Active user rate and change by unit
- Proficiency: Change in test results and key risks
- Business results: Before-and-after comparison in pilot workflows
- Barriers: The most frequent issues from the field
- Next steps: Use cases to scale and decisions needed
Common mistakes
- Looking only at usage: A flattering picture that doesn't prove value.
- No baseline: You can't show what changed.
- Trying to measure everything: A few indicators tracked regularly are worth more.
- Creating a sense of individual surveillance: The aim is to improve the programme, not to monitor employees; say so openly and assess the measurement method from a data protection perspective.
Conclusion
Measuring AI adoption is more than counting licences. A measurement system that tracks usage, proficiency and business results together, starts with a baseline and is repeated regularly both proves the programme's value and shows where to focus.
Our Corporate AI Adoption Programme starts with a baseline assessment and ends with a results report; after the programme, our Ongoing Support module provides quarterly measurement and a management report.
Sources