The most expensive mistake in AI is starting the wrong project. The ten questions below are a simple but effective checklist for assessing an idea before turning it into an investment. If you can't answer most of them clearly, the project isn't ready to start.
Problem and value
1. Which business problem are we solving?
You should be able to state the problem in one sentence without using a technology word. "Routing customer requests to the right team takes two days on average" is a good problem statement; "let's build a GenAI assistant" isn't.
2. What is it worth if we solve it?
Roughly estimate today's cost of the problem: time spent, cost of errors, lost revenue. If the value is small, even the best technology won't justify the investment.
3. How will we measure success?
Decide up front what you'll look at to say "we succeeded". The criterion should be a business metric: time, cost, error rate, customer satisfaction.
4. Do we really need AI?
Some problems can be solved with simple rules, process improvements or better use of existing software. AI makes a difference where rules fall short, and where there is uncertainty and large volumes of data.
Data and legal basis
5. Do we have the data we need?
What data does the solution require, where does it live, how far back does it go and what is its quality? If labelled data is needed, who will produce the labels?
6. Do we have the right to use the data?
If personal data is involved, what is the legal basis under data protection law? Will data leave the country? What do sector rules require (e.g. data localisation in banking)? The answers also determine whether the solution runs in the cloud or on-premise.
Users and process
7. Who will use the solution, and how?
On which screen, at which step of their workflow, will users see the solution? If users aren't involved, even a technically successful solution may go unused.
8. What happens when the AI is wrong?
Every AI system makes mistakes sometimes. What does an error cost, who will notice it and how will it be corrected? In decisions that affect people, where does human approval come in?
Implementation and scale
9. Which systems will it integrate with?
Which systems will the solution read data from and write results to? Integration is often the longest and riskiest part of a project.
10. Who will run it in production, and at what cost?
After the pilot, who will monitor, update and support the model? What will infrastructure and usage cost in production, especially per-use costs for LLM-based solutions?
Checklist
| # | Question | Clear answer? |
|---|---|---|
| 1 | Which business problem are we solving? | ☐ |
| 2 | What is it worth if solved? | ☐ |
| 3 | How will we measure success? | ☐ |
| 4 | Do we really need AI? | ☐ |
| 5 | Do we have the data? | ☐ |
| 6 | Do we have the right to use it? | ☐ |
| 7 | Who will use it, and how? | ☐ |
| 8 | What happens when it's wrong? | ☐ |
| 9 | Which systems will it integrate with? | ☐ |
| 10 | Who runs it in production, at what cost? | ☐ |
Next step: a rapid prototype
If the answers look good, the next step shouldn't be a big investment but a short proof of concept (PoC). A prototype shows, at low cost, whether the idea really works with your own data.
Through our Use Cases & Feasibility service we answer these questions with you and build a working prototype for the most promising use case. If you need to prioritise between several ideas, the AI Strategy & Roadmap Workshop is a good place to start.
Sources