Three questions to help clarify which AI solution to build in your business
If you are a business leader wanting to get your organisation using AI, but your teams seem to be spinning their wheels considering many options. How do you help them pin down scope, and focus?
Here are three questions that might help focus the discussion and match the business need to what the new technology can and cannot deliver.
Question 1 — If this system works, what will be the impact for my business?
This is an obvious first question to help focus on processes that have some significant impact if they can be automated. But this is particularly important where there is so much hype and possibility in the AI world. It is easy for the team to become distracted by shiny things and new ideas — I regularly have this problem myself.
Grounding the discussion in a handful of valuable use cases helps narrow the discussions. The sizing can be very much back of the envelope of course. It’s mainly to help weed out less impactful options.
Question 2 — How accurate will the system need to be for us to actually use it in production?
One of the big paradigm changes in using AI systems compared to traditional IT systems is that AI systems are inherently ‘fuzzy’. This is quite different to the traditional IT paradigm that we are used to in projects where the systems that are built are precise, reliable and testable.
In practical terms this means that an AI system cannot give you 100% guarantees as to the accuracy of what it does. This is just the nature of this new paradigm. ChatGPT ‘hallucinations’ are a feature of this fuzziness that many people have experienced. The errors can be weeded out to an extent, but full guarantees are either impossible or very expensive to achieve. In a way, this new AI systems paradigm is much more akin to human resources where again we cannot give 100% guarantees that our human staff get things right all the time. But we learn to manage the risk.
Obtaining accuracy is where the time-sink will be in many AI systems projects — getting your accuracy up from 90% to 99% to 99.9% or whereever you need it to be. Even working out what accuracy metrics are appropriate can be a challenge and measuring them can be tricky.
So to avoid this ‘fuzziness’ becoming a problem in the implementation phase of a project, it is important to have a common understanding of how critical accuracy is for your use case. If it is a use case where there will be a human in the loop preventing errors, then you might be fine with 90–99% and development will be swift. If it is a use case in a regulated industry with a customer harm if things are wrong, then this accuracy bar may be much much higher and getting to that level of accuracy and proving it to yourself and your regulators may take quite some time.
The result of all of this is that when you have a choice between use cases, perhaps it is better to focus first on use cases with a lower accuracy bar?
Question 3— Describe the architecture of the prototype, which building blocks we will use, which of these exist and which we will build?
Getting a clear solution architecture drawn up, is a good way to break down the project into building blocks. Focusing on solution architecture early is already a pretty common process in many organisational IT projects.
However doing this work to pin down the building blocks is particularly important in the AI system space, because things are moving so quickly in both open-source and vended tools. There is a tempation to reconsider and constantly swap in the latest tool as each month something new comes out. This may seem ‘agile’ but can be quite confusing and can delay a team.
Also, content and data also take on a bigger role in AI paradigm projects. They are a key part of solution architecture that should be considered. For example with large language model AI solutions, a significant block of work is in curating and refining content for the AI to use. In other AI projects, putting together and refining representative data is a critical piece of work.
If the team doesn’t already have a pinned down picture of all these pieces of the jigsaw, then usually they should probably do this first.
Clearly these three questions are not the only way to think about focusing the team on an AI project — circumstances matter a lot — but can help clarify the way forward.
It is better to do something than to do nothing while waiting to do everything.
