The importance of uncertainty model estimation in Artificial Intelligence for business.
The social and economic value that Artificial Intelligence (AI) is bringing is undeniable. From web advertising to assisting the development of the Covid-19 vaccine, adoption is booming even in traditional industries like finance and banking, where AI is being used to underwrite loans, fight money laundering, and prevent fraudulent transactions. Other emerging applications where AI is increasingly used is in medical diagnostic devices, as well as in autonomous cars to develop navigation systems.
As AI continues to penetrate into areas of high-risk decision making such as life sciences and the financial sector, more care has to be taken in the design and implementation of AI models. The following are examples where measuring the uncertainty of our models becomes very important:
- Should I prescribe this treatment or not?
- Should the car autopilot accelerate or not?
First we need to define what “uncertainty” is in the context of AI-Machine learning. Uncertainty refers to a situation where there is no perfect information about the system but rather some degree of the unknown (like most business applications). In practical terms, measuring uncertainty is how much we trust our AI model outputs. Probabilistic Machine Learning is a field of AI that allows us to compute uncertainty.
How does it work in practice?
Imagine you are an online retailer planning your inventory for the next quarter and you have a tight budget. It is critical to have an accurate forecast of your expected sales in order to allocate the inventory efficiently. Traditional AI models would tell you something like: “the expected sales of product X for the next quarter is 126.” In contrast, a probabilistic AI-Machine learning model would tell you a range of probable sales instead: “you could sell between 115 and 135 units of product X”. In short, a non-probabilistic model will forecast a point value for the sales in the future, but the probabilistic AI will give you a probability distribution of the sales, being the median of the distribution equivalent to the non-probabilistic output. By having a probability distribution you can make better decisions, for example; the confidence intervals of the distribution can help you estimate what would be your potential maximum (or minimal) sales next quarter. This opens the door to simulation-based decision making using AI. You can proceed by adding more information to enrich the system like supplier lead time, price-discounts, etc. Furthermore, this could identify potential risks in your business and/or optimize your income.
Uncertainty comes from both the data and the AI model (aleatoric and epistemic). In practice, measuring uncertainty may improve the model accuracy with less data. Probabilistic machine learning relies on Bayes theorem. Building a model with the theorem allows to incorporate a prior belief of the probability distribution of our data. A prior belief is a way to incorporate domain expertise in our model in benefit of the learning process. The efficiency of the data is not directly related to the uncertainty measure, but a high uncertainty may be an indication of the quantity and quality of the data. Being data efficient is key for a business application, where data sometimes is missing or we have insufficient data from a particular client or market.
Why is Probabilistic Machine learning not more widely used?
The benefits of having uncertainty and good accuracy does not come for free. The machinery used by probabilistic machine learning is more computationally intensive, but recent advances in algorithms and hardware are opening the door to efficiently deploy models that solve business problems at scale. We routinely run models that would be unthinkable a few years ago, even in a laptop (of course it is faster and more convenient to rely on cloud computing instances that leverage multi CPU/GPU nodes).
Giving machines a measure of uncertainty, brings them closer to how we humans make decisions. When we hesitate to make a decision in scenarios where we don’t have enough information, the feeling of uncertainty prompts us to collect more data, or changes our decision making process. Uncertainty estimation is a must in AI applications, where the cost of taking the wrong decision is high. In the next few years, we are going to see more industrial applications of probabilistic machine learning that hopefully helps us to get closer to a true general AI.
