Powerful machine learning models are used to help humans solve complex problems such as detecting diseases in medical images or detecting road obstacles in autonomous vehicles. But machine learning models can make mistakes, so when the stakes are high, it’s critical that people know when to trust the model’s predictions.
Uncertainty quantification is a tool that improves model reliability; along with the forecast, the model provides a score that expresses the level of confidence that the forecast is correct. While uncertainty quantification can be useful, existing methods typically require retraining the entire model to give it this ability. Training involves showing the model millions of examples so it can learn the task. Training then requires millions of new data inputs, which can be expensive and difficult to obtain, and use enormous amounts of computing resources.
Researchers at MIT and the MIT-IBM Watson AI Lab have now developed a technique that allows the model to perform more efficient uncertainty quantification while using far fewer computing resources than other methods, and without additional data. Their technique, which does not require the user to train or modify the model, is flexible enough for many applications.
The technique involves creating a simpler companion model that helps the original machine learning model estimate uncertainty. This small model is designed to identify different types of uncertainty, which can help researchers identify the root cause of inaccurate predictions.
“Quantifying uncertainty is essential for both developers and users of machine learning models. Developers can use uncertainty metrics to help develop more robust models, while for users it can add another layer of confidence and reliability when deploying models in the real world. Our work leads to a more flexible and practical solution for uncertainty quantification,” said Maohao Shen, a graduate student in electrical engineering and computer science and lead author of a paper on the technique.
Shen co-authored the paper with Yuheng Bu, a former postdoctoral fellow at the Research Laboratory of Electronics (RLE) who is now an assistant professor at the University of Florida. Prasanna Sattigeri, Soumya Ghosh and Subhro Das, MIT-IBM Watson AI Lab research staff members; and senior author Gregory Wornell, the Sumitomo Professor of Engineering who directs the Signals, Information and Algorithms Laboratory RLE and is a member of the MIT-IBM Watson AI Lab. The research will be presented at the AAAI Conference on Artificial Intelligence.
Quantifying uncertainty
When quantifying uncertainty, the machine learning model creates a numerical score with each result to reflect its confidence in the accuracy of that prediction. Quantifying uncertainty by building a new model from scratch or training an existing model usually requires large amounts of data and expensive computations that are often impractical. Furthermore, existing methods sometimes have the unintended consequence of degrading the quality of model predictions.
Researchers at MIT and the MIT-IBM Watson AI Lab have thus zeroed in on the following problem: With a pre-built model, how can they enable it to perform effective uncertainty quantification?
They solve this by creating a smaller and simpler model, known as a metamodel, that attaches to a larger, pre-trained model and uses features that the larger model has already learned to help it make quantitative estimates of uncertainty.
“The metamodel can be applied to any pre-made model. It’s better to have access to the internals of the model because we can get a lot more information about the base model, but it will also work if you just have the final output. It can still predict the confidence score,” Satigeri says.
They design the metamodel to produce a quantitative output of uncertainty using a technique that incorporates both types of uncertainty: data uncertainty and model uncertainty. Data uncertainty is caused by corrupt data or inaccurate labels and can only be reduced by correcting the database or collecting new data. With model uncertainty, the model is unsure how to explain the newly observed data and may make incorrect predictions, most likely because it has not seen enough similar training examples. This is a particularly difficult but common problem when models are fitted. In the real world, they often encounter data that is different from the training data.
“Has the reliability of your decisions changed when you use the model in a new environment? You want to somehow be sure if it’s working in this new mode or if you need to collect training data for this specific new setting,” Warnell says.
Quantification Validation
When a model produces a quantitative estimate of uncertainty, the user still needs some assurance that the unit itself is accurate. Researchers often confirm accuracy by creating a smaller dataset drawn from the initial training data and then testing the model on the stored data. However, this technique doesn’t work well in quantifying uncertainty because a model can achieve good predictive accuracy while being overconfident, Shen says.
They developed a new validation technique by adding noise to the data in the validation set; these noisy data are more like out-of-distribution data that can introduce model uncertainty. Researchers use this noisy dataset to quantify uncertainty.
They tested their approach by seeing how well the meta-model could capture different types of uncertainty for different downstream tasks, including out-of-distribution detection and misclassification detection. Their method not only outperformed all baselines in every downstream task, but also required less training time to achieve those results.
This technique could help researchers enable more machine learning models to effectively quantify uncertainty, ultimately helping users make better decisions about when to trust predictions.
Moving forward, the researchers want to adapt their technique for newer classes of models, such as large language models, which have a different structure than a traditional neural network, Shen says.
The work was partially funded by the MIT-IBM Watson AI Lab and the US National Science Foundation.