Full prediction uncertainty quantification: a plea from science and decision making

Abstract

It is often tacitly assumed that the usefulness of a model depends primarily on the accuracy of its predictions. We disagree with this viewpoint and argue that in both applied decision making and theory development, understanding prediction uncertainty is often equally important. Unfortunately, many researchers still see the quantification and description of uncertainties as a nuisance that is unnecessary at best and counterproductive to the success of a model or theory at worst. Here, we argue that many objections against full prediction uncertainty quantification (FPUQ) are incoherent, or are reflecting a non-probabilistic tradition of process-oriented modelling communities. However, there are some real, at times substantial, problems when attempting a FPUQ. Those include quantifying structural model uncertainty, weighting of non-independent model predictions, estimating the fat-tailed prediction distributions and the challenge to obtain truly independent data. We discuss these challenges and argue that nevertheless these issues are outweighed by the reward of FPUQ, in particular the identification of real gaps in knowledge, and the establishment of credibility in decision making.