Recent Technology Cost Forecasts Underestimate Pace of Technological Change
Governance. “Such a comparison is essential to ensure researchers and analysts have more empirically-grounded evidence in integrated assessment models, cost benefit analyses and broader policy design efforts.” She suggests that undertaking this type of comparison to assess and better understand different forecasting methods should become more common among modellers and forecasting practitioners, as more data is available. “Our analysis is focused on a particular period of time and on correlated energy technologies, so although our results point to current methods underestimating technological progress in this space, more research is needed”.
A number of key results emerge from this analysis.
As Dr. Way of the University of Oxford explains, “the comparison of expert- and model-based forecasts with observed 2019 costs over a short time frame (a maximum of 10 years) shows that model-based approaches outperformed expert elicitations. More specifically, the 5th-95th percentile range of the four model-based approaches were much more likely to contain the observed value than that of EE forecasts. Among the model-based methods, some captured 2019 observed costs more often than others”.
“In addition”, notes Dr. Meng from University College London “the 2019 medians of model-based forecasts were closer to the average observed 2019 cost for five out of the six technologies. However, this comparison was possible only for a small number of technologies; furthermore, some of the EE forecasts included the observed value”. For these reasons, the authors argue, this should not be taken as evidence that model-based approaches perform better than expert-based methods for all or most cases.
Prof. Verdolini, from UniBrescia/EIEE points to the fact that both expert-based methods and model-based methods underestimated technological progress in most of the energy technologies analysed in this paper. “That is, in five out of six technologies analyzed, the methods produced 2019 cost forecast medians that were higher than the observed 2019 costs. This indicates that the rate of progress in cost reduction has been higher than what both historical data and expert opinions predicted. But the extent to which this faster pace of progress compared to forecasts will continue (or not) in the future remains to be seen”.
The urgency of developing policies for deep decarbonisation, as outlined in the IPCC 1.5 °C report, makes this systematic analysis timely and necessary. Taken together, results point to various worthwhile avenues for future research. Concerning expert elicitations, this paper calls attention to the need to continue methodological improvements to reduce overconfidence. For model-based methods, this work highlights the challenge of finding (and collecting) data for many key energy technologies. It also calls for increased efforts in data collection and publication by international organizations and other entities. The underestimation of technological progress also points to the value of further method development to reflect structural changes and technology correlations. Lastly, given the large uncertainty ranges and major policy decisions associated with the energy transition and with addressing climate change, additional research comparing the performance of different probabilistic forecasting approaches with observed values across a wider range of technologies should be carried out as more data becomes available and more time passes.
The article is complemented with a database containing a large number of data points on the costs of 32 energy technologies relevant to support the energy transition. These data points include 25 sets of data from expert elicitations conducted between 2007 and 2016 covering a range of geographies and 25 sets of observed technology data including the evolution of cost and deployment over different periods of time. This data was made publicly available here.
