When most people have to predict micro and macro issues this is an important question. If you cannot predict Brexit why would you think you can predict next years sales, profits, waiting times in A&E or budget deficit? Yet the way so many pundits rubbish Brexit predictions and scenario modelling suggests they have an over-inflated opinion of their own decision-making effectiveness.
In "Superforecasting the Future of Europe" Warren Hatch explains how rigorous the forecasting process is and gives real-life examples.
Which brings us back to an essential fact- arguably the best forecasting organisation in the world is Good Judgement led by Philip Tetlock.
"In 2011, the Good Judgment Project was one of five research teams selected by the US intelligence community to compete in an unprecedented government-funded forecasting tournament.Four years, five hundred questions and over one million forecasts later, GJP emerged as the undisputed victor in the tournament, outperforming in accuracy even professional intelligence analysts with access to classified data." Good Judgement website
They attribute four critical success factors to the science of forecasting.
"
Good Judgment's research findings point to four main determinants of forecasting accuracy:
- talent-spotting to identify the best forecasters;
- training to reduce cognitive biases and implement the best forecasting practices;
- placing forecasters in teams to deliver the benefits of collaboration and cognitive diversity; and
- aggregating forecasts in a way that 'extremizes' the wisdom of the crowd towards a higher degree of confidence and gives more weight to forecasters who have a proven track record and update frequently."
Otherwise a chimpanzee throwing a dart at multiple choice questions will get better results. That's not fanciful as shown by Hans Roslin in the book Factfulness. Even Bill Gates was shocked at how wrong he was in an area which he works in- eliminating illness, disease and poverty.
Test yourself with real questions you should know the answers to at Factfulness and why chimpanzees make better decisions than humans.
Which again makes the point that we all must be more proficient at forecasting, analytics, data and not relying on pundits and even experts when they pontificate outside their narrow domains of knowledge and skill.
What he found was that pundits’ calibration, on average, was no better than random guessing – or, as Tetlock described it, “than a dart-throwing chimpanzee”. But some were significantly better. What predicted who was better was not whether they were conservative or liberal, or even, particularly, their expertise in the field, but how they approached the problem. People who assumed the world was simple and had simple solutions did badly; people who thought it was complex, who realised they could be wrong, and who learnt from mistakes did better. It also helped if you were good with numbers, and good at spotting patterns, as in IQ tests.
https://unherd.com/2018/11/even-superforecasters-dark-brexit/
unknownx500
