Seeing the World with a Prediction Mindset

I am fascinated by the topic of prediction. As machine learning experts, we are skilled at creating prediction tasks and “solving” them by training models such as random forests and neural networks. We have a unique perspective on prediction, a very empirical and scientific one. But prediction is much bigger than just machine learning. Predictions are everywhere: your uncle warning you that the financial market is about to crash, business plans, your worries about the future, astrology, playing poker games, weather forecasts, scrum estimates, and TV pundits.

Many of the prediction principles we know from machine learning apply to all these predictions. For example, we know that ensembles tend to work better, we know that errors can be due to bias or variance, and we know how to evaluate predictions. But if we only look at predictions from this calculated, empirical point of view, we miss the human and social side. Predictions are an integral part of how our society works, sometimes regardless of how accurate they are.

And if we take a really critical look at predictions, we can see how unpredictable the future really is. There are Clever Hans predictors, distribution shifts, black swans, and butterfly effects. Predictions about geopolitics and politics are particularly difficult. We don't have much data, only one world to look at. And the less data we have, the more assumptions we have to make, the more predictions become narratives. For example, one way to make geopolitical or social predictions is to assume that there are 'cycles' in which change happens. On the other hand, we have success stories like weather forecasting.

Interesting things also happen when predictors get good at predicting. I'm still amazed at how next-token prediction can produce such interesting models (GPT) that do much more than just predict the next word. Or there's Rainbolt, who excels at GeoGuessr: Get a Google Streetview image, guess the location. By playing this prediction game extensively, he can work out people's locations from a single photo.

In a way, we might be better off making more things into prediction problems. In Germany, we have the “Sonntagsfrage”, which asks people who they would vote for if there were an election on Sunday. It has been shown to have a bias that underestimates extremist parties. Many complicated solutions were suggested that would further anonymize this survey. But what stood out for me was the suggestion to turn it directly into a prediction problem.

Prediction Mindset

I have just returned from a 7-day yoga retreat. Now completely refreshed, I'm working on a new book with the working title 'The Prediction Mindset'. One of my life goals is to publish a book that my parents can read. They're not machine learning experts, so it's non-fiction rather than technical.

The Prediction Mindset is a book about all these aspects of prediction and when to trust predictions. “The Prediction Mindset” takes a much broader view of predictions, including human biases like the Barnum effect and confirmation bias, and understanding predictions as narratives.

This book will be a mix of philosophy, pragmatic advice for the streets, and unusual stories. Still trying to decide whether to make it ultra-short, a bit like Modeling Mindsets, or much longer. Tempted to make it short as the long segment on forecasting is already crowded (The Signal and the Noise, Taleb's Incerto, Superforecasting). I'm still trying to work out the scope and narrative of the book. Any thoughts you have on this would be very welcome.

If you are interested in such a book, you can sign up for updates here: https://christophmolnar.com/books/prediction-mindset/