Explore Your Modeling Mindset With A Quiz

A fun way to shed light on your modeling assumptions

If you follow me on Twitter or LinkedIn, you might have seen these spider charts:

These are from fellow modelers who took my fun quiz about modeling mindsets. If you haven’t tried out the quiz, you can take the quiz here.

I’ve reworked the quiz a little and it looks like this now. By the way, this is my profile

What the quiz tells you about yourself

While the quiz is not to be taken too seriously, it still touches open a topic in the world of modeling data that we don’t talk about enough: the often hidden assumptions and worldviews when modeling data.

If you fit, for example, a linear model, it wildly differs from what you are “allowed” to do with it. The usability of the model depends on the mindset and the assumptions from which the model originated.

• Did you select variables based on causal considerations? You might make causal claims.

• Did you use cross-validation for the model performance and selection? You can make statements about expected predictive performance.

• Did you put in frequentist assumptions? You may interpret the confidence intervals of the coefficients in your linear model.

• Did you define prior distributions of the coefficients? You may interpret the posterior distribution of the parameters as your degree of belief.

• Did you use the linear regression model to map state and action to a state-action value function? You may use the model to guide an agent through a (simple) environment.

The quiz is an entertaining way to make you think about what your assumptions are when modeling data. You might already be aware of that, but I’ve found it valuable to revisit those assumptions. Because sometimes the mindset with which we approach a new problem might limit us.

I, for example, have a strong background in supervised machine learning and frequentist statistics. That means I like it when problems can be turned into prediction problems or be framed in terms of decisions (via hypothesis tests and confidence intervals). But sometimes a problem might be interactive and profit from reinforcement learning, or it might require a causal inference mindset. I’ve stumbled often because of limitations in my mindset. This ultimately led me to write the Modeling Mindsets book, which summarizes mindsets from Bayesian inference to unsupervised machine learning.

How I built the quiz

The quiz is a simple static website: 1x HTML file, 1x CSS file, and 1x Javascript file.

The only problem: I’m bad at HTML, CSS, and Javascript.

I know some scraps, understand functions, and can read the stuff, but this quiz would have taken me days without help.

Such simple, static websites are great for ChatGPT. I used GPT-4 to build the quiz. And it worked surprisingly well. The first prompt got me almost to 80% of the project if you exclude the creation of the questions which also took me some time. You can find the prompt here.

It worked like a charm, but the last 20% of a project can be very annoying. The remainder was a mixture of me fiddling around with the html/js/css and asking GPT-4 for changes. For example, I prompted GPT-4 to optimize the website for mobile use and to add an “advertisement” slot for my book.

My summary: For such small projects, GPT-4 works exceptionally well. However, knowing a bit of Javascript/HTML/CSS helped, because to finish the last 20% I had to understand the code.

If you haven’t taken the quiz, it’s still online!

And if you haven’t read Modeling Mindsets yet, it’s also still available 😉