I’ve got an early Christmas present for you 🎁

Starting next week, this newsletter will temporarily turn into an intuitive introductory course for conformal prediction.

Here’s the outline:

I’ve covered conformal prediction before. The more I learn about this uncertainty quantification method, the more I’m convinced it should be standard for predictive models.

Especially because conformal prediction works with any predictive model and wraps the prediction function like a gift (and therefore doesn’t require changing the underlying model).

Conformal Prediction

Conformal prediction is a method that turns point predictions into prediction sets. These prediction sets are similar to confidence intervals because they come with a probability guarantee of covering the true outcome. Except that conformal prediction requires no distribution assumption and works for any model.

Examples

class scores → set of classes
quantile regression → conformalized quantile regression
uncalibrated class probability → interval of calibrated probabilities
regression (point prediction) → prediction interval

Conformal prediction is a growing research topic that is slowly making its way into applications.

But how do you get started with conformal prediction?

Just wait until next week.

This newsletter will turn into a 5-week e-mail course on conformal prediction.

But first, I have a confession to make: I’m new to conformal prediction. I did a deep dive last week, but I’m still a beginner and I’ll be learning as much as you will in the coming weeks.

But you know how it is: experts tend to forget the obstacles in their learning paths and everything seems to be trivial — in hindsight.

I’ll be handling this course the same way as with my Interpretable Machine Learning book: learning by writing. When I started the Interpretable ML book, I was by no means an expert on interpretability. But that’s the secret ingredient that made the book down-to-earth and focused on intuitive explanations.

Course On Conformal Prediction

Good news: the e-mail course is free and it will be delivered conveniently to your inbox in the coming weeks. If you are not signed up yet, you can sign up right here:

Schedule

Week 1: Getting Started With Conformal Prediction For Classification
Week 2: Intuition Behind Conformal Prediction
Week 3: Conformal Prediction For Regression
Week 4: Overview of Conformal Predictors
Week 5: Advanced Topics

Don’t worry, there will be no homework and no tests. 😉

If you can’t wait to get started, have a look at these 2 great resources in the meantime:

Valeriy’s Awesome Conformal Prediction list is a great collection of resources on conformal prediction: papers, software, books, and everything else.
The academic introduction paper “A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification” is the best introduction I know of so far.

Both resources help(ed) me a lot to create this course and learn about conformal prediction myself.

Looking forward to learning about conformal together with you 🤓

Looking For A Gift Idea?

For this week, and this week only, I reduced the price of the Modeling Mindsets paperback by 25%.

If you haven’t bought the book yet, now would be the best time.

Modeling Mindsets is a short yet insightful book that also makes a great present. Why not give Modeling Mindsets away to a fellow data scientist?

Get Paperback (-25%)

The offer is only available until Friday, December 17th. Order it now and get it before the holidays.

Mindful Modeler

E-Mail Course On Conformal Prediction