How to implement permutation feature importance

Of all the methods for interpreting machine learning models, permutation feature importance is the easiest to implement. In the basic version, it’s just a few lines of code:

import numpy as np

def get_pfi(model, X, y, feature_idx):
    baseline_error = np.mean(np.abs(y - model.predict(X)))
    X_permuted = X.copy()
    X_permuted[:, feature_idx] = np.random.permutation(X[:,feature_idx])
    permutation_error = np.mean(np.abs(y - model.predict(X_permuted)))
    return permutation_error / baseline_error

For a more in-depth explanation of the code, you can watch my video below. With the video, I'm testing new formats, so there might be more videos in the future. Any constructive feedback is welcome. My wife has already told me that I should smile more when making videos ... 😅

Because of this algorithmic simplicity, permutation feature importance (PFI) is the best entry point into interpretable machine learning (at least for model-agnostic methods). Even if you already have experience with interpretability, PFI is a great way to teach non-experts how interpretability can work:

• PFI is a great example of how model-agnostic interpretability works in general.

• More specifically, PFI demonstrates how model-agnostic methods work without looking “inside” the model, but by manipulating the data.

• The code example allows you to pinpoint where correlated features may cause problems: When the feature is shuffled, any correlations are ignored, which can cause problems for the interpretation.

• PFI is estimated and subject to randomness. This becomes clear in the code because we shuffle the feature, which relies on a random number generator. Many other interpretability methods are also estimates and therefore have an estimation uncertainty.

The implementation shown here is the simplest possible version of permutation feature importance. For a more rigorous implementation, you might want to add a few things (ignoring here software development best practices such as type-checking and documentation):

• You would want to implement the function to loop through all features.

• Results with only one permutation can be unstable, especially with small datasets, so an option to repeat the permutations and average the results is helpful.

• Instead of hard-coding the error metric, you could offer multiple options or even allow custom error metrics.

• You might also want to visualize the results, e.g. with a horizontal bar chart.

Still, it’s useful to know how to implement a simple version of PFI for education and communication purposes.

Just 6 more days

Next week, on Monday, November 4th, Timo and I will publish our book Supervised ML for Science. It has been over 2 years in the making, and with all the Nobel prizes going to machine learning, I think it’s a timely book.

Tomorrow I should receive the author copies. I am especially excited to hold the hardcover in my hands. It's the first book I've done a hardcover version for.

I’m looking forward to sharing the book with you.