Love this, I am a big fan of permutation importance.
For the first scenario, since the model was overfit, is SHAP more useful with a validation set? This way at least you would see the clear failure of the model for certain examples. Is it worthwhile looking at shap for correct versus incorrect predictions?
(Obviously this doesn’t hold if there is some sort of distribution shift in production, etc.)
Love this, I am a big fan of permutation importance.
For the first scenario, since the model was overfit, is SHAP more useful with a validation set? This way at least you would see the clear failure of the model for certain examples. Is it worthwhile looking at shap for correct versus incorrect predictions?
(Obviously this doesn’t hold if there is some sort of distribution shift in production, etc.)
Haha 🙈 I totally get your rant. The two quotes speak for themselves...