I agree, the comment about counterfactual questions deserves elaboration.
> Don’t use SHAP for counterfactual questions
We could ask, in the diagram, if the value for this feature was increased to the average value on the baseline data, would it affect the prediction positively or negatively? This is a counterfactual question and seems to be exactly what SHAP should answer correctly. But I haven't read the book.
I think if the question is "what is the prediction if I change only the value of this feature but keep all other features the same?" then shapley values (not sure about the shap package specifically) will not answer the question.
I think shapley values, answer the question "what is the prediction if the value of that feature changes, averaged over many possible combinations of other features being 'on' or 'off'?"
Could you elaborate further on the definition of counterfactual questions?
I agree, the comment about counterfactual questions deserves elaboration.
> Don’t use SHAP for counterfactual questions
We could ask, in the diagram, if the value for this feature was increased to the average value on the baseline data, would it affect the prediction positively or negatively? This is a counterfactual question and seems to be exactly what SHAP should answer correctly. But I haven't read the book.
I think if the question is "what is the prediction if I change only the value of this feature but keep all other features the same?" then shapley values (not sure about the shap package specifically) will not answer the question.
I think shapley values, answer the question "what is the prediction if the value of that feature changes, averaged over many possible combinations of other features being 'on' or 'off'?"