Thanks for your post. It was a great one. Recently, we have done a couple of research projects to extend LIME-based approaches for LLMs. You can see the results here:
I really like the idea of LLMs as classifiers! Would kv caching help here at all with previously predicted tokens?
I would like to shamefully plug my own work as well, I have a paper in TMLR showing how to find an exactly equivalent linear system that reproduces a given next token prediction in models like Gemma 3 and Qwen 3. This gives you linear token contributions that exactly reproduce the predicted output embedding. It’s nice because there is no approximation error like in Shap.
Depending on the estimation method used for the Shapley values, a bit of cashing is helpful. For example, for the permutation method, each coalition appears at least twice, so there it's quite obvious that we want to only do one call to the LLM per coalition.
But also the Monte Carlo sampling method may produce the same call more than once.
Thanks for your post. It was a great one. Recently, we have done a couple of research projects to extend LIME-based approaches for LLMs. You can see the results here:
https://arxiv.org/pdf/2505.21657
We also expanded it for an image editing scenario
https://arxiv.org/pdf/2412.16277
https://www.kaggle.com/code/zeinabdehghani/explain-gemini-image-editing-with-smile
I really like the idea of LLMs as classifiers! Would kv caching help here at all with previously predicted tokens?
I would like to shamefully plug my own work as well, I have a paper in TMLR showing how to find an exactly equivalent linear system that reproduces a given next token prediction in models like Gemma 3 and Qwen 3. This gives you linear token contributions that exactly reproduce the predicted output embedding. It’s nice because there is no approximation error like in Shap.
https://arxiv.org/abs/2505.24293
Thank you for all of the books and posts over the years!
Thanks for the paper!
Depending on the estimation method used for the Shapley values, a bit of cashing is helpful. For example, for the permutation method, each coalition appears at least twice, so there it's quite obvious that we want to only do one call to the LLM per coalition.
But also the Monte Carlo sampling method may produce the same call more than once.