Embrace the clash between domain expertise and machine learning
How to deal with ML model being at odds with domain knowledge
Machine learning and domain expertise sometimes don’t align.
Some examples:
I trained a water supply forecasting model with the most important feature being the snowpack in the surrounding mountains. I also included data from stations outside the basin, meaning from snow stations where the snow would melt into a different river. This clashes with physics-based intuition. But including outside stations improved the predictions (my theory: it stabilizes the snowpack estimates).
Some data scientists I know don’t use machine learning algorithms because they aren’t interpretable to the domain experts.
From the paper by Caruana et. al 2015: They trained a model to predict the probability of death from pneumonia for patients coming to the emergency room. Asthma patients were predicted to have a lower risk. This is at odds with medical knowledge: Asthma patients have an increased risk, not a reduced one. Reason: Patients with asthma tend to get more aggressive treatments for pneumonia.
The root of the problem is that domain expertise and machine learning have different foundations:
Domain expertise is (often) process-focused; machine learning is prediction-focused.
Domain knowledge is designed for humans; machine learning is designed for machines.
Domain expertise is (often) theory-driven; machine learning is data-driven.
What to do, as a machine learning expert, when the machine learning model is at odds with domain knowledge?
Embrace it.
The gaps between the ML model and domain knowledge can be the most influential leverage point and whenever such gaps occur you should investigate. Your model might have picked up some bias, like in the asthma case, that renders your model unusable unless you intervene. Or, together with domain experts, you might generate a new understanding of the prediction task. You will learn more about the domain. Addressing such clashes can get domain experts on board.
The interaction between domain expertise and the ML model is a two-way street and you can close the gaps from both directions:
Make the machine learning model adhere to domain knowledge (see this book chapter). This can mean making the model more causal, interpretable, and robust. It can mean putting constraints on the model or differently engineered features. It can also mean designing a custom loss function to optimize for.
Communicate your modeling approach to domain experts. Methods and models from machine learning interpretability can help. But also skills like drawing a directed acyclic graph (DAG) to discuss causality. It's magical once you have brought domain experts on board: they will understand the model better, connect it with current knowledge, and maybe even draw new connections.
Let’s chat about machine learning in science
Do you apply machine learning in the context of science? If so, I would be interested in chatting with you. What you will get out of it: We can discuss solutions to concrete problems with your scientific application of machine learning. I have experience in interpretability, uncertainty quantification, ML, stats, etc. What I’m hoping to get out of the call: Insights about problems scientists face with machine learning and inspiration for writing, like for my book “Supervised Machine Learning for Science”.
Book a free call with me:
Looking forward to talking to you!
Great article! I am very much interested in the gaps mentioned here. A large segment of my research work is on data visualization for ML and often the goal is to identify (and hopefully close) those gaps.