Interpreting Machine Learning Models SHAP – Christoph Molnar

There are two “lines”: One line represents fans who are >7 away from the stage (x2). Here we see the large jump, which is expected since fans taller than 160cm have no chance of getting on someone’s shoulders. The other line represents values of x2 below 7. It has a smaller jump, but why is there a jump at all? Fans in this “cluster” don’t get to sit on someone’s shoulders when they are smaller than 160cm.

The reason why the interaction also “bleeds” into the cluster where we wouldn’t expect it has to do with how SHAP values function. 10.3 SHAP values have a “global” component Let’s consider two fans: 1. Mia, who is 159cm tall and 2 units away from the stage. 2. Tom, who is 161cm tall and standing right next to Mia, also 2 units away from the stage. Here are the model’s predictions for how much they will enjoy the concert: Mia: 5.88 # Creating data for Mia and Tom Xnew = pd.DataFrame({‘x1’: [159, 161], ‘x2’: [2,2]}) print(“”” Mia: {mia} Tom: {tom} Expected: {exp} “””.format( mia=round(rf_model.predict(Xnew)[0], 2), tom=round(rf_model.predict(Xnew)[1], 2), exp=round(explainer.expected_value[0], 2) )) Tom: 6.07 Expected: 4.86 They have a rather similar predicted joy for the concert, with Mia having a slightly worse prediction – makes sense given she is slightly smaller and neither of them qualify for shoulders.

Let’s examine their SHAP values. Mia [-0.15944093 1.18150926] Tom [-1.37794848 2.5913748 ] The SHAP values for Mia and Tom differ significantly. Mia’s slightly negative value for height is understandable, given her relative shortness compared to the majority of simulated heights. Mia’s positive SHAP value for distance makes sense as she is quite near the front. Tom’s SHAP values follow similar trends, which is expected since they share the same distance and similar heights.

However, Tom’s SHAP values are more pronounced. But shouldn’t Mia have a smaller SHAP value for height than Tom? Neither of them benefits from the shoulder bonus, so Mia being smaller than Tom should mean that her SHAP value for “height” should be smaller than Tom’s, right?

But surprisingly, Mia’s SHAP value is influenced by the interaction term, despite her not being directly affected by the shoulder bonus!

In my first book, “Interpretable Machine Learning,” I overlooked the inclusion of SHAP. I conducted a Twitter survey to determine the most frequently used methods for interpreting machine learning models. Options included LIME, permutation feature importance, partial dependence plots, and “Other.” SHAP was not an option.

To my surprise, the majority of respondents selected “Other,” with many comments highlighting the absence of SHAP. Although I was aware of SHAP at that time, I underestimated its popularity in machine learning explainability. This popularity was a double-edged sword. My PhD research on interpretable machine learning was centered around partial dependence plots and permutation feature importance. On multiple occasions, when submitting a paper to a conference, we were advised to focus on SHAP or LIME instead.

This advice was misguided because we should make progress for all interpretation methods, not just SHAP, but it underscores the popularity of SHAP. SHAP has been subjected to its fair share of criticism: it’s costly to compute, challenging to interpret, and overhyped. I agree with some of these criticisms. In the realm of interpretable machine learning, there’s no perfect method; we must learn to work within constraints, which this book also addresses. However, SHAP excels in many areas: it can work with any model, it’s modular in building global interpretations, and it has a vast ecosystem of SHAP adaptations.

As you can see, my relationship with SHAP is a mix of admiration and frustration – perhaps a balanced standpoint for writing about SHAP. I don’t intend to overhype it, but I believe it’s a beneficial tool worth understanding. OceanofPDF.com 2 Introduction Machine learning models are powerful tools, but their lack of interpretability is a challenge.

It’s often unclear why a certain prediction was made, what the most important features were, and how the features influenced the predictions in general. Many people argue that as long as a machine learning model performs well, interpretability is unnecessary. However, there are many practical reasons why you need interpretability, ranging from debugging to building trust in your model. I think of “interpretability” in the context of machine learning as a keyword.

Under this keyword, you find a colorful variety of approaches that attempt to extract information about how the model makes predictions. 2.1 Interpreting to debug Interpretability is valuable for model debugging, as illustrated by a study predicting pneumonia (Caruana et al. 2015). The authors trained a rule- based model, which learned that if a patient has asthma, they have a lower risk of pneumonia.

Seriously?

This is a short excerpt from the opening of “” by Unknown, quoted for review and introduction purposes. All rights belong to the copyright holders.

Book Information

• Unique ID: c5638a823a9067eb
• File Extension: .pdf
• File Size: 8,162,973 bytes (7.785 MB)
• Title: –
• Author: Unknown
• Pages: 221
• Language: English (en)

Reading & Word Statistics

• Estimated Reading Time: 189.59 minutes
• Total Words: 37,918
• Total Characters: 246,733
• Average Words per Page: 171.57
• Average Characters per Page: 1116.44

Most Frequent Words

shap (840), values (575), model (334), data (298), feature (271), features (224), value (184), prediction (155), explainer (140), shapley (131), models (113), linear (110), learning (104), plot (104), machine (100), marginal (91), test (89), use (88), method (84), train (80), however (78), let’s (78), also (77), background (77), alcohol (76), one (74), using (74), import (74), estimator (73), sampling (72), coalition (72), coalitions (72), image (71), example (71), function (69), two (64), chapter (64), predictions (62), contribution (62), correlation (61), plots (59), regression (59), text (58), instance (58), contributions (58), based (57), input (57), masker (57), tree (56), between (54), bob (54), random (53), plt (53), permutation (52), alice (52), interpretation (51), explanations (49), estimation (49), it’s (48), point (48), mask (47), first (46), average (46), methods (45), importance (45), number (45), interactions (45), like (44), explanation (43), different (43), print (42), predicted (41), interaction (41), charlie (41), used (40), dataset (40), additive (40), now (38), classification (38), possible (38), taxi (38), wine (38), theory (37), neural (37), game (37), set (37), time (36), compute (36), individual (36), explain (36), wines (36), clustering (36), interpretable (35), gradient (35), expected (35), instead (34), deep (34), size (34), index (34), output (34).

PDF Download