Day041 — Why should I trust you?

Two minutes paper is a channel that I will recommend to others. I appreciate the channel owner spending time to summary academic paper to a few minutes so that I could quickly browser about. This week it presents an interesting topic that I would like to share with you.

“Why Should I Trust You?” Explaining the Predictions of Any Classifier. It is a popular paper just by looking at the number of times it got cited. Over 300. WOW.

In the field of machine learning, people often focus on the held-out accuracy. A common approach to develop a machine learning model is to have the data set split into training, testing and validating. A high accuracy means the model is working and the model is ready to deploy. However, the accuracy should not be the only factor you look at. The model could be wrong even if its predication is correct. In a picture containing a wolf with a snow background, the model predicts the image contains a wolf simply because it thinks the white snow is the fur of a wolf. Without knowing how the model comes to its conclusion, we will never know it is the model is truly correct. It might just get lucky. We would never truly understand why the model thinks it is wolf unless we know for sure what portion of the image contributes the most when it decides it is a wolf.

The paper introduces LIME, Local Interpretable Model-agnostic Explanations. It is a novel technique to explain the predication of any machine learning classifier in an interpretable and faithful manner. With the explanation, it clarifies the rationale behind the decision making of a model. Then, people could see through the black box of machine learning model and really trust the predication it gives out. It also helps to gain insight of how the model perform or pinpoint the weakness if it presents.

The author describes his point of view on the Desired Characteristics for Explainers, which LIME meets all of them:

• Interpretable, explanation should be easy enough to understand depending on the target audience
• Local Fidelity, at least locally faithful in the vicinity of the instance being predicted
• Model Agnostic, should explain any model
• Global Perspective, select a few explanations to be representative of the model

In the paper, it demonstrates the flexibility of the framework by explaining models of text and image classification.

Explaining an image classification prediction made by Google’s Inception neural network. The top 3 classes predicted are “Electric Guitar” (p = 0.32), “Acoustic guitar” (p = 0.24) and “Labrador” (p = 0.21)

Here is the open source project posted on GitHub:

marcotcr/lime

The author also writes an easy-to-read blog post on O’Reilly. It helps a lot when it comes to understanding the paper.

Introduction to Local Interpretable Model-Agnostic Explanations (LIME)

Additional material:

powerpoint for above video:

kdd_lime_v2.pptx