What problem does this paper solve?
This paper comes up with a new method called GAM(Global Attribution Mapping).GAM provides global explanations across sub-populations by taking pre-existing local explanations as input. The global explanations also have additional information of the number of samples that the explanation is applicable to and what those samples are. Think of it as a summarizer/aggregator of multiple local explanations!
Background and Context
There are multiple methods (LIME, DeepLift, etc) that allow one to get local explanations for a particular test point. However, if the dataset is huge, it becomes difficult to aggregate this and understand the whole landscape of the model's predictions. Also, completely global explanations are too broad and do not allow one to understand specific sub-populations where one set of explanation is different from others.
In such a scenario, a middle ground between local explanations for each point and global explanations for the entire model is found using a system where global explanations are given in terms of subsets of the population. For instance, such an explanation would partition the points into 5 sets and each of those sets will have an explanation applicable to all the points belonging to that set.
Methodology
GAM uses a combination of ranking and clustering to group points with similar explanations together. Once they are grouped, each group represents a sub-population and will have a corresponding set of explanations specific to itself.
The basic idea is that given the local explanations for the points in the dataset, we first represent the attributions as a ranking and then use the difference between rankings as a distance metric (instead of the usual Manhattan distance) in clustering to group similar attributions together. Similar to K-Means clustering, the user has to specify the number of clusters (K) beforehand. In our context, this K is the number of subset groups we will get the explanation for, ie, the subpopulations.
Representing Attributions as Rankings
The goal in this first step is to take the raw local attributions from methods like LIME and DeepLift and convert the attribution vector to a weighted ranked vector for each point. By doing this we consider both the value of attribution and the ordering. This will allow us later to compute our similarity metric.
Distance between Rankings
We first normalize the raw values for each row(each instance) to a percentange between 0 and 100. When we do this, we use absolute values and the sign does not matter.
Now there are two ways stated in the paper to find rank distances between two such vectors. The Weighted Kendall’s Tau method and Weighted Spearman’s Rho squared rank distance. Both are similar, and the authors primarily use the former so we will discuss it here.
Weighted Kendall’s Tau This method has the following distance calculation logic-
a. If two rankings containing all elements in the same order (even if weights differ)-> Zero distance b. If Two rankings have at least one element in a different order -> distance proportional to the weights of the element that are out of order. The total distance between two rankings is then the sum of the distance found for all such features that are in a different order. Hence, the more features that appear in a different order, the greater the distance between the rankings.
Clustering
K-mediods ( a variant of K-means) is used to cluster similar explanations together. Instead of the standard Manhattan distance which is used in clustering for input data points, here we use the Weighted Kendall’s Tau distance between the local attributions obtained for data points using any local explainability method. In the end, we present global explanations using the cluster centres of the clusters obtained after running the algorithm.
We can tune the number of sub-populations we want the global explanation to be partitioned into using the number of clusters(K) parameter in the algorithm. This allows us to set the granularity level that we desire ranging from local(high K) to global(lower K).
Although the paper says that this method is valid for Neural Networks specifically and all experiments are done for Neural Nets, yet, this can effectively be used for any model's explanation. Just explain the model using your favourite local model agnostic method such as LIME/SHAP and then run this method on top of those local explanations!
Resources
Last updated