LRP Rule Overview

Notation

We use the following notation for LRP rules:

$W$ is the weight matrix of the layer
$b$ is the bias vector of the layer
$a^k$ is the activation vector at the input of layer $k$
$a^{k+1}$ is the activation vector at the output of layer $k$
$R^k$ is the relevance vector at the input of layer $k$
$R^{k+1}$ is the relevance vector at the output of layer $k$
$\rho$ is a function that modifies parameters (what we call modify_parameters)
$\epsilon$ is a small positive constant to avoid division by zero

Subscript characters are used to index vectors and matrices (e.g. $b_i$ is the $i$-th entry of the bias vector), while the superscripts $^k$ and $^{k+1}$ indicate the relative positions of activations $a$ and relevances $R$ in the model. For any $k$, $a^k$ and $R^k$ have the same shape.

Note that all terms in the following equations are scalar value, which removes the need to differentiate between matrix and element-wise operations. For more information, refer to the developer documentation.

Basic rules

RelevancePropagation.ZeroRule — Type

ZeroRule()

LRP-$0$ rule. Commonly used on upper layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i \frac{W_{ij}a_j^k}{\sum_l W_{il}a_l^k+b_i} R_i^{k+1}\]

References

S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

source

RelevancePropagation.EpsilonRule — Type

EpsilonRule([epsilon=1.0e-6])

LRP-$ϵ$ rule. Commonly used on middle layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{W_{ij}a_j^k}{\epsilon +\sum_{l}W_{il}a_l^k+b_i} R_i^{k+1}\]

Optional arguments

epsilon: Optional stabilization parameter, defaults to 1.0e-6.

References

S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

source

RelevancePropagation.PassRule — Type

PassRule()

Pass-through rule. Passes relevance through to the lower layer.

Supports layers with constant input and output shapes, e.g. reshaping layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = R_j^{k+1}\]

source

Lower layer rules

RelevancePropagation.GammaRule — Type

GammaRule([gamma=0.25])

LRP-$γ$ rule. Commonly used on lower layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{(W_{ij}+\gamma W_{ij}^+)a_j^k} {\sum_l(W_{il}+\gamma W_{il}^+)a_l^k+(b_i+\gamma b_i^+)} R_i^{k+1}\]

Optional arguments

gamma: Optional multiplier for added positive weights, defaults to 0.25.

References

G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

source

RelevancePropagation.AlphaBetaRule — Type

AlphaBetaRule([alpha=2.0, beta=1.0])

LRP-$αβ$ rule. Weights positive and negative contributions according to the parameters alpha and beta respectively. The difference $α-β$ must be equal to one. Commonly used on lower layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\left( \alpha\frac{\left(W_{ij}a_j^k\right)^+}{\sum_l\left(W_{il}a_l^k+b_i\right)^+} -\beta\frac{\left(W_{ij}a_j^k\right)^-}{\sum_l\left(W_{il}a_l^k+b_i\right)^-} \right) R_i^{k+1}\]

Optional arguments

alpha: Multiplier for the positive output term, defaults to 2.0.
beta: Multiplier for the negative output term, defaults to 1.0.

References

S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

source

RelevancePropagation.ZPlusRule — Type

ZPlusRule()

LRP-$z⁺$ rule. Commonly used on lower layers.

Equivalent to AlphaBetaRule(1.0f0, 0.0f0), but slightly faster. See also AlphaBetaRule.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{\left(W_{ij}a_j^k\right)^+}{\sum_l\left(W_{il}a_l^k+b_i\right)^+} R_i^{k+1}\]

References

S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
G. Montavon et al., Explaining Nonlinear Classification Decisions with Deep Taylor Decomposition

source

Input layer rules

RelevancePropagation.FlatRule — Type

FlatRule()

LRP-Flat rule. Similar to the WSquareRule, but with all weights set to one and all bias terms set to zero.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{1}{\sum_l 1} R_i^{k+1} = \sum_i\frac{1}{n_i} R_i^{k+1}\]

where $n_i$ is the number of input neurons connected to the output neuron at index $i$.

References

S. Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn

source

RelevancePropagation.WSquareRule — Type

WSquareRule()

LRP-$w²$ rule. Commonly used on the first layer when values are unbounded.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{W_{ij}^2}{\sum_l W_{il}^2} R_i^{k+1}\]

References

G. Montavon et al., Explaining Nonlinear Classification Decisions with Deep Taylor Decomposition

source

RelevancePropagation.ZBoxRule — Type

ZBoxRule(low, high)

LRP-$zᴮ$-rule. Commonly used on the first layer for pixel input.

The parameters low and high should be set to the lower and upper bounds of the input features, e.g. 0.0 and 1.0 for raw image data. It is also possible to provide two arrays of that match the input size.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k=\sum_i \frac{W_{ij}a_j^k - W_{ij}^{+}l_j - W_{ij}^{-}h_j} {\sum_l W_{il}a_l^k+b_i - \left(W_{il}^{+}l_l+b_i^{+}\right) - \left(W_{il}^{-}h_l+b_i^{-}\right)} R_i^{k+1}\]

References

G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

source

Specialized rules

RelevancePropagation.LayerNormRule — Type

LayerNormRule()

LRP-LN rule. Used on LayerNorm layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_i^k = \sum_j\frac{a_i^k\left(\delta_{ij} - 1/N\right)}{\sum_l a_l^k\left(\delta_{lj}-1/N\right)} R_j^{k+1}\]

Relevance through the affine transformation is by default propagated using the ZeroRule.

If you would like to assign a special rule to the affine transformation inside of the LayerNorm layer, call canonize on your model. This will split the LayerNorm layer into

a LayerNorm layer without affine transformation
a Scale layer implementing the affine transformation

You can then assign separate rules to these two layers.

References

A. Ali et al., XAI for Transformers: Better Explanations through Conservative Propagation

source

RelevancePropagation.GeneralizedGammaRule — Type

GeneralizedGammaRule([gamma=0.25])

Generalized LRP-$γ$ rule. Can be used on layers with leakyrelu activation functions.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac {(W_{ij}+\gamma W_{ij}^+)a_j^+ +(W_{ij}+\gamma W_{ij}^-)a_j^-} {\sum_l(W_{il}+\gamma W_{il}^+)a_j^+ +(W_{il}+\gamma W_{il}^-)a_j^- +(b_i+\gamma b_i^+)} I(z_k>0) \cdot R^{k+1}_i +\sum_i\frac {(W_{ij}+\gamma W_{ij}^-)a_j^+ +(W_{ij}+\gamma W_{ij}^+)a_j^-} {\sum_l(W_{il}+\gamma W_{il}^-)a_j^+ +(W_{il}+\gamma W_{il}^+)a_j^- +(b_i+\gamma b_i^-)} I(z_k<0) \cdot R^{k+1}_i\]

Optional arguments

gamma: Optional multiplier for added positive weights, defaults to 0.25.

References

L. Andéol et al., Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization

source