analyze(input, method)
analyze(input, method, output_selection)

Apply the analyzer method for the given input, returning an Explanation. If output_selection is specified, the explanation will be calculated for that output. Otherwise, the output with the highest activation is automatically chosen.

See also Explanation.

Explanation(val, output, output_selection, analyzer, heatmap, extras)

Return type of analyzers when calling analyze.

Fields

• val: numerical output of the analyzer, e.g. an attribution or gradient
• output: model output for the given analyzer input
• output_selection: index of the output used for the explanation
• analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP
• heatmap: symbol indicating a preset heatmapping style, e.g. :attribution, :sensitivity or :cam
• extras: optional named tuple that can be used by analyzers to return additional information.

LRP(model, rules)
LRP(model, composite)

Analyze model by applying Layer-Wise Relevance Propagation. The analyzer can either be created by passing an array of LRP-rules or by passing a composite, see Composite for an example.

Keyword arguments

• normalize_output_relevance: Selects whether output relevance should be set to 1 before applying LRP backward pass. Defaults to true to match literature. If false, values of output activations are used.
• skip_checks::Bool: Skip checks whether model is compatible with LRP and contains output softmax. Defaults to false.
• verbose::Bool: Select whether the model checks should print a summary on failure. Defaults to true.

References

[1] G. Montavon et al., Layer-Wise Relevance Propagation: An Overview [2] W. Samek et al., Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications

strip_softmax(model)
strip_softmax(layer)

Remove softmax activation on layer or model if it exists.

canonize(model)

Canonize model by flattening it and fusing BatchNorm layers into preceding Dense and Conv layers with linear activation functions.

flatten_model(model)

Flatten a Flux Chain containing Chains.

ZeroRule()

LRP-$0$ rule. Commonly used on upper layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i \frac{W_{ij}a_j^k}{\sum_l W_{il}a_l^k+b_i} R_i^{k+1}\]

References

• S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

EpsilonRule([epsilon=1.0e-6])

LRP-$ϵ$ rule. Commonly used on middle layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{W_{ij}a_j^k}{\epsilon +\sum_{l}W_{il}a_l^k+b_i} R_i^{k+1}\]

Optional arguments

• epsilon: Optional stabilization parameter, defaults to 1.0e-6.

References

• S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

GammaRule([gamma=0.25])

LRP-$γ$ rule. Commonly used on lower layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{(W_{ij}+\gamma W_{ij}^+)a_j^k} {\sum_l(W_{il}+\gamma W_{il}^+)a_l^k+(b_i+\gamma b_i^+)} R_i^{k+1}\]

Optional arguments

• gamma: Optional multiplier for added positive weights, defaults to 0.25.

References

• G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

WSquareRule()

LRP-$w²$ rule. Commonly used on the first layer when values are unbounded.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{W_{ij}^2}{\sum_l W_{il}^2} R_i^{k+1}\]

References

• G. Montavon et al., Explaining Nonlinear Classification Decisions with Deep Taylor Decomposition

FlatRule()

LRP-Flat rule. Similar to the WSquareRule, but with all weights set to one and all bias terms set to zero.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{1}{\sum_l 1} R_i^{k+1} = \sum_i\frac{1}{n_i} R_i^{k+1}\]

where $n_i$ is the number of input neurons connected to the output neuron at index $i$.

References

• S. Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn

AlphaBetaRule([alpha=2.0, beta=1.0])

LRP-$αβ$ rule. Weights positive and negative contributions according to the parameters alpha and beta respectively. The difference $α-β$ must be equal to one. Commonly used on lower layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\left( \alpha\frac{\left(W_{ij}a_j^k\right)^+}{\sum_l\left(W_{il}a_l^k+b_i\right)^+} -\beta\frac{\left(W_{ij}a_j^k\right)^-}{\sum_l\left(W_{il}a_l^k+b_i\right)^-} \right) R_i^{k+1}\]

Optional arguments

• alpha: Multiplier for the positive output term, defaults to 2.0.
• beta: Multiplier for the negative output term, defaults to 1.0.

References

• S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
• G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

ZPlusRule()

LRP-$z⁺$ rule. Commonly used on lower layers.

Equivalent to AlphaBetaRule(1.0f0, 0.0f0), but slightly faster. See also AlphaBetaRule.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac{\left(W_{ij}a_j^k\right)^+}{\sum_l\left(W_{il}a_l^k+b_i\right)^+} R_i^{k+1}\]

References

• S. Bach et al., On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
• G. Montavon et al., Explaining Nonlinear Classification Decisions with Deep Taylor Decomposition

ZBoxRule(low, high)

LRP-$zᴮ$-rule. Commonly used on the first layer for pixel input.

The parameters low and high should be set to the lower and upper bounds of the input features, e.g. 0.0 and 1.0 for raw image data. It is also possible to provide two arrays of that match the input size.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k=\sum_i \frac{W_{ij}a_j^k - W_{ij}^{+}l_j - W_{ij}^{-}h_j} {\sum_l W_{il}a_l^k+b_i - \left(W_{il}^{+}l_l+b_i^{+}\right) - \left(W_{il}^{-}h_l+b_i^{-}\right)} R_i^{k+1}\]

References

• G. Montavon et al., Layer-Wise Relevance Propagation: An Overview

PassRule()

Pass-through rule. Passes relevance through to the lower layer.

Supports layers with constant input and output shapes, e.g. reshaping layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = R_j^{k+1}\]

GeneralizedGammaRule([gamma=0.25])

Generalized LRP-$γ$ rule. Can be used on layers with leakyrelu activation functions.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_j^k = \sum_i\frac {(W_{ij}+\gamma W_{ij}^+)a_j^+ +(W_{ij}+\gamma W_{ij}^-)a_j^-} {\sum_l(W_{il}+\gamma W_{il}^+)a_j^+ +(W_{il}+\gamma W_{il}^-)a_j^- +(b_i+\gamma b_i^+)} I(z_k>0) \cdot R^{k+1}_i +\sum_i\frac {(W_{ij}+\gamma W_{ij}^-)a_j^+ +(W_{ij}+\gamma W_{ij}^+)a_j^-} {\sum_l(W_{il}+\gamma W_{il}^-)a_j^+ +(W_{il}+\gamma W_{il}^+)a_j^- +(b_i+\gamma b_i^-)} I(z_k<0) \cdot R^{k+1}_i\]

Optional arguments

• gamma: Optional multiplier for added positive weights, defaults to 0.25.

References

• L. Andéol et al., Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization

LayerNormRule()

LRP-LN rule. Used on LayerNorm layers.

Definition

Propagates relevance $R^{k+1}$ at layer output to $R^k$ at layer input according to

\[R_i^k = \sum_j\frac{a_i^k\left(\delta_{ij} - 1/N\right)}{\sum_l a_l^k\left(\delta_{lj}-1/N\right)} R_j^{k+1}\]

Relevance through the affine transformation is by default propagated using the ZeroRule.

If you would like to assign a special rule to the affine transformation inside of the LayerNorm layer, call canonize on your model. This will split the LayerNorm layer into

1. a LayerNorm layer without affine transformation
2. a Scale layer implementing the affine transformation

You can then assign separate rules to these two layers.

References

• A. Ali et al., XAI for Transformers: Better Explanations through Conservative Propagation

Composite(primitives...)
Composite(default_rule, primitives...)

Automatically contructs a list of LRP-rules by sequentially applying composite primitives.

Primitives

To apply a single rule, use:

• LayerMap to apply a rule to the n-th layer of a model
• GlobalMap to apply a rule to all layers
• RangeMap to apply a rule to a positional range of layers
• FirstLayerMap to apply a rule to the first layer
• LastLayerMap to apply a rule to the last layer

To apply a set of rules to layers based on their type, use:

• GlobalTypeMap to apply a dictionary that maps layer types to LRP-rules
• RangeTypeMap for a TypeMap on generalized ranges
• FirstLayerTypeMap for a TypeMap on the first layer of a model
• LastLayerTypeMap for a TypeMap on the last layer
• FirstNTypeMap for a TypeMap on the first n layers

Example

Using a VGG11 model:

julia> composite = Composite(
           GlobalTypeMap(
               ConvLayer => AlphaBetaRule(),
               Dense => EpsilonRule(),
               PoolingLayer => EpsilonRule(),
               DropoutLayer => PassRule(),
               ReshapingLayer => PassRule(),
           ),
           FirstNTypeMap(7, Conv => FlatRule()),
       );

julia> analyzer = LRP(model, composite)
LRP(
  Conv((3, 3), 3 => 64, relu, pad=1)    => FlatRule(),
  MaxPool((2, 2))                       => EpsilonRule{Float32}(1.0f-6),
  Conv((3, 3), 64 => 128, relu, pad=1)  => FlatRule(),
  MaxPool((2, 2))                       => EpsilonRule{Float32}(1.0f-6),
  Conv((3, 3), 128 => 256, relu, pad=1) => FlatRule(),
  Conv((3, 3), 256 => 256, relu, pad=1) => FlatRule(),
  MaxPool((2, 2))                       => EpsilonRule{Float32}(1.0f-6),
  Conv((3, 3), 256 => 512, relu, pad=1) => AlphaBetaRule{Float32}(2.0f0, 1.0f0),
  Conv((3, 3), 512 => 512, relu, pad=1) => AlphaBetaRule{Float32}(2.0f0, 1.0f0),
  MaxPool((2, 2))                       => EpsilonRule{Float32}(1.0f-6),
  Conv((3, 3), 512 => 512, relu, pad=1) => AlphaBetaRule{Float32}(2.0f0, 1.0f0),
  Conv((3, 3), 512 => 512, relu, pad=1) => AlphaBetaRule{Float32}(2.0f0, 1.0f0),
  MaxPool((2, 2))                       => EpsilonRule{Float32}(1.0f-6),
  MLUtils.flatten                       => PassRule(),
  Dense(25088 => 4096, relu)            => EpsilonRule{Float32}(1.0f-6),
  Dropout(0.5)                          => PassRule(),
  Dense(4096 => 4096, relu)             => EpsilonRule{Float32}(1.0f-6),
  Dropout(0.5)                          => PassRule(),
  Dense(4096 => 1000)                   => EpsilonRule{Float32}(1.0f-6),
)

lrp_rules(model, composite)

Apply a composite to obtain LRP-rules for a given Flux model.

LayerMap(index, rule)

Composite primitive that maps an LRP-rule to all layers in the model at the given index. The index can either be an integer or a tuple of integers to map a rule to a specific layer in nested Flux Chains.

See show_layer_indices to print layer indices and Composite for an example.

GlobalMap(rule)

Composite primitive that maps an LRP-rule to all layers in the model.

See Composite for an example.

RangeMap(range, rule)

Composite primitive that maps an LRP-rule to the specified positional range of layers in the model.

See Composite for an example.

FirstLayerMap(rule)

Composite primitive that maps an LRP-rule to the first layer in the model.

See Composite for an example.

LastLayerMap(rule)

Composite primitive that maps an LRP-rule to the last layer in the model.

See Composite for an example.

show_layer_indices(model)

Print layer indices of Flux models. This is primarily a utility to help define LayerMap primitives.

GlobalTypeMap(map)

Composite primitive that maps layer types to LRP rules based on a list of type-rule-pairs map.

See Composite for an example.

RangeTypeMap(range, map)

Composite primitive that maps layer types to LRP rules based on a list of type-rule-pairs map within the specified range of layers in the model.

See Composite for an example.

FirstLayerTypeMap(map)

Composite primitive that maps the type of the first layer of the model to LRP rules based on a list of type-rule-pairs map.

See Composite for an example.

LastLayerTypeMap(map)

Composite primitive that maps the type of the last layer of the model to LRP rules based on a list of type-rule-pairs map.

See Composite for an example.

FirstNTypeMap(n, map)

Composite primitive that maps layer types to LRP rules based on a list of type-rule-pairs map within the first n layers in the model.

See Composite for an example.

Union type for convolutional layers.

Union type for pooling layers.

Union type for dropout layers.

Union type for reshaping layers such as flatten.

Union type for normalization layers.

EpsilonGammaBox(low, high; [epsilon=1.0f-6, gamma=0.25f0])

Composite using the following primitives:

julia> EpsilonGammaBox(-3.0f0, 3.0f0)
Composite(
  GlobalTypeMap(  # all layers
    Flux.Conv               => RelevancePropagation.GammaRule{Float32}(0.25f0),
    Flux.ConvTranspose      => RelevancePropagation.GammaRule{Float32}(0.25f0),
    Flux.CrossCor           => RelevancePropagation.GammaRule{Float32}(0.25f0),
    Flux.Dense              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.Scale              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.LayerNorm          => RelevancePropagation.LayerNormRule(),
    typeof(NNlib.dropout)   => RelevancePropagation.PassRule(),
    Flux.AlphaDropout       => RelevancePropagation.PassRule(),
    Flux.Dropout            => RelevancePropagation.PassRule(),
    Flux.BatchNorm          => RelevancePropagation.PassRule(),
    typeof(Flux.flatten)    => RelevancePropagation.PassRule(),
    typeof(MLUtils.flatten) => RelevancePropagation.PassRule(),
    typeof(identity)        => RelevancePropagation.PassRule(),
 ),
  FirstLayerTypeMap(  # first layer
    Flux.Conv          => RelevancePropagation.ZBoxRule{Float32}(-3.0f0, 3.0f0),
    Flux.ConvTranspose => RelevancePropagation.ZBoxRule{Float32}(-3.0f0, 3.0f0),
    Flux.CrossCor      => RelevancePropagation.ZBoxRule{Float32}(-3.0f0, 3.0f0),
 ),
)

EpsilonPlus(; [epsilon=1.0f-6])

Composite using the following primitives:

julia> EpsilonPlus()
Composite(
  GlobalTypeMap(  # all layers
    Flux.Conv               => RelevancePropagation.ZPlusRule(),
    Flux.ConvTranspose      => RelevancePropagation.ZPlusRule(),
    Flux.CrossCor           => RelevancePropagation.ZPlusRule(),
    Flux.Dense              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.Scale              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.LayerNorm          => RelevancePropagation.LayerNormRule(),
    typeof(NNlib.dropout)   => RelevancePropagation.PassRule(),
    Flux.AlphaDropout       => RelevancePropagation.PassRule(),
    Flux.Dropout            => RelevancePropagation.PassRule(),
    Flux.BatchNorm          => RelevancePropagation.PassRule(),
    typeof(Flux.flatten)    => RelevancePropagation.PassRule(),
    typeof(MLUtils.flatten) => RelevancePropagation.PassRule(),
    typeof(identity)        => RelevancePropagation.PassRule(),
 ),
)

EpsilonAlpha2Beta1(; [epsilon=1.0f-6])

Composite using the following primitives:

julia> EpsilonAlpha2Beta1()
Composite(
  GlobalTypeMap(  # all layers
    Flux.Conv               => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.ConvTranspose      => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.CrossCor           => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.Dense              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.Scale              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.LayerNorm          => RelevancePropagation.LayerNormRule(),
    typeof(NNlib.dropout)   => RelevancePropagation.PassRule(),
    Flux.AlphaDropout       => RelevancePropagation.PassRule(),
    Flux.Dropout            => RelevancePropagation.PassRule(),
    Flux.BatchNorm          => RelevancePropagation.PassRule(),
    typeof(Flux.flatten)    => RelevancePropagation.PassRule(),
    typeof(MLUtils.flatten) => RelevancePropagation.PassRule(),
    typeof(identity)        => RelevancePropagation.PassRule(),
 ),
)

EpsilonPlusFlat(; [epsilon=1.0f-6])

Composite using the following primitives:

julia> EpsilonPlusFlat()
Composite(
  GlobalTypeMap(  # all layers
    Flux.Conv               => RelevancePropagation.ZPlusRule(),
    Flux.ConvTranspose      => RelevancePropagation.ZPlusRule(),
    Flux.CrossCor           => RelevancePropagation.ZPlusRule(),
    Flux.Dense              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.Scale              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.LayerNorm          => RelevancePropagation.LayerNormRule(),
    typeof(NNlib.dropout)   => RelevancePropagation.PassRule(),
    Flux.AlphaDropout       => RelevancePropagation.PassRule(),
    Flux.Dropout            => RelevancePropagation.PassRule(),
    Flux.BatchNorm          => RelevancePropagation.PassRule(),
    typeof(Flux.flatten)    => RelevancePropagation.PassRule(),
    typeof(MLUtils.flatten) => RelevancePropagation.PassRule(),
    typeof(identity)        => RelevancePropagation.PassRule(),
 ),
  FirstLayerTypeMap(  # first layer
    Flux.Conv          => RelevancePropagation.FlatRule(),
    Flux.ConvTranspose => RelevancePropagation.FlatRule(),
    Flux.CrossCor      => RelevancePropagation.FlatRule(),
 ),
)

EpsilonAlpha2Beta1Flat(; [epsilon=1.0f-6])

Composite using the following primitives:

julia> EpsilonAlpha2Beta1Flat()
Composite(
  GlobalTypeMap(  # all layers
    Flux.Conv               => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.ConvTranspose      => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.CrossCor           => RelevancePropagation.AlphaBetaRule{Float32}(2.0f0, 1.0f0),
    Flux.Dense              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.Scale              => RelevancePropagation.EpsilonRule{Float32}(1.0f-6),
    Flux.LayerNorm          => RelevancePropagation.LayerNormRule(),
    typeof(NNlib.dropout)   => RelevancePropagation.PassRule(),
    Flux.AlphaDropout       => RelevancePropagation.PassRule(),
    Flux.Dropout            => RelevancePropagation.PassRule(),
    Flux.BatchNorm          => RelevancePropagation.PassRule(),
    typeof(Flux.flatten)    => RelevancePropagation.PassRule(),
    typeof(MLUtils.flatten) => RelevancePropagation.PassRule(),
    typeof(identity)        => RelevancePropagation.PassRule(),
 ),
  FirstLayerTypeMap(  # first layer
    Flux.Conv          => RelevancePropagation.FlatRule(),
    Flux.ConvTranspose => RelevancePropagation.FlatRule(),
    Flux.CrossCor      => RelevancePropagation.FlatRule(),
 ),
)

ChainTuple(xs)

Thin wrapper around Tuple for use with Flux.jl models.

Combining ChainTuple, ParallelTuple and SkipConnectionTuple, data xs can be stored while preserving the structure of a Flux model without risking type piracy.

ParallelTuple(xs)

Thin wrapper around Tuple for use with Flux.jl models.

Combining ChainTuple, ParallelTuple and SkipConnectionTuple, data xs can be stored while preserving the structure of a Flux model without risking type piracy.

SkipConnectionTuple(xs)

Thin wrapper around Tuple for use with Flux.jl models.

Combining ChainTuple, ParallelTuple and SkipConnectionTuple, data xs can be stored while preserving the structure of a Flux model without risking type piracy.

modify_input(rule, input)

Modify input activation before computing relevance propagation.

modify_denominator(rule, d)

Modify denominator $z$ for numerical stability on the forward pass.

modify_parameters(rule, parameter)

Modify parameters before computing the relevance.

Note

Use of a custom function modify_layer will overwrite functionality of modify_parameters, modify_weight and modify_bias for the implemented combination of rule and layer types. This is due to the fact that internally, modify_weight and modify_bias are called by the default implementation of modify_layer. modify_weight and modify_bias in turn call modify_parameters by default.

The default call structure looks as follows:

┌─────────────────────────────────────────┐
│              modify_layer               │
└─────────┬─────────────────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│   modify_weight   │ │    modify_bias    │
└─────────┬─────────┘ └─────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│ modify_parameters │ │ modify_parameters │
└───────────────────┘ └───────────────────┘

modify_weight(rule, weight)

Modify layer weights before computing the relevance.

Note

Use of a custom function modify_layer will overwrite functionality of modify_parameters, modify_weight and modify_bias for the implemented combination of rule and layer types. This is due to the fact that internally, modify_weight and modify_bias are called by the default implementation of modify_layer. modify_weight and modify_bias in turn call modify_parameters by default.

The default call structure looks as follows:

┌─────────────────────────────────────────┐
│              modify_layer               │
└─────────┬─────────────────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│   modify_weight   │ │    modify_bias    │
└─────────┬─────────┘ └─────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│ modify_parameters │ │ modify_parameters │
└───────────────────┘ └───────────────────┘

modify_bias(rule, bias)

Modify layer bias before computing the relevance.

Note

Use of a custom function modify_layer will overwrite functionality of modify_parameters, modify_weight and modify_bias for the implemented combination of rule and layer types. This is due to the fact that internally, modify_weight and modify_bias are called by the default implementation of modify_layer. modify_weight and modify_bias in turn call modify_parameters by default.

The default call structure looks as follows:

┌─────────────────────────────────────────┐
│              modify_layer               │
└─────────┬─────────────────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│   modify_weight   │ │    modify_bias    │
└─────────┬─────────┘ └─────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│ modify_parameters │ │ modify_parameters │
└───────────────────┘ └───────────────────┘

modify_layer(rule, layer)

Modify layer before computing the relevance.

Note

Use of a custom function modify_layer will overwrite functionality of modify_parameters, modify_weight and modify_bias for the implemented combination of rule and layer types. This is due to the fact that internally, modify_weight and modify_bias are called by the default implementation of modify_layer. modify_weight and modify_bias in turn call modify_parameters by default.

The default call structure looks as follows:

┌─────────────────────────────────────────┐
│              modify_layer               │
└─────────┬─────────────────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│   modify_weight   │ │    modify_bias    │
└─────────┬─────────┘ └─────────┬─────────┘
          │ calls               │ calls
┌─────────▼─────────┐ ┌─────────▼─────────┐
│ modify_parameters │ │ modify_parameters │
└───────────────────┘ └───────────────────┘

is_compatible(rule, layer)

Check compatibility of a LRP-Rule with layer type.

LRP_CONFIG.supports_layer(layer)

Check whether LRP can be used on a layer or a Chain. To extend LRP to your own layers, define:

LRP_CONFIG.supports_layer(::MyLayer) = true          # for structs
LRP_CONFIG.supports_layer(::typeof(mylayer)) = true  # for functions

LRP_CONFIG.supports_activation(σ)

Check whether LRP can be used on a given activation function. To extend LRP to your own activation functions, define:

LRP_CONFIG.supports_activation(::typeof(myactivation)) = true  # for functions
LRP_CONFIG.supports_activation(::MyActivation) = true          # for structs

CRP(lrp_analyzer, layer, features)

Use Concept Relevance Propagation to explain the output of a neural network with respect to specific features in a given layer.

Arguments

• lrp_analyzer::LRP: LRP analyzer
• layer::Int: Index of layer after which the concept is located
• features: Concept / feature to explain.

See also TopNFeatures and IndexedFeatures.

References

[1] R. Achtibat et al., From attribution maps to human-understandable explanations through Concept Relevance Propagation

TopNFeatures(n)

Select top-n features.

For outputs of convolutional layers, the relevance is summed across height and width channels for each feature.

See also IndexedFeatures.

Note

The XAIBase interface currently assumes that features have either 2 or 4 dimensions ((features, batchsize) or (width, height, features, batchsize)).

It also assumes that the batch dimension is the last dimension of the feature.

Example

julia> feature_selector = TopNFeatures(2)
 TopNFeatures(2)

julia> feature = rand(3, 2)
3×2 Matrix{Float64}:
 0.265312  0.953689
 0.674377  0.172154
 0.649722  0.570809

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{2, Tuple{UnitRange{Int64}, UnitRange{Int64}}}}}:
 [CartesianIndices((2:2, 1:1)), CartesianIndices((1:1, 2:2))]
 [CartesianIndices((3:3, 1:1)), CartesianIndices((3:3, 2:2))]

julia> feature = rand(3, 3, 3, 2);

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{4, NTuple{4, UnitRange{Int64}}}}}:
 [CartesianIndices((1:3, 1:3, 2:2, 1:1)), CartesianIndices((1:3, 1:3, 1:1, 2:2))]
 [CartesianIndices((1:3, 1:3, 1:1, 1:1)), CartesianIndices((1:3, 1:3, 3:3, 2:2))]

IndexedFeatures(indices...)

Select features by indices.

For outputs of convolutional layers, the index refers to a feature dimension.

See also See also TopNFeatures.

Note

The XAIBase interface currently assumes that features have either 2 or 4 dimensions ((features, batchsize) or (width, height, features, batchsize)).

It also assumes that the batch dimension is the last dimension of the feature.

Example

julia> feature_selector = IndexedFeatures(2, 3)
 IndexedFeatures(2, 3)

julia> feature = rand(3, 3, 3, 2);

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{4, NTuple{4, UnitRange{Int64}}}}}:
 [CartesianIndices((1:3, 1:3, 2:2, 1:1)), CartesianIndices((1:3, 1:3, 2:2, 2:2))]
 [CartesianIndices((1:3, 1:3, 3:3, 1:1)), CartesianIndices((1:3, 1:3, 3:3, 2:2))]

julia> feature = rand(3, 2);

julia> feature_selector(feature)
 1-element Vector{Vector{CartesianIndices{2, Tuple{UnitRange{Int64}, UnitRange{Int64}}}}}:
  [CartesianIndices((2:2, 1:1)), CartesianIndices((2:2, 2:2))]