API Reference

Abstract super type of all XAI methods.

It is expected that all XAI methods are callable types that return an Explanation:

(method::AbstractXAIMethod)(input, output_selector::AbstractOutputSelector)

If this function is implemented, XAIBase will provide the analyze functionality and heatmap functionality by loading either VisionHeatmaps.jl or TextHeatmaps.jl.

analyze(input, method)
analyze(input, method, output_selection)

Apply the analyzer method for the given input, returning an Explanation. If output_selection is specified, the explanation will be calculated for that output. Otherwise, the output with the highest activation is automatically chosen.

See also Explanation.

Explanation(val, output, output_selection, analyzer, heatmap, extras)

Return type of analyzers when calling analyze.

Fields

• val: numerical output of the analyzer, e.g. an attribution or gradient
• output: model output for the given analyzer input
• output_selection: index of the output used for the explanation
• analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP
• heatmap: symbol indicating a preset heatmapping style, e.g. :attribution, :sensitivity or :cam
• extras: optional named tuple that can be used by analyzers to return additional information.

Abstract super type of all feature selectors in XAIBase.

Feature selectors are expected to be callable and to return a vector of CartesianIndices for each selected feature.

Note

The XAIBase interface currently assumes that features have either 2 or 4 dimensions ((features, batchsize) or (width, height, features, batchsize)).

It also assumes that the batch dimension is the last dimension of the feature.

IndexedFeatures(indices...)

Select features by indices.

For outputs of convolutional layers, the index refers to a feature dimension.

See also See also TopNFeatures.

Note

The XAIBase interface currently assumes that features have either 2 or 4 dimensions ((features, batchsize) or (width, height, features, batchsize)).

It also assumes that the batch dimension is the last dimension of the feature.

Example

julia> feature_selector = IndexedFeatures(2, 3)
 IndexedFeatures(2, 3)

julia> feature = rand(3, 3, 3, 2);

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{4, NTuple{4, UnitRange{Int64}}}}}:
 [CartesianIndices((1:3, 1:3, 2:2, 1:1)), CartesianIndices((1:3, 1:3, 2:2, 2:2))]
 [CartesianIndices((1:3, 1:3, 3:3, 1:1)), CartesianIndices((1:3, 1:3, 3:3, 2:2))]

julia> feature = rand(3, 2);

julia> feature_selector(feature)
 1-element Vector{Vector{CartesianIndices{2, Tuple{UnitRange{Int64}, UnitRange{Int64}}}}}:
  [CartesianIndices((2:2, 1:1)), CartesianIndices((2:2, 2:2))]

TopNFeatures(n)

Select top-n features.

For outputs of convolutional layers, the relevance is summed across height and width channels for each feature.

See also IndexedFeatures.

Note

The XAIBase interface currently assumes that features have either 2 or 4 dimensions ((features, batchsize) or (width, height, features, batchsize)).

It also assumes that the batch dimension is the last dimension of the feature.

Example

julia> feature_selector = TopNFeatures(2)
 TopNFeatures(2)

julia> feature = rand(3, 2)
3×2 Matrix{Float64}:
 0.265312  0.953689
 0.674377  0.172154
 0.649722  0.570809

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{2, Tuple{UnitRange{Int64}, UnitRange{Int64}}}}}:
 [CartesianIndices((2:2, 1:1)), CartesianIndices((1:1, 2:2))]
 [CartesianIndices((3:3, 1:1)), CartesianIndices((3:3, 2:2))]

julia> feature = rand(3, 3, 3, 2);

julia> feature_selector(feature)
2-element Vector{Vector{CartesianIndices{4, NTuple{4, UnitRange{Int64}}}}}:
 [CartesianIndices((1:3, 1:3, 2:2, 1:1)), CartesianIndices((1:3, 1:3, 1:1, 2:2))]
 [CartesianIndices((1:3, 1:3, 1:1, 1:1)), CartesianIndices((1:3, 1:3, 3:3, 2:2))]

Abstract super type of all output selectors in XAIBase.

Output selectors are expected to be callable and to return a vector of CartesianIndex of the selected outputs.

Note

XAIBase assumes that the output of a model is a matrix of logits and that the batch dimension is the last dimension of the output.

MaxActivationSelector()

Output selector that picks the output with the highest activation.

Note

XAIBase assumes that the output of a model is a matrix of logits and that the batch dimension is the last dimension of the output.

Example

julia> output = rand(3, 3)
3×3 Matrix{Float64}:
 0.411871  0.313366  0.13402
 0.885562  0.136938  0.465622
 0.498235  0.627209  0.298911

julia> output_selector = MaxActivationSelector()
 MaxActivationSelector()

julia> output_selector(output)
 3-element Vector{CartesianIndex{2}}:
  CartesianIndex(2, 1)
  CartesianIndex(3, 2)
  CartesianIndex(2, 3)

IndexSelector(index)

Output selector that picks the output at the given index. If multiple indices are provided, outputs of a batch are selected separately.

Note

XAIBase assumes that the output of a model is a matrix of logits and that the batch dimension is the last dimension of the output.

Example

julia> output = rand(3, 3)
3×3 Matrix{Float64}:
 0.411871  0.313366  0.13402
 0.885562  0.136938  0.465622
 0.498235  0.627209  0.298911

julia> output_selector = IndexSelector(1)
 IndexSelector{Int64}(1)

julia> output_selector(output)
 3-element Vector{CartesianIndex{2}}:
  CartesianIndex(1, 1)
  CartesianIndex(1, 2)
  CartesianIndex(1, 3)

julia> output_selector = IndexSelector((1, 1, 2));

julia> output_selector(output)
3-element Vector{CartesianIndex{2}}:
 CartesianIndex(1, 1)
 CartesianIndex(1, 2)
 CartesianIndex(2, 3)