Analyzer augmentations

All analyzers implemented in ExplainableAI.jl can be augmented by two types of augmentations: NoiseAugmentations and InterpolationAugmentations. These augmentations are wrappers around analyzers that modify the input before passing it to the analyzer.

We build on the basics shown in the Getting started section and start out by loading the same pre-trained LeNet5 model and MNIST input data:

using ExplainableAI
using VisionHeatmaps
using Zygote
using Flux

model

Chain(
  Conv((5, 5), 1 => 6, relu),           # 156 parameters
  MaxPool((2, 2)),
  Conv((5, 5), 6 => 16, relu),          # 2_416 parameters
  MaxPool((2, 2)),
  Flux.flatten,
  Dense(256 => 120, relu),              # 30_840 parameters
  Dense(120 => 84, relu),               # 10_164 parameters
  Dense(84 => 10),                      # 850 parameters
)                   # Total: 10 arrays, 44_426 parameters, 174.359 KiB.

using MLDatasets
using ImageCore, ImageIO, ImageShow

index = 10
x, y = MNIST(Float32, :test)[10]
input = reshape(x, 28, 28, 1, :)

convert2image(MNIST, x)

Noise augmentation

The NoiseAugmentation wrapper computes explanations averaged over noisy inputs. Let's demonstrate this on the Gradient analyzer. First, we compute the heatmap of an explanation without augmentation:

analyzer = Gradient(model)
heatmap(input, analyzer)

(a vector displayed as a row to save space)

Now we wrap the analyzer in a NoiseAugmentation with 10 samples of noise. By default, the noise is sampled from a Gaussian distribution with mean 0 and standard deviation 1.

Augmentations require a pooling keyword argument, which determines how the averaged attribution is reduced over color channels. The right choice depends on the wrapped analyzer: like Gradient, we use NormPooling.

analyzer = NoiseAugmentation(Gradient(model), 50; pooling = NormPooling())
heatmap(input, analyzer)

(a vector displayed as a row to save space)

Note that a higher sample size is desired, as it will lead to a smoother heatmap. However, this comes at the cost of a longer computation time.

We can also set the standard deviation of the Gaussian distribution:

analyzer = NoiseAugmentation(Gradient(model), 50, 0.1; pooling = NormPooling())
heatmap(input, analyzer)

(a vector displayed as a row to save space)

When used with a Gradient analyzer, this is equivalent to SmoothGrad:

analyzer = SmoothGrad(model, 50)
heatmap(input, analyzer)

(a vector displayed as a row to save space)

We can also use any distribution from Distributions.jl, for example Poisson noise with rate $\lambda=0.5$:

using Distributions

analyzer = NoiseAugmentation(Gradient(model), 50, Poisson(0.5); pooling = NormPooling())
heatmap(input, analyzer)

(a vector displayed as a row to save space)

Is is also possible to define your own distributions or mixture distributions.

NoiseAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

Integration augmentation

The InterpolationAugmentation wrapper computes explanations integrated over n points of linear interpolation between a reference input and the input, using the trapezoidal rule. The reference input is set to zero(input) by default. Multiplying with the difference between input and reference input turns gradients into signed attributions, so we pool them by summation:

analyzer = InterpolationAugmentation(Gradient(model), 50; pooling = SumPooling())
heatmap(input, analyzer)

(a vector displayed as a row to save space)

When used with a Gradient analyzer, this is equivalent to IntegratedGradients:

analyzer = IntegratedGradients(model, 50)
heatmap(input, analyzer)

(a vector displayed as a row to save space)

To select a different reference input, pass it to the analyze function using the keyword argument input_ref. Note that this is an arbitrary example for the sake of demonstration.

matrix_of_ones = ones(Float32, size(input))

analyzer = InterpolationAugmentation(Gradient(model), 50; pooling = SumPooling())
attr = analyzer(input; input_ref = matrix_of_ones)
heatmap(attr)

(a vector displayed as a row to save space)

Once again, InterpolationAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

This page was generated using Literate.jl.