shap
Explain and audit machine-learning predictions with SHAP. Use for selecting SHAP explainers and maskers, computing and validating feature attributions, handling multi-output explanations, and producing local or global SHAP visualizations.
Install this skill
Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.
npx skills add KalarisLabs/research-agent-skills --skill shapuvx research-agent-skills install shapDocumentation & Instructions
SHAP
Use SHAP to describe how a fitted predictive model maps inputs to outputs. Work from the modern shap.Explanation API, make the explained output and background distribution explicit, and validate every explanation before interpreting it.
This skill is aligned with SHAP 0.52.0 (released 2026-05-28). That release requires Python 3.12 or newer.
Operating Rules
- Explain a fixed, evaluated model; do not use SHAP as a substitute for predictive validation.
- Use held-out or clearly labeled analysis rows for explanations. Choose background rows only from an appropriate training or reference population.
- State the explained output: regression value, raw margin, probability, log loss, logit, or another model method.
- Keep explanations as
shap.Explanationobjects. Callexplainer(X); use.shap_values(X)only when maintaining legacy code. - For multi-output models, select one output before using tabular plots:
explanation[..., output_index]. - Check
base_values + values.sum(...)against the exact model output being explained. - Treat SHAP as a description of model behavior under a masking/background choice. It does not establish causality, fairness, recourse, or scientific mechanism.
- Never silence an additivity failure until input shape, preprocessing, model version, output space, and row ordering have been checked.
- Do not load untrusted pickle, joblib, model, or explainer artifacts; those formats can execute code during deserialization.
Install
Create an isolated environment and pin the documented release:
uv venv --python 3.12
source .venv/bin/activate
uv pip install "shap[plots]==0.52.0"
shap[plots] installs the plotting dependencies. Add the fitted model's package at a version compatible with the project. For older Python compatibility, read references/migration.md instead of silently installing a different SHAP release.
Confirm the environment before debugging an API mismatch:
import platform
import shap
print("Python:", platform.python_version())
print("SHAP:", shap.__version__)
Standard Workflow
1. Define the explanation target
Record:
- model and preprocessing version;
- exact callable or model method being explained;
- output name/index and units;
- evaluation rows;
- background/reference population;
- masker and explainer algorithm;
- SHAP and model-library versions.
For classifiers, decide whether the task needs raw margins or probabilities. Defaults differ by model family; never infer units from the plot color or sign.
2. Select an explainer and masker
Start with shap.Explainer(model, masker) when automatic dispatch is sufficient. Instantiate a specialized explainer when its assumptions or output controls matter.
| Situation | Preferred choice | Important constraint |
|---|---|---|
| Supported tree ensemble | TreeExplainer |
model_output="probability" and "log_loss" require interventional masking and background data |
| Linear model | LinearExplainer |
The masker determines interventional versus correlation-aware behavior |
| Small feature space | ExactExplainer |
Cost grows quickly with unconstrained feature count |
| General tabular callable | PermutationExplainer |
Budget at least one full forward/reverse permutation |
| Hierarchical feature groups, text, or image | PartitionExplainer |
The partition tree changes the cooperative game |
| Differentiable neural network | DeepExplainer or GradientExplainer |
Framework support, output shape, and background choice require testing |
| Legacy Kernel SHAP workflow | KernelExplainer |
Usually much slower than model-specific methods |
Use the detailed decision guide in references/explainers.md. Use references/data-maskers.md when features are correlated, structured, sparse, or semantically grouped.
3. Compute a modern Explanation
This complete binary-classification example uses an explicit background and selects the positive-class output:
import numpy as np
import shap
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(as_frame=True, return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
stratify=y,
random_state=7,
)
model = RandomForestClassifier(
n_estimators=200,
min_samples_leaf=3,
random_state=7,
n_jobs=-1,
).fit(X_train, y_train)
background = shap.sample(X_train, 100, random_state=7)
explainer = shap.Explainer(model, background, algorithm="tree")
all_outputs = explainer(X_test)
# sklearn tree classifiers expose one output per class.
positive = all_outputs[..., 1]
assert positive.values.shape == X_test.shape
reconstructed = np.asarray(positive.base_values) + positive.values.sum(axis=1)
expected = model.predict_proba(X_test)[:, 1]
np.testing.assert_allclose(reconstructed, expected, rtol=1e-5, atol=1e-6)
shap.plots.beeswarm(positive, max_display=15)
shap.plots.waterfall(positive[0], max_display=15)
Output shape is model-dependent:
- one tabular output:
(samples, features); - multiple tabular outputs:
(samples, features, outputs); - multiple model inputs: often a list of arrays or explanations;
- image/text explanations: feature axes follow the input representation, with output selection on the final axis when present.
Do not use the pre-0.45 pattern values[class_index] for a modern multi-output array. Use values[..., class_index] or slice the Explanation itself.
4. Control tree output semantics when needed
For a supported tree classifier, probability-space explanations must be explicit:
background = shap.sample(X_train, 200, random_state=7)
explainer = shap.TreeExplainer(
model,
data=background,
feature_perturbation="interventional",
model_output="probability",
)
probability_exp = explainer(X_test)
In SHAP 0.52:
feature_perturbation="auto"uses interventional semantics when background data is supplied and tree-path-dependent semantics otherwise;- probability and log-loss output modes are supported only with interventional semantics;
- pass
approximate=Truetoexplainer(X, approximate=True)if deliberately using the lower-fidelity tree approximation; do not pass it to the constructor.
5. Use a model-agnostic callable deliberately
Pass the exact callable whose outputs will be interpreted:
masker = shap.maskers.Independent(background, max_samples=100)
explainer = shap.Explainer(
model.predict_proba,
masker,
algorithm="permutation",
output_names=[str(label) for label in model.classes_],
seed=7,
)
budget = 2 * X_test.shape[1] + 1
all_outputs = explainer(X_test.iloc[:20], max_evals=budget)
positive = all_outputs[..., 1]
Increase max_evals to average over more permutations when estimates are unstable. Keep the seed, background sample, and evaluation budget in the report.
6. Visualize the question, not merely the available plot
| Question | Plot |
|---|---|
| Which features have the largest average attribution magnitude? | shap.plots.bar(exp) |
| How do direction, magnitude, and observed values vary globally? | shap.plots.beeswarm(exp) |
| Why did one prediction differ from its baseline? | shap.plots.waterfall(exp[i]) |
| How does one feature's attribution vary over its values? | shap.plots.scatter(exp[:, feature]) |
| Do explanations form sample-level patterns? | shap.plots.heatmap(exp) |
| How do predefined cohorts differ descriptively? | shap.plots.bar(exp.cohorts(labels).abs.mean(0)) |
| Which tokens or image regions contribute to an output? | shap.plots.text(exp) or shap.plots.image(exp) |
Read references/plots.md before customizing or saving figures.
7. Report limitations with results
At minimum, report:
- output and units;
- baseline/reference population;
- explainer and masker;
- sample count and selection;
- output index/name;
- additivity error or applicable approximation diagnostics;
- known correlated/grouped features;
- whether results are local, aggregated, or cohort-specific;
- a clear non-causal statement.
Common Tasks
Global and local analysis
Use global plots to locate important patterns, scatter plots to inspect those patterns, and local plots to investigate selected rows. Do not select only visually dramatic rows without documenting the selection rule.
Multiclass models
Set output_names where possible, inspect explanation.output_names, and slice an output before plotting:
class_exp = explanation[..., "class_name"]
# or
class_exp = explanation[..., class_index]
Never average signed attributions across classes. For cross-class comparison, preserve the same model, rows, background, output space, and aggregation.
Cohorts, subgroup analysis, and fairness
SHAP can compare how a model uses features across cohorts, but this is not a fairness test. A protected feature with small SHAP magnitude does not rule out proxy discrimination, and removing a protected feature does not establish fairness. Pair attribution analysis with performance, calibration, error-rate, and domain-appropriate fairness metrics.
See references/workflows.md for cohort construction, model comparison, error analysis, log-loss explanations, monitoring, and production records.
Text and images
Use domain maskers rather than treating tokens or pixels as ordinary independent columns:
shap.maskers.Text(tokenizer)withPartitionExplainerfor token groups;shap.maskers.Image(...)withPartitionExplainerfor image regions;- restrict expensive multi-output models with
outputs=....
Read references/modalities.md for current examples and output-shape guidance.
Troubleshooting Order
- Print Python, SHAP, model-library, NumPy, and framework versions.
- Verify the model receives exactly the same transformed columns, order, dtype, and missing-value representation used during fitting.
- Print
values.shape,base_values.shape,data.shape,feature_names, andoutput_names. - Confirm the selected output and output units.
- Recompute predictions on the same rows in the same order.
- Test a smaller batch and representative background.
- Only then investigate package-specific compatibility or approximation settings.
Use references/troubleshooting.md for additivity failures, shape mismatches, categorical features, pipelines, deep-learning frameworks, plotting, and performance.
Bundled Script
Run a deterministic, self-contained tabular example that writes importance data, metadata, and plots:
uv run --no-project --python 3.12 --with "shap[plots]==0.52.0" \
skills/shap/scripts/tabular_report.py --output-dir /tmp/shap-report
The script does not download data or deserialize models. Read it as a template, then replace the built-in dataset and model while preserving output selection and additivity validation.
Reference Map
| File | Load when |
|---|---|
| references/explainers.md | Selecting or configuring explainers |
| references/data-maskers.md | Choosing background data, masking semantics, or feature groups |
| references/plots.md | Selecting, composing, or saving visualizations |
| references/workflows.md | Running audits, comparisons, cohorts, monitoring, or production workflows |
| references/modalities.md | Explaining text, images, or deep models |
| references/migration.md | Updating legacy SHAP code or supporting older Python |
| references/theory.md | Explaining estimands, guarantees, dependence, interactions, and limitations |
| references/troubleshooting.md | Diagnosing runtime, shape, additivity, and compatibility problems |
Primary Sources
- Documentation: https://shap.readthedocs.io/en/latest/
- API reference: https://shap.readthedocs.io/en/latest/api.html
- Release notes: https://shap.readthedocs.io/en/latest/release_notes.html
- Repository: https://github.com/shap/shap
Related skills
alphagenome: Look up precomputed AlphaGenome Atlas effects for any GRCh38 single-nucleotide variant (AVI score with Phred and 18 SHAP feature attributio…
Trigger Prompts & Evaluation Corpus
These evaluation prompts test that the agent correctly selects shap for relevant tasks and leaves it out for near misses.
Should trigger (3)
- should trigger
“How do I get SHAP values for my XGBoost classifier and make a beeswarm plot?”
expects Uses the modern shap.Explanation API (explainer(X)) with an explicit background, states the explained output (raw margin vs probability), selects the positive-class output if multi-output, checks additivity, and calls shap.plots.beeswarm.
- should trigger
“I'm working on a LightGBM model predicting 30-day readmission from train.parquet (about 40 features, many correlated like creatinine and eGFR). I ran shap.TreeExplainer(model).shap_values(X_test) and got a list of two arrays, then plotting with values[1] fails with a shape error on the newer shap version. Also I get an additivity check error when I switch to model_output='probability'. How do I fix this?”
expects Explains that modern SHAP returns (samples, features, outputs) so one should slice explanation[..., 1] rather than values[1]; notes probability output requires interventional perturbation with background data; advises checking input columns/order/preprocessing and output space before suppressing additivity; mentions correlated features limitation and non-causal interpretation.
- should trigger
“I trained a gradient boosted model on patient lab data and a reviewer asked us to show why the model flagged certain patients as high risk and which inputs drive predictions overall. How should I go about producing that for the paper, including making sure the explanations are trustworthy?”
expects Recommends feature-attribution analysis (SHAP): choose explainer and background population, specify output units, use held-out rows, validate additivity against model outputs, produce global (bar/beeswarm) and local (waterfall) plots, document selection rule, and state that attributions are descriptive and not causal.
Near misses (2)
- near miss
“Tune the hyperparameters of my random forest with Optuna and report cross-validated AUROC; I don't need any interpretation yet.”
expects Handles hyperparameter optimization and cross-validated evaluation without invoking SHAP explanation workflows; the task is predictive model tuning, not explaining predictions.
- near miss
“Can you prove my model is fair across sex and race groups using a causal analysis of why the model discriminates?”
expects Should not rely on SHAP as a fairness or causal test; would use dedicated fairness metrics and causal inference methods (performance, calibration, error rates by group) instead of SHAP attributions.