adversarial-robustness-toolbox
Adversarial Robustness Toolbox
Use ART to implement a bounded experiment against an explicitly authorized model and dataset; ART is an API, not evidence that a model is robust or vulnerable. Pin the installed ART version and check the matching source before relying on API behavior. This guide is grounded in ART 1.20.1, tag commit cbc47a890a191809baa292992761030e7f46ab18.
Workflow and evidence contract
- Define model access, attacker knowledge, eligible inputs, data splits, perturbation or query budget, and success condition before choosing an attack.
- Wrap the actual model with an ART estimator that exposes the interfaces required by the selected attack. Verify input domain, shape, labels, outputs, loss/gradient behavior, and clipping in that wrapper.
- Keep training, tuning, and final evaluation data distinct. Record ART and dependency versions, estimator/backend, attack parameters, seeds, and any query accounting.
- Check generated data against finite-value, shape, domain/clip, and stated perturbation-budget invariants. Report clean performance and attack metrics with explicit numerators and denominators. A result from one model, split, seed, or attack is not a general robustness claim.
For each report, include: (a) art.__version__ and relevant framework versions; (b) model/estimator type and input/output contract; (c) preprocessing and clip domain; (d) threat model, attack/access mode, norm and epsilon units or query/training-data budget; (e) clean baseline, metric definition and denominator, attack parameters, seed, measured perturbation/query counts, and limitations. Separate source-established API facts from behavior observed in the run.
Load the focused reference
- Adapter selection, preprocessing, loss gradients, output scores/logits, clipping, or an ART version change: references/estimator-contract.md.
- White-box evasion with FGM/PGD or a defensible perturbation/query-budget metric: references/evasion-evaluation.md.
- Authorized poisoning, membership-inference/privacy, or model-extraction workflow: references/privacy-poisoning-extraction.md.
- Reproducible CPU FGM/PGD API and invariant smoke test, without downloads or external models: assets/cpu_evasion_smoke.py.