MiniFool -- Physics-Constraint-Aware Minimizer-Based Adversarial Attacks in Deep Neural Networks

AI summary

Overview

Research area: Machine learning security and robustness, applied to particle and astroparticle physics (adversarial attacks on deep neural network classifiers).

Technical level: Intermediate. The core idea is intuitive, but the paper assumes familiarity with softmax classifiers, chi-squared test statistics, and physics event reconstruction.

Scope: The paper introduces MiniFool, a minimizer-based adversarial attack that constrains perturbations by experimental uncertainties, and demonstrates it on three classification tasks: MNIST digits, IceCube tau-neutrino identification, and CMS b-jet tagging.

What This Paper Is About

Standard adversarial attacks search for the smallest possible change to an input that flips a neural network's classification. In physics applications this is a problem: the resulting perturbed input is often unphysical (negative energy deposits, implausible sensor readings, violations of conservation laws), so the attack success rate cannot be interpreted statistically. MiniFool instead searches for the most plausible perturbation — one weighted by the actual measurement uncertainties of the detector — so that a successful attack means the classification genuinely could have gone the other way within experimental error.

Key Contributions

1. A physics-constraint-aware attack cost function. MiniFool minimizes a combined metric consisting of a chi-squared-like term (mean squared input deviation, divided by each feature's uncertainty) plus a target-score term that rewards changing the network output. Unlike FGSM, PGD, or DeepFool, perturbations that are statistically implausible are automatically penalized rather than merely clipped.

2. The "attack parameter" as a robustness dial. A single scalar s scales all nominal uncertainties. Sweeping s produces a per-event robustness curve: events that flip at s <= 1 are fragile, events that only flip at s >> 1 are robust. This yields a per-event robustness score, not just an aggregate attack success rate.

3. Three cross-domain demonstrations. The algorithm is validated on MNIST (image recognition), IceCube (7 reported tau-neutrino candidates plus simulated events), and CMS Open Data (b-jet tagging with DeepJet), showing the method is not tied to one detector or data type.

4. Open-source implementations. Published under the AISafety project as both TensorFlow and PyTorch code, with an included MNIST example.

Main Findings

• Misclassified events are systematically less robust. Across MNIST, IceCube, and CMS, events whose labels the network got wrong require much smaller perturbations to flip than events classified correctly. On MNIST, roughly 80% of initially misclassified digits received the correct label after a MiniFool attack with s = 0.2.

• IceCube tau-neutrino candidates mostly resist the attack. Of the seven tau-neutrino candidates reported by IceCube in 2024, only one could be flipped at nominal uncertainties (s = 1). The other six required s ≈ 10, corresponding to roughly 100% uncertainty on the recorded amplitudes — experimentally excluded. This independently corroborates the background estimate of 0.5 events in that analysis.

• Simulation reproduces the pattern. Simulated misclassified electron-neutrino events were attackable at s ≲ 1, while correctly identified tau neutrinos needed s >> 1, matching the behavior of the real candidate events.

• CMS b-jet tagging degrades non-linearly. A nominally trained DeepJet model achieves AUC 0.932. MiniFool perturbations begin to degrade performance around s = 3 × 10⁻⁴ (AUC 0.887), degrade steeply thereafter, and converge to AUC 0.752 in the artificially large-uncertainty limit s = 1. The takeaway is that DeepJet's decisions are largely insensitive to perturbations below roughly 0.02% of the normalized input scale.

• Confidence alone does not predict robustness. In the CMS study, 200 jets were selected with at least 87% classification confidence and split evenly between correct and incorrect labels. Despite equally high confidence, the incorrectly labeled jets still flipped at lower attack parameters — confidence is not a proxy for robustness.

• Computational cost is a real constraint. MiniFool uses the Adam optimizer (default learning rate 1e-5) and individual attacks can take seconds on a workstation, making it markedly slower than single-step methods like FGSM.

Methodology in Plain English

Ordinary adversarial attacks ask: how little do I have to change this input to change the answer? MiniFool asks: how little do I have to change this input to change the answer, given how precisely each input value is actually measured?

For each event, the algorithm defines a cost with two parts. The first part measures how far the modified input has drifted from the original, with each pixel or feature divided by its own uncertainty — essentially a chi-squared goodness-of-fit value. If a feature is measured very precisely, moving it is expensive; if it is noisy, moving it is cheap. The second part measures how far the network's output on the modified input is from a desired target score (typically zero for the originally predicted class). A gradient-based minimizer then adjusts the input to reduce the combined cost.

Because the target-score term always pulls the classifier away from its original answer while the chi-squared term always pulls the input back toward what was actually measured, the minimizer settles at a compromise. If the compromise reaches a changed classification cheaply, the attack succeeds; if the required drift is too large relative to the measurement precision, the optimiser leaves the classification alone.

The attack parameter s simply multiplies all uncertainties. At s = 1, uncertainties are the true experimental values. At s = 0.1, the attacker is pretending the detector is ten times more precise than it is. By scanning s upward and recording where each event flips, the researchers obtain a per-event robustness curve that can be compared between correctly and incorrectly classified samples, and between real data and simulation.

Why This Matters

Impact on research: Adversarial robustness studies in machine learning usually report an aggregate attack success rate. That number is hard to translate into physics because an unphysical perturbation has no meaningful interpretation. MiniFool makes the attack success rate a statement about statistical plausibility, which lets physicists use it as a cross-check on analysis results — for example, verifying that a set of candidate signal events is not an artifact of an overconfident network.

Real-world applications:

• Neutrino astronomy. IceCube's tau-neutrino identification is a flagship result; MiniFool provides an independent robustness check on the seven reported candidates without requiring new data.
• LHC event classification. b-tagging underpins most LHC measurements and searches. Quantifying the stability of taggers under realistic detector uncertainty informs systematic error budgets.
• Detector simulation and calibration. Identifying which input features are cheapest to perturb highlights where mismodeling would most damage a classifier, guiding where simulation effort should be invested.
• Safety-critical ML generally. The core idea — constrain adversarial perturbations by measurement or sensor uncertainty rather than an arbitrary norm ball — transfers to any domain where inputs have known error bars, such as medical imaging or sensor fusion.

Industry relevance: Adversarial robustness is a central concern for deployed ML systems. Norm-ball attacks (L-infinity, L2) are convenient but arbitrary; they do not reflect how much a real sensor can drift. MiniFool's formulation gives a principled alternative that maps attacks onto domain-specific tolerances, and its per-sample robustness scores are directly usable for triage — flagging the inputs on which a model is most fragile.

Future Directions

• Realistic uncertainty and correlation models. The paper's uncertainty treatment is deliberately simple (a single global scale factor, or a fixed percentage of signal amplitude). The authors explicitly call for covariance matrices that capture correlated features, which would allow s to be interpreted physically rather than purely as a stress-test knob.

• Adversarial training with MiniFool. Using MiniFool-generated examples during training could improve robustness, but the current numerical minimizer is too slow for large datasets. Faster minimizers or amortized attack generation would be required.

• Extension to regression. The target-score term is defined for classification outputs. Regression tasks — energy reconstruction, direction estimation — would need a different significance criterion for "the answer changed meaningfully."

• Application to systematic uncertainty estimation. The authors suggest the attack parameter scan could serve as an independent verification of uncertainties that feed into classification-based selections, potentially improving data purity.

Target Audience

Physicists and machine learning researchers working on robustness in scientific applications. Particle and astroparticle physicists who use neural network classifiers and need to quantify systematic uncertainties will find the IceCube and CMS case studies directly relevant. ML safety researchers interested in alternatives to norm-ball adversarial attacks — particularly attacks constrained by semantically meaningful error models — will find the cost-function formulation the most transferable element. Readers need working familiarity with softmax classifiers and basic statistics; no deep physics background is assumed for the MNIST section, which serves as the gentlest entry point.

Authors’ abstract

In this paper, we present a new algorithm, MiniFool, that implements physics-inspired adversarial attacks for testing neural network-based classification tasks in particle and astroparticle physics. While we initially developed the algorithm for the search for astrophysical tau neutrinos with the IceCube Neutrino Observatory, we apply it to further data from other science domains, thus demonstrating its general applicability. Here, we apply the algorithm to the well-known MNIST data set and furthermore, to Open Data data from the CMS experiment at the Large Hadron Collider. The algorithm is based on minimizing a cost function that combines a $χ^2$ based test-statistic with the deviation from the desired target score. The test statistic quantifies the probability of the perturbations applied to the data based on the experimental uncertainties. For our studied use cases, we find that the likelihood of a flipped classification differs for both the initially correctly and incorrectly classified events. When testing changes of the classifications as a function of an attack parameter that scales the experimental uncertainties, the robustness of the network decision can be quantified. Furthermore, this allows testing the robustness of the classification of unlabeled experimental data.