The Pulse
Researchers Extract Trained Neural Networks From Hard Labels
A new study demonstrates end-to-end extraction of trained ReLU networks using only output labels from black-box queries. The method reaches at least 98.477% label agreement on MNIST and Fashion-MNIST test models.
AI.info Team ·
Researchers have demonstrated a fully black-box attack that reconstructs trained neural networks while seeing only the final class label returned for each query. The method extracted four- and six-hidden-layer ReLU networks trained on MNIST and Fashion-MNIST, producing between 98.477% and 100% agreement with the target models on 100,000 evaluation inputs.
Akira Ito, Takayuki Miura and Yosuke Todo describe the technique in a paper submitted to arXiv on September 18, 2026. Their work addresses the hardest version of neural-model extraction: the attacker receives labels such as “dog” or “cat,” but not confidence scores, logits or hidden activations.
Hard labels reveal more than they appear to
Model extraction attacks seek to reproduce a deployed neural network from access to its predictions. Providers generally treat trained parameters as proprietary because producing them requires substantial data, computation and engineering. An extracted model can expose the behavior of a service without giving an attacker direct access to its files.
The researchers focus on fully connected multilayer perceptrons with ReLU hidden layers and an affine output layer. Their attack assumes that the adversary knows the network architecture, can submit arbitrary inputs and can obtain sufficiently precise predictions from the target system. Only the winning class label is used during extraction.
ReLU networks divide input space into regions in which the network behaves like an affine function. The boundaries between those regions change when individual neurons switch between active and inactive states. The attack searches for locations where a neuron’s activation boundary intersects the boundary separating two output classes. Those intersections expose geometric information about the hidden layer.
The cosine method removes the hardest bottleneck
Earlier hard-label extraction methods used a boundary-walking procedure to resolve an ambiguity in the direction of recovered neuron weights. That process required repeated queries, detection of changes in the decision boundary and additional numerical estimation. The paper says a practical black-box implementation was difficult because errors could look like genuine changes in the network.
Ito, Miura and Todo replace that procedure with a family of offline calculations called the cosine method. The method compares cosine similarities between a recovered neuron signature and the normal vectors of nearby decision boundaries. It reuses intersection spaces already collected for signature recovery, so sign recovery requires no dedicated queries.
Experiments on an MNIST-trained model with five hidden layers of width 128 recovered 506 of 510 tested neuron signs using 50 intersection spaces per neuron, excluding persistent and dead neurons. The authors also introduce weighted versions of the calculation to reduce distortion caused by the network’s local linear mapping and by directional bias in trained models.
Four small networks give the end-to-end test
The full attack targets networks with 784 inputs, 10 outputs and hidden layers containing 16 neurons each. The models use four or six hidden layers, are trained for 30 epochs with the Adam optimizer and classify either MNIST or Fashion-MNIST images.
For the four-hidden-layer models, the researchers collect 20,000 intersection spaces per model. The six-hidden-layer experiments use 400,000 intersection spaces per model. Collection dominates the query budget; the sign-recovery stage reuses those spaces and adds no queries of its own.
The extracted networks reproduce the target classifications with high agreement under standard Gaussian inputs. The four-layer MNIST model reaches 98.477%, while the six-layer MNIST model reaches 99.948%. The corresponding Fashion-MNIST models reach 100.000% and 99.744%.
The paper reports that every neuron was recovered except dead and almost-dead neurons. After matching and normalizing recovered hidden-layer signatures, the largest reported L2 distance from the corresponding true weight vector was 9.34 × 10-6.
The attack still depends on a large query budget
The results do not show that any deployed neural service can be copied cheaply. The experiments use small, fully connected ReLU networks and grant the attacker unrestricted access to arbitrary inputs. Modern production systems may use different architectures, input restrictions, rate limits, monitoring or randomized behavior.
Query cost also remains a major limitation. The authors say their method reduces the extra queries needed for sign and signature recovery, but gathering enough intersection spaces still requires a substantial number of oracle calls. Their six-layer experiments illustrate the scale of the problem: each model requires 400,000 collected intersection spaces before layer recovery begins.
For model providers, the finding narrows the protection offered by hiding confidence scores. A service that returns only the top label exposes less information than one that returns logits or probabilities, but the paper shows that label-only access can still support parameter recovery under the stated conditions.
A controlled result with a direct security implication
The study is a proof of feasibility rather than an attack on a named commercial AI service. Its target models are trained benchmarks, not proprietary systems, and the researchers do not claim that the method extracts large language models or other architectures outside their experimental setting.
The result matters because it closes a gap in prior work. Polynomial-time hard-label extraction had been proposed, but a complete demonstration on trained deep ReLU networks remained difficult because of sign recovery and missing or misleading geometric information. The new pipeline combines sign recovery with procedures for rejecting spurious signatures, recovering missing coordinates and handling deeper layers.
The authors identify query reduction as the next technical problem. Their experiments show that the recovered networks can closely match the target models, but they also show where the attack remains expensive: the system must first map enough of the target network’s decision geometry to reconstruct its hidden structure.