Sparse Methods for Vector Embeddings of TPC Data

AI summary

Overview

• Research area: Machine learning for nuclear physics instrumentation — specifically, sparse convolutional representation learning on Time Projection Chamber (TPC) data.
• Technical level: Intermediate. Familiarity with convolutional neural networks and basic particle-detector concepts helps, but the paper is short and the core ideas (sparse tensors, embeddings, linear probes, PCA) are explained plainly.
• Scope: The paper demonstrates that a sparse ResNet trained with the Minkowski Engine produces event-level vector embeddings of TPC data that are useful both on the detector they were trained on (GADGET II) and on a geometrically and physically different detector (AT-TPC).

What This Paper Is About

TPCs produce massive, sparse, high-dimensional datasets of charged-particle tracks, and analysis pipelines are typically built separately for each detector and each physics task. The authors ask whether a single sparse convolutional encoder can learn general-purpose event representations that transfer across tasks and across very different TPC systems. Their goal is not a state-of-the-art classifier but evidence that such transfer is viable, as a step toward general-purpose TPC foundation models.

Key Contributions

1. A demonstration that raw pad-level TPC signals can be represented as four-dimensional sparse point clouds of the form (x, y, z, q), where convolutions operate on the three spatial dimensions and q is carried as a feature channel, preserving pad-level sparsity and avoiding dense voxelization.
2. An evaluation of a sparse ResNet14 backbone (ten convolutional layers plus one linear layer) that handles variable-length events natively, without the padding or truncation required by common PointNet-style pipelines.
3. A finding that a randomly initialized, entirely untrained sparse ResNet (ResNet_rand) already yields structured embeddings, while pretraining on a simple physics-motivated binary proton–alpha classification task (ResNet_train) improves embedding quality.
4. A cross-detector transfer test in which the same encoder is applied to AT-TPC data and probed on a track-counting task, showing that features learned on GADGET II retain utility on a detector with different geometry and physics goals.

Main Findings

• Untrained networks already encode structure: Even a randomly initialized sparse ResNet14 with no training produces embeddings in which event classes are partially separated in PCA projection, which the authors attribute to the architectural bias of sparse convolutions toward local spatial continuity.
• Pretraining improves in-domain embeddings: On GADGET II, a linear support vector machine probe reached accuracy and F1 of 0.97 with ResNet_train embeddings on a three-class task (800 keV protons, 1600 keV protons, 2 MeV alphas), compared to 0.85 for ResNet_rand. A naive model that always predicts the mode scored 0.33 accuracy and 0.17 F1.
• The probe task is harder than the training task: The three-class separation probed for GADGET II is more challenging than the binary proton–alpha classification used for pretraining, indicating the embeddings capture physically meaningful substructure beyond the training labels.
• Pretraining on one detector helps another: On AT-TPC track counting ({0,1,2} vs. {3} vs. {4,5} tracks), ResNet_train embeddings reached 0.74 accuracy and 0.70 F1, versus 0.58 accuracy and 0.53 F1 for ResNet_rand, and 0.48 accuracy and 0.31 F1 for the naive baseline.
• Training converges quickly and accurately: The binary proton–alpha classifier converged within approximately 15 epochs, achieving about 0.995 accuracy and about 0.99 F1, with the best checkpoint selected by lowest validation loss.
• Deeper is not better here: Deeper variants such as ResNet50 with bottleneck blocks increased training time without clear performance gains, so the shallow ResNet14 design was retained.
• PCA shows cleaner separation after pretraining: ResNet_train projections show cleaner separation of all three particle classes than ResNet_rand, and AT-TPC embeddings from the GADGET-trained encoder are more dispersed in PCA space and separate more successfully than those from the untrained encoder.
• Not reported: The paper does not report training wall-clock times, dataset sizes, parameter counts, or precision and recall values for the probe tasks, although precision, recall and macro-F1 are listed among the training metrics in the optimization settings table.

Methodology in Plain English

The authors treat each TPC event as a sparse cloud of points, where each point has three spatial coordinates and a charge value. Instead of converting events into dense images, they use the Minkowski Engine to run convolutions only where data actually exist, which keeps the computation proportional to the sparse signal rather than to empty space.

The encoder is a shallow residual network (ResNet14) with an initial stem, four residual stages, a pre-pooling block, global max pooling, and a fully connected head. It uses a dropout rate of 0.8 and a stride-3 sparse convolution in the pre-pooling block. Because it operates on sparse point lists, it accepts events of widely varying length without padding.

Two versions of the model are studied. ResNet_train is trained to distinguish protons from alphas on GADGET II data, with labels obtained by gating distinct bands in a range-versus-energy plot and excluding events in the overlap region. Inputs are raw pad-level hits quantized with a voxel size of 0.05, batched using the engine's batched_coordinates function without padding. Training uses cross-entropy loss, Adam with a learning rate of 5×10⁻⁴ and weight decay of 10⁻⁴, cosine annealing with T_max = 13, gradient clipping at max-norm 1.0, a batch size of 64, and up to 15 epochs. ResNet_rand is simply the same architecture with random weights and no training.

To test what the embeddings contain, the authors extract latent vectors from the penultimate layer and train a deliberately low-capacity linear support vector machine probe on tasks the model never saw during training. They also apply principal component analysis to project embeddings onto their first two principal components, making the latent structure visually interpretable.

Why This Matters

The work suggests that detector-specific, task-specific deep learning pipelines may not be necessary for TPC analysis. If an encoder trained on one detector can produce embeddings that remain useful on another, the field gains a path toward shared, reusable models rather than a separate network per experiment.

Potential applications:

• Particle identification and decay-channel separation in low-energy nuclear physics experiments such as GADGET II.
• Track counting and reaction-topology classification in inverse-kinematics experiments such as the AT-TPC.
• Pretrained encoders that reduce the labeled-data burden for new detectors or new physics tasks.
• General-purpose event representations that make downstream classification tasks linearly separable with simple, interpretable probes.

Industry relevance: Sparse convolutional processing of event-like point clouds is directly relevant to any domain producing sparse three-dimensional sensor data — including medical imaging, LiDAR and autonomous navigation, and other scientific imaging pipelines where dense voxelization is computationally wasteful. The finding that untrained sparse architectures already provide usable embeddings also matters for practitioners who lack large labeled datasets.

Future Directions

• Scaling beyond the binary proton–alpha pretext task and the shallow ResNet14 backbone to determine how pretext-task difficulty and model capacity affect embedding quality.
• Expanding transfer tests to additional TPC designs and physics goals, since the current cross-detector test covers only GADGET II and the AT-TPC.
• Building toward general-purpose TPC foundation models, which the authors explicitly name as the motivation for this line of work.
• Quantifying embedding quality more rigorously than the paper does, for example through larger probe studies and additional reported metrics, given that precision and recall are listed as training metrics but not reported for the probe tasks.

Target Audience

This paper is most useful to machine learning researchers working on scientific and physics applications, nuclear and particle physicists interested in adopting deep learning for detector data, and practitioners exploring sparse convolutional networks for point-cloud-like data. It is also relevant to readers interested in foundation models for scientific instruments and in transfer learning across heterogeneous data domains.

Authors’ abstract

Time Projection Chambers (TPCs) are versatile detectors that reconstruct charged-particle tracks in an ionizing medium, enabling sensitive measurements across a wide range of nuclear physics experiments. We explore sparse convolutional networks for representation learning on TPC data, finding that a sparse ResNet architecture, even with randomly set weights, provides useful structured vector embeddings of events. Pre-training this architecture on a simple physics-motivated binary classification task further improves the embedding quality. Using data from the GAseous Detector with GErmanium Tagging (GADGET) II TPC, a detector optimized for measuring low-energy $β$-delayed particle decays, we represent raw pad-level signals as sparse tensors, train Minkowski Engine ResNet models, and probe the resulting event-level embeddings which reveal rich event structure. As a cross-detector test, we embed data from the Active-Target TPC (AT-TPC) -- a detector designed for nuclear reaction studies in inverse kinematics -- using the same encoder. We find that even an untrained sparse ResNet model provides useful embeddings of AT-TPC data, and we observe improvements when the model is trained on GADGET data. Together, these results highlight the potential of sparse convolutional techniques as a general tool for representation learning in diverse TPC experiments.