CORDS: Continuous Representations of Discrete Structures

AI summary

Authors: Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent Affiliations: UvA-Bosch Delta Lab and University of Amsterdam; AMLab, University of Amsterdam; Tübingen AI Center, Eberhard Karls Universität Tübingen (Michael Tiemann notes part of the work was carried out while still with the Bosch Center for Artificial Intelligence, Renningen, Germany) arXiv: 2601.21583v2 [cs.LG], 17 Sep 2026 · License CC BY 4.0 · Category: Machine Learning

Overview

• Research area: Machine learning for variable-cardinality set prediction, spanning generative molecular modeling, object detection, and simulation-based inference.
• Technical level: Intermediate. The core idea is geometric and visually intuitive, but the invertibility arguments rely on kernel superposition, Gram matrices, and positive-definiteness.
• Scope (1 sentence): The paper proposes CORDS, a bijective mapping between finite sets of positioned objects with attributes and a pair of continuous fields (a density field and a feature field), and evaluates it on molecules, images, light curves, and a synthetic maxima benchmark.

What This Paper Is About

Many learning problems require predicting a set of objects when the number of objects is not known in advance, such as object detection, molecular modeling, and astrophysical source detection. Existing methods typically pre-allocate capacity (fixed slots, padding, truncation) or must infer the set size explicitly, which the authors describe as a recurring source of difficulty. CORDS addresses this by casting variable-sized set prediction as a continuous inference problem: a set of spatial objects is mapped invertibly into a density field encoding positions and count plus a feature field carrying attributes over the same support, so models can learn entirely in field space yet decode back to discrete sets exactly.

Key Contributions

1. A bijective encoding of discrete sets as continuous fields. Objects at positions $\mathbf{r}_i$ with features $\mathbf{x}_i$ are encoded by superimposing a fixed positive kernel $K$ with location-independent mass $\alpha$: $\rho(\mathbf{r}) = \frac{1}{\alpha}\sum_i K(\mathbf{r}, \mathbf{r}_i)$ for density and $\mathbf{h}(\mathbf{r}) = \frac{1}{\alpha}\sum_i \mathbf{x}_i K(\mathbf{r}, \mathbf{r}_i)$ for features.
2. A constructive, three-step decoding procedure. Cardinality is read as total density mass, $N = \int_\Omega \rho(\mathbf{r}),d\mathbf{r}$; positions are recovered by solving a kernel-matching least-squares problem; features are recovered by a linear projection using the Gram matrix $G_{ij} = \int_\Omega \kappa_i(\mathbf{r})\kappa_j(\mathbf{r}),d\mathbf{r}$, giving $\mathbf{X} = \alpha G^{-1} B$ with $B_{i:} = \int_\Omega \mathbf{h}(\mathbf{r})\kappa_i(\mathbf{r}),d\mathbf{r}$.
3. Formal correctness results. Propositions and lemmas in Appendix A establish cardinality recovery, symmetric positive-definiteness of the Gram matrix for distinct centers, exact feature inversion, and exact position recovery (with optimal value 0) under stated identifiability assumptions (referred to as A2 and A3).
4. A unified empirical evaluation across four domains — molecular generation and regression, object detection, simulation-based inference for fast radio bursts, and recovery of local maxima — using the same encoding, decoding, and training objectives, plus an appendix (B) sketching an extension to graphs via a node/edge density and feature "Field of Graph" quadruple $(\rho_{\text{n}}, \rho_{\text{e}}, \mathbf{h}{\text{n}}, \mathbf{h}{\text{e}})$.

Main Findings

• Molecular generation on QM9 (standard RDKit evaluation, Table 1). CORDS reports atom stability 97.9%, molecule stability 82.3%, validity 91.0%, uniqueness 97.1%. For reference, the table lists GeoLDM at 98.9 / 89.4 / 93.8 / 98.8 and Rapidash at 99.4 / 92.9 / 98.1 / 97.2. The paper states CORDS reaches the overall performance range of E(3)-equivariant GNNs despite using a non-equivariant, domain-agnostic backbone.
• Molecular generation on QM9 (OpenBabel postprocessing, Figure 3 table). CORDS records atom 99.2, molecule 93.8, validity 98.7, uniqueness 97.1, compared with VoxMol at 99.2 / 89.3 / 98.7 / 92.1 and FuncMol at 99.0 / 89.2 / 100.0 / 92.8 (the "data" row is 99.8 / 98.7 / 98.9 / 99.9). The paper states CORDS outperforms prior continuous approaches such as VoxMol on QM9.
• GeomDrugs unconditional generation. CORDS reports atom stability 78.4% and validity 94.6%. Baseline entries in the same table include GeoLDM at 84.4 / 99.3 and EDM at 81.3 / 92.6. The paper emphasizes that non-categorical features such as charges can be represented directly in field space and decoded back without post-processing heuristics.
• Conditional generation generalizes past held-out conditioning ranges. CORDS conditions directly on continuous polarizability $\alpha$ without discretizing either the property or $N$; when a range of $c$ is removed during training and used at inference, the model still induces a coherent conditional atom-count distribution $p(N|c)$ (Figure 3, left).
• Object detection under out-of-distribution counts (MultiMNIST, Table 2). CORDS shows the smallest relative drop under OOD counts: AP 16.4% drop (76.8 in-distribution to 64.2 OOD), AP50 11.9% drop (81.5 to 71.8), AP75 21.0% drop (68.0 to 53.7). DETR drops 19.5% / 14.6% / 25.8% and YOLO 24.5% / 18.5% / 28.0%. All methods are described as competitive in-distribution; all networks were allocated 8 million parameters.
• Simulation-based inference sidesteps explicit cardinality modeling. For FRB light curves, the posterior over the number of components $p(N \mid \ell)$ arises from the learned field distribution rather than being modeled explicitly; Figure 4 shows posterior light-curve reconstructions and the induced count posterior.
• Decoding is exact in principle, approximate in practice. Exact recovery holds when the field genuinely originates from the forward transform; in practice gradient-based optimization (with L-BFGS refinement available for higher accuracy) is described as sufficient.

Methodology in Plain English

The authors replace the question "how many objects are there, and what are they?" with "what does a smooth field look like?" Each object is turned into a small "bump" — a Gaussian kernel $K(\mathbf{r}, \mathbf{r}_i) = \exp(-|\mathbf{r}-\mathbf{r}_i|^2 / 2\sigma^2)$ — placed at its position. Adding up all the bumps gives a density field, and adding up the same bumps weighted by each object's attributes gives a feature field. Because every bump has the same total mass, the density field's total integral is exactly the number of objects.

Decoding runs in reverse in three stages: integrate the density to get the count; fit kernel centers to the density shape to get positions; and project the feature field onto those recovered kernels using a Gram matrix to get attributes back. Models then never need fixed slots or padding — they see sampled field values instead. For images and time series the authors sample on uniform grids and use standard 2D and 1D CNNs; for 3D molecules, where a uniform grid would be wasteful, they importance-sample points in proportion to the density and feed the resulting unordered point sets to the Erwin architecture, a hierarchical permutation-invariant transformer. Training for detection minimizes a pixel-wise MSE on both fields plus a count penalty $\mathcal{L} = \mathcal{L}{\text{MSE}} + \lambda(\hat{N} - N)^2$, where $\hat{N} = \int \rho(x,y),dx,dy$, making the object count a differentiable quantity. For FRBs, they discretize light curves on a grid of $K = 1000$ points, evaluate $(\rho, \mathbf{h})$ there, append the observed counts $\ell(t)$, and train a flow-matching model to approximate $p(\rho(t), \mathbf{h}(t) \mid \ell)$; at inference, sampled fields are decoded into component sets with parameters $\theta = (t_0, A, \tau{\text{rise}}, \text{skew})$.

Why This Matters

• Research impact. CORDS reframes variable cardinality as a representational choice rather than a modeling workaround, offering a counterpart to padded representations, truncations, and explicit size inference. It connects to kernel mean embeddings while using the same kernel-superposition principle constructively, with an inverse map. The authors position it as a single representation that carries across pixel grids, 3D space, time series, and abstract domains.
• Real-world applications.
  • Molecular design and drug discovery: conditional generation where a target property does not uniquely determine the number of atoms, including direct handling of continuous attributes such as partial charges.
  • Astronomical and physics inference: decomposing noisy photon-count light curves into a variable number of burst components, and reconstructing catalogs of astrophysical sources.
  • Object detection and counting in imagery: scenes with unknown or unusually dense object counts, where fixed-slot detectors must implicitly cap their predictions.
  • Mathematical and scientific structure recovery: finding local maxima of functions represented as continuous fields.
• Industry relevance. The approach replaces slot-budget engineering with a representation whose capacity scales with scene density, which is attractive for detection pipelines with unpredictable object counts. The paper also flags a practical cost: high-fidelity molecular reconstruction benefits from dense sampling (approximately $10^3$ points per molecule), and refinements such as L-BFGS add latency.

Future Directions

• Evaluate CORDS on larger-scale detection benchmarks such as COCO to test robustness under heavy occlusion, class diversity, and crowding, and explore learned, spatially adaptive kernels to separate nearby instances.
• Extend molecular conditional generation to pocket-conditioned ligand design, regional inpainting, and multi-property control, to probe the benefits of continuous attributes such as charges, spins, and partial occupancies.
• Pursue hybrid settings combining discrete and continuous modalities — for example molecules or materials described as discrete atomic graphs alongside continuous DFT-derived electron densities (Elsborg et al., 2026).
• Address the stated limitations: sampling cost at scale, the speed–accuracy trade-off of kernel-center refinement, and the difficulty of separating overlapping kernels. The paper points to Appendices C.1 and C.2 for further discussion of approximate decoding and limitations.

Target Audience

Researchers and practitioners working on generative modeling or prediction of variable-sized sets, particularly those in molecular machine learning, object detection, and simulation-based inference. Readers who benefit most will be comfortable with kernel methods, continuous/field representations, and diffusion or flow-matching models; the mathematical development in Appendix A assumes familiarity with Hilbert-space arguments and positive-definite kernels.

Authors’ abstract

Many learning problems require predicting sets of objects when the number of objects is not known beforehand. Examples include object detection, molecular modeling, and scientific inference tasks such as astrophysical source detection. Existing methods often rely on padded representations or must explicitly infer the set size, which often poses challenges. We present a novel strategy for addressing this challenge by casting prediction of variable-sized sets as a continuous inference problem. Our approach, CORDS (Continuous Representations of Discrete Structures), provides an invertible mapping that transforms a set of spatial objects into continuous fields: a density field that encodes object locations and count, and a feature field that carries their attributes over the same support. Because the mapping is invertible, models operate entirely in field space while remaining exactly decodable to discrete sets. We evaluate CORDS across molecular generation and regression, object detection, simulation-based inference, and a mathematical task involving recovery of local maxima, demonstrating robust handling of unknown set sizes with competitive accuracy.