Conditional Clifford-Steerable CNNs for PDE Modeling

AI summary

Overview

Research area: Geometric deep learning and scientific machine learning — specifically equivariant neural architectures for solving partial differential equations (PDEs).

Technical level: Advanced. The paper assumes familiarity with group equivariance, representation theory, steerable convolutions, and Clifford algebras. The high-level motivation, however, is accessible without deep mathematical background.

Scope: The paper diagnoses an expressivity flaw in Clifford-Steerable CNNs (CSCNNs) and proposes an input-conditioned kernel formulation that restores missing degrees of freedom while preserving exact $\mathrm{E}(p,q)$ equivariance.

What This Paper Is About

Physical systems obey symmetries (rotations, reflections, Lorentz transformations), and neural networks that respect these symmetries generalize better from limited data. Clifford-Steerable CNNs were designed for this purpose, but their convolutional kernel basis is provably incomplete — some allowed interactions are missing — which caps their expressive power. This paper shows that making the kernel depend on the input field itself (conditioning) fills in those missing basis elements, yielding a more expressive yet still exactly equivariant model.

Key Contributions

1. Conditional Clifford-Steerable CNNs (C-CSCNNs): A framework that augments $G$-steerable kernels with auxiliary variables derived from the input feature field, addressing the incomplete kernel basis of standard CSCNNs.
2. A derivation of the conditional steerability constraint: The authors formally state the equivariance condition that input-dependent kernels must satisfy, and show how to solve it efficiently through implicit parameterization rather than closed-form analytical kernels.
3. An efficient implementation via template matching: By conditioning on a translation-invariant global summary (mean pooling) of the input field, the method retains the parameter-sharing and parallelization benefits of standard convolution.
4. Empirical validation across four PDE benchmarks: The approach is tested on Navier-Stokes, shallow-water, 3D Maxwell, and relativistic Maxwell equations, with consistent gains over CSCNNs and competitive results against state-of-the-art solvers.

Main Findings

• Standard CSCNNs are provably over-constrained: For the $\mathrm{O}(2)$ case, the vector-vector kernel interaction collapses to a scalar with no angular dependence. The analytical solution requires both frequency-0 and frequency-2 components; the frequency-2 part is unreachable by the original implicit kernel network.
• Conditioning restores the missing frequencies: Interacting the relative position vector with an auxiliary multivector derived from the input allows angular information to propagate into all grades, including grade-0 — enabling the kernel to reproduce the full analytical basis.
• Large empirical gains on PDE forecasting: Reported improvements over baseline CSCNNs: ≈53.7% on Navier-Stokes, ≈54.1% on shallow-water, and ≈33.9% on Maxwell, with the advantage widening as more training data is added.
• State-of-the-art-competitive results: C-CSCNNs achieve the lowest MSE on fluid dynamics tasks (NS, SWE-1) across all dataset sizes and second-best on electrodynamics tasks (MW3, MW2), trailing only the Transformer-based Transolver.
• Strong parameter efficiency in scaling tests: On the 5-step shallow-water benchmark, a 10M-parameter C-CSCNN reaches 3.51% relative $L^2$ error, beating a 13M CViT-S (4.47%), and a 55M variant reaches 2.94%, outperforming FNO at 268M and UNO at 440M parameters.
• Equivariance is preserved: The conditional convolution shows a relative equivariance error of $3.4 \times 10^{-7}$, essentially the same order as the original CSCNN's $2.4 \times 10^{-7}$, confirming the theoretical proof.
• The completeness claim is a conjecture, not a theorem: The authors state that a full proof would require knowing all irreducible representations of $\mathrm{O}(p,q)$ and mapping them onto Clifford algebra grades — a task they leave open.

Methodology in Plain English

The starting point is a standard convolutional layer, where the kernel is a function of the spatial offset between points. In steerable CNNs, that kernel is chosen so the layer commutes with a symmetry group. The trick used by CSCNNs is to learn that kernel implicitly with a small equivariant neural network, avoiding hand-derived analytical formulas.

The problem: because that small network only takes the offset vector as input, it cannot express every allowed kernel — certain angular patterns are mathematically unreachable.

The fix: also feed it a summary of the input feature field itself. Instead of the kernel depending only on the offset, it depends on the offset plus a translation-invariant global summary (in practice, the mean of the input field). This is like letting each convolution filter "know something about" the broader field it is operating on, rather than treating every location identically.

The authors prove this conditioning is still consistent with equivariance as long as the summary operator is itself equivariant, and they show that computing the summary is computationally cheap — mean pooling is negligible next to the convolution itself. The model is a ResNet backbone with these conditional Clifford-steerable layers substituted in, implemented in JAX/Flax on four PDE datasets.

Why This Matters

Impact on research: The paper identifies a structural limitation in a widely used equivariant architecture and offers a principled repair. It demonstrates that implicit kernel parameterization — previously treated as complementary to analytical construction — can, when conditioned appropriately, recover the completeness of analytical kernels without deriving them. This connects equivariant deep learning with attention- and message-passing-style conditioning, and provides a template for extending CSCNNs to arbitrary pseudo-Euclidean groups.

Real-world applications:

• Computational fluid dynamics surrogates — fast approximate solvers for turbulent flow, replacing expensive numerical simulation in design loops.
• Weather and climate modeling — shallow-water equations are a standard testbed for atmospheric dynamics; parameter-efficient models matter where compute budgets are tight.
• Electromagnetics and plasma physics — Maxwell's equations, including the relativistic variant relevant to fusion and astrophysics.
• Relativistic and spacetime-aware simulation — the $\mathbb{R}^{1,2}$ formulation is applicable to Lorentz-covariant field problems.

Industry relevance: Physics-informed surrogates directly reduce simulation cost in aerospace, automotive, energy, and semiconductor design, where PDE solvers dominate engineering workflows. The demonstrated parameter efficiency (a 10M model beating a 13M Transformer variant, a 55M model competing with 268M–440M parameter operators) is a practical argument for deployment in compute-constrained settings.

Future Directions

• Prove the completeness conjecture. The authors defer a formal result; it would require characterizing the irreducible representations of $\mathrm{O}(p,q)$ and constructing isomorphisms to Clifford multivector grades — a nontrivial representation-theoretic problem.
• Design richer conditioning operators. Mean pooling is a deliberately simple choice. More expressive equivariant operators (attention-based, non-local, or multi-scale summaries) could recover the performance gap on electrodynamics tasks, where attention-based baselines currently lead.
• Improve scaling. The paper notes that the scalable variant already competes with far larger models, suggesting that more expressive conditioning could unlock further gains at higher parameter counts.
• Extend to broader PDE families and geometries. The framework works for general pseudo-Euclidean groups, but empirical validation is limited to four tasks in two or three spatial dimensions; generalization to higher dimensions, irregular domains, or coupled multi-physics problems remains open.

Target Audience

This paper is aimed at machine learning researchers working on equivariant and geometric deep learning, scientific ML practitioners building neural PDE solvers, and computational scientists interested in symmetry-aware surrogate models. A reader needs a solid grasp of group theory and representation theory to follow the derivations, though the architectural contribution and empirical results are legible to anyone familiar with convolutional networks and PDE modeling. Those primarily interested in applying neural surrogates to engineering problems will find the data-efficiency and parameter-efficiency results the most immediately actionable.

Authors’ abstract

We introduce Conditional Clifford-Steerable CNNs (C-CSCNNs), a unified framework that incorporates equivariance to arbitrary pseudo-Euclidean groups and significantly improves the expressivity of standard CSCNNs. We show that the kernel basis of the standard formulation is incomplete, limiting model capacity. To address this, we augment the kernels with equivariant representations of the input feature field. We derive the equivariance constraint for these input-dependent kernels and show how it can be solved efficiently via implicit parameterization. We empirically validate on multiple PDE forecasting tasks, including fluid dynamics and relativistic electrodynamics, where our method consistently outperforms standard CSCNNs and performs on par with state-of-the-art baselines.