Drift Estimation for Diffusion Processes Using Neural Networks Based on Discretely Observed Independent Paths

AI summary

Overview

• Research area: Nonparametric statistics for stochastic processes, specifically drift estimation for time-homogeneous diffusion processes, combined with statistical learning theory for neural networks (stat.ML).
• Technical level: Advanced. The paper is centered on a non-asymptotic risk bound, covering-number arguments, and approximation theory for ReLU networks, and assumes familiarity with stochastic differential equations and empirical process theory.
• Scope in one sentence: The paper proposes a ReLU neural network estimator for the drift of a diffusion observed at high frequency along N independent paths over a fixed time horizon, proves a non-asymptotic L² risk bound, and benchmarks the estimator against a B-spline method.

What This Paper Is About

The authors study how to recover the unknown drift function b of a diffusion process dX_t = b(X_t) dt + σ(X_t) dB_t from data consisting of N independent trajectories, each recorded at a fine discrete grid of time points with step Δ = T/M over a fixed horizon T. The goal is to estimate one component of the drift restricted to the compact domain [0,1]^d, without assuming the process is ergodic, and to quantify how the estimation error depends on the number of trajectories N, the time step Δ, and the size of the neural network. A central difficulty the authors highlight is that the classical regression targets built from increments, Y = (1/Δ)(X_{t_{m+1}} − X_{t_m}), have errors that are neither normally distributed nor independent of the drift value at the observation point.

Key Contributions

1. A neural network drift estimator with a non-asymptotic risk bound (Theorem 2.1). The estimation risk is bounded by four terms: the training error Ψ^F(f̂) (the gap to the empirical risk minimizer), the approximation error inf over the network class, a term involving the network size and N, and a term proportional to Δ. The authors stress that the bound does not require ergodicity, unlike the long-time-horizon neural network approach of Oga and Koike (2024).
2. An explicit rate for compositional drift functions (Corollary 2.3). When f₀ belongs to the composition-structured function space 𝒢(q, d, t, β, K) of Schmidt-Hieber (2020), the risk is bounded by C φ_N log³ N, where φ_N is defined in equation (2.8) as a maximum over composition layers of N^{-2β_i*/(2β_i*+t_i)}, under conditions on network depth, width, sparsity, and the time step Δ.
3. An empirical study showing dimension-independent convergence (Section 3). Using a drift with local oscillations of double-layer compositional form, the empirical convergence rate is reported as closely matching the theoretical bound for d ∈ {1, 2, 50}, and the authors state the empirical rate becomes independent of the input dimension d.
4. A favorable comparison against a B-spline estimator from Denis et al. (2021). The neural network estimator shows a better convergence rate in N and recovers local fluctuations of the drift that the B-spline method misses, with a scalability argument based on memory requirements of tensor-product B-spline bases.

Main Findings

• Risk decomposes into three interpretable pieces. Remark 2.2 states the Theorem 2.1 bound splits into a training error from optimizing the empirical loss 𝒬_{𝒟_N}, an approximation/regression error that is independent of the diffusion setting (bounded as in Schmidt-Hieber (2020)), and a term capturing temporal dependence in the diffusion.
• Explicit bound in Theorem 2.1. For constants depending on C_b, C_σ, L_b, L_σ, T and a universal constant C, the risk is at most 4Ψ^F(f̂) + 6 inf_f ℛ(f, f₀) + ℭ F²(Δ + [s(L log s + log d) + s log 4F]/N + s log N / N).
• Compositional rate in Corollary 2.3. With L ≍ log₂ N, N φ_N ≲ min_i p_i, s ≍ N φ_N log N, Δ ≲ φ_N log³ N and Ψ^F(f̂) ≤ C φ_N log³ N, the risk satisfies ℛ(f̂, f₀) ≤ C φ_N log³ N.
• The diffusion-induced term scales as log N / N. This matches the "diffusion-related term scaling as log N/N" described in the abstract.
• Empirical convergence confirms theory for most dimensions. The log–log plot of estimation risk against N follows the theoretical envelope for d ∈ {1, 2, 50}. For d = 10 the empirical error slightly exceeds the theoretical envelope for certain values of N; the authors attribute this to the restricted hyperparameter search grid (learning rates, epochs, widths, depths, regularization), which was deliberately constrained for comparability and computational cost.
• Neural networks beat B-splines in the reported experiments. Figure 2 is described as showing a better convergence rate in N for the neural estimator, and Figure 3 (with N = 5000, network width p = (d, 16, 32, 16, 1) and sparsity ratio s_ratio = 0.75) is described as showing that the neural estimator captures local variations of the drift while the B-spline method fails to recover those fine-scale features.
• Memory scaling favors neural networks. The paper states that with d = 5 and 8 basis functions per dimension, the B-spline coefficient matrix has size MN × 32768; at 64-bit floats (8 bytes) this exceeds 24GB for M = 100 and N = 1000, before accounting for matrix inversion. The authors conclude that the B-spline estimator becomes computationally infeasible on a standard laptop for d ≥ 3, whereas the neural estimator only needs to store network parameters and minibatch data.
• Theoretical machinery. Proposition 4.1 bounds the risk by 3ε̄ + 2ℛ̂ + 44F² log(Cs(L log s + log d + log 4F − log ε̄))/N, and Proposition 4.4 bounds the expected empirical training risk by 2Ψ^F(f̂) + 3 inf_f ℛ(f, f₀) + ℭ′(F²Δ + [s(L log s + log d) + F]/N + s log N/N).

Methodology in Plain English

The authors set up the problem as component-by-component nonparametric regression. For each subject n and grid point m, they build a response variable from the observed increment divided by the time step, Y = (1/Δ)(X_{t_{m+1}} − X_{t_m}). This response is approximately the drift value at the current position plus a noise term that comes from the Brownian motion and from how much the drift changes within the interval. They then fit a ReLU feedforward network with shift vectors and a sparsity constraint to minimize the average squared error over all N trajectories and all M grid points, and they measure performance by the L² risk against the true drift on an independent test path (empirically, equation (2.7) with N′ = 1000 fresh trajectories).

The theory is proved in two steps. First, they control how far the true risk can drift from the empirical training risk by treating the whole grid vector as one large observation and applying a uniform deviation inequality over the network class, using a covering-number bound for sparse ReLU networks. Second, they expand the expected training risk in terms of the training error Ψ^F(f̂), the best approximation error within the class, and a term that accounts for the fact that the regression noise is correlated with the state of the diffusion. For the numerical study, they fix T = 1 and Δ = 0.01, estimate the first drift component, use X₀ ~ N(0, I_d) and σ the identity matrix, search over three network shapes — (d, 16, 16, 1), (d, 16, 32, 16, 1), (d, 16, 32, 32, 16, 1) — and three sparsity ratios {0.25, 0.5, 0.75}, train with Adam at learning rate 10⁻³ using early stopping after 20 consecutive epochs, repeat each configuration 50 times with 95% confidence intervals, and implement everything in PyTorch. The B-spline baseline follows Denis et al. (2021) with tensor-product B-spline bases in the multivariate case and the number of knots chosen by validation performance.

Why This Matters

• Fills a theoretical gap. The paper notes that neural network drift estimation in the high-frequency, fixed-time-horizon regime (Gao et al. 2024; Bae et al. 2025) is only beginning to be explored and that theoretical analyses remain limited; this work supplies convergence guarantees without assuming ergodicity.
• Practical estimator selection. The comparison gives practitioners a concrete reason to prefer neural networks over B-spline or ridge-style estimators when the drift has local fluctuation structure and the dimension is moderate to high.

Real-world applications implied by the paper's framing (diffusions are described as fundamental stochastic models in physics, biology, and mathematical finance):

• Molecular dynamics and biophysics: inferring the deterministic force (drift) that governs the motion of particles or molecules from many independently recorded trajectories.
• Mathematical finance: estimating the drift of an asset-price or rate diffusion over a fixed trading window, where long-time ergodicity is not a credible assumption.
• Cell biology and single-particle tracking: using many independent, densely sampled trajectories to recover spatially varying drift fields with sharp local features.
• Physics of nonequilibrium systems: reconstructing force fields from high-frequency position data in settings where the process is not stationary.

Industry relevance. The memory argument is the most directly industrial point: tensor-product B-spline bases blow up combinatorially in dimension, whereas the neural estimator's cost is governed by network parameters and minibatch size. The authors state that the B-spline approach becomes infeasible on a standard laptop for d ≥ 3 in their setting, which matters for any workflow that needs drift estimation in tens of dimensions.

Future Directions

• Relaxing the hyperparameter search constraints. The authors attribute the d = 10 deviation from the theoretical envelope to the limited search grid over learning rates, epochs, widths, depths, and regularization, suggesting broader tuning as a natural next step.
• Choosing the compact domain in practice. The paper states the choice of [0,1]^d is for convenience of theoretical analysis and can be replaced by any compact K ⊂ ℝ^d, and suggests determining K from the training data via sample coverage or a sample-splitting procedure; how to do this rigorously is left open.
• Verifying or relaxing the optimization assumption. Corollary 2.3 requires Ψ^F(f̂) ≤ C φ_N log³ N, i.e., that the training procedure nearly attains the global empirical minimizer; understanding when this holds for practical training is an open question the bound inherits.
• Extending beyond the fixed, finite time horizon. The framework assumes a fixed T with high-frequency sampling; whether the guarantees carry over to long-time or partially observed settings, and to estimating the diffusion coefficient jointly with the drift, is not addressed in the provided content.
• Scaling experiments further. The numerical study covers d ∈ {1, 2, 10, 50} and N up to 5000; whether the dimension-independent empirical rate persists for larger N, larger d, or other drift classes is an empirical question the paper leaves open.

Target Audience

• Theoretical statisticians and machine learning researchers working on nonparametric estimation for stochastic processes, empirical process theory, or approximation rates of neural networks.
• Applied researchers in physics, biology, and quantitative finance who need to estimate drift fields from many short, densely sampled trajectories and want to know which estimator to use.
• Practitioners comparing neural estimators against spline or ridge baselines in high-dimensional or locally structured settings, who will find the memory-scaling argument and the released code at https://github.com/yuzhen3001/nn-drift-estimation-diffusion-process directly useful.
• Readers of the related literature on compositional function estimation (Schmidt-Hieber 2020) and on neural drift estimation with ergodicity assumptions (Oga and Koike 2024), who want to see the high-frequency fixed-horizon alternative.

Authors’ abstract

This paper addresses the nonparametric estimation of the drift function over a compact domain for a time-homogeneous diffusion process, based on high-frequency discrete observations from $N$ independent trajectories. We propose a neural network-based estimator and derive a non-asymptotic convergence rate, decomposed into a training error, an approximation error, and a diffusion-related term scaling as ${\log N}/{N}$. For compositional drift functions, we establish an explicit rate. In the numerical experiments, we consider a drift function with local fluctuations generated by a double-layer compositional structure featuring local oscillations, and show that the empirical convergence rate becomes independent of the input dimension $d$. Compared to the $B$-spline method, the neural network estimator achieves better convergence rates and more effectively captures local features, particularly in higher-dimensional settings.