Skewness-Robust Causal Discovery in Location-Scale Noise Models

AI summary

Overview

Research area: Causal discovery / functional causal models — specifically location-scale noise models (LSNMs) and the skew-normal distribution, in the statistics and machine learning literature (stat.ML).

Technical level: Advanced. The paper assumes familiarity with structural causal models, identifiability beyond Markov equivalence classes, likelihood-based estimation, and EM-type algorithms. The paper is formatted for ICML.

One-sentence scope: The paper introduces SkewD, a likelihood-based causal discovery method for location-scale noise models whose noise term is allowed to be skewed rather than symmetric, with a bivariate focus and a multivariate extension.

What This Paper Is About

Many causal discovery methods that go beyond Markov equivalence classes rely on location-scale noise models of the form Y = f(X) + g(X)N, and most existing estimators assume the noise term N is symmetrically distributed. The authors show that when N is actually skewed — which they argue is likely in real-world domains — those methods lose performance, because the misspecified symmetric likelihood leaves dependence between the cause and the estimated residuals. Their goal is a method that remains reliable under both symmetric and skewed noise by modeling the noise explicitly as a skew-normal random variable.

Key Contributions

1. A skew-normal LSNM formulation. The paper defines an LSNM in which the noise follows N ~ SN(0, 1, λ), so that the conditional distribution of the effect given its causes is Y | X = x ~ SN(f(x), g(x), λ), with f and g modeled as generalized additive models using B-spline bases.
2. A parameter estimation procedure. The authors derive a penalized log-likelihood and estimate it with a two-stage scheme: Bayesian optimization to set the roughness penalty parameters, CMA-ES heuristic search for the model parameters, and then an expectation conditional maximization (ECM) algorithm to refine that heuristic solution.
3. SkewD as a practical discovery method. SkewD performs bivariate cause-effect inference by either independence testing (HSIC test on extracted residuals) or likelihood scoring, and the likelihood scoring variant is extended to the multivariate setting as SkewD-MV.
4. An empirical comparison. The authors benchmark SkewD against state-of-the-art algorithms on established and novel bivariate benchmarks with varying skewness levels, plus an evaluation of the multivariate extension. They report that SkewD is the most reliable approach for LSNMs under high skewness while performing on par with state-of-the-art in other settings.

Main Findings

• Symmetric-noise assumptions break under skewness. The paper shows that when the noise variable N is skewed, approaches that assume symmetric noise drop in performance. Figure 1 illustrates this on Pair 6 of the new LSs(1.750) dataset, where the normal-based method LOCI produces residuals that still depend on the cause X — a dependence traced to an inadequate variance fit for small values of x — causing a wrong inference.
• Misspecification persists in likelihood-based methods too. The authors state that the misspecification issue persists when resorting to likelihood-based approaches, consistent with prior work on the pitfalls of Gaussian likelihood scoring in linear ANMs (Reisach et al., 2021) and LSNMs (Schultheiss and Bühlmann, 2023).
• SkewD recovers the correct direction in the illustrated case. In the same Figure 1 example, SkewD recovers the independent noise variable and infers the correct direction, where the normal-based method fails.
• Skew-normal skewness is bounded. The skewness of the skew-normal is approximately bounded in (−γ1max, γ1max) with γ1max ≈ 0.9953. The authors nevertheless report that its practical applicability extends even to settings that exceed its theoretical skewness and excess kurtosis range.
• Nonparametric alternatives underperform. The paper reports that nonparametric approaches lead to unsatisfactory performance in skewed LSNMs, motivating the parametric skew-normal choice.
• Identifiability carries over with caveats. LSNMs are identifiable in most cases; prior work characterized the non-identifiable cases for Gaussian noise (Khemakhem et al., 2021), exponential family models (Bodik and Chavez-Demoulin, 2025), and uniform and continuous Bernoulli (Sun and Schulte, 2023). The paper's argument in Section 5.4 is that inference via independence testing or likelihood scoring is valid only when the skew-normal LSNM is identifiable; the supplied content is truncated mid-sentence in that discussion, so the full statement of the identifiability caveats is not available here.
• Multivariate scalability is structure-dependent. With d = 10 nodes in their experiments, the maximum number of admissible parent-child configurations was 40; the authors expect SkewD-MV to remain applicable when configurations are in the order of 100.

Methodology in Plain English

The idea is to keep the flexible location-scale form Y = f(X) + g(X)N but stop pretending the noise N is symmetric.

• The model. Both f (the mean function) and g (the scale function) are represented as generalized additive models, where each covariate gets its own B-spline basis. The scale is passed through an exponential function to keep it positive. The noise is a standardized skew-normal, controlled by a shape parameter λ: λ = 0 gives back the ordinary normal, λ > 0 gives right skew, λ < 0 gives left skew.
• Fitting the model. The authors write down the observed log-likelihood and add roughness penalties based on second-order differences of the spline coefficients, giving a penalized log-likelihood. Two practical difficulties arise: choosing how strong the penalties should be, and actually maximizing a non-convex objective for a distribution whose likelihood is known to be hard to maximize (it can lack a finite maximum, and there is a stationary point at λ = 0).
• Three-step optimization. First, Bayesian optimization picks the penalty parameters. Second, with penalties fixed, CMA-ES — an evolutionary optimizer for non-linear, non-convex functions — searches for the parameters using multiple starting values. Third, the best heuristic solution is handed to an ECM algorithm that refines it. ECM alternates an expectation step, which computes conditional expectations of a latent variable introduced to represent the skew-normal, with conditional maximization steps that update the mean/shape parameters in closed form and the scale parameter numerically via CMA-ES. The algorithm stops when the parameter change over a lag of Δ = 25 iterations falls below 10⁻⁶ or a maximum iteration count is reached.
• Deciding the direction (bivariate). SkewD either extracts residuals in each direction and tests cause–noise independence with the HSIC test, choosing the direction with the higher p-value, or it compares the joint log-likelihoods of the two directions and picks the higher one. The likelihood variant assumes standardized data and normal marginals, which lets the marginal terms be dropped; the authors also ran an ablation with kernel density estimation for the marginals and describe those initial results as encouraging.
• Scaling up (multivariate). SkewD-MV first estimates the Markov equivalence class using the order-independent PC algorithm, then compares the partial log-likelihood sum over only those nodes that have at least one unoriented edge in that class. Proposition 5.1 states this partial sum selects the same maximum-likelihood DAG as the full sum, since nodes without unoriented edges contribute identical likelihood terms across all DAGs in the class.

Why This Matters

Impact on research. The paper targets a specific and often unexamined assumption — symmetric noise — inside otherwise general LSNM identifiability results. By explicitly modeling skewness, it offers a route to retain identifiability guarantees in data regimes where Gaussian or Student's t noise models (as in LOCI and ROCHE) are misspecified. The work also connects to a complementary result by Lin et al. (2024), who show that under symmetric noise, skewness appears in the anti-causal model; SkewD instead models the skew directly. The authors make implementations and experimental runs publicly available at github.com/causality-lab/skewd.

Real-world applications. The paper cites Genton (2004) for data examples exhibiting skewed distributions:

• Coastal flooding
• Spatial rainfall prediction
• Stochastic frontier analysis
• Earth sciences more broadly, along with neuroscience and manufacturing, which the authors list as fields where causal discovery is of strong interest

Industry relevance. Methods like SkewD are relevant wherever observational data are used to infer direction of effect rather than mere association, and where outcome distributions are asymmetric — risk, failure, demand, and environmental extremes are typical examples. The authors note that SkewD-MV is most suitable for smaller-scale problems; on larger graphs, omitting the ECM refinement step is suggested as a practical runtime/accuracy trade-off, which matters for deployment.

Future Directions

• Nonparametric marginals. The authors report encouraging initial results from estimating marginal distributions via kernel density estimation instead of assuming normal marginals, and state this motivates further investigation.
• Scalability of the multivariate variant. Runtime is dominated by fitting a skew-normal model per admissible parent-child configuration, which grows as 2^υ − 1 for a parent with υ undirected edges. The paper suggests omitting the ECM step as a trade-off and notes that the heuristic solution alone performed sufficiently for bivariate likelihood scoring, but a systematic analysis is left open.
• Tighter identifiability characterization for skew-normal LSNMs. The paper states that inference is valid only when the model is identifiable and references prior work characterizing non-identifiable cases for specific noise families, while noting no exhaustive enumeration of non-identifiable cases for LSNMs exists. The supplied content is truncated before the authors' full treatment of this point.
• Broader experimental reporting. The available content describes the experimental design — established and novel bivariate benchmarks with varying skewness levels, plus a multivariate evaluation at d = 10 — but the truncated text does not include the Section 6 result tables, so the specific benchmark scores, dataset sizes, and competing method names beyond those cited earlier in the paper are not reported in the material provided here.

Target Audience

Researchers and graduate students working on causal discovery, structural causal models, and functional causal models, particularly those using additive noise models, post-nonlinear models, or location-scale noise models. It is also relevant to statisticians interested in skew-normal modeling and EM/ECM estimation, and to practitioners who need cause-effect inference on asymmetric real-world data but are not served well by Gaussian-noise assumptions. Readers need a working knowledge of likelihood-based estimation and causal identifiability to follow the derivations; the empirical claims are accessible to a broader machine learning audience.

Authors’ abstract

To distinguish Markov equivalent graphs in causal discovery, it is necessary to restrict the structural causal model. Crucially, we need to be able to distinguish cause $X$ from effect $Y$ in bivariate models, that is, distinguish the two graphs $X \to Y$ and $Y \to X$. Location-scale noise models (LSNMs), in which the effect $Y$ is modeled based on the cause $X$ as $Y = f(X) + g(X)N$, form a flexible class of models that is general and identifiable in most cases. Estimating these models for arbitrary noise terms $N$, however, is challenging. Therefore, practical estimators are typically restricted to symmetric distributions, such as the normal distribution. As we showcase in this paper, when $N$ is a skewed random variable, which is likely in real-world domains, the reliability of these approaches decreases. To approach this limitation, we propose SkewD, a likelihood-based algorithm for bivariate causal discovery under LSNMs with skewed noise distributions. SkewD extends the usual normal-distribution framework to the skew-normal setting, enabling reliable inference under symmetric and skewed noise. For parameter estimation, we employ a combination of a heuristic search and an expectation conditional maximization algorithm. We evaluate SkewD on novel synthetically generated datasets with skewed noise as well as established benchmark datasets. Throughout our experiments, SkewD exhibits a strong performance and, in comparison to prior work, remains robust under high skewness.