Transferring Causal Effects using Proxies

AI summary

Overview

Research area: Causal inference / machine learning — specifically proximal causal inference, identifiability theory, and domain transfer under unmeasured confounding.

Technical level: Advanced. The paper is a theoretical statistics and causal inference contribution: it relies on structural causal models, d-separation, matrix invertibility conditions, maximum likelihood asymptotics, and pseudoinverses.

Scope: The paper develops identifiability results and two estimators for a target-domain interventional distribution of a treatment on an outcome when an unobserved confounder is present, a proxy for that confounder is observed, and only the proxy is available in the target domain.

What This Paper Is About

Estimating the causal effect of a treatment X on an outcome Y is confounded whenever an unobserved variable U influences both. The authors consider a multi-domain setting in which U shifts across domains, so the causal effect of X on Y can differ between a source domain and the target domain of interest. Their goal is to estimate the interventional distribution in the target domain, where only a proxy W of U is observed, by transferring information from source domains where E, W, X and Y are all observed.

Key Contributions

1. Identifiability of the target interventional distribution from proxy data. The authors prove (Theorem 1) that the target-domain interventional distribution ℚ(Y | do(x)) is identified from the source-domain distribution of (E, W, X, Y) and the target-domain distribution of W alone, provided the source domains are sufficiently diverse — formalised as Assumption 1, a rank condition rank(P(W | E, x)) ≥ k_U on the matrix of conditional proxy probabilities.

2. Relaxation beyond the latent shift assumption. The identifiability result is shown to hold for a broader class of causal graphs, including settings with additional covariates Z, with unmediated covariate shift through an E → X edge, and without an edge between W and Y. This is characterised by d-separation statements and treated further in Appendix B (Theorem 8, Remark 9).

3. Two consistent estimators with inference guarantees. The paper introduces a causal parametrisation estimator (Proposition 4, maximum-likelihood plug-in based on the decomposition q(y | do(x)) = diag(P(y | U, W, x) P(W | U)) Q(U)) and a reduced parametrisation estimator (Proposition 5, based directly on the identifying formula). The reduced estimator is asymptotically normal, and the paper derives pointwise asymptotic confidence intervals, clipped to [0, 1].

4. Empirical validation. Simulation studies with M = 25 distinct data generating processes, N = 5 datasets each, k_E = 2 source domains and n = 20,000 realisations, plus a real-world application on hotel ranking data studying the causal effect of website rankings on consumer choices. Code is released at github.com/manueligal/proxy-intervention.

Main Findings

• Closed-form identification. Under Assumption 1, for all x in supp(X) and y in supp(Y): q(y | do(x)) = P(y | E, x) · P(W | E, x)† · Q(W), where † denotes the right pseudoinverse. The intuition is that the dependence on U vanishes once it is expressed through the factor P(W | U), which is invariant across domains by modularity.

• The rank condition is essentially necessary. If Assumption 1 fails, identifiability is in general lost; the paper provides a counterexample in §B.2, making the condition necessary in the sense that Eq. 4 cannot hold without it.

• Two estimators, one with asymptotic normality. The causal parametrisation estimator q̂_C,n is consistent as n_src, n_tgt → ∞ (Proposition 4). The reduced parametrisation estimator q̂_R,n is consistent and satisfies √n (q̂_R,n − q) / σ̂_{x,y} → N(0, 1), giving confidence intervals q̂_R,n ± (σ̂_{x,y} / √n) z_{1−α/2} (Proposition 5, Eq. 13).

• Estimation error grows with the condition number. Simulations show that the absolute estimation error of both estimators increases with the condition number κ(P(W | E, x)), a measure of near-violation of Assumption 1. The estimated condition number κ̂(P(W | E, x)) closely tracks the true one, so unreliable estimates can be flagged from data alone.

• Continuous treatment, outcome and proxy are partially covered. Theorem 1 continues to hold when X and Y are continuous (with P(y | E, x) interpreted as evaluations of a conditional density), and §B.5 proposes handling a continuous proxy W by discretisation such that Assumption 1 holds (Assumption 3, Proposition 10).

• ATE and CATE follow. Identifiability of q(y | do(x)) directly implies identifiability of the target-domain average treatment effect E[Y | do(X := 1)] − E[Y | do(X := 0)], and identifiability of the conditional effect given observed confounders Z = z yields the target-domain conditional average treatment effect.

• Distinct from prior proxy-transfer work. Unlike Tsai et al. [35], who estimate the conditional mean E[Y | X = x] under the target distribution, this paper targets the interventional distribution ℚ(Y | do(x)); unlike Miao et al. [13], the causal effect of X on Y here may differ across domains because of shifts in U.

• Real-data example reported qualitatively. The abstract and §1.2 state that the theoretical results are supported by a real-world example studying the causal effect of website rankings on consumer choices on hotel ranking data. The numerical results of that study are not contained in the provided paper content.

Methodology in Plain English

The authors assume data come from a structural causal model with a specific chain structure: a domain indicator E shifts the hidden confounder U; W is a noisy measurement of U; X depends on U; and Y depends on U, W and X. Crucially, the mechanisms connecting U, W, X and Y are assumed invariant across domains — only the distribution of U changes — so the target domain differs from the source domains purely through a shift in the hidden confounder.

The identifying trick is to write the standard covariate adjustment formula, which sums over the unobserved U, in a way that isolates the dependence on U into a single matrix P(W | U). Because this matrix is invariant across domains, the domain shifts provide the variation needed to "invert" it. The rank condition in Assumption 1 guarantees that the relevant conditional probability matrix has enough variability across domains for this inversion to succeed; the right pseudoinverse then yields a closed-form expression involving only observable quantities: source-domain conditionals of Y and W given E and x, and the target-domain marginal of W.

Two estimation routes follow. The first models the whole causal system explicitly — the matrices P(U | E), Q(U), P(W | U), P(X | U) and P(y | U, W, x) — and fits them by maximum likelihood with softmax reparametrisation, plugging the fitted values into the causal formula. The second skips the latent structure entirely and estimates the observable quantities in the identification formula by empirical frequencies, then applies a plug-in based on the pseudoinverse. The second approach is lighter and admits a delta-method variance estimate, which is what delivers the confidence intervals. Both are checked in simulations across many randomly generated parameter settings, and both are illustrated on real ranking data.

Why This Matters

Impact on research. The paper expands the reach of proximal causal inference by making it work in a domain-transfer regime where the target domain only contains proxy measurements — not the treatment, the outcome, or the confounder. It bridges two literatures that usually sit apart: proximal causal inference (which uses proxies to adjust for unmeasured confounding) and domain adaptation/transportability (which moves predictions across domains). It also shows that identifiability survives relaxations such as additional covariates and unmediated covariate shift, and that it extends to continuous treatment and outcome variables.

Real-world applications:

• Drug safety and effectiveness studies, where patient severity is an unmeasured confounder observed only through a proxy such as a diagnostic code, and the target population differs from the source cohort.
• Online marketplaces and recommendation systems, where a hidden user intent confounds clicks or purchases and a noisy behavioural signal serves as its proxy; the featured hotel-ranking study is an instance of this.
• Epidemiology with imperfect measurements, where a biomarker measured with error stands in for an unmeasured risk factor and the effects must be transferred to a new region or time period.
• Policy evaluation across sites, where a latent community characteristic drives both treatment uptake and outcomes, and outcomes are unavailable at the site where a decision must be made.

Industry relevance. Companies routinely face exactly this data shape: rich labelled data in historical markets or experiments, and only a cheap proxy signal in the deployment environment. The closed-form estimator requires no outcome labels in the target domain, and the confidence intervals make its uncertainty auditable — with the condition number providing a practical diagnostic for when to trust a given estimate.

Future Directions

• Generalising the estimators and inference, not just the identifiability result. The authors state that they believe their estimation and inference results can be generalised analogously to the broader class of causal graphs (with covariates Z and covariate shift) for which they already prove identifiability, but they do not carry this out.

• Making the continuous-proxy treatment practical. The discretisation route for a continuous proxy W is only sketched through Assumption 3 and Proposition 10; how to choose the discretisation well, and what it costs in efficiency, is left open.

• Understanding the near-violation of Assumption 1. The simulations show error blowing up with the condition number and show that the condition number is estimable from data, raising the question of principled sample-size guidance or regularised estimators when the rank condition is only nearly satisfied.

• Extending beyond the discrete/finite-dimensional framework. The paper notes that its work omits counterfactual statements and focuses on discrete variables for most of the development; extensions to richer functional or longitudinal settings are natural next steps.

Target Audience

Researchers and graduate students in causal inference, identifiability theory, and statistical machine learning who are already comfortable with structural causal models, do-calculus, and matrix-based identification arguments. It is also relevant to applied statisticians and quantitative practitioners in tech, epidemiology, and the social sciences who have proxy measurements, multiple data sources, and no outcome labels in the domain they actually care about — though those readers will likely need the appendices and a statistics background to follow the estimators and confidence interval derivations in detail.

Authors’ abstract

We consider the problem of estimating a causal effect in a multi-domain setting. The causal effect of interest is confounded by an unobserved confounder and can change between the different domains. We assume that we have access to a proxy of the hidden confounder and that all variables are discrete or categorical. We propose methodology to estimate the causal effect in the target domain, where we assume to observe only the proxy variable. Under these conditions, we prove identifiability (even when treatment and response variables are continuous). We introduce two estimation techniques, prove consistency, and derive confidence intervals. The theoretical results are supported by simulation studies and a real-world example studying the causal effect of website rankings on consumer choices.