Targeted Maximum Likelihood Estimation

Machine learning is built to predict. Predicting well and estimating a causal effect are not the same job. The gap between them is the reason this method exists, and one 2016 abstract puts it in its first two sentences: "Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters." That is Chernozhukov and six co-authors, on double/debiased machine learning. The Econometrics Journal published the paper in 2018.

Read the second sentence again. It is the one that gets skipped. An analyst fits a flexible ensemble to the outcome and checks it the way prediction work is normally checked. Held-out outcomes come back close to observed ones. By every measure anyone applies to a predictor, the fit is excellent. The treated and untreated predictions are then averaged, differenced, and reported as the treatment effect. Achieving the first goal did not imply the second. The reported number carries a bias nothing in the held-out error would have shown.

The reason is structural rather than a matter of effort. An ensemble minimizes error across everyone, spreading its accuracy where the data is thickest. The average treatment effect depends on one specific contrast. Treated and untreated groups are rarely balanced in a finite sample. A fit can be excellent overall and still be tilted exactly where that contrast is taken.

The repair is not a better predictor. It is a second step. That step takes the fitted predictions and moves them, using information about who was treated, in the one direction the target quantity cares about. The rest of this lesson is about what the step buys and what it does not.

What it does not buy is a design. The updated estimate still needs exchangeability. It still needs positivity. It still needs nuisance models estimated well enough to be worth moving.

Targeted maximum likelihood estimation arrived in 2006, from van der Laan and Rubin. Their diagnosis is the one the previous section quotes, reached from the density side rather than the machine-learning side: "A substitution estimator evaluating the parameter of the density estimator is typically too biased and might not even converge at the parametric rate: that is, the density estimator was targeted to be a good estimator of the density and might therefore result in a poor estimator of a particular smooth functional of the density."

The algorithm they define is a one-step procedure — a k-th step procedure, if you iterate — with exactly three operations. Build a "hardest" parametric submodel through the initial density estimator, with parameter epsilon, whose score equals the efficient influence curve. Estimate epsilon by maximum likelihood. Define the update. That is the entire apparatus, and the order is the whole idea. Describe the data first, with whatever learners perform, since nothing about that stage is specific to the question being asked. Only then nudge the estimate toward the one quantity being reported, along a direction derived from the efficient influence curve for that quantity.

The paper also proves how little the starting point has to be worth. Take a target parameter that is linear and a model that is convex. The estimator is then often achieved in the first step from an arbitrary starting density, even a heavily misspecified one. The step is doing work the initial fit was never asked to do.

Focus is the nearest everyday version of this. A photograph can be sharp across the whole frame and still fail to resolve the object you meant to measure. Pulling that object into focus costs sharpness elsewhere, and it is a trade worth making when the object is the reason for the picture. What no adjustment does is bring in an object that was never in the frame. Exchangeability and positivity decide the frame here, and no amount of targeting supplies either.

Built with a loss and link chosen for the outcome, the update has a second useful property. The final number stays inside the parameter's natural bounds, and it leaves behind the influence values that inference is later built from. None of that rescues a broken design.

The vocabulary sounds more exotic than the procedure. Each of these four terms names something in the sequence above — the submodel, the maximum-likelihood step, the update, the learner that supplies the initial fit. Keeping them distinct is mostly a matter of remembering which stage each one belongs to.

The workflow runs in one direction, and every link constrains the next. Initial nuisance fits come first, for the outcome and for treatment assignment. The clever covariate is constructed out of those treatment estimates and out of the estimand being reported. The fluctuation is fitted along it. The updated distribution is substituted into the target parameter. Inference comes last, from the influence values the targeted fit leaves behind.

The assumptions do not sit neatly at one link. Positivity does its damage at the second. A propensity crowded against zero or one produces a clever covariate large enough to dominate everything that follows. A 2010 review by Petersen and colleagues states the mechanism: "Positivity violations occur when certain subgroups in a sample rarely or never receive some treatments of interest. The resulting sparsity in the data may increase bias with or without an increase in variance and can threaten valid inference."

That review is worth reading at exactly this link. It catalogues the four things analysts actually do about sparsity: restricting the covariate adjustment set, changing the projection function in a marginal structural working model, restricting the sample (trimming), and modifying the target intervention. All four change the parameter being estimated. The review also supplies something to run rather than something to worry about. ETA.Bias, a parametric bootstrap diagnostic, estimates the finite-sample bias the sparsity produces.

Exchangeability, by contrast, does its work at every link at once and never appears in any output at all. Reading a diagram of this usefully means stopping at each arrow and asking what would have to be true for the next box to mean anything.

The acronym does not specify an implementation. Two teams can run this on the same table, describe their work accurately, and report different effects. Several choices are made before a single quantity is computed, and none of them follow from the name. Each one belongs in writing, next to the estimate.

Both TMLE and AIPW descend from the same influence-function argument, and under the usual conditions both are doubly robust. What separates them is where the correction is applied. AIPW computes an initial estimate and then adds a correction term to it — arithmetic performed on the answer. TMLE moves the fitted distribution instead, and reads the parameter off the moved distribution. That is the substitution step from the first sections doing its job.

The practical consequence is boundedness, and it has a price tag. One 2010 simulation put a number on it. Gruber and van der Laan drew 1,000 samples of size n = 1,000 in a sparse design. True treatment assignment probabilities ranged from 0.047 to 0.998, and the true additive causal effect was psi_0 = 1. The initial outcome fit was misspecified. Fluctuated on the logistic scale, the TMLE returned an average estimate of 0.722, with mean squared error 0.291. Fluctuated on the linear scale — a fluctuation that does not respect the bounds of the observed-data model — it returned −0.263, with mean squared error 4.173. That is the wrong sign for an effect of 1, and roughly fourteen times the mean squared error. Nothing changed but where the correction was allowed to live.

The authors state the rule the table illustrates: "However, it is very important that the fluctuations stay within the semiparametric model for the observed data distribution, even if the parameter can be defined on fluctuations that fall outside the assumed observed data model."

So a correction added at the end can push an estimate past what the parameter permits, and so can one fluctuated on a scale that ignores the model's bounds. An estimate read out of a valid distribution cannot. That is worth 0.722 against −0.263 when the outcome is bounded and the design is sparse. It is worth much less when nothing about the parameter constrains the range. There, the choice between them is mostly a question of which one your team can implement and diagnose without mistakes.

Do this once by hand, on a binary outcome, with every intermediate quantity printed rather than swallowed by a wrapper that returns a single estimate and a standard error.

Fit the outcome model and the propensity model, and look at both before going further. Look in particular at how close the estimated assignment probabilities come to zero and one. That is the quantity Gruber and van der Laan let range from 0.047 to 0.998 to build a sparse design on purpose. Build the clever covariate and look at its distribution before you use it. Fit the fluctuation on a scale that respects the bounds of the observed-data model, and record the coefficient. Substitute, read out the estimate, then compute the influence values and get the standard error from them.

Then run the same data twice more, changing one thing each time. Fluctuate on the unbounded scale and watch what the published comparison predicts: an estimate that can come back the wrong side of zero for a positive effect. Swap the nuisance fits for a super learner library and see how far the standard error moves. The benchmark above measured that swap at a factor of 8.5 for TMLE_Y* and 1.5 for C-TMLE_Y*.

The moment worth waiting for is the comparison between the initial contrast and the targeted one. That gap is this lesson made numerical. It shows exactly how much of the reported effect came from the ensemble's description of the data, and how much came from aiming that description at the question you were actually asked.

This is expensive to build, expensive to explain, and expensive to review, so it should be buying something specific. It buys three properties at once: flexible nuisance estimation, inference with a defensible basis, and a final number that cannot leave the space its parameter lives in. Problems that genuinely need all three have few alternatives.

Problems that need one of them, or none, do not repay the cost. There is a published counter-example behind that sentence rather than a slogan. Kang and Schafer, in 2007, built a population with mean mu = 210.0, a respondent mean of 200.0, a nonrespondent mean of 220.0 and a 50% response rate. They drew 1,000 samples at n = 1,000. Inverse-probability weighting with a misspecified propensity model had bias 5.05 and RMSE 12.10, against bias −0.01 and RMSE 1.81 when the propensity model was correct. And no doubly robust estimator they tried beat plain regression prediction. Their own summary of what the experiment licenses is deliberately narrow: "But it does demonstrate that, in at least some settings, two wrong models are not better than one."

That design is not a museum piece. It is the same missing-data design used four years later to benchmark ten estimators, which is what makes the pair instructive. The doubly robust machinery is not a guarantee that arrives with the acronym. It is a set of conditions, and where the nuisance models are both wrong the extra structure can buy nothing at all.

A clean randomized comparison, or a setting where a simple well-specified model already fits, is usually better served by a simpler estimator. Not because simplicity is a virtue in itself, but because a reviewer can check it end to end, and an estimate nobody can check is not evidence of anything.

The honest failure mode is worth naming plainly. The acronym sounds more causal than a difference in means, and that is not a reason to use it. Nothing in the targeting step manufactures exchangeability or overlap. Where the design cannot support the claim, all this produces is a carefully targeted estimate of a quantity you still cannot interpret.