The G-Formula and Standardization

The whole mistake runs in public, on a dataset anyone can download. NHEFS smoking cessation: 1,566 people, quitting smoking as the treatment, weight change in kilograms as the outcome. Fit one outcome model, with a single product term between quitting and smoking intensity. The coefficient sitting next to the treatment prints as 2.6 kg. Hernán and Robins work exactly this example in Causal Inference: What If.

Read that number off, call it the effect of quitting on weight, and you have reported something correct that nobody asked for. The product term is in the model. So quitting does different things to people who smoke different amounts. The conditional effect is 2.8 kg (95% CI 1.5, 4.1) at 5 cigarettes a day and 4.4 kg (95% CI 2.8, 6.1) at 40. The 2.6 kg is a comparison made at fixed covariate values, between people already alike. Nobody in the room had asked about that comparison. The question was about the 1,566 as they actually are.

The number that answers it is 3.5 kg (95% CI 2.5, 4.5). It comes out of the same fitted model, on the same people, in four moves rather than one.

Three assumptions do the work, and they are worth naming before the formula rather than after it. Consistency says the outcome you observed under the treatment someone actually received is the outcome they would have had under that treatment. Conditional exchangeability says that once the covariates are held fixed, who ended up treated tells you nothing more about how they would have fared. Positivity says every covariate pattern in your target population had a real chance of receiving each treatment.

Grant those three and the g-formula follows. The mean potential outcome under treatment a equals the average of your conditional outcome model, evaluated at that treatment, over the covariate distribution of the population you care about.

Notice what it is. It is an identification result: a statement about which observable quantity equals the causal one you cannot see. It is not a promise that your estimate is any good. Whatever the outcome model gets wrong — nonlinearities, interactions, treatment-covariate combinations it barely saw — walks straight into the average and comes out the other side wearing the word effect.

Reprice every customer's basket under one fee rule, then under the other, and average the totals across the same customer mix. The gap between the two totals is what switching rules does to that mix, not to any one shopper. Baskets are the generous version of the problem, because the second total can be computed exactly. On NHEFS it cannot. Each of the 1,566 people was observed under exactly one of quitting and continuing. So every one of them contributes a prediction that was never checked against anything, to one of the two standardized means.

Four words mark four different places in that move, and they were not all coined at once. Swapping any two of them is how an analysis ends up claiming more than it estimated.

Once you recognise the move — predict twice, average, contrast — you start finding it under names that sound like separate methods and are not. What differs between them is the outcome model, the target population, or both. Each of the four below has a dated, published instance with its numbers on the record.

In most software those four moves collapse into a single command. Which is precisely why they are worth separating by hand once, on a dataset small enough to print. NHEFS is the obvious candidate, because someone has already published the numbers you should get. Tom Palmer's R and Stata companion to Causal Inference: What If prints them.

Fit the outcome model — weight change on quitting, on smoking intensity, and on the product term between them. The coefficient on quitting comes out as 2.5595941 and the product term as 0.0466628. Then stop. Do not read the first of those out loud yet.

Predict the entire dataset as though every one of the 1,566 had quit. Predict it again as though none had. Average each column of predictions on its own: 5.178841 under quitting, 1.660267 under continuing. Only then take the difference, 3.518574 — the 3.5 kg. For contrast, the conditional route on the same fit gives 2.7929 at 5 cigarettes a day and 4.4261 at 40, and never gives 3.5 anywhere.

Doing it in that order makes something obvious that no amount of arguing will. The treated average is not an average over treated units. It is an average over everybody, priced under treatment. That is the claim the 2.5595941 was never in a position to make.

The workflow is short enough to hold in your head, and its order is not arbitrary. You define the target first, because every quantity downstream is an average over it. You fit the outcome model. You predict under treatment, then under the comparison, using the same units both times. You average and contrast.

Follow it on the map with the assumptions written next to the stages they actually govern. Exchangeability is a claim about the fitting step. Positivity is a claim about the two prediction steps, and it is the one that quietly fails. That is why it, and not the others, has a diagnostic literature of its own. The choice of target is not an estimate at all. It is a decision, and it belongs to whoever asked the question rather than to whoever fitted the model.

A conditional quantity answers what happens among units that match on the covariates. A marginal quantity answers what happens to a population with a particular mix of them. Different questions, so different answers, and no contradiction anywhere in sight. On NHEFS both come out of one fit: 2.6 kg beside the treatment, 3.5 kg (95% CI 2.5, 4.5) after standardization, and 2.8 kg and 4.4 kg at 5 and 40 cigarettes a day.

Causal Inference: What If marks the fork explicitly when it reaches it. Its chapter on outcome regression notes: “In Section 13.2, outcome regression was an intermediate step towards the estimation of a standardized outcome mean. Here, outcome regression is the end of the procedure. Rather than standardizing the estimates of the conditional means to estimate a marginal mean, we just compare the conditional mean estimates.” Same model, two destinations. The software will take you to either without comment.

The two coincide only when several things hold at once: a model linear in the right way, no interaction between treatment and covariates, an effect that is the same for everyone. The product term between quitting and smoking intensity breaks the second of those on its own. The coefficient then drifts away from the standardized contrast while both remain correctly estimated.

Which means choosing between them is not a statistical decision. It follows from who will act on the number. Someone advising one person who smokes 40 a day wants 4.4 kg. Someone deciding whether to run a cessation policy across the whole population wants 3.5 kg. Handing them 2.6 kg instead is the error the opening analysis made.

Standardization earns trust in a narrow band of conditions. The outcome model has to be estimable across the regions you are averaging over, not merely fittable on the sample as a whole. The target population has to be stated out loud, rather than inherited by accident from whoever assembled the data.

That second requirement is no longer advice. A standards body has written it down. ICH E9(R1), the Addendum on Estimands and Sensitivity Analysis in Clinical Trials, was adopted on 20 November 2019, after a public consultation that opened on 30 August 2017. It requires the treatment effect to be built from named attributes before any estimator is chosen: the treatment condition, the population of patients targeted by the clinical question, the variable, the handling of intercurrent events, and a population-level summary. Its glossary defines an estimand as “A precise description of the treatment effect reflecting the clinical question posed by the trial objective. It summarises at a population-level what the outcomes would be in the same patients under different treatment conditions being compared.” The same patients under different treatment conditions is the predict-twice step, written out by a standards body. The FDA issued the guideline as final guidance for industry on 12 May 2021.

Flexible learners help with the first requirement, since they reduce the damage a wrong functional form can do. They buy you nothing at all on support. So they arrive with obligations attached: check where treatment is supported, check that predictions are calibrated, and refit under different specifications to see how far the answer travels.

When the answer turns out to rest on unsupported tails, there are two honest exits, and neither of them is trusting the model to be smooth. Shrink the estimand to the population you can actually speak about — restriction of the sample, in the positivity paper's taxonomy. Or change what you are asking of it, which is their modification of the target intervention. They are candid about the cost: their responses “can be understood as trading off proximity to the initial target of inference for identifiability”. The third exit is to go back and get a stronger design.