Global Sensitivity Analysis for Engineering Design Based on Individual Conditional Expectations

AI summary

Overview

• Research area: Explainable machine learning and global sensitivity analysis (GSA) for engineering design, with aerospace applications (surrogate models, aerodynamic shape design, wind-turbine fatigue analysis).
• Technical level: Intermediate. Familiarity with surrogate modeling, variance-based sensitivity analysis, and explainability concepts such as Partial Dependence Plots (PDPs), Individual Conditional Expectation (ICE) curves, and SHAP is helpful, but the paper motivates each concept.
• Scope (one sentence): The paper proposes ICE-based global sensitivity metrics — the mean and variance of per-instance ICE curve standard deviations, plus an ICE-based correlation value — proves that PDP-based sensitivity is a lower bound of the ICE-based metric under additive and multiplicative separability, and compares these metrics against PDP, SHAP, and Sobol' indices on three benchmark problems.

What This Paper Is About

PDP is a widely used way to interpret black-box models: it shows the average effect of an input variable on the prediction. But because PDP averages over the other inputs, it can hide interaction effects — even producing a misleadingly flat curve when an input has opposite effects in different regions of the design space. The paper replaces that averaged view with a global sensitivity metric built from ICE curves, computing the importance of each individual curve first and then aggregating, so that interactions are not cancelled out by averaging. The goal is to give engineers a scalar, model-agnostic sensitivity ranking that preserves the diagnostic richness of ICE visualizations.

Key Contributions

1. An ICE-based global sensitivity metric. The authors define I_ice(x_j; x_C^(i)) as the standard deviation of the i-th ICE curve for variable x_j, then aggregate across realizations of the complementary inputs x_C using two metrics: the mean mu_Iice,xj (global importance, which averages magnitudes of variation rather than predictions) and the variance sigma^2_Iice,xj (an interaction / heterogeneity indicator). A high dispersion indicates that the effect of x_j depends strongly on the values of the remaining variables.

2. A theoretical lower bound. The paper provides a mathematical proof that the PDP-based sensitivity is a lower bound of the proposed ICE-based metric under additive and multiplicative separability — formally E_{x_C}[I_ice(x_j; x_C)] >= I_pdp(x_j) (Eq. 19). The proof is given for classes of functions expressible as finite expansions of a linear combination of bases with some combination of additive and multiplicative separability; because a Polynomial Chaos Expansion (PCE) surrogate is such a finite expansion, the inequality holds when PCE is used.

3. An ICE-based correlation value. A metric is introduced to quantify how interactions modify the relationship between inputs and the output — specifically, whether interactions purely scale the output or fundamentally alter the functional relationship, and its nonlinearity.

4. Benchmarking on three engineering cases. The metrics are evaluated on a 5-variable analytical function, a 5-variable wind-turbine fatigue problem, and a 9-variable airfoil aerodynamics case, against PDP, SHAP, and Sobol' indices, along with a discussion of computational cost (O(NK) model evaluations per feature, O(mNK) for all m features, versus O(mN) for total Sobol' indices with the standard Saltelli estimator).

Main Findings

• PDP averaging can be misleading. Because PDP averages model predictions over complementary variables, heterogeneous effects can cancel out, potentially producing a flat profile even when the input matters strongly in specific regions. The authors state this is a serious hazard in safety-critical cases.

• ICE-based importance acts on magnitudes, not on averaged predictions. mu_Iice,xj averages the magnitude of variations across ICE curves, mitigating cancellation. sigma^2_Iice,xj measures dispersion in I_ice across realizations, indicating that the impact of x_j varies significantly with the complementary features, i.e. strong feature interactions.

• The two proposed metrics have distinct interpretations. mu_Iice,xj characterizes conditional response behavior across conditional settings, whereas the total Sobol' index quantifies the fraction of output variance attributable to x_j including interactions. The paper states the two can lead to different variable rankings, and illustrates this with the first demonstration. sigma^2_Iice,xj reflects heterogeneity in conditional response behavior, whereas Sobol' interaction indices reflect variance attribution within a functional ANOVA decomposition.

• Standard deviation versus variance as the instance-wise measure. The authors note that using conditional variance (without the square root) has formal appeal because the law of total variance decomposes Var(y) into Var_{x_j}(E[y | x_j]) plus E_{x_j}[Var(y | x_j)], matching the identification of first-order and residual/interaction-plus components in Sobol' analysis under input independence. Standard deviations are more interpretable because they share the units of the model output but do not decompose additively. The recommendation is to use variance for variance decomposition and additivity, and standard deviation for interpretability in the original output units.

• Relationship to optimal-transport GSA. The paper relates its metrics to the Wasserstein–Bures decomposition of the squared 2-Wasserstein distance (Eq. 18) between the unconditional and conditional output distributions, which splits into a mean shift term and a covariance shift term under Gaussian or elliptical assumptions. The authors state the correspondence to their metrics is not algebraic in general: the covariance shift term is defined through the conditional covariance Sigma_{y|x_j} (variability across x_C at fixed x_j), whereas ICE-based dispersion looks across x_j at fixed x_C^(i) — so the two probe variability along different conditional directions. The ICE-based dispersion is therefore to be interpreted as a complementary measure of response heterogeneity, not a direct estimator of the Wasserstein covariance shift.

• Benchmark outcome as reported in the abstract. ICE-based feature importance provides richer insights than the traditional PDP-based approach, and visual interpretations from PDP, ICE, and SHAP complement one another by offering multiple perspectives. Detailed numerical results for the three case studies are not included in the provided content.

• Computational cost. For a single feature, the method requires O(NK) model evaluations (aggregation is negligible) and O(mNK) for all m variables — identical to the cost of computing PDPs, since PDPs are obtained by averaging the same set of ICE curves — compared with O(mN) for total Sobol' indices using the standard Saltelli estimator. The proposed method therefore costs more than total Sobol' indices in exchange for resolving the conditional response structure.

Methodology in Plain English

1. Set up the surrogate. Inputs x = (x_1, ..., x_m) are assumed independent, with domain Omega and joint density pi(x). A surrogate f_hat(x) is fitted to data D = {(x^(i), y^(i))}_{i=1}^n. The authors use data-driven, non-intrusive PCE built with the UQLab module, solved with least-angle regression, with polynomial orders explored from A_{p=1} up to A_{p=p_max} and a total-order truncation A_p = {gamma in N^m : ||gamma||_1 <= p}. Crucially, the proposed framework is described as surrogate-agnostic; PCE is used because its spectral and additive structure makes the theoretical derivation analytically tractable and because it achieves excellent predictive accuracy in the demonstrated problems.

2. Compute ICE curves. For each variable x_j, draw N samples of the complementary inputs x_C^(i) from pi_{x_C} and evaluate the model on a grid of K values of x_j, giving one ICE curve per instance. Both PDP and ICE curves are anchored at the mean of the input space (x_S^a = E(x_S)) by subtracting the value at the anchor point, which shifts the curve to zero at the anchor without altering its shape.

3. Score each curve, then aggregate. For each ICE curve, compute its standard deviation over x_j (empirically via the 1/(K-1) sum of squared deviations in Eq. 13). Then average these values over instances to get mu_Iice,xj (Eq. 16) and compute their variance as sigma^2_Iice,xj (Eq. 17), both by Monte Carlo. Algorithm 1 spells this out.

4. Visualize and compare. The metrics can be shown together in a bar plot of mu_Iice,xj with sigma_Iice,xj as error bars. PDP-based importance I_pdp(x_j) is computed as the standard deviation of the partial dependence function (Eq. 9), which corresponds exactly to the unnormalised first-order effect in the Hoeffding–Sobol' (functional ANOVA) decomposition and should not be read as a proportion of total variance unless normalised. A PDP-based interaction measure is also described: build the joint partial dependence f_pdp(x_i, x_j), compute I_pdp(x_i|x_j) and I_pdp(x_j|x_i) across values of the conditioning variable, and average their standard deviations.

5. State the assumptions. The work assumes independent inputs, which the authors call reasonable for design exploration or parametric studies; PDP and ICE may be misleading with correlated inputs. The paper notes ICE is still restricted to independent features. For expensive models such as CFD, the metrics can be evaluated on a surrogate provided it achieves sufficient predictive accuracy.

Why This Matters

Impact on research. The paper builds a bridge between global sensitivity analysis and explainable ML: it turns the visual richness of ICE curves into scalar indices suitable for automated variable ranking in engineering design pipelines, where previously no standardised quantitative ICE summaries existed. It also positions these dispersion-based metrics relative to variance decomposition (Sobol'), Shapley-based attribution (SHAP), and optimal-transport GSA (Wasserstein–Bures), clarifying what each does and does not capture.

Real-world applications:

• Aerodynamic design. The 9-variable airfoil aerodynamics case demonstrates use for interpreting surrogate models of geometric or kinematic parameters, in the same family of problems the introduction cites (multi-element airfoils, hovering flapping wings, high-pressure capturing wings).
• Wind energy / structural fatigue. The 5-variable wind-turbine fatigue problem applies the metrics to a durability-relevant quantity of interest.
• Design-space screening and prioritisation. Scalar rankings from mu_Iice,xj with sigma_Iice,xj error bars help engineers decide which variables to prioritise, and flag variables whose influence is strongly context-dependent.
• Black-box model verification. Because the metrics are model-agnostic, they support checking that a learned model behaves consistently with established domain knowledge, in line with the paper's stated goal of using explainability for knowledge discovery rather than prediction alone.

Industry relevance. High-fidelity simulation budgets limit how many designs can be evaluated, so surrogates are standard in aerospace workflows. PDP-derived importance alone can mask interactions — a risk the authors describe as serious in safety-critical cases. The proposed metrics cost the same as computing PDPs (O(NK) per feature, O(mNK) for all features), so they add no new class of expense beyond what is already incurred when PDP and ICE plots are produced, though they cost more than total Sobol' indices via the Saltelli estimator.

Future Directions

• Extension beyond independent inputs. PDP and ICE can be misleading under correlated inputs, and the present work explicitly assumes independence; handling correlation with conditional or causal-aware variants is an open problem.
• Tightening the theory. The lower-bound inequality E_{x_C}[I_ice] >= I_pdp is proven for specific function classes (finite expansions with additive and multiplicative separability, including PCE); generalising the result and clarifying conditions under which the correspondence to Wasserstein–Bures decomposition becomes algebraic remain open.
• Cost reduction for high-dimensional problems. The O(mNK) scaling for all variables, and the difficulty of estimating indices with few data points in high dimensions, motivate cheaper estimators or adaptive sampling.
• Systematic comparison with interaction indices. The paper notes the metrics can lead to different variable rankings from total Sobol' indices; mapping when and why rankings diverge across Sobol', SHAP, PDP, and ICE metrics would help practitioners choose tools.

Target Audience

• Aerospace and mechanical engineers using surrogate models for design exploration, variable screening, and trade studies.
• Sensitivity-analysis and uncertainty-quantification researchers interested in metrics beyond variance decomposition.
• Explainable-AI practitioners who need model-agnostic, output-unit-interpretable summaries of feature influence, and who want to understand the trade-offs between PDP, ICE, SHAP, and Sobol' methods.
• Graduate students and method developers working on interpretable surrogate models, who will benefit from the derivations, the pseudocode, and the explicit statement of assumptions.

Authors’ abstract

Explainable machine learning techniques have gained increasing attention in engineering applications, especially in aerospace design and analysis, where understanding how input variables influence data-driven models is essential. Partial Dependence Plots (PDPs) are widely used for interpreting black-box models by showing the average effect of an input variable on the prediction. However, their global sensitivity metric can be misleading when strong interactions are present, as averaging tends to obscure interaction effects. To address this limitation, we propose a global sensitivity metric based on Individual Conditional Expectation (ICE) curves. The method computes the expected feature importance across ICE curves, along with their standard deviation, to more effectively capture the influence of interactions. We provide a mathematical proof demonstrating that the PDP-based sensitivity is a lower bound of the proposed ICE-based metric under truncated orthogonal polynomial expansion. In addition, we introduce an ICE-based correlation value to quantify how interactions modify the relationship between inputs and the output. Comparative evaluations were performed on three cases: a 5-variable analytical function, a 5-variable wind-turbine fatigue problem, and a 9-variable airfoil aerodynamics case, where ICE-based sensitivity was benchmarked against PDP, SHapley Additive exPlanations (SHAP), and Sobol' indices. The results show that ICE-based feature importance provides richer insights than the traditional PDP-based approach, while visual interpretations from PDP, ICE, and SHAP complement one another by offering multiple perspectives.