Knowledge-guided generative surrogate modeling for high-dimensional design optimization under scarce data

AI summary

Overview

Research area: Machine learning for engineering design—specifically surrogate modeling that combines kernel methods (radial basis functions) with generative modeling and expert-supplied domain knowledge.

Technical level: Intermediate. The paper assumes familiarity with RBF interpolation, Gaussian processes/kriging, null-space linear algebra, and generative models, but the core ideas are explained with worked examples.

Scope: The paper introduces RBF-Gen, a surrogate modeling framework that builds an overcomplete RBF basis, exploits its null space with a generator network, and injects expert structural and distributional priors, then evaluates it on a 1D cantilever beam, a 2D cantilever shell, and a real-world semiconductor etching dataset.

What This Paper Is About

In mechanical design and manufacturing process optimization, high-fidelity simulations are often unavailable or too expensive, so engineers must build surrogate models from very few measurements. Standard RBF and kriging surrogates interpolate sparse data well in low dimensions but produce a single interpolant, offer no way to encode what subject matter experts already know about the physics, and degrade badly when the number of samples is much smaller than the number of design variables. The goal of this paper is to give sparse data somewhere useful to go: instead of one arbitrary interpolant, generate a whole family of data-consistent interpolants and use expert knowledge to steer that family toward physically meaningful solutions.

Key Contributions

1. An overcomplete RBF formulation. The RBF centers are decoupled from the data locations: K > N centers are placed uniformly or quasi-randomly over the design domain, yielding an underdetermined interpolation system with infinitely many exact interpolants rather than a single unique one.

2. A generator that explores the null space. A generator network G(z; θ) maps latent variables z ~ N(0, I) to null-space coefficients, so each latent sample produces a valid interpolant that exactly fits the training data while varying in the degrees of freedom that the data do not constrain.

3. A systematic mechanism for encoding expert knowledge. Two complementary loss mechanisms are defined: soft penalty terms for structural priors (monotonicity, positivity/bounds, Lipschitz/slope bounds, smoothness/curvature, convexity/concavity, boundary and symmetry conditions) and KL divergence terms for distributional priors on function statistics (point values, regional averages, extrema, gradient magnitude, curvature, and functional integrals).

4. Validation across three case studies. The framework is tested on a 1D cantilever beam compliance problem, a 2D cantilever shell thickness problem using the Reissner-Mindlin shell model in FEniCS, and a semiconductor etching process with real-world manufacturing data and expert-provided monotonicity information.

The paper states that no existing surrogate modeling framework systematically integrates such knowledge into the training process, and positions RBF-Gen as a natural extension of the standard RBF method rather than a physics-informed model relying on governing equations or physical residuals.

Main Findings

• 1D demonstration with four data points: For the quadratic function f(x) = 20x² + 20x + 1 sampled at only 4 locations, relaxing the RBF centers produced a wide variety of interpolants that exactly match the training points, and adding priors based on target function values, curvature distributions, and monotonicity trends shaped the generated functions to better reflect the true function's structure.

• 1D cantilever beam, very scarce data (N/D = 1): RBF-Gen consistently outperformed the standard RBF surrogate, achieving substantially higher design improvements across dimensions ranging from D = 10 to D = 80.

• 1D cantilever beam, more data (N/D = 2): The result reverses—the standard RBF surrogate achieved stronger improvement on average while RBF-Gen tended to underperform, which the authors attribute to priors introducing unnecessary bias once enough data exist to constrain the mapping.

• Slice-prediction quality: Comparing true FEM response against standard RBF and RBF-Gen along the first design variable (slices for D = 2, N = 4 and D = 10, N = 20), RBF-Gen functions followed the true response much more closely. Accuracy degraded as dimensionality increased from D = 2 to D = 10, but RBF-Gen still outperformed standard RBF.

• Computational cost: RBF-Gen requires considerably more training time than standard RBF due to null-space computation and generator training, and its training time grows superlinearly with problem dimension. The authors argue this cost (on the order of seconds to minutes) is negligible when each experiment or high-fidelity simulation takes hours or days.

• 2D cantilever shell, N/D = 1: The standard RBF surrogate degraded drastically as the number of design variables increased, finding an optimal design worse than the initial design for all but the smallest case considered. RBF-Gen outperformed it for all but the lowest-dimensional problem and consistently found designs better than the initial design.

• 2D cantilever shell, N/D = 2: Standard RBF improved relative to the N/D = 1 case but still failed to obtain any performance improvement in the higher-dimensional cases, while RBF-Gen consistently found optimal designs that improved on the initial design.

• Semiconductor etching case: The abstract reports superior predictive accuracy on a real-world semiconductor manufacturing dataset. The truncated content shows the setting—vertical profiles and top views during etch and ash processes, an equipment schematic, and a table of expert monotonicity information covering 17 input variables and 5 quantities of interest (with +1 for increasing, −1 for decreasing, and 0 for no prior information, and variable/output names redacted for confidentiality)—but the numerical accuracy results themselves are not included in the available text. Table 2 is labeled "Leave-T…" and is truncated.

Methodology in Plain English

The approach has three stages.

Stage one—build a richer basis. Classical RBF places one basis function at each training point, giving exactly as many unknowns as data and therefore one unique interpolant. RBF-Gen instead scatters K centers across the design domain with K larger than the number N of data points. Fitting the data now leaves the system underdetermined, so there is a whole family of functions that pass exactly through every measurement.

Stage two—describe the family. All solutions can be written as a particular solution plus any combination of the null-space directions that the data cannot see. Those null-space coefficients are the free dials: turning them moves the surrogate around without ever violating the measured data.

Stage three—turn the dials with knowledge. A neural network generator takes random latent inputs and outputs those null-space coefficients, so sampling the latent space samples admissible interpolants. The generator is then trained with a combined loss: soft penalty terms that fire when a generated function breaks a physical expectation (for example, compliance increasing where it should decrease, or going negative), and KL divergence terms that pull the distribution of generated function statistics toward target distributions the expert specifies. No adversarial discriminator is used.

For the engineering problems, data were deliberately drawn from a narrow neighborhood of the design space (for example, element heights between 0.05 and 0.1 rather than the full feasible range of 0.05 to 0.5) to emulate the practical situation where experimental data cluster around nominal recipes. Priors used were monotonicity (compliance decreasing with thickness), positivity (compliance non-negative), and slice-based Gaussian targets formed by perturbing true finite element values by ±30 percent. The number of centers is chosen by a simple heuristic between two and five times the number of design variables.

Why This Matters

Impact on research. The paper offers a different route to knowledge-guided learning than physics-informed neural networks or physics-guided generative models: rather than embedding governing equations or physical residuals, it embeds qualitative expert judgment—monotonicity, bounds, curvature, distributional expectations—as penalties and divergences applied to a family of exactly interpolating functions. It also frames surrogate construction as a null-space exploration problem, connecting classical kernel interpolation to information-maximization ideas from generative modeling.

Real-world applications (as framed by the paper):

• Mechanical and structural design, such as beam and shell thickness optimization where compliance must be minimized under volume constraints.
• Semiconductor manufacturing process optimization, including etching and ash processes modeled from limited fab measurements.
• Aerospace system design, where surrogate models are already standard practice.
• Materials and chemical process design, both listed among the domains where surrogate models are established.

Industry relevance. The motivating setting is industrial: proprietary processes where high-fidelity simulation is unavailable, experiments are expensive, and experts already know directional trends. The paper reports the semiconductor dataset as real manufacturing data with anonymity-redacted variable names, which signals an industrial collaboration, and two of the authors are affiliated with Samsung Electronics.

Future Directions

• Resolving the N/D = 2 reversal. In the 1D beam case, RBF-Gen underperformed standard RBF once twice as many samples as variables were available. Understanding when priors help versus bias the surrogate, and possibly relaxing prior weights adaptively as data accumulate, is an open question.

• Improving high-dimensional behavior. Accuracy degraded as dimensionality grew from D = 2 to D = 10 in the slice comparison, and the 2D shell case showed both methods struggling in higher dimensions. Scaling the approach to larger design spaces is a stated concern rather than a solved problem.

• Managing hyperparameter and computational cost. The number of centers K, the scaling factor on the null-space coefficients, and the prior weights λ and γ all affect results, and generator training time grows superlinearly with dimension. More principled selection and cheaper training are natural next steps.

• Full reporting of the semiconductor case. The truncated content does not include the accuracy numbers, the design of the leave-out validation in Table 2, or the details behind the abstract's claim of superior predictive accuracy, and the paper's Section 5 conclusion and future work are not included in the available text.

Target Audience

Engineers and researchers in design optimization, surrogate modeling, and manufacturing process development who work with datasets far smaller than their design space. It is also relevant to machine learning researchers interested in knowledge-guided learning, generative approaches to function spaces, or the intersection of kernel methods and neural networks. Readers need working familiarity with RBF interpolation, linear algebra including null spaces, and generative modeling; the paper's own 1D demonstration makes the central idea accessible without deep background in the application domains.

Authors’ abstract

Surrogate models are widely used in mechanical design and manufacturing process optimization, where high-fidelity computational models may be unavailable or prohibitively expensive. Their effectiveness, however, is often limited by data scarcity, as purely data-driven surrogates struggle to achieve high predictive accuracy in such situations. Subject matter experts (SMEs) frequently possess valuable domain knowledge about functional relationships, yet few surrogate modeling techniques can systematically integrate this information with limited data. We address this challenge with RBF-Gen, a knowledge-guided surrogate modeling framework that combines scarce data with domain knowledge. This method constructs a radial basis function (RBF) space with more centers than training samples and leverages the null space via a generator network, inspired by the principle of maximum information preservation. The introduced latent variables provide a principled mechanism to encode structural relationships and distributional priors during training, thereby guiding the surrogate toward physically meaningful solutions. Numerical studies demonstrate that RBF-Gen significantly outperforms standard RBF surrogates on 1D and 2D structural optimization problems in data-scarce settings, and achieves superior predictive accuracy on a real-world semiconductor manufacturing dataset. These results highlight the potential of combining limited experimental data with domain expertise to enable accurate and practical surrogate modeling in mechanical and process design problems.