Learning When to Transfer: Reinforcement Learning for Carbon-Footprint Modeling in New Manufacturing Domains

AI summary

Overview

• Research area: Machine learning for manufacturing and life-cycle assessment — specifically transfer learning and reinforcement learning applied to carbon-footprint (greenhouse-gas) modeling for new materials in material-extrusion additive manufacturing.
• Technical level: Intermediate. The paper is written for a machine-learning audience familiar with transfer learning, ensembling, and policy-gradient methods, though the framing (a decision rather than a prediction) is explained in plain terms.
• Scope (1 sentence): The paper trains a single material-agnostic reinforcement-learning policy that decides when to reuse data from existing manufacturing domains (and in what combination) when modeling the embodied carbon of a newly introduced polymer, using 729 printing experiments across five materials.

What This Paper Is About

Manufacturers introducing a new material or process need carbon-footprint and energy estimates before they have enough operating data to train a reliable model for that new domain. Data from existing operations can help, but reusing it indiscriminately can make predictions worse — a phenomenon the paper calls negative transfer. The authors reframe the question from "how do I train a predictor?" to "which sources, if any, should I transfer from?", and use reinforcement learning to make that choice dynamically for each new material and each available amount of target data.

Key Contributions

1. A material-agnostic decision formulation for transfer. One shared policy represents domain distance, source competence (source-expert errors on the target fitting subset), predictor disagreement, and the target-data budget in its state, and outputs weights over a bank of candidate predictors.
2. A controlled leave-one-material-out evaluation protocol. Four materials generate policy-training cases while the fifth is held out as an unseen target; every material is split into mutually exclusive 50% training, 20% validation, and 30% test partitions, so fitting, model-selection, and test evidence are separated.
3. A validation gate that guards deployment. The policy's mixture is deployed only when its target-validation NRMSE is at least as good as the stronger supervised alternative (target-only or all-sources-plus-target); otherwise the supervised fallback is used.
4. An empirical analysis of accuracy and negative transfer across five polymer AM domains and seven target-fitting budgets against direct learning, prediction-level, data-level, and parameter-transfer baselines, plus policy ablations.

Main Findings

• Best overall accuracy. Across 385 held-out cases (5 materials × 7 budgets × 11 repetitions), the complete policy ("actor–critic + gate") achieves the lowest mean test RMSE at 5.2 gCO₂e/part (0.0052 kgCO₂e/part, ±0.0021), which is 20.0% lower than target-only fitting (0.0065 kgCO₂e/part) and 10.3% lower than fully pooled sources plus target (0.0058 kgCO₂e/part).
• Largest gains when target data are scarce. At fitting budgets of 5–10% of the target training partition, the complete policy lowers RMSE by 32.5% relative to target-only fitting, with negative-transfer rates of 2–4%, against 15–20% for pooling.
• The validation gate is what cuts negative transfer. Raw actor–critic exceeds target-only test RMSE in 33% of evaluated cases; the gate reduces the observed rate to 17%, compared with 49% for all-sources-plus-target pooling. The gate lowers the negative-transfer rate from 28% to 17% relative to behaviour cloning with the gate (McNemar p = 4 × 10⁻⁵), while mean RMSE improves by only 0.1 gCO₂e/part.
• Ablations separate the two mechanisms. Raw actor–critic obtains 5.6 gCO₂e/part, behaviour cloning with the gate 5.3 gCO₂e/part, and actor–critic with the gate 5.2 gCO₂e/part, indicating that the central advantage comes from learning over a diverse candidate bank while retaining a strong supervised alternative. Test R² values are 0.776 ± 0.215 (actor–critic + gate), 0.726 ± 0.213 (behaviour cloning + gate), 0.720 ± 0.189 (raw actor–critic), 0.663 ± 0.265 (all sources + target), and 0.493 ± 0.661 (target only).
• Transfer is directional and not predicted by material similarity. PETG and TPU transfer effectively in both directions, but PETG transfers effectively to rPLA and not vice versa. ABS–PETG has a standardized material distance of 0.810 yet poor transfer error near 0.019–0.024 kgCO₂e/part, whereas PETG–TPU has a larger distance of 1.803 and transfers well in both directions.
• Useful information is compositional. Frequently competitive subsets include TPU+rPLA, PETG+TPU, and ABS+PLA; a source that is weak alone (for example ABS) can still contribute in multi-source subsets with target data, structure that both nearest-source selection and indiscriminate pooling miss.
• Transfer can become inappropriate as target data grow. Across all 35 material–budget cells the complete policy stays within 15% of the best fixed rule chosen in hindsight, whereas behaviour cloning with the gate and raw actor–critic are up to 86% and 134% higher. TPU reverses from weak target-only performance at 5–10% to target-only superiority after roughly 40%, and fallback frequency is itself a result: ABS reaches a fallback rate of 1.0 by b = 0.80, and TPU uses fallback in all cases at every budget.
• Errors concentrate where source experts disagree. For PLA, mean absolute error rises from approximately 2.1 × 10⁻³ in the first disagreement quantile to 8.4 × 10⁻³ in the third; ABS also deteriorates in the highest quantile.
• Paired comparisons. The proposed method lowers RMSE by 0.64 g per part (95% interval 0.36–0.87 g) relative to all sources plus target. The text also reports a reduction of 1.31 g (0.63–1.90 g) relative to a further baseline whose identity is cut off in the provided content, so that comparison cannot be stated here.
• Baselines that rely on naive source reuse perform poorly. Averaging and similarity weighting do not resolve incompatible sources, and nearest-source pretrain–finetune shows a negative-transfer rate of 0.98 and test RMSE of 0.0528 ± 0.0769 kgCO₂e/part.

Methodology in Plain English

The authors treat each polymer (ABS, PETG, PLA, recycled PLA, and TPU) as a separate manufacturing domain. The dataset is 729 controlled material-extrusion prints — 145 each for ABS, PETG, PLA, and rPLA, and 149 for TPU — all using the same specimen geometry and a harmonized schema, with electrical power recorded at 1 Hz. Each experiment is described by 22 predictors: 11 material descriptors, six controllable process parameters, and five toolpath aggregates. The carbon response per part is computed as the measured fabrication electricity times a location-specific grid factor of 0.587 kgCO₂e/kWh, plus the filament mass times a polymer-specific cradle-to-gate production factor. The authors note this is an operational-plus-material inventory indicator, not a complete comparative LCA.

For any new target material and any given amount of target data, the method assembles a bank of candidate predictors: each source expert trained alone, supervised models retrained on selected source subsets of size at least two combined with the target fitting subset, the target-only model, and the model trained on all sources plus target. Source experts are chosen per source from Random Forest, Extra Trees, Gradient Boosting, AdaBoost, and XGBoost by lowest source-validation RMSE. A policy network reads a state describing the target-data budget, summaries of the target fitting set, standardized material distances, source-expert fitting errors, disagreement among expert predictions, and the validation performance of the two supervised references, then outputs weights over candidates. It is initialized by behaviour cloning toward the validation-optimal convex mixture, then refined by one-step actor–critic (REINFORCE with a learned baseline) using a reward that penalizes validation error, rewards improvement over the stronger supervised reference, penalizes underperformance relative to that reference, and mildly penalizes distance from the best available mixture. Because each material–budget case is one state, one action, and one reward, the authors describe it as a contextual bandit rather than sequential process control. A separate gate compares the policy mixture against the better supervised model on the target validation split and falls back when the policy is not better. Evaluation is leave-one-material-out with eleven repetitions per budget across budgets of 5, 10, 20, 40, 60, 80, and 100% of the training partition.

Why This Matters

Impact on research. The paper moves transfer selection from a post-hoc diagnosis into a forward decision: rather than choosing a transfer pipeline in advance and discovering negative transfer afterwards, a learned policy scores each transfer opportunity against supervised alternatives and can fall back safely. It also shows that material-descriptor similarity is not a sufficient rule for source selection, motivating response-level rather than distance-level criteria in cross-domain manufacturing learning.

Real-world applications.

• Carbon-footprint and life-cycle inventory modeling for a newly introduced polymer or feedstock, where a full characterization campaign has not yet been run.
• Early-stage design and process decisions in additive manufacturing, where G-code and process settings can be linked to predicted energy and material consumption.
• Auditable reuse of historical operating data across plants, machines, or product lines that share a predictor schema.
• Prioritizing where to spend scarce measurement effort, using expert disagreement and fallback frequency as signals of high-value measurement targets.

Industry relevance. A manufacturer needs only a common predictor schema across historical and new operations, a small target fitting set, and a separate target validation set to apply the method. The candidate bank turns historical operations into auditable alternatives, and because geometry, printer, and response construction are held consistent in this study, observed differences can be attributed to the transfer decision rather than to changes in system boundaries. The authors are explicit that the demonstrated benefit is improved prediction of an operational-plus-material carbon indicator, and that realized emissions reductions depend on using those predictions to choose lower-footprint, technically feasible alternatives.

Future Directions

• Recalibrating the reward weights. The reward coefficients (3.0 for underperformance versus 1.5 for improvement, and 0.75 for distance from the best mixture) were set from deployment preference rather than tuned on held-out data; the paper calls for systematic study and validation-based recalibration for other processes, response scales, or risk tolerances.
• Alternatives to the score-function estimator. The authors note that direct differentiation through the mixture reward, and per-target direct optimization of mixture weights on target validation data, are viable alternatives; the present experiments do not establish an advantage of the score-function estimator over these, and comparisons are left to future work.
• Generalization beyond five materials. The hierarchical bootstrap used for generalization intervals resamples materials and then cases, and with five materials those intervals are approximate; broader domain coverage would sharpen the estimates.
• Extending beyond the indicator studied. The studied response is an operational-plus-material inventory indicator, not a complete comparative LCA, and the approach is described as designed for other manufacturing variables such as process energy and material consumption, as well as performance and quality outcomes — applying it in those settings remains to be demonstrated.

Target Audience

Researchers and practitioners in machine learning for manufacturing, transfer learning, and reinforcement learning, particularly those working on data-efficient modeling with small target datasets and heterogeneous source domains. It is also relevant to industrial engineers and sustainability analysts building life-cycle inventories for new materials or processes, and to readers interested in contextual bandit formulations where the "label" for the correct decision never exists and the decision's value is defined relative to a supervised alternative.

Authors’ abstract

Manufacturers often need to characterize a new material, process, or product before enough operating data are available to train reliable predictive models for this new, target domain. Existing manufacturing operations contain potentially useful knowledge to accelerate this characterization, but indiscriminate reuse of data from existing source domains can cause negative transfer in learning of the new, target domain when the manufacturing domains differ sufficiently. This work proposes a reinforcement-learning method for selective knowledge transfer, where its reward scores each opportunity to transfer against supervised alternatives, so negative transfer is avoided and controlled during forward prediction instead of being diagnosed afterwards. Given limited data from a new target domain, the learned policy chooses among predictors based on individual source operations, subsets of the sources, or the supervised alternatives: fully pooled sources, or target data alone. A validation gate deploys the transferred prediction only when it improves upon the supervised alternatives. The method is designed for learning variables supporting carbon accounting, as well as performance and quality outcomes. We demonstrate the approach by quantifying the embodied greenhouse-gas emissions of parts produced via material-extrusion additive manufacturing, using data from 729 experiments spanning five materials: ABS, PETG, TPU, PLA, and recycled PLA. The response combines measured fabrication electricity with polymer-specific cradle-to-gate production factors, while 22 material, process, and toolpath features characterize each experiment. In leave-one-material-out evaluation, this method achieves a mean test RMSE of 5.2 g CO2e per part, 20.00% lower than target-only training (32.5% lower when only 5-10% of the target training data are available) and 10.30% lower than indiscriminate source-target pooling.