When the Cross-Silo Federation Goes Offline: Continual Learning for Site Onboarding with Limited Unlabeled Data

AI summary

Overview

Research area: Federated learning and continual learning, applied to cross-silo industrial condition monitoring (specifically ultrasonic metal welding process data).

Technical level: Advanced. The paper assumes familiarity with federated averaging, class-incremental learning, catastrophic forgetting, autoencoders, and prototype-based classifiers, though its central argument is stated in terms of deployment policy rather than mathematics.

Scope: The paper proposes and evaluates a three-stage procedure by which a single late-joining site can autonomously expand an already-deployed federated classifier to cover unlabeled, previously unseen conditions, using one of two alternative sources of old-class evidence that differ in what they require an organization to disclose.

What This Paper Is About

In cross-silo federated learning, several organizations train a shared model without releasing their records, but the arrangement normally fixes in advance which sites participate and which classes the model can predict. This paper studies what happens when both assumptions break at once: a new site (a plant, hospital, or business unit) joins after the federation's original sites have gone offline, and its data arrive unlabeled, mixing conditions the model already recognizes with conditions no participant has ever observed. The goal is a procedure the joining site can run entirely on its own to discover the new conditions and expand the model's output layer, while keeping the old classes intact using only summaries the federation left behind or a few of its own records.

Key Contributions

1. An onboarding protocol for a late-joining site. The incremental task is built autonomously at the joining site while the established sites stay offline, with novelty screening, grouping, and classifier expansion all carried out inside one shared representation.

2. An evaluation on real ultrasonic-welding process data. The protocol is tested across different patterns of class arrival and compared against distillation-based (LwF) and regularization-based (EWC) continual learning methods.

3. A privacy-indexed account of old-class memory. Federation-level prototypes and site-level retained records are dissimilar kinds of evidence, yet their accuracy differs by at most 0.022, so a deployment can choose between them by the disclosure it permits rather than by the accuracy it delivers.

4. A data-budget study. Under the Local configuration, the paper maps how far the joining site's budget can be cut, finding the balance holds with only 6 labeled records per arriving condition and 3 retained per old class.

Main Findings

• Screening performs well at a high-recall operating point. On the joining site's pool of 36 familiar and 72 unfamiliar records, the class-conditional reconstruction detector reaches recall 0.975 ± 0.027 and acceptance 0.750 ± 0.068 at the federated threshold, with ROC-AUC 0.941 ± 0.018. On average it misses under two unfamiliar records and flags one familiar record in four in error.

• Screening beats post-hoc alternatives on the same classifier. Maximum softmax probability reaches ROC-AUC 0.884 ± 0.033 and the energy score 0.875 ± 0.040, against 0.941 ± 0.018 for the reconstruction bank. The margins, 0.057 and 0.066, are larger than any separation among the memory configurations. The bank ranks ahead of the energy score on 5 of 5 splits and ahead of maximum softmax probability on 4 of 5.

• Without old-class evidence, the model forgets badly. Under controlled arrivals with verified labels, plain fine-tuning falls to 0.560 ± 0.027 old-class accuracy with forgetting of 0.379 ± 0.037, and FrozenHead (old outputs frozen, representation free) reaches 0.655 ± 0.048 with forgetting of 0.284 ± 0.060.

• Both forms of replay restore old-class accuracy to comparable levels. Global reaches 0.909 ± 0.048, Local 0.924 ± 0.042, and Dual 0.931 ± 0.035 with forgetting of 0.008 ± 0.020. The largest separation among the three is 0.022, smaller than the 0.028 that a single test record is worth, while each is separated from FrozenHead alone by 0.25 to 0.28.

• Reference continual-learning methods lose more of the old classes. At their most favorable balanced settings, LwF reaches 0.707 ± 0.039 old-class accuracy with 0.956 ± 0.023 on new conditions, and EWC reaches 0.809 ± 0.054 with 0.935 ± 0.042. Both obtain higher new-class accuracy and markedly lower old-class accuracy than the replay configurations, so the difference lies in retention rather than plasticity.

• Replay keeps the two tasks balanced. All three replay configurations keep old- and new-class accuracy within 0.026 of one another, whereas every other entry in the table separates them by 0.126 to 0.419.

• The full autonomous procedure holds up end to end. With no label consulted — screening, grouping, and expansion chained together and errors propagating forward — Dual retains 0.889 ± 0.055 on the old classes while reaching 0.889 ± 0.039 on conditions it discovered without supervision, against 0.149 ± 0.084 for fine-tuning. The three replay configurations fall inside one band, from 0.868 ± 0.036 to 0.889 ± 0.055, with forgetting at most 0.063.

• Retained records buy retention, labeled records buy plasticity. In the budget grid, one labeled record per arriving condition leaves new-class accuracy at 0.556 ± 0.104; three records reach 0.800 ± 0.115 and six reach 0.900 ± 0.046. On the old classes, the b = 0 row averages 0.513 ± 0.209, and a single retained record per class lifts that column to 0.876 ± 0.053.

• A broad, forgiving operating plateau exists. Every cell with n ≥ 3 labeled records per arriving condition and b ≥ 1 retained per old class lies within 0.054 of the best cell in the grid (n = 6, b = 3, at 0.917 ± 0.065 old and 0.900 ± 0.046 new). Outside that region one of the two tasks gives way.

Methodology in Plain English

The procedure runs entirely at the joining site and has three stages, all operating on the same frozen hidden-layer activation, written as φ_θ.

Screening. The base federation trains one denoising autoencoder per known class, each trained only on representations labeled with that class. A record is scored by how badly the best-matching expert fails to reconstruct it, divided by that expert's typical error on its own class. If the score exceeds a threshold computed from known-class records alone, the record is flagged as unfamiliar. Dividing by per-class typical error matters because classes are not equally easy to reconstruct.

Grouping. The flagged records have their frozen representations reduced by principal component analysis and clustered with DBSCAN, which requires no target cluster count and leaves sparse points unassigned. Each recovered group opens one new output unit. No label is consulted.

Expansion. The output layer grows from K to K + R units, with the weights producing the old-class logits copied from the base model and their gradients set to zero, so the map from representation to old outputs survives adaptation unchanged. Freezing the old outputs alone is not enough, because nothing in the objective refers to the old classes. Two sources of old-class evidence can be added: federation-level prototypes (the mean activation of each site's records for each old class, computed before the sites disconnect) injected through the frozen representation layer, or site-level local rehearsal (records the joining site screens as familiar, pseudo-labels with the frozen base classifier, and retains up to b per old class).

Experimental instantiation. The data are 270 ultrasonic metal welding observations, each a 252-feature signal vector, in nine balanced classes formed by crossing three tool states with three surface states. The six new- and worn-tool combinations are the old classes; the three damaged-tool combinations arrive later. Each class's 30 records are split so no observation is used twice: six test, six per each of three established sites, and the remainder (six per old class, 24 per new class) to the joining site. Every accuracy is measured on the 54-record test set, with old-class accuracy measured on its 36 old-class records, so one record is worth 0.028. The base classifier is a multilayer perceptron with hidden widths 150, 90 and 40, trained for 20 federated rounds, with φ_θ the 150-dimensional activation of the first hidden layer. Adaptation runs 20 site-local rounds with prototype loss weight one and b = 3. Two experiments are run: one that isolates the expansion step using verified labels (averaged over all seven non-empty subsets of the three new classes and over 5 seeded splits), and one that runs the procedure as deployed. Reference methods are reported at the most favorable balanced setting each reaches in a sweep, which the paper notes favors them. The paper does not report wall-clock training time, energy use, or quantified communication cost beyond noting that screening constants add K+1 scalars per site to the model updates.

Why This Matters

The paper reframes a technical question — how to keep an expanded model accurate on old classes — as a governance question. Because the two viable sources of old-class memory achieve statistically indistinguishable accuracy, the deciding factor becomes what a deployment is permitted to reveal: representation-level class means released by the established sites, or a handful of records retained locally by the joining site. That is a materially different framing from the usual accuracy-versus-forgetting trade-off, and it also shows that existing reference methods (LwF, EWC) trade one task away for the other rather than balancing them.

Real-world applications:

• Manufacturing condition monitoring. A new production line or plant onboards to a shared fault-classification model without sending its process measurements to other firms, and without waiting for the original participants to come back online.
• Healthcare consortia. A hospital joining a federated diagnostic model can surface patient conditions that the founding hospitals never observed, under whichever disclosure rule its regulators permit.
• Financial services. A bank or business unit joining a shared fraud or risk model can extend it to locally observed patterns while transaction histories stay in place.
• Battery and electronics production. The paper's own instantiation is ultrasonic metal welding, where tool damage develops with use and may not have occurred anywhere in the federation during base training.

Industry relevance: the setting mirrors how real cross-silo federations are governed — small numbers of organizations under persistent identities, bound by regulation and contracts, holding data they cannot release. The finding that a configuration can be selected from the binding legal constraint, with its accuracy cost known in advance, is directly actionable for organizations planning an onboarding.

Future Directions

• Aligning heterogeneous feature spaces. The protocol assumes every site measures the same quantities the same way and that this never changes. Re-instrumenting a line, replacing a sensor, or altering feature extraction breaks the correspondence between prototypes and thresholds; the authors identify this as the extension they would attempt first.
• Rare conditions and dissimilar sites. Every class here contributes the same number of records and every site observes one process on one machine. Rare conditions would arrive as a handful of records and shift the screen's operating point, and sites differing by machine or operator would shift the representation that prototypes summarize.
• Longer drifts and larger sequences. Sites carrying related but distinct tasks, and sequences in which the process drifts between arrivals, lie outside what the paper measures. Expansion is also stated in terms of output units and would need restating for a regression or segmentation head.
• Stage-by-stage comparison. The reference methods are compared as published and receive none of the masking, freezing, or retained records the replay configurations use. A fuller study would evaluate screening, grouping, and expansion each against its own alternatives, then assemble the strongest option at each stage into the autonomous procedure.

Target Audience

Researchers and practitioners working on federated learning, continual learning, or class-incremental learning, particularly those concerned with deployment settings where participants leave and data cannot be pooled. It is also relevant to engineers and architects designing cross-silo federated systems in manufacturing, healthcare, or finance who must decide, under a given privacy regime, what summaries a federation should carry forward after its founding members go offline. Readers without a background in federated or continual learning will find the paper's motivation and conclusions accessible, but the method section and results tables assume that background.

Authors’ abstract

An organization often holds too little labeled data to train a model that generalizes, and the records that would supply the rest sit with organizations that cannot release them. Cross-silo federated learning offers a way through, since participants exchange model parameters rather than records, but it ordinarily settles two aspects of the arrangement in advance, the participating sites and the classes the model can predict, and deployment can breach both. A new site joins after training, once the established sites have finished their engagement and gone offline, and its records arrive unlabeled, mixing conditions the model already recognizes with conditions no participant has observed. We present an autonomous three-stage procedure that expands the model entirely at the joining site: reconstruction experts screen for novelty, clustering separates the flagged records into candidate conditions, and class means describe the old classes, all inside one shared representation. Those classes were learned from records that never leave their owners, so the usual defenses against forgetting are unavailable, and the procedure supplies the evidence they would have carried from either of two dissimilar sources, prototypes held by the federation or records held by the joining site. On a real industrial condition-monitoring dataset, run end to end with no label consulted, either source holds old-class accuracy at 0.868 or above with forgetting at most 0.063, and the two differ by 0.021, so a configuration can be chosen by the disclosure it permits rather than the accuracy it delivers. Both keep old- and new-class accuracy in balance where every alternative we measure gives up one for the other, and both retain more of the old classes than distillation- and regularization-based baselines. The balance still holds with only 6 labeled records per arriving condition and 3 retained per old class.