GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation

AI summary

arXiv:2511.00097v2 [cs.LG] — Zihao Guo, Qingyun Sun, Ziwei Zhang, Haonan Yuan, Huiping Zhuang, Xingcheng Fu, Jianxin Li (Beihang University; South China University of Technology; Guangxi Normal University). Code: github.com/RingBDStack/GraphKeeper. License: CC BY 4.0.

Overview

Research area: Graph incremental learning (GIL), and specifically the newly posed setting of graph domain-incremental learning (Domain-IL), positioned as a capability needed by graph foundation models (GFMs).

Technical level: Advanced. The paper assumes familiarity with graph neural networks, catastrophic forgetting, LoRA-style parameter-efficient fine-tuning, contrastive objectives, ridge regression, the Woodbury matrix identity, and t-SNE visualization.

Scope: The paper defines the Domain-IL setting for graphs, argues that catastrophic forgetting there arises from embedding shifts and decision boundary deviations, and proposes a framework (GraphKeeper) that mitigates both, evaluated on 15 real-world datasets across six groups of graph domains.

What This Paper Is About

Existing graph incremental learning research targets task-incremental (Task-IL) and class-incremental (Class-IL) scenarios, where new tasks or classes appear but all data stays within a single domain. In practice, newly arriving graphs often come from different domains, and the paper shows that representative GIL methods (for example SSM) degrade sharply when moved from Class-IL to Domain-IL. GraphKeeper's goal is to let a pre-trained GNN keep absorbing graph domains one after another without losing accuracy on domains it has already learned, even when test graphs arrive without a known domain label.

Key Contributions

1. First formulation of graph Domain-IL. The authors state they are the first to explore graph incremental learning where each incoming graph belongs to a different domain, distinct from Task-IL and Class-IL.

2. Multi-domain graph disentanglement. Domain-specific parameter-efficient fine-tuning (PEFT) built on LoRA isolates each domain's parameters so previously learned domains are frozen, combined with intra-domain (contrastive) and inter-domain (prototype-scattering) disentanglement objectives to prevent embeddings from overlapping across domains or classes.

3. Deviation-free knowledge preservation. The classifier is separated from the embedding model and updated by a closed-form ridge regression solution with a recursive update rule, so the decision boundary is not disturbed by gradient-based adaptation to new domains.

4. Domain-aware distribution discrimination. For test graphs whose domain is unobservable, a randomly initialized frozen GNN maps features into a high-dimensional space, and the graph is matched to the nearest domain prototype to select the correct PEFT module. Results show GraphKeeper achieves 6.5% ~ 16.6% improvement over the runner-up with negligible forgetting and can be integrated with representative GFMs.

Main Findings

• Large gains over GIL baselines in Domain-IL. Across six groups of graph domains, GraphKeeper reaches average accuracy (AA) of 69.2, 73.1, 80.6, 79.9, 75.5 and 77.5 in Groups 1–6 respectively, versus runner-ups DeLoMe (49.3, 58.2, 70.2, 73.4, 63.2, 64.2) and PDGNNs (52.4, 53.5, 65.5, 65.5, 64.3, 60.2). This corresponds to the reported 6.5% ~ 16.6% improvement over the runner-up.

• Negligible forgetting. GraphKeeper's average forgetting (AF) values are -0.4, -2.8, 0.0, -1.0, -0.1 and -0.1 across the six groups. For comparison, Fine-Tune ranges from -72.7 to -81.3, EWC from -72.0 to -80.9, LWF from -72.5 to -80.7, and SSM from -31.0 to -45.7. The paper defines higher AF as less forgetting. TPP reports AF of exactly 0.0 ± 0.0 in all six groups, but its AA (52.6, 49.7, 57.1, 53.0, 56.7, 48.3) is below GraphKeeper's in every group.

• GraphKeeper beats the Joint baseline. Joint, which trains on all previous graph domains at once, scores AA of 66.6, 67.6, 78.0, 75.7, 74.5 and 71.5 — lower than GraphKeeper in each group. The authors interpret this as evidence that a single GNN struggles to integrate multi-domain knowledge, while parameter isolation across domains works better.

• Existing GIL methods fail in Domain-IL. The paper reports a significant decline for general IL methods (EWC, MAS, GEM, LWF) and GIL methods (TWP, ER-GNN, SSM, DeLoMe, PDGNNs, TPP) relative to their Task-IL/Class-IL behaviour, with most memory-based methods limited by the small quantity of replayed data.

• DeLoMe and PDGNNs rely on special backbones. The paper attributes their comparatively passable results to using SGC and APPNP backbones respectively, which "sacrifice plasticity to mitigate forgetting." The authors note (Table E.3) that replacing these backbones with GCN causes a significant performance decline.

• Integration with graph foundation models. Combined with GCOPE, AA rises from 20.6, 10.5, 13.2, 12.6, 13.6, 12.6 to 56.8, 36.6, 47.4, 51.6, 44.3, 44.9 in the few-shot Domain-IL setting, with AF falling from values between -36.7 and -53.5 to between -0.2 and -1.6. Combined with MDGPT, AA rises from 19.9, 10.7, 12.1, 12.8, 12.8, 12.3 to 59.7, 32.7, 49.5, 62.7, 50.9, 43.9, with AF between -0.2 and -2.8. The paper argues that the original GFMs have strong few-shot capability but no continuous updating ability.

• Ablation: all three components matter. Removing the disentanglement objectives (w/o DT) entangles embeddings and causes cross-domain confusion; removing domain-specific PEFT (w/o PEFT) causes large parameter updates that shift previously learned embeddings; removing knowledge preservation (w/o KP) lets classifier parameters drift, deviating the decision boundary. All variants show a noticeable drop.

• Visualization confirms separation. t-SNE plots show GraphKeeper produces compact embeddings with clear boundaries between domains, whereas PDGNNs and DeLoMe show noticeable domain overlap and become increasingly entangled over learning stages.

• Hyperparameter behaviour. A small LoRA rank r limits tunable parameters and hurts adaptation; increasing r improves performance while still using far fewer parameters than full-model tuning. For the trade-off ratio γ₁/γ₂, dominance of the inter-domain objective degrades class discrimination, dominance of the intra-domain objective causes partial cross-domain confusion, and a balanced trade-off performs best.

Methodology in Plain English

GraphKeeper starts from a frozen pre-trained GNN and attacks forgetting from two directions, based on the authors' diagnosis that forgetting comes from (1) embeddings of old graphs shifting and (2) the decision boundary moving.

Keeping embeddings still. Before anything else, node features from different domains are projected into a unified dimension using truncated singular value decomposition. Then, instead of fine-tuning the whole GNN for each new domain, GraphKeeper attaches a small LoRA module (two low-rank matrices, a "down" and an "up" projection) to each domain. When a new domain arrives, the LoRA modules of earlier domains are frozen, so the parameters that produced old embeddings never change.

Keeping embeddings apart. Two objectives shape the embedding space. The intra-domain objective is a contrastive loss: augmented views of a graph are created by perturbing features and structure, and nodes of the same class are pulled together while nodes of different classes are pushed apart. The inter-domain objective pushes the current domain's embeddings away from clustering-derived "prototypes" of all previous domains. The prototypes are computable without revisiting old graphs because frozen parameters mean old-domain embeddings stay compact and unchanged. The total loss is a weighted sum of the two objectives.

Keeping the decision boundary still. Rather than an end-to-end-trained classifier, GraphKeeper uses ridge regression to map embeddings to labels, which has a closed-form solution and needs no back-propagation. Since historical data is inaccessible, the paper derives a recursive update (using the Woodbury matrix identity / matrix inversion lemma) that reproduces the exact same optimal solution using only the current domain's data, with no memory replay. Inference is a softmax over the resulting linear map.

Handling unknown domains at test time. Because each domain has its own PEFT module, the model must know which module to use. For training graphs this is trivial, but test graphs may arrive without a domain label. The authors pass test graphs through a randomly initialized frozen GNN, average-pool the resulting features into a prototype, and match the test graph to the nearest stored domain prototype. The random high-dimensional mapping is used specifically because raw features from different domains risk being too similar or overlapping; the paper reports a prototype confusion matrix before and after this mapping in Figure E.2.

Evaluation protocol. Experiments use 15 real-world datasets, follow the GIL benchmark, average results over 5 independent runs, and compare against general IL methods (EWC, MAS, GEM, LWF), representative GIL methods (TWP, ER-GNN, SSM, DeLoMe, PDGNNs, TPP) plus two self-designed baselines (Fine-Tune, Joint). Because performance varies with incremental order, multiple orders are set per group and averaged. Metrics are Average Accuracy (AA) and Average Forgetting (AF). Exact dataset characteristics, the domain groupings, and the full complexity analysis are placed in appendices that are not included in the provided content, so they are not reported here.

Why This Matters

Research impact. The paper opens a setting — graph domain-incremental learning — that the authors argue is unaddressed in the literature, and connects it directly to the practical needs of graph foundation models, which must keep absorbing graphs from many sources rather than freezing after pre-training. It also offers a diagnosis-based decomposition of forgetting (embedding shift versus decision boundary deviation) that gives other researchers two separable levers to work on.

Real-world applications. The paper does not itself enumerate deployed applications; its stated relevance is the construction of GFMs over a large corpus of graphs and the avoidance of memory explosion that replay-based methods risk on large graph collections. Plausible settings implied by the problem framing include:

• Systems that continuously onboard graph data from new scientific, social, or infrastructure domains without retraining from scratch.
• Graph foundation model pipelines that need to be extended with new downstream domains after deployment.
• Multi-domain recommendation or fraud-detection graphs, where the graph distribution drifts as new data sources are added.
• Any setting where old training graphs cannot be stored for legal or storage reasons, which fits GraphKeeper's no-replay design.

Industry relevance. The claim that GraphKeeper "can be seamlessly integrated" with GCOPE and MDGPT, together with the reported AA increases and near-zero AF, is the paper's main industrial hook: it is a bolt-on continual-learning layer rather than a replacement model. The no-memory-replay property is also relevant where retaining historical graph data is expensive or disallowed. Note that the integration experiments are the few-shot Domain-IL setting, so this framing is specific to that setup.

Future Directions

• Verify the domain-discrimination mechanism beyond prototypes. The paper's own analysis notes that raw features across domains can overlap, motivating the random frozen GNN mapping. Whether prototype matching stays reliable as the number of learned domains grows is not tested in the provided content.

• Determine how the method scales with domain count. The recursive ridge-regression update and per-domain LoRA modules grow with each new domain; the complexity analysis is deferred to Appendix B and is not included in the provided text, so the scaling limits are not established here.

• Broaden integration with graph foundation models. Only two GFMs (GCOPE and MDGPT) are tested, in few-shot Domain-IL. Testing other GFMs and non-few-shot regimes is a natural next step.

• Test outside graph node-level benchmarks. The paper reports results on 15 real-world datasets in node-classification-style GIL benchmarks; its behaviour on graph-level tasks, other GNN backbones (the paper already observes that swapping DeLoMe's and PDGNNs' backbones hurts those baselines), and settings where the domain gap is smaller than in the chosen groups is not reported.

Target Audience

This paper is most useful to researchers and practitioners working on continual or incremental learning over graphs, graph foundation models, and multi-domain graph learning, particularly those who need to add new graph domains to a deployed model without storing old graph data. It is also relevant to readers interested in parameter-efficient fine-tuning beyond language and vision models, and to engineers evaluating whether a continual-learning layer can be bolted onto an existing GFM. Readers without a background in GNNs, LoRA, or ridge regression will need supporting material, as the method and appendix material are written at an advanced level.

Authors’ abstract

Graph incremental learning (GIL), which continuously updates graph models by sequential knowledge acquisition, has garnered significant interest recently. However, existing GIL approaches focus on task-incremental and class-incremental scenarios within a single domain. Graph domain-incremental learning (Domain-IL), aiming at updating models across multiple graph domains, has become critical with the development of graph foundation models (GFMs), but remains unexplored in the literature. In this paper, we propose Graph Domain-Incremental Learning via Knowledge Dientanglement and Preservation (GraphKeeper), to address catastrophic forgetting in Domain-IL scenario from the perspectives of embedding shifts and decision boundary deviations. Specifically, to prevent embedding shifts and confusion across incremental graph domains, we first propose the domain-specific parameter-efficient fine-tuning together with intra- and inter-domain disentanglement objectives. Consequently, to maintain a stable decision boundary, we introduce deviation-free knowledge preservation to continuously fit incremental domains. Additionally, for graphs with unobservable domains, we perform domain-aware distribution discrimination to obtain precise embeddings. Extensive experiments demonstrate the proposed GraphKeeper achieves state-of-the-art results with 6.5%~16.6% improvement over the runner-up with negligible forgetting. Moreover, we show GraphKeeper can be seamlessly integrated with various representative GFMs, highlighting its broad applicative potential.