BFHD: Bidirectional Feature Harmonization Decomposition for Heterogeneous Clinical Assessments

AI summary

Overview

Research area: Machine learning for clinical data integration, sitting at the intersection of psychometric test linking, multiview/bidirectional representation learning, and neural feature gating.

Technical level: Advanced. The paper assumes familiarity with psychometric equating (IRT, total-score linking), cycle-consistent and invertible neural architectures (NICE-style normalizing flows), hard-concrete L0 gating relaxation, and feasibility-constrained optimization.

Scope: The paper proposes BFHD, a framework that decomposes two heterogeneous clinical assessment systems into measurements that can be reliably translated in both directions within a tolerance, and measurements that cannot.

What This Paper Is About

Clinical records are often mixtures of different assessment systems, because centers, cohorts, and time periods use different rating scales, cognitive tests, and questionnaires. Even when two systems target the same clinical domain, they share some measurements and contain system-specific ones. Existing harmonization practice links only total scores, leaving no measurement-level guidance on which individual items can be translated across systems and which cannot. The paper reframes this as a bidirectional recoverability problem: given paired observations from two systems, find the largest subsets of measurements that can be reliably translated in both directions within an application-defined tolerance, and explicitly set aside the rest as non-translatable.

Key Contributions

1. A measurement-level formulation of harmonization as bidirectional recoverability. The authors define ε-recoverability: a measurement X_i is ε-recoverable from Y if the expected feature-wise discrepancy between X_i and the reconstruction g_Φ(Y)_i is below a tolerance ε, with the symmetric condition for Y_j. Recoverability is defined relative to a chosen bidirectional model class, extending classical total-score linking to multivariate feature-level harmonization.

2. A formal optimization problem and feasibility criterion. A triple (S_X, S_Y, Φ) is ε-feasible if every selected feature satisfies its recoverability condition. BFHD then maximizes w|S_X| + (1−w)|S_Y| over subsets and shared-model parameters subject to ε-feasibility, with w fixed by the user (w = 0.5 unless otherwise stated). An optimal solution yields the decomposition into X_rec / X_nt and Y_rec / Y_nt in the original measurement space.

3. A single-operator, feasibility-driven implementation. BFHD enforces strict bidirectional parameter sharing using an invertible NICE-style backbone (the X→Y and Y→X mappings are the forward and inverse passes of one operator), applies feature-wise hard-concrete L0 gates at the outputs of the cross-view mappings, and uses gated hinge reconstruction losses with a gate reward term (rather than the usual sparsity penalty) to favor large feasible subsets.

4. Empirical validation on synthetic and real clinical assessment pairs. Experiments cover synthetic data with ground-truth recoverable labels, real paired systems (ADNI cognitive assessments and a co-administered ADHD symptom-scale dataset), a temporal deployment test under protocol drift, and a component-wise ablation.

Main Findings

• Synthetic structure recovery. On the synthetic dataset with ground-truth recoverability at a generation seed of s_gen = 5, BFHD achieved index-level structure recovery accuracy of 0.880 ± 0.024 (system X) and 0.830 ± 0.024 (system Y), with Jaccard stability of 0.933 (X) and 0.924 (Y). CMLP reached 0.855 ± 0.015 and 0.810 ± 0.049, with Jaccard 0.975 and 0.789; NICE reached 0.735 ± 0.039 and 0.725 ± 0.040, with Jaccard 0.837 and 0.814. Baselines without gating were scored post hoc using a relaxed 2ε_gen feasibility criterion.

• Broader coverage on real data. At ε_cutoff = 0.6, BFHD matched or exceeded baselines on the number of recoverable measurements. On CAS ↔ SNAP, BFHD and NICE both reported 11 recoverable measurements for system A and 9 for system B, while IRT reported 11 and 7, and XGB, FRF, and CMLP reported 9 and 7. On EMBIC ↔ NeuroBAT, BFHD reported 7 and 2, versus IRT 6 and 0, XGB 6 and 1, FRF 6 and 0, CMLP 5 and 0, and NICE 7 and 2.

• Improved subset stability. On CAS ↔ SNAP, BFHD's Jaccard similarity was 0.930 (A) and 0.817 (B), versus 0.867 and 0.774 for NICE and 0.733 and 0.785 for IRT. On EMBIC ↔ NeuroBAT, BFHD's Jaccard was 0.910 (A) and 0.689 (B), versus 0.806 and 0.523 for NICE, 0.739 and 0.491 for CMLP, and 0.841 and N/A for IRT.

• Additional measurements are not poorly reconstructed. The authors report that the extra measurements BFHD identifies match baselines on the count of strongly aligned measurements (CCC > 0.7) while increasing the number and stability of marginally aligned ones (CCC > 0.5). On CAS ↔ SNAP, BFHD counted 4 measurements above CCC 0.7 and 11 above CCC 0.5 in system A, and 5 and 9 in system B. On EMBIC ↔ NeuroBAT, BFHD counted 1 and 7 in system A, and 0 and 2 in system B.

• Smooth coverage–feasibility frontier. In the ε-sweep on EMBIC ↔ NeuroBAT, BFHD recovered more measurements than all baselines across a broad range of thresholds, with the advantage especially pronounced in the intermediate regime of approximately ε_cutoff ∈ [0.55, 0.8]. Baselines were described as either overly conservative at low thresholds or showing abrupt, unstable increases as ε_cutoff relaxed.

• Better temporal deployment under protocol drift. On EMBIC (A) ↔ NeuroBAT (B) with a time-based split, BFHD retained 7 measurements on system A with a 100.0 ± 0.0 percent retention rate and 3 measurements on system B with 100.0 ± 0.0 percent retention. Comparable baselines were NICE at 6 and 77.4 ± 9.9 percent (A) and 2 and 49.2 ± 10.5 percent (B), and CMLP at 6 and 71.0 ± 7.5 percent (A) and 1 and 90.0 ± 10.0 percent (B).

• Ablations identify which components matter. Replacing the L0 gate reward with a standard L0 penalty produced degenerate solutions (0 recoverable measurements in both systems, marked as trivial). Removing output gating preserved the count of 7 and 2 but reduced stability (Jaccard 0.806 and 0.523 versus 0.910 and 0.689 for the full model). Replacing the invertible backbone with a non-invertible tied-weight MLP (TWMLP) preserved subset size (7 and 1) with dataset-dependent stability effects (Jaccard 0.886 and 0.600), suggesting strict bidirectional parameter sharing contributes to robustness but invertibility itself is not a prerequisite. Removing the hinge loss produced trivial gating solutions (13 and 9 recoverable measurements).

Methodology in Plain English

The authors start from paired data: for the same individuals, measurements from two different assessment systems. Their core claim is that a model translating between the systems should not be two independently trained functions linked only by a loss. Instead, the two directions should come from one shared operator, so that improving the X→Y mapping automatically constrains the Y→X mapping.

To achieve this, they use a NICE-style invertible neural network. The forward pass maps X to Y; the inverse pass maps Y back to X. Because the network is invertible by construction, there is exactly one set of parameters and no opportunity for the two directions to drift apart. When the two systems have different numbers of measurements, both are embedded into a common dimension D = max(d_X, d_Y) by zero-padding the smaller one; padded dimensions are never supervised and never interact with the gates.

On top of this backbone, each output measurement gets a gate value in [0, 1]. A gate near 1 means the measurement is treated as recoverable; near 0 means it is assigned to the non-translatable component. Crucially, gates sit at the outputs, so the model always sees the full input, but only the gated subset is required to be reconstructed. Gates are trained with the hard-concrete relaxation of the L0 penalty, which gives stochastic binary behavior during training while remaining differentiable in expectation.

The training loss has three parts. Two gated hinge terms measure per-feature normalized mean squared error (NMSE); a gated feature incurs no penalty once its reconstruction error drops below a training tolerance ε_train, so the model is not pushed to over-optimize beyond feasibility. The third term is unusual: instead of penalizing gate activation for sparsity, BFHD rewards open gates. Gates should stay open unless reconstruction error forces them closed. The tension between the reward and the hinge losses produces the empirical approximation to the largest feasible subsets. The authors also note that ε_train is set well below the evaluation tolerance ε_cutoff so that borderline features do not remain open due to noise, and that training uses a staged procedure (details deferred to Appendix A.2).

For evaluation, the paper distinguishes two tolerances: ε_train used only for optimizing BFHD, and ε_cutoff = 0.6 used uniformly across all methods for reporting feasible measurements. All experiments use 10 random seeds (s_train ∈ {1, ..., 10}); synthetic datasets use fixed generation seeds (s_gen ∈ {1, ..., 5}).

Why This Matters

Impact on research. The paper moves harmonization from total-score linking to a measurement-level, bidirectionally coupled feasibility statement. Because feasibility is defined "relative to a model class" — much as classical linking defines equivalence relative to a family of allowable linking functions — it gives a principled vocabulary for talking about which individual items are genuinely shared across instruments and which encode system-specific information. Crucially, it treats system-specific content as something to preserve and report, not nuisance variation to be removed, in contrast to ComBat-style approaches. It also departs from feature selection: gates are optimized for bidirectional recoverability under a shared operator rather than for downstream task performance.

Real-world applications:

• Cross-protocol integration, where records from different scales or instrument versions must be combined into a single analyzable dataset.
• Retrospective cohort pooling, where studies run at different times and sites used different batteries of instruments.
• Longitudinal analysis of patients who move between centers or whose local protocol evolves over time.
• Deployment under protocol drift, where harmonization decisions learned on an earlier cohort must remain valid on later data — the paper's temporal experiment directly tests this, along with robustness to test-time missingness (reported in Appendix C).

Industry relevance. The framework is relevant to clinical trial data management, electronic health record interoperability, and digital-health or cognitive-assessment vendors whose products must interoperate with legacy instruments. The gate outputs give an auditable, measurement-level statement of what can and cannot be translated, which matters for regulatory and scientific defensibility. The code and supplementary materials are released at https://github.com/UOW-AMRL/IJCAI26-BFHD, and the work is an extended version of a paper published in the Proceedings of IJCAI-ECAI 2026 (DOI 10.25963/ijcai.2026/784).

Future Directions

1. Choosing ε in practice. The paper states that the feasibility tolerance ε_cutoff is system- and context-dependent rather than universal, and adopts 0.6 as a conservative, measurement-anchored reference point anchored on reported intrinsic measurement uncertainty for NeuroBAT subtests in ADNI and SNAP-IV items in ADHD. How to set it from first principles for a new instrument pair — rather than by anchoring or sensitivity analysis — remains open.

2. Relaxing the paired-data requirement. All real-world datasets consist of paired, co-administered assessment systems. The framework's recoverability definition assumes paired observations and access to true expectations, so extending it to unpaired or partially overlapping cohorts is a natural next step not addressed by the reported experiments.

3. Handling non-numerical measurements. All assessment features are treated as numerical variables, and the same normalized squared-error discrepancy is used throughout; ordinal items are modeled as numerical scores under this approximation. Whether ordinal-aware discrepancy measures change the recovered subsets is an open question.

4. Whether invertibility is needed. The ablation with a non-invertible tied-weight backbone preserved recoverable subset size, which the authors read as evidence that strict bidirectional parameter sharing matters more than invertibility itself. This suggests further investigation into lighter-weight coupling mechanisms, and into the identifiability question the paper raises: when multiple maximal feasible solutions exist, BFHD recovers one, and stability across random initializations is used as a practical check.

Target Audience

This paper is most useful to machine learning researchers working on clinical and biomedical data integration, especially those interested in feasibility-constrained learning, coupled bidirectional architectures, and gated feature selection. It is also relevant to psychometricians and clinical researchers dealing with instrument linking and cross-protocol cohort pooling, who will care about the measurement-level decomposition and the CCC-based diagnostics. Practitioners building clinical data pipelines or translational tooling — for instance, teams merging cognitive batteries across centers or tracking patients across changing protocols — will find the deployment and drift experiments most directly actionable. Readers will need comfort with the psychometric linking literature and with invertible-network and hard-concrete gating machinery to follow the methodology in full detail; Appendices A–D carry the implementation settings, hyperparameters, sensitivity analyses, and additional results that the truncated main text references but does not reproduce.

Authors’ abstract

Clinical assessments are often collected using heterogeneous assessment systems across centers and time, leading to records that mix different but related sets of measurements. This motivates harmonization beyond total-score linking. We formulate clinical harmonization at the measurement level as a bidirectional recoverability problem: given paired observations from two assessment systems, the goal is to identify which measurements can be reliably translated in both directions within an application-defined tolerance, while separating non-translatable components. We propose Bidirectional Feature Harmonization Decomposition (BFHD), a feasibility-driven framework that enforces bidirectionally coupled translation and uses feature-wise output gating to produce an explicit decomposition in the original measurement space. Experiments on synthetic data and real clinical assessment pairs show that BFHD achieves broader feasible harmonization coverage and improved subset stability compared to baselines.