LTR-ICD: A Ranking-Aware Framework for Automatic ICD Coding

AI summary

Overview

Research area: automatic ICD coding, clinical natural language processing, medical informatics, and ranking/recommender systems. Technical level: Intermediate. Scope: This paper introduces LTR-ICD, a ranking-aware framework that formulates automatic ICD coding as a combined classification and ranking task and evaluates it on the MIMIC-III dataset.

What This Paper Is About

Clinical notes contain unstructured text from patient encounters, and they are usually paired with a sequence of ICD codes. Correctly assigning and ordering these codes matters for diagnosis, billing, and research, but most prior work treats ICD coding only as an extreme multi-label classification problem and ignores code order. The paper's goal is to recommend an ordered list of ICD codes for a clinical note by combining classification with ranking.

Key Contributions

1. The paper formulates automatic ICD coding as a combined classification and ranking task, which the authors state is the first attempt to do so.
2. It proposes LTR-ICD, a language model-based framework with a classification module and a generative module that share an encoder, trained jointly through a two-stage learning process.
3. It introduces a ranking algorithm that integrates classifier and generative outputs to produce a final ordered list of recommended ICD codes.
4. It reports experiments on the MIMIC-III benchmark showing improved ranking quality and classification performance compared with previous state-of-the-art classifier models.

Main Findings

• Primary diagnosis ranking: The model's reported accuracy in correctly ranking primary diagnosis codes is ~47%, compared with ~20% for the state-of-the-art classifier.
• Main procedure code identification: LTR-ICD has a precision of ~57% in detecting the main procedure code, compared with ~43% for PLM-ICD and ~38% for GKI-ICD.
• Classification metrics: LTR-ICD achieves micro-F1 of 0.6065 and macro-F1 of 0.2904, surpassing the previous best scores of 0.6035 and 0.2741. The paper states the macro-F1 improvement from 27.41 to 29.04 indicates better handling of label imbalance and rare codes.
• Ranking over K: LTR-ICD outperforms baseline models in F1@K over important ranking positions for both diagnosis and procedure codes, meaning it places high-priority codes at higher ranks.
• MAP and NDCG: The paper reports that MAP and NDCG also show LTR-ICD's superiority at identifying and prioritizing top-ranked codes. These are macro metrics that average scores across queries.
• Cumulative gain: Using a cumulative gain approach up to rank K, the proposed model achieves superior early-ranking performance and consistently outperforms PLM-ICD and GKI-ICD for both diagnosis and procedure codes.
• Ablation study: The classifier-only module identifies relevant codes reasonably well but is weaker at early ranking positions. The generative-only module is stronger at early ranks because of its sequential nature but has lower coverage and misses relevant codes. The full model consistently outperforms both individual modules across all ranking positions, improving both ranking quality and coverage.
• Label ordering: The paper tested three orderings for diagnosis and procedure codes in the generative module. The third ordering, which mixes diagnosis and procedure codes by priority from high to low, produced the best results compared with ordered diagnosis followed by procedures, or ordered procedures followed by diagnoses.

Methodology in Plain English

The researchers reframe ICD coding as a recommendation task: given a clinical note, the system should produce an ordered list of ICD codes. They use T5, an encoder-decoder transformer model, specifically ClinicalT5 initialized with pretrained Clinical-T5-Base weights. They modify T5 by adding a label attention mechanism and a classification head on top of the encoder.

The model has two parts. The classification module predicts a set of relevant ICD codes without considering order. It uses an encoder, two label attention blocks, and multi-hot encoded labels. The generative module produces ICD codes sequentially, from most to least relevant, using the shared encoder through cross-attention. A ranking algorithm then merges the two outputs: it first takes codes from the generative predictions that also appear in the classifier predictions, then appends any remaining classifier predictions.

Training happens in two phases. In phase one, the model is trained jointly using Focal Loss for classification and cross-entropy loss for generation, with a weighting coefficient alpha of 0.02 and Focal loss gamma of 2, and a learning rate of 0.001. In phase two, the encoder and decoder are frozen, and only the classifier head is trained with Dice loss at a learning rate of 0.0001. The best model is selected by highest micro-F1 on validation data.

At inference, the generative module uses beam search with a maximum beam size of 5. Repeated labels and labels that are not ICD codes are removed. The dataset is MIMIC-III, using hospital discharge summaries with ICD-9 codes from 2001 to 2012, covering about 46,000 patients and 15 types of clinical notes. The paper uses the data splits introduced by Edin et al. (2023) and compares primarily with reproduced PLM-ICD and GKI-ICD baselines. On average, each discharge summary has 11 diagnosis codes and 4 procedure codes; the maximum is 39 diagnosis codes and 40 procedure codes.

Why This Matters

This work matters because it shifts automatic ICD coding from pure classification toward ranking-aware recommendation, which better matches how coders actually sequence codes. It also introduces ranking metrics such as MAP and NDCG to evaluate code priority, not just presence or absence.

Real-world applications:

• Medical billing and reimbursement, where the primary diagnosis code must be listed first.
• Clinical documentation and coding support, helping human coders review and prioritize suggested codes.
• Public health and clinical research, where the order of codes can affect disease trend analysis and study validity.
• Clinical decision support and patient care, where high-priority diagnoses and procedures need to be surfaced early.

Industry relevance: EHR vendors, hospital coding departments, healthcare payers, and clinical NLP tool developers could use ranking-aware ICD coding to improve coding accuracy, efficiency, and administrative workflows.

Future Directions

• Evaluate the framework on ICD-10 or other coding systems beyond the ICD-9 codes in MIMIC-III.
• Develop graded relevance or priority labels for ICD codes, since the current evaluation uses binary relevance and lacks intermediate relevance levels.
• Compare with methods such as Wang et al. (2024) and Wu et al. (2025), which the authors could not reproduce because public code or sufficient implementation details were unavailable.
• Explore combinations with agentic large language model frameworks, knowledge injection, and human-in-the-loop coding to improve rare code prediction and real-world deployment.

Target Audience

This paper benefits clinical NLP researchers, medical informatics researchers, machine learning practitioners working on healthcare text, healthcare data scientists, and industry teams building EHR or medical coding systems. It is also useful for graduate students with some background in machine learning, transformers, and information retrieval metrics who want to understand ranking-aware approaches to ICD coding.

Authors’ abstract

Clinical notes contain unstructured text provided by clinicians during patient encounters. These notes are usually accompanied by a sequence of diagnostic codes following the International Classification of Diseases (ICD). Correctly assigning and ordering ICD codes is essential for medical diagnosis and reimbursement. However, automating this task remains challenging. State-of-the-art methods treated this problem as a classification task, leading to ignoring the order of ICD codes that is essential for different purposes. In this work, as a first attempt, we approach this task from a retrieval system perspective to consider the order of codes, thus formulating this problem as a classification and ranking task. Our results and analysis show that the proposed framework has a superior ability to identify high-priority codes compared to other methods. For instance, our model's accuracy in correctly ranking primary diagnosis codes is 47%, compared to 20% for the state-of-the-art classifier. Additionally, in terms of classification metrics, the proposed model achieves a micro- and macro-F1 scores of 0.6065 and 0.2904, respectively, surpassing the previous best model with scores of 0.6035 and 0.2741.