Papers › Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classification

Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classification

27 Apr 2023arXiv:2304.13998archive 2025-07-28

Thanh-Tung Nguyen, Viktor Schlegel, Abhinav Kashyap, Stefan Winkler, Shao-Syuan Huang, Jie-Jyun Liu, Chih-Jen Lin

Clinical notes are assigned ICD codes - sets of codes for diagnoses and procedures. In the recent years, predictive machine learning models have been built for automatic ICD coding. However, there is a lack of widely accepted benchmarks for automated ICD coding models based on large-scale public EHR data. This paper proposes a public benchmark suite for ICD-10 coding using a large EHR dataset derived from MIMIC-IV, the most recent public EHR dataset. We implement and compare several popular methods for ICD coding prediction tasks to standardize data preprocessing and establish a comprehensive ICD coding benchmark dataset. This approach fosters reproducibility and model comparison, accelerating progress toward employing automated ICD coding in future studies. Furthermore, we create a new ICD-9 benchmark using MIMIC-IV data, providing more data points and a higher number of ICD codes than MIMIC-III. Our open-source code offers easy access to data processing steps, benchmark creation, and experiment replication for those with MIMIC-IV access, providing insights, guidance, and protocols to efficiently develop ICD coding models.

PaperPDFConference PDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

thomasnguyen92/MIMIC-IV-ICD-data-processing officialmentioned on GitHubSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Multi-Label Classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Medical Code Prediction MIMIC-IV-ICD-10-full MSMN Macro-AUC 97.07 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full MSMN Macro-F1 5.42 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full MSMN Micro-AUC 99.61 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full MSMN Micro-F1 55.91 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full MSMN Precision@8 67.66 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full Joint LAAT Macro-AUC 93.64 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full Joint LAAT Macro-F1 5.71 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full Joint LAAT Micro-AUC 99.27 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full Joint LAAT Micro-F1 55.89 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full Joint LAAT Precision@8 66.89 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full LAAT Macro-AUC 92.96 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full LAAT Macro-F1 4.47 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full LAAT Micro-AUC 99.14 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full LAAT Micro-F1 55.40 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full LAAT Precision@8 66,97 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full PLM Macro-AUC 91.85 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full PLM Macro-F1 4.90 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full PLM Micro-AUC 99.02 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full PLM Micro-F1 56.95 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full PLM Precision@8 69.47 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full CAML Macro-AUC 89.91 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full CAML Macro-F1 4.07 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full CAML Micro-AUC 98.79 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full CAML Micro-F1 52.67 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD-10-full CAML Precision@8 64.43 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 MSMN AUC (Macro) 93.60 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 MSMN AUC (Micro) 95.61 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 MSMN F1 (macro) 69.01 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 MSMN F1 (micro) 74.15 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 MSMN Precision@5 65.16 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 PLM-ICD AUC (Macro) 93.37 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 PLM-ICD AUC (Micro) 95.69 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 PLM-ICD F1 (macro) 70.31 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 PLM-ICD F1 (micro) 73.27 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 PLM-ICD Precision@5 64.57 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 Joint LAAT AUC (Macro) 93.39 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 Joint LAAT AUC (Micro) 95.57 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 Joint LAAT F1 (macro) 68.41 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 Joint LAAT F1 (micro) 72.85 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 Joint LAAT Precision@5 64.49 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 LAAT AUC (Macro) 93.21 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 LAAT AUC (Micro) 95.49 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 LAAT F1 (macro) 68.15 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 LAAT F1 (micro) 72.56 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 LAAT Precision@5 64.39 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 CAML AUC (Macro) 91.05 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 CAML AUC (Micro) 93.18 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 CAML F1 (macro) 64.30 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 CAML F1 (micro) 67.56 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD10-top50 CAML Precision@5 59.58 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full MSMN F1 Macro 13.94 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full MSMN F1 Micro 61.15 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full MSMN Macro AUC 96.79 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full MSMN Micro AUC 99.56 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full MSMN Precision@8 68.89 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full PLM-ICD F1 Macro 14.40 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full PLM-ICD F1 Micro 62.45 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full PLM-ICD Macro AUC 96.61 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full PLM-ICD Micro AUC 99.53 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full PLM-ICD Precision@8 70.34 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full Joint LAAT F1 Macro 14.17 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full Joint LAAT F1 Micro 60.37 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full Joint LAAT Macro AUC 95.57 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full Joint LAAT Micro AUC 99.49 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full Joint LAAT Precision@8 67.46 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full LAAT F1 Macro 13.12 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full LAAT F1 Micro 60.31 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full LAAT Macro AUC 95.18 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full LAAT Micro AUC 99.47 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full LAAT Precision@8 67.47 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full CAML F1 Macro 11.06 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full CAML F1 Micro 57.28 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full CAML Macro AUC 93.45 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full CAML Micro AUC 99.29 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-full CAML Precision@8 64.91 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 MSMN AUC Macro 95.13 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 MSMN AUC Micro 96.46 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 MSMN F1 Macro 71.85 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 MSMN F1 Micro 75.78 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 MSMN Precision @5 62.60 #1 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 PLM-ICD AUC Macro 94.97 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 PLM-ICD AUC Micro 96.41 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 PLM-ICD F1 Macro 71.35 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 PLM-ICD F1 Micro 75.46 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 PLM-ICD Precision @5 62.44 #2 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 Joint LAAT AUC Macro 94.92 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 Joint LAAT AUC Micro 96.31 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 Joint LAAT F1 Macro 69.93 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 Joint LAAT F1 Micro 74.33 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 Joint LAAT Precision @5 61.95 #3 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 LAAT AUC Macro 94.88 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 LAAT AUC Micro 96.29 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 LAAT F1 Macro 69.99 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 LAAT F1 Micro 74.46 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 LAAT Precision @5 62.01 #4 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 CAML AUC Macro 93.07 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 CAML AUC Micro 94.05 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 CAML F1 Macro 65.33 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 CAML F1 Micro 69.23 #5 of 5 Archive leaderboard report
Medical Code Prediction MIMIC-IV-ICD9-top50 CAML Precision @5 58.64 #5 of 5 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections