Each one is a failure nobody was measuring, with a fix attached and, where the code is public, a repo you can run.
Temporal OOD audit 路 clinical NLP
A model that scores 0.73, then zero
0.73 → 0.00
BiomedBERT F1 on post-2019 FDA drug interactions
A drug interaction classifier holding 0.7273 F1 in distribution collapsed to
exactly zero on a 600-pair post-2019 out-of-distribution set,
reproduced across four seeds. It did not degrade. It flatlined. A name-only
in-distribution control reproduced the same 0.0000, which pins the cause on
structural dependence on annotated sentence context, not drug novelty. I recovered
0.2640 F1 with KnowledgeRAG-DDI, a hybrid BM25 and FAISS retrieval probe over FDA
labels with leakage-controlled indexing and reciprocal rank fusion.
IEEE ICDM 2026 路 accepted
Five models, wrong in unison
19.9×
more unanimous-wrong answers than chance predicts
Across 4,089 USMLE and MedMCQA questions, five models (Llama-3-8B, Qwen-2.5-7B,
Gemma-3n, GPT-4o, DeepSeek-V3) agreed on a wrong answer 24.6% of the time
against a 1.23% chance baseline. A distractor shuffle isolated the cause as content
driven, not positional. The resulting parameter-free routing rule flags these cases
at 83.0 to 96.1% precision (96.1% on MedQA, 83.0% on MedMCQA) for
6.7 to 8.0% human review cost, holding 77.5% under a held-out frontier-model
substitution.
Fairness benchmark 路 under review, EMNLP
DART: dialect robustness in essay scoring
480 pairs
six U.S. English dialect families, plus 160 controls
A controlled benchmark validated by a three-gate pipeline (length drift,
DeBERTa-v3 NLI entailment, regex feature presence) at a 92.71% pass rate.
Mixed-effects regression exposed directional bias against African American English
in a fine-tuned scorer (Pearson r = 0.829, QWK = 0.714) as the only family with
significant negative drift (β = −0.029, p < 0.001), concentrated at
the equalised-odds decision threshold. Chain-of-thought scaffolding
amplified the drift across three LLM judges rather than reducing it.
Multi-omic fusion 路 real patient cohort
SGOF: when more data makes it worse
−0.149 → 0.012
AUROC drop under an adversarial modality: MOGONET vs. SGOF
On a real 111-patient antidepressant response cohort, adding an
uninformative 258,782-SNP genotype layer dragged four fusion families below the best
single modality (methylation alone, AUROC 0.692), collapsing naive concatenation to
0.334. I attributed it to uninformativeness with a synthetic noise control, then
built Signal-Gated Omic Fusion, which weights modalities by held-out validation
signal and held at 0.664 (paired-bootstrap change 0.012, 95% CI
[−0.035, 0.059], not significant) where MOGONET dropped 0.149. The claim is
stability under a bad modality, not beating the best single one.