More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
# Cross-Architecture Proxy Tuning for Clinical LLMs
The paper tackles a real problem in healthcare AI: every time a new general-purpose language model comes out, hospitals and clinics have to spend massive computational resources retraining it on clinical data from scratch. This creates a lag where cutting-edge models can't be deployed in clinical settings without months of expensive adaptation work. The researchers propose CAPT (Cross-Architecture Proxy Tuning), a method that lets you combine an older clinical model with a new general-purpose model at inference time—no retraining needed.
The key innovation is handling models with completely different vocabularies and architectures. Previous ensemble methods like proxy tuning require models to share the same tokenizer, which locks you into old architectures. CAPT uses contrastive decoding to selectively pull clinical knowledge from the legacy model while letting the newer general model handle reasoning and fluency. When generating text, the system amplifies tokens where the clinical model is confident and the general model isn't, essentially letting each model contribute where it's strongest. On six clinical tasks—both classification and text generation—CAPT beat UniTE (the previous best ensemble method) by 17.6% on average and proxy tuning by 41.4%.
The results matter beyond benchmark numbers. Physician case studies showed CAPT outputs had more precise clinical terminology and better clinical reasoning. Token-level analysis revealed the method actually works as intended: clinical tokens (drug names, symptoms, procedures) come from the clinical model, while structural and formatting tokens come from the general model. This is especially valuable for resource-constrained hospitals that can't afford to continuously retrain models but want to adopt new capabilities as they emerge. The code and datasets are public, which means institutions can actually implement this without building it from scratch.
Questions about this article
No questions yet.