Foundation models in pathology are failing not due to size or training duration but because they are built on flawed assumptions about data scalability and generalization. Clinical performance has plateaued, as models struggle with variability across institutions and real-world applications, highlighting a need for task-specific approaches instead of generalized solutions. Alternative methods, like weakly supervised learning, have shown promise in achieving high accuracy without the limitations of foundation models.

Text-to-LoRA (T2L) is a hypernetwork that enables the instant adaptation of large language models to specific tasks using only natural language descriptions, eliminating the need for extensive fine-tuning and dataset curation. Trained on various pre-existing LoRA adapters, T2L can generate task-specific adapters in a single forward pass, demonstrating performance comparable to traditional methods while significantly reducing computational requirements and allowing zero-shot generalization to new tasks.

The article discusses the emerging role of foundation models in processing tabular data, highlighting their potential to improve data analysis and machine learning tasks. It examines the benefits of leveraging these models to enhance predictive performance and streamline workflows in various applications. Additionally, the article explores the challenges and future directions for integrating foundation models in tabular datasets.