This article introduces Mixture-of-Recursions (MoR), a framework that enhances the efficiency of language models by combining parameter sharing and adaptive computation. MoR dynamically adjusts recursion depths for individual tokens, improving memory access and reducing computational costs while maintaining model performance. It shows significant improvements in validation perplexity and few-shot accuracy across various model sizes.

This article discusses the advancements in on-device language models, highlighting their advantages in latency, privacy, cost, and availability. It examines the constraints of mobile devices and explores effective strategies for building smaller, efficient models that can still perform complex tasks.

The article serves as an introduction to VLLM, a framework designed for serving large language models efficiently. It discusses the benefits of using VLLM, including reduced latency and improved resource management, making it suitable for production environments. Key features and implementation steps are also highlighted to assist users in adopting this technology.

The article discusses the phenomenon that shorter tokens in language models tend to have a higher likelihood of being selected in various contexts. It explores the implications of this tendency for understanding how language processing works in computational models. Additionally, the author examines how the length of tokens can affect the efficiency and accuracy of these models.

The paper introduces the Chain of Draft (CoD) paradigm, which enables Large Language Models (LLMs) to generate concise intermediate reasoning outputs, mimicking human draft strategies. By focusing on essential information and reducing verbosity, CoD achieves comparable or superior accuracy to Chain-of-Thought prompting while utilizing significantly fewer tokens, thus lowering costs and latency in reasoning tasks.