This article discusses the advancements in on-device language models, highlighting their advantages in latency, privacy, cost, and availability. It examines the constraints of mobile devices and explores effective strategies for building smaller, efficient models that can still perform complex tasks.

A collection of on-device AI primitives for React Native is available, supporting low-latency inference without server costs and ensuring data privacy. The toolkit includes features such as text generation, embeddings, transcription, and speech synthesis, all optimized for Apple devices and compatible with the Vercel AI SDK. Additionally, users can run popular open-source language models directly on their devices using MLC's optimized runtime.