More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Deep inside the latest papers from NVIDIA, Anthropic, Google and others lies a push for what Jose Crespo calls “Geometric AI”—a shift away from today’s flat-transformer models toward architectures that handle meaning as a curved space. Crespo argues that transformers, with their positional encodings and attention layers, treat context the wrong way: they force millions of examples into memory instead of letting the model’s geometry adapt to shifting semantics. He illustrates with “justice” and “law”—words that sit close together in ordinary use but drift apart under different political regimes. No amount of extra computation can fix that; you need a structure that understands context as the shape of the ground beneath the words, not just another input token.
Crespo points readers to several recent research papers they’ve probably missed. Anthropic’s “When Models Manipulate Manifolds” shows its own LLMs twisting into curved manifolds during inference—even though the company trained them on flat-space math. Months earlier, Anthropic quietly admitted that its published mathematical framework was wrong and that real progress requires geometric features. Then there’s “The Curved Spacetime of Transformer Architectures,” co-authored by ex-Google, Apple and Cohere researchers. They map query-key attention onto a Riemannian metric, with value vectors undergoing parallel transport across tokens—literally treating transformer layers like paths through curved space.
That body of work contradicts the public narrative. While CEOs and marketing decks promise bigger GPU farms and ever-larger transformer flavors, the smartest teams are building the geometry these models crave. In practice, that means fewer training examples, less brute-force memorization and more elegant math borrowed from general relativity. Crespo warns that until companies adopt curved embeddings and profunctor-style operations, their real-world systems will keep breaking when contexts shift or stakes rise.
Questions about this article
No questions yet.