1 link tagged with all of: world-models + sigreg + lecun + jepa
Click any tag below to further narrow down your results
Links
The article unpacks the long history of “world models,” from Schmidhuber’s 1990 proposal through Sutton’s Dyna and Ha & Schmidhuber’s 2018 implementation, to LeCun’s JEPA framework and the new LeWorldModel paper. It explains LeCun’s real contribution—predicting in latent space with SIGReg to prevent collapse—while calling out overhyped funding claims and noting limitations and prior art.
- SIGReg replaces the usual grab-bag of anti-collapse heuristics with a simple sketched Gaussianity test, projecting embeddings onto random lines and pushing them toward normality—loss is just L_pred + λ·SIGReg.
- LeWorldModel is tiny (15M params, ViT-Tiny + Transformer) and trains on one GPU in hours, with no stop-gradients, teacher networks, or pretrained backbones.
- Latent-space CEM planning hits ~1 second versus DINO-WM's 47 seconds, a 48x speedup from compression.
- The core ideas (predicting in latent space, imagining futures to plan) trace back to Schmidhuber's 1990 work and Sutton's 1991 Dyna, making LeCun's contribution more of a refinement than an invention—undercutting the hype around the funding round.