The article compares LLMs’ frozen knowledge to the amnesiac in Memento, showing how they rely on context prompts, retrieval systems, and external memory instead of updating their own weights. It reviews in-context learning and state-space memory layers, then argues that only continual learning—letting models compress new information into their parameters after deployment—can bridge the gap to genuine, scalable understanding.

• LLMs are like Memento's amnesiac: frozen weights mean no new memories form after training, so chat histories, retrieval, and system prompts serve as external scaffolding rather than real learning.
• These workarounds fail for novel discovery, adversarial situations, or tacit knowledge that can't be captured in a prompt.
• Even massive context windows (via state space models pushing reasoning from ~20 steps to tens of thousands) just act as a giant filing cabinet—retrieving info without integrating it.
• Real knowledge acquisition requires continual learning that updates model weights post-deployment, not just non-parametric tricks.