Language Models Can Rewrite Their Own Weights

Open original

This arXiv paper claims an open 27B model can diagnose its own failures and edit its own weights, with no gradients and no training, and end up measurably more accurate. The authors frame it as "self-rewriting" and a possible step toward recursive self-improvement.

  • The model's own edit raised its accuracy 6.6 points on problems it never saw during the edit.
  • Across eight consecutive sessions it tested edits on practice problems and submitted only ones that worked, and accuracy only went up.
  • Given a damaged copy of itself, it restored that copy to over 80% of original accuracy on unseen image tasks, without reading the original weights.

The core claim is that a language model can rewrite its own weights using only its own reasoning, with no gradient updates and no training run. The authors call this "self-rewriting." In earlier work, a frontier model edited a much smaller one. Here, an open 27B model is given its own weights, its graded answers, and the ability to run itself. It diagnoses where it failed and edits the weights to fix it. The reported result is a 6.6-point accuracy gain on problems it never saw during the editing process. The authors also say the model improves across repeated sessions: it tests candidate edits on practice problems, keeps only the ones that help, and over eight consecutive sessions its accuracy rises each time. The repair experiment is the most striking part. The authors damaged a copy of the model so it solves almost nothing. Then the model, without retraining and without ever reading the original weights, edited the broken copy back to over 80% of the original's accuracy on image tasks it had not seen. That is a strong result if it holds up, because it suggests the model can recover from degradation using only forward passes. The paper also reports a scale effect within one model family: the smaller models either make no useful edit or damage themselves, and only the largest one improves and repairs itself. So the capability seems to appear only above some size threshold, and the paper does not establish where that threshold sits. The framing matters for why this is interesting. Test-time scaling, prompting, and harnesses all add cost per answer and disappear when the task ends. Training changes weights but needs gradients and a separate training run. Self-rewriting claims to write gains into the weights at no per-answer cost while using only forward passes, the same operations serving already performs. The authors present this as a step toward recursive self-improvement. The paper is an arXiv preprint with a November 2026 submission date, and its claims rest on the experiments described; a reader should check the evaluation setup, the size and diversity of the benchmarks, and whether the gains survive comparison against strong baselines before treating the self-improvement story as established.

Ask about this article