This gist provides a single-file, dependency-free implementation of a GPT-style transformer, complete with a custom autograd engine, training loop using Adam, and inference routine. It trains on a list of names, demonstrating both the core algorithm and a brief benchmark discussion for a GPU-based microgpt.cu variant.

• A complete GPT training pipeline—autodiff engine, transformer architecture, and Adam optimizer—fits in under 300 lines of pure Python with zero dependencies.
• The custom Value class implements backpropagation from scratch, proving you don't need PyTorch/TensorFlow to understand how gradients flow through a transformer.
• The model is genuinely tiny (16-dim embeddings, 4 heads, 16-token context, a few thousand params) yet demonstrates every core GPT component: embeddings, RMS-norm, scaled dot-product attention, residuals, and a feed-forward MLP.
• It's explicitly a stripped-down teaching tool, sacrificing batching and GPU speed to expose the raw mechanics of training a character-level name generator in 1,000 steps.