GitHub - microsoft/SkillOpt: SkillOpt is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts.
Short summary
SkillOpt is a Microsoft framework that optimizes AI agent skills (prompt documents) using supervised training techniques—epochs, learning rates, validation gates—without modifying model weights. It treats skill documents as trainable state and uses an optimizer model to iteratively edit them based on scored rollouts.
- SkillOpt achieves +23.5 to +24.8 point accuracy gains across six benchmarks and seven models (GPT, Claude, Codex, Qwen), with optimized skills transferring across model scales and execution environments without retraining.
- The system generates compact, deployable skill artifacts (300–2,000 tokens) that add zero inference-time overhead—skills are static documents that run against unmodified target models.
- SkillOpt-Sleep, a nightly offline self-evolution engine, lets local coding agents (Claude Code, Copilot) automatically harvest past sessions, replay tasks, and consolidate validated skills behind held-out validation gates.
Full summary
SkillOpt is a training framework from Microsoft Research that optimizes AI agent skills the way you'd optimize neural network weights — through epochs, learning rates, validation gates, and structured feedback loops. Instead of tweaking model parameters, it treats a skill document (typically 300–2,000 tokens) as the trainable artifact. An optimizer model reads scored rollouts from agent attempts and makes bounded edits to the skill: adding, deleting, or replacing instructions. Crucially, changes only stick if they improve performance on a held-out validation set. The approach adds no inference-time overhead since the final artifact is just a markdown file that runs against an unchanged base model.
The results are concrete. Across six benchmarks, seven target models (GPT, Claude, Qwen, others), and three execution environments (direct chat, Codex CLI, Claude Code CLI), SkillOpt matched or beat all 52 tested combinations. On GPT-5.5, it lifted accuracy by +23.5 points in direct chat, +24.8 inside Codex, and +19.1 inside Claude Code. The optimized skills transfer surprisingly well — they work across different model sizes, between different agent harnesses, and even to related benchmarks without retraining. Version 0.2.0 shipped in July 2026 with SkillOpt-Sleep, a nightly offline self-improvement engine that harvests past sessions, replays recurring tasks, and consolidates validated skills behind validation gates.
The framework supports multiple LLM backends (OpenAI, Azure, Claude, Qwen, MiniMax) and includes six built-in benchmarks plus a web dashboard for monitoring. Adding new backends or benchmarks follows documented contracts — chat backends need a model module that registers through common.py, and benchmarks need an adapter, data loader, and config. The code is on GitHub with full documentation, a technical blog, and a paper (Yang et al., 2026) covering the method and ablations. Installation is straightforward: `pip install skillopt`, with optional WebUI support via `pip install -e ".[webui]"`.