General Hill-Climbing: The One Main Loop in AI
Daniel Miessler argues that nearly all AI work follows one loop: define the ideal state, measure progress with evals, build, evaluate, and iterate. He says the main bottleneck isn't AI's ability to execute but people's ability to clearly describe what they want.
- AI failures often come from incomplete or conflicting descriptions of what the user wants, not from poor execution. The AI can only work from the written artifact, not the idea in your head.
- Evals are simple checks that score work against defined criteria, such as accuracy, format, or passing tests. Failing criteria give you specific things to improve.
- The system is meant to be general, applying to software, writing, sales, and business planning alike, with the evals themselves refined over time when a result passes but still feels wrong.
Daniel Miessler argues there is one core loop behind most AI-assisted work, which he calls General Hill-Climbing: know what you want, write it down, measure progress, do the work, then adjust and repeat. He frames the first step as the most important. His claim is that AI failures are rarely about execution anymore. They come from the human side, where the idea in someone's head gets lost or garbled on the way into a document, a prompt, or a voice note. He describes three layers: the ground-truth idea, the artifact that captures it (what he calls an Ideal State Artifact, which he says might take an hour or three to write well for a big project), and the built output, which can only be as good as the artifact. The AI never sees the original idea, only the written version, so the quality of that writing caps the result.
The second step is measurement, which Miessler ties to "Evals," a term borrowed from software development that he thinks will spread to every kind of work. An Eval is just a set of checks for whether something is good. His example is a report scored against five pass/fail criteria: facts accurate, data under a year old, no banned mentions, a required topic covered, no spelling or grammar errors. He lists a range of Eval types, from plain code checks for a required phrase or field, to AI judges comparing two outputs, to tournaments where AI pairs off twenty candidates until one wins, to running the same test ten times to see whether results hold. He also suggests training an AI judge on your own past ratings so it scores the way you do. Some criteria can be weighted above others, so a report that fails on accuracy shouldn't count as "good" even if it passes everything else.
The final step is doing the work and then using the Evals to see where it falls short, fixing the failing parts, and running the checks again until the output passes. Miessler ranks Evals second only to the ideal-state articulation in importance, and he warns that teams who skip or do Evals poorly will struggle to use AI at scale. The piece is short on case studies or measured results, so its value rests mostly on the framework itself: a practical way to structure AI work that treats clear specification and testable criteria as the real bottlenecks.