What Jobs Are Made Of

Open original

Thomas Wolf, a startup founder, argues that AI's strong benchmark scores don't match its limited economic impact because real work depends on judgment and agency, which benchmarks rarely measure. He ties this to a drop in entry-level hiring, since execution is the part of junior work AI handles best, and that junior work is where people usually build the judgment that later makes them senior.

  • A Stanford analysis found employment for workers aged 22–25 in the most AI-exposed occupations fell about 6% between late 2022 and mid-2025, while older workers in those jobs gained 6–9%.
  • Models now solve around 75% of real-world coding tasks on SWE-bench, yet the Remote Labor Index shows the strongest agents succeed on only about 2.5% of paid freelance projects.
  • Wolf's hiring framework splits workers into execution, judgment, and agency; AI is advancing fastest on execution, so the entry layer gets squeezed first, weakening the pipeline for future senior people.

A Stanford analysis from summer 2025 (Brynjolfsson, Chandar and Chen) found that employment for workers aged 22 to 25 in the most AI-exposed occupations fell about 6% between late 2022 and mid-2025, while employment for older workers in those same jobs rose 6 to 9%. Thom Wolf, a startup founder, uses that data as a jumping-off point for a personal reflection: he's reminded of a 2010 job interview in Paris where he was rejected for lacking "experience," despite knowing the tools. The timing he points to is ChatGPT's release in fall 2022, though he's careful to note that correlation isn't causation. He also lays out the capability side: Claude Opus 4.5 solves roughly 75% of real-world coding tasks on SWE-bench, Gemini 3 and GPT-5 reached gold-medal level on science Olympiads, and ChatGPT is approaching a billion weekly users. The gap he's interested in is between those benchmark numbers and the economic picture, which he describes as muted. AI's effect on GDP is small, many generative AI pilots fail to deliver sustained value, and on the Remote Labor Index, which tests agents on paid freelance work, even the best systems succeed only a small fraction of the time (about 2.5% for ManusAI). The usual explanations are organizational inertia and not yet crossing some capability threshold, but Wolf thinks both framings treat work too narrowly as task execution. Drawing on his experience hiring for his own startup, he argues that good workers combine three things: execution (doing the task well), judgment (understanding why it matters in a broader goal), and agency or taste (knowing what to do next, what to avoid, and when to stop). Execution is easy to test. The other two are not, and they matter most when problems are ill-defined or the right move is to question the task itself. That framing explains why entry-level roles are hit first. Early-career jobs are mostly execution, and AI is improving fastest at exactly that layer. Wolf's concern is long-term: judgment and agency are partly learned through execution-heavy work, so if the entry rung erodes, the pipeline for future senior people erodes with it. He also suggests the same logic explains the limits on automating longer-horizon work. The hard part isn't generating code or text in isolation but fitting it into a wider team context, interpreting vague requirements, and making trade-offs. He closes by arguing that evaluations need to measure these cross-team, non-static qualities if we want a realistic read on AI's economic potential, since most real jobs aren't fixed rule-following in stable environments.

Ask about this article