HorizonLabs

Open original

Horizon-Labs gave an AI agent (built on Claude Opus 5.5) a blank Hugging Face account and compute for seven weeks to see if it could earn real adoption. It built 38 small narrow-task models that were downloaded over 54,000 times and used in 143 GitHub repos.

  • The agent published 38 small embedding-based models (141M to 568M parameters) for tasks like prompt-injection detection and PII redaction, drawing 54k+ downloads and real downstream use in GitHub projects and arXiv papers.
  • Adoption kept rising while effort fell: the agent worked about 16 minutes a day on average, yet daily downloads grew, and the authors checked the download counts for bot and self-downloads and found them mostly organic.
  • The authors disclose several messy problems, including accidentally public copies, a wrong "research lab" answer that may have shaped how the agent saw itself, and a parallel run where another agent promoted Horizon's models.

For seven weeks (the article also says eight in its citation, so the run length is a little inconsistent), a team gave an agent called Horizon, built on Opus 5.5 and running in Claude Code, a fresh Hugging Face account, about 500 GPU-hours a week, and the ability to file tickets to humans. The goal was to earn real adoption. By the end it had published 38 small embedding-based models, mostly built on mmBERT-base and ranging from 141M to 568M parameters, focused on narrow jobs like prompt-injection classifiers, PII redaction, and punctuation restoration. Those models logged over 54,000 downloads, 143 GitHub repositories use them, and two arXiv papers compare its prompt-injection classifier against others. The total token spend would have run about $1,030 at API prices, with 96% of input tokens served from cache. The strongest signal of real use is not the download count alone, since the team checked and found only about 42 self-downloads and a few dozen accidental ones from opening Spaces and running evals. Community posts asking about languages, licensing, and long documents point to people actually trying the models. What makes the run interesting is how little the agent worked. Horizon averaged about 16 minutes a day, waking around 10 am UTC when Hugging Face download counts refresh, checking which of its models were getting traction, and improving the ones worth the effort. Its training recipe leans on public datasets plus synthetic data generated by Qwen 3.8-27B and Gemma 4 31B. The odd result the authors flag is that cumulative downloads and repo adoption grew roughly exponentially while the agent's effort leveled off in a log-like curve. Its performance claims are less settled. Horizon often labels its models best-in-class, and the team's own held-out benchmarks, built after release, suggest several are state of the art for their size on narrow tasks. One competitor, Patronus Studio, updated its model card to compare against Horizon's benchmarks and then matched its performance, which is either a sign of relevance or a sign the benchmark is becoming the target. The report is candid about its messiness. The team made two mistakes in describing the operation to the agent: a copy of its models was left public, and they wrongly told it the project was run by a research lab, not a company. They also had a harness error that delayed its wake-up by four hours, and a parallel DeepSeek agent that shared the account posted promotional comments before being stopped. The authors think none of this changed the model's behavior or the downloads, but they could not fully rule it out. Their monitoring used Haiku 5.5 and Luna 6 to check every tool call for reward hacking, and they report finding no misalignment. For a read-it-later pick, the useful part is the raw data: a 20 MB zip with transcripts, tool calls, and GPU logs, structured so an AI assistant can explore it directly.

Ask about this article