transformer-tuning

# gpu-optimization → n-gram-embeddings → automated-research → transformer-tuning

1 link tagged with all of: gpu-optimization + n-gram-embeddings + automated-research + transformer-tuning

Click any tag below to further narrow down your results

Links

First Steps Toward Automated AI Research - Recursive

Recursive built an AI agent that autonomously proposes, implements, and validates research ideas over long horizons. On the NanoChat benchmark it cut validation loss by 0.0263 BPB versus the community best and rediscovered and combined techniques like hashed bigram/trigram embeddings, optimizer tweaks, and architecture modifications.

Last saved Jun 13, 2026 · 6 min read

automated-research + language-model-training gpu-optimization n-gram-embeddings transformer-tuning