Click any tag below to further narrow down your results
Links
Redis published benchmark results showing its memory system outperforms other production systems on LongMemEval, a standard test for long-term agent memory. The report also flags that published memory benchmarks often don't replicate when re-tested independently.
- Redis' Remis + Instruct strategy achieved 86.5% task-averaged accuracy on LongMemEval, the highest among production systems tested on GPT-4o, at lower cost than competitors
- Oracle's managed service scored slightly higher (within 0.5 points) but uses a stronger answer model
- Vertex Memory Bank's actual performance (42.4%) significantly underperformed its published results (~70%), suggesting benchmark results don't always hold up in real-world setups
Simon Willison runs Claude Fable 5 through its paces, finding it slower and pricier than Opus 4.8 but far more knowledgeable thanks to its 1 million-token context. He tests it on real-world coding tasks—upgrading a MicroPython sandbox to full CPython in WASM and adding pause-resume hooks to Datasette Agent—showing it can build complex features end-to-end.
- Fable 5 costs twice as much as Opus 4.5–4.8 ($10/$50 per million tokens vs their pricing) and runs slower, but handles every complex prompt thrown at it.
- Its 1M-token context gives it dramatically deeper recall than Opus 4.8—listing a dozen detailed open-source projects with dates versus Opus's brief list of four.
- It successfully converted a MicroPython-wasm project to full CPython in WASM, ultimately producing a working 13.9 MB wheel runnable via uv, demonstrating real end-to-end coding capability.