@mmastrac - Jev Pairs EmbeddingGemma2
@mmastrac on X Matt Mastrac is wiring Google DeepMind's EmbeddingGemma2 into a fast, trainable /v1/systemone endpoint called Jev. He claims it runs in about 20ms on text and 100ms on images, and that a model this small can punch above its weight with the right harness.
- The endpoint answers in roughly 20ms for text and 100ms for images.
- The model can be trained in real time and can distill knowledge from a larger upstream teacher model.
- It is multi-modal by default, handling both text and images.
Matt Mastrac, who has been posting about a project called Jev, says he's built a way to run Google DeepMind's EmbeddingGemma2 as a fast, real-time endpoint he's calling /v1/systemone. The headline numbers: about 20ms per request for text and around 100ms for images. Those latencies are the reason the post exists. A model that quick can sit in the hot path of an application instead of being a batch job or an expensive call out to a large model.
Three capabilities stand out. The system can be trained in real time, which means it adapts as new data comes in rather than needing a separate offline training run. It can distill from an upstream teacher model, so a larger model's behavior gets compressed into this smaller embedding model. And it is multi-modal by default, handling both text and images without bolting on a separate pipeline. Mastrac's point is that an embedding model of this size can punch well above its weight when wrapped in the right harness, and he seems more impressed by the harness than by the base model itself.
The post is short on detail. There's no benchmark against other embedding models, no accuracy figures, and no description of what the harness actually does. Treat the speed claims and the distillation story as the author's own results until there's a repo, a write-up, or independent numbers to check them against.