Click any tag below to further narrow down your results
Links
This article explains how flattening structured JSON data into natural language improves vector search performance. It details the challenges of tokenization and attention mechanisms in raw JSON, demonstrating that a simple preprocessing step can enhance retrieval metrics significantly.
TOON is a compact format designed for encoding JSON data, making it easier for large language models to process. It combines YAML's structure with a CSV-like layout to reduce token usage while maintaining accuracy. While effective for uniform arrays, it's less suitable for deeply nested data.