6 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
LinkedIn replaced its outdated FollowFeed system with FishDB, a new retrieval engine built in Rust. FishDB improves memory efficiency, reduces hardware usage, and enhances query capabilities, addressing the limitations of its predecessor. The article details the architecture, migration challenges, and the benefits of using Rust for performance.
If you do, here's more
LinkedIn replaced its decade-old FollowFeed system with FishDB, a new retrieval engine built in Rust. FollowFeed struggled with scalability and usability, leading to inefficiencies like 18% of heap memory wasted on Java object overhead and an average of 1.7x content duplication per document. Tail latency issues arose due to Java's unpredictable garbage collection, which hindered performance. FishDB aims to address these problems by improving efficiency, cutting hardware usage by 50%, and offering more flexible APIs.
FishDB employs a scatter-gather architecture, utilizing a broker layer to distribute requests across multiple shards. Data is partitioned by a user-defined key, with a current setup of 48 partitions and 16 replicas. For indexing, FishDB combines offline batch processing and real-time stream processing to maintain an up-to-date index. This structure allows easy backfilling of data and schema evolution, addressing previous challenges with FollowFeed.
The query capabilities of FishDB include a functional programming style, supporting complex filtering and ranking logic. The initial syntax allows users to compose queries using higher-order functions, making it adaptable for various business needs. This flexibility is essential for LinkedIn as it evolves its content delivery, ensuring that the feed remains relevant and efficient for its users.
Questions about this article
No questions yet.