1 link tagged with all of: algorithms + sketches + streaming
Click any tag below to further narrow down your results
Links
This article discusses a library of stochastic streaming algorithms designed for fast approximate analysis of big data. It highlights the library's ability to handle complex queries efficiently, reducing processing times significantly while maintaining mathematically proven error bounds. Adaptors for various platforms and languages are included to facilitate integration.
- Yahoo cut data processing times from days/hours down to minutes or seconds using these sketch algorithms.
- The library provides sketches (approximate streaming algorithms) with mathematically proven error bounds for queries like count distinct, quantiles, and graph analysis.
- It integrates with Apache Hive, PostgreSQL, and Google BigQuery, and supports Java, C++, Python, Rust, and Go.
- Built-in Theta Sketch set operators allow complex set expression calculations, outperforming traditional Include/Exclude methods.