This MIT course focuses on estimating answers and tackling problems through intuitive shortcuts instead of formal proofs or exact calculations. It covers guesswork techniques and heuristic strategies to quickly arrive at useful results.

• MIT's Street Fighting Mathematics (IAP 2008, taught by Scott Aaronson, J. Michael Steele, Steven G. Johnson) teaches estimation and heuristic problem-solving instead of rigorous proofs.
• Key techniques include dimensional analysis, bounding arguments, perturbation methods, and picking simpler functions to bound nasty integrals.
• The course derives results like the √N law for random walks and Stirling's formula through physical intuition and rough approximation rather than formal derivation.
• These heuristics extend beyond physics to algorithm analysis, offering quick mental shortcuts for estimating runtime or complexity (e.g., spotting O(N log N) behavior without formal analysis).

This article discusses a library of stochastic streaming algorithms designed for fast approximate analysis of big data. It highlights the library's ability to handle complex queries efficiently, reducing processing times significantly while maintaining mathematically proven error bounds. Adaptors for various platforms and languages are included to facilitate integration.

• Yahoo cut data processing times from days/hours down to minutes or seconds using these sketch algorithms.
• The library provides sketches (approximate streaming algorithms) with mathematically proven error bounds for queries like count distinct, quantiles, and graph analysis.
• It integrates with Apache Hive, PostgreSQL, and Google BigQuery, and supports Java, C++, Python, Rust, and Go.
• Built-in Theta Sketch set operators allow complex set expression calculations, outperforming traditional Include/Exclude methods.

This article explores the challenges of performing exact queries on large datasets and introduces data sketches as a solution. Sketches provide approximate answers quickly and efficiently, allowing for scalable data analysis without the need for massive storage. The piece outlines how these probabilistic structures work and their advantages in handling big data.

• Exact COUNT DISTINCT on billions of daily events can turn a simple task into a multi-hour or multi-day query nightmare
• Data sketches like HyperLogLog trade a bit of precision for massive speed and memory savings by hashing and summarizing data instead of storing every unique value
• The technique traces back to Philippe Flajolet's 1980s streaming algorithm research
• Spark and BigQuery already have sketch-based functions built in, so engineers can use them without extra libraries