This article explores the challenges of performing exact queries on large datasets and introduces data sketches as a solution. Sketches provide approximate answers quickly and efficiently, allowing for scalable data analysis without the need for massive storage. The piece outlines how these probabilistic structures work and their advantages in handling big data.

• Exact COUNT DISTINCT on billions of daily events can turn a simple task into a multi-hour or multi-day query nightmare
• Data sketches like HyperLogLog trade a bit of precision for massive speed and memory savings by hashing and summarizing data instead of storing every unique value
• The technique traces back to Philippe Flajolet's 1980s streaming algorithm research
• Spark and BigQuery already have sketch-based functions built in, so engineers can use them without extra libraries