6 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
The author reflects on a decade in observability, highlighting the rampant waste in data and the vendor's lack of support for cost management. They reveal that many companies experience significant data waste, leading to inflated bills and operational challenges. The article argues for a shift in how observability is approached, emphasizing understanding over sheer volume of data.
If you do, here's more
The author reflects on a decade in the observability industry, starting from their founding of a logging platform, Timber, which evolved into Vector. Despite initial optimism, they express frustration over the industry's failure to address persistent cost issues. Many engineering teams struggle with rising costs from observability tools, often feeling like they're policing expenses rather than benefiting from the insights these tools provide. Engineers frequently encounter unexpected costs tied to log lines or metric tags, leading to stress and pressure during renewal periods. Even when they seek help from vendors, they find little support; vendors often deflect responsibility, claiming the data belongs to the customers.
The author zeroes in on a critical question: how much of observability data is waste? They suggest that this issue is the crux of the industry's problems, pointing out that cost is the primary concern, overshadowing other challenges. While vendors have the ability to help, they often choose to focus on profit over customer success. After leaving Vector, the author aimed to answer the waste question by analyzing customer data, discovering an average of 40% waste across various services. This figure highlights a significant disconnect between the data engineers generate and its actual utility.
The findings led to a system that automated the identification of waste in logs. By refining the process, teams were able to clean up their logging practices and reduce costs without recklessly dropping data. The outcome was not just financial savings; the clarity around data worth keeping simplified pipelines and improved overall observability. The author emphasizes that the complexity and challenges in observability stem largely from managing unnecessary data, which distracts teams from their core objectives.
Questions about this article
No questions yet.