Click any tag below to further narrow down your results
+ performance
(1)
+ engineering
(1)
+ domain specific context
(1)
+ vedder
(1)
+ startups
(1)
+ inference efficiency
(1)
+ factual grounding
(1)
+ hallucination detection
(1)
+ probably
(1)
+ open source
(1)
+ proprietary lock in
(1)
+ agent scaffolding
(1)
+ llama.cpp
(1)
+ ollama
(1)
+ security
(1)
Links
PostHog shares concrete lessons from shipping 10 AI products used by 100,000 monthly users, covering when to build AI, how to structure agents, and what fails in practice. The core insight: most AI failures are about reliability and context, not capability, and you need to rethink product architecture for agents rather than bolt them on.
- Most AI product failures come from poor reliability, unclear limits, and insufficient context—not lack of capability. PostHog's install wizard improved activation 2x and conversion 5x because it automated a 2-hour task, not because it was technically impressive.
- Agents need complete tool coverage and flexible abstractions to work. Forcing agents through preset tools or leaving out endpoint coverage (like missing experiment-start when experiment-create exists) makes them abandon tasks and ask users to finish manually.
- Context matters more than reasoning power. PostHog improved wizard effectiveness more by giving agents better context than by adding multi-step planning, subagents, or better models—and model upgrades bulldoze your assumptions anyway, so don't over-invest in harnesses.
- Build shared infrastructure and let product teams own agent features. PostHog's MCP server has 5x more daily users than their AI product because it's easier to maintain and lets teams integrate agents into their own products rather than using a monolithic assistant.
New Relic surveyed 2,575 engineering leaders and found that AI has become the top driver of observability demand—but it's creating a dangerous gap. Teams are shipping AI-generated code faster than they can monitor it, leading to more production incidents even as code reviews look good on paper.
- One in four organizations deploy AI agents in production with zero monitoring, creating unmitigated operational risk when autonomous systems alter code, configs, or infrastructure without visibility.
- 62% of teams ship AI-generated code without manual line-by-line verification; 78% report more production incidents and 86% see senior engineers spending extra time fixing code, despite 94% rating the code quality high during review.
- Tool sprawl is reversing years of consolidation—organizations now use multiple AI coding tools (GitHub Copilot at 49.7%, Gemini at 40.7%, Codex at 36.6%) that generate code in incompatible formats, making source-code review unreliable and runtime observability the only trustworthy signal.
Independent processes with fixed intervals naturally converge into synchronized waves after any shared event (deploys, network partitions, cache flushes), creating traffic spikes. Adding jitter to these intervals is a one-line fix that spreads load without changing average rates or requiring coordination.
- Fixed intervals act as attractors: after any event that resets all clients simultaneously, they stay synchronized forever, creating predictable spikes instead of distributed load
- Exponential backoff without jitter keeps retry storms aligned—AWS measurements show full jitter (random delay across the entire backoff window) recovers faster than backoff alone because synchronized waves prevent the dependency from recovering
- Most systems have hidden synchronized intervals nobody thinks about: cron jobs at midnight, health checks from rolling restarts, reconnect logic after disconnects, token refresh—jitter on these costs almost nothing but cuts peak load dramatically
Snowflake interviews Tom Lane, a core Postgres developer for 25 years, about the architectural decisions that have kept the database stable and reliable. He discusses why the process-per-connection model works, how the write-ahead log handles crash recovery, and the trade-offs the project continues to navigate.
- Postgres uses isolated OS processes per connection instead of threads, which simplifies code and improves crash resistance—if one session crashes, it doesn't corrupt shared state or require a full system restart.
- The write-ahead log (WAL) lets transactions commit as soon as log data hits disk, even though table changes may still sit in shared buffers; this batching approach reduces random disk I/O compared to writing scattered changes immediately.
- Connection pooling tools like PgBouncer solve the overhead problem of managing thousands of connections, though the core project hasn't fully integrated pooling into Postgres itself.
Grab built an automated system called Data Production Issues (DPIs) that detects when certified data products fail their contracts, diagnoses root causes without manual intervention, and fixes common failures automatically. The system keeps data quality issues actionable by filtering noise, assigning problems to the right owner, and closing issues only when the underlying data actually recovers.
- DPIs use a standardized Data Health API with four error categories (upstream, platform, job, data) to route failures to the correct team without requiring platform-specific knowledge, enabling consistent incident management across Grab's heterogeneous data systems.
- The triage phase deduplicates alerts and groups related failures by root cause, preventing responders from drowning in repetitive notifications while ensuring only real contract breaches become operational issues.
- Hugo, Grab's ingestion platform, consolidates signals from multiple sources into a three-layer diagnosis workflow that automatically identifies root causes and recommends fixes, eliminating manual log inspection and shortening remediation time.
Spotify built Vedder, an AI data assistant that uses domain-specific “clusters” curated by expert teams to provide reliable SQL-generated insights. Each cluster combines relevant datasets, vetted question–SQL pairs, and docs, with automated health scoring and feedback loops to keep context accurate and scalable.
- Auto-generating question-SQL examples from query logs mostly failed—only 12.5% passed expert review, proving curated human examples beat scraped ones
- Vedder's reliability comes from domain-expert-curated "clusters" (tables + vetted SQL examples + docs), not a bigger LLM
- Health scoring tracks schema drift and broken examples so experts know which clusters need fixing as data evolves
- Adopted by 2,100+ employees for 13,000+ conversations since August 2025, showing real traction beyond a pilot
Probably raised $9 million to build an AI system that catches hallucinations and factual errors before they reach users. Their data-science tool wraps LLM outputs in a deterministic validator “mech suit,” letting it run smaller models locally while ensuring each answer matches the source data.
- Probably raised $9M from a16z to build a validator system that blocks LLM outputs unless they match source data exactly, aiming for 99.99% accuracy.
- This validation approach lets them use models "four classes weaker" than frontier LLMs, cheap enough to run on a desktop instead of a GPU farm.
- Elias argues big AI labs won't build this themselves because their revenue model benefits from users paying per interaction, including ones spent correcting errors.
This article argues that Ollama’s custom backend runs 30–70% slower than llama.cpp, locked users’ models in a proprietary format, and dropped proper attribution before grudgingly reverting. It also criticizes Ollama Cloud’s reliability and warns that the platform has strayed from its local-first, open-source roots.
- Ollama's custom ggml fork runs 30-70% slower than llama.cpp (e.g., 30 vs 52 tokens/sec on RTX 5090 with Qwen3 Coder 32B), despite reverting to upstream llama.cpp in v0.30.0-rc15
- Ollama locked models into a proprietary hashed-filename registry format between 2024 and early 2026, preventing reuse with other tools
- Ollama Cloud Pro users report up to 95% failure rates, long timeouts, broken tool calling, and unanswered support tickets, undermining its local-first pitch
- Ollama mislabeled distilled DeepSeek-R1 variants as the full 671B model and launched its desktop app without source code despite open-source claims
Anthropic’s Claude Cowork introduces live artifacts as an alternative to static dashboards. The feature is still in early testing with no formal release, and users have reported reliability and scaling challenges. Organizations will need to set up permissions, access controls, and audit trails before connecting live data sources.
- Claude Cowork is testing live artifacts that pull real-time data from databases, spreadsheets, or warehouses to generate dynamic charts and reports instead of static dashboards, replacing manual design with conversational queries.
- It's still unreleased and access-tier limited, with reported glitches like dropped queries and timeouts when scaled up.
- Live data connections require strict permissions, encryption, and audit logging to prevent leaks or manipulation of sensitive figures.
- Recommended for now only as a proof-of-concept on small datasets with tight access controls, not for mission-critical use.
The author describes a pattern of prototyping workflows with AI agents then refactoring into code-driven processes, using agents only for tasks that require human-like judgment. A security vulnerability alert system illustrates how webhooks filter and route high-priority issues, delegating owner identification to an agent and formatting alerts via a second agent for reliable Slack notifications.
- Prototyping a workflow fully inside an agent prompt (filtering, assigning, formatting all in one) broke down under load, letting high/medium severity alerts slip through undetected.
- The fix was refactoring deterministic steps (filtering, batching, data extraction) into plain code, reserving agents only for tasks needing human-like judgment, like matching people to code ownership.
- The resulting hybrid pipeline (code filters/batches → agent assigns owners via CODEOWNERS/commits → second agent formats Slack messages) runs reliably without spamming non-critical alerts.
- The author's general pattern—prototype in pure agent, then refactor toward code with only a few targeted agent calls—is framed as producing faster, cheaper, more reliable automation than prompt tinkering or building strict evals.