Click any tag below to further narrow down your results
Links
Kestra 2.0 splits into a control plane and data plane, letting workers connect via gRPC instead of requiring direct database access, which means you can deploy them across regions and networks. The release also adds AI agent integration, a revamped no-code editor, and decouples the queue from the database for simpler configuration.
- Workers now connect to the control plane over a single outbound gRPC stream instead of needing direct database access, enabling deployment in different regions, clouds, or restricted networks without opening inbound ports.
- Queue and repository are now independent, so you pick them separately (Postgres, Kafka, Redis, or AMQP) instead of being locked into paired options; this simplified the codebase to one implementation instead of multiple backends.
- MCP Tool Trigger exposes any existing flow as a callable tool for AI agents like Claude, skipping the need for custom integration layers.
- AI Copilot is now a persistent sidebar with three modes (Edit, Plan, Ask) that refines flows iteratively rather than generating from scratch, and reads your namespace configuration to ground suggestions.
Independent processes with fixed intervals naturally converge into synchronized waves after any shared event (deploys, network partitions, cache flushes), creating traffic spikes. Adding jitter to these intervals is a one-line fix that spreads load without changing average rates or requiring coordination.
- Fixed intervals act as attractors: after any event that resets all clients simultaneously, they stay synchronized forever, creating predictable spikes instead of distributed load
- Exponential backoff without jitter keeps retry storms aligned—AWS measurements show full jitter (random delay across the entire backoff window) recovers faster than backoff alone because synchronized waves prevent the dependency from recovering
- Most systems have hidden synchronized intervals nobody thinks about: cron jobs at midnight, health checks from rolling restarts, reconnect logic after disconnects, token refresh—jitter on these costs almost nothing but cuts peak load dramatically
American Express redesigned their payment platform using cell-based architecture—independent, self-contained processing units that isolate failures and keep transactions moving even when parts of the system fail. The system routes transactions deterministically to cells holding the right data, avoiding cross-cell dependencies during payment processing.
- Cells are failure boundaries, not functional units: each cell contains everything needed to process a transaction (microservices, databases, DNS) so problems stay contained and don't cascade across the platform.
- Data strategy varies by change frequency: immutable and semi-static data gets pushed to all cells before transactions arrive, while dynamic data triggers deterministic routing to send transactions to cells holding current state.
- The Global Transaction Router enforces cell isolation by being the only path between cells and external systems, kept deliberately simple to avoid becoming a bottleneck—it just parses messages and routes, leaving all business logic to the cells.
Email became one of computing's most durable interfaces because it was the only truly universal platform available—developers used it to distribute software, manage tasks, and control systems without needing to build separate apps or support multiple operating systems. The article traces how this hack, from 1985's Netlib to modern tools like Trello and Readwise, persists because email solved a real problem: letting users interact with software without friction or commitment.
- Netlib (1985) distributed mathematical software via email commands, hitting 4,000 downloads monthly—proof that email worked as a functional app store before the web existed.
- Email eliminated platform fragmentation: when developers faced supporting Unix variants, Mac, Windows, and other systems, email offered a single interface that worked everywhere.
- The pattern stuck because email solved two sides of the problem simultaneously—users didn't need new accounts or apps, developers didn't need to build separate interfaces for each platform.
This article traces how blockchain didn't emerge from nowhere but instead synthesized decades of work across distributed systems, cryptography, and economics. a16z crypto assembled interviews with Turing Award winners and Nobel laureates—including Leslie Lamport, Barbara Liskov, Ron Rivest, and others—who explain how their foundational research on Byzantine fault tolerance, consensus algorithms, cryptography, and market design eventually became core components of blockchain systems.
- Blockchain's core components (Byzantine fault tolerance, consensus, public-key crypto, zero-knowledge proofs, auction/market design) were all developed decades earlier for unrelated problems, then repurposed.
- Lamport's algorithm was debugged by an editor, which pushed him toward rigorous proofs for concurrent systems.
- Noam Nisan built theoretical tools he was certain had no practical use, only to see them become central to blockchain scaling years later.
- Goldwasser argues understanding the narrative of a problem has to come before the math in cryptography research.
This article outlines how Honeycomb’s observability platform handles massive, distributed systems by shortening time-to-understanding, reducing alert fatigue with SLOs, and consolidating legacy tools. A Forrester TEI study reports a 296% ROI over three years, $2.68 M in incident-related savings, and a break-even point under six months.
- Forrester TEI study found 296% ROI over three years with break-even in under six months, driven by ~$2.68M in incident-related savings and $4.43M NPV.
- A travel-booking company cut observability costs 16% by replacing legacy tools with Honeycomb and using BubbleUp anomaly detection to monitor multi-leg reservations.
- Another customer caught a search-cluster slowdown in seconds and scaled capacity before customers noticed, calling it "night-and-day" versus their old tooling.
- Honeycomb offers a no-signup public sandbox for hands-on testing before engaging sales for a custom demo.
This course covers core concepts of concurrency control—threads, locks, transactions, and crash recovery—in the first half, then shifts to distributed systems topics like network models, clocks, replication, consensus, and fault tolerance. It lists lecture topics, objectives, prerequisites, and recommended readings for a Part IB CST Michaelmas module led by Dr. Martin Kleppmann.
- Kleppmann's 16-lecture Part IB course splits evenly: 8 lectures on concurrency control, 8 on distributed systems.
- The concurrency half moves from basic threads/locks through bakery algorithm, semaphores, monitors, and message-passing (actors, CSP) to ACID transactions, 2PL, optimistic concurrency, and crash recovery.
- The distributed half covers network/failure models, physical and logical clocks, replication and consensus (Raft, FLP impossibility, 2PC, CAP), ending with case studies on CRDTs and Google Spanner's TrueTime.
- Core texts backing the course are Tanenbaum's Modern Operating Systems, Goetz's Java Concurrency in Practice, and Kleppmann's own Designing Data-Intensive Applications.
Martin Kleppmann discusses his journey from startups to academia, and the new edition of his book Designing Data-Intensive Applications. They cover trade-offs in modern infrastructure, cloud scalability, distributed system challenges, and emerging topics like formal verification and local-first software.
- Kleppmann rewrote and reorganized Designing Data-Intensive Applications' second edition to add consensus algorithm case studies and deeper cloud failure-handling coverage
- Cloud infrastructure has shifted the core scaling trade-offs from raw hardware capacity to network behavior and API guarantees
- Kleppmann is now researching local-first software algorithms for offline-first collaboration and cryptographic methods for supply-chain transparency that preserve pricing/manufacturing secrecy
- He predicts formal verification will become more important as AI-assisted coding introduces subtle bugs that are hard to catch otherwise
In this Pragmatic Engineer episode, Martin Kleppmann walks through updates in the second edition of Designing Data-Intensive Applications and shares how his LinkedIn experience shaped the book’s core concepts. He breaks down trade-offs in multi-region and cloud architectures, explains why replication still matters more than sharding, and predicts a rise in formal verification and local-first software.
- Second edition of DDIA cuts MapReduce coverage in favor of Spark and Flink, reflecting how the field has moved on
- Manual sharding has become rare thanks to bigger hardware and managed services, making replication the fault-tolerance skill every team now needs
- Kleppmann predicts LLMs will make formal verification mainstream by automating proof generation, shifting the bottleneck to human review
- Architecture decisions like multi-region or multi-cloud are business trade-offs between risk and cost, not universal best practices