Designing Data-intensive Applications with Martin Kleppmann

Martin Kleppmann spent years building startups before he wrote Designing Data-Intensive Applications, and now he’s back with a heavily updated second edition. He began at companies like Rapportive, which LinkedIn bought, then moved into academia. In this episode of Pragmatic Engineer, Kleppmann traces how his hands-on work with real systems fed into the book’s deep dive on data models, stream processing and replication. He also breaks down why he chose to reorganize chapters, add new case studies on consensus algorithms and expand sections on failure handling in the cloud. The conversation drills into trade-offs you face when you build distributed systems today. Kleppmann explains why “scale” once meant throwing hardware at a problem, and how cloud services shift those trade-offs toward network behavior and API guarantees. He walks through reliability patterns—leader election, quorum reads, consistent hashing—and why repeatable tests matter as much as code coverage. On the engineering ethics front, he questions how much responsibility we carry for data misuse and bias when we design those systems. Toward the end, Kleppmann turns to what’s next. He argues that formal verification will gain traction in an AI-assisted world where subtle bugs can spiral out of control. He’s exploring local-first software, proving that offline-first collaboration needs fresh algorithms. And he’s researching cryptographic methods to boost transparency in supply chains without revealing secret pricing or manufacturing details. Every topic circles back to one core idea: you can’t separate theory from practice once your app handles serious data loads.