Software is slow because of bad design, not slow languages

Open original

Rewriting programs in C or Rust rarely makes them much faster, because most real software spends its time waiting on I/O or calling optimized libraries already. The author argues the big speedups come from redesigning what a program does, not from changing its language.

  • Most CRUD apps are I/O-bound: if half a request's time goes to database queries, even infinitely fast code caps the gain at about 2x (Amdahl's law).
  • Slow languages like Python already offload heavy work to C through libraries such as NumPy and cmark-gfm, so rewriting rarely helps the hot path.
  • Bryan Cantrill's Rust rewrite of a C program ran 30% faster, not slower, because of a data structure choice (B-tree over binary tree) rather than the language.
  • Real gains come from caching slow calls, moving work async, fixing N+1 queries, and skipping unneeded operations, none of which depend on language speed.

Most software isn't slow because of its language, and rewriting it in C or Rust usually won't help much. The core argument is that real programs spend most of their time waiting, not computing. Take a typical CRUD website backed by a database: if half the request time goes to database queries, rewriting the rest in C caps your gain at roughly 2x, and realistically closer to 1.5x. The author points to curl as an extreme case: you could make its internal code 100x faster and barely notice, because it spends nearly all its time waiting on the network. Where a program is genuinely CPU-bound, slow languages usually already hand the heavy lifting to fast C libraries. Python's machine learning stack is the standard example, since its matrix math runs in optimized C or assembly anyway. The more interesting claim is that performance comes from design, and the author backs it with experience and an anecdote. Bryan Cantrill's 2018 attempt to port a carefully tuned C program to Rust failed on the borrow checker, so he wrote a naive Rust version instead, expecting a 20% slowdown. It came out 30% faster, largely because Rust's standard library uses B-trees, which are more cache-friendly than the binary search trees in the C code. The author's own list of big wins is revealing: caching slow external calls, moving work into background jobs, running operations in parallel, collapsing N+1 query patterns into one SQL query, swapping a slow query for a fast one, and fetching only the data a paginated page actually displays. None of these depend on the language. He also argues that a high-level language can make these problems easier to spot, since you can read what the program is doing at a glance. The author is careful about where his view could be wrong. Commenters on Lobsters pushed back that CRUD apps are not really I/O-bound, since database queries take around 5ms while full responses take around 500ms, and they noted that browser layout engines, 3D renderers, and word processors are CPU-bound. He concedes the second point but argues such programs make up a small share of real-world code, perhaps five percent of the number of CRUD programs, even if they dominate programming discussions. He also accepts that language ecosystems affect performance, but folds that into design, on the grounds that some languages make fast designs more natural. His practical advice for anyone using AI tools is to have the agent profile code, find hotspots, and propose structural changes, reserving rewrites in faster languages for the rare hotspot that is truly CPU-bound and can't be restructured.

Ask about this article