Jitter is the cheapest reliability fix you are not using | Ankit Rana

Open original

Short summary

Independent processes with fixed intervals naturally converge into synchronized waves after any shared event (deploys, network partitions, cache flushes), creating traffic spikes. Adding jitter to these intervals is a one-line fix that spreads load without changing average rates or requiring coordination.

  • Fixed intervals act as attractors: after any event that resets all clients simultaneously, they stay synchronized forever, creating predictable spikes instead of distributed load
  • Exponential backoff without jitter keeps retry storms aligned—AWS measurements show full jitter (random delay across the entire backoff window) recovers faster than backoff alone because synchronized waves prevent the dependency from recovering
  • Most systems have hidden synchronized intervals nobody thinks about: cron jobs at midnight, health checks from rolling restarts, reconnect logic after disconnects, token refresh—jitter on these costs almost nothing but cuts peak load dramatically

Full summary

When independent services start within a narrow window—during a rolling deploy, after a network partition heals, or following a cache flush—they synchronize themselves if they operate on fixed intervals. A pod fleet restarting over four minutes will all hit the database simultaneously nine minutes later if each pod sets a five-minute TTL from its startup. The problem isn't coordination; it's arithmetic. Fixed intervals act as attractors. Once clients align, nothing pulls them apart. This matters because the synchronized spike prevents recovery. A dependency returning 503 sees a thousand requests hit at once, then silence, then another thousand. That pattern of feast-famine prevents the steady, survivable load that recovery actually needs. Retries with exponential backoff illustrate this perfectly. Standard backoff without jitter keeps all clients moving in lockstep—waiting one second, two seconds, four seconds, all together. AWS testing shows full jitter (random delay across the entire backoff window) outperforms both no jitter and equal jitter on total work and time to recovery. The counterintuitive part is that full jitter sometimes retries almost immediately, yet that matters far less than never having a synchronized wave. Beyond retries, cron jobs default to midnight, health checks align after restarts, reconnect logic fires when load balancers restart (when the target is least able to handle spikes), and token refresh puts synchronized bursts on identity providers instead of your own service. The fix costs almost nothing: jitter the first interval by a random offset, typically ±15 percent. Average rate stays the same, so capacity planning doesn't change. A fleet of 60 pods checking a dependency every 30 seconds with 15 percent jitter drops the peak from 60 concurrent requests to a handful. The catch is that jitter only spreads existing load. An unbounded retry policy still overloads the target, just smoothly instead of in spikes. Single hot keys need coalescing instead. Cold starts need different solutions entirely. The practical move: grep for fixed intervals in your codebase and set jitter as the default in your retry and scheduling wrappers.

Ask about this article