7 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
On November 18, 2025, Cloudflare experienced a significant outage due to a change in database permissions that led to an oversized feature file for their Bot Management system. This caused widespread HTTP 5xx errors across various services until the issue was resolved later that day. The article details the incident, its impact, and steps for future prevention.
If you do, here's more
On November 18, 2025, at 11:20 UTC, Cloudflare's network faced severe disruptions, preventing users from accessing various sites. The issue stemmed from a change in database permissions that led to a corrupted feature file for their Bot Management system. This file, which grew unexpectedly large, was distributed across the network, causing software failures due to size limitations. Initially misidentified as a DDoS attack, the problem persisted until the team stopped the faulty file's propagation and replaced it with a stable version by 14:30 UTC.
The outage affected multiple services, resulting in a spike in HTTP 5xx error codes. Core CDN services, Turnstile authentication, and Workers KV experienced significant failures. Users struggled to log into the dashboard due to Turnstile issues, while Access authentication failures compounded the problem. Although email delivery remained intact, the temporary loss of an IP reputation source impacted spam detection. Increased latency was noted as debugging systems consumed excessive CPU resources during the incident.
Cloudflare outlined the request process through their network, highlighting the role of the core proxy system in applying security measures like the Bot Management model. This model relies on a feature configuration file to evaluate traffic for automated requests. The file, which is updated frequently to adapt to changing threats, became problematic due to a change in the ClickHouse query behavior, resulting in duplicates. This incident serves as a critical learning point for Cloudflare as they aim to prevent similar outages in the future.
Questions about this article
No questions yet.