On February 17, 2017, Tavis Ormandy of Google’s Project Zero was working on something unrelated to Cloudflare when he started seeing things that should not have been in the web pages he was fetching. Not garbage — secrets. Chunks of what looked like other people’s HTTPS requests, cookies, private messages, fragments that had no business being in the response to his request. He assumed at first he was misreading his own tooling. He wasn’t. He fired off a message to Cloudflare that, in the retelling, has the tone of a man who has found a very large hole and is trying to stay calm: he thought they had an active data leak, and he was right.
He called it Cloudbleed, an echo of Heartbleed from three years earlier. The name stuck because the shape was the same: a program handing out memory it had no business handing out. But Heartbleed leaked one server’s memory to whoever asked. Cloudbleed did something worse, and the reason it was worse is the whole point of this story.
One character
Start with the bug, because it is almost insultingly small.
Cloudflare’s edge servers rewrite HTML on the fly. To do that they parse it, and for years the parser was built with Ragel — a tool that takes a state-machine grammar and generates C code to walk through text. Ragel-generated parsers are fast and they are also famously easy to get subtly wrong, because you’re not writing the pointer arithmetic by hand; the tool is, from your grammar, and the tool will faithfully generate whatever you told it to.
Somewhere in that generated code, the check for “have we reached the end of this buffer?” used an equality test. In Cloudflare’s own words afterward: reaching the end of a buffer was checked with the equality operator, and a pointer was able to step past the end of the buffer. Written out, it’s the difference between == and >=. If the pointer lands exactly on the end, == catches it. If something lets the pointer skip the end by a byte — jump from just-before to just-after in a single step — then == never fires, the guard sails by, and the code keeps reading memory that isn’t part of the buffer at all.
That is a textbook buffer overrun. What made it a catastrophe is what lives just past the end of a buffer on a busy edge server: other requests. The memory the parser wandered into was heap that had recently held — or still held — data from entirely different connections. Other people’s cookies. Other people’s POST bodies. Other people’s Authorization headers. The parser scooped that up and appended it to the HTML it was rewriting, and that HTML got sent back to whoever made the request.
The bug was old; the trigger was new
Here’s the part that should feel familiar if you’ve read enough postmortems. The == was not a fresh mistake. It had been sitting in that generated parser code for a long time, harmless, because nothing ever let the pointer skip the end byte. The conditions to trigger it simply didn’t occur.
Then Cloudflare introduced a new parser, cf-html, and migrated features onto it. cf-html was better, and moving to it changed how buffers got filled and handed around between the parsing stages. That change — reasonable, well-intentioned, an upgrade — created exactly the condition the old Ragel code had never faced: input that could leave the pointer one step beyond the end. The new parser didn’t contain the bug. It woke up a dormant bug in the old one by changing the shape of what the old one was fed.
This is the recurring structure of infrastructure disasters, and I’ve written before about how Cloudflare’s own outages keep repeating it: a latent fragility that everybody’s been standing on safely for years, plus a change that alters the one condition keeping it asleep. The change looks safe in isolation. The fragility looks safe in isolation. They are only dangerous together, and nobody tests the together.
Three features ran content through the vulnerable parser chain, and each could trip the leak: Email Obfuscation, Server-side Excludes, and Automatic HTTPS Rewrites. That last one matters for the timeline, because Automatic HTTPS Rewrites launched on September 22, 2016 — which is the day the leaking actually began. It ran quietly for months. The worst stretch came February 13–18, 2017, after Email Obfuscation was migrated in a way that raised the odds. At the peak, Cloudflare estimated the bug triggered in roughly 1 in every 3.3 million HTTP requests — about 0.00003%.
Stop on that number for a second, because it’s the seductive one. One in 3.3 million sounds like nothing. It’s the kind of number you’d put in a slide to reassure people. And it is completely irrelevant to how bad this was, for a reason that has nothing to do with probability.
Why 0.00003% was a disaster anyway
Cloudflare sat in front of millions of websites. That’s the value proposition: one shared fleet of edge servers, in front of everybody, so that everybody gets the caching and the DDoS protection and the TLS termination cheaply. Shared. That word is doing enormous work.
Because the servers were shared, the memory was shared. And because the memory was shared, the data the overrunning parser scooped up wasn’t the requesting site’s own data — it was whatever had recently passed through that edge machine, from any of the millions of sites behind it. A request to a small blog could come back carrying a chunk of someone’s session on a major dating site. A request to a forum could surface a password manager’s traffic. The leak didn’t respect tenant boundaries because at the level where it happened — raw heap memory on a shared box — there were no tenant boundaries. That’s the thing multi-tenancy quietly trades away. The isolation you see in the dashboard is not the isolation that exists in RAM.
So “1 in 3.3 million requests” is the wrong frame. The right frame is: every leaked response could contain secrets from any customer, and the pool of victims was the entire customer base, not the site that happened to receive the spill. A tiny per-request probability times an astronomically large and cross-contaminated request volume, over five months, is not a small incident. It’s a slow, broad, indiscriminate dusting of the internet’s secrets across the internet’s pages.
The part that made it un-cleanable
Cloudflare’s response, once Ormandy reported it, was genuinely fast. They have a global kill switch for features, and they used it: Email Obfuscation was disabled worldwide 47 minutes after they got the details, Automatic HTTPS Rewrites about three hours after that, and full global mitigation was done in under seven hours. For a bug of this class — where the industry norm for a coordinated fix is measured in weeks or months — seven hours is a real achievement. If the story were only about the servers, it would end there, a fire put out fast.
It didn’t end there, because of who else had been reading those pages.
Search engine crawlers spend all day fetching pages and storing copies. For five months, some of the pages they fetched from Cloudflare-fronted sites came back with leaked memory baked into the HTML — and the crawlers cached those pages, secrets and all. Turning off the feature stopped new leaks instantly. It did nothing about the copies already sitting in Google’s, Bing’s, and Yahoo’s caches, quietly searchable, waiting for anyone who knew the right query. Cloudflare had to go work with the search engines to hunt down and purge cached responses containing leaked data — a cleanup with no clean end state, because you can never be sure you found every copy.
That’s the durable lesson, and it’s not “buffer overruns are bad” or even “test your parser changes,” true as both are. It’s that leaked data has a second life you don’t control. The moment a secret lands in an HTTP response, it can be cached, logged, screenshotted, scraped — copied into systems you’ve never heard of by machines doing their ordinary jobs. Your incident timeline says “resolved in seven hours.” The data’s timeline says “indefinite.” Those are different clocks, and the second one is the one that matters to the person whose password went out.
The bill for sharing
It would be easy to file Cloudbleed under “careless C” and move on. I don’t think that’s the right filing. The == was a mistake, sure, and a memory-safe language would have turned that overrun into a crash instead of a silent leak — which is a real argument, and part of why so much edge infrastructure has since moved toward Rust and sandboxing.
But the thing that turned a one-byte pointer slip into a mass secret-disclosure event wasn’t the language. It was the architecture everyone chose on purpose: put the whole internet behind a shared fleet, because that’s what makes the economics work. CDNs concentrate risk by design — we accept a single point of failure for availability because the alternative is worse. Cloudbleed is the confidentiality version of that same bargain. When the memory is shared, a bug in the shared part isn’t one customer’s bug. It’s a bug in the commons, and it spills the commons.
The overrun has been patched for years. The bargain hasn’t changed at all. We are still, all of us, running our secrets through the same handful of shared machines, and the reason we usually get away with it is not that the boundaries are strong. It’s that the pointer usually lands exactly on the end of the buffer.