Cloudflare has patched a vulnerability in its Containers product that could have allowed a paying customer to pull residual data out of disk storage blocks previously used by a different tenant. The company disclosed the issue on September 24, three weeks after security researcher Oren Yomtov from the firm Accomplish filed a report through Cloudflare's HackerOne bug bounty program.
The flaw was rooted in how Cloudflare configured the Linux storage subsystem underpinning its container infrastructure. Cloudflare Containers run each workload inside a dedicated virtual machine powered by the Firecracker virtual machine monitor. Each VM gets a writable root disk backed by Linux device mapper thin provisioning, known as dm-thin, a storage technology that allocates physical disk space on demand rather than upfront. When a container's thin volume was deleted, the physical 64 KiB blocks it had occupied were handed back to a shared pool that served workloads from multiple customer accounts.
The problem was a single configuration option: `skip_block_zeroing`. With this flag set, dm-thin does not wipe a block before reassigning it. That is a performance trade-off operators sometimes make deliberately, but in a multi-tenant environment the consequences were significant. A freshly assigned block would carry the previous tenant's data intact unless the incoming workload happened to overwrite every byte of it.
Yomtov and his team worked out a way to exploit this behavior without needing any privileged access. A tenant with a standard Workers Paid account could open their container's raw root disk at `/dev/vdc` and identify regions that the guest ext4 filesystem had marked as free space. Writing a small 4 KiB block into a 64 KiB-aligned free region would force dm-thin to pull a physical block from the shared pool. Because zeroing was disabled, only the 4 KiB the attacker wrote got replaced. The remaining 60 KiB stayed exactly as the previous owner had left it. A subsequent raw-device read could then pull those bytes out.
What the researchers found across production runs was striking in scope. They tested the technique across 24 placements and found residual data on 18 of them, across 20 of 22 underlying nodes and spanning four continents. The recovered material included directory structures, database pages, and structurally complete SQLite databases. Using ext4's `metadata_csum` checksum feature, the team was able to confirm that recovered directory blocks did not originate from their own test filesystem. Across six placements they identified 2,700 distinct foreign directory inodes. All proof-of-concept materials submitted to Cloudflare were scrubbed of third-party identifiers and content values, and the researchers confirmed they securely deleted the recovered data after submission.
The vulnerability carried real limits. An attacker could not pick a target. Which blocks dm-thin reassigned to a new container depended entirely on Cloudflare's workload scheduler, so exploitation was opportunistic rather than directed. The technique also could not touch any actively mounted disk or modify another tenant's live data.
Cloudflare moved fast. Yomtov filed the report on September 4 at 15:26 UTC. The engineering team opened a security incident and confirmed the production setup behind the flaw within about three hours. A runtime fix was merged by 21:27 UTC the same day. Rolling out the change across the fleet began by 23:15 UTC. But removing `skip_block_zeroing` only stops future misallocation. Blocks already mapped into running containers or cached in pre-built snapshot layers were unaffected. To clean those up, Cloudflare drained hosts during off-peak hours, restarted their VMs, and wiped each host's image cache so every disk would be rebuilt using zeroed allocations. That final cleanup finished on September 19. The researchers confirmed their proof of concept stopped working on September 14.
Cloudflare said it reviewed all available historical disk I/O telemetry and found no activity consistent with the exploit technique other than what came from the researchers and from Cloudflare engineers during authorized validation. No customer-side action is needed.
The disclosure adds to a recent pattern in cloud infrastructure research. Yomtov's team at Accomplish has a track record of finding platform-level flaws; they also reported a sandbox escape in Anthropic's Cowork tool this year. In the wider cloud industry, Wiz researchers disclosed a separate cross-tenant issue in Microsoft Azure Cosmos DB this year, called CosmosEscape, which could have let attackers escalate from a crafted Gremlin query to retrieving primary account keys for other customers' databases. Microsoft said it found no evidence of customer impact in that case either.
Cloudflare co-authored its disclosure with Yomtov and the Accomplish research team, a relatively transparent move for a company of its size. The company's bug bounty sits on HackerOne and remains open for
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article: