How a Single Bit Flip Brought Down an Entire Data Center

Failed to add items

Sorry, we are unable to add the item because your shopping cart is already at capacity.

Add to basket failed.

Please try again later

Add to wishlist failed.

Please try again later

Remove from wishlist failed.

Please try again later

Adding to library failed

Please try again

Follow podcast failed

Unfollow podcast failed

How a Single Bit Flip Brought Down an Entire Data Center

Listen for free

View show details

In episode 36 of The Software Engineering Podcast with Fexingo, Lucas and Luna dive into one of the most infamous hardware-induced software bugs in recent memory: the 2021 Facebook outage caused by a single bit flip. Lucas explains how a routine configuration change triggered a cascading failure that took down Facebook, Instagram, and WhatsApp for over six hours. He walks through the exact sequence — a BGP withdrawal, DNS failures, and a data center network meltdown — and why a single incorrect bit in a router's memory was the root cause. Luna challenges the conventional wisdom about redundancy and asks whether engineers can realistically guard against single-bit errors at scale. They discuss cosmic rays, memory error-correcting codes, and the trade-offs between software abstraction and hardware reality. Along the way, they share practical lessons for engineers designing resilient systems: from careful change management to the dangers of assuming hardware is perfect. #SoftwareEngineering #Technology #FacebookOutage #BitFlip #BGP #DNS #DataCenter #Resilience #Infrastructure #Networking #CosmicRays #ECC #ErrorCorrection #ChangeManagement #CascadingFailure #EngineeringLessons #FexingoBusiness #BusinessPodcast Keep every episode free: buymeacoffee.com/fexingo

No reviews yet