𝗖𝗵𝗮𝗼𝘀 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗠𝗶𝘀𝘁𝗮𝗸𝗲𝘀

Five chaos engineering mistakes to avoid

Chaos engineering doesn’t create resiliency, it reveals where resiliency is missing based on real world failures “chaos variables” such as hardware or network failures. It literally break things on purpose to build more resilient systems.

Popularized by Netflix, chaos engineering has been gaining attention and acceptance in the software engineering community. As more teams start using chaos engineering, there are a few common seen mistakes are worth discussing.

𝗠𝗶𝘀𝘁𝗮𝗸𝗲 𝟭: 𝗡𝗼𝘁 𝗗𝗲𝗳𝗶𝗻𝗶𝗻𝗴 𝗮 𝗦𝘁𝗲𝗮𝗱𝘆 𝗦𝘁𝗮𝘁𝗲 𝗙𝗶𝗿𝘀𝘁

Not defining a steady state first means we don’t have a clear baseline of how our system should work under normal conditions. Without this, chaos engineering becomes a mess—we can’t tell if the chaos broke something or if it was already off. It’s like trying to fix a car without knowing how it runs when it’s healthy. We need that “normal” snapshot to compare against, so we can spot what’s failing and why.

“We must know what "normal" looks like before breaking things, or we can’t measure what’s broken.”

𝗠𝗶𝘀𝘁𝗮𝗸𝗲 𝟮: 𝗦𝗸𝗶𝗽𝗽𝗶𝗻𝗴 𝗥𝗼𝗹𝗹𝗯𝗮𝗰𝗸 𝗣𝗹𝗮𝗻𝘀

We don’t have a way to reverse the chaos we introduce. In chaos engineering, we’re intentionally breaking things—like killing a server or spiking latency—to see how the system works. But if we can’t undo it fast, we might tank the whole operation.

“Failing to plan for rollback can turn a controlled experiment into uncontrolled chaos.”

For example, we cut a database connection to test resilience. Without a rollback—like a script to reconnect or a backup ready—we could leave users stuck, data lost, or the app dead. It’s not just about starting the chaos; it’s about controlling it. A good rollback keeps the experiment safe and lets us learn without burning everything down.

𝗠𝗶𝘀𝘁𝗮𝗸𝗲 𝟯: 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝘁𝗼𝗼 𝗺𝘂𝗰𝗵 𝗮𝘁 𝗼𝗻𝗰𝗲

Testing too much at once such as cutting servers, spiking traffic, and dropping connections all together. Let’s say we aim to test by killing two services and faking a network lag simultaneously. If the app crashes, is it one failure, both, or the combo? We’re left guessing instead of learning. Chaos engineering works best when we break one thing at a time—clean, clear results show us exactly what’s weak.

“Overloading with multiple failures(chaos variables) obscures root causes.”

𝗠𝗶𝘀𝘁𝗮𝗸𝗲 𝟰: 𝗙𝗼𝗿𝗴𝗲𝘁𝘁𝗶𝗻𝗴 𝘁𝗼 𝗺𝗼𝗻𝗶𝘁𝗼𝗿 𝗿𝗲𝘀𝘂𝗹𝘁𝘀

Forgetting to monitor results means we’re running chaos experiments blind. We break stuff such as killing a server or spiking load but if we don’t watch what happens, we’ve got no clue what it proves.

“Neglecting to monitor defeats the purpose—you need visibility into the chaos.”

For instance, we cut a database link to test resilience, but without tracking response times or error rates, we won’t know if it failed quietly or crashed hard. Monitoring is how we catch the weak spots and fix them. No data, no insight—just wasted chaos.

𝗠𝗶𝘀𝘁𝗮𝗸𝗲 𝟱: 𝗜𝗴𝗻𝗼𝗿𝗶𝗻𝗴 𝘁𝗲𝗮𝗺 𝗯𝘂𝘆-𝗶𝗻

Ignoring team buy-in means starting chaos experiments without team agreeing to it. If the people who run and fix the system aren’t on board, it’s a waste. They won’t get ready, won’t learn, and won’t fix anything. It’s like throwing a party no one wants to come to—nothing happens.

“No team support means chaos engineering doesn’t work—it’s about people too.”

For example, if you crash a service and the team doesn’t know why, they’ll just be annoyed instead of making things tougher. Getting everyone to say “yes” first makes sure they’re in on it and ready to improve.

𝗖𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻

To succeed, start with a clear baseline, plan rollbacks, test one thing at a time, watch the results, and get the team on board—miss any of these, and the chaos just creates more problems.

Disclaimer: The statements and opinions expressed in this article are based on personal experiences, faqs, and references. It do not necessarily reflect the positions of Miahlouge.

𝗜𝗻𝘀𝗽𝗶𝗿𝗮𝘁𝗶𝗼𝗻𝘀 𝗮𝗻𝗱 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀

• Chaos engineering frequently asked questions

• John Allspaw discussion on Resilience Engineering to industry

• Thoughtworks article on avoiding mistakes in chaos engineering

• Reddit discussion on Is Chaos Enginnering still a Thing?