Why do customer find bugs even after our extensive testing

Everything is interconnected in this world. Take butterfly effect as an example. The butterfly effect (coined by Edward Lorenz) is the sensitive dependence on initial conditions in which a small change in one state of a deterministic nonlinear system can result in large differences in a later state. An another example from the same article - The randomness of the outcomes of throwing dice depends on this characteristic to amplify small differences in initial conditions—the precise direction, thrust, and orientation of the throw—into significantly different dice paths and outcomes, which makes it virtually impossible to throw dice exactly the same way twice.

Now coming back to our world of software – In a small subsystem, it’s easy to predict the results of an action. Provide a specific input and the result will always be consistent. Now think of bigger subsystems with many connected systems passing through different layers. The precise amount of data, load, network and the parameters might cause something to fail. A software update to a different subsystem can cause failure somewhere else. A series of small changes can influence the output. This is why customers find bugs that we miss. In a closed room, it’s easy to predict the result of an action, like bouncing a ball. In an open world, it depends on many criteria’s, like the direction of wind, the UFO’s that might suddenly fly through and many more possibilities.

What do we need to do to find customer bugs? Switch from testing a smallest unit of code to test scenarios, switch your testing from smaller input and output to large customer like apps, switch from tests that run for an hour to 24*7 tests, switch to bigger subsystems where many pieces are interconnected, where there is sensitive dependency and you can see the butterfly effect. Call us if you still see issues from customers after this :)