I worked in a big project and while doing some testing I noticed a bug that was previously fixed had reappeared. Since I was the one that had originally corrected the bug it was quite easy for me to spot the regression. I reported my findings to the project manager and after some investigation it came clear that several bug fixes were gone. Someone had forgot to merge the last fixes from the x.0 release to the x.1 branch.
Even after the missing merge was done, the project manager still didn’t trust that everything was fixed. There was a consensus among the developers that the merge was the cause and everything should be fine, but the project manager still felt unsure of the state of the system. To mitigate this an execution of a big regression test was ordered. In good shoot the messenger fashion I got the honor to do the testing. I had found the problem in the first place, so people thought it was logical that I should do it. Since this was seen as a risk, it was critical to get this done fast.
I spent a little more than two weeks doing insane hours. Mostly alone in the lab executing scripted manual regression tests. I had to repeat the same tests over and over again on different but similar equipment. Making an already boring task, soul destroying. The result was as expected and I didn’t find any regression related to the initial problem. Everything was fixed by doing the missing merge.
A non-existing continuous integration was the reason that this fault could slip through in the first place. The lack of technical safety that continuous integration and test automation would have brought, made the project make decisions based on feelings rather than to base them on facts. This uncertainty prevented people from trusting each other. What at first was a technical problem started to affect the psychological safety. Needless to say I wasn’t that keen on bringing up any inconvenient findings for a while.