Who Would Take That Gamble?!
I was a software engineer at a large company with little tolerance for downtime. Understandably, they were very strict about software change management. Even the most minor change to the production environment had to be carefully documented in great detail.
Once the change was fully documented, it had to be approved by every group that would be impacted in any way. This was usually the most difficult step, requiring nagging emails and often desperate phone calls over a two-week period to people who had no motivation to take the time to open the change management tool, find the change, read through the details, and take responsibility for approving it for their department.
But sometimes emergencies arise, so there was a simple process to bypass that final time-consuming step. The only approval required to implement an emergency change was from the manager of the person making the change. But the person making the change would then have only three days to collect all the rest of the approvals or the change would be classified as “Unauthorized”.
One application team deployed a software update that was failing intermittently. My manager approached me to see if I could provide a simple fix.
Manager: “Do you think you can mitigate this issue by modifying the container configuration, or does the application team need to fall back to the last release?”
Me: “Would I have to modify it as an Emergency Change?”
Manager: “Yes, of course.”
Me: “And if I don’t get all the sign-offs in three days, I get dinged?”
Manager: “An Unauthorized Change would be a very serious violation.”
Me: “Serious like a note in my file or like losing a bonus?”
Manager: “That’s up to the Change Team. Anything up to and including termination.”
Me: “No, I don’t think modifying the configuration will work.”
A couple of weeks later, after this issue was resolved, my manager asked if I really could have fixed the problem. I didn’t directly answer, but I told him that I would never, under any circumstances, take responsibility for an Emergency Change.
And I never did.