Places like that never learn their lesson until The Event™ happens. At my last place, The Event™ was a derecho that knocked out power for a few days, and then when it came back on, the SAN was all kinds of fucked. On top of that, we didn’t have backups for everything because they didn’t want to pay for more storage. They were losing like $100K+ every hour they were down.
The speed at which they approved all-new hardware inside a colocation facility after The Event™ was absolutely hilarious, I’d never seen anything approved that quickly.
Trust me, they’re going to keep putting it off until you have your own version of The Event™, and they’ll deny that they ever disregarded the risk of it happening in the first place, even though you have years’ worth of emails saying “If we don’t do X, Y will occur.” And when when Y occurs, they’ll scream “Oh my God, Y has occurred, no one could have ever foreseen this!”
It’ll happen. Wait and watch.
Places I’m at usually end up bricking robots and causing tens of thousands of dollars of damage to them because they insist on running the robot without allowing small fixes.
Usually a big robot crash will be The Event that teaches people to respect early warning signs…for about 3 months. Then the old attitude slides back.
Good thing we aren’t building something that requires precision, like semi-conductor wafers. Oh wait.
Sounds like a universal experience for pretty much all fields of work.
Government and policy? Climate change? A fucking pandemic?!
We’ve seen it all happen time and time again. People in positions of authority get overconfident that if things are working right now, they’ll keep working indefinitely. And then despite being warned for decades, when things finally break, they’ll claim no one could have foreseen the consequences of their lack of responsibility. Some people will even chime in and begin theorising that surely, those that warned them, had to be responsible for all the chaos. It was an act of sabotage, and not of foresight.
