SysAdmin Talk

A few months back I wrote an article titled “7 reasons why High Availability will help you fail in even more spectacular ways than ever!” It was a humorous but legitimate look at some pitfalls you might face when implementing highly available systems (e.g. clustered servers, failover firewalls and etc.). However, after finishing that article I felt I hadn’t quite addressed the issue in toto. Then it struck me. The previous article focused mostly on the hardware and technology, but paid scant attention to the technologists working the hardware, and the more personal assumptions that can be made when considering and implementing highly available systems.

Make no mistake, we SysAdmins can make some silly assumptions about HA and its impact on our lives. Let’s take a look at some and see if we can clear them up:

1. High availability will not inherently make your job more secure. That responsibility is, as it always has been and always will be, largely up to you personally. As was pointed out when I discussed 7 reasons why High Availability will help you fail in even more spectacular ways than ever, you are afforded some amazing ways to fail if you view high availability solutions as a panacea. HA is simply one of the many tools that are used to secure a resource. Remember, a carpenter isn’t paid because he has an awesome hammer.
2. High availability will not make you a better SysAdmin. Don’t kid yourself. If you didn’t have a cluster last week but you do this week, you’re not inherently a better SysAdmin. Or if you are, it wasn’t the cluster that did it. Anyone with a room temperature IQ and a support contract can implement most HA solutions, but you need good processes regardless of whether or not you have an unstoppable juggernaut of a failover cluster. SysAdmins with expensive new clustering software are not magically given excellent time management skills, interpersonal tact, clairvoyant troubleshooting insights and minty fresh breath. That takes intentionality, hard work, perseverance and humility. Just like everything else worth earning.
3. High availability will not save the company any money. HA costs money, and a lot of it. The only time it (hopefully) saves anything is in the case of a disaster, which really isn’t true savings. Yes, it prevents the active loss of money should a worst case scenario happen, but that’s not true financial savings. Thus, you shouldn’t present it to your superiors (or even yourself, for that matter) as a cost-saver. Present it like it really is: A butt saver.
4. High availability will not help you with your bad attitude. If you’re bitter that management doesn’t give you what you want or need, and makes you hobble along on a shoe-string budget, then I’m afraid there’s only one thing that will help you: getting a better attitude. And there’s only one person that can give you a better attitude: you. Don’t think that you’d be less grumpy, happier or generally live a halcyon lifestyle if you could just get some HA implemented on your critical systems. You’ll still be grumpy, unhappy and agitated, just with one more expensive system which you’re expected to maintain. Figure yourself out, and don’t expect anyone or anything to magically make you a more contented person.
5. High availability will not make your bosses appreciate you more. Or if they do appreciate you more, it’s not so much the HA system’s magic as your communication with them while you implemented it. If you want appreciation, or you fear that IT’s role is not understood and largely ignored, then it’s once again your responsibility to show up, step up and speak up. Take responsibility for what you can do to change the perception of IT. Clearly communicate what you are doing. Be visible with the business leaders and align yourself with business goals. HA will not garner you appreciation; visibility will. Unless your work practices are unsightly, in which case you have a completely different set of problems to take care of.
6. High availability does not allow you to skimp on hardware or run your equipment into the ground. As the saying goes, you can’t make a silk purse out of a sow’s ear. If you have two thrift-store servers that you synchronize in a cluster, do you know what you end up with? A really grotesque looking handbag that will probably get you in trouble with PETA. Skimping on hardware, or treating existing hardware poorly, and then looking to some kind of clustering solution to stave off the Bad Things from happening is not addressing the root of the problem. Buy the proper hardware and then run it in proper operating conditions before you even think about some form of clustering.
7. Lastly, High Availability will not make your stuff highly available. You will make your stuff highly available. You are the one who should be researching compatibility, scenario testing, questioning assumptions and generally being a complete pain in your vendor’s tender spots as you kick the virtual tires on a HA product. Just because the salesman (whose teeth are so white you get radiation burns from his smile) says his product implements high availability doesn’t mean it will do so automagically. If you want proof, then remind me to tell you about the time my team learned that a certain load balancer would fail in a “closed” state, thus not flowing traffic to the secondary load balancer that it was connected to. Ultimately, it’s you and your team of technologists that makes anything highly available, and don’t you forget it.

Clustering, failover, mirroring, duplication, synchronization… they all solve fewer problems than most of us think they do. Sure, those things serve a great purpose and so HA (if you’ll forgive me for using a nebulous term) can deliver some useful tools! However, let’s put high availability in a proper perspective and to do that, we might need to first see ourselves in the proper perspective. Once we free ourselves from misperceptions, we can be free to approach things with greater clarity.