Of bash and LVM

Wow, has it actually been 2 months since I posted anything to this site? I suppose it has.. What has kept me busy you might ask? Well, I have been working on my backup script rather faithfully lately due to some unfortunate happenings at work lately, more on that later. I have introduced a simple database dump feature to the script so it can dump a database (mysqldump) into the directory it is backing up, then you get a nice bundled backup in one shiny tarball!

At this point I am pondering how to keep the script from turning into a massive procedural nightmare of bash code. Sure, it does everything I need it to do right now, but adding features could prove to be an exercise in masochism if I had to do anything major. Right now the script it sitting at 357 lines of code, a lot of which is debugging output, so in reality the script it probably closer to around 250 lines, but the whole thing it still bigger than I anticipated it getting. Bash has turned out to be surprising powerful, but surprisingly limited. I suppose it was never meant to be used as a full programming language.

At this point I am trying to figure out how to get a single instance of the script to backup multiple directories into separate tarballs. The only thing I have been able to come up with thus far has been a separate configuration file with some sort of structure to define each backup job. I will probably go this route, but not before I have exhausted other options first.

Ok, back to the unfortunate happenings at work. One of our web servers runs CentOS, if you have read any of my previous posts you will know that I absolutely adore CentOS…wait did I say adore, I mean despise. When I got the colocated server CentOS had already been set up on it so I just went with it. Little did I know, they enabled the Linux Volume Manager to manage the disks. I applied a routine kernel upgrade, then rebooted and the machine died a horrible silent death right then and there. It never came back and never relinquished any of the data off the drives, even though I was running the drives in Raid 1. FSCK basically came back with a “Your screwed” error message after trying to read the Volume groups for a couple of hours. How did this happen, I have no idea. I do know that this was the worse server melt down in my entire history of being an admin, and it was also the only time I have ever ran CentOS with LVM in a production environment. I am open to the idea that it might be possible that I did something in the recovery effort to completely hose all the data. However, take a guess at who I am blaming for the whole fiasco 😉

Comments are closed.