Classic tech archive.
From the original averyjparker.com tech blog — historical context; pair with modern guides where noted.
Full archive · Maker projects ·
Network Ninja
Classic tip · Linux
Ubuntu Linux Software Raid - Replacing a Failing Drive
Sometime back I setup my home server running Ubuntu Linux (6.06 - Dapper Drake LTS). I used two pairs of drives to do raid cloning. Two IDE drives were for the main system structure and two SATA driv…
Sometime back I setup my home server running Ubuntu Linux (6.06 - Dapper Drake LTS). I used two pairs of drives to do raid cloning. Two IDE drives were for the main system structure and two SATA drives for Audio/Video storage as well as CDimages and other large file sharing on the local network. Well... I noticed the hard drive light was on solid and sure enough one of the two SATA drives had failed. (I didn't get my status email because I'd done a network structure change and didn't update my local mail setup...) Anyway... replacing it was a pain in the neck only for the physical access to the box. Everything else worked as it should.
I had made a file with the contents of the partition table when I first set things up which helped...
So from another pc (with the new drive attached as a usb disk) I ran...
sudo sfdisk /dev/sdb < partitiontable.sda(sda was the one from the old setup that had failed.) Then I logged in and removed the failed drive from the arrays.
sudo su mdadm /dev/md2 --fail /dev/sda6 --remove /dev/sda6 mdadm /dev/md1 --fail /dev/sda5 --remove /dev/sda5 mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1(Remember those long --'s are really a double hyphen - wordpress is funny that way and interprets them differently.) I shut down the machine in question and carefully pulled it out where I could work on it. (Those front loading hard drive trays would be REALLY nice for my home setup.) I figured out which was sda according to the system board, disconnected it and tested the bios to make sure. Sure enough I had pulled the correct one. I shut down again and removed the old drive entirely, replacing it with the new drive and hooking up the cables. Then I powered back up to check bios again. All is good. I powered things back down to put the cover on the case and move it back into it's cubby hole. All booted up just fine - doing a "cat /proc/mdstat" showed just one drive for the raid drives of md0, md1 and md2. So, I just did the following...
mdadm /dev/md2 --add /dev/sda6 mdadm /dev/md1 --add /dev/sda5 mdadm /dev/md0 --add /dev/sda1and checked "cat /proc/mdstat" again to find that they were in process of syncing and all looked healthy. The syncing process will take a while (these are 400GB drives) especially if there's is change to the content during this proces, but once that's done it should be a perfectly healthy raid array again. While I was at it I ordered a pair of drives to start rebuilding my desktop system with a raid array for the primary system file structure and the home partitions. I like the redundancy that simple software raid gives. (I've talked before about even adding a third drive in temporarily as a backup drive. USB may be a bit slow for doing that frequently, but as a snapshot it's not a bad approach from time to time.)