Registered: Aug 2000
Location:
Posts: 1725

Managing Software RAIDs
March 9th, 2006 by cvcrud

Ok, so previously, I gave a small intro inito mdadm, and software raids with Linux. It wasnt very informative, it just simply “got the job done”. Well, this one isnt much better, but it explains things a little further.

[I] CREATING THE SOFTWARE RAID
I always choose to create the software RAID during the install of the Linux OS. It is literally a pain in the ass to set it up on the fly… we will go over that shortly…

To setup the RAID during the install, refer to my old post:

First off, begin your installation. Once you get to the partition section, choose to “Manually Partition” your drives.
Next, create your first set of partitions (I did hda’s first):

- Create 1st partition at 100 MB with a type of “Software RAID”

* you must do this so that you have GRUB installed on both hard drives

- Create a 2nd partition with 1024 MB with a type of “SWAP”
- Last, Create a 3rd partition that “fills” the rest of your drive with a type of “Software RAID”

Do the same thing to your hdb drive.

After this is done, and you have to drives with the same partitions, click the RAID button. This will bring up a box with 3 options. Choose to “Create a RAID Device”.

A new window will open, and you need to choose the 2 100 MB partitions. Make them the /boot partition, and create the device (/dev/md0). This goes back to having a bootloader on both devices. Without it hdb will not boot alone.

Next click the RAID button again, and do the same thing with the last partitions you created. Make those partitions mount point /. Create the device (/dev/md1). Ensure that the bootloader is installing to /dev/md0.

Finish up the install of FC4 and enjoy your mirror.

[I.I] CREATE RAID ON THE FLY
In order to attempt (and I use the word “attempt” for a reason) to get a RAID built “on the fly” you will first need to have already installed your second hard disk. Ensure that Linux is seeing the disk by checking for /dev/hda* or /dev/sd*.

Once your sure of everything, you will need to tag your partitions as Linux Raid Autodetect (fd on the tag list in fdisk). So pretty much:

- fdisk /dev/sda
- “p” - print the partition table (write down the start and end sectors of each partition)
- “t” - tag a partition type (change all partitions to a Linux Raid Autodetect)
- “w” - write the partition table

You will need to create the exact same partitions and partition types, thus the reason you wrote down the start and end sectors of each patition:

- fdisk /dev/sdb
- “p” - should be nothing there, unless its a previously used drive. delete any partitions there.
- “n” - create new partion, same as the other one, matching /dev/sda1
- “n” - create second new partition, matching /dev/sda2
- “t” - change the partition types of both new partitions
- “w” - write the partition table

Now there are various methods in which you could actually create the mirror on the fly… my method:

mdadm –create /dev/md0 –level=1 –raid-devices=2 /dev/hd[ac]1

This creates /dev/md0 as a RAID1 array consisting of /dev/sda1 and /dev/sdb1. That should be your boot partition. Do the same thing with sda2 and sdb2 for a /dev/md1. As I said before, Good luck with this, its a pain in the ass; but it is do-able.

[I.II] SETTING UP MDADM FOR NOTIFICATION

MDADM also gives you the ability to monitor your RAID, all in one effectivness. First, you will need to edit your /etc/mdadm.conf. Look for the mailaddr line, and put your email address there. In order to monitor your RAID, you will need to run the following:

mdadm –Ff /dev/md1(and 0; once for each mirror)

When mdadm is invoked with -F option, it daemonizes itself, and “follows” the mirror, alerting your defined email address when something goes wrong:

Subject: Fail event on /dev/md1:blah.blahlblah.com

This is an automatically generated mail message from mdadm
running on blah1.blahblah.com

A Fail event had been detected on md device /dev/md1.

Faithfully yours, etc.

*you will need to start sendmail for this to work

Because it can run as a daemon, its a good idea to throw those commands in your /etc/rc.local so they run on startup.

(The other “f” just forces it to run)

[I.III] TESTING THE RAID…

In order to simulate a failed device with mdadm you can issue the following command:

mdadm /dev/md0 -f /dev/sda1 (Fail the device in the raid)

You will next need to remove the device from the raid by executing this:

mdadm /dev/md0 -r /dev/sda1 (remove the davice from raid)

To re-add the device to the RAID, and start the re-sync, you would do this:

mdadm /dev/md0 -a /dev/sda1 (”hotadd” the device back to the array)

You can issue these commands as a string:

mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1 -a /dev/sda1

This will do everything listed above at once… you should recieve email notification after running this command.

[I.IV] REBUILDING A RAID IF A DEVICE FAILS

Lets say one day you show up to work, or wake up to find that dreadful “delightfully yours” email from mdadm…. what do you do? What does your common sense tell you? Heres what I would do:

First I would remove the device from the RAID and try re-adding it back:

mdadm /dev/md0 -r /dev/sda1 -a /dev/sda1

If that works… your in luck. I would have my spare disc next to me, just in case that didnt work. And if it didnt work, this is what I would do:

First I would remove both devices that are on the disk with the failed partition:

mdadm /dev/md0 -r /dev/sda1
mdadm /dev/md1 -r /dev/sdb1

Then, I would physically remove the disc from the server. After that I would insert my spare, and write my RAID partitions just like before. Then re-add the devices back to the array:

mdadm /dev/md0 -a /dev/sda1
mdadm /dev/md1 -a /dev/sdb1

I would check my RAID details as this is happening… mdadm –detail /dev/md0 [md1]
and: more /proc/mdstat

#EOF

Report this post to a moderator | IP: Logged