Kinscoe: Created page with "==Summary of RAID software in Linux== There are several options available to Linux: these are all software based solutions as opposed to hardware RAID we used to see in physi..."

2018-03-29T16:07:55Z

Created page with "==Summary of RAID software in Linux== There are several options available to Linux: these are all software based solutions as opposed to hardware RAID we used to see in physi..."

New page

==Summary of RAID software in Linux==

There are several options available to Linux: these are all software based solutions as opposed to hardware RAID we used to see in physical disk controllers or Storage Area Networks.

In Linux the three main commands for creating a software RAID between two or more volumes are: lvm, mdadm or dmraid. dmraid ([https://en.wikipedia.org/wiki/Device_mapper Device Mapper]) is limited to ATA type disks (commonly known as "[http://superuser.com/questions/721795/how-fake-raid-communicates-with-operating-systemlinux/721796#721796 fakeRAID"]) so we will not use that here. Mirroring with [http://hydra.geht.net/tino/howto/linux/lvm2/mirror/ LVM2 (which ships with RHEL 7) is considered broken] when it come to mirroring so we will focus here on [http://neil.brown.name/blog/mdadm mdadm].

Some notes:

https://jreypo.wordpress.com/tag/device-mapper/

http://stackoverflow.com/questions/23164384/what-is-the-difference-between-dm-and-md-in-linux-kernel

The Linux RAID FAQ: https://raid.wiki.kernel.org/index.php/Linux_Raid

RAID on AWS instance: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/raid-config.html

==mdadm mirrors==

You will likely need to install the madam softwware as it is not installed by default:

<pre>yum install mdadm</pre>

Notes: https://raid.wiki.kernel.org/index.php/Partitioning_RAID_/_LVM_on_RAID

An example of mirroring two AWS EBS volumes in an instance:

1. Create two same sized EBS volumes and attach to the instance as known devices say (/dev/xvde and /dev/xvds). Note if you are re-using a currently attached EBS volume drop to Step 4 below.

2. Verify the volumes are attached to your instance:

<pre>
$ sudo lsblk /dev/xvde
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvde 202:64 0 34G 0 disk
</pre>

3. You do not need to fdisk or partition these two volumes. This will be done through the mirror volume.

4. If this volume has been used before we need to erase it. This can take quite a long time. It is also known as "pre-warming". You may want to consier removing and creating a new EBS volume from sratch it would be quicker.

To erase an already connected EBS volume we will use output from fdisk to supply parameters to the dd command.

<pre>
$ sudo fdisk -l /dev/xvdf

Disk /dev/xvdf: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

$ sudo dd if=/dev/zero of=/dev/xvdf bs=512
20428009+0 records in
20428009+0 records out
10459140608 bytes (10 GB) copied, 1356.81 s, 7.7 MB/s

real 22m36.817s
user 0m0.325s
sys 0m59.720s
</pre>

As you can see above it took 23 minutes just to zero out 10GB.

5. Run this command to create mirrored volume of both EBS volumes. The resulting volume will be /dev/md/oracle0.

<pre>
$ sudo mdadm --create --auto=mdp --verbose /dev/md/oracle0 --name=oracle0 --level=mirror --raid-devices=2 /dev/xvde /dev/xvds
mdadm: /dev/xvde appears to be part of a raid array:
level=raid0 devices=0 ctime=Wed Dec 31 19:00:00 1969
mdadm: partition table exists on /dev/xvde but will be lost or
meaningless after creating array
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: /dev/xvds appears to be part of a raid array:
level=raid0 devices=0 ctime=Wed Dec 31 19:00:00 1969
mdadm: partition table exists on /dev/xvds but will be lost or
meaningless after creating array
mdadm: size set to 35618816K
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/oracle0 started.
</pre>

6. Partition your mirror device:

<pre>
$ sudo fdisk /dev/md/oracle0
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x70a07f5e.

Command (m for help): p

Disk /dev/md/oracle0: 36.5 GB, 36473667584 bytes, 71237632 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x70a07f5e

Device Boot Start End Blocks Id System

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-71237631, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-71237631, default 71237631):
Using default value 71237631
Partition 1 of type Linux and of size 34 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

$ sudo fdisk -l /dev/md/oracle0

Disk /dev/md/oracle0: 36.5 GB, 36473667584 bytes, 71237632 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x70a07f5e

Device Boot Start End Blocks Id System
/dev/md/oracle0p1 2048 71237631 35617792 83 Linux
</pre>

7. Now format the filesystem (ext4):

<pre>
$ sudo mkfs.ext4 -L /oracle /dev/md/oracle0p1
</pre>

8. Check on the raid volume:

<pre>
$ sudo mdadm --detail /dev/md/oracle0
/dev/md/oracle0:
Version : 1.2
Creation Time : Mon Feb 22 19:14:59 2016
Raid Level : raid1
Array Size : 35618816 (33.97 GiB 36.47 GB)
Used Dev Size : 35618816 (33.97 GiB 36.47 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

Update Time : Mon Feb 22 19:24:56 2016
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Name : awsmdmqld02.hmco.com:oracle0 (local to host awsmdmqld02.hmco.com)
UUID : de418038:1019361b:90a24022:83f16131
Events : 18

Number Major Minor RaidDevice State
0 202 64 0 active sync /dev/xvde
1 202 4608 1 active sync /dev/xvds
</pre>

9. Update the madam.conf file for reboots.

<pre>
$ sudo mdadm --examine --scan --config=mdadm.conf >> /etc/mdadm.conf
</pre>

==Monitoring the RAID==

You can always take a look at /proc/mdstat. You can also use mdadm command to check the RAID array status.

mdadm --detail we looked at above.

This command will show spare and failed disks.

==Notes of interest==

Cheatsheet: http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/

HOWTO: http://tldp.org/HOWTO/Software-RAID-HOWTO-6.html

http://edoceo.com/howto/mdadm-raid1

Linux - Software RAID and Mirrors - Revision history

Kinscoe: Created page with "==Summary of RAID software in Linux== There are several options available to Linux: these are all software based solutions as opposed to hardware RAID we used to see in physi..."