Raid on Raspberry Pi with Compute Blade

Goal:

• Use two NVMe drives in RAID 1 for the root filesystem "/" to increase reliability
• SD card (or eMMC) is used only for booting “/boot/firmware“

1. Install OS and Required Utilities

• Install the OS on the SD card or eMMC

• Start the system and wait for a full boot, shutdown

• Install 2 NVMe disks via the Twins adapter on the Compute Blade, boot

• Connect via SSH and execute:

sudo apt update && sudo apt install -y mdadm initramfs-tools

• (Raspberry Pi CM5 optional) enable PCIe 3.0 in Advanced Options:

sudo raspi-config

2. Create RAID 1

Make sure that both disks are available on the system:

sudo fdisk -l | grep "nvme"

There should be two lines like:

Disk /dev/nvme0n1: xxx
Disk /dev/nvme1n1: xxx

or use command:

lsblk -do NAME,TYPE,SIZE,MODEL

Prepare NVMe Drives.
Wipe drives if they were previously used:

sudo wipefs -a /dev/nvme0n1
sudo wipefs -a /dev/nvme1n1

Create RAID 1. Note: if the disks are different sizes, the raid will match the size of the smaller disk.

sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/nvme0n1 /dev/nvme1n1

Check RAID status:

cat /proc/mdstat

This may take some time (5-30 min), you can track progress with the command:

Note: You don’t have to wait until the process is complete before moving on further

watch cat /proc/mdstat

finished:

BTW. For the example, I created a Raid1 of two disks of different sizes:
256GB and 1TB:

3. Copy Root Filesystem to RAID

Copy the root filesystem from the SD card/eMMC to RAID. This can be done in a number of ways, I prefer good old-fashioned dd. But partition /dev/mmcblk0p2 must be smaller than md0 (you can shrink it in advance with GParted if needed).

sudo dd if=/dev/mmcblk0p2 of=/dev/md0 bs=4M status=progress

After copying, the partition UUID will also be copied:

change the UUID to avoid conflicts:

sudo fsck -yf /dev/md0
sudo tune2fs -U random /dev/md0

Verify:

lsblk -o NAME,UUID,PARTUUID,MOUNTPOINT

4. Expand Filesystem to Use Full Raid1 Partition

After copying, the filesystem will be the same size as the original SD card:

Expand it:

sudo resize2fs /dev/md0

Mount and check the result:

sudo mount /dev/md0 /mnt
df -h /dev/md0

5. Update Boot Configuration

Mount Radi partition /dev/md0 (if you didn’t do it in the last step)

sudo mount /dev/md0 /mnt

Get md0 UUID:

sudo blkid /dev/md0

copy your UUID

UUID=”5e80d0bc-ca52-4f68-bc39-4a304f096b41″ in my case

Edit /etc/fstab

sudo nano /mnt/etc/fstab

Replace PARTUUID=xxxxxxx-xx with UUID=<Your UUID> for the root “/” only. And keep /boot/firmware on the SD card. Fstab example with my UUIDs (they are unique!):

from:

to:

Update cmdline.txt

sudo nano /boot/firmware/cmdline.txt

I’ll just give you an example of my file, you can use it (there’s only one line in the file), root=/dev/md0 changed:

console=serial0,115200 console=tty1 root=/dev/md0 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles

6. Add RAID to initramfs

echo "raid1" | sudo tee -a /etc/initramfs-tools/modules
echo "md_mod" | sudo tee -a /etc/initramfs-tools/modules
sudo update-initramfs -u -k $(uname -r)

to check:

lsinitramfs /boot/initrd.img-$(uname -r) | grep mdadm

7. Reboot and final check

mount | grep " / "
cat /proc/mdstat
df -h

Expected results:

• / is mounted from md0
• raid active

The /dev/mmcblk0p2 partition now can be deleted. But if you want to boot from it for some reason, it is enough to correct the cmdline.txt file

What about reliability?

You can disconnect either of the two disks and the system will continue to function and the RAID will degrade. If you put the same disk back in place, it will not be available until the OS reboots. But after rebooting the array will be restored.
To replace a disk, follow the general mdadm instructions.

Ivan Kuleshov

I’m a systems engineer in JetBrains company. Uptime Lab founder. I’m glad to see you on my website! I hope you find my content useful. Please subscribe to my Instagram and Twitter. I post the newest updates there.

uplab.pro