Booting grub from a removable device

Basics of grub

Recently I went into the problem of having to boot two different hard drives on a single notebook. The reason for this was that I wanted to keep the data from the first hard drive aside when using the notebook for work which might require remote access by third parties and screen monitoring. Yeah, buying an additional notebook is another possible solution (as someone suggested), but why spend thousands of bucks when you can achieve the same with just buying the hard drive, having in mind that additional cases as this are not quite often, and I could just swap a hard drive with additional setup already prepared. Please note that the idea here is to keep the SSD drive inside the notebook and swap only the second one.

In my case, the notebook has an SSD drive which has a Windows OS (and all the windows recovery partition stuff on it), and another regular SATA hard drive which includes my Kali Linux and a partition for data storage.

The SSD drive:
The SATA hard drive:
So as the /dev/sda has the windows bootloader and grub is installed into the MBR of /dev/sda, which then points to /dev/sda1 (where the windows bootloader is stored) and to /dev/sdb3 (where the linux box is located), what happens if you just swap the /dev/sdb hard drive?

As the UID of the device and UUID of the hard drive’s partitions and the wohle partition table have changed, grub is unable to recognize the device within its configuration and goes into rescue mode stating “error: device not found“.

As I’m writing this post as a quick reference to grub boot issues, I’ll describe the main concept of grub command line, which is as follows:

Basicly in order to review the hard drive devices and partition tables, you just type “ls”.  Once you type “ls”, you would see a list of hard drives and partitions.

Enumerating where the Linux installation is

In order to enumerate where the linux partitions is located, you need to type “ls” and the appropriate partition. If you just supply the partition it would output the filesystem, and if you add a forward slash at the end “/” it would list its directory contents.
ls

ls (hd0, gpt1)

ls (hd2,gpt2)

ls (hd2,gpt2)/
Once you get an answer from grub that the requested filesystem is “ext4” or whatever your linux partition is, you should proceed further and make sure that this is the correct one by listing its directory tree structure and looking for the “boot directory”:
ls (hd2,gpt2)/boot
In case you get directory listing to the above command then you should have the right partition reference.

Booting your Linux installation

In order to boot the selected partition, you basically have to let grub know where is its configuration and where kernel image files are being stored. From the above command you should have already have the exact names of the images, which are usually something like “vmlinuz-4.19.0-kali4-amd64” and “initrd.img-4.19.0-kali4-amd64”. In order to boot the kernel using these image files there are two ways:

option #1

root=(hd2,gpt1)
prefix=/boot/grub
insmod linux
normal
OR the last command could also be substituted with the following one:
configfile /boot/grub/grub.cfg

option #2

root=(hd2,gpt1)
ls /boot/grub
insmod linux
linux /boot/vmlinuz-4.19.0-kali4-amd64 root=UUID=[ID]
# ( ID can be retrieved using cat /etc/grub/grub.cfg or blkid)
initrd /boot/initrd.img-4.19.0-kali4-amd64
boot
After booting into grub in order to recover the filesystem try running update-grub and grub-install:
update-grub
grub-install

Possible issues: The annoying “grub-install: error: cannot find EFI directory.” error

When grub boots the kernel image, you should usually have an entry in /etc/fstab which mounts the EFI system, which is something like this:
UUID=[Some UID] /boot/efi vfat utf8,fmask=0133 0 1
In case for some reason the efi directory could not be mounted, or you are missing the entry within /etc/fstab and run “grub-install”, it would report that the EFI directory is missing: In order to resolve this, you have to mount the EFI System partition in /boot/efi. Running “fdisk -l” would tell you where the partition with the UEFI bootloader resides – you have to look for a 500M sized partition labeled “EFI System” or something similar: It’s usually the first partition of the first hard drive within your partition table.
mount /dev/sda1 /boot
grub-update
grub-install

cannot set efi variable boot000: no space left on device

Occasionally grub may refuse to update the mbr or partition you are trying to install it on with a message that there is no space left to write to. To resolve this issue, you need to delete the temporary files created during previous attempts:
rm /sys/firmware/efi/efivars/dump-*

How to configure grub to boot from a removable device

In order to boot two different hard drive devices and have separate configuration for each other, while keeping the MBR of /dev/sda, my solution was to configure grub to boot from a usb flash drive. A quick diagram to describe the process would look something like this: In order to prepare the usb stick for the second hard drive (HDD2), I swapped the hard drives and used a bootable live kali image in order to be able to interact with the filesystem and to avoid dealing with reconfiguring grub several times with each hard drive.

Prerequisites:
  • 1 bootable linux flash drive
  • 1 empty usb flash drive *remember to create a 500M fat32 partition, mark a boot and esp flag on it, and create a gpt partition table)
  • 1 additional internal or external hard drive (or whereever the installation you want to boot is)
The below steps suggest that the /dev/sda1 is the EFI System drive where the boot manager is located, and /dev/sdc is the usb stick you want to configure to boot the second hard drive with. Prior to going for this you have to create a 500M fat32 partition using gparted, mark it ESP and boot, as well as create a gpt partition table so it could be mounted as an efi filesystem.

1. Swap hard drives with the one you want to boot grub using a usb stick

2. Insert the live bootable usb of the distribution you are using (I used Kali, but any bootable distribution which has fdisk and grub-install would do) and an empty usb drive

3. Mount your empty usb drive which you want to confgure grub to boot
mount /dev/sdc1 /mnt
4. Mount the EFI partition
mount /dev/sda1 /mnt/efi
5. Install grub configuration into the MBR of /dev/sdc, pointing /mnt as the boot directory and /mnt/efi as the EFI directory
grub-install --boot-directory=/mnt --efi-directory=/mnt/efi --removable /dev/sdc
6. Generate grub config file (Required!)
grub-mkconfig -o /mnt/grub/grub.cfg


Power off the device using udisksctl power-off /dev/sdc to be safe, reboot and test. When the system reboots select the usb stick as the boot device.

Now you should have a second hard drive and a usb stick which can boot it, and once you swap it with your original configuration you would be able to boot normally.

Remember that you can neither boot the first hard drive with the USB stick nor the second hard drive without the USB stick. The grub configurations are now attached with the relevant device UUID.



Resources:

https://current.workingdirectory.net/posts/2009/grub-on-usb/ – Most useful thread and closely related to my issue, required some updates and slight modifications. https://help.ubuntu.com/community/Grub2/Troubleshooting https://blog.viktorpetersson.com/2014/07/29/how-to-boot-from-usb-with-grub2.html https://askubuntu.com/questions/493826/grub-rescue-problem-after-deleting-ubuntu-partition (1st answer) https://www.howtoforge.com/tutorial/repair-linux-boot-with-grub-rescue/ https://www.gnu.org/software/grub/manual/grub/grub.html https://askubuntu.com/questions/740253/how-to-install-grub-in-an-external-hard-drive https://www.cyberciti.biz/faq/howto-copy-mbr/