Booting FreeBSD from a USB disk

Open source software has found its way in every corner of the IT industry. Which is great, as it provides possibilities that were otherwise to expensive to pursue. In general, there is (at least) one down-side to using open source software: Support. In some cases a company might offer you a community edition of their software in the form of Open Source software, allowing you to choose between a free, “as is” version or a full-blown – and paid for – enterprise edition of their software. The latter usually has a help desk or technical support team to help you, but with former you’re on your own. There’s nearly always some form of community to help you out, but in general that is all “best effort”. And some are very good, but results vary is my experience. So if you ever happen to be in the position where there is no else but yourself to provide support, you’d best double check first to see if the software matches requirements, or expectations (if requirements are missing). Obviously, this is best done before using it in an environment where people depend on it. All too often, features turn out to work different than expected, or have other side effects.

The same applies to software updates. You’d normally expect the supplying party to have tested what they’ve built, but I have learned that it’s best to rule out any ill side effects for myself. This is especially true for updates or upgrades for the operating system you are using. FreeBSD is no exception. Although it hasn’t failed me that often compared to some others, I still like to verify it’s behavior “just in case”. In order to test OS updates – with a minimum amount of risks – you’d need a working copy of the machine you’d like to apply them to. There are several ways to do this, and if you happen to use jails, this becomes fairly straightforward. For a OS running on bare metal it is somewhat different.

After some tinkering with various methods, I found it more convient to create a bootable mirror-image USB-stick of the running system. This script creates 3 filesystems: 2 copies of the OS, and the rest for other data. It installs a bootloader so you can choose which copy to boot. This way you can easily verify differences side-by-side on another system.
For example: You could run freebsd-update in one, and if you don’t like the results – instead of doing a rollback – simply use the second copy to replace the first. Another advantage is that there is no need to play digital Russian roulette on the original host. Downside is that you do need an extra system to prepare the actual update/upgrade on.

Since disk-size is ever increasing, the days of commands like bsdlabel and fdisk are numbered, as they only allow you to allocate 2 Terabyte of diskspace. GEOM seems to fill in this gap. So below is an overview based on more modern FreeBSD commands, resulting in a bootable FreeBSD USB disk.


Start with a clean sheet. Although it would be trivial to overwrite the first sectors of the disk with zero’s, it doesn’t work that way: You first need to delete the partitions from geom and then destroy the current partitioning scheme. The following bourne shell snippet does just that. Note that I’ve used the variable DISK to prevent copy-paste disasters. Set this to match your disk, eg. DISK=’da9′ or so (Check dmesg output).

gpart status ${DISK} | awk 'NR>;1 {print $1}' | sed "s/${DISK}.//" | sort -ur | while read i
        gpart delete -i $i ${DISK}
gpart destroy ${DISK}

After that, clean the first few bytes to make absolutely sure there isn’t anything left that could be thought of as valid disk-layout information. In theory you could “watermark” your full disk with something like ‘echo “YOUR NAME” | dd of=/dev/${DISK}’. That way, if you would ever lose the disk, analysis would reveal “YOUR NAME”, unless you’re using the -E option of newfs. I’ve never tried that, and since this bit is only needed for the first few sectors of the disk and not the entire disk, I just stick to zero’s here:

dd if=/dev/zero of=/dev/${DISK} &
for w in 1 2 3 4 5
        echo -n "${w}..."
        sleep 3
echo "Done"
kill $c
sleep 5

…Next, recreate the disk and add FreeBSD partitions. This uses the GUID Partition Table (GPT) scheme, versus the old MBR. This is basically identical to the first example found in man page of gpart:

gpart create -s GPT ${DISK} >/dev/null 2>&1
while [ $Z -ne 0 ]; do
    echo "Trying to create ${DISK}..."
    sleep 5
    gpart create -s GPT ${DISK} >/dev/null 2>&1
gpart bootcode -b /boot/pmbr ${DISK}
gpart add -b 34 -s 128 -t freebsd-boot ${DISK}
gpart bootcode -p /boot/gptboot -i 1 ${DISK}
gpart add -b 162 -t freebsd-ufs -s 1G ${DISK}
gpart add -t freebsd-ufs ${DISK}

…Then, create the filesystems. p1 Is used for booting, p2 is 1 Gigabyte for the root-filesystem and p3 is used for data:

newfs /dev/${DISK}p2
newfs /dev/${DISK}p3

…Copy the root filesystem to the p2 partition of your $DISK. Note that this script actually tries to make an educated guess on how big it actually should be. This here below blindly assumes 1G to suffice.

mount -t ufs /dev/${DISK}p2 /mnt
mkdir -p /mnt/var /mnt/tmp /mnt/dev /mnt/root/bin
for d in boot bin sbin etc lib libexec usr/bin usr/sbin usr/share usr/lib usr/libexec usr/libdata
        c=`expr $c + 1`
        echo -n "${c}..."
        tar -C / -c -f - $d | ( cd /mnt && tar xf - )
echo "Done"

…Now setup the proper configuration so the system will boot properly. Note that if your system has SCSI-disks that this may actually be a different disk (da0 instead of da1 for example):

cat <<EOF > /mnt/etc/fstab
/dev/${DISK}p2        /       ufs     rw,noatime      1       1

cat /etc/rc.conf >> /mnt/etc/rc.conf.`hostname -s`
cat <<EOF > /mnt/etc/rc.conf

umount /mnt

…And voila! You’ve just cloned the system you’ve booted from!



As usual, these scripts aren’t nearly halfway finished. I assumed the use of USB-sticks and have made /var a virtual filesystem, although in real life var may actually have very useful content when performing updates or upgrades. It wouldn’t take a lot of effort to fix that. On the plus-side: Both methods used in this post don’t really depend on USB-sticks and should also work for any disk. I tend to use the first script , because of the luxury of having an identical backup root-filesystem at the cost of less than 1 Gigabyte of disk-space. I still need to convert that to fit a GPT scheme. As both scripts are still riddled with numerous imperfections I would like to highly recommend that you test these scripts in a virtual environment such as VirtualBox before using it on a critical server.

Please note that in FreeBSD, well… I’ve checked 8 and 9, the naming-scheme for SCSI-disks is identical to USB-disks, which is neat as it allows you to experiment with these scripts without actually having a real USB-stick. In VirtualBox, simply adding a virtual SCSI-disk will give you a ‘da0’ to use. Adding a real USB-stick will introduce ‘da1’. Reboot the virtual machine and hit F12 and there you should the (virtual) SCSI-disk. VirtualBox does not allow you to boot from real USB-stick unfortunately. So if you’ve created your bootable USB-stick in that setup, this will introduce a flaw: When booting that USB-stick on bare metal the virtual SCSI-disk will be missing, making the USB-stick da0, instead of da1. Then, the information in /etc/fstab is wrong and the kernel won’t find the root filesystem by itself. The same applies to card-readers instead of SCSI-disks.

This entry was posted in FreeBSD and tagged , , . Bookmark the permalink.