Solving UEFI Boot Problems on Bhyve / FreeNAS VM

Background:

I’ve been using FreeNAS for a couple of years now as my main go-to file server at home and am super happy with it. As my install is getting old now, before diving straight in and updating my ageing 9.10 setup to the latest 11.2 I dragged out my old N40L Microserver to give the latest version a whirl, mainly to try out the VM hosting that now seems to be (almost!) now properly baked into FreeNAS without any unofficial tinkering. In short, it’s great. It’s pretty simple once you know what you’re doing too so I thought I’d give it a spin.

Scenario:

After installing the lightweight Raspberry Pi Desktop (yes, they do an x86 one!) the newly installed OS did its updates, rebooted, brought me back to the desktop and all was well. Until …

Problem:

Bhyve UEFI Interactive Shell

My VM will not boot!

After gracefully shutting down the VM and then starting it up again later on that day, my machine would not boot. Rather, the VM would start but the operating system would not boot past a UEFI Interactive Shell.

Reason:

After a bit of digging it seems this is a well known problem where the finger is being pointed towards jointly between the GRUB boot manager not saving files where Bhyve expects and Bhyve not looking for bootfiles in the location saved in by GRUB.

Solution:

The trick here is to manually boot the machine by pointing it to the EFI boot file, then to place a copy of the boot file into the place expected by Bhyve.

  1. At the UEFI Interactive Shell, simply type exit
  2. At the Bhyve boot screen, select Boot Maintenance Manager

    then Boot From File
  3. You should now see the details of the first virtual disk to boot from.
    Press Enter to select the disk
  4. Press Enter to select the single EFI option
  5. At this point you should see an OS name, mine here being Debian.

    Use the cursors and Enter key to select this.
  6. Finally, select the file visible: grubx64.efi
  7. Your VM will now boot up to your selected OS.
  8. Your OS has now booted but at this point in time if you were to shutdown and start up the machine again you would be faced with the same at next boot. To fix this from within the OS you now need to open a root shell. For the uninitiated, you can do this in most modern OSs by opening a standard shell and running sudo su .
  9. Now, navigate to the EFI directory of your operating system. Do this is by typing cd /boot/efi/ followed by the directory as seen in the previous steps.
    In my case: cd /boot/efi/EFI/debian/

    To confirm you are in the right place, type ls and you should see a line reading grubx64.efi.
    This file contains the boot instructions that the machine is looking for on bootup. It’s just not in the correct location as yet.
  10. Next, make a copy of the boot file to the correct location: /boot/efi/EFI/BOOT/bootx64.efi
    You can do this by doing:  mkdir /boot/efi/EFI/BOOT/
    Then: cp grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi
  11. You can now close your shell prompt.
    Type exit
    Then again exit

This should be all that’s required to get your machine booting without any extra manual interaction. Just be aware though that if the grubx64.efi file ever changes (maybe after an OS update), a new copy will have to be made over the bootx64.efi file again.

Now, there is a lot of stuff that goes on in the background to do with VMs and booting that I’m not going to pretend I know everything/anything about but if you’re interested in the detail, KrisBee in the FreeNAS forums explains it all here.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.