Konrad Scherer

PXE install on UEFI using Foreman and GRUB2

Most of the bare metal hardware that I manage now supports or defaults to UEFI. Many have the option to use “Legacy BIOS” mode, but the main feature I find that I require from UEFI is support for the boot volume to be 2TB+. I prefer one single RAID0 volume for all the builders for operational simplicity.

Foreman

My preferred solution for installing the base OS on the hardware is Foreman. It makes automated installs very simple and reproducible but has only recently supported UEFI and PXE. I will describe my previous attempts to get this working and how I was able to get it working with Foreman 1.14.2.

Pxelinux and UEFI

Pxelinux is part of the syslinux project and provides many different types of bootloaders. Pxelinux depends on a custom ROM inside the network card to run DHCP and download kernel+initrd using TFTP. It also has support for displaying interactive menus to the user.

UEFI contains all this functionality but unfortunately did not think to extend it or preserve backwards compatibility. All boot time programs like grub2 and pxelinux required significant rework. I was able to use the syslinux git tree and compile a working EFI version of pxelinux that was able to boot the 14.04 Ubuntu installer. But there were limitations:

  • Foreman only supported non-efi Pxelinux and I had to manually swap the binaries on the TFTP server
  • The menu system didn’t work so I could not use the Foreman feature of leaving the system to boot PXE by default and booting the local hard drive if rebuild was not enabled for that host in Foreman.
  • I could not get this pxelinux to work with 16.04 installer. The initrd would be downloaded and would hang and trigger a system reset.

UEFI and GRUB2

Foreman 1.13 added support for GRUB2 and UEFI, but my initial attempts failed. When I changed the boot template from PXELinux to PXEGRUB2 the update of the DHCP server would fail. The DHCP entry was added properly to the DHCP server using the Foreman Proxy, but it would cause a traceback on the server and prevent the Host change from being saved. This bug was fixed in 1.14 and I was finally able to get this working. There was one more bug in the PXEGRUB2 boot template involving an assumption about Profiles. I opened an issue and have submitted a PR to the community templates for this.

Foreman 1.14.2 was also missing the Preseed default PXEGrub2 template, but one had already been submitted to the community templates repo, so I had to manually add this template to my provisioning templates.

TFTP preparation

Foreman adds a DHCP record which contains the following:

server.filename = "grub2/grubx64.efi";

First step was to find the proper grub2 binary. Fortunately the Ubuntu wiki had a helpful post covering UEFI PXE netboot

I was able to find the xenial grubnetx64.efi here. But it turns out the Debian/Ubuntu grub2 is missing a few useful features that have been added to the Fedora grub2. The Ubuntu/vanilla grub2 only looks for grub/grub.conf whereas the Fedora grub2 has patches to search the grub2 directory and search for grub.cfg-[mac address] which is a convention that Foreman expects. Since Foreman is a project mostly run by RedHat employees it makes sense. The Fedora prebuilt grub2 bootloader is here.

There is PR which adds a default grub/grub.cfg and uses the grub2 regexp feature to search for $prefix/grub.cfg-[mac address]. This means that will support vanilla Grub2 soon.

Foreman will also place the correct kernel and initrd into the boot directory. It will not replace an older kernel, so sometimes a newer kernel and initrd need to be download from here and manually added to the boot directory.

How does it work?

Here is how this works:

  1. Put the host in build mode. This sets up the grub2/grub.cfg- file with the automated build setup. It also adds a DHCP entry specifying to download the "grub2/grubx64.efi" file.
  2. Start PXE boot and UEFI retrieves IP, filename and next-server/TFTP from DHCP server
  3. UEFI downloads grub2/grubx64.efi from TFTP
  4. GRUB2 looks for grub2/grub.cfg-[mac address]
  5. Grub2 template contains the automated install configuration generated by Foreman
  6. GRUB2 downloads kernel and initrd and boots the kernel and starts the installer
  7. After install is complete, PXELinux template is changed back to chainload local disk

Conclusion

The deficiencies of the previous process have been addressed. GRUB2 can boot the 16.04 kernels and even the hwe kernels and installer if I want to. The menus and boot to local disk are working.

Pages