Practical Use of GRUB's DSL
With the examples inexplicably left out of the GRUB documentation
by Giles Orr
ed. Daniel Wayne Armstrong
2018, GRUBlishing company
Toronto, ON
Canada
License: CC-BY-SA (Creative Commons with Attribution, Share-alike)
Last modified: 2018-10-15 22:01
Table of Contents
- Durability of This Document
- Document Conventions
- Getting to the GRUB Boot Prompt
- Creating a GRUB Command Line USB Stick
- The GRUB Command Line
- grub.cfg
- Language Similarities to Shell - and Caveats
- Special Environment Variables
- Menus
- Video Modes
- Fonts
- Background Images
- Functions
- The grubenv Variable
- Playing Music
Durability of This Document
I often find myself with an itch to scratch and get fascinated by a particular topic - such as GRUB's DSL - and proceed to dig in ... to a certain level. Usually until my own requirement is satisfied. And then I'm happy, and I move on to some other project. I'll do the same thing here. Feel free to contact me with changes, fixes, and updates, which will probably be applied. If you have an interest in the subject and would like to maintain this document, contact me (gilesorr+grubdsldocument@gmail.com). If this document hasn't been updated in 18 months, go to town - take it over. It's CC-BY-SA, so you can update it and republish it.
Document Conventions
Commands executed at the Linux command line will appear in this style: grub-mkfont. Commands executed at the GRUB command line will appear like this: ls (hd0). A block of interactions at the Linux command line will appear like this:
Similarly, a block of interactions at the GRUB command line will look like this:
Finally, code will be presented like this:
# define a variable:
function_folder='(hd0,msdos1)/boot/grub/grubbits'
# define a function:
function modules {
insmod regexp
insmod ext2
}
Getting to the GRUB Boot Prompt
The simplest way to get to a GRUB prompt is to reboot your Linux computer and just press "c" when you get to the GRUB menu to go to the command line, but since my intent is to learn GRUB to control a bootable USB stick, I'm going to recommend the extra step of creating a USB stick. As has been noted, some Linux versions seem to handle their own GRUB menu system and commands differently, and I think there's more consistency in the bootable USB sticks that can be created by the process described next.
Creating a GRUB Command Line USB Stick
Several (possibly) important assumptions here:
- I'm working on x86 (both 32-bit and 64-bit), I don't know if this will work on other platforms
- I have yet to successfully create a EFI-only bootable GRUB USB stick, although most (not all, and I can't explain the exceptions) BIOS GRUB sticks will boot in "legacy" mode on EFI systems
- I'm not trying to boot kernels (a common use for the GRUB command line) - although I do eventually use it as an example. I'm doing this to learn the GRUB DSL
- I've always used 'vfat' formatted USB sticks: this would probably work with other formats, but I haven't tested
Start by inserting a USB stick you don't mind wiping. Size isn't important: the material you're going to install is considerably less than a Gigabyte in size. And we're not even going to wipe the stick, but you probably shouldn't take my word for that. Determine the stick's ID/mount point using whatever utility suits you: lsblk, fdisk -l, even dmesg - whatever works.
As a user:
Be very sure that the USB stick is in fact /dev/sdx1, and if it's not, substitute the correct value here and in any following commands. If you're not using pmount to mount your removable media, just get that partition mounted however works for you. In my case, the target media is now mounted at /media/sdx1. This won't wipe any files (unless you already have GRUB-specific stuff on the stick). As root:
Fedora users will need to use grub2-install. Notice that I've named a directory that didn't previously exist: grub-install is willing to create it. And more important, notice that the target device at the end of the line is the ENTIRE DEVICE, not just the partition we're installing the files to. I made the mistake of doing a grub-install to /dev/sdx1 because I was putting the boot directory there: that doesn't work.
This command is often listed on the internet with --force and --recheck. Read the man page if you have problems before using either of these: I didn't need either, and you'd only need them if you're trying to achieve something different from what I'm doing here.
There are undoubtedly a lot of possible errors that can happen with grub-install, but this is one we've seen:
Fix: Note the sector where first partition begins:
Zero out the gap between boot sector and first partition:
GRUB should now install properly. (Source)
Unmount the USB stick, remove the stick, and put it into a computer that will boot from USB (this won't make any changes to the computer you're using ... unless you manage to issue some complex and nasty GRUB command - the boot process itself won't hurt it). When you boot to the USB stick, you won't see a menu because we haven't installed a grub.cfg file: you'll go immediately to the grub command line:
(Keep in mind that screenshots and/or text dumps aren't implemented in GRUB so I'm writing all of this by hand: there could be typos.)
The GRUB Command Line
You'll find that the GRUB command line uses the Scholes (aka "Qwerty") keyboard layout. For those of you that use Dvorak (as I do) or Colemak, sorry - you're out of luck. If you do an online search, you may find references for Dvorak. Unfortunately, it appears this was a GRUB v1 thing that's no longer maintained - I spent a LOT of time trying to get that feature working ... I'm less clear about more common other-country layouts like Azerty.
After using one of the previously discussed methods to get to the GRUB command line, try this:
GRUB's command line behaves a great deal like a Unix shell, albeit more like 'sh' or the old Bourne Shell rather than Bash or ZSH. It's borrowed a number of commands from the shell, as well as similar logical constructs.
What we see above is a list of available devices. In this case, we have hd0 which is the USB stick I booted from, and hd1, the hard drive in the laptop I'm using. Both appear to have only one partition: the one on the USB stick is shown as hd0,msdos1. It's important to note that drive numbering is zero-based, and yet partition numbering starts at one. Details of GRUB's Naming Convention: https://www.gnu.org/software/grub/manual/grub/html_node/Naming-convention.html#Naming-convention. It's also worth keeping in mind that the zero-based and one-based numbering HAVE CHANGED IN THE PAST, and may do so again.
ls can also be used to view files on partitions ... but don't rush to type ls (hd0) yet. While that's a correct command, you need to inform GRUB about the file system types it will be looking at (it already knows about the one on the USB stick or it wouldn't have been able to boot):
One of the things the grub-install command did was install a bunch of modules to /media/sdb1/boot/grub/i386-pc/ (Fedora users will find this under /media/sdb1/boot/grub2/i386-pc/). To see a list of these modules and their sizes, run (at the grub> prompt) ls -l (hd0,msdos1)/boot/grub/i386-pc/. But there's a problem: there are a LOT of modules, and they scroll off screen. To fix this, type pager=1 and enter, then press the up arrow key twice (happily, GRUB supports command line replay and editing). The command output will now pause every page of output until you press a key. You'll find that GRUB's implementations of both ls and the pager are dumb as stumps - not exactly what you're used to from your regular command line environment. But this isn't surprising: this is a boot loader, not a full OS - or even a shell.
Rudimentary help for these commands can be obtained by typing help insmod or whatever command you're interested in.
GRUB has a LOT of modules for filesystem support, including many more that you'll ever use. Knowing which ones to use and when - and what the module names are - is tricky. Do you remember if this machine has LVM on it? Or is it raw partitions? GRUB won't tell you, you have to keep using insmod <filesystem-module-name> until ls shows you something. Filesystem modules are worthy of another entire section: as I write, I'm forced to admit I don't know if I'll get that far.
grub.cfg
This document is mostly about the grub command line, but if you're going to tinker with that much you'll want to know how to set up your command line when your USB stick boots. The grub.cfg file will help with that. If you're familiar with Bash, think of it as the /etc/profile for your shell.
On your USB stick, create and edit the file boot/grub/grub.cfg. It can look something like this:
loadfont $prefix/fonts/unicode.pf2
insmod all_video
insmod gfxterm
terminal_output gfxterm
export pager=1
This loads the default font (always supplied when you create a GRUB boot device), inserts the modules for video and a graphical terminal, and then switches from the default text terminal to a graphical terminal. The last line turns on the pager by default.
Language Similarities to Shell - and Caveats
I'm afraid I'm not up to doing a comparison of Backus-Naur forms for the two languages ... or even a full syntax listing. This is a list of similarities and differences I've noticed - in no particular order.
- GRUB has variables, very similar to sh
- GRUB does NOT support pipes
- no command substitution
- you can pass parameters
- looping is supported (over a list ... what about numbers?)
- math??
- yes file globbing - but only with after insmod regexp!
- command line replay and editing (but no 'history' command)
- uses a lot of the same keywords
- adds a lot of GRUB-specific commands
- tab completion works only for commands (GRUB's original commands, not any you've created) on the first word, files on everything thereafter
Special Environment Variables
As usual, a language that supports variables has a bunch of names that are off bounds. These are "Special Environment Variables," and they're listed here: https://www.gnu.org/software/grub/manual/grub/html_node/Special-environment-variables.html#Special-environment-variables.
The full list: biosnum, check_signatures, chosen, cmdpath, color_highlight, color_normal, config_directory, config_file, debug, default, fallback, gfxmode, gfxpayload, gfxterm_font, grub_cpu, grub_platform, icondir, lang, locale_dir, menu_color_highlight, menu_color_normal, net_<interface>_boot_file, net_<interface>_dhcp_server_name, net_<interface>_domain, net_<interface>_extensionspath, net_<interface>_hostname, net_<interface>_ip, net_<interface>_mac, net_<interface>_next_server, net_<interface>_rootpath, net_default_interface, net_default_ip, net_default_mac, net_default_server, pager, prefix, pxe_blksize, pxe_default_gateway, pxe_default_server, root, superusers, theme, timeout, timeout_style.
Don't use any of these names as variables, except to reference the GRUB-special-variable. Some of these are quite useful (you've already seen pager and you may know timeout) and we'll be using several as we work through examples.
Menus
As soon as you add a menuentry entry anywhere in the grub.cfg file, GRUB will boot to a menu rather than a command line. Add the following to your grub.cfg:
menuentry "Show the contents of grub.cfg" {
cat /boot/grub/grub.cfg
}
This will do what it suggests, dumping the contents of the grub.cfg to the screen.
Once you're in a menu, you can always go to the command line just by typing the letter "c".
menuentry is most commonly used to add bootable operating systems and/or kernels. This is one of the simplest examples - although it won't work unless you copy the named file to your USB stick):
menuentry "MemTest86+ 5.01 Binary" {
linux16 /boot/iso/memtest86+-5.01.bin
}
I usually get a MemTest86+ binary by installing the package on my OS and then copying the binary it installs, but it can also be downloaded from http://www.memtest.org/#downiso - get the on labelled "Pre-Compiled Bootable Binary," and un(g)zip it before you put it on the GRUB stick: if you use the folder named above, you'll need to create the iso/ folder on the USB stick before placing the binary in the folder.
NOTE: MemTest86+ works on the vast majority of computers, but there are some machines (the Acer c720 Chromebook is known for it) that MemTest86+ freezes on.
You can add multiple menuentry items to your grub.cfg, and they will be displayed in the order added.
If the menu structure becomes complex, it can be divided and controlled with the submenu command:
submenu "Documentation >" {
menuentry "Show the README" {
cat /boot/grub/README.txt
}
menuentry "Show the contents of grub.cfg" {
cat /boot/grub/grub.cfg
}
}
menuentry "MemTest86+ 5.01 Binary" {
linux16 /boot/iso/memtest86+-5.01.bin
}
(The README.txt file isn't there by default: I added it. It includes comments about the USB stick, including instructions on how to use the menus and functions.
I add a '>' at the end of a submenu name to indicate to the user that they're entering a submenu: this is NOT a required element, just a personal preference.
It's possible to programmatically generate menuentry items: I attempted it looping over
the list of available fonts with each menuentry saying "Change to font
Video Modes
GRUB is able to detect and use multiple video resolutions and colour depths. I've encountered video hardware where it didn't detect all available modes (the Intel i915 chip ... more than a decade old), but even in that case you could make it work if you knew what you were doing.
GRUB usually - not always (may depend on the opinion of the person who put GRUB on your Linux distribution that made the USB stick) - defaults to the equivalent of an 80x25 terminal, but the pseudo-graphical modes allow higher resolutions (meaning more text on screen), user-defined fonts, and background images.
At the GRUB font:
The first three commands do nothing visible, but you are 1) loading GRUB's one font (seems to be standard across all distributions), 2) loading the all_video module, 3) loading the graphical text terminal, and finally 4) activating the graphical text terminal. On some versions you may already be in the gxfterm, in which case you won't see any change at the last command.
Now that we're in a graphical mode, we can change the screen resolution. To find out what video modes are available (and which one is being used):
The mode with the leading '*' is the currently chosen mode. Since I have to copy this by hand (GRUB doesn't have screen capture that I'm aware of), I've excised a great deal of the output. There was an intro, including information on the video drivers available and lots of other modes - none of which were 1280x800 (a mode I know this monitor supports), presumably because GRUB doesn't support that even if the monitor prefers it. The output varies considerably from machine to machine. NOTE that the videoinfo command won't show you the available modes until you've loaded the all_video driver.
To change to another video mode:
Make sure the value you set is a valid one from the output of videoinfo. But just setting a new value isn't enough: you have to restart the graphical terminal by switching to another terminal mode and then back to gfxterm - thus the second (compound) command.
A useful feature of gfxterm is that it accepts multiple, comma-separated resolutions. Each will be tried in order until it finds a working mode: so try desirable modes first and make sure to have a good fallback:
This is less useful at the command line where you can run videoinfo and see what's available, but it's a huge boon in scripts when you don't know the capabilities of the machine in advance.
Fonts
The GRUB manual says "The fonts GRUB uses 'PFF2 font format' bitmap fonts." This is a GRUB-specific file format that doesn't seem to be well described: the best I could find was here: http://fileformats.archiveteam.org/wiki/PFF2.
To create and use different fonts, use the grub-mkfont command (at the Linux prompt, not the GRUB prompt!) - I recommend you read man grub-mkfont. The command claims to be able to convert "common font file formats" although it doesn't appear to expand on what those might be. So I assumed that this included standard X fonts:
This is a Debian system and apt-get and dpkg commands are specific to Debian. If you're not using Debian, you'd have to figure out your OS's equivalent commands. The font locations may also be different.
dpkg -L <package-name> lists all the files in the package. I filtered to find anything with my preferred code page (8859-1) and 'n' (indicating 'normal' rather than 'b' for 'bold'). Then convert the fonts. Since there was no output, I assumed it was succeeding. I copied the resulting PF2 files onto the GRUB USB stick under /boot/grub/fonts/ (the location isn't essential given that you can specify a full path to load the font files, but GRUB's loadfont command assumes this is the default location). Then, after booting GRUB:
And I have a different font. I've always been a fan of the Terminus fonts, so it was nice to be able to use them here.
Changing fonts (as opposed to setting the initial font) turns out to have the same problem as changing resolutions. You have to give the terminal mode a kick in the ass to get its attention:
Font Warning / Follow-up
The Terminus fonts converted perfectly, but several other attempts have resulted in fonts that ... work, but not well. Some conversions fail entirely. The "Artwiz Kates" (pcf) font converted without complaint, but has underlining under all words, and when I converted Droid Sans Mono Slashed (TTF) I ended up with a very similar problem - except at the 24 size, at which point I got some cropping at the bottom. So apparently converting fonts is a crapshoot.
Another problem with converting fonts is that most of them don't have line-draw characters, so if you use them in combination with GRUB's default menu system the box around the menu looks really horrible. (Or perhaps the linedraw characters are in the font - but unicode.pf2 stores them in an odd place so calls to other fonts get the wrong characters.) This is why unicode.pf2 is the default ...
Background Images
GRUB supports background images if you're in graphical (not text) mode. Background images can be JPG, PNG, or TGA files according to this Ubuntu documentation: https://help.ubuntu.com/community/Grub2/Displays. To support any of these, you have to have the proper module(s) inserted:
The first double-command ensures you're in a graphical terminal. If you've been following through this document, you were probably already in that mode. Note that, unlike the commonest usage, they've included an "E" in the "jpeg" module name.
The image must be 8-bit, RGB, and non-indexed. Unlike font or screen resolution changes, background images change immediately when the file is named:
That's all that's needed (although I created a new directory called pictures/ under boot/grub/).
Note that by default the image will be squished and stretched to fit the current screen size. The resizing appears to be beyond our control, but you can force GRUB to not change the image ratio with the option background_image --mode 'normal' <image>.
There appear to be limitations on the size of the file, as an attempt to load what I believe was an otherwise valid 2560x1440 JPG image failed, and size seemed the likeliest cause. There used to be a max size of something like 800x600, but this clearly no longer applies as I was able to load a 1280x720 TGA file without problems.
With a standard JPG image from a modern camera, the Gimp can be used to convert it to the needed format (you should probably reduce the image size before you do the colour conversion): go to the "Image" menu, select "Mode," and then "Indexed." Use the resulting dialogue to reduce the colour palette to 256 colours. (This colour reduction can make some photos look considerably worse: you'll have to be judge whether or not the damage is acceptable.) But now you have an indexed image: select "Image", then "Mode" and "RGB" to get back to RGB mode. Export the image in the format you want.
Keep in mind that colourful images can make the GRUB text very hard to read. Changing the colour of the GRUB text may help, but I usually try to use images that only have colour variety on the right third of the image or I deliberately mute the image colours.
Functions
What we're doing in this document is writing scripts in a language nobody bothers to script (aside from building menu systems), so I have very few examples to work from ... The stuff I've created may be a bit rough around the edges.
This example shows several things:
- item the strong similarity between shell and GRUB scripting
- item the basic structure of a function
- item and, as a function, how to see what GRUB knows about your processor
function cpuinfo {
# only able to determine: 32/64 bit, and is it PAE
echo "GRUB's ability to analyse processors is limited, we can only tell you:"
if cpuid -p; then
pae_assessment="PAE"
else
pae_assessment="NO PAE"
fi
if cpuid -l; then
echo "64-bit processor, $pae_assessment"
else
echo "32-bit processor, $pae_assessment"
fi
}
I store functions in a file with the same name (in the case above it would be cpuinfo, in a new folder under boot/grub/ called grubbits/. A block in the grub.cfg can be used to load them:
function_folder='(hd0,msdos1)/boot/grub/grubbits'
insmod regexp # needed for file globbing to work!
for file in "$function_folder"/*; do
source "$file"
done
Note the insmod regexp - if you don't do that, file globbing does not work and you can't say '*' to list all files in a folder.
Once you boot a stick with these two pieces in place - the function in its own function file, and the code block above in your grub.cfg, you can then type cpuinfo at the command line. Note that tab completion doesn't work for functions, only for GRUB's own built-in commands.
The videoinfo command (not available at the Fedora GRUB HD command line, but available everywhere else, including USB sticks made by Fedora) will show you what resolutions are available on your monitor. You can change to one of the other named resolutions:
function chres {
if [ ${1}x = x ]
then
echo "Please supply a resolution (ie. '800x600') from the"
echo "output of the 'videoinfo' command."
exit 1
else
set gfxmode=${1} # load the new resolution
terminal_output console # toggle the console to activate the new resolution
terminal_output gfxterm
fi
}
If you have alternate .pf2 fonts, you can change to them with this command (this assumes they're stored in the boot/grub/fonts/ folder):
function _chfont_help {
echo "Usage:"
echo " chfont <fontname>"
echo "Please supply a font name (without the .pf2 extension) from the"
echo "${prefix}/fonts/ directory."
}
function chfont {
newfont=${prefix}/fonts/${1}.pf2
if [ ${2}x != x ]
then
_chfont_help
elif [ ${1}x = x ]
then
_chfont_help
elif ! [ -f ${newfont} ]
then
echo "'${newfont}' doesn't appear to exist."
_chfont_help
else
loadfont ${newfont} # load the named font
terminal_output console # toggle the console to activate the new font
terminal_output gfxterm
fi
}
The above example defines two functions, one of which is a help function for the other.
Apparently pretty colours have been a priority for the GRUB community: they're easy to use and there are quite a few (well - by command line standards). Note that colour availability varies by platform: I use i386 and amd64, and colours are well supported there.
# This file echoes a bunch of color codes to the terminal to demonstrate
# what's available.
function colours {
for bg in 'black' 'blue' 'green' 'cyan' 'red' 'magenta' 'brown' \
'light-gray' 'dark-gray' 'light-blue' 'light-green' \
'light-cyan' 'light-red' 'light-magenta' 'yellow' 'white'
do
color_normal=light-gray/black # reset to readable
echo -n "$bg bg: "
for fg in 'black' 'blue' 'green' 'cyan' 'red' 'magenta' 'brown' \
'light-gray' 'dark-gray' 'light-blue' 'light-green' \
'light-cyan' 'light-red' 'light-magenta' 'yellow' 'white'
do
color_normal=${fg}/${bg}
echo -n "${fg} "
done
echo ""
done
color_normal=light-gray/black # reset to readable
echo ""
}
Try running the above on your system: you may decide to change your default GRUB colours.
I have an old Dell laptop that has a specific and slightly problematic video card. It was common enough that people wrote a lot of code to support it under Linux (and GRUB). Don't try running this unless you know you have a 915 video card and it supports this specific resolution, although it could act as an example to access a different 915-supported resolution:
function dell {
# override video mode 30 (usually 640x480x8) with a res for the
# Dell Inspiron 700m:
# ( http://915resolution.mango-lang.org/ )
915resolution 30 1280 800
set gfxmode=1280x800,1024x768,800x600,640x480,auto
terminal_output console ; terminal_output gfxterm
}
GRUB is mostly used as a menu system, but you also boot from the command line. This requires the memtest86+-5.01.bin binary in the isos subfolder of boot/grub/. Most ISOs or kernels are A) bigger, and B) require more settings to boot properly: I chose this as the simplest possible test (it works everywhere I've tested it).
function memtest {
memtestbin=${prefix}/isos/memtest86+-5.01.bin
if [ -f ${memtestbin} ]
then
linux16 ${memtestbin}
boot
else
echo "File ${memtestbin} not found to boot, quitting."
fi
}
Note the addition of the boot keyword to start the boot process: it's assumed in menu entries in GRUB, but it's required if you're writing scripts like this.
The grubenv Variable
When you create GRUB bootable media, one of the files created is /boot/grub/grubenv (or /boot/grub2/grubenv if you're using Fedora). This appears to be a simple text file. It can be shown with cat /boot/grub/grubenv from the GRUB command line (assuming your '$root' variable is still set to the default drive), but it can also be manipulated by a couple GRUB commands to interesting effect.
This command will change the colours you see in GRUB from the default light gray on black to green on black (color_normal is a GRUB "Special Environment Variable").
The system will reboot. But this is where it gets interesting.
This will pull the saved setting from the /boot/grub/grubenv file and change the colour of your GRUB terminal.
The most obvious use for this would probably be to save your previous OS boot choice as the default for the next boot. Other possible uses would include saving colour, font, or background picture choices from one boot to the next.
Here's a very basic implementation with a GRUB submenu. If you choose one of the last three entries, it changes the colours and saves the setting. If you choose the first entry, it reloads the last saved entry (including on another machine because the file grubenv is stored on the USB stick).
submenu "Ugly Colours and Environment Settings" {
menuentry "Load previous colour choices" {
load_env color_normal
load_env color_highlight
}
menuentry "light-gray/black and white/dark-gray" {
set color_normal=light-gray/black
set color_highlight=white/dark-gray
save_env color_normal
save_env color_highlight
}
menuentry "yellow/blue and light-blue/brown" {
set color_normal=yellow/blue
set color_highlight=light-blue/brown
save_env color_normal
save_env color_highlight
}
menuentry "white/green and green/black" {
set color_normal=white/green
set color_highlight=green/black
save_env color_normal
save_env color_highlight
}
}
The contents of the grubenv file now look like this:
That's not a literal space and three periods at the end - there's a single line of 947 hash marks (confirmed by piping to wc -c) that I'm not going to show here.
Playing Music
When I say "playing music" in the context of GRUB, we're talking about PC speaker single note unmodulated sound: it leaves a little to be desired. But it can be done. Another important detail you should keep in mind is that GRUB isn't multitasking: that means that as long as music is being played, nothing else can happen: you won't be able to use the menu or the prompt.
CAVEAT: This doesn't work on all computers, I have at least one where the commands given below run, but exit in about a second having not made a sound.
With all that said, here's an example that will play you a small piece of the "Close Encounters" theme (a favourite of this book's editor):
The GRUB play command (documentation) requires first a single number that is the "tempo." In this case, '480' sets the base as each note of duration '1' lasting for a quarter second. After the tempo follow pairs of numbers, the first of which is a pitch (in Hz), the second of which is duration (in this case in multiples of one-quarter second).
Here's the "Super Mario" theme:
Search online for a fine selection of other (short!) tunes.