Libreboot Build System Audit 6

Leah Rowe

19 July 2024


Return to index

Article published by: Leah Rowe

Date of publication: 19 July 2024

Heavy amount of code reduction in this audit, and general cleanup. A new Libreboot release is planned, for the early days of August 2024.

Introduction

Libreboot is a free/opensource boot firmware project. It replaces your proprietary BIOS/UEFI firmware, on supported x86 and ARM computers. It does this by providing an automated build system to download, patch and compile the various upstream sources (e.g. coreboot, GRUB, SeaBIOS). Coreboot is used for hardware initialisation, configuring everything from your CPU, memory controller all way to peripherals, readying the hardware so that it can run software, e.g. Linux/BSD operating systems. You can essentially think of lbmk, which is Libreboot’s build system, as a source-based package manager. It is what the Libreboot releases are built with. The lbmk build system essentially implements a coreboot distro, the same way you might think of a Linux distribution; it systematically downloads, resets (to specific revisions) and patches various upstream project sources such as coreboot, U-Boot and GRUB, automatically building entire coreboot images. This build system is what creates Libreboot release images and source tarballs.

Extensive auditing has been performed on lbmk, since the Libreboot 20240612 release. These audits fix bugs, reduce code bloat and generally improve the efficiency of lbmk, adding and removing features in a careful, conservative way, with a focus on clean code.

Remember the magic words: code equals bugs. Fewer lines of code will yield fewer bugs.

This article covers changes from Libreboot 20240612, up to revision 31f1e4dadfcc1ceecacec50dd2a14e63a44364bd from 19 July 2024.

You can read about the previous audit in the article for Libreboot Build System Audit 5.

Notable code size reduction

There are 1109 lines of shell script in the build system, versus 1482 in the Libreboot 20240612 release. Libreboot’s build system is written entirely in POSIX sh; not BASH, not KSH, not ZSH, jush sh! It is the build system of build systems; the design focuses on re-using (read: abusing) the build systems of the projects it uses, avoiding as much bloat as possible. More information is available in the lbmk build system documentation.

This is a difference of 373 lines, or a 25% reduction. The main focus has been on removing legacy bloat from the build system, vastly increasing the overall code cleanliness and efficiency. Several new safety and redundancy features were added, such as auto-rebuild when configs change(and auto-redownload) for sources, and additional redundant handling of both repositories and file downloads, cached during operation to avoid over-use of internet bandwidth.

Plan for next release

This code reduction has been the main focus; the focus now, at the next release, will be new payloads and boards, in addition to testing newer upstream revisions of projects such as coreboot, on every machine supported by Libreboot. A release is planned for early August 2024.

A lot of work on new ports is planned. There are a number of new mainboards that will be available, in the next Libreboot release.

Summarised list of changes

The most interesting changes are marked in bold. “Interesting” means that the change greatly improves the usefulness/reliability of Libreboot, or that it affects the user in a profound and noticeable way.

Changes are in order per category, from newest to oldest:

Feature changes

Some unused features have been removed, and yet more added. The overall focus of Audit 6 has been to remove legacy cruft from lbmk, and in general to simplify the operation of lbmk, Libreboot’s build system. Several new safety features were implementated for this audit, in addition to features that generally improve the redundancy of certain operations, and overall reliability in lbmk.

The changes are as follows:

Configuration changes

This pertains to anything under config/, for any changes that are of note, but it does not pertain to revisions for specific projects, nor does it cover specific patches applied to projects; the latter will be covered in other sections of this audit announcement.

The changes are as follows:

Bug fixes

There are fewer acute bug fixes in Audit 6, because bugfixes was the primary focus of Audit 5. The word acute is used, referring to triggered bugs, because many of these changes are pre-emptive bug fixes, preventing future bugs that have been foreseen, based on future modification to existing logic. The focus of Audit 6 has been general code reduction, feature simplification and improvements to safety features / error handling (many pre-emptive bugfixes implement newer safety features, but are listed as pre-emptive fixes, not features).

The changes are as follows:

General code cleanup

Extensive code cleanup has been performed, as was the main purpose of Audit 6.

The changes are as follows:

Revision updates

U-Boot

Alper Nebi Yasak is the maintainer of U-Boot, within Libreboot, and submitted a patch updating U-Boot to v2024.07, on the gru_bob and gru_kevin Chromebooks. Several improvements have been merged upstream, like Alper’s patches for Qemu, so these patches have been dropped in lbmk because they’re now included by default, in the new U-Boot revision.

Alper also tested the latest coreboot revision and it worked fine, on these boards, but revision updates will be done immediately after Audit 6 is announced, as a priority for further work that is to be completed for the next Libreboot release, ETA August 2024.

Git log

This git log covers all changes in this audit, relative to Libreboot 20240612.

* 31f1e4dad vendor.sh: don't use XBMK_CACHE for appdir 
* 2b50b3ea9 put cachedir in environmental variable 
* 0e0b12a63 git.sh: warn when a cached clone fails 
* 9b1b95576 git.sh: fix typo in git command 
* 82bdf2707 git.sh: fix lack of error exits on fault
* 64283a1fb build: remove tmp/ on release archives
* bf85246c4 trees: hardcode makeargs for xgcc 
* c6e6d96ce allow using coreboot's build system to add payload 
* 78cdc56ae trees: remove unnecessary command 
* e69903789 build: remove cache/ on making releases 
* 79f50b379 unify caching of files into cache/ 
* a9f9de885 unified cache file handling for vendorfile/subfile 
* ef867eccc git.sh: remove previous tmprepo if args>5 
* 4d5385a14 git.sh: try direct clone if cached git fails 
* 0cd52fc7f git.sh: re-try git pull three times 
* 0b9cd77fe trees: auto-delete+auto-rebuild if project changes 
* 4438368c0 trees: also remove single-tree repo/p/ 
* 570e3abb2 trees: remove repo/p, not src/p/p 
* 583502027 git.sh: don't download to src/project/project/ 
* 9f09728ac git.sh: cache git downloads to repo/ 
* f367afabc remove util/autoport 
* 373c2eb23 trees: move dependency building to new function 
*   12c3956f5 Merge pull request 'Update U-Boot to v2024.07' (#225) from alpernebbi/lbmk:uboot-v2024.07 into master 
|\  
| * 708fc14c6 u-boot: Fix display initialization on gru boards 
| * 0cc7736b4 u-boot: Avoid breaking build for U-Boot-only binman images 
| * 2ecec55af u-boot: Update to v2024.07 
|/  
* 090cf7ad5 trees: fix bad comparison in configure_project() 
* e37779c11 trees: don't continue if no argument given 
* 3263eeb6b trees: general code cleanup 
* 17f3e61a0 trees: merge build_targets() with handle_targets() 
* c0a4df688 trees: use wrapper for dry-running make commands 
* 459db1cc2 trees: remove project-specific hacks 
* e9f66ec4a remove executable permission on include/ 
* 91994b785 rom.sh: actually say if a cbutil exists 
* 97b777715 rom.sh: avoid re-building cbutils if built 
* 494b94799 rom.sh: only before cbutils before coreboot 
* 469cc64f9 trees: fix bad comparison 
* 5a1d2401c minor cleanup 
* 964617214 trees: support -d (dry run) for custom build logic 
* e850c06bd rom.sh: only make rom tarball if release=y 
* 2f3cc5d37 rom.sh: new file, to replace script/roms 
* 889afe168 roms: remove unnecessary trees checks 
* c241a3ef4 coreboot: set build_depend on target.cfg files 
* 6d4fb200a trees: rename load_target_config() 
* 87df6ae36 trees: support build dependencies, not just fetch 
* 23ca49bee GRUB: only load xhci from grub.cfg 
* 065453b72 trees: just do makeargs on coreboot, not cbmakearg 
* 2ab1d9949 trees: fix bad rm -Rf command (-Rf, not Rf) 
* d6ff009e2 roms: fix bad comparison in configure_target() 
* 3ee045f9a GRUB: use mkhelper.cfg for common variables 
* 3ef84af9f trees: allow global config on multi-tree projects 
* 2b49714a6 trees: handle all projects if project undefined 
* b7f12ade0 vendor.sh: remove mkdirs() 
* 0018600d6 git.sh: simpler for loop in git_am_patches() 
* 5882056a2 git.sh: merge for loops in git_am_patches() 
* 67421a21f trees and git.sh: tidy up global variables 
* 55dbd72aa git.sh: simplified initialisation of "loc" 
* 0cb84a8dd trees: simplified distclean directory check 
* 5ba0433b5 git.sh: condense fetch_targets() a bit 
* 4772186b1 git.sh: short git_prep command in fetch_targets() 
* c62dbdbe8 trees: only do bare multi-tree clone on git 
* 89a81a291 trees: simplified multi-tree bare repo clone 
* 017fd8259 git.sh: merge prepare_new_tree with fetch_targets 
* f937a1142 trees run_make_command: simpler distclean handling 
* 88f741db9 trees: condense copy_elf() a bit 
* d5928c9ba trees: add return to handle_defconfig() 
* 1a7c8acb0 trees: rename check_config to check_defconfig 
* ba7b3a3b2 trees: remove variable "config_name" 
* 030f1afd8 trees: rename variable "config" to defconfig 
* 739496061 git.sh: remove duplicate "xtree" variable 
* 0c4ad24b0 trees: remove unnecessary commonts 
* eacc41e82 trees: condense run_make_command() a bit 
* 563948141 trees: condense handle_makefile() a bit 
* 1ec9fc336 trees: mv load_project_config load_target_config 
* 789631ccb trees, multi: download bare project *before* trees 
* a5e724e51 trees: unified multi-tree configuration handling 
* cfc9f62ff trees: unified handling of source downloads 
* 7bf283520 git.sh: rename Fetch_project_trees fetch_targets 
* e0c244f37 git.sh: rename fetch_project_repo to fetch_project 
* 02f741795 trees: better skip-clean string in handle_src_tree 
* f7ec42d37 trees: add return to check_coreboot_utils() 
* 8d02adfbb trees: simplify "utilmode" variable initialisation 
* d1b7882c5 vendor.sh readcfg: split the -b coreboot command 
* fbe36b01a trees mkpayload_grub: remove unnecessary check 
* 088a79918 vendor.sh: remove unnecessary check 
* 8c5f78d3d vendor.sh: condense inject() a bit 
* 42925ce79 vendor.sh extract_kbc1126: use quote on file check 
* ddcc74777 vendor.sh extract_kbc1126ec: simplify build check 
* 7a12cd770 vendor.sh: simplify e6400 vga rom file check 
* 163bcfcdf vendor.sh: simplify variable checks for e6400vga 
* cbac2087c vendor.sh: condense fetch() a bit more 
* 675d2c037 vendor.sh: remove unnecessary check 
* 5e46b9f43 vendor.sh: simplify kbc1126 util handling 
* 7086b6868 vendor.sh: simplify bootstrap() utils handling 
* 6a05487ef vendor.sh: simplified initialisation of _7ztest 
* 972e61070 vendor.sh: condense detect_board() a bit 
* bf5acedb1 vendor.sh: condense patch_rom() a bit 
* 8b8cf070d vendor.sh: add return to end of bootstrap() 
* d85b9de2d vendor.sh: rename getcfg() to readkconfig() 
* 53b69f673 vendor.sh: merge cfgutils() into readcfg() 
* 55677d821 vendor.sh: remove build_dependencies_inject() 
* c76419861 vendor.sh: simplify initialisation of variables 
* d63da943c vendor.sh inject: skip serprog targets 
* c6df9666a vendor.sh: don't inject /dev/null 
* f7ab70f60 vendor.sh: simplify inject() 
* ee5bab2c9 vendor.sh: clean up GbE handling 
* f2d53eb84 vendor.sh: condense fetch() a bit 
* 85c2eb0f7 vendor.sh: general code cleanup 
* a8f272536 lib.sh: stricter check in chkvars() 
* fa5af956c vendor.sh: simplified modify_gbe handling 
* 155ce49b1 vendor.sh: simplify modify_gbe() 
* 08bce7ad3 vendor.sh: minor code cleanup 
* 636b86237 vendor.sh: re-add modify_gbe() 
* 2eb0163cc Revert "vendor.sh: remove unnecessary check" 
* 3f37c3788 tidy up some setvars lists 
* ed3cce84b roms: explicitly shift by 1 on the "all" target 
* 359d00bd5 roms: add return value for the list command 
* 3f540e756 roms: build u-boot *before* checking ubootelf 
* afc003068 remove more unnecessary checks on trees commands 
* 35c487106 vendor.sh: remove unnecessary checks 
* ce03786f6 vendor.sh: remove unnecessary command 
* e01b5c974 vendor.sh: remove unnecesessary cbutils check 
* 8196ce6dd vendor.sh: remove unnecessary check 
* 8f50131e1 vendor.sh: condense check_board() a bit 
* d12a68601 vendor fetch(): condense variable initialisation 
* 708e306e5 vendor.sh: remove unnecessary cbfstool check 
* 75951c469 vendor.sh: simplified cbfstoolref check 
* d36c07847 lib.sh: keep versiondate check to 80 characters 
* 40d3bb19b lib.sh: condense for loop 
* 5725e3c4e lib.sh: condense err_() a bit 
* e3546f77b lib.sh: add a return to the end of chkvars() 
* d0f68a0fb vendor.sh: rename release to vrelease 
* 473f27784 vendor.sh: unified reading of board configs 
* a2f4eba58 vendor.sh: simplify variable check in getcfg() 
* 4afcbcb64 vendor.sh: remove unnecessary cbdir check 
* fd037722a remove use of _xm variable in the build system 
* 136787185 trees: don't hardcode use of mkpayload_grub 
* c59fbb365 trees: simplify single/multi handling in main() 
* 902b98d1a trees: rename build_projects to build_project 
* e1e04aa80 trees: err if target.cfg not given if multi-tree 
* 93ff80d96 trees: set btype if target.cfg doesn't exist 
* 732c5908d lib.sh: remove unused cbdir variable 
* 119cebc4d roms: remove unnecessary assignment to cbdir 
* 7a15ba18c trees: avoid kconfig make commands generically 
* eb9c1872b git.sh: remove unnecessary check 
* 9aec992ff lib.sh: move git_err() to build 
* 6fc7cd3c1 lib.sh: condense singletree() a bit 
* 35d09e426 lib.sh: add a return to the end of check_defconfig 
* 144b3c93a trees: condense elfcheck() a bit 
* 7b8c2bd41 lib.sh: condense e() a bit 
* ffdecb0ce trees: shorten the final confirmation message 
* ed6acfee8 lib.sh: make elf/coreboot* a dot directory 
* 44fb98736 build: don't rm TMPDIR if it's /tmp 
* fac74cd60 lib.sh: simplified TMPDIR handling 
* 671893a80 lib.sh: condense setcfg() if/else logic 
* 19bc5845a trees: remove redundant space in printf 
* 9154ba5b8 trees: explicitly err if OPTARG is not set 
* f383b1ad7 trees: only permit one single-tree project 
* a64e2db58 trees: call err if multiple flags are provided 
* 0e97e98fe trees: explicitly set mode for -b 
* 7f7b640d3 roms: re-add compression of coreboot images 
* 1ab9189cb roms: build coreboot *after* checking variables 
* 2b4c6ee5c lib.sh: introduce mandatory check of projectname 
* c79fb125e lib.sh: condense setvars() a bit 
* 1387dc0ae simplified lock message 
* 20ac7ec67 lib.sh: simplify reading of version files 
* 4e48fa808 lib.sh: simplify use of environment variables 
* b4fb25100 roms main(): confirm what serprog images are built 
* 5e2308ce0 roms: remove unused variable name 
* 2960abd00 roms: remove redundant printf 
* c661eca6b roms: optimise u-boot elf check 
* 8af9f904c roms: simplify build_roms() 
* c9f26dbc9 roms: make the bin/elf message more polite 
* 9484eda5a roms: re-add final confirmation of targets 
* 2b7df7ab2 roms: rename functions for extra clarity 
* 6753222d0 roms: build coreboot early to avoid duplicate work 
* 19e7c1eab trees: try xgcc build twice if first attempt fails 
* 6468bdb3a trees: don't check if xgcc is already built 
* d5baaff0e lib.sh: fix error running ./build dependencies 
* 0158a0811 roms: general code cleanup 
* e67cd1716 roms: only support SeaBIOS/SeaGRUB on x86 
* 71137b12b roms: remove support for "grubonly" seabios 
* d4d5d2902 use backticks on eval commands, not subshells 
* c1527b611 lib.sh: remove badcmd() 
* fc7ae3e59 lib.sh: more unified config handling 
* ad1602569 trees: more robust check to avoid "make fetch" 
* a6b1a6bdd roms: fix lack of backslash on multi-line command 
* 8c4a91850 vendor.sh: more cleanup 
* 471129684 Revert "roms: remove build_payloads() and split it up" 
* 5a4fc97c6 vendor.sh: correction (s/scancfg/scan_config) 
* d65e4fac1 git.sh: revert modification to for loop 
* bc61c39ec vendor.sh: minor code cleanup 
* 808458ced minor code cleanup in the build system 
* 4ab99d546 git.sh: general code cleanup in fetch_submodule() 
* cd1d84789 git.sh: reduced indentation on repo/file check 
* 4f6dda136 git.sh: simplified repo/backup check 
* 185d76f57 roms: merge mkserprog() into main() 
* 893e88bc8 roms: don't insert timeout.cfg 
* abfc799fd correction 
* a0da8fdef roms: reduce indentation in build_grub_roms() 
* 383433d4f roms: re-introduce accidentally disabled check 
* 3610667e3 roms: remove build_payloads() and split it up 
* 29a7123c0 roms: group some commands that are similar 
* 2d6946775 roms: remove mt86bin variable 
* 920e5ba2d roms: merge build_uboot_payload to build_payloads 
* a96c4b59f roms: simplify payload_uboot y/n check 
* 28682b1a4 roms: simplify the check for serprog srcdir 
* b61dd4c25 roms: simplify the loop for building serprog roms 
* 6df17860e roms: shorten variable serprog_boards_dir 
* ea5b5b072 roms: simplified serprog image copy 
* ea9bdfce4 roms: rename picosrc variable to rp2040src 
* 1a4f97ffd roms: remove useless confirmation in mkserprogfw 
* 1881d34db roms: merge serprog build into one function 
* f7e28964a roms: remind the user about gkb files 
* f928ac5c7 roms: rename x variable to it in for loop 
* 58a451865 roms: don't use x_ to call cproms() 
* bc853fbb8 roms build_uboot_roms(): move rom, don't copy 
* 12b26f207 roms cproms(): allow other commands besides cp 
* e67628c6a unify coreboot elfdir (DO_NOT_FLASH) 
* 8b58c1eac roms: merge mkUbootRom() into build_uboot_roms() 
* c3f0a109c roms: simplify mkSeabiosRom() 
* c8944f1ca roms: simplify mkUbootRom() 
* 92aa83a23 roms: simplify build_roms() 
* d3e788645 roms: remove unnecessary check 
* ef411c596 build: remove unused variables 
* 6dc051558 roms: further clean up build_grub_roms() 
* 0e9b36c58 roms: simplify timeout/scandisk insertion 
* 3a7b3660f roms: simplify seagrub check in build_grub_roms 
* 4b764d26f roms: simplify mkserprog() 
* 167e7447a roms: simplify the serprog build functions 
* 7bc9fcc34 script/roms: fix serprog build commands 
* 0dfe3aed9 roms: simplified ubootelf check 
* a9166898d roms: simplify grubonly check in configure_target 
* 90017cdc5 roms: simplify seagrub check in configure_target 
* 817004e15 roms: don't use x_ to call build_grub_roms 
* 0812d5321 trees: simplify copy_elf() 
* 331c4097f trees: remove unnecessary check in copy_elf 
* 340eea0b1 grub: insert background in memdisk instead 
* ed9c90e59 roms: unify all add-payload commands 
* 26451775d roms: don't add grub keymaps at all 
* af8296ce6 roms: merge handle_coreboot_target into main() 
* d1c0c3464 roms: simplify target check (whether roms built) 
* f626b25db roms: simplify main() again 
* eb9a688ee roms: remove redundant check on grub_scan_disk 
* a4328cb11 roms: remove dangerous runtime p/s/d options 
* 167e1a0fb unified checks for variable initialisation 
* 98724d701 lib.sh: remove the items() function 
* baea03c67 roms: simplify main() 

This is 247 changes in total, between 12 June 2024 and 19 July 2024.

Markdown file for this page: https://libreboot.org/news/audit6.md

Subscribe to RSS for this site

Site map

This HTML page was generated by the Untitled Static Site Generator.