Add distributed ThinLTO build support for the Linux kernel. This new mode offers several advantages: (1) Increased flexibility in handling user-specified build options. (2) Improved user-friendliness for developers. (3) Greater convenience for integrating with objtool and livepatch. Note that "distributed" in this context refers to a term that differentiates in-process ThinLTO builds by invoking backend compilation through the linker, not necessarily building in distributed environments. Distributed ThinLTO is enabled via the `CONFIG_LTO_CLANG_THIN_DIST` Kconfig option. For example: > make LLVM=1 defconfig > scripts/config -e LTO_CLANG_THIN_DIST > make LLVM=1 oldconfig > make LLVM=1 vmlinux -j <..> The build flow proceeds in four stages: 1. Perform FE compilation, mirroring the in-process ThinLTO mode. 2. Thin-link the generated IR files and object files. 3. Find all IR files and perform BE compilation, using the flags stored in the .*.o.cmd files. 4. Link the BE results to generate the final vmlinux.o. NOTE: This patch currently implements the build for the main kernel image (vmlinux) only. Kernel module support is planned for a subsequent patch. Tested on the following arch: x86, arm64, loongarch, and riscv. The earlier implementation details can be found here: https://discourse.llvm.org/t/rfc-distributed-thinlto-build-for-kernel/85934
Showing the changes to scripts/Makefile.thinlto (1 of 7 files). Show all
Unified · Split
| @@ -0,0 +1,40 @@ | ||
| 1 | +PHONY := __default | |
| 2 | +__default: | |
| 3 | + | |
| 4 | +include include/config/auto.conf | |
| 5 | +include $(srctree)/scripts/Kbuild.include | |
| 6 | +include $(srctree)/scripts/Makefile.lib | |
| 7 | + | |
| 8 | +native-objs := $(patsubst %.o,%.thinlto-native.o,$(call read-file, vmlinux.thinlto-index)) | |
| 9 | + | |
| 10 | +__default: $(native-objs) | |
| 11 | + | |
| 12 | +# Generate .thinlto-native.o (obj) from .o (bitcode) and .thinlto.bc (summary) files | |
| 13 | +# --------------------------------------------------------------------------- | |
| 14 | +quiet_cmd_cc_o_bc = CC $(quiet_modtag) $@ | |
| 15 | + be_flags = $(shell sed -n '/saved_c_flags_/s/.*:= //p' \ | |
| 16 | + $(dir $(<)).$(notdir $(<)).cmd) | |
| 17 | + cmd_cc_o_bc = \ | |
| 18 | + $(CC) $(be_flags) -x ir -fno-lto -Wno-unused-command-line-argument \ | |
| 19 | + -fthinlto-index=$(word 2, $^) -c -o $@ $< | |
| 20 | + | |
| 21 | +targets += $(native-objs) | |
| 22 | +$(native-objs): %.thinlto-native.o: %.o %.o.thinlto.bc FORCE | |
| 23 | + $(call if_changed,cc_o_bc) | |
| 24 | + | |
| 25 | +# Add FORCE to the prerequisites of a target to force it to be always rebuilt. | |
| 26 | +# --------------------------------------------------------------------------- | |
| 27 | + | |
| 28 | +PHONY += FORCE | |
| 29 | +FORCE: | |
| 30 | + | |
| 31 | +# Read all saved command lines and dependencies for the $(targets) we | |
| 32 | +# may be building above, using $(if_changed{,_dep}). As an | |
| 33 | +# optimization, we don't need to read them if the target does not | |
| 34 | +# exist, we will rebuild anyway in that case. | |
| 35 | + | |
| 36 | +existing-targets := $(wildcard $(sort $(targets))) | |
| 37 | + | |
| 38 | +-include $(foreach f, $(existing-targets),$(dir $(f)).$(notdir $(f)).cmd) | |
| 39 | + | |
| 40 | +.PHONY: $(PHONY) | |