Add distributed ThinLTO build support for the Linux kernel. This new mode offers several advantages: (1) Increased flexibility in handling user-specified build options. (2) Improved user-friendliness for developers. (3) Greater convenience for integrating with objtool and livepatch. Note that "distributed" in this context refers to a term that differentiates in-process ThinLTO builds by invoking backend compilation through the linker, not necessarily building in distributed environments. Distributed ThinLTO is enabled via the `CONFIG_LTO_CLANG_THIN_DIST` Kconfig option. For example: > make LLVM=1 defconfig > scripts/config -e LTO_CLANG_THIN_DIST > make LLVM=1 oldconfig > make LLVM=1 vmlinux -j <..> The build flow proceeds in four stages: 1. Perform FE compilation, mirroring the in-process ThinLTO mode. 2. Thin-link the generated IR files and object files. 3. Find all IR files and perform BE compilation, using the flags stored in the .*.o.cmd files. 4. Link the BE results to generate the final vmlinux.o. NOTE: This patch currently implements the build for the main kernel image (vmlinux) only. Kernel module support is planned for a subsequent patch. Tested on the following arch: x86, arm64, loongarch, and riscv. The earlier implementation details can be found here: https://discourse.llvm.org/t/rfc-distributed-thinlto-build-for-kernel/85934
Showing the changes to scripts/Makefile.thinlto (1 of 7 files). Show all
Unified · Split
| @@ -0,0 +1,40 @@ | |||
| 1 | PHONY := __default | ||
| 2 | __default: | ||
| 3 | |||
| 4 | include include/config/auto.conf | ||
| 5 | include $(srctree)/scripts/Kbuild.include | ||
| 6 | include $(srctree)/scripts/Makefile.lib | ||
| 7 | |||
| 8 | native-objs := $(patsubst %.o,%.thinlto-native.o,$(call read-file, vmlinux.thinlto-index)) | ||
| 9 | |||
| 10 | __default: $(native-objs) | ||
| 11 | |||
| 12 | # Generate .thinlto-native.o (obj) from .o (bitcode) and .thinlto.bc (summary) files | ||
| 13 | # --------------------------------------------------------------------------- | ||
| 14 | quiet_cmd_cc_o_bc = CC $(quiet_modtag) $@ | ||
| 15 | be_flags = $(shell sed -n '/saved_c_flags_/s/.*:= //p' \ | ||
| 16 | $(dir $(<)).$(notdir $(<)).cmd) | ||
| 17 | cmd_cc_o_bc = \ | ||
| 18 | $(CC) $(be_flags) -x ir -fno-lto -Wno-unused-command-line-argument \ | ||
| 19 | -fthinlto-index=$(word 2, $^) -c -o $@ $< | ||
| 20 | |||
| 21 | targets += $(native-objs) | ||
| 22 | $(native-objs): %.thinlto-native.o: %.o %.o.thinlto.bc FORCE | ||
| 23 | $(call if_changed,cc_o_bc) | ||
| 24 | |||
| 25 | # Add FORCE to the prerequisites of a target to force it to be always rebuilt. | ||
| 26 | # --------------------------------------------------------------------------- | ||
| 27 | |||
| 28 | PHONY += FORCE | ||
| 29 | FORCE: | ||
| 30 | |||
| 31 | # Read all saved command lines and dependencies for the $(targets) we | ||
| 32 | # may be building above, using $(if_changed{,_dep}). As an | ||
| 33 | # optimization, we don't need to read them if the target does not | ||
| 34 | # exist, we will rebuild anyway in that case. | ||
| 35 | |||
| 36 | existing-targets := $(wildcard $(sort $(targets))) | ||
| 37 | |||
| 38 | -include $(foreach f, $(existing-targets),$(dir $(f)).$(notdir $(f)).cmd) | ||
| 39 | |||
| 40 | .PHONY: $(PHONY) | ||