[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Tue, 8 Sep 2020 00:30:14 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: Sami Tolvanen <samitolvanen@...gle.com>
Cc: Will Deacon <will@...nel.org>, Peter Zijlstra <peterz@...radead.org>,
Steven Rostedt <rostedt@...dmis.org>,
Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
"Paul E. McKenney" <paulmck@...nel.org>,
Kees Cook <keescook@...omium.org>,
Nick Desaulniers <ndesaulniers@...gle.com>,
clang-built-linux <clang-built-linux@...glegroups.com>,
Kernel Hardening <kernel-hardening@...ts.openwall.com>,
linux-arch <linux-arch@...r.kernel.org>,
linux-arm-kernel <linux-arm-kernel@...ts.infradead.org>,
Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
linux-pci@...r.kernel.org, X86 ML <x86@...nel.org>
Subject: Re: [PATCH v2 09/28] kbuild: add support for Clang LTO
On Fri, Sep 4, 2020 at 5:31 AM Sami Tolvanen <samitolvanen@...gle.com> wrote:
>
> This change adds build system support for Clang's Link Time
> Optimization (LTO). With -flto, instead of ELF object files, Clang
> produces LLVM bitcode, which is compiled into native code at link
> time, allowing the final binary to be optimized globally. For more
> details, see:
>
> https://llvm.org/docs/LinkTimeOptimization.html
>
> The Kconfig option CONFIG_LTO_CLANG is implemented as a choice,
> which defaults to LTO being disabled. To use LTO, the architecture
> must select ARCH_SUPPORTS_LTO_CLANG and support:
>
> - compiling with Clang,
> - compiling inline assembly with Clang's integrated assembler,
> - and linking with LLD.
>
> While using full LTO results in the best runtime performance, the
> compilation is not scalable in time or memory. CONFIG_THINLTO
> enables ThinLTO, which allows parallel optimization and faster
> incremental builds. ThinLTO is used by default if the architecture
> also selects ARCH_SUPPORTS_THINLTO:
>
> https://clang.llvm.org/docs/ThinLTO.html
>
> To enable LTO, LLVM tools must be used to handle bitcode files. The
> easiest way is to pass the LLVM=1 option to make:
>
> $ make LLVM=1 defconfig
> $ scripts/config -e LTO_CLANG
> $ make LLVM=1
>
> Alternatively, at least the following LLVM tools must be used:
>
> CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm
>
> To prepare for LTO support with other compilers, common parts are
> gated behind the CONFIG_LTO option, and LTO can be disabled for
> specific files by filtering out CC_FLAGS_LTO.
>
> Note that support for DYNAMIC_FTRACE and MODVERSIONS are added in
> follow-up patches.
>
> Signed-off-by: Sami Tolvanen <samitolvanen@...gle.com>
> ---
> Makefile | 18 +++++++-
> arch/Kconfig | 68 +++++++++++++++++++++++++++++++
> include/asm-generic/vmlinux.lds.h | 11 +++--
> scripts/Makefile.build | 9 +++-
> scripts/Makefile.modfinal | 9 +++-
> scripts/Makefile.modpost | 24 ++++++++++-
> scripts/link-vmlinux.sh | 32 +++++++++++----
> 7 files changed, 154 insertions(+), 17 deletions(-)
> #define TEXT_MAIN .text
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 6ecf30c70ced..a5f4b5d407e6 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -111,7 +111,7 @@ endif
> # ---------------------------------------------------------------------------
>
> quiet_cmd_cc_s_c = CC $(quiet_modtag) $@
> - cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS), $(c_flags)) $(DISABLE_LTO) -fverbose-asm -S -o $@ $<
> + cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS) $(CC_FLAGS_LTO), $(c_flags)) -fverbose-asm -S -o $@ $<
>
> $(obj)/%.s: $(src)/%.c FORCE
> $(call if_changed_dep,cc_s_c)
> @@ -428,8 +428,15 @@ $(obj)/lib.a: $(lib-y) FORCE
> # Do not replace $(filter %.o,^) with $(real-prereqs). When a single object
> # module is turned into a multi object module, $^ will contain header file
> # dependencies recorded in the .*.cmd file.
> +ifdef CONFIG_LTO_CLANG
> +quiet_cmd_link_multi-m = AR [M] $@
> +cmd_link_multi-m = \
> + rm -f $@; \
> + $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(filter %.o,$^)
KBUILD_ARFLAGS no longer exists in the mainline.
(commit 13dc8c029cabf52ba95f60c56eb104d4d95d5889)
> +else
> quiet_cmd_link_multi-m = LD [M] $@
> cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(filter %.o,$^)
> +endif
>
> $(multi-used-m): FORCE
> $(call if_changed,link_multi-m)
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 411c1e600e7d..1005b147abd0 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -6,6 +6,7 @@
> PHONY := __modfinal
> __modfinal:
>
> +include $(objtree)/include/config/auto.conf
> include $(srctree)/scripts/Kbuild.include
>
> # for c_flags
> @@ -29,6 +30,12 @@ quiet_cmd_cc_o_c = CC [M] $@
>
> ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)
>
> +ifdef CONFIG_LTO_CLANG
> +# With CONFIG_LTO_CLANG, reuse the object file we compiled for modpost to
> +# avoid a second slow LTO link
> +prelink-ext := .lto
> +endif
> +
> quiet_cmd_ld_ko_o = LD [M] $@
> cmd_ld_ko_o = \
> $(LD) -r $(KBUILD_LDFLAGS) \
> @@ -37,7 +44,7 @@ quiet_cmd_ld_ko_o = LD [M] $@
> -o $@ $(filter %.o, $^); \
> $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)
>
> -$(modules): %.ko: %.o %.mod.o $(KBUILD_LDS_MODULE) FORCE
> +$(modules): %.ko: %$(prelink-ext).o %.mod.o $(KBUILD_LDS_MODULE) FORCE
> +$(call if_changed,ld_ko_o)
>
> targets += $(modules) $(modules:.ko=.mod.o)
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> index f54b6ac37ac2..a70f1f7da6aa 100644
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -102,12 +102,32 @@ $(input-symdump):
> @echo >&2 'WARNING: Symbol version dump "$@" is missing.'
> @echo >&2 ' Modules may not have dependencies or modversions.'
>
> +ifdef CONFIG_LTO_CLANG
> +# With CONFIG_LTO_CLANG, .o files might be LLVM bitcode,
or, .o files might be even thin archives.
For example,
$ file net/ipv6/netfilter/nf_defrag_ipv6.o
net/ipv6/netfilter/nf_defrag_ipv6.o: thin archive with 6 symbol entries
Now we have 3 possibilities for .o files:
- ELF (real .o)
- LLVM bitcode (.bc)
- Thin archive (.a)
Let me discuss how to proceed with this...
--
Best Regards
Masahiro Yamada
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
