Commit Graph

220 Commits

Author SHA1 Message Date
Masahiro Yamada
7d32358be8 kbuild: avoid split lines in .mod files
"xargs echo" is not a safe way to remove line breaks because the input
may exceed the command line limit and xargs may break it up into
multiple invocations of echo. This should never happen because
scripts/gen_autoksyms.sh expects all undefined symbols are placed in
the second line of .mod files.

One possible way is to replace "xargs echo" with
"sed ':x;N;$!bx;s/\n/ /g'" or something, but I rewrote the code by
using awk because it is more readable.

This issue was reported by Sami Tolvanen; in his Clang LTO patch set,
$(multi-used-m) is no longer an ELF object, but a thin archive that
contains LLVM bitcode files. llvm-nm prints out symbols for each
archive member separately, which results a lot of dupications, in some
places, beyond the system-defined limit.

This problem must be fixed irrespective of LTO, and we must ensure
zero possibility of having this issue.

Link: https://lkml.org/lkml/2020/12/1/1658
Reported-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
2020-12-06 21:09:28 +09:00
Sami Tolvanen
0f6372e522 treewide: remove DISABLE_LTO
This change removes all instances of DISABLE_LTO from
Makefiles, as they are currently unused, and the preferred
method of disabling LTO is to filter out the flags instead.

Note added by Masahiro Yamada:
DISABLE_LTO was added as preparation for GCC LTO, but GCC LTO was
not pulled into the mainline. (https://lkml.org/lkml/2014/4/8/272)

Suggested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-10-21 00:28:53 +09:00
Masahiro Yamada
85569d19d0 kbuild: sort hostprogs before passing it to ifneq
The conditional:

  ifneq ($(hostprogs),)

... is evaluated to true if $(hostprogs) does not contain any word but
whitespace characters.

  ifneq ($(strip $(hostprogs)),)

... is a safe way to avoid interpreting whitespace as a non-empty value,
but I'd rather want to use the side-effect of $(sort ...) to do the
equivalent.

$(sort ...) is used in scripts/Makefile.host in order to drop duplication
in $(hostprogs). It is also useful to strip excessive spaces.

Move $(sort ...) before evaluating the ifneq.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-10 01:32:59 +09:00
Masahiro Yamada
42640b134b kbuild: move host .so build rules to scripts/gcc-plugins/Makefile
The host shared library rules are currently implemented in
scripts/Makefile.host, but actually GCC-plugin is the only user of
them. (The VDSO .so files are built for the target by different
build rules) Hence, they do not need to be treewide available.

Move all the relevant build rules to scripts/gcc-plugins/Makefile.

I also optimized the build steps so *.so is directly built from .c
because every upstream plugin is compiled from a single source file.

I am still keeping the multi-file plugin support, which Kees Cook
mentioned might be needed by out-of-tree plugins.
(https://lkml.org/lkml/2019/1/11/1107)

If the plugin, foo.so, is compiled from two files foo.c and foo2.c,
then you can do like follows:

  foo-objs := foo.o foo2.o

Single-file plugins do not need the *-objs notation.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
2020-08-10 01:32:59 +09:00
Masahiro Yamada
cc8a51ca6f kbuild: always create directories of targets
Currently, the directories of objects are automatically created
only for O= builds.

It should not hurt to cater to this for in-tree builds too.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-10 01:32:58 +09:00
Luc Van Oostenryck
0c33f12573 kbuild: run the checker after the compiler
Since the pre-git time the checker is run first, before the compiler.
But if the source file contains some syntax error, the warnings from
the compiler are more useful than those from sparse (and other
checker most probably too).

So move the 'check' command to run after the compiler.

Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-07-07 11:13:10 +09:00
Masahiro Yamada
b2c8855491 kbuild: update modules.order only when contained modules are updated
Make modules.order depend on $(obj-m), and use if_changed to build it.
This will avoid unneeded update of modules.order, which will be useful
to optimize the modpost stage.

Currently, the second pass of modpost is always invoked. By checking the
timestamp of modules.order, we can avoid the unneeded modpost.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-03 13:22:17 +09:00
Masahiro Yamada
f3908ab3ff kbuild: refactor tagets caluculation for KBUILD_{BUILTIN,KBUILD_MODULES}
Remove lib-target, builtin-target, modorder-target, and modtargets.

Instead, add targets-for-builtin and targets-for-modules.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-01 21:50:46 +09:00
Masahiro Yamada
454753d9f6 kbuild: make modules.order rule consistent with built-in.a
built-in.a contains the built-in object paths from the current and sub
directories.

module.order collects the module paths from the current and sub
directories.

Make their build rules look more symmetrical.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-26 00:03:16 +09:00
Masahiro Yamada
aaa385ba9a kbuild: rename subdir-obj-y to subdir-builtin
I think subdir-builtin is clearer.

While I was here, I made its build rule explicit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-26 00:03:16 +09:00
Masahiro Yamada
6ba3bcb013 kbuild: move subdir-obj-y to scripts/Makefile.build
Save $(addprefix ...) for subdir-obj-y.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-26 00:03:16 +09:00
Masahiro Yamada
b480fec988 kbuild: clear KBUILD_MODULES in top Makefile if CONFIG_MODULES=n
Do not try to build any module-related artifacts when CONFIG_MODULES
is disabled.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-26 00:03:16 +09:00
Masahiro Yamada
e578edc722 kbuild: remove ifdef builtin-target / lib-target
I do not see a good reason to add ifdef here.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-26 00:03:16 +09:00
Masahiro Yamada
7f3a59db27 kbuild: add infrastructure to build userspace programs
Kbuild supports the infrastructure to build host programs, but there
was no support to build userspace programs for the target architecture
(i.e. the same architecture as the kernel).

Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154),
but it was not merged. One problem at that time was, there was no good way
to know whether $(CC) can link standalone programs. In fact, pre-built
kernel.org toolchains [1] are often used for building the kernel, but they
do not provide libc.

Now, we can handle this cleanly because the compiler capability is
evaluated at the Kconfig time. If $(CC) cannot link standalone programs,
the relevant options are hidden by 'depends on CC_CAN_LINK'.

The implementation just mimics scripts/Makefile.host

The userspace programs are compiled with the same flags as the host
programs. In addition, it uses -m32 or -m64 if it is found in
$(KBUILD_CFLAGS).

This new syntax has two usecases.

- Sample programs

  Several userspace programs under samples/ include UAPI headers
  installed in usr/include. Most of them were previously built for
  the host architecture just to use the 'hostprogs' syntax.

  However, 'make headers' always works for the target architecture.
  This caused the arch mismatch in cross-compiling. To fix this
  distortion, sample code should be built for the target architecture.

- Bpfilter

  net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper,
  and embeds it into the kernel. Currently, it overrides HOSTCC with
  CC to use the 'hostprogs' syntax. This hack should go away.

[1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
2020-05-17 18:52:01 +09:00
Masahiro Yamada
7273ad2b08 kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y
Kbuild supports not only obj-y but also lib-y to list objects linked to
vmlinux.

The difference between them is that all the objects from obj-y are
forcibly linked to vmlinux, whereas the objects from lib-y are linked
as needed; if there is no user of a lib-y object, it is not linked.

lib-y is intended to list utility functions that may be called from all
over the place (and may be unused at all), but it is a problem for
EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need
to keep exported symbols for the use from loadable modules.

Commit 7f2084fa55 ("[kbuild] handle exports in lib-y objects reliably")
worked around it by linking a dummy object, lib-ksyms.o, which contains
references to all the symbols exported from lib.a in that directory.
It uses the linker script command, EXTERN. Unfortunately, the meaning of
EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does
not work with LD=ld.lld (CBL issue #515).

Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to
get rid of it.

At first, I was thinking of accumulating lib-y objects into obj-y
(or even replacing lib-y with obj-y entirely), but the lib-y syntax
is used beyond the ordinary use in lib/ and arch/*/lib/.

Examples:

 - drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked
   into vmlinux in the own way (arm64), or linked to the decompressor
   (arm, x86).

 - arch/alpha/lib/Makefile builds lib.a which is linked not only to
   vmlinux, but also to bootloaders in arch/alpha/boot/Makefile.

 - arch/xtensa/boot/lib/Makefile builds lib.a for use from
   arch/xtensa/boot/boot-redboot/Makefile.

One more thing, adding everything to obj-y would increase the vmlinux
size of allnoconfig (or tinyconfig).

For less impact, I tweaked the destination of lib.a at the top Makefile;
when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is
forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS
as before.

The size impact for normal usecases is quite small since at lease one
symbol in every lib-y object is eventually called by someone. In case
you are intrested, here are the figures.

x86_64_defconfig:

   text	   data	    bss	    dec	    hex	filename
19566602 5422072 1589328 26578002 1958c52 vmlinux.before
19566932 5422104 1589328 26578364 1958dbc vmlinux.after

The case with the biggest impact is allnoconfig + CONFIG_MODULES=y.

ARCH=x86 allnoconfig + CONFIG_MODULES=y:

   text	   data	    bss	    dec	    hex	filename
1175162	 254740	1220608	2650510	 28718e	vmlinux.before
1177974	 254836	1220608	2653418	 287cea	vmlinux.after

Hopefully this is still not a big deal. The per-file trimming with the
static library is not so effective after all.

If fine-grained optimization is desired, some architectures support
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol
basis. When LTO is supported in mainline, even better optimization will
be possible.

Link: https://github.com/ClangBuiltLinux/linux/issues/515
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reported-by: kbuild test robot <lkp@intel.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-09 00:13:45 +09:00
Masahiro Yamada
77342a02ff gcc-plugins: drop support for GCC <= 4.7
Nobody was opposed to raising minimum GCC version to 4.8 [1]
So, we will drop GCC <= 4.7 support sooner or later.

We always use C++ compiler for building plugins for GCC >= 4.8.

This commit drops the plugin support for GCC <= 4.7 a bit earlier,
which allows us to dump lots of code.

[1] https://lkml.org/lkml/2020/1/23/545

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
2020-04-09 00:13:45 +09:00
Masahiro Yamada
5f2fb52fac kbuild: rename hostprogs-y/always to hostprogs/always-y
In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.

It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.

This commit renames like follows:

  always       ->  always-y
  hostprogs-y  ->  hostprogs

So, scripts/Makefile will look like this:

  always-$(CONFIG_BUILD_BIN2C) += ...
  always-$(CONFIG_KALLSYMS)    += ...
      ...
  hostprogs := $(always-y) $(always-m)

I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.

The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-04 01:53:07 +09:00
Masahiro Yamada
a749926797 kbuild: use pattern rule for building built-in.a in sub-directories
The built-in.a in a sub-directory is created by descending into that
directory. It does not depend on the other sub-directories. Loosen
the dependency.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-07 02:18:38 +09:00
Masahiro Yamada
56d5893615 kbuild: do not create orphan built-in.a or obj-y objects
Both 'obj-y += foo/' and 'obj-m += foo/' request Kbuild to visit the
sub-directory foo/, but the difference is that only the former combines
foo/built-in.a into the built-in.a of the current directory because
everything in sub-directories visited by obj-m is supposed to be modular.

So, it makes sense to create built-in.a only if that sub-directory is
reachable by the chain of obj-y. Otherwise, built-in.a will not be
linked into vmlinux anyway. For the same reason, it is pointless to
compile obj-y objects in the directory visited by obj-m.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-07 02:18:38 +09:00
Masahiro Yamada
fcbb8461fd kbuild: remove header compile test
There are both positive and negative options about this feature.
At first, I thought it was a good idea, but actually Linus stated a
negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it
is ugly and annoying.

The baseline I'd like to keep is the compile-test of uapi headers.
(Otherwise, kernel developers have no way to ensure the correctness
of the exported headers.)

I will maintain a small build rule in usr/include/Makefile.
Remove the other header test functionality.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-15 00:22:35 +09:00
Masahiro Yamada
2dffd23f81 kbuild: make single target builds much faster
Since commit 394053f4a4 ("kbuild: make single targets work more
correctly"), building single targets is really slow.

Speed it up by not descending into unrelated directories.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Masahiro Yamada
203126293c kbuild: reduce KBUILD_SINGLE_TARGETS as descending into subdirectories
KBUILD_SINGLE_TARGETS does not need to contain all the targets.
Change it to keep track the targets only from the current directory
and its subdirectories.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Masahiro Yamada
13dc8c029c kbuild: remove ar-option and KBUILD_ARFLAGS
Commit 40df759e2b ("kbuild: Fix build with binutils <= 2.19")
introduced ar-option and KBUILD_ARFLAGS to deal with old binutils.

According to Documentation/process/changes.rst, the current minimal
supported version of binutils is 2.21 so you can assume the 'D' option
is always supported. Not only GNU ar but also llvm-ar supports it.

With the 'D' option hard-coded, there is no more user of ar-option
or KBUILD_ARFLAGS.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2019-10-01 09:20:33 +09:00
Masahiro Yamada
e27128db62 kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN
KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning
options for GCC, but now it is a historical misnomer since we use it
also for Clang, DTC, and even kernel-doc.

Rename it to more sensible, shorter KBUILD_EXTRA_WARN.

For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still
supported (but not advertised in the documentation).

I also fixed up 'make help', and updated the documentation.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>
2019-09-06 23:46:52 +09:00
Masahiro Yamada
eb27ea5ce7 kbuild: move modkern_{c,a}flags to Makefile.lib from Makefile.build
Makefile.lib is included by Makefile.modfinal as well as Makefile.build.

Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c
in Makefile.modfinal. Move modkern_cflags as well for consistency.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-22 01:14:11 +09:00
Masahiro Yamada
2ff2b7ec65 kbuild: add CONFIG_ASM_MODVERSIONS
Add CONFIG_ASM_MODVERSIONS. This allows to remove one if-conditional
nesting in scripts/Makefile.build.

scripts/Makefile.build is run every time Kbuild descends into a
sub-directory. So, I want to avoid $(wildcard ...) evaluation
where possible although computing $(wildcard ...) is so cheap that
it may not make measurable performance difference.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
2019-08-22 01:14:11 +09:00
Masahiro Yamada
394053f4a4 kbuild: make single targets work more correctly
Currently, the single target build directly descends into the directory
of the target. For example,

  $ make foo/bar/baz.o

... directly descends into foo/bar/.

On the other hand, the normal build usually descends one directory at
a time, i.e. descends into foo/, and then foo/bar/.

This difference causes some problems.

[1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles

    The options in subdir-{as,cc}flags-y take effect in the current
    and its sub-directories. In other words, they are inherited
    downward. In the example above, the single target will miss
    subdir-{as,cc}flags-y if they are defined in foo/Makefile.

[2] could be built in a different directory

    As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can
    handle files that are spread over several sub-directories.

    The build rule of foo/bar/baz.o may not necessarily be specified in
    foo/bar/Makefile. It might be specifies in foo/Makefile as follows:

    [foo/Makefile]
    obj-y := bar/baz.o

    This often happens when a module is so big that its source files
    are divided into sub-directories.

    In this case, there is no Makefile in the foo/bar/ directory, yet
    the single target descends into foo/bar/, then fails due to the
    missing Makefile. You can still do 'make foo/bar/' for partial
    building, but cannot do 'make foo/bar/baz.s'. I believe the single
    target '%.s' is a useful feature for inspecting the compiler output.

    Some modules work around this issue by putting an empty Makefile
    in every sub-directory.

This commit fixes those problems by making the single target build
descend in the same way as the normal build does.

Another change is the single target build will observe the CONFIG
options. Previously, it allowed users to build the foo.o even when
the corresponding CONFIG_FOO is disabled:

   obj-$(CONFIG_FOO) += foo.o

In the new behavior, the single target build will just fail and show
"No rule to make target ..." (or "Nothing to be done for ..." if the
stale object already exists, but cannot be updated).

The disadvantage of this commit is the build speed. Now that the
single target build visits every directory and parses lots of
Makefiles, it is slower than before. (But, I hope it will not be
too slow.)

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-21 21:05:21 +09:00
Masahiro Yamada
c2290f3286 kbuild: fix modkern_aflags implementation
For the single target building %.symtypes from %.S, $(a_flags) is
expanded into the _KERNEL flags even if the object is a part of a
module.

$(real-obj-m:.o=.symtypes): modkern_aflags := $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE)

... would fix the issue, but it is not nice to duplicate similar code
for every suffix.

Implement modkern_aflags in the same way as modkern_cflags.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-15 02:25:11 +09:00
Masahiro Yamada
986662b903 kbuild: refactor part-of-module more
Make it even shorter.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-15 02:25:11 +09:00
Masahiro Yamada
49d5089d92 kbuild: use $(basename ...) for cmd_asn1_compiler
$(basename ...) trims the last suffix. Using it is more intuitive in
my opinion.

This pattern rule makes %.asn1.c and %.asn1.h at the same time.
Previously, the short log showed only either of them, depending on
the target file in question.

To clarify that two files are being generated by the single recipe,
I changed the log as follows:

Before:

  ASN.1   crypto/asymmetric_keys/x509.asn1.c

After:

  ASN.1   crypto/asymmetric_keys/x509.asn1.[ch]

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 01:10:42 +09:00
Masahiro Yamada
c07d8d47bc kbuild: show hint if subdir-y/m is used to visit module Makefile
Since commit ff9b45c55b ("kbuild: modpost: read modules.order instead
of $(MODVERDIR)/*.mod"), a module is no longer built in the following
pattern:

  [Makefile]
  subdir-y := some-module

  [some-module/Makefile]
  obj-m := some-module.o

You cannot write Makefile this way in upstream because modules.order is
not correctly generated. subdir-y is used to descend to a sub-directory
that builds tools, device trees, etc.

For external modules, the modules order does not matter. So, the
Makefile above was known to work.

I believe the Makefile should be re-written as follows:

  [Makefile]
  obj-m := some-module/

  [some-module/Makefile]
  obj-m := some-module.o

However, people will have no idea if their Makefile suddenly stops
working. In fact, I received questions from multiple people.

Show a warning for a while if obj-m is specified in a Makefile visited
by subdir-y or subdir-m.

I touched the %/ rule to avoid false-positive warnings for the single
target.

Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Tom Stonecypher <thomas.edwardx.stonecypher@intel.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-10 01:45:31 +09:00
Masahiro Yamada
4f2c8f3089 kbuild: generate modules.order only in directories visited by obj-y/m
The modules.order files in directories visited by the chain of obj-y
or obj-m are merged to the upper-level ones, and become parts of the
top-level modules.order. On the other hand, there is no need to
generate modules.order in directories visited by subdir-y or subdir-m
since they would become orphan anyway.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-10 01:45:31 +09:00
Masahiro Yamada
d9f78edfd8 kbuild: fix false-positive need-builtin calculation
The current implementation of need-builtin is false-positive,
for example, in the following Makefile:

  obj-m := foo/
  obj-y := foo/bar/

..., where foo/built-in.a is not required.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-10 01:45:31 +09:00
Masahiro Yamada
9f69a496f1 kbuild: split out *.mod out of {single,multi}-used-m rules
Currently, *.mod is created as a side-effect of obj-m.

Split out *.mod as a dedicated build rule, which allows to unify
the %.c -> %.o rule, and remove the single-used-m rule.

This also makes the incremental build of allmodconfig faster because
it saves $(NM) invocation when there is no change in the module.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-18 02:19:31 +09:00
Masahiro Yamada
60ae1b194b kbuild: remove the first line of *.mod files
The current format of *.mod is like this:

  line 1: directory path to the .ko file
  line 2: a list of objects linked into this module
  line 3: unresolved symbols (only when CONFIG_TRIM_UNUSED_KSYMS=y)

Now that *.mod and *.ko are created in the same directory, the line 1
provides no valuable information. It can be derived by replacing the
extension .mod with .ko. In fact, nobody uses the first line any more.

Cut down the first line.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-18 02:19:31 +09:00
Masahiro Yamada
b7dca6dd1e kbuild: create *.mod with full directory path and remove MODVERDIR
While descending directories, Kbuild produces objects for modules,
but do not link final *.ko files; it is done in the modpost.

To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR)
for every module it is building. Some post-processing steps read the
necessary information from *.mod files. This avoids descending into
directories again. This mechanism was introduced in 2003 or so.

Later, commit 551559e13a ("kbuild: implement modules.order") added
modules.order. So, we can simply read it out to know all the modules
with directory paths. This is easier than parsing the first line of
*.mod files.

$(MODVERDIR) has a flat directory structure, that is, *.mod files
are named only with base names. This is based on the assumption that
the module name is unique across the tree. This assumption is really
fragile.

Stephen Rothwell reported a race condition caused by a module name
conflict:

  https://lkml.org/lkml/2019/5/13/991

In parallel building, two different threads could write to the same
$(MODVERDIR)/*.mod simultaneously.

Non-unique module names are the source of all kind of troubles, hence
commit 3a48a91901 ("kbuild: check uniqueness of module names")
introduced a new checker script.

However, it is still fragile in the build system point of view because
this race happens before scripts/modules-check.sh is invoked. If it
happens again, the modpost will emit unclear error messages.

To fix this issue completely, create *.mod with full directory path
so that two threads never attempt to write to the same file.

$(MODVERDIR) is no longer needed.

Since modules with directory paths are listed in modules.order, Kbuild
is still able to find *.mod files without additional descending.

I also killed cmd_secanalysis; scripts/mod/sumversion.c computes MD4 hash
for modules with MODULE_VERSION(). When CONFIG_DEBUG_SECTION_MISMATCH=y,
it occurs not only in the modpost stage, but also during directory
descending, where sumversion.c may parse stale *.mod files. It would emit
'No such file or directory' warning when an object consisting a module is
renamed, or when a single-obj module is turned into a multi-obj module or
vice versa.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
2019-07-18 02:19:31 +09:00
Masahiro Yamada
e0e1b1ec39 kbuild: remove duplication from modules.order in sub-directories
Currently, only the top-level modules.order drops duplicated entries.

The modules.order files in sub-directories potentially contain
duplication. To list out the paths of all modules, I want to use
modules.order instead of parsing *.mod files in $(MODVERDIR).

To achieve this, I want to rip off duplication from modules.order
of external modules too.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-17 22:39:27 +09:00
Masahiro Yamada
1bd9a46801 kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin}
Removing the 'kernel/' prefix will make our life easier because we can
simply do 'cat modules.order' to get all built modules with full paths.

Currently, we parse the first line of '*.mod' files in $(MODVERDIR).
Since we have duplicated functionality here, I plan to remove MODVERDIR
entirely.

In fact, modules.order is generated also for external modules in a
broken format. It adds the 'kernel/' prefix to the absolute path of
the module, like this:

  kernel//path/to/your/external/module/foo.ko

This is fine for now since modules.order is not used for external
modules. However, I want to sanitize the format everywhere towards
the goal of removing MODVERDIR.

We cannot change the format of installed module.{order,builtin}.
So, 'make modules_install' will add the 'kernel/' prefix while copying
them to $(MODLIB)/.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-17 22:39:27 +09:00
Masahiro Yamada
7e13191879 kbuild: do not create empty modules.order in the prepare stage
Currently, $(objtree)/modules.order is touched in two places.

In the 'prepare0' rule, scripts/Makefile.build creates an empty
modules.order while processing 'obj=.'

In the 'modules' rule, the top-level Makefile overwrites it with
the correct list of modules.

While this might be a good side-effect that modules.order is made
empty every time (probably this is not intended functionality),
I personally do not like this behavior.

Create modules.order only when it is sensible to do so.

This avoids creating the following pointless files:

  scripts/basic/modules.order
  scripts/dtc/modules.order
  scripts/gcc-plugins/modules.order
  scripts/genksyms/modules.order
  scripts/mod/modules.order
  scripts/modules.order
  scripts/selinux/genheaders/modules.order
  scripts/selinux/mdp/modules.order
  scripts/selinux/modules.order

Going forward, $(objtree)/modules.order lists the modules that
was built in the last successful build.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-17 22:39:27 +09:00
Masahiro Yamada
4bd01de8f2 kbuild: compile-test headers listed in header-test-m as well
It will be useful to control the header-test by a tristate option.

If CONFIG_FOO is a tristate option, you can write like this:

  header-test-$(CONFIG_FOO) += foo.h

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-17 22:37:51 +09:00
Masahiro Yamada
051f278e9d kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree
Commit 25b146c5b8 ("kbuild: allow Kbuild to start from any directory")
deprecated KBUILD_SRCTREE.

It is only used in tools/testing/selftest/ to distinguish out-of-tree
build. Replace it with a new boolean flag, building_out_of_srctree.

I also replaced the conditional ($(srctree),.) because the next commit
will allow an absolute path to be used for $(srctree) even when building
in the source tree.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-11 00:05:09 +09:00
Masahiro Yamada
c93a0368aa kbuild: do not create wrappers for header-test-y
header-test-y does not work with headers in sub-directories.

For example, you may want to write a Makefile, like this:

include/linux/Kbuild:

  header-test-y += mtd/nand.h

This entry will create a wrapper include/linux/mtd/nand.hdrtest.c
with the following content:

  #include "mtd/nand.h"

To make this work, we need to add $(srctree)/include/linux to the
header search path. It would be tedious to add ccflags-y.

Instead, we could change the *.hdrtest.c rule to wrap:

  #include "nand.h"

This works for in-tree build since #include "..." searches in the
relative path from the header with this directive. For O=... build,
we need to add $(srctree)/include/linux/mtd to the header search path,
which will be even more tedious.

After all, I thought it would be handier to compile headers directly
without creating wrappers.

I added a new build rule to compile %.h into %.h.s

The target is %.h.s instead of %.h.o because it is slightly faster.
Also, as for GCC, an empty assembly is smaller than an empty object.

I wrote the build rule:

  $(CC) $(c_flags) -S -o $@ -x c /dev/null -include $<

instead of:

  $(CC) $(c_flags) -S -o $@ -x c $<

Both work fine with GCC, but the latter is bad for Clang.

This comes down to the difference in the -Wunused-function policy.
GCC does not warn about unused 'static inline' functions at all.
Clang does not warn about the ones in included headers, but does
about the ones in the source. So, we should handle headers as
headers, not as source files.

In fact, this has been hidden since commit abb2ea7dfd ("compiler,
clang: suppress warning for unused static inline functions"), but we
should not rely on that.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Tested-by: Jani Nikula <jani.nikula@intel.com>
2019-07-09 10:10:27 +09:00
Jani Nikula
e846f0dc57 kbuild: add support for ensuring headers are self-contained
Sometimes it's useful to be able to explicitly ensure certain headers
remain self-contained, i.e. that they are compilable as standalone
units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add
headers to be tested if CONFIG_HEADER_TEST is enabled. This will
generate a dummy C file per header that gets built as part of extra-y.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-15 19:57:02 +09:00
Linus Torvalds
2646719a48 Kbuild updates for v5.2
- allow users to invoke 'make' out of the source tree
 
 - refactor scripts/mkmakefile
 
 - deprecate KBUILD_SRC, which was used to track the source tree
   location for O= build.
 
 - fix recordmcount.pl in case objdump output is localized
 
 - turn unresolved symbols in external modules to errors from warnings
   by default; pass KBUILD_MODPOST_WARN=1 to get them back to warnings
 
 - generate modules.builtin.modinfo to collect .modinfo data from
   built-in modules
 
 - misc Makefile cleanups
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJc0upmAAoJED2LAQed4NsGreoP/jkKQojVdzTjM/nn/Xe9FrE9
 elQ59Wu/gW/q7jouN733WTJ2fRPAEVaGPw/CT/cCRWjNJjd6MM4voDKVxifZE+u4
 qz5XUGq4Mjozd0MJdU1LA1dIRJrEaPofOj7E0JEIerEdLPKpv4kjCCbbHOZwN2Kl
 YrpPFJkYspCRVG+txEWY8YaZeU/+OAdNckJnMkX8hnUwsNplAlPw4L/5Y12Uncuz
 7g/3T1f701pJhStoO4OPR/+Rivi0EX5AP7TZyv2/FOwTO+Oau5G1rF0j1BT0+ceg
 d+bApuaHLR4ocZ6GvWfACMYhH7ais0BUgAwi6HY/b78SGMmNB6xlD5biVqcklx6c
 mjrPNaj6WJEQIflZ3yr83tRNhTu7xXRKZkrGHHUjOGNsRULuD8RFrXtusl06A22S
 hA3vt2SF4FuLBabPw5yFkKNOTX+P5dG79BC1fWIQjglBGoYqWTRQNYoottr/7t2T
 Si+E6J2lAD0EDNuc/EFKWfrgONfatgcdgQuTibwCcE2KfYnMG/C+9DvxRV4lAQtn
 fYpap9gws+0AisbVx0m1W088NMFiwrMbN3n3KITGY/15XDmeySEA1QgPKFN8JHJd
 Do0duaMx8BHjkA5ms/Bf0AZd0tWUkWVdt0epwU2KafWNGK7VaPnhKxn5eQOGkWV+
 gT3J3XZc3b7OEOQ4XX3Z
 =zj9w
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - allow users to invoke 'make' out of the source tree

 - refactor scripts/mkmakefile

 - deprecate KBUILD_SRC, which was used to track the source tree
   location for O= build.

 - fix recordmcount.pl in case objdump output is localized

 - turn unresolved symbols in external modules to errors from warnings
   by default; pass KBUILD_MODPOST_WARN=1 to get them back to warnings

 - generate modules.builtin.modinfo to collect .modinfo data from
   built-in modules

 - misc Makefile cleanups

* tag 'kbuild-v5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (21 commits)
  .gitignore: add more all*.config patterns
  moduleparam: Save information about built-in modules in separate file
  Remove MODULE_ALIAS() calls that take undefined macro
  .gitignore: add leading and trailing slashes to generated directories
  scripts/tags.sh: fix direct execution of scripts/tags.sh
  scripts: override locale from environment when running recordmcount.pl
  samples: kobject: allow CONFIG_SAMPLE_KOBJECT to become y
  samples: seccomp: turn CONFIG_SAMPLE_SECCOMP into a bool option
  kbuild: move Documentation to vmlinux-alldirs
  kbuild: move samples/ to KBUILD_VMLINUX_OBJS
  modpost: make KBUILD_MODPOST_WARN also configurable for external modules
  kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build
  kbuild: remove unneeded dependency for include/config/kernel.release
  memory: squash drivers/memory/Makefile.asm-offsets
  kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build
  kbuild: mkmakefile: generate a simple wrapper of top Makefile
  kbuild: mkmakefile: do not check the generated Makefile marker
  kbuild: allow Kbuild to start from any directory
  kbuild: pass $(MAKECMDGOALS) to sub-make as is
  kbuild: fix warning "overriding recipe for target 'Makefile'"
  ...
2019-05-08 12:25:12 -07:00
Peter Zijlstra
ea24213d80 objtool: Add UACCESS validation
It is important that UACCESS regions are as small as possible;
furthermore the UACCESS state is not scheduled, so doing anything that
might directly call into the scheduler will cause random code to be
ran with UACCESS enabled.

Teach objtool too track UACCESS state and warn about any CALL made
while UACCESS is enabled. This very much includes the __fentry__()
and __preempt_schedule() calls.

Note that exceptions _do_ save/restore the UACCESS state, and therefore
they can drive preemption. This also means that all exception handlers
must have an otherwise redundant UACCESS disable instruction;
therefore ignore this warning for !STT_FUNC code (exception handlers
are not normal functions).

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-04-03 11:02:24 +02:00
Masahiro Yamada
a9a49c2ad9 kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build
KBUILD_SRC was conventionally used for some different purposes:
 [1] To remember the source tree path
 [2] As a flag to check if sub-make is already done
 [3] As a flag to check if Kbuild runs out of tree

For [1], we do not need to remember it because the top Makefile
can compute it by $(realpath $(dir $(lastword $(MAKEFILE_LIST))))

[2] has been replaced with self-commenting 'sub_make_done'.

For [3], we can distinguish in-tree/out-of-tree by comparing
$(srctree) and '.'

This commit converts [3] to prepare for the KBUILD_SRC removal.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-04-02 23:28:04 +09:00
Joe Lawrence
1a49b2fd8f kbuild: strip whitespace in cmd_record_mcount findstring
CC_FLAGS_FTRACE may contain trailing whitespace that interferes with
findstring.

For example, commit 6977f95e63 ("powerpc: avoid -mno-sched-epilog on
GCC 4.9 and newer") introduced a change such that on my ppc64le box,
CC_FLAGS_FTRACE="-pg -mprofile-kernel ".  (Note the trailing space.)
When cmd_record_mcount is now invoked, findstring fails as the ftrace
flags were found at very end of _c_flags, without the trailing space.

  _c_flags=" ... -pg -mprofile-kernel"
  CC_FLAGS_FTRACE="-pg -mprofile-kernel "
                                       ^
    findstring is looking for this extra space

Remove the redundant whitespaces from CC_FLAGS_FTRACE in
cmd_record_mcount to avoid this problem.

[masahiro.yamada: This issue only happens in the released versions
of GNU Make. CC_FLAGS_FTRACE will not contain the trailing space if
you use the latest GNU Make, which contains commit b90fabc8d6f3
("* NEWS: Do not insert a space during '+=' if the value is empty.") ]

Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com> (refactoring)
Fixes: 6977f95e63 ("powerpc: avoid -mno-sched-epilog on GCC 4.9 and newer").
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-28 23:46:55 +09:00
Masahiro Yamada
898f5a009f kbuild: move archive command to scripts/Makefile.lib
scripts/Makefile.build and arch/s390/boot/Makefile use the same
command (thin archiving with symbol table creation).

Avoid the code duplication, and move it to scripts/Makefile.lib.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-14 02:39:10 +09:00
Masahiro Yamada
88110713ca kbuild: hardcode genksyms path and remove GENKSYMS variable
The genksyms source was integrated into the kernel tree in 2003.

I do not expect anybody still using the external /sbin/genksyms.
Kbuild does not need to provide the ability to override GENKSYMS.

Let's remove the GENKSYMS variable, and use the hardcoded path.

Since it occurred in the pre-git era, I attached the commit message
in case somebody is interested in the historical background.

  | Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
  | Date:   Wed Feb 19 04:17:28 2003 -0600
  |
  | kbuild: [PATCH] put genksyms in scripts dir
  |
  | This puts genksyms into scripts/genksyms/.
  |
  | genksyms used to be maintained externally, though the only possible user
  | was the kernel build. Moving it into the kernel sources makes it easier to
  | keep it uptodate, like for example updating it to generate linker scripts
  | directly instead of postprocessing the generated header file fragments
  | with sed, as we do currently.
  |
  | Also, genksyms does not handle __typeof__, which needs to be fixed since
  | some of the exported symbol in the kernel are defined using __typeof__.
  |
  | (Rusty Russell/me)

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-27 21:41:26 +09:00
Masahiro Yamada
1d8001ef35 kbuild: generate modules.order only when CONFIG_MODULES=y
Do not generate pointless modules.order when the module support is
disabled.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-20 09:42:46 +09:00