linux/scripts/Kbuild.include

477 lines
18 KiB
Plaintext
Raw Normal View History

####
# kbuild: Generic definitions
# Convenient variables
comma := ,
quote := "
squote := '
empty :=
space := $(empty) $(empty)
kbuild: fix if_change and friends to consider argument order Currently, arg-check is implemented as follows: arg-check = $(strip $(filter-out $(cmd_$(1)), $(cmd_$@)) \ $(filter-out $(cmd_$@), $(cmd_$(1))) ) This does not care about the order of arguments that appear in $(cmd_$(1)) and $(cmd_$@). So, if_changed and friends never rebuild the target if only the argument order is changed. This is a problem when the link order is changed. Apparently, obj-y += foo.o obj-y += bar.o and obj-y += bar.o obj-y += foo.o should be distinguished because the link order determines the probe order of drivers. So, built-in.o should be rebuilt when the order of objects is changed. This commit fixes arg-check to compare the old/current commands including the argument order. Of course, this change has a side effect; Kbuild will react to the change of compile option order. For example, "-DFOO -DBAR" and "-DBAR -DFOO" should give no difference to the build result, but false positive should be better than false negative. I am moving space_escape to the top of Kbuild.include just for a matter of preference. In practical terms, space_escape can be defined after arg-check because arg-check uses "=" flavor, not ":=". Having said that, collecting convenient variables in one place makes sense from the point of readability. Chaining "%%%SPACE%%%" to "_-_SPACE_-_" is also a matter of taste at this point. Actually, it can be arbitrary as long as it is an unlikely used string. The only problem I see in "%%%SPACE%%%" is that "%" is a special character in "$(patsubst ...)" context. This commit just uses "$(subst ...)" for arg-check, but I am fixing it now in case we might want to use it in $(patsubst ...) context in the future. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-07 06:48:26 +00:00
space_escape := _-_SPACE_-_
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
right_paren := )
left_paren := (
Kbuild: fix # escaping in .cmd files for future Make I tried building using a freshly built Make (4.2.1-69-g8a731d1), but already the objtool build broke with orc_dump.c: In function ‘orc_dump’: orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations] if (elf_getshdrnum(elf, &nr_sections)) { Turns out that with that new Make, the backslash was not removed, so cpp didn't see a #include directive, grep found nothing, and -DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS. Now, that new Make behaviour is documented in their NEWS file: * WARNING: Backward-incompatibility! Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes: thus a call such as: foo := $(shell echo '#') is legal. Previously the number sign needed to be escaped, for example: foo := $(shell echo '\#') Now this latter will resolve to "\#". If you want to write makefiles portable to both versions, assign the number sign to a variable: C := \# foo := $(shell echo '$C') This was claimed to be fixed in 3.81, but wasn't, for some reason. To detect this change search for 'nocomment' in the .FEATURES variable. This also fixes up the two make-cmd instances to replace # with $(pound) rather than with \#. There might very well be other places that need similar fixup in preparation for whatever future Make release contains the above change, but at least this builds an x86_64 defconfig with the new make. Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847 Cc: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-08 21:35:28 +00:00
pound := \#
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
dot-target = $(dir $@).$(notdir $@)
###
# The temporary file to save gcc -MD generated dependencies must not
# contain a comma
depfile = $(subst $(comma),_,$(dot-target).d)
###
# filename of target with directory and extension stripped
basetarget = $(basename $(notdir $@))
###
# filename of first prerequisite with directory and extension stripped
baseprereq = $(basename $(notdir $<))
###
# Escape single quote for use in echo statements
escsq = $(subst $(squote),'\$(squote)',$1)
###
# Easy method for doing a status message
kecho := :
quiet_kecho := echo
silent_kecho := :
kecho := $($(quiet)kecho)
###
# filechk is used to check if the content of a generated file is updated.
# Sample usage:
# define filechk_sample
# echo $KERNELRELEASE
# endef
# version.h : Makefile
# $(call filechk,sample)
# The rule defined shall write to stdout the content of the new file.
# The existing file will be compared with the new one.
# - If no file exist it is created
# - If the content differ the new file is used
# - If they are equal no change, and no timestamp update
# - stdin is piped in from the first prerequisite ($<) so one has
# to specify a valid file as first prerequisite (often the kbuild file)
define filechk
$(Q)set -e; \
$(kecho) ' CHK $@'; \
mkdir -p $(dir $@); \
$(filechk_$(1)) < $< > $@.tmp; \
if [ -r $@ ] && cmp -s $@ $@.tmp; then \
rm -f $@.tmp; \
else \
$(kecho) ' UPD $@'; \
mv -f $@.tmp $@; \
fi
endef
######
# gcc support functions
# See documentation in Documentation/kbuild/makefiles.txt
# cc-cross-prefix
# Usage: CROSS_COMPILE := $(call cc-cross-prefix, m68k-linux-gnu- m68k-linux-)
# Return first prefix where a prefix$(CC) is found in PATH.
# If no $(CC) found in PATH with listed prefixes return nothing
cc-cross-prefix = \
$(word 1, $(foreach c,$(1), \
$(shell set -e; \
if (which $(strip $(c))$(CC)) > /dev/null 2>&1 ; then \
echo $(c); \
fi)))
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
# Tools for caching Makefile variables that are "expensive" to compute.
#
# Here we want to help deal with variables that take a long time to compute
# by making it easy to store these variables in a cache.
#
# The canonical example here is testing for compiler flags. On a simple system
# each call to the compiler takes 10 ms, but on a system with a compiler that's
# called through various wrappers it can take upwards of 100 ms. If we have
# 100 calls to the compiler this can take 1 second (on a simple system) or 10
# seconds (on a complicated system).
#
# The "cache" will be in Makefile syntax and can be directly included.
# Any time we try to reference a variable that's not in the cache we'll
# calculate it and store it in the cache for next time.
# Include values from last time
make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk
$(make-cache): ;
-include $(make-cache)
cached-data := $(filter __cached_%, $(.VARIABLES))
# If cache exceeds 1000 lines, shrink it down to 500.
ifneq ($(word 1000,$(cached-data)),)
$(shell tail -n 500 $(make-cache) > $(make-cache).tmp; \
mv $(make-cache).tmp $(make-cache))
endif
create-cache-dir := $(if $(KBUILD_SRC),$(if $(cache-data),,1))
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
# Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios)
#
# Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_'
__sanitize-opt = $(subst $$,_,$(subst $(right_paren),_,$(subst $(left_paren),_,$(subst \,_,$(subst =,_,$(subst $(space),_,$(subst $(comma),_,$(subst :,_,$(1)))))))))
# Usage: $(call shell-cached,shell_command)
# Example: $(call shell-cached,md5sum /usr/bin/gcc)
#
# If we've already seen a call to this exact shell command (even in a
# previous invocation of make!) we'll return the value. If not, we'll
# compute it and store the result for future runs.
#
# This is a bit of voodoo, but basic explanation is that if the variable
# was undefined then we'll evaluate the shell command and store the result
# into the variable. We'll then store that value in the cache and finally
# output the value.
#
# NOTE: The $$(2) here isn't actually a parameter to __run-and-store. We
# happen to know that the caller will have their shell command in $(2) so the
# result of "call"ing this will produce a reference to that $(2). The reason
# for this strangeness is to avoid an extra level of eval (and escaping) of
# $(2).
define __run-and-store
ifeq ($(origin $(1)),undefined)
$$(eval $(1) := $$(shell $$(2)))
ifeq ($(create-cache-dir),1)
$$(shell mkdir -p $(dir $(make-cache)))
$$(eval create-cache-dir :=)
endif
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
$$(shell echo '$(1) := $$($(1))' >> $(make-cache))
endif
endef
__shell-cached = $(eval $(call __run-and-store,$(1)))$($(1))
shell-cached = $(call __shell-cached,__cached_$(call __sanitize-opt,$(1)),$(1))
# output directory for tests below
TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/)
# try-run
# Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)
# Exit code chooses option. "$$TMP" serves as a temporary file and is
# automatically cleaned up.
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
__try-run = set -e; \
TMP="$(TMPOUT).$$$$.tmp"; \
TMPO="$(TMPOUT).$$$$.o"; \
if ($(1)) >/dev/null 2>&1; \
then echo "$(2)"; \
else echo "$(3)"; \
fi; \
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
rm -f "$$TMP" "$$TMPO"
try-run = $(shell $(__try-run))
# try-run-cached
# This works like try-run, but the result is cached.
try-run-cached = $(call shell-cached,$(__try-run))
# as-option
# Usage: cflags-y += $(call as-option,-Wa$(comma)-isa=foo,)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
as-option = $(call try-run-cached,\
$(CC) $(KBUILD_CFLAGS) $(1) -c -x assembler /dev/null -o "$$TMP",$(1),$(2))
# as-instr
# Usage: cflags-y += $(call as-instr,instr,option1,option2)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
as-instr = $(call try-run-cached,\
printf "%b\n" "$(1)" | $(CC) $(KBUILD_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3))
# __cc-option
# Usage: MY_CFLAGS += $(call __cc-option,$(CC),$(MY_CFLAGS),-march=winchip-c6,-march=i586)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
__cc-option = $(call try-run-cached,\
$(1) -Werror $(2) $(3) -c -x c /dev/null -o "$$TMP",$(3),$(4))
# Do not attempt to build with gcc plugins during cc-option tests.
# (And this uses delayed resolution so the flags will be up to date.)
CC_OPTION_CFLAGS = $(filter-out $(GCC_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# cc-option
# Usage: cflags-y += $(call cc-option,-march=winchip-c6,-march=i586)
cc-option = $(call __cc-option, $(CC),\
$(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS),$(1),$(2))
# hostcc-option
# Usage: cflags-y += $(call hostcc-option,-march=winchip-c6,-march=i586)
hostcc-option = $(call __cc-option, $(HOSTCC),\
$(HOSTCFLAGS) $(HOST_EXTRACFLAGS),$(1),$(2))
# cc-option-yn
# Usage: flag := $(call cc-option-yn,-march=winchip-c6)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-option-yn = $(call try-run-cached,\
$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) $(1) -c -x c /dev/null -o "$$TMP",y,n)
# cc-disable-warning
# Usage: cflags-y += $(call cc-disable-warning,unused-but-set-variable)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-disable-warning = $(call try-run-cached,\
$(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
# cc-name
# Expands to either gcc or clang
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-name = $(call shell-cached,$(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc)
# cc-version
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-version = $(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC))
# cc-fullversion
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-fullversion = $(call shell-cached,$(CONFIG_SHELL) \
$(srctree)/scripts/gcc-version.sh -p $(CC))
# cc-ifversion
# Usage: EXTRA_CFLAGS += $(call cc-ifversion, -lt, 0402, -O1)
cc-ifversion = $(shell [ $(cc-version) $(1) $(2) ] && echo $(3) || echo $(4))
x86/build: Mostly disable '-maccumulate-outgoing-args' The GCC '-maccumulate-outgoing-args' flag is enabled for most configs, mostly because of issues which are no longer relevant. For most configs, and with most recent versions of GCC, it's no longer needed. Clarify which cases need it, and only enable it for those cases. Also produce a compile-time error for the ftrace graph + mcount + '-Os' case, which will otherwise cause runtime failures. The main benefit of '-maccumulate-outgoing-args' is that it prevents an ugly prologue for functions which have aligned stacks. But removing the option also has some benefits: more readable argument saves, smaller text size, and (presumably) slightly improved performance. Here are the object size savings for 32-bit and 64-bit defconfig kernels: text data bss dec hex filename 10006710 3543328 1773568 15323606 e9d1d6 vmlinux.x86-32.before 9706358 3547424 1773568 15027350 e54c96 vmlinux.x86-32.after text data bss dec hex filename 10652105 4537576 843776 16033457 f4a6b1 vmlinux.x86-64.before 10639629 4537576 843776 16020981 f475f5 vmlinux.x86-64.after That comes out to a 3% text size improvement on x86-32 and a 0.1% text size improvement on x86-64. Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Andrew Lutomirski <luto@kernel.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Pavel Machek <pavel@ucw.cz> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/20170316193133.zrj6gug53766m6nn@treble Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-03-16 19:31:33 +00:00
# cc-if-fullversion
# Usage: EXTRA_CFLAGS += $(call cc-if-fullversion, -lt, 040502, -O1)
cc-if-fullversion = $(shell [ $(cc-fullversion) $(1) $(2) ] && echo $(3) || echo $(4))
# cc-ldoption
# Usage: ldflags += $(call cc-ldoption, -Wl$(comma)--hash-style=both)
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
cc-ldoption = $(call try-run-cached,\
kbuild: fix linker feature test macros when cross compiling with Clang I was not seeing my linker flags getting added when using ld-option when cross compiling with Clang. Upon investigation, this seems to be due to a difference in how GCC vs Clang handle cross compilation. GCC is configured at build time to support one backend, that is implicit when compiling. Clang is explicit via the use of `-target <triple>` and ships with all supported backends by default. GNU Make feature test macros that compile then link will always fail when cross compiling with Clang unless Clang's triple is passed along to the compiler. For example: $ clang -x c /dev/null -c -o temp.o $ aarch64-linux-android/bin/ld -E temp.o aarch64-linux-android/bin/ld: unknown architecture of input file `temp.o' is incompatible with aarch64 output aarch64-linux-android/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000400078 $ echo $? 1 $ clang -target aarch64-linux-android- -x c /dev/null -c -o temp.o $ aarch64-linux-android/bin/ld -E temp.o aarch64-linux-android/bin/ld: warning: cannot find entry symbol _start; defaulting to 00000000004002e4 $ echo $? 0 This causes conditional checks that invoke $(CC) without the target triple, then $(LD) on the result, to always fail. Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Matthias Kaehlcke <mka@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-06 18:47:54 +00:00
$(CC) $(1) $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
# ld-option
# Usage: LDFLAGS += $(call ld-option, -X)
kbuild: simplify ld-option implementation Currently, linker options are tested by the coordination of $(CC) and $(LD) because $(LD) needs some object to link. As commit 86a9df597cdd ("kbuild: fix linker feature test macros when cross compiling with Clang") addressed, we need to make sure $(CC) and $(LD) agree the underlying architecture of the passed object. This could be a bit complex when we combine tools from different groups. For example, we can use clang for $(CC), but we still need to rely on GCC toolchain for $(LD). So, I was searching for a way of standalone testing of linker options. A trick I found is to use '-v'; this not only prints the version string, but also tests if the given option is recognized. If a given option is supported, $ aarch64-linux-gnu-ld -v --fix-cortex-a53-843419 GNU ld (Linaro_Binutils-2017.11) 2.28.2.20170706 $ echo $? 0 If unsupported, $ aarch64-linux-gnu-ld -v --fix-cortex-a53-843419 GNU ld (crosstool-NG linaro-1.13.1-4.7-2013.04-20130415 - Linaro GCC 2013.04) 2.23.1 aarch64-linux-gnu-ld: unrecognized option '--fix-cortex-a53-843419' aarch64-linux-gnu-ld: use the --help option for usage information $ echo $? 1 Gold works likewise. $ aarch64-linux-gnu-ld.gold -v --fix-cortex-a53-843419 GNU gold (Linaro_Binutils-2017.11 2.28.2.20170706) 1.14 masahiro@pug:~/ref/linux$ echo $? 0 $ aarch64-linux-gnu-ld.gold -v --fix-cortex-a53-999999 GNU gold (Linaro_Binutils-2017.11 2.28.2.20170706) 1.14 aarch64-linux-gnu-ld.gold: --fix-cortex-a53-999999: unknown option aarch64-linux-gnu-ld.gold: use the --help option for usage information $ echo $? 1 LLD too. $ ld.lld -v --gc-sections LLD 7.0.0 (http://llvm.org/git/lld.git 4a0e4190e74cea19f8a8dc625ccaebdf8b5d1585) (compatible with GNU linkers) $ echo $? 0 $ ld.lld -v --fix-cortex-a53-843419 LLD 7.0.0 (http://llvm.org/git/lld.git 4a0e4190e74cea19f8a8dc625ccaebdf8b5d1585) (compatible with GNU linkers) $ echo $? 0 $ ld.lld -v --fix-cortex-a53-999999 ld.lld: error: unknown argument: --fix-cortex-a53-999999 LLD 7.0.0 (http://llvm.org/git/lld.git 4a0e4190e74cea19f8a8dc625ccaebdf8b5d1585) (compatible with GNU linkers) $ echo $? 1 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2018-02-23 04:56:53 +00:00
ld-option = $(call try-run-cached, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2))
# ar-option
# Usage: KBUILD_ARFLAGS := $(call ar-option,D)
# Important: no spaces around options
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
ar-option = $(call try-run-cached, $(AR) rc$(1) "$$TMP",$(1),$(2))
# ld-version
# Note this is mainly for HJ Lu's 3 number binutil versions
kbuild: Add a cache for generated variables While timing a "no-op" build of the kernel (incrementally building the kernel even though nothing changed) in the Chrome OS build system I found that it was much slower than I expected. Digging into things a bit, I found that quite a bit of the time was spent invoking the C compiler even though we weren't actually building anything. Currently in the Chrome OS build system the C compiler is called through a number of wrappers (one of which is written in python!) and can take upwards of 100 ms to invoke even if we're not doing anything difficult, so these invocations of the compiler were taking a lot of time. Worse the invocations couldn't seem to take advantage of the multiple cores on my system. Certainly it seems like we could make the compiler invocations in the Chrome OS build system faster, but only to a point. Inherently invoking a program as big as a C compiler is a fairly heavy operation. Thus even if we can speed the compiler calls it made sense to track down what was happening. It turned out that all the compiler invocations were coming from usages like this in the kernel's Makefile: KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,) Due to the way cc-option and similar statements work the above contains an implicit call to the C compiler. ...and due to the fact that we're storing the result in KBUILD_CFLAGS, a simply expanded variable, the call will happen every time the Makefile is parsed, even if there are no users of KBUILD_CFLAGS. Rather than redoing this computation every time, it makes a lot of sense to cache the result of all of the Makefile's compiler calls just like we do when we compile a ".c" file to a ".o" file. Conceptually this is quite a simple idea. ...and since the calls to invoke the compiler and similar tools are centrally located in the Kbuild.include file this doesn't even need to be super invasive. Implementing the cache in a simple-to-use and efficient way is not quite as simple as it first sounds, though. To get maximum speed we really want the cache in a format that make can natively understand and make doesn't really have an ability to load/parse files. ...but make _can_ import other Makefiles, so the solution is to store the cache in Makefile format. This requires coming up with a valid/unique Makefile variable name for each value to be cached, but that's solvable with some cleverness. After this change, we'll automatically create a ".cache.mk" file that will contain our cached variables. We'll load this on each invocation of make and will avoid recomputing anything that's already in our cache. The cache is stored in a format that it shouldn't need any invalidation since anything that might change should affect the "key" and any old cached value won't be used. Signed-off-by: Douglas Anderson <dianders@chromium.org> Tested-by: Ingo Molnar <mingo@kernel.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-10-16 17:12:45 +00:00
ld-version = $(call shell-cached,$(LD) --version | $(srctree)/scripts/ld-version.sh)
# ld-ifversion
# Usage: $(call ld-ifversion, -ge, 22252, y)
ld-ifversion = $(shell [ $(ld-version) $(1) $(2) ] && echo $(3) || echo $(4))
######
[PATCH] vDSO hash-style fix The latest toolchains can produce a new ELF section in DSOs and dynamically-linked executables. The new section ".gnu.hash" replaces ".hash", and allows for more efficient runtime symbol lookups by the dynamic linker. The new ld option --hash-style={sysv|gnu|both} controls whether to produce the old ".hash", the new ".gnu.hash", or both. In some new systems such as Fedora Core 6, gcc by default passes --hash-style=gnu to the linker, so that a standard invocation of "gcc -shared" results in producing a DSO with only ".gnu.hash". The new ".gnu.hash" sections need to be dealt with the same way as ".hash" sections in all respects; only the dynamic linker cares about their contents. To work with older dynamic linkers (i.e. preexisting releases of glibc), a binary must have the old ".hash" section. The --hash-style=both option produces binaries that a new dynamic linker can use more efficiently, but an old dynamic linker can still handle. The new section runs afoul of the custom linker scripts used to build vDSO images for the kernel. On ia64, the failure mode for this is a boot-time panic because the vDSO's PT_IA_64_UNWIND segment winds up ill-formed. This patch addresses the problem in two ways. First, it mentions ".gnu.hash" in all the linker scripts alongside ".hash". This produces correct vDSO images with --hash-style=sysv (or old tools), with --hash-style=gnu, or with --hash-style=both. Second, it passes the --hash-style=sysv option when building the vDSO images, so that ".gnu.hash" is not actually produced. This is the most conservative choice for compatibility with any old userland. There is some concern that some ancient glibc builds (though not any known old production system) might choke on --hash-style=both binaries. The optimizations provided by the new style of hash section do not really matter for a DSO with a tiny number of symbols, as the vDSO has. If someone wants to use =gnu or =both for their vDSO builds and worry less about that compatibility, just change the option and the linker script changes will make any choice work fine. Signed-off-by: Roland McGrath <roland@redhat.com> Cc: "Luck, Tony" <tony.luck@intel.com> Cc: Kyle McMartin <kyle@mcmartin.ca> Cc: Paul Mackerras <paulus@samba.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Jeff Dike <jdike@addtoit.com> Cc: Andi Kleen <ak@muc.de> Cc: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-07-30 10:04:06 +00:00
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.build obj=
# Usage:
# $(Q)$(MAKE) $(build)=dir
build := -f $(srctree)/scripts/Makefile.build obj
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.modbuiltin obj=
# Usage:
# $(Q)$(MAKE) $(modbuiltin)=dir
modbuiltin := -f $(srctree)/scripts/Makefile.modbuiltin obj
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.dtbinst obj=
# Usage:
# $(Q)$(MAKE) $(dtbinst)=dir
dtbinst := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.dtbinst obj
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.clean obj=
# Usage:
# $(Q)$(MAKE) $(clean)=dir
clean := -f $(srctree)/scripts/Makefile.clean obj
###
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.headersinst obj=
# Usage:
# $(Q)$(MAKE) $(hdr-inst)=dir
hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj
# Prefix -I with $(srctree) if it is not an absolute path.
# skip if -I has no parameter
addtree = $(if $(patsubst -I%,%,$(1)), \
$(if $(filter-out -I/% -I./% -I../%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1)),$(1)))
# Find all -I options and call addtree
flags = $(foreach o,$($(1)),$(if $(filter -I%,$(o)),$(call addtree,$(o)),$(o)))
# echo command.
# Short version is used, if $(quiet) equals `quiet_', otherwise full one.
echo-cmd = $(if $($(quiet)cmd_$(1)),\
echo ' $(call escsq,$($(quiet)cmd_$(1)))$(echo-why)';)
# printing commands
cmd = @$(echo-cmd) $(cmd_$(1))
# Add $(obj)/ for paths that are not absolute
objectify = $(foreach o,$(1),$(if $(filter /%,$(o)),$(o),$(obj)/$(o)))
###
# if_changed - execute command if any prerequisite is newer than
# target, or command line has changed
# if_changed_dep - as if_changed, but uses fixdep to reveal dependencies
# including used config symbols
# if_changed_rule - as if_changed but execute rule instead
# See Documentation/kbuild/makefiles.txt for more info
ifneq ($(KBUILD_NOCMDDEP),1)
kbuild: fix if_change and friends to consider argument order Currently, arg-check is implemented as follows: arg-check = $(strip $(filter-out $(cmd_$(1)), $(cmd_$@)) \ $(filter-out $(cmd_$@), $(cmd_$(1))) ) This does not care about the order of arguments that appear in $(cmd_$(1)) and $(cmd_$@). So, if_changed and friends never rebuild the target if only the argument order is changed. This is a problem when the link order is changed. Apparently, obj-y += foo.o obj-y += bar.o and obj-y += bar.o obj-y += foo.o should be distinguished because the link order determines the probe order of drivers. So, built-in.o should be rebuilt when the order of objects is changed. This commit fixes arg-check to compare the old/current commands including the argument order. Of course, this change has a side effect; Kbuild will react to the change of compile option order. For example, "-DFOO -DBAR" and "-DBAR -DFOO" should give no difference to the build result, but false positive should be better than false negative. I am moving space_escape to the top of Kbuild.include just for a matter of preference. In practical terms, space_escape can be defined after arg-check because arg-check uses "=" flavor, not ":=". Having said that, collecting convenient variables in one place makes sense from the point of readability. Chaining "%%%SPACE%%%" to "_-_SPACE_-_" is also a matter of taste at this point. Actually, it can be arbitrary as long as it is an unlikely used string. The only problem I see in "%%%SPACE%%%" is that "%" is a special character in "$(patsubst ...)" context. This commit just uses "$(subst ...)" for arg-check, but I am fixing it now in case we might want to use it in $(patsubst ...) context in the future. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-07 06:48:26 +00:00
# Check if both arguments are the same including their order. Result is empty
# string if equal. User may override this check using make KBUILD_NOCMDDEP=1
arg-check = $(filter-out $(subst $(space),$(space_escape),$(strip $(cmd_$@))), \
$(subst $(space),$(space_escape),$(strip $(cmd_$1))))
else
arg-check = $(if $(strip $(cmd_$@)),,1)
endif
# Replace >$< with >$$< to preserve $ when reloading the .cmd file
# (needed for make)
Kbuild: fix # escaping in .cmd files for future Make I tried building using a freshly built Make (4.2.1-69-g8a731d1), but already the objtool build broke with orc_dump.c: In function ‘orc_dump’: orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations] if (elf_getshdrnum(elf, &nr_sections)) { Turns out that with that new Make, the backslash was not removed, so cpp didn't see a #include directive, grep found nothing, and -DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS. Now, that new Make behaviour is documented in their NEWS file: * WARNING: Backward-incompatibility! Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes: thus a call such as: foo := $(shell echo '#') is legal. Previously the number sign needed to be escaped, for example: foo := $(shell echo '\#') Now this latter will resolve to "\#". If you want to write makefiles portable to both versions, assign the number sign to a variable: C := \# foo := $(shell echo '$C') This was claimed to be fixed in 3.81, but wasn't, for some reason. To detect this change search for 'nocomment' in the .FEATURES variable. This also fixes up the two make-cmd instances to replace # with $(pound) rather than with \#. There might very well be other places that need similar fixup in preparation for whatever future Make release contains the above change, but at least this builds an x86_64 defconfig with the new make. Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847 Cc: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-08 21:35:28 +00:00
# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file
# (needed for make)
# Replace >'< with >'\''< to be able to enclose the whole string in '...'
# (needed for the shell)
Kbuild: fix # escaping in .cmd files for future Make I tried building using a freshly built Make (4.2.1-69-g8a731d1), but already the objtool build broke with orc_dump.c: In function ‘orc_dump’: orc_dump.c:106:2: error: ‘elf_getshnum’ is deprecated [-Werror=deprecated-declarations] if (elf_getshdrnum(elf, &nr_sections)) { Turns out that with that new Make, the backslash was not removed, so cpp didn't see a #include directive, grep found nothing, and -DLIBELF_USE_DEPRECATED was wrongly put in CFLAGS. Now, that new Make behaviour is documented in their NEWS file: * WARNING: Backward-incompatibility! Number signs (#) appearing inside a macro reference or function invocation no longer introduce comments and should not be escaped with backslashes: thus a call such as: foo := $(shell echo '#') is legal. Previously the number sign needed to be escaped, for example: foo := $(shell echo '\#') Now this latter will resolve to "\#". If you want to write makefiles portable to both versions, assign the number sign to a variable: C := \# foo := $(shell echo '$C') This was claimed to be fixed in 3.81, but wasn't, for some reason. To detect this change search for 'nocomment' in the .FEATURES variable. This also fixes up the two make-cmd instances to replace # with $(pound) rather than with \#. There might very well be other places that need similar fixup in preparation for whatever future Make release contains the above change, but at least this builds an x86_64 defconfig with the new make. Link: https://bugzilla.kernel.org/show_bug.cgi?id=197847 Cc: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-08 21:35:28 +00:00
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
# Find any prerequisites that is newer than target or that does not exist.
# PHONY targets skipped in both cases.
any-prereq = $(filter-out $(PHONY),$?) $(filter-out $(PHONY) $(wildcard $^),$^)
# Execute command if command has changed or prerequisite(s) are updated.
if_changed = $(if $(strip $(any-prereq) $(arg-check)), \
@set -e; \
$(echo-cmd) $(cmd_$(1)); \
printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
# Execute the command and also postprocess generated .d dependencies file.
if_changed_dep = $(if $(strip $(any-prereq) $(arg-check) ), \
@set -e; \
$(cmd_and_fixdep), @:)
kbuild: add fine grained build dependencies for exported symbols Like with kconfig options, we now have the ability to compile in and out individual EXPORT_SYMBOL() declarations based on the content of include/generated/autoksyms.h. However we don't want the entire world to be rebuilt whenever that file is touched. Let's apply the same build dependency trick used for CONFIG_* symbols where the time stamp of empty files whose paths matching those symbols is used to trigger fine grained rebuilds. In our case the key is the symbol name passed to EXPORT_SYMBOL(). However, unlike config options, we cannot just use fixdep to parse the source code for EXPORT_SYMBOL(ksym) because several variants exist and parsing them all in a separate tool, and keeping it in synch, is not trivially maintainable. Furthermore, there are variants such as EXPORT_SYMBOL_GPL(pci_user_read_config_##size); that are instanciated via a macro for which we can't easily determine the actual exported symbol name(s) short of actually running the preprocessor on them. Storing the symbol name string in a special ELF section doesn't work for targets that output assembly or preprocessed source. So the best way is really to leverage the preprocessor by having it output actual symbol names anchored by a special sequence that can be easily filtered out. Then the list of symbols is simply fed to fixdep to be merged with the other dependencies. That implies the preprocessor is executed twice for each source file. A previous attempt relied on a warning pragma for each EXPORT_SYMBOL() instance that was filtered apart from stderr by the build system with a sed script during the actual compilation pass. Unfortunately the preprocessor/compiler diagnostic output isn't stable between versions and this solution, although more efficient, was deemed too fragile. Because of the lowercasing performed by fixdep, there might be name collisions triggering spurious rebuilds for similar symbols. But this shouldn't be a big issue in practice. (This is the case for CONFIG_* symbols and I didn't want to be different here, whatever the original reason for doing so.) To avoid needless build overhead, the exported symbol name gathering is performed only when CONFIG_TRIM_UNUSED_KSYMS is selected. Signed-off-by: Nicolas Pitre <nico@linaro.org> Acked-by: Rusty Russell <rusty@rustcorp.com.au>
2016-01-22 18:41:57 +00:00
ifndef CONFIG_TRIM_UNUSED_KSYMS
cmd_and_fixdep = \
$(echo-cmd) $(cmd_$(1)); \
scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).tmp;\
rm -f $(depfile); \
mv -f $(dot-target).tmp $(dot-target).cmd;
kbuild: add fine grained build dependencies for exported symbols Like with kconfig options, we now have the ability to compile in and out individual EXPORT_SYMBOL() declarations based on the content of include/generated/autoksyms.h. However we don't want the entire world to be rebuilt whenever that file is touched. Let's apply the same build dependency trick used for CONFIG_* symbols where the time stamp of empty files whose paths matching those symbols is used to trigger fine grained rebuilds. In our case the key is the symbol name passed to EXPORT_SYMBOL(). However, unlike config options, we cannot just use fixdep to parse the source code for EXPORT_SYMBOL(ksym) because several variants exist and parsing them all in a separate tool, and keeping it in synch, is not trivially maintainable. Furthermore, there are variants such as EXPORT_SYMBOL_GPL(pci_user_read_config_##size); that are instanciated via a macro for which we can't easily determine the actual exported symbol name(s) short of actually running the preprocessor on them. Storing the symbol name string in a special ELF section doesn't work for targets that output assembly or preprocessed source. So the best way is really to leverage the preprocessor by having it output actual symbol names anchored by a special sequence that can be easily filtered out. Then the list of symbols is simply fed to fixdep to be merged with the other dependencies. That implies the preprocessor is executed twice for each source file. A previous attempt relied on a warning pragma for each EXPORT_SYMBOL() instance that was filtered apart from stderr by the build system with a sed script during the actual compilation pass. Unfortunately the preprocessor/compiler diagnostic output isn't stable between versions and this solution, although more efficient, was deemed too fragile. Because of the lowercasing performed by fixdep, there might be name collisions triggering spurious rebuilds for similar symbols. But this shouldn't be a big issue in practice. (This is the case for CONFIG_* symbols and I didn't want to be different here, whatever the original reason for doing so.) To avoid needless build overhead, the exported symbol name gathering is performed only when CONFIG_TRIM_UNUSED_KSYMS is selected. Signed-off-by: Nicolas Pitre <nico@linaro.org> Acked-by: Rusty Russell <rusty@rustcorp.com.au>
2016-01-22 18:41:57 +00:00
else
# Filter out exported kernel symbol names from the preprocessor output.
# See also __KSYM_DEPS__ in include/linux/export.h.
# We disable the depfile generation here, so as not to overwrite the existing
# depfile while fixdep is parsing it.
flags_nodeps = $(filter-out -Wp$(comma)-M%, $($(1)))
ksym_dep_filter = \
case "$(1)" in \
cc_*_c|cpp_i_c) \
$(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;; \
as_*_S|cpp_s_S) \
$(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;; \
boot*|build*|cpp_its_S|*cpp_lds_S|dtc|host*|vdso*) : ;; \
*) echo "Don't know how to preprocess $(1)" >&2; false ;; \
esac | tr ";" "\n" | sed -n 's/^.*=== __KSYM_\(.*\) ===.*$$/_\1/p'
kbuild: add fine grained build dependencies for exported symbols Like with kconfig options, we now have the ability to compile in and out individual EXPORT_SYMBOL() declarations based on the content of include/generated/autoksyms.h. However we don't want the entire world to be rebuilt whenever that file is touched. Let's apply the same build dependency trick used for CONFIG_* symbols where the time stamp of empty files whose paths matching those symbols is used to trigger fine grained rebuilds. In our case the key is the symbol name passed to EXPORT_SYMBOL(). However, unlike config options, we cannot just use fixdep to parse the source code for EXPORT_SYMBOL(ksym) because several variants exist and parsing them all in a separate tool, and keeping it in synch, is not trivially maintainable. Furthermore, there are variants such as EXPORT_SYMBOL_GPL(pci_user_read_config_##size); that are instanciated via a macro for which we can't easily determine the actual exported symbol name(s) short of actually running the preprocessor on them. Storing the symbol name string in a special ELF section doesn't work for targets that output assembly or preprocessed source. So the best way is really to leverage the preprocessor by having it output actual symbol names anchored by a special sequence that can be easily filtered out. Then the list of symbols is simply fed to fixdep to be merged with the other dependencies. That implies the preprocessor is executed twice for each source file. A previous attempt relied on a warning pragma for each EXPORT_SYMBOL() instance that was filtered apart from stderr by the build system with a sed script during the actual compilation pass. Unfortunately the preprocessor/compiler diagnostic output isn't stable between versions and this solution, although more efficient, was deemed too fragile. Because of the lowercasing performed by fixdep, there might be name collisions triggering spurious rebuilds for similar symbols. But this shouldn't be a big issue in practice. (This is the case for CONFIG_* symbols and I didn't want to be different here, whatever the original reason for doing so.) To avoid needless build overhead, the exported symbol name gathering is performed only when CONFIG_TRIM_UNUSED_KSYMS is selected. Signed-off-by: Nicolas Pitre <nico@linaro.org> Acked-by: Rusty Russell <rusty@rustcorp.com.au>
2016-01-22 18:41:57 +00:00
cmd_and_fixdep = \
$(echo-cmd) $(cmd_$(1)); \
$(ksym_dep_filter) | \
scripts/basic/fixdep -e $(depfile) $@ '$(make-cmd)' \
> $(dot-target).tmp; \
rm -f $(depfile); \
mv -f $(dot-target).tmp $(dot-target).cmd;
endif
# Usage: $(call if_changed_rule,foo)
# Will check if $(cmd_foo) or any of the prerequisites changed,
# and if so will execute $(rule_foo).
if_changed_rule = $(if $(strip $(any-prereq) $(arg-check) ), \
@set -e; \
$(rule_$(1)), @:)
###
# why - tell why a target got built
# enabled by make V=2
# Output (listed in the order they are checked):
# (1) - due to target is PHONY
# (2) - due to target missing
# (3) - due to: file1.h file2.h
# (4) - due to command line change
# (5) - due to missing .cmd file
# (6) - due to target not in $(targets)
# (1) PHONY targets are always build
# (2) No target, so we better build it
# (3) Prerequisite is newer than target
# (4) The command line stored in the file named dir/.target.cmd
# differed from actual command line. This happens when compiler
# options changes
# (5) No dir/.target.cmd file (used to store command line)
# (6) No dir/.target.cmd file and target not listed in $(targets)
# This is a good hint that there is a bug in the kbuild file
ifeq ($(KBUILD_VERBOSE),2)
why = \
$(if $(filter $@, $(PHONY)),- due to target is PHONY, \
$(if $(wildcard $@), \
$(if $(strip $(any-prereq)),- due to: $(any-prereq), \
$(if $(arg-check), \
$(if $(cmd_$@),- due to command line change, \
$(if $(filter $@, $(targets)), \
- due to missing .cmd file, \
- due to $(notdir $@) not in $$(targets) \
) \
) \
) \
), \
- due to target missing \
) \
)
echo-why = $(call escsq, $(strip $(why)))
endif
###############################################################################
#
# When a Kconfig string contains a filename, it is suitable for
# passing to shell commands. It is surrounded by double-quotes, and
# any double-quotes or backslashes within it are escaped by
# backslashes.
#
# This is no use for dependencies or $(wildcard). We need to strip the
# surrounding quotes and the escaping from quotes and backslashes, and
# we *do* need to escape any spaces in the string. So, for example:
#
# Usage: $(eval $(call config_filename,FOO))
#
# Defines FOO_FILENAME based on the contents of the CONFIG_FOO option,
# transformed as described above to be suitable for use within the
# makefile.
#
# Also, if the filename is a relative filename and exists in the source
# tree but not the build tree, define FOO_SRCPREFIX as $(srctree)/ to
# be prefixed to *both* command invocation and dependencies.
#
# Note: We also print the filenames in the quiet_cmd_foo text, and
# perhaps ought to have a version specially escaped for that purpose.
# But it's only cosmetic, and $(patsubst "%",%,$(CONFIG_FOO)) is good
# enough. It'll strip the quotes in the common case where there's no
# space and it's a simple filename, and it'll retain the quotes when
# there's a space. There are some esoteric cases in which it'll print
# the wrong thing, but we don't really care. The actual dependencies
# and commands *do* get it right, with various combinations of single
# and double quotes, backslashes and spaces in the filenames.
#
###############################################################################
#
define config_filename
ifneq ($$(CONFIG_$(1)),"")
$(1)_FILENAME := $$(subst \\,\,$$(subst \$$(quote),$$(quote),$$(subst $$(space_escape),\$$(space),$$(patsubst "%",%,$$(subst $$(space),$$(space_escape),$$(CONFIG_$(1)))))))
ifneq ($$(patsubst /%,%,$$(firstword $$($(1)_FILENAME))),$$(firstword $$($(1)_FILENAME)))
else
ifeq ($$(wildcard $$($(1)_FILENAME)),)
ifneq ($$(wildcard $$(srctree)/$$($(1)_FILENAME)),)
$(1)_SRCPREFIX := $(srctree)/
endif
endif
endif
endif
endef
#
###############################################################################