linux_dsm_epyc7002/Makefile

1956 lines
63 KiB
Makefile
Raw Normal View History

License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 21:07:57 +07:00
# SPDX-License-Identifier: GPL-2.0
2019-01-07 08:08:20 +07:00
VERSION = 5
2020-10-26 05:14:11 +07:00
PATCHLEVEL = 10
SUBLEVEL = 10
2020-12-14 05:41:30 +07:00
EXTRAVERSION =
2019-10-28 00:19:19 +07:00
NAME = Kleptomaniac Octopus
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
# More info can be located in ./README
# Comments in this file are targeted only to the developer, do not
# expect to learn how to build the kernel reading this file.
$(if $(filter __%, $(MAKECMDGOALS)), \
$(error targets prefixed with '__' are only for internal use))
# That's our default target when none is given on the command line
PHONY := __all
__all:
# We are using a recursive build, so we need to do a little thinking
# to get the ordering right.
#
# Most importantly: sub-Makefiles should only ever modify files in
# their own directory. If in some directory we have a dependency on
# a file in another dir (which doesn't happen often, but it's often
# unavoidable when linking the built-in.a targets which finally
# turn into vmlinux), we will call a sub make in that other dir, and
# after that we are sure that everything which is in that other dir
# is now up to date.
#
# The only cases where we need to modify files which have global
# effects are thus separated out and done before the recursive
# descending is started. They are now explicitly listed as the
# prepare rule.
ifneq ($(sub_make_done),1)
kbuild: make -r/-R effective in top Makefile for old Make versions Adding -rR to MAKEFLAGS is important because we do not want to be bothered by built-in implicit rules or variables. One problem that used to exist in older GNU Make versions is MAKEFLAGS += -rR ... does not become effective in the current Makefile. When you are building with O= option, it becomes effective in the top Makefile since it recurses via 'sub-make' target. Otherwise, the top Makefile tries implicit rules. That is why we explicitly add empty rules for Makefiles, but we often miss to do that. In fact, adding -d option to older GNU Make versions shows it is trying a bunch of implicit pattern rules. Considering target file `scripts/Makefile.kcov'. Looking for an implicit rule for `scripts/Makefile.kcov'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.o'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.c'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.cc'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.C'. ... This issue was fixed by GNU Make commit 58dae243526b ("[Savannah #20501] Handle adding -r/-R to MAKEFLAGS in the makefile"). So, it is no longer a problem if you use GNU Make 4.0 or later. However, older versions are still widely used. So, I decided to patch the kernel Makefile to invoke sub-make regardless of O= option. This will allow further cleanups. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-22 14:40:07 +07:00
# Do not use make's built-in rules and variables
# (this increases performance and avoids hard-to-debug behaviour)
MAKEFLAGS += -rR
# Avoid funny character set dependencies
unexport LC_ALL
LC_COLLATE=C
LC_NUMERIC=C
export LC_COLLATE LC_NUMERIC
# Avoid interference with shell env settings
unexport GREP_OPTIONS
# Beautify output
# ---------------------------------------------------------------------------
#
# Normally, we echo the whole command before executing it. By making
# that echo $($(quiet)$(cmd)), we now have the possibility to set
# $(quiet) to choose other forms of output instead, e.g.
#
# quiet_cmd_cc_o_c = Compiling $(RELDIR)/$@
# cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
#
# If $(quiet) is empty, the whole command will be printed.
# If it is set to "quiet_", only the short version will be printed.
# If it is set to "silent_", nothing will be printed at all, since
# the variable $(silent_cmd_cc_o_c) doesn't exist.
#
# A simple variant is to prefix commands with $(Q) - that's useful
# for commands that shall be hidden in non-verbose mode.
#
# $(Q)ln $@ :<
#
# If KBUILD_VERBOSE equals 0 then the above command will be hidden.
# If KBUILD_VERBOSE equals 1 then the above command is displayed.
# If KBUILD_VERBOSE equals 2 then give the reason why each target is rebuilt.
#
# To put more focus on warnings, be less verbose as default
# Use 'make V=1' to see the full commands
ifeq ("$(origin V)", "command line")
KBUILD_VERBOSE = $(V)
endif
ifndef KBUILD_VERBOSE
KBUILD_VERBOSE = 0
endif
ifeq ($(KBUILD_VERBOSE),1)
quiet =
Q =
else
quiet=quiet_
Q = @
endif
# If the user is running make -s (silent mode), suppress echoing of
# commands
ifneq ($(findstring s,$(filter-out --%,$(MAKEFLAGS))),)
quiet=silent_
endif
export quiet Q KBUILD_VERBOSE
# Kbuild will save output files in the current working directory.
# This does not need to match to the root of the kernel source tree.
#
# For example, you can do this:
#
# cd /dir/to/store/output/files; make -f /dir/to/kernel/source/Makefile
#
# If you want to save output files in a different location, there are
# two syntaxes to specify it.
#
# 1) O=
# Use "make O=dir/to/store/output/files/"
#
# 2) Set KBUILD_OUTPUT
# Set the environment variable KBUILD_OUTPUT to point to the output directory.
# export KBUILD_OUTPUT=dir/to/store/output/files/; make
#
# The O= assignment takes precedence over the KBUILD_OUTPUT environment
# variable.
# Do we want to change the working directory?
ifeq ("$(origin O)", "command line")
KBUILD_OUTPUT := $(O)
endif
ifneq ($(KBUILD_OUTPUT),)
# Make's built-in functions such as $(abspath ...), $(realpath ...) cannot
# expand a shell special character '~'. We use a somewhat tedious way here.
abs_objtree := $(shell mkdir -p $(KBUILD_OUTPUT) && cd $(KBUILD_OUTPUT) && pwd)
$(if $(abs_objtree),, \
$(error failed to create output directory "$(KBUILD_OUTPUT)"))
# $(realpath ...) resolves symlinks
abs_objtree := $(realpath $(abs_objtree))
else
abs_objtree := $(CURDIR)
endif # ifneq ($(KBUILD_OUTPUT),)
ifeq ($(abs_objtree),$(CURDIR))
# Suppress "Entering directory ..." unless we are changing the work directory.
MAKEFLAGS += --no-print-directory
else
need-sub-make := 1
endif
abs_srctree := $(realpath $(dir $(lastword $(MAKEFILE_LIST))))
ifneq ($(words $(subst :, ,$(abs_srctree))), 1)
$(error source directory cannot contain spaces or colons)
endif
ifneq ($(abs_srctree),$(abs_objtree))
# Look for make include files relative to root of kernel src
#
# This does not become effective immediately because MAKEFLAGS is re-parsed
# once after the Makefile is read. We need to invoke sub-make.
MAKEFLAGS += --include-dir=$(abs_srctree)
need-sub-make := 1
endif
kbuild: make -r/-R effective in top Makefile for old Make versions Adding -rR to MAKEFLAGS is important because we do not want to be bothered by built-in implicit rules or variables. One problem that used to exist in older GNU Make versions is MAKEFLAGS += -rR ... does not become effective in the current Makefile. When you are building with O= option, it becomes effective in the top Makefile since it recurses via 'sub-make' target. Otherwise, the top Makefile tries implicit rules. That is why we explicitly add empty rules for Makefiles, but we often miss to do that. In fact, adding -d option to older GNU Make versions shows it is trying a bunch of implicit pattern rules. Considering target file `scripts/Makefile.kcov'. Looking for an implicit rule for `scripts/Makefile.kcov'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.o'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.c'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.cc'. Trying pattern rule with stem `Makefile.kcov'. Trying implicit prerequisite `scripts/Makefile.kcov.C'. ... This issue was fixed by GNU Make commit 58dae243526b ("[Savannah #20501] Handle adding -r/-R to MAKEFLAGS in the makefile"). So, it is no longer a problem if you use GNU Make 4.0 or later. However, older versions are still widely used. So, I decided to patch the kernel Makefile to invoke sub-make regardless of O= option. This will allow further cleanups. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-22 14:40:07 +07:00
this-makefile := $(lastword $(MAKEFILE_LIST))
ifneq ($(filter 3.%,$(MAKE_VERSION)),)
# 'MAKEFLAGS += -rR' does not immediately become effective for GNU Make 3.x
# We need to invoke sub-make to avoid implicit rules in the top Makefile.
need-sub-make := 1
# Cancel implicit rules for this Makefile.
$(this-makefile): ;
endif
export abs_srctree abs_objtree
export sub_make_done := 1
ifeq ($(need-sub-make),1)
PHONY += $(MAKECMDGOALS) __sub-make
$(filter-out $(this-makefile), $(MAKECMDGOALS)) __all: __sub-make
@:
# Invoke a second make in the output directory, passing relevant variables
__sub-make:
$(Q)$(MAKE) -C $(abs_objtree) -f $(abs_srctree)/Makefile $(MAKECMDGOALS)
endif # need-sub-make
endif # sub_make_done
# We process the rest of the Makefile if this is the final invocation of make
ifeq ($(need-sub-make),)
# Do not print "Entering directory ...",
# but we want to display it when entering to the output directory
# so that IDEs/editors are able to understand relative filenames.
MAKEFLAGS += --no-print-directory
# Call a source code checker (by default, "sparse") as part of the
# C compilation.
#
# Use 'make C=1' to enable checking of only re-compiled files.
# Use 'make C=2' to enable checking of *all* source files, regardless
# of whether they are re-compiled or not.
#
# See the file "Documentation/dev-tools/sparse.rst" for more details,
# including where to get the "sparse" utility.
ifeq ("$(origin C)", "command line")
KBUILD_CHECKSRC = $(C)
endif
ifndef KBUILD_CHECKSRC
KBUILD_CHECKSRC = 0
endif
# Use make M=dir or set the environment variable KBUILD_EXTMOD to specify the
# directory of external module to build. Setting M= takes precedence.
ifeq ("$(origin M)", "command line")
KBUILD_EXTMOD := $(M)
endif
$(if $(word 2, $(KBUILD_EXTMOD)), \
$(error building multiple external modules is not supported))
export KBUILD_CHECKSRC KBUILD_EXTMOD
kbuild: make single targets work more correctly Currently, the single target build directly descends into the directory of the target. For example, $ make foo/bar/baz.o ... directly descends into foo/bar/. On the other hand, the normal build usually descends one directory at a time, i.e. descends into foo/, and then foo/bar/. This difference causes some problems. [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles The options in subdir-{as,cc}flags-y take effect in the current and its sub-directories. In other words, they are inherited downward. In the example above, the single target will miss subdir-{as,cc}flags-y if they are defined in foo/Makefile. [2] could be built in a different directory As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can handle files that are spread over several sub-directories. The build rule of foo/bar/baz.o may not necessarily be specified in foo/bar/Makefile. It might be specifies in foo/Makefile as follows: [foo/Makefile] obj-y := bar/baz.o This often happens when a module is so big that its source files are divided into sub-directories. In this case, there is no Makefile in the foo/bar/ directory, yet the single target descends into foo/bar/, then fails due to the missing Makefile. You can still do 'make foo/bar/' for partial building, but cannot do 'make foo/bar/baz.s'. I believe the single target '%.s' is a useful feature for inspecting the compiler output. Some modules work around this issue by putting an empty Makefile in every sub-directory. This commit fixes those problems by making the single target build descend in the same way as the normal build does. Another change is the single target build will observe the CONFIG options. Previously, it allowed users to build the foo.o even when the corresponding CONFIG_FOO is disabled: obj-$(CONFIG_FOO) += foo.o In the new behavior, the single target build will just fail and show "No rule to make target ..." (or "Nothing to be done for ..." if the stale object already exists, but cannot be updated). The disadvantage of this commit is the build speed. Now that the single target build visits every directory and parses lots of Makefiles, it is slower than before. (But, I hope it will not be too slow.) Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 22:19:18 +07:00
extmod-prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/)
ifeq ($(abs_srctree),$(abs_objtree))
# building in the source tree
srctree := .
building_out_of_srctree :=
else
ifeq ($(abs_srctree)/,$(dir $(abs_objtree)))
# building in a subdirectory of the source tree
srctree := ..
else
srctree := $(abs_srctree)
endif
building_out_of_srctree := 1
endif
ifneq ($(KBUILD_ABS_SRCTREE),)
srctree := $(abs_srctree)
endif
objtree := .
VPATH := $(srctree)
export building_out_of_srctree srctree objtree VPATH
# To make sure we do not include .config for any of the *config targets
# catch them early, and hand them over to scripts/kconfig/Makefile
# It is allowed to specify more targets when calling make, including
# mixing *config targets and build targets.
# For example 'make oldconfig all'.
# Detect when mixed targets is specified, and make a second invocation
# of make so .config is not included in this case either (for *config).
version_h := include/generated/uapi/linux/version.h
old_version_h := include/linux/version.h
clean-targets := %clean mrproper cleandocs
no-dot-config-targets := $(clean-targets) \
cscope gtags TAGS tags help% %docs check% coccicheck \
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
$(version_h) headers headers_% archheaders archscripts \
%asm-generic kernelversion %src-pkg dt_binding_check \
outputmakefile
no-sync-config-targets := $(no-dot-config-targets) %install kernelrelease
kbuild: make single targets work more correctly Currently, the single target build directly descends into the directory of the target. For example, $ make foo/bar/baz.o ... directly descends into foo/bar/. On the other hand, the normal build usually descends one directory at a time, i.e. descends into foo/, and then foo/bar/. This difference causes some problems. [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles The options in subdir-{as,cc}flags-y take effect in the current and its sub-directories. In other words, they are inherited downward. In the example above, the single target will miss subdir-{as,cc}flags-y if they are defined in foo/Makefile. [2] could be built in a different directory As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can handle files that are spread over several sub-directories. The build rule of foo/bar/baz.o may not necessarily be specified in foo/bar/Makefile. It might be specifies in foo/Makefile as follows: [foo/Makefile] obj-y := bar/baz.o This often happens when a module is so big that its source files are divided into sub-directories. In this case, there is no Makefile in the foo/bar/ directory, yet the single target descends into foo/bar/, then fails due to the missing Makefile. You can still do 'make foo/bar/' for partial building, but cannot do 'make foo/bar/baz.s'. I believe the single target '%.s' is a useful feature for inspecting the compiler output. Some modules work around this issue by putting an empty Makefile in every sub-directory. This commit fixes those problems by making the single target build descend in the same way as the normal build does. Another change is the single target build will observe the CONFIG options. Previously, it allowed users to build the foo.o even when the corresponding CONFIG_FOO is disabled: obj-$(CONFIG_FOO) += foo.o In the new behavior, the single target build will just fail and show "No rule to make target ..." (or "Nothing to be done for ..." if the stale object already exists, but cannot be updated). The disadvantage of this commit is the build speed. Now that the single target build visits every directory and parses lots of Makefiles, it is slower than before. (But, I hope it will not be too slow.) Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 22:19:18 +07:00
single-targets := %.a %.i %.ko %.lds %.ll %.lst %.mod %.o %.s %.symtypes %/
config-build :=
mixed-build :=
need-config := 1
may-sync-config := 1
kbuild: make single targets work more correctly Currently, the single target build directly descends into the directory of the target. For example, $ make foo/bar/baz.o ... directly descends into foo/bar/. On the other hand, the normal build usually descends one directory at a time, i.e. descends into foo/, and then foo/bar/. This difference causes some problems. [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles The options in subdir-{as,cc}flags-y take effect in the current and its sub-directories. In other words, they are inherited downward. In the example above, the single target will miss subdir-{as,cc}flags-y if they are defined in foo/Makefile. [2] could be built in a different directory As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can handle files that are spread over several sub-directories. The build rule of foo/bar/baz.o may not necessarily be specified in foo/bar/Makefile. It might be specifies in foo/Makefile as follows: [foo/Makefile] obj-y := bar/baz.o This often happens when a module is so big that its source files are divided into sub-directories. In this case, there is no Makefile in the foo/bar/ directory, yet the single target descends into foo/bar/, then fails due to the missing Makefile. You can still do 'make foo/bar/' for partial building, but cannot do 'make foo/bar/baz.s'. I believe the single target '%.s' is a useful feature for inspecting the compiler output. Some modules work around this issue by putting an empty Makefile in every sub-directory. This commit fixes those problems by making the single target build descend in the same way as the normal build does. Another change is the single target build will observe the CONFIG options. Previously, it allowed users to build the foo.o even when the corresponding CONFIG_FOO is disabled: obj-$(CONFIG_FOO) += foo.o In the new behavior, the single target build will just fail and show "No rule to make target ..." (or "Nothing to be done for ..." if the stale object already exists, but cannot be updated). The disadvantage of this commit is the build speed. Now that the single target build visits every directory and parses lots of Makefiles, it is slower than before. (But, I hope it will not be too slow.) Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 22:19:18 +07:00
single-build :=
ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),)
ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),)
need-config :=
endif
endif
kbuild: do not update config when running install targets "make syncconfig" is automatically invoked when any of the following happens: - .config is updated - any of Kconfig files is updated - any of environment variables referenced in Kconfig is changed Then, it updates configuration files such as include/config/auto.conf include/generated/autoconf.h, etc. Even install targets (install, modules_install, etc.) are no exception. However, they should never ever modify the source tree. Install targets are often run with root privileges. Once those configuration files are owned by root, "make mrproper" would end up with permission error. Install targets should just copy things blindly. They should not care whether the configuration is up-to-date or not. This makes more sense because we are interested in the configuration that was used in the previous kernel building. This issue has existed since before, but rarely happened. I expect more chance where people are hit by this; with the new Kconfig syntax extension, the .config now contains the compiler information. If you cross-compile the kernel with CROSS_COMPILE, but forget to pass it for "make install", you meet "any of environment variables referenced in Kconfig is changed" because $(CC) is referenced in Kconfig. Another scenario is the compiler upgrade before the installation. Install targets need the configuration. "make modules_install" refer to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_* to decide which dtb files to install. However, the auto-update of the configuration files should be avoided. We already do this for external modules. Now, Make targets are categorized into 3 groups: [1] Do not need the kernel configuration at all help, coccicheck, headers_install etc. [2] Need the latest kernel configuration If new config options are added, Kconfig will show prompt to ask user's selection. Build targets such as vmlinux, in-kernel modules are the cases. [3] Need the kernel configuration, but do not want to update it Install targets except headers_install, and external modules are the cases. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-20 14:46:34 +07:00
ifneq ($(filter $(no-sync-config-targets), $(MAKECMDGOALS)),)
ifeq ($(filter-out $(no-sync-config-targets), $(MAKECMDGOALS)),)
may-sync-config :=
kbuild: do not update config when running install targets "make syncconfig" is automatically invoked when any of the following happens: - .config is updated - any of Kconfig files is updated - any of environment variables referenced in Kconfig is changed Then, it updates configuration files such as include/config/auto.conf include/generated/autoconf.h, etc. Even install targets (install, modules_install, etc.) are no exception. However, they should never ever modify the source tree. Install targets are often run with root privileges. Once those configuration files are owned by root, "make mrproper" would end up with permission error. Install targets should just copy things blindly. They should not care whether the configuration is up-to-date or not. This makes more sense because we are interested in the configuration that was used in the previous kernel building. This issue has existed since before, but rarely happened. I expect more chance where people are hit by this; with the new Kconfig syntax extension, the .config now contains the compiler information. If you cross-compile the kernel with CROSS_COMPILE, but forget to pass it for "make install", you meet "any of environment variables referenced in Kconfig is changed" because $(CC) is referenced in Kconfig. Another scenario is the compiler upgrade before the installation. Install targets need the configuration. "make modules_install" refer to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_* to decide which dtb files to install. However, the auto-update of the configuration files should be avoided. We already do this for external modules. Now, Make targets are categorized into 3 groups: [1] Do not need the kernel configuration at all help, coccicheck, headers_install etc. [2] Need the latest kernel configuration If new config options are added, Kconfig will show prompt to ask user's selection. Build targets such as vmlinux, in-kernel modules are the cases. [3] Need the kernel configuration, but do not want to update it Install targets except headers_install, and external modules are the cases. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-20 14:46:34 +07:00
endif
endif
ifneq ($(KBUILD_EXTMOD),)
may-sync-config :=
kbuild: do not update config when running install targets "make syncconfig" is automatically invoked when any of the following happens: - .config is updated - any of Kconfig files is updated - any of environment variables referenced in Kconfig is changed Then, it updates configuration files such as include/config/auto.conf include/generated/autoconf.h, etc. Even install targets (install, modules_install, etc.) are no exception. However, they should never ever modify the source tree. Install targets are often run with root privileges. Once those configuration files are owned by root, "make mrproper" would end up with permission error. Install targets should just copy things blindly. They should not care whether the configuration is up-to-date or not. This makes more sense because we are interested in the configuration that was used in the previous kernel building. This issue has existed since before, but rarely happened. I expect more chance where people are hit by this; with the new Kconfig syntax extension, the .config now contains the compiler information. If you cross-compile the kernel with CROSS_COMPILE, but forget to pass it for "make install", you meet "any of environment variables referenced in Kconfig is changed" because $(CC) is referenced in Kconfig. Another scenario is the compiler upgrade before the installation. Install targets need the configuration. "make modules_install" refer to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_* to decide which dtb files to install. However, the auto-update of the configuration files should be avoided. We already do this for external modules. Now, Make targets are categorized into 3 groups: [1] Do not need the kernel configuration at all help, coccicheck, headers_install etc. [2] Need the latest kernel configuration If new config options are added, Kconfig will show prompt to ask user's selection. Build targets such as vmlinux, in-kernel modules are the cases. [3] Need the kernel configuration, but do not want to update it Install targets except headers_install, and external modules are the cases. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-20 14:46:34 +07:00
endif
ifeq ($(KBUILD_EXTMOD),)
ifneq ($(filter %config,$(MAKECMDGOALS)),)
config-build := 1
ifneq ($(words $(MAKECMDGOALS)),1)
mixed-build := 1
endif
endif
endif
kbuild: make single targets work more correctly Currently, the single target build directly descends into the directory of the target. For example, $ make foo/bar/baz.o ... directly descends into foo/bar/. On the other hand, the normal build usually descends one directory at a time, i.e. descends into foo/, and then foo/bar/. This difference causes some problems. [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles The options in subdir-{as,cc}flags-y take effect in the current and its sub-directories. In other words, they are inherited downward. In the example above, the single target will miss subdir-{as,cc}flags-y if they are defined in foo/Makefile. [2] could be built in a different directory As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can handle files that are spread over several sub-directories. The build rule of foo/bar/baz.o may not necessarily be specified in foo/bar/Makefile. It might be specifies in foo/Makefile as follows: [foo/Makefile] obj-y := bar/baz.o This often happens when a module is so big that its source files are divided into sub-directories. In this case, there is no Makefile in the foo/bar/ directory, yet the single target descends into foo/bar/, then fails due to the missing Makefile. You can still do 'make foo/bar/' for partial building, but cannot do 'make foo/bar/baz.s'. I believe the single target '%.s' is a useful feature for inspecting the compiler output. Some modules work around this issue by putting an empty Makefile in every sub-directory. This commit fixes those problems by making the single target build descend in the same way as the normal build does. Another change is the single target build will observe the CONFIG options. Previously, it allowed users to build the foo.o even when the corresponding CONFIG_FOO is disabled: obj-$(CONFIG_FOO) += foo.o In the new behavior, the single target build will just fail and show "No rule to make target ..." (or "Nothing to be done for ..." if the stale object already exists, but cannot be updated). The disadvantage of this commit is the build speed. Now that the single target build visits every directory and parses lots of Makefiles, it is slower than before. (But, I hope it will not be too slow.) Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 22:19:18 +07:00
# We cannot build single targets and the others at the same time
ifneq ($(filter $(single-targets), $(MAKECMDGOALS)),)
single-build := 1
ifneq ($(filter-out $(single-targets), $(MAKECMDGOALS)),)
mixed-build := 1
endif
endif
# For "make -j clean all", "make -j mrproper defconfig all", etc.
ifneq ($(filter $(clean-targets),$(MAKECMDGOALS)),)
ifneq ($(filter-out $(clean-targets),$(MAKECMDGOALS)),)
mixed-build := 1
endif
endif
# install and modules_install need also be processed one by one
ifneq ($(filter install,$(MAKECMDGOALS)),)
ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
mixed-build := 1
endif
endif
ifdef mixed-build
# ===========================================================================
# We're called with mixed targets (*config and build targets).
# Handle them one by one.
PHONY += $(MAKECMDGOALS) __build_one_by_one
$(MAKECMDGOALS): __build_one_by_one
@:
__build_one_by_one:
$(Q)set -e; \
for i in $(MAKECMDGOALS); do \
$(MAKE) -f $(srctree)/Makefile $$i; \
done
else # !mixed-build
include scripts/Kbuild.include
# Read KERNELRELEASE from include/config/kernel.release (if it exists)
KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null)
KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION)
export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
include scripts/subarch.include
# Cross compiling and selecting different set of gcc/bin-utils
# ---------------------------------------------------------------------------
#
# When performing cross compilation for other architectures ARCH shall be set
# to the target architecture. (See arch/* for the possibilities).
# ARCH can be set during invocation of make:
# make ARCH=ia64
# Another way is to have ARCH set in the environment.
# The default ARCH is the host where make is executed.
# CROSS_COMPILE specify the prefix used for all executables used
# during compilation. Only gcc and related bin-utils executables
# are prefixed with $(CROSS_COMPILE).
# CROSS_COMPILE can be set on the command line
# make CROSS_COMPILE=ia64-linux-
# Alternatively CROSS_COMPILE can be set in the environment.
# Default value for CROSS_COMPILE is not to prefix executables
# Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile
ARCH ?= $(SUBARCH)
# Architecture as present in compile.h
UTS_MACHINE := $(ARCH)
SRCARCH := $(ARCH)
# Additional ARCH settings for x86
ifeq ($(ARCH),i386)
SRCARCH := x86
endif
ifeq ($(ARCH),x86_64)
SRCARCH := x86
endif
# Additional ARCH settings for sparc
ifeq ($(ARCH),sparc32)
SRCARCH := sparc
endif
ifeq ($(ARCH),sparc64)
SRCARCH := sparc
endif
# Additional ARCH settings for sh
ifeq ($(ARCH),sh64)
SRCARCH := sh
endif
KCONFIG_CONFIG ?= .config
export KCONFIG_CONFIG
kbuild: use KBUILD_DEFCONFIG as the fallback for DEFCONFIG_LIST Most of the Kconfig commands (except defconfig and all*config) read the .config file as a base set of CONFIG options. When it does not exist, the files in DEFCONFIG_LIST are searched in this order and loaded if found. I do not see much sense in the last two lines in DEFCONFIG_LIST. [1] ARCH_DEFCONFIG The entry for DEFCONFIG_LIST is guarded by 'depends on !UML'. So, the ARCH_DEFCONFIG definition in arch/x86/um/Kconfig is meaningless. arch/{sh,sparc,x86}/Kconfig define ARCH_DEFCONFIG depending on 32 or 64 bit variant symbols. This is a little bit strange; ARCH_DEFCONFIG should be a fixed string because the base config file is loaded before the symbol evaluation stage. Using KBUILD_DEFCONFIG makes more sense because it is fixed before Kconfig is invoked. Fortunately, arch/{sh,sparc,x86}/Makefile define it in the same way, and it works as expected. Hence, replace ARCH_DEFCONFIG with "arch/$(SRCARCH)/configs/$(KBUILD_DEFCONFIG)". [2] arch/$(ARCH)/defconfig This file path is no longer valid. The defconfig files are always located in the arch configs/ directories. $ find arch -name defconfig | sort arch/alpha/configs/defconfig arch/arm64/configs/defconfig arch/csky/configs/defconfig arch/nds32/configs/defconfig arch/riscv/configs/defconfig arch/s390/configs/defconfig arch/unicore32/configs/defconfig The path arch/*/configs/defconfig is already covered by "arch/$(SRCARCH)/configs/$(KBUILD_DEFCONFIG)". So, this file path is not necessary. I moved the default KBUILD_DEFCONFIG to the top Makefile. Otherwise, the 7 architectures listed above would end up with endless loop of syncconfig. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-28 10:46:40 +07:00
# Default file for 'make defconfig'. This may be overridden by arch-Makefile.
export KBUILD_DEFCONFIG := defconfig
# SHELL used by kbuild
CONFIG_SHELL := sh
HOST_LFS_CFLAGS := $(shell getconf LFS_CFLAGS 2>/dev/null)
HOST_LFS_LDFLAGS := $(shell getconf LFS_LDFLAGS 2>/dev/null)
HOST_LFS_LIBS := $(shell getconf LFS_LIBS 2>/dev/null)
kbuild: support LLVM=1 to switch the default tools to Clang/LLVM As Documentation/kbuild/llvm.rst implies, building the kernel with a full set of LLVM tools gets very verbose and unwieldy. Provide a single switch LLVM=1 to use Clang and LLVM tools instead of GCC and Binutils. You can pass it from the command line or as an environment variable. Please note LLVM=1 does not turn on the integrated assembler. You need to pass LLVM_IAS=1 to use it. When the upstream kernel is ready for the integrated assembler, I think we can make it default. We discussed what we need, and we agreed to go with a simple boolean flag that switches both target and host tools: https://lkml.org/lkml/2020/3/28/494 https://lkml.org/lkml/2020/4/3/43 Some items discussed, but not adopted: - LLVM_DIR When multiple versions of LLVM are installed, I just thought supporting LLVM_DIR=/path/to/my/llvm/bin/ might be useful. CC = $(LLVM_DIR)clang LD = $(LLVM_DIR)ld.lld ... However, we can handle this by modifying PATH. So, we decided to not do this. - LLVM_SUFFIX Some distributions (e.g. Debian) package specific versions of LLVM with naming conventions that use the version as a suffix. CC = clang$(LLVM_SUFFIX) LD = ld.lld(LLVM_SUFFIX) ... will allow a user to pass LLVM_SUFFIX=-11 to use clang-11 etc., but the suffixed versions in /usr/bin/ are symlinks to binaries in /usr/lib/llvm-#/bin/, so this can also be handled by PATH. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> # build Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-08 08:36:23 +07:00
ifneq ($(LLVM),)
HOSTCC = clang
HOSTCXX = clang++
else
HOSTCC = gcc
HOSTCXX = g++
endif
kbuild: add infrastructure to build userspace programs Kbuild supports the infrastructure to build host programs, but there was no support to build userspace programs for the target architecture (i.e. the same architecture as the kernel). Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154), but it was not merged. One problem at that time was, there was no good way to know whether $(CC) can link standalone programs. In fact, pre-built kernel.org toolchains [1] are often used for building the kernel, but they do not provide libc. Now, we can handle this cleanly because the compiler capability is evaluated at the Kconfig time. If $(CC) cannot link standalone programs, the relevant options are hidden by 'depends on CC_CAN_LINK'. The implementation just mimics scripts/Makefile.host The userspace programs are compiled with the same flags as the host programs. In addition, it uses -m32 or -m64 if it is found in $(KBUILD_CFLAGS). This new syntax has two usecases. - Sample programs Several userspace programs under samples/ include UAPI headers installed in usr/include. Most of them were previously built for the host architecture just to use the 'hostprogs' syntax. However, 'make headers' always works for the target architecture. This caused the arch mismatch in cross-compiling. To fix this distortion, sample code should be built for the target architecture. - Bpfilter net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper, and embeds it into the kernel. Currently, it overrides HOSTCC with CC to use the 'hostprogs' syntax. This hack should go away. [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Sam Ravnborg <sam@ravnborg.org>
2020-04-29 10:45:14 +07:00
export KBUILD_USERCFLAGS := -Wall -Wmissing-prototypes -Wstrict-prototypes \
-O2 -fomit-frame-pointer -std=gnu89
export KBUILD_USERLDFLAGS :=
KBUILD_HOSTCFLAGS := $(KBUILD_USERCFLAGS) $(HOST_LFS_CFLAGS) $(HOSTCFLAGS)
kbuild: add -Wall to KBUILD_HOSTCXXFLAGS Add -Wall to catch more warnings for C++ host programs. When I submitted the previous version, the 0-day bot reported -Wc++11-compat warnings for old GCC: HOSTCXX -fPIC scripts/gcc-plugins/latent_entropy_plugin.o In file included from /usr/lib/gcc/x86_64-linux-gnu/4.8/plugin/include/tm.h:28:0, from scripts/gcc-plugins/gcc-common.h:15, from scripts/gcc-plugins/latent_entropy_plugin.c:78: /usr/lib/gcc/x86_64-linux-gnu/4.8/plugin/include/config/elfos.h:102:21: warning: C++11 requires a space between string literal and macro [-Wc++11-compat] fprintf ((FILE), "%s"HOST_WIDE_INT_PRINT_UNSIGNED"\n",\ ^ /usr/lib/gcc/x86_64-linux-gnu/4.8/plugin/include/config/elfos.h:170:24: warning: C++11 requires a space between string literal and macro [-Wc++11-compat] fprintf ((FILE), ","HOST_WIDE_INT_PRINT_UNSIGNED",%u\n", \ ^ In file included from /usr/lib/gcc/x86_64-linux-gnu/4.8/plugin/include/tm.h:42:0, from scripts/gcc-plugins/gcc-common.h:15, from scripts/gcc-plugins/latent_entropy_plugin.c:78: /usr/lib/gcc/x86_64-linux-gnu/4.8/plugin/include/defaults.h:126:24: warning: C++11 requires a space between string literal and macro [-Wc++11-compat] fprintf ((FILE), ","HOST_WIDE_INT_PRINT_UNSIGNED",%u\n", \ ^ The source of the warnings is in the plugin headers, so we have no control of it. I just suppressed them by adding -Wno-c++11-compat to scripts/gcc-plugins/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Kees Cook <keescook@chromium.org>
2020-03-25 10:14:32 +07:00
KBUILD_HOSTCXXFLAGS := -Wall -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS)
KBUILD_HOSTLDFLAGS := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS)
KBUILD_HOSTLDLIBS := $(HOST_LFS_LIBS) $(HOSTLDLIBS)
# Make variables (CC, etc...)
CPP = $(CC) -E
kbuild: support LLVM=1 to switch the default tools to Clang/LLVM As Documentation/kbuild/llvm.rst implies, building the kernel with a full set of LLVM tools gets very verbose and unwieldy. Provide a single switch LLVM=1 to use Clang and LLVM tools instead of GCC and Binutils. You can pass it from the command line or as an environment variable. Please note LLVM=1 does not turn on the integrated assembler. You need to pass LLVM_IAS=1 to use it. When the upstream kernel is ready for the integrated assembler, I think we can make it default. We discussed what we need, and we agreed to go with a simple boolean flag that switches both target and host tools: https://lkml.org/lkml/2020/3/28/494 https://lkml.org/lkml/2020/4/3/43 Some items discussed, but not adopted: - LLVM_DIR When multiple versions of LLVM are installed, I just thought supporting LLVM_DIR=/path/to/my/llvm/bin/ might be useful. CC = $(LLVM_DIR)clang LD = $(LLVM_DIR)ld.lld ... However, we can handle this by modifying PATH. So, we decided to not do this. - LLVM_SUFFIX Some distributions (e.g. Debian) package specific versions of LLVM with naming conventions that use the version as a suffix. CC = clang$(LLVM_SUFFIX) LD = ld.lld(LLVM_SUFFIX) ... will allow a user to pass LLVM_SUFFIX=-11 to use clang-11 etc., but the suffixed versions in /usr/bin/ are symlinks to binaries in /usr/lib/llvm-#/bin/, so this can also be handled by PATH. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> # build Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-08 08:36:23 +07:00
ifneq ($(LLVM),)
CC = clang
LD = ld.lld
AR = llvm-ar
NM = llvm-nm
OBJCOPY = llvm-objcopy
OBJDUMP = llvm-objdump
READELF = llvm-readelf
STRIP = llvm-strip
else
CC = $(CROSS_COMPILE)gcc
LD = $(CROSS_COMPILE)ld
AR = $(CROSS_COMPILE)ar
NM = $(CROSS_COMPILE)nm
OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
READELF = $(CROSS_COMPILE)readelf
kbuild: support LLVM=1 to switch the default tools to Clang/LLVM As Documentation/kbuild/llvm.rst implies, building the kernel with a full set of LLVM tools gets very verbose and unwieldy. Provide a single switch LLVM=1 to use Clang and LLVM tools instead of GCC and Binutils. You can pass it from the command line or as an environment variable. Please note LLVM=1 does not turn on the integrated assembler. You need to pass LLVM_IAS=1 to use it. When the upstream kernel is ready for the integrated assembler, I think we can make it default. We discussed what we need, and we agreed to go with a simple boolean flag that switches both target and host tools: https://lkml.org/lkml/2020/3/28/494 https://lkml.org/lkml/2020/4/3/43 Some items discussed, but not adopted: - LLVM_DIR When multiple versions of LLVM are installed, I just thought supporting LLVM_DIR=/path/to/my/llvm/bin/ might be useful. CC = $(LLVM_DIR)clang LD = $(LLVM_DIR)ld.lld ... However, we can handle this by modifying PATH. So, we decided to not do this. - LLVM_SUFFIX Some distributions (e.g. Debian) package specific versions of LLVM with naming conventions that use the version as a suffix. CC = clang$(LLVM_SUFFIX) LD = ld.lld(LLVM_SUFFIX) ... will allow a user to pass LLVM_SUFFIX=-11 to use clang-11 etc., but the suffixed versions in /usr/bin/ are symlinks to binaries in /usr/lib/llvm-#/bin/, so this can also be handled by PATH. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> # build Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-08 08:36:23 +07:00
STRIP = $(CROSS_COMPILE)strip
endif
kbuild: add ability to generate BTF type info for vmlinux This patch adds new config option to trigger generation of BTF type information from DWARF debuginfo for vmlinux and kernel modules through pahole, which in turn relies on libbpf for btf_dedup() algorithm. The intent is to record compact type information of all types used inside kernel, including all the structs/unions/typedefs/etc. This enables BPF's compile-once-run-everywhere ([0]) approach, in which tracing programs that are inspecting kernel's internal data (e.g., struct task_struct) can be compiled on a system running some kernel version, but would be possible to run on other kernel versions (and configurations) without recompilation, even if the layout of structs changed and/or some of the fields were added, removed, or renamed. This is only possible if BPF loader can get kernel type info to adjust all the offsets correctly. This patch is a first time in this direction, making sure that BTF type info is part of Linux kernel image in non-loadable ELF section. BTF deduplication ([1]) algorithm typically provides 100x savings compared to DWARF data, so resulting .BTF section is not big as is typically about 2MB in size. [0] http://vger.kernel.org/lpc-bpf2018.html#session-2 [1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Alexei Starovoitov <ast@fb.com> Cc: Yonghong Song <yhs@fb.com> Cc: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Andrii Nakryiko <andriin@fb.com> Acked-by: David S. Miller <davem@davemloft.net> Acked-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-02 23:49:50 +07:00
PAHOLE = pahole
RESOLVE_BTFIDS = $(objtree)/tools/bpf/resolve_btfids/resolve_btfids
LEX = flex
YACC = bison
AWK = awk
kbuild: use INSTALLKERNEL to select customized installkernel script Replace the use of CROSS_COMPILE to select a customized installkernel script with the possibility to set INSTALLKERNEL to select a custom installkernel script when running make: make INSTALLKERNEL=arm-installkernel install With this patch we are now more consistent across different architectures - they did not all support use of CROSS_COMPILE. The use of CROSS_COMPILE was a hack as this really belongs to gcc/binutils and the installkernel script does not change just because we change toolchain. The use of CROSS_COMPILE caused troubles with an upcoming patch that saves CROSS_COMPILE when a kernel is built - it would no longer be installable. [Thanks to Peter Z. for this hint] This patch undos what Ian did in commit: 0f8e2d62fa04441cd12c08ce521e84e5bd3f8a46 ("use ${CROSS_COMPILE}installkernel in arch/*/boot/install.sh") The patch has been lightly tested on x86 only - but all changes looks obvious. Acked-by: Peter Zijlstra <peterz@infradead.org> Acked-by: Mike Frysinger <vapier@gentoo.org> [blackfin] Acked-by: Russell King <linux@arm.linux.org.uk> [arm] Acked-by: Paul Mundt <lethal@linux-sh.org> [sh] Acked-by: "H. Peter Anvin" <hpa@zytor.com> [x86] Cc: Ian Campbell <icampbell@arcom.com> Cc: Tony Luck <tony.luck@intel.com> [ia64] Cc: Fenghua Yu <fenghua.yu@intel.com> [ia64] Cc: Hirokazu Takata <takata@linux-m32r.org> [m32r] Cc: Geert Uytterhoeven <geert@linux-m68k.org> [m68k] Cc: Kyle McMartin <kyle@mcmartin.ca> [parisc] Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> [powerpc] Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> [s390] Cc: Thomas Gleixner <tglx@linutronix.de> [x86] Cc: Ingo Molnar <mingo@redhat.com> [x86] Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
2009-07-21 02:37:11 +07:00
INSTALLKERNEL := installkernel
DEPMOD = depmod
PERL = perl
PYTHON = python
kbuild: add PYTHON2 and PYTHON3 variables The variable 'PYTHON' allows users to specify a proper executable name in case the default 'python' does not work. However, this does not address the case where both Python 2.x and 3.x scripts are used in one source tree. PEP 394 (https://www.python.org/dev/peps/pep-0394/) provides a convention for Python scripts portability. Here is a quotation: In order to tolerate differences across platforms, all new code that needs to invoke the Python interpreter should not specify 'python', but rather should specify either 'python2' or 'python3'. This distinction should be made in shebangs, when invoking from a shell script, when invoking via the system() call, or when invoking in any other context. One exception to this is scripts that are deliberately written to be source compatible with both Python 2.x and 3.x. Such scripts may continue to use python on their shebang line without affecting their portability. To meet this requirement, this commit adds new variables 'PYTHON2' and 'PYTHON3'. arch/ia64/scripts/unwcheck.py is the only script that has ever used $(PYTHON). Recent commit bd5edbe67794 ("ia64: convert unwcheck.py to python3") converted it to be compatible with both Python 2.x and 3.x, so this is the exceptional case where the use of 'python' is allowed. So, I did not touch arch/ia64/Makefile. tools/perf/Makefile.config sets PYTHON and PYTHON2 by itself, so it is not affected by this commit. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-13 16:12:02 +07:00
PYTHON3 = python3
CHECK = sparse
BASH = bash
KGZIP = gzip
KBZIP2 = bzip2
KLZOP = lzop
LZMA = lzma
LZ4 = lz4c
XZ = xz
ZSTD = zstd
CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
-Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
NOSTDINC_FLAGS :=
kbuild: allow assignment to {A,C,LD}FLAGS_MODULE on the command line It is now possible to assign options to AS, CC and LD on the command line - which is only used when building modules. {A,C,LD}FLAGS_MODULE was all used both in the top-level Makefile in the arch makefiles, thus users had no way to specify additional options to AS, CC, LD when building modules without overriding the original value. Introduce a new set of variables KBUILD_{A,C,LD}FLAGS_MODULE that is used by arch specific files and free up {A,C,LD}FLAGS_MODULE so they can be assigned on the command line. All arch Makefiles that used the old variables has been updated. Note: Previously we had a MODFLAGS variable for both AS and CC. But in favour of consistency this was dropped. So in some cases arch Makefile has one assignmnet replaced by two assignmnets. Note2: MODFLAGS was not documented and is dropped without any notice. I do not expect much/any breakage from this. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Cc: Denys Vlasenko <vda.linux@googlemail.com> Cc: Haavard Skinnemoen <hskinnemoen@atmel.com> Cc: Mike Frysinger <vapier@gentoo.org> Cc: Tony Luck <tony.luck@intel.com> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Chen Liqin <liqin.chen@sunplusct.com> Acked-by: Mike Frysinger <vapier@gentoo.org> [blackfin] Acked-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com> [avr32] Signed-off-by: Michal Marek <mmarek@suse.cz>
2010-07-28 22:33:09 +07:00
CFLAGS_MODULE =
AFLAGS_MODULE =
LDFLAGS_MODULE =
CFLAGS_KERNEL =
AFLAGS_KERNEL =
LDFLAGS_vmlinux =
# Use USERINCLUDE when you must reference the UAPI directories only.
USERINCLUDE := \
-I$(srctree)/arch/$(SRCARCH)/include/uapi \
-I$(objtree)/arch/$(SRCARCH)/include/generated/uapi \
-I$(srctree)/include/uapi \
-I$(objtree)/include/generated/uapi \
-include $(srctree)/include/linux/kconfig.h
# Use LINUXINCLUDE when you must reference the include/ directory.
# Needed to be compatible with the O= option
LINUXINCLUDE := \
-I$(srctree)/arch/$(SRCARCH)/include \
-I$(objtree)/arch/$(SRCARCH)/include/generated \
$(if $(building_out_of_srctree),-I$(srctree)/include) \
-I$(objtree)/include \
$(USERINCLUDE)
KBUILD_AFLAGS := -D__ASSEMBLY__ -fno-PIE
KBUILD_CFLAGS := -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs \
-fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE \
-Werror=implicit-function-declaration -Werror=implicit-int \
-Werror=return-type -Wno-format-security \
-std=gnu89
KBUILD_CPPFLAGS := -D__KERNEL__
KBUILD_AFLAGS_KERNEL :=
KBUILD_CFLAGS_KERNEL :=
kbuild: allow assignment to {A,C,LD}FLAGS_MODULE on the command line It is now possible to assign options to AS, CC and LD on the command line - which is only used when building modules. {A,C,LD}FLAGS_MODULE was all used both in the top-level Makefile in the arch makefiles, thus users had no way to specify additional options to AS, CC, LD when building modules without overriding the original value. Introduce a new set of variables KBUILD_{A,C,LD}FLAGS_MODULE that is used by arch specific files and free up {A,C,LD}FLAGS_MODULE so they can be assigned on the command line. All arch Makefiles that used the old variables has been updated. Note: Previously we had a MODFLAGS variable for both AS and CC. But in favour of consistency this was dropped. So in some cases arch Makefile has one assignmnet replaced by two assignmnets. Note2: MODFLAGS was not documented and is dropped without any notice. I do not expect much/any breakage from this. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Cc: Denys Vlasenko <vda.linux@googlemail.com> Cc: Haavard Skinnemoen <hskinnemoen@atmel.com> Cc: Mike Frysinger <vapier@gentoo.org> Cc: Tony Luck <tony.luck@intel.com> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Chen Liqin <liqin.chen@sunplusct.com> Acked-by: Mike Frysinger <vapier@gentoo.org> [blackfin] Acked-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com> [avr32] Signed-off-by: Michal Marek <mmarek@suse.cz>
2010-07-28 22:33:09 +07:00
KBUILD_AFLAGS_MODULE := -DMODULE
KBUILD_CFLAGS_MODULE := -DMODULE
KBUILD_LDFLAGS_MODULE :=
KBUILD_LDFLAGS :=
CLANG_FLAGS :=
export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE LD CC
export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
kbuild: allow assignment to {A,C,LD}FLAGS_MODULE on the command line It is now possible to assign options to AS, CC and LD on the command line - which is only used when building modules. {A,C,LD}FLAGS_MODULE was all used both in the top-level Makefile in the arch makefiles, thus users had no way to specify additional options to AS, CC, LD when building modules without overriding the original value. Introduce a new set of variables KBUILD_{A,C,LD}FLAGS_MODULE that is used by arch specific files and free up {A,C,LD}FLAGS_MODULE so they can be assigned on the command line. All arch Makefiles that used the old variables has been updated. Note: Previously we had a MODFLAGS variable for both AS and CC. But in favour of consistency this was dropped. So in some cases arch Makefile has one assignmnet replaced by two assignmnets. Note2: MODFLAGS was not documented and is dropped without any notice. I do not expect much/any breakage from this. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Cc: Denys Vlasenko <vda.linux@googlemail.com> Cc: Haavard Skinnemoen <hskinnemoen@atmel.com> Cc: Mike Frysinger <vapier@gentoo.org> Cc: Tony Luck <tony.luck@intel.com> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Chen Liqin <liqin.chen@sunplusct.com> Acked-by: Mike Frysinger <vapier@gentoo.org> [blackfin] Acked-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com> [avr32] Signed-off-by: Michal Marek <mmarek@suse.cz>
2010-07-28 22:33:09 +07:00
export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
# Files to ignore in find ... statements
export RCS_FIND_IGNORE := \( -name SCCS -o -name BitKeeper -o -name .svn -o \
-name CVS -o -name .pc -o -name .hg -o -name .git \) \
-prune -o
export RCS_TAR_IGNORE := --exclude SCCS --exclude BitKeeper --exclude .svn \
--exclude CVS --exclude .pc --exclude .hg --exclude .git
# ===========================================================================
# Rules shared between *config targets and build targets
# Basic helpers built in scripts/basic/
PHONY += scripts_basic
scripts_basic:
$(Q)$(MAKE) $(build)=scripts/basic
$(Q)rm -f .tmp_quiet_recordmcount
PHONY += outputmakefile
# Before starting out-of-tree build, make sure the source tree is clean.
# outputmakefile generates a Makefile in the output directory, if using a
# separate output directory. This allows convenient use of make in the
# output directory.
# At the same time when output Makefile generated, generate .gitignore to
# ignore whole output directory
outputmakefile:
ifdef building_out_of_srctree
$(Q)if [ -f $(srctree)/.config -o \
-d $(srctree)/include/config -o \
-d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
echo >&2 "***"; \
echo >&2 "*** The source tree is not clean, please run 'make$(if $(findstring command line, $(origin ARCH)), ARCH=$(ARCH)) mrproper'"; \
echo >&2 "*** in $(abs_srctree)";\
echo >&2 "***"; \
false; \
fi
$(Q)ln -fsn $(srctree) source
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/mkmakefile $(srctree)
$(Q)test -e .gitignore || \
{ echo "# this is build directory, ignore it"; echo "*"; } > .gitignore
endif
ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
ifneq ($(CROSS_COMPILE),)
CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))
GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
Makefile: Fix GCC_TOOLCHAIN_DIR prefix for Clang cross compilation When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-elfedit, GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to /usr/bin/ and Clang as of 11 will search for both $(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle. GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle, $(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice, $(prefix)aarch64-linux-gnu/$needle rarely contains executables. To better model how GCC's -B/--prefix takes in effect in practice, newer Clang (since https://github.com/llvm/llvm-project/commit/3452a0d8c17f7166f479706b293caf6ac76ffd90) only searches for $(prefix)$needle. Currently it will find /usr/bin/as instead of /usr/bin/aarch64-linux-gnu-as. Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) (/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the appropriate cross compiling GNU as (when -no-integrated-as is in effect). Cc: stable@vger.kernel.org Reported-by: Nathan Chancellor <natechancellor@gmail.com> Signed-off-by: Fangrui Song <maskray@google.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Link: https://github.com/ClangBuiltLinux/linux/issues/1099 Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-07-22 00:31:23 +07:00
CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE))
GCC_TOOLCHAIN := $(realpath $(GCC_TOOLCHAIN_DIR)/..)
endif
ifneq ($(GCC_TOOLCHAIN),)
CLANG_FLAGS += --gcc-toolchain=$(GCC_TOOLCHAIN)
endif
ifneq ($(LLVM_IAS),1)
CLANG_FLAGS += -no-integrated-as
endif
kbuild: Add -Werror=unknown-warning-option to CLANG_FLAGS In commit ebcc5928c5d9 ("arm64: Silence gcc warnings about arch ABI drift"), the arm64 Makefile added -Wno-psabi to KBUILD_CFLAGS, which is a GCC only option so clang rightfully complains: warning: unknown warning option '-Wno-psabi' [-Wunknown-warning-option] https://clang.llvm.org/docs/DiagnosticsReference.html#wunknown-warning-option However, by default, this is merely a warning so the build happily goes on with a slew of these warnings in the process. Commit c3f0d0bc5b01 ("kbuild, LLVMLinux: Add -Werror to cc-option to support clang") worked around this behavior in cc-option by adding -Werror so that unknown flags cause an error. However, this all happens silently and when an unknown flag is added to the build unconditionally like -Wno-psabi, cc-option will always fail because there is always an unknown flag in the list of flags. This manifested as link time failures in the arm64 libstub because -fno-stack-protector didn't get added to KBUILD_CFLAGS. To avoid these weird cryptic failures in the future, make clang behave like gcc and immediately error when it encounters an unknown flag by adding -Werror=unknown-warning-option to CLANG_FLAGS. This can be added unconditionally for clang because it is supported by at least 3.0.0, according to godbolt [1] and 4.0.0, according to its documentation [2], which is far earlier than we typically support. [1]: https://godbolt.org/z/7F7rm3 [2]: https://releases.llvm.org/4.0.0/tools/clang/docs/DiagnosticsReference.html#wunknown-warning-option Link: https://github.com/ClangBuiltLinux/linux/issues/511 Link: https://github.com/ClangBuiltLinux/linux/issues/517 Suggested-by: Peter Smith <peter.smith@linaro.org> Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-12 01:43:31 +07:00
CLANG_FLAGS += -Werror=unknown-warning-option
KBUILD_CFLAGS += $(CLANG_FLAGS)
KBUILD_AFLAGS += $(CLANG_FLAGS)
export CLANG_FLAGS
endif
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
# The expansion should be delayed until arch/$(SRCARCH)/Makefile is included.
# Some architectures define CROSS_COMPILE in arch/$(SRCARCH)/Makefile.
# CC_VERSION_TEXT is referenced from Kconfig (so it needs export),
# and from include/config/auto.conf.cmd to detect the compiler upgrade.
CC_VERSION_TEXT = $(shell $(CC) --version 2>/dev/null | head -n 1)
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
ifdef config-build
# ===========================================================================
# *config targets only - make sure prerequisites are updated, and descend
# in scripts/kconfig to make the *config target
# Read arch specific Makefile to set KBUILD_DEFCONFIG as needed.
# KBUILD_DEFCONFIG may point out an alternative default configuration
# used for 'make defconfig'
include arch/$(SRCARCH)/Makefile
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
export KBUILD_DEFCONFIG KBUILD_KCONFIG CC_VERSION_TEXT
config: outputmakefile scripts_basic FORCE
$(Q)$(MAKE) $(build)=scripts/kconfig $@
%config: outputmakefile scripts_basic FORCE
$(Q)$(MAKE) $(build)=scripts/kconfig $@
else #!config-build
# ===========================================================================
# Build targets only - this includes vmlinux, arch specific targets, clean
# targets and others. In general all targets except *config targets.
# If building an external module we do not care about the all: rule
# but instead __all depend on modules
PHONY += all
ifeq ($(KBUILD_EXTMOD),)
__all: all
else
__all: modules
endif
# Decide whether to build built-in, modular, or both.
# Normally, just do built-in.
KBUILD_MODULES :=
KBUILD_BUILTIN := 1
# If we have only "make modules", don't compile built-in objects.
ifeq ($(MAKECMDGOALS),modules)
KBUILD_BUILTIN :=
endif
# If we have "make <whatever> modules", compile modules
# in addition to whatever we do anyway.
# Just "make" or "make all" shall build modules as well
Makefile: Add clang-tidy and static analyzer support to makefile This patch adds clang-tidy and the clang static-analyzer as make targets. The goal of this patch is to make static analysis tools usable and extendable by any developer or researcher who is familiar with basic c++. The current static analysis tools require intimate knowledge of the internal workings of the static analysis. Clang-tidy and the clang static analyzers expose an easy to use api and allow users unfamiliar with clang to write new checks with relative ease. ===Clang-tidy=== Clang-tidy is an easily extendable 'linter' that runs on the AST. Clang-tidy checks are easy to write and understand. A check consists of two parts, a matcher and a checker. The matcher is created using a domain specific language that acts on the AST (https://clang.llvm.org/docs/LibASTMatchersReference.html). When AST nodes are found by the matcher a callback is made to the checker. The checker can then execute additional checks and issue warnings. Here is an example clang-tidy check to report functions that have calls to local_irq_disable without calls to local_irq_enable and vice-versa. Functions flagged with __attribute((annotation("ignore_irq_balancing"))) are ignored for analysis. (https://reviews.llvm.org/D65828) ===Clang static analyzer=== The clang static analyzer is a more powerful static analysis tool that uses symbolic execution to find bugs. Currently there is a check that looks for potential security bugs from invalid uses of kmalloc and kfree. There are several more general purpose checks that are useful for the kernel. The clang static analyzer is well documented and designed to be extensible. (https://clang-analyzer.llvm.org/checker_dev_manual.html) (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf) The main draw of the clang tools is how accessible they are. The clang documentation is very nice and these tools are built specifically to be easily extendable by any developer. They provide an accessible method of bug-finding and research to people who are not overly familiar with the kernel codebase. Signed-off-by: Nathan Huckleberry <nhuck@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-22 21:56:18 +07:00
ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),)
KBUILD_MODULES := 1
endif
ifeq ($(MAKECMDGOALS),)
KBUILD_MODULES := 1
endif
export KBUILD_MODULES KBUILD_BUILTIN
ifdef need-config
include include/config/auto.conf
endif
ifeq ($(KBUILD_EXTMOD),)
# Objects we will link into vmlinux / subdirs we need to visit
core-y := init/ usr/
drivers-y := drivers/ sound/
drivers-$(CONFIG_SAMPLES) += samples/
drivers-y += net/ virt/
libs-y := lib/
endif # KBUILD_EXTMOD
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
# The all: target is the default when no target is given on the
# command line.
# This allow a user to issue only 'make' to build a kernel including modules
# Defaults to vmlinux, but the arch makefile usually adds further targets
all: vmlinux
CFLAGS_GCOV := -fprofile-arcs -ftest-coverage \
$(call cc-option,-fno-tree-loop-im) \
$(call cc-disable-warning,maybe-uninitialized,)
export CFLAGS_GCOV
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
tracing/Makefile: Fix handling redefinition of CC_FLAGS_FTRACE As a Kernel developer, I make heavy use of "make targz-pkg" in order to locally compile and remotely install my development Kernels. The nice feature I rely on is that after a normal "make", "make targz-pkg" only generates the tarball without having to recompile everything. That was true until commit f28bc3c32c05 ("tracing: Handle CC_FLAGS_FTRACE more accurately"). After it, running "make targz-pkg" after "make" will recompile the whole Kernel tree, making my development workflow much slower. The Kernel is choosing to recompile everything because it claims the command line has changed. A diff of the .cmd files show a repeated -mfentry in one of the files. That is because "make targz-pkg" calls "make modules_install" and the environment is already populated with the exported variables, CC_FLAGS_FTRACE being one of them. Then, -mfentry gets duplicated because it is not protected behind an ifndef block, like -pg. To complicate the problem a little bit more, architectures can define their own version CC_FLAGS_FTRACE, so our code not only has to consider recursive Makefiles, but also architecture overrides. So in this patch we move CC_FLAGS_FTRACE up and unconditionally define it to -pg. Then we let the architecture Makefiles possibly override it, and finally append the extra options later. This ensures the variable is always fully redefined at each invocation so recursive Makefiles don't keep appending, and hopefully it maintains the intended behavior on how architectures can override the defaults.. Thanks Steven Rostedt and Vasily Gorbik for the help on this regression. Cc: Michal Marek <michal.lkml@markovi.net> Cc: Ingo Molnar <mingo@redhat.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: linux-kbuild@vger.kernel.org Fixes: commit f28bc3c32c05 ("tracing: Handle CC_FLAGS_FTRACE more accurately") Acked-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-09-11 00:59:56 +07:00
# The arch Makefiles can override CC_FLAGS_FTRACE. We may also append it later.
ifdef CONFIG_FUNCTION_TRACER
CC_FLAGS_FTRACE := -pg
endif
RETPOLINE_CFLAGS_GCC := -mindirect-branch=thunk-extern -mindirect-branch-register
RETPOLINE_VDSO_CFLAGS_GCC := -mindirect-branch=thunk-inline -mindirect-branch-register
RETPOLINE_CFLAGS_CLANG := -mretpoline-external-thunk
RETPOLINE_VDSO_CFLAGS_CLANG := -mretpoline
RETPOLINE_CFLAGS := $(call cc-option,$(RETPOLINE_CFLAGS_GCC),$(call cc-option,$(RETPOLINE_CFLAGS_CLANG)))
RETPOLINE_VDSO_CFLAGS := $(call cc-option,$(RETPOLINE_VDSO_CFLAGS_GCC),$(call cc-option,$(RETPOLINE_VDSO_CFLAGS_CLANG)))
export RETPOLINE_CFLAGS
export RETPOLINE_VDSO_CFLAGS
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
include arch/$(SRCARCH)/Makefile
ifdef need-config
ifdef may-sync-config
kbuild: fix endless syncconfig in case arch Makefile sets CROSS_COMPILE Commit 21c54b774744 ("kconfig: show compiler version text in the top comment") was intended to detect the compiler upgrade, but Geert reported a breakage on the m68k build. The compiler upgrade is detected by the change of the environment variable, CC_VERSION_TEXT, which contains the first line of the output from $(CC) --version. Currently, this works well when CROSS_COMPILE is given via the environment variable or the Make command line. However, some architectures such as m68k can specify CROSS_COMPILE from arch/$(SRCARCH)/Makefile as well. In this case, "make ARCH=m68k" ends up with endless syncconfig loop. $ make ARCH=m68k defconfig *** Default configuration is based on 'multi_defconfig' # # configuration written to .config # $ make ARCH=m68k scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig scripts/kconfig/conf --syncconfig Kconfig Things are happening like this: Because arch/$(SRCARCH)/Makefile is included after CC_VERSION_TEXT is set, it contains the host compiler version in the defconfig phase. To create or update auto.conf, the following line is triggered: include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig This recurses the top Makefile after arch/$(SRCARCH)/Makefile is included. CROSS_COMPILE is set to a m68k toolchain prefix and exported to the recursed Make. Then, syncconfig is invoked with the target compiler version in CC_VERSION_TEXT. The Make will restart because auto.conf and auto.conf.cmd have been updated. At this point, CROSS_COMPILE is reset, so CC_VERSION_TEXT is set to the host compiler version again. Then, syncconfig is triggered due to the change of CC_VERSION_TEXT. This loop continues eternally. To fix this problem, $(CC_VERSION_TEXT) must be evaluated only after arch/$(SRCARCH)/Makefile. Setting it earlier is OK as long as it is defined by using the '=' operator instead of ':='. For the defconfig phase, $(CC_VERSION_TEXT) is evaluated when Kbuild descends into scripts/kconfig/, so it contains the target compiler version correctly. include/config/auto.conf.cmd references $(CC_VERSION_TEXT) as well, so it must be included after arch/$(SRCARCH)/Makefile. Fixes: 21c54b774744 ("kconfig: show compiler version text in the top comment") Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2018-06-08 07:21:43 +07:00
# Read in dependencies to all Kconfig* files, make sure to run syncconfig if
# changes are detected. This should be included after arch/$(SRCARCH)/Makefile
# because some architectures define CROSS_COMPILE there.
kbuild: turn auto.conf.cmd into a mandatory include file syncconfig is responsible for keeping auto.conf up-to-date, so if it fails for any reason, the build must be terminated immediately. However, since commit 9390dff66a52 ("kbuild: invoke syncconfig if include/config/auto.conf.cmd is missing"), Kbuild continues running even after syncconfig fails. You can confirm this by intentionally making syncconfig error out: diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c index 08ba146..307b9de 100644 --- a/scripts/kconfig/confdata.c +++ b/scripts/kconfig/confdata.c @@ -1023,6 +1023,9 @@ int conf_write_autoconf(int overwrite) FILE *out, *tristate, *out_h; int i; + if (overwrite) + return 1; + if (!overwrite && is_present(autoconf_name)) return 0; Then, syncconfig fails, but Make would not stop: $ make -s mrproper allyesconfig defconfig $ make scripts/kconfig/conf --syncconfig Kconfig *** Error during sync of the configuration. make[2]: *** [scripts/kconfig/Makefile;69: syncconfig] Error 1 make[1]: *** [Makefile;557: syncconfig] Error 2 make: *** [include/config/auto.conf.cmd] Deleting file 'include/config/tristate.conf' make: Failed to remake makefile 'include/config/auto.conf'. SYSTBL arch/x86/include/generated/asm/syscalls_32.h SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h SYSTBL arch/x86/include/generated/asm/syscalls_64.h [ continue running ... ] The reason is in the behavior of a pattern rule with multi-targets. %/auto.conf %/auto.conf.cmd %/tristate.conf: $(KCONFIG_CONFIG) $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig GNU Make knows this rule is responsible for making all the three files simultaneously. As far as examined, auto.conf.cmd is the target in question when this rule is invoked. It is probably because auto.conf.cmd is included below the inclusion of auto.conf. The inclusion of auto.conf is mandatory, while that of auto.conf.cmd is optional. GNU Make does not care about the failure in the process of updating optional include files. I filed this issue (https://savannah.gnu.org/bugs/?56301) in case this behavior could be improved somehow in future releases of GNU Make. Anyway, it is quite easy to fix our Makefile. Given that auto.conf is already a mandatory include file, there is no reason to stick auto.conf.cmd optional. Make it mandatory as well. Cc: linux-stable <stable@vger.kernel.org> # 5.0+ Fixes: 9390dff66a52 ("kbuild: invoke syncconfig if include/config/auto.conf.cmd is missing") Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-05-12 09:13:48 +07:00
include include/config/auto.conf.cmd
$(KCONFIG_CONFIG):
@echo >&2 '***'
@echo >&2 '*** Configuration file "$@" not found!'
@echo >&2 '***'
@echo >&2 '*** Please run some configurator (e.g. "make oldconfig" or'
@echo >&2 '*** "make menuconfig" or "make xconfig").'
@echo >&2 '***'
@/bin/false
2018-02-13 14:58:20 +07:00
# The actual configuration files used during the build are stored in
# include/generated/ and include/config/. Update them if .config is newer than
# include/config/auto.conf (which mirrors .config).
kbuild: invoke syncconfig if include/config/auto.conf.cmd is missing If include/config/auto.conf.cmd is lost for some reasons, it is not self-healing, so the top Makefile misses to run syncconfig. Move include/config/auto.conf.cmd to the target side. I used a pattern rule instead of a normal rule here although it is a bit gross. If the rule were written with a normal rule like this, include/config/auto.conf \ include/config/auto.conf.cmd \ include/config/tristate.conf: $(KCONFIG_CONFIG) $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig ... syncconfig would be executed per target. Using a pattern rule makes sure that syncconfig is executed just once because Make assumes the recipe will create all of the targets. Here is a quote from the GNU Make manual [1]: "Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule's recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. When searching for a pattern rule to match a target, the target patterns of a rule other than the one that matches the target in need of a rule are incidental: make worries only about giving a recipe and prerequisites to the file presently in question. However, when this file's recipe is run, the other targets are marked as having been updated themselves." [1]: https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-22 14:40:10 +07:00
#
# This exploits the 'multi-target pattern rule' trick.
# The syncconfig should be executed only once to make all the targets.
# (Note: use the grouped target '&:' when we bump to GNU Make 4.3)
quiet_cmd_syncconfig = SYNC $@
cmd_syncconfig = $(MAKE) -f $(srctree)/Makefile syncconfig
%/config/auto.conf %/config/auto.conf.cmd %/generated/autoconf.h: $(KCONFIG_CONFIG)
+$(call cmd,syncconfig)
else # !may-sync-config
kbuild: do not update config when running install targets "make syncconfig" is automatically invoked when any of the following happens: - .config is updated - any of Kconfig files is updated - any of environment variables referenced in Kconfig is changed Then, it updates configuration files such as include/config/auto.conf include/generated/autoconf.h, etc. Even install targets (install, modules_install, etc.) are no exception. However, they should never ever modify the source tree. Install targets are often run with root privileges. Once those configuration files are owned by root, "make mrproper" would end up with permission error. Install targets should just copy things blindly. They should not care whether the configuration is up-to-date or not. This makes more sense because we are interested in the configuration that was used in the previous kernel building. This issue has existed since before, but rarely happened. I expect more chance where people are hit by this; with the new Kconfig syntax extension, the .config now contains the compiler information. If you cross-compile the kernel with CROSS_COMPILE, but forget to pass it for "make install", you meet "any of environment variables referenced in Kconfig is changed" because $(CC) is referenced in Kconfig. Another scenario is the compiler upgrade before the installation. Install targets need the configuration. "make modules_install" refer to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_* to decide which dtb files to install. However, the auto-update of the configuration files should be avoided. We already do this for external modules. Now, Make targets are categorized into 3 groups: [1] Do not need the kernel configuration at all help, coccicheck, headers_install etc. [2] Need the latest kernel configuration If new config options are added, Kconfig will show prompt to ask user's selection. Build targets such as vmlinux, in-kernel modules are the cases. [3] Need the kernel configuration, but do not want to update it Install targets except headers_install, and external modules are the cases. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-20 14:46:34 +07:00
# External modules and some install targets need include/generated/autoconf.h
# and include/config/auto.conf but do not care if they are up-to-date.
# Use auto.conf to trigger the test
PHONY += include/config/auto.conf
include/config/auto.conf:
$(Q)test -e include/generated/autoconf.h -a -e $@ || ( \
echo >&2; \
echo >&2 " ERROR: Kernel configuration is invalid."; \
echo >&2 " include/generated/autoconf.h or $@ are missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it."; \
echo >&2 ; \
/bin/false)
kbuild: do not update config when running install targets "make syncconfig" is automatically invoked when any of the following happens: - .config is updated - any of Kconfig files is updated - any of environment variables referenced in Kconfig is changed Then, it updates configuration files such as include/config/auto.conf include/generated/autoconf.h, etc. Even install targets (install, modules_install, etc.) are no exception. However, they should never ever modify the source tree. Install targets are often run with root privileges. Once those configuration files are owned by root, "make mrproper" would end up with permission error. Install targets should just copy things blindly. They should not care whether the configuration is up-to-date or not. This makes more sense because we are interested in the configuration that was used in the previous kernel building. This issue has existed since before, but rarely happened. I expect more chance where people are hit by this; with the new Kconfig syntax extension, the .config now contains the compiler information. If you cross-compile the kernel with CROSS_COMPILE, but forget to pass it for "make install", you meet "any of environment variables referenced in Kconfig is changed" because $(CC) is referenced in Kconfig. Another scenario is the compiler upgrade before the installation. Install targets need the configuration. "make modules_install" refer to CONFIG_MODULES etc. "make dtbs_install" also needs CONFIG_ARCH_* to decide which dtb files to install. However, the auto-update of the configuration files should be avoided. We already do this for external modules. Now, Make targets are categorized into 3 groups: [1] Do not need the kernel configuration at all help, coccicheck, headers_install etc. [2] Need the latest kernel configuration If new config options are added, Kconfig will show prompt to ask user's selection. Build targets such as vmlinux, in-kernel modules are the cases. [3] Need the kernel configuration, but do not want to update it Install targets except headers_install, and external modules are the cases. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-20 14:46:34 +07:00
endif # may-sync-config
endif # need-config
KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)
KBUILD_CFLAGS += $(call cc-disable-warning,frame-address,)
KBUILD_CFLAGS += $(call cc-disable-warning, format-truncation)
KBUILD_CFLAGS += $(call cc-disable-warning, format-overflow)
KBUILD_CFLAGS += $(call cc-disable-warning, address-of-packed-member)
ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
KBUILD_CFLAGS += -O2
else ifdef CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3
KBUILD_CFLAGS += -O3
else ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS += -Os
endif
./Makefile: tell gcc optimizer to never introduce new data races We have been chasing a memory corruption bug, which turned out to be caused by very old gcc (4.3.4), which happily turned conditional load into a non-conditional one, and that broke correctness (the condition was met only if lock was held) and corrupted memory. This particular problem with that particular code did not happen when never gccs were used. I've brought this up with our gcc folks, as I wanted to make sure that this can't really happen again, and it turns out it actually can. Quoting Martin Jambor <mjambor@suse.cz>: "More current GCCs are more careful when it comes to replacing a conditional load with a non-conditional one, most notably they check that a store happens in each iteration of _a_ loop but they assume loops are executed. They also perform a simple check whether the store cannot trap which currently passes only for non-const variables. A simple testcase demonstrating it on an x86_64 is for example the following: $ cat cond_store.c int g_1 = 1; int g_2[1024] __attribute__((section ("safe_section"), aligned (4096))); int c = 4; int __attribute__ ((noinline)) foo (void) { int l; for (l = 0; (l != 4); l++) { if (g_1) return l; for (g_2[0] = 0; (g_2[0] >= 26); ++g_2[0]) ; } return 2; } int main (int argc, char* argv[]) { if (mprotect (g_2, sizeof(g_2), PROT_READ) == -1) { int e = errno; error (e, e, "mprotect error %i", e); } foo (); __builtin_printf("OK\n"); return 0; } /* EOF */ $ ~/gcc/trunk/inst/bin/gcc cond_store.c -O2 --param allow-store-data-races=0 $ ./a.out OK $ ~/gcc/trunk/inst/bin/gcc cond_store.c -O2 --param allow-store-data-races=1 $ ./a.out Segmentation fault The testcase fails the same at least with 4.9, 4.8 and 4.7. Therefore I would suggest building kernels with this parameter set to zero. I also agree with Jikos that the default should be changed for -O2. I have run most of the SPEC 2k6 CPU benchmarks (gamess and dealII failed, at -O2, not sure why) compiled with and without this option and did not see any real difference between respective run-times" Hopefully the default will be changed in newer gccs, but let's force it for kernel builds so that we are on a safe side even when older gcc are used. The code in question was out-of-tree printk-in-NMI (yeah, surprise suprise, once again) patch written by Petr Mladek, let me quote his comment from our internal bugzilla: "I have spent few days investigating inconsistent state of kernel ring buffer. It went out that it was caused by speculative store generated by gcc-4.3.4. The problem is in assembly generated for make_free_space(). The functions is called the following way: + vprintk_emit(); + log = MAIN_LOG; // with logbuf_lock or log = NMI_LOG; // with nmi_logbuf_lock cont_add(log, ...); + cont_flush(log, ...); + log_store(log, ...); + log_make_free_space(log, ...); If called with log = NMI_LOG then only nmi_log_* global variables are safe to modify but the generated code does store also into (main_)log_* global variables: <log_make_free_space>: 55 push %rbp 89 f6 mov %esi,%esi 48 8b 05 03 99 51 01 mov 0x1519903(%rip),%rax # ffffffff82620868 <nmi_log_next_id> 44 8b 1d ec 98 51 01 mov 0x15198ec(%rip),%r11d # ffffffff82620858 <log_next_idx> 8b 35 36 60 14 01 mov 0x1146036(%rip),%esi # ffffffff8224cfa8 <log_buf_len> 44 8b 35 33 60 14 01 mov 0x1146033(%rip),%r14d # ffffffff8224cfac <nmi_log_buf_len> 4c 8b 2d d0 98 51 01 mov 0x15198d0(%rip),%r13 # ffffffff82620850 <log_next_seq> 4c 8b 25 11 61 14 01 mov 0x1146111(%rip),%r12 # ffffffff8224d098 <log_buf> 49 89 c2 mov %rax,%r10 48 21 c2 and %rax,%rdx 48 8b 1d 0c 99 55 01 mov 0x155990c(%rip),%rbx # ffffffff826608a0 <nmi_log_buf> 49 c1 ea 20 shr $0x20,%r10 48 89 55 d0 mov %rdx,-0x30(%rbp) 44 29 de sub %r11d,%esi 45 29 d6 sub %r10d,%r14d 4c 8b 0d 97 98 51 01 mov 0x1519897(%rip),%r9 # ffffffff82620840 <log_first_seq> eb 7e jmp ffffffff81107029 <log_make_free_space+0xe9> [...] 85 ff test %edi,%edi # edi = 1 for NMI_LOG 4c 89 e8 mov %r13,%rax 4c 89 ca mov %r9,%rdx 74 0a je ffffffff8110703d <log_make_free_space+0xfd> 8b 15 27 98 51 01 mov 0x1519827(%rip),%edx # ffffffff82620860 <nmi_log_first_id> 48 8b 45 d0 mov -0x30(%rbp),%rax 48 39 c2 cmp %rax,%rdx # end of loop 0f 84 da 00 00 00 je ffffffff81107120 <log_make_free_space+0x1e0> [...] 85 ff test %edi,%edi # edi = 1 for NMI_LOG 4c 89 0d 17 97 51 01 mov %r9,0x1519717(%rip) # ffffffff82620840 <log_first_seq> ^^^^^^^^^^^^^^^^^^^^^^^^^^ KABOOOM 74 35 je ffffffff81107160 <log_make_free_space+0x220> It stores log_first_seq when edi == NMI_LOG. This instructions are used also when edi == MAIN_LOG but the store is done speculatively before the condition is decided. It is unsafe because we do not have "logbuf_lock" in NMI context and some other process migh modify "log_first_seq" in parallel" I believe that the best course of action is both - building kernel (and anything multi-threaded, I guess) with that optimization turned off - persuade gcc folks to change the default for future releases Signed-off-by: Jiri Kosina <jkosina@suse.cz> Cc: Martin Jambor <mjambor@suse.cz> Cc: Petr Mladek <pmladek@suse.cz> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Marek Polacek <polacek@redhat.com> Cc: Jakub Jelinek <jakub@redhat.com> Cc: Steven Noonan <steven@uplinklabs.net> Cc: Richard Biener <richard.guenther@gmail.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-08-07 06:08:43 +07:00
# Tell gcc to never replace conditional load with a non-conditional one
KBUILD_CFLAGS += $(call cc-option,--param=allow-store-data-races=0)
KBUILD_CFLAGS += $(call cc-option,-fno-allow-store-data-races)
./Makefile: tell gcc optimizer to never introduce new data races We have been chasing a memory corruption bug, which turned out to be caused by very old gcc (4.3.4), which happily turned conditional load into a non-conditional one, and that broke correctness (the condition was met only if lock was held) and corrupted memory. This particular problem with that particular code did not happen when never gccs were used. I've brought this up with our gcc folks, as I wanted to make sure that this can't really happen again, and it turns out it actually can. Quoting Martin Jambor <mjambor@suse.cz>: "More current GCCs are more careful when it comes to replacing a conditional load with a non-conditional one, most notably they check that a store happens in each iteration of _a_ loop but they assume loops are executed. They also perform a simple check whether the store cannot trap which currently passes only for non-const variables. A simple testcase demonstrating it on an x86_64 is for example the following: $ cat cond_store.c int g_1 = 1; int g_2[1024] __attribute__((section ("safe_section"), aligned (4096))); int c = 4; int __attribute__ ((noinline)) foo (void) { int l; for (l = 0; (l != 4); l++) { if (g_1) return l; for (g_2[0] = 0; (g_2[0] >= 26); ++g_2[0]) ; } return 2; } int main (int argc, char* argv[]) { if (mprotect (g_2, sizeof(g_2), PROT_READ) == -1) { int e = errno; error (e, e, "mprotect error %i", e); } foo (); __builtin_printf("OK\n"); return 0; } /* EOF */ $ ~/gcc/trunk/inst/bin/gcc cond_store.c -O2 --param allow-store-data-races=0 $ ./a.out OK $ ~/gcc/trunk/inst/bin/gcc cond_store.c -O2 --param allow-store-data-races=1 $ ./a.out Segmentation fault The testcase fails the same at least with 4.9, 4.8 and 4.7. Therefore I would suggest building kernels with this parameter set to zero. I also agree with Jikos that the default should be changed for -O2. I have run most of the SPEC 2k6 CPU benchmarks (gamess and dealII failed, at -O2, not sure why) compiled with and without this option and did not see any real difference between respective run-times" Hopefully the default will be changed in newer gccs, but let's force it for kernel builds so that we are on a safe side even when older gcc are used. The code in question was out-of-tree printk-in-NMI (yeah, surprise suprise, once again) patch written by Petr Mladek, let me quote his comment from our internal bugzilla: "I have spent few days investigating inconsistent state of kernel ring buffer. It went out that it was caused by speculative store generated by gcc-4.3.4. The problem is in assembly generated for make_free_space(). The functions is called the following way: + vprintk_emit(); + log = MAIN_LOG; // with logbuf_lock or log = NMI_LOG; // with nmi_logbuf_lock cont_add(log, ...); + cont_flush(log, ...); + log_store(log, ...); + log_make_free_space(log, ...); If called with log = NMI_LOG then only nmi_log_* global variables are safe to modify but the generated code does store also into (main_)log_* global variables: <log_make_free_space>: 55 push %rbp 89 f6 mov %esi,%esi 48 8b 05 03 99 51 01 mov 0x1519903(%rip),%rax # ffffffff82620868 <nmi_log_next_id> 44 8b 1d ec 98 51 01 mov 0x15198ec(%rip),%r11d # ffffffff82620858 <log_next_idx> 8b 35 36 60 14 01 mov 0x1146036(%rip),%esi # ffffffff8224cfa8 <log_buf_len> 44 8b 35 33 60 14 01 mov 0x1146033(%rip),%r14d # ffffffff8224cfac <nmi_log_buf_len> 4c 8b 2d d0 98 51 01 mov 0x15198d0(%rip),%r13 # ffffffff82620850 <log_next_seq> 4c 8b 25 11 61 14 01 mov 0x1146111(%rip),%r12 # ffffffff8224d098 <log_buf> 49 89 c2 mov %rax,%r10 48 21 c2 and %rax,%rdx 48 8b 1d 0c 99 55 01 mov 0x155990c(%rip),%rbx # ffffffff826608a0 <nmi_log_buf> 49 c1 ea 20 shr $0x20,%r10 48 89 55 d0 mov %rdx,-0x30(%rbp) 44 29 de sub %r11d,%esi 45 29 d6 sub %r10d,%r14d 4c 8b 0d 97 98 51 01 mov 0x1519897(%rip),%r9 # ffffffff82620840 <log_first_seq> eb 7e jmp ffffffff81107029 <log_make_free_space+0xe9> [...] 85 ff test %edi,%edi # edi = 1 for NMI_LOG 4c 89 e8 mov %r13,%rax 4c 89 ca mov %r9,%rdx 74 0a je ffffffff8110703d <log_make_free_space+0xfd> 8b 15 27 98 51 01 mov 0x1519827(%rip),%edx # ffffffff82620860 <nmi_log_first_id> 48 8b 45 d0 mov -0x30(%rbp),%rax 48 39 c2 cmp %rax,%rdx # end of loop 0f 84 da 00 00 00 je ffffffff81107120 <log_make_free_space+0x1e0> [...] 85 ff test %edi,%edi # edi = 1 for NMI_LOG 4c 89 0d 17 97 51 01 mov %r9,0x1519717(%rip) # ffffffff82620840 <log_first_seq> ^^^^^^^^^^^^^^^^^^^^^^^^^^ KABOOOM 74 35 je ffffffff81107160 <log_make_free_space+0x220> It stores log_first_seq when edi == NMI_LOG. This instructions are used also when edi == MAIN_LOG but the store is done speculatively before the condition is decided. It is unsafe because we do not have "logbuf_lock" in NMI context and some other process migh modify "log_first_seq" in parallel" I believe that the best course of action is both - building kernel (and anything multi-threaded, I guess) with that optimization turned off - persuade gcc folks to change the default for future releases Signed-off-by: Jiri Kosina <jkosina@suse.cz> Cc: Martin Jambor <mjambor@suse.cz> Cc: Petr Mladek <pmladek@suse.cz> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Marek Polacek <polacek@redhat.com> Cc: Jakub Jelinek <jakub@redhat.com> Cc: Steven Noonan <steven@uplinklabs.net> Cc: Richard Biener <richard.guenther@gmail.com> Cc: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-08-07 06:08:43 +07:00
ifdef CONFIG_READABLE_ASM
# Disable optimizations that make assembler listings hard to read.
# reorder blocks reorders the control in the function
# ipa clone creates specialized cloned functions
# partial inlining inlines only parts of functions
KBUILD_CFLAGS += $(call cc-option,-fno-reorder-blocks,) \
$(call cc-option,-fno-ipa-cp-clone,) \
$(call cc-option,-fno-partial-inlining)
endif
ifneq ($(CONFIG_FRAME_WARN),0)
KBUILD_CFLAGS += -Wframe-larger-than=$(CONFIG_FRAME_WARN)
endif
stackp-flags-y := -fno-stack-protector
Kbuild: rename CC_STACKPROTECTOR[_STRONG] config variables The changes to automatically test for working stack protector compiler support in the Kconfig files removed the special STACKPROTECTOR_AUTO option that picked the strongest stack protector that the compiler supported. That was all a nice cleanup - it makes no sense to have the AUTO case now that the Kconfig phase can just determine the compiler support directly. HOWEVER. It also meant that doing "make oldconfig" would now _disable_ the strong stackprotector if you had AUTO enabled, because in a legacy config file, the sane stack protector configuration would look like CONFIG_HAVE_CC_STACKPROTECTOR=y # CONFIG_CC_STACKPROTECTOR_NONE is not set # CONFIG_CC_STACKPROTECTOR_REGULAR is not set # CONFIG_CC_STACKPROTECTOR_STRONG is not set CONFIG_CC_STACKPROTECTOR_AUTO=y and when you ran this through "make oldconfig" with the Kbuild changes, it would ask you about the regular CONFIG_CC_STACKPROTECTOR (that had been renamed from CONFIG_CC_STACKPROTECTOR_REGULAR to just CONFIG_CC_STACKPROTECTOR), but it would think that the STRONG version used to be disabled (because it was really enabled by AUTO), and would disable it in the new config, resulting in: CONFIG_HAVE_CC_STACKPROTECTOR=y CONFIG_CC_HAS_STACKPROTECTOR_NONE=y CONFIG_CC_STACKPROTECTOR=y # CONFIG_CC_STACKPROTECTOR_STRONG is not set CONFIG_CC_HAS_SANE_STACKPROTECTOR=y That's dangerously subtle - people could suddenly find themselves with the weaker stack protector setup without even realizing. The solution here is to just rename not just the old RECULAR stack protector option, but also the strong one. This does that by just removing the CC_ prefix entirely for the user choices, because it really is not about the compiler support (the compiler support now instead automatially impacts _visibility_ of the options to users). This results in "make oldconfig" actually asking the user for their choice, so that we don't have any silent subtle security model changes. The end result would generally look like this: CONFIG_HAVE_CC_STACKPROTECTOR=y CONFIG_CC_HAS_STACKPROTECTOR_NONE=y CONFIG_STACKPROTECTOR=y CONFIG_STACKPROTECTOR_STRONG=y CONFIG_CC_HAS_SANE_STACKPROTECTOR=y where the "CC_" versions really are about internal compiler infrastructure, not the user selections. Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-06-14 10:21:18 +07:00
stackp-flags-$(CONFIG_STACKPROTECTOR) := -fstack-protector
stackp-flags-$(CONFIG_STACKPROTECTOR_STRONG) := -fstack-protector-strong
KBUILD_CFLAGS += $(stackp-flags-y)
ifdef CONFIG_CC_IS_CLANG
KBUILD_CPPFLAGS += -Qunused-arguments
KBUILD_CFLAGS += -Wno-format-invalid-specifier
KBUILD_CFLAGS += -Wno-gnu
# CLANG uses a _MergedGlobals as optimization, but this breaks modpost, as the
# source of a reference will be _MergedGlobals and not on of the whitelisted names.
# See modpost pattern 2
KBUILD_CFLAGS += -mno-global-merge
else
# These warnings generated too much noise in a regular build.
# Use make W=1 to enable them (see scripts/Makefile.extrawarn)
KBUILD_CFLAGS += -Wno-unused-but-set-variable
kbuild: Do not enable -Wimplicit-fallthrough for clang for now This functionally reverts commit bfd77145f35c ("Makefile: Convert -Wimplicit-fallthrough=3 to just -Wimplicit-fallthrough for clang"). clang enabled support for -Wimplicit-fallthrough in C in r369414 [1], which causes a lot of warnings when building the kernel for two reasons: 1. Clang does not support the /* fall through */ comments. There seems to be a general consensus in the LLVM community that this is not something they want to support. Joe Perches wrote a script to convert all of the comments to a "fallthrough" keyword that will be added to compiler_attributes.h [2] [3], which catches the vast majority of the comments. There doesn't appear to be any consensus in the kernel community when to do this conversion. 2. Clang and GCC disagree about falling through to final case statements with no content or cases that simply break: https://godbolt.org/z/c8csDu This difference contributes at least 50 warnings in an allyesconfig build for x86, not considering other architectures. This difference will need to be discussed to see which compiler is right [4] [5]. [1]: https://github.com/llvm/llvm-project/commit/1e0affb6e564b7361b0aadb38805f26deff4ecfc [2]: https://lore.kernel.org/lkml/61ddbb86d5e68a15e24ccb06d9b399bbf5ce2da7.camel@perches.com/ [3]: https://lore.kernel.org/lkml/1d2830aadbe9d8151728a7df5b88528fc72a0095.1564549413.git.joe@perches.com/ [4]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91432 [5]: https://github.com/ClangBuiltLinux/linux/issues/636 Given these two problems need discussion and coordination, do not enable -Wimplicit-fallthrough with clang right now. Add a comment to explain what is going on as well. This commit should be reverted once these two issues are fully flushed out and resolved. Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Nathan Chancellor <natechancellor@gmail.com> Acked-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> Acked-by: Nick Desaulniers <ndesaulniers@google.com> Acked-by: Gustavo A. R. Silva <gustavo@embeddedor.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-27 07:41:55 +07:00
# Warn about unmarked fall-throughs in switch statement.
# Disabled for clang while comment to attribute conversion happens and
# https://github.com/ClangBuiltLinux/linux/issues/636 is discussed.
KBUILD_CFLAGS += $(call cc-option,-Wimplicit-fallthrough,)
endif
KBUILD_CFLAGS += $(call cc-disable-warning, unused-const-variable)
ifdef CONFIG_FRAME_POINTER
KBUILD_CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls
else
# Some targets (ARM with Thumb2, for example), can't be built with frame
# pointers. For those, we don't have FUNCTION_TRACER automatically
# select FRAME_POINTER. However, FUNCTION_TRACER adds -pg, and this is
# incompatible with -fomit-frame-pointer with current GCC, so we don't use
# -fomit-frame-pointer with FUNCTION_TRACER.
ifndef CONFIG_FUNCTION_TRACER
KBUILD_CFLAGS += -fomit-frame-pointer
endif
endif
security: allow using Clang's zero initialization for stack variables In addition to -ftrivial-auto-var-init=pattern (used by CONFIG_INIT_STACK_ALL now) Clang also supports zero initialization for locals enabled by -ftrivial-auto-var-init=zero. The future of this flag is still being debated (see https://bugs.llvm.org/show_bug.cgi?id=45497). Right now it is guarded by another flag, -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang, which means it may not be supported by future Clang releases. Another possible resolution is that -ftrivial-auto-var-init=zero will persist (as certain users have already started depending on it), but the name of the guard flag will change. In the meantime, zero initialization has proven itself as a good production mitigation measure against uninitialized locals. Unlike pattern initialization, which has a higher chance of triggering existing bugs, zero initialization provides safe defaults for strings, pointers, indexes, and sizes. On the other hand, pattern initialization remains safer for return values. Chrome OS and Android are moving to using zero initialization for production builds. Performance-wise, the difference between pattern and zero initialization is usually negligible, although the generated code for zero initialization is more compact. This patch renames CONFIG_INIT_STACK_ALL to CONFIG_INIT_STACK_ALL_PATTERN and introduces another config option, CONFIG_INIT_STACK_ALL_ZERO, that enables zero initialization for locals if the corresponding flags are supported by Clang. Cc: Kees Cook <keescook@chromium.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Alexander Potapenko <glider@google.com> Link: https://lore.kernel.org/r/20200616083435.223038-1-glider@google.com Reviewed-by: Maciej Żenczykowski <maze@google.com> Signed-off-by: Kees Cook <keescook@chromium.org>
2020-06-16 15:34:35 +07:00
# Initialize all stack variables with a 0xAA pattern.
ifdef CONFIG_INIT_STACK_ALL_PATTERN
KBUILD_CFLAGS += -ftrivial-auto-var-init=pattern
endif
security: allow using Clang's zero initialization for stack variables In addition to -ftrivial-auto-var-init=pattern (used by CONFIG_INIT_STACK_ALL now) Clang also supports zero initialization for locals enabled by -ftrivial-auto-var-init=zero. The future of this flag is still being debated (see https://bugs.llvm.org/show_bug.cgi?id=45497). Right now it is guarded by another flag, -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang, which means it may not be supported by future Clang releases. Another possible resolution is that -ftrivial-auto-var-init=zero will persist (as certain users have already started depending on it), but the name of the guard flag will change. In the meantime, zero initialization has proven itself as a good production mitigation measure against uninitialized locals. Unlike pattern initialization, which has a higher chance of triggering existing bugs, zero initialization provides safe defaults for strings, pointers, indexes, and sizes. On the other hand, pattern initialization remains safer for return values. Chrome OS and Android are moving to using zero initialization for production builds. Performance-wise, the difference between pattern and zero initialization is usually negligible, although the generated code for zero initialization is more compact. This patch renames CONFIG_INIT_STACK_ALL to CONFIG_INIT_STACK_ALL_PATTERN and introduces another config option, CONFIG_INIT_STACK_ALL_ZERO, that enables zero initialization for locals if the corresponding flags are supported by Clang. Cc: Kees Cook <keescook@chromium.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Alexander Potapenko <glider@google.com> Link: https://lore.kernel.org/r/20200616083435.223038-1-glider@google.com Reviewed-by: Maciej Żenczykowski <maze@google.com> Signed-off-by: Kees Cook <keescook@chromium.org>
2020-06-16 15:34:35 +07:00
# Initialize all stack variables with a zero value.
ifdef CONFIG_INIT_STACK_ALL_ZERO
# Future support for zero initialization is still being debated, see
# https://bugs.llvm.org/show_bug.cgi?id=45497. These flags are subject to being
# renamed or dropped.
KBUILD_CFLAGS += -ftrivial-auto-var-init=zero
KBUILD_CFLAGS += -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
endif
# Workaround for GCC versions < 5.0
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61801
ifdef CONFIG_CC_IS_GCC
DEBUG_CFLAGS := $(call cc-ifversion, -lt, 0500, $(call cc-option, -fno-var-tracking-assignments))
endif
Fix gcc-4.9.0 miscompilation of load_balance() in scheduler Michel Dänzer and a couple of other people reported inexplicable random oopses in the scheduler, and the cause turns out to be gcc mis-compiling the load_balance() function when debugging is enabled. The gcc bug apparently goes back to gcc-4.5, but slight optimization changes means that it now showed up as a problem in 4.9.0 and 4.9.1. The instruction scheduling problem causes gcc to schedule a spill operation to before the stack frame has been created, which in turn can corrupt the spilled value if an interrupt comes in. There may be other effects of this bug too, but that's the code generation problem seen in Michel's case. This is fixed in current gcc HEAD, but the workaround as suggested by Markus Trippelsdorf is pretty simple: use -fno-var-tracking-assignments when compiling the kernel, which disables the gcc code that causes the problem. This can result in slightly worse debug information for variable accesses, but that is infinitely preferable to actual code generation problems. Doing this unconditionally (not just for CONFIG_DEBUG_INFO) also allows non-debug builds to verify that the debug build would be identical: we can do export GCC_COMPARE_DEBUG=1 to make gcc internally verify that the result of the build is independent of the "-g" flag (it will make the compiler build everything twice, toggling the debug flag, and compare the results). Without the "-fno-var-tracking-assignments" option, the build would fail (even with 4.8.3 that didn't show the actual stack frame bug) with a gcc compare failure. See also gcc bugzilla: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61801 Reported-by: Michel Dänzer <michel@daenzer.net> Suggested-by: Markus Trippelsdorf <markus@trippelsdorf.de> Cc: Jakub Jelinek <jakub@redhat.com> Cc: stable@kernel.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-07-27 04:52:01 +07:00
ifdef CONFIG_DEBUG_INFO
kbuild: Support split debug info v4 This is an alternative approach to lower the overhead of debug info (as we discussed a few days ago) gcc 4.7+ and newer binutils have a new "split debug info" debug info model where the debug info is only written once into central ".dwo" files. This avoids having to copy it around multiple times, from the object files to the final executable. It lowers the disk space requirements. In addition it defaults to compressed debug data. More details here: http://gcc.gnu.org/wiki/DebugFission This patch adds a new option to enable it. It has to be an option, because it'll undoubtedly break everyone's debuginfo packaging scheme. gdb/objdump/etc. all still work, if you have new enough versions. I don't see big compile wins (maybe a second or two faster or so), but the object dirs with debuginfo get significantly smaller. My standard kernel config (slightly bigger than defconfig) shrinks from 2.9G disk space to 1.1G objdir (with non reduced debuginfo). I presume if you are IO limited the compile time difference will be larger. Only problem I've seen so far is that it doesn't play well with older versions of ccache (apparently fixed, see https://bugzilla.samba.org/show_bug.cgi?id=10005) v2: various fixes from Dirk Gouders. Improve commit message slightly. v3: Fix clean rules and improve Kconfig slightly v4: Fix merge error in last version (Sam Ravnborg) Clarify description that it mainly helps disk size. Cc: Dirk Gouders <dirk@gouders.net> Signed-off-by: Andi Kleen <ak@linux.intel.com> Acked-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Michal Marek <mmarek@suse.cz>
2014-07-31 01:50:18 +07:00
ifdef CONFIG_DEBUG_INFO_SPLIT
DEBUG_CFLAGS += -gsplit-dwarf
kbuild: Support split debug info v4 This is an alternative approach to lower the overhead of debug info (as we discussed a few days ago) gcc 4.7+ and newer binutils have a new "split debug info" debug info model where the debug info is only written once into central ".dwo" files. This avoids having to copy it around multiple times, from the object files to the final executable. It lowers the disk space requirements. In addition it defaults to compressed debug data. More details here: http://gcc.gnu.org/wiki/DebugFission This patch adds a new option to enable it. It has to be an option, because it'll undoubtedly break everyone's debuginfo packaging scheme. gdb/objdump/etc. all still work, if you have new enough versions. I don't see big compile wins (maybe a second or two faster or so), but the object dirs with debuginfo get significantly smaller. My standard kernel config (slightly bigger than defconfig) shrinks from 2.9G disk space to 1.1G objdir (with non reduced debuginfo). I presume if you are IO limited the compile time difference will be larger. Only problem I've seen so far is that it doesn't play well with older versions of ccache (apparently fixed, see https://bugzilla.samba.org/show_bug.cgi?id=10005) v2: various fixes from Dirk Gouders. Improve commit message slightly. v3: Fix clean rules and improve Kconfig slightly v4: Fix merge error in last version (Sam Ravnborg) Clarify description that it mainly helps disk size. Cc: Dirk Gouders <dirk@gouders.net> Signed-off-by: Andi Kleen <ak@linux.intel.com> Acked-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Michal Marek <mmarek@suse.cz>
2014-07-31 01:50:18 +07:00
else
kbuild: Disable extra debugging info in .s output Modern gcc adds view assignments, reset assertion checking in .loc directives and a couple more additional debug markers, which clutters the asm output unnecessarily: For example: bsp_resume: .LFB3466: .loc 1 1868 1 is_stmt 1 view -0 .cfi_startproc .loc 1 1869 2 view .LVU73 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 14 is_stmt 0 view .LVU74 movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 5 view .LVU75 testq %rax, %rax # _2 je .L8 #, .loc 1 1870 3 is_stmt 1 view .LVU76 movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax or .loc 2 57 9 view .LVU478 .loc 2 57 9 view .LVU479 .loc 2 57 9 view .LVU480 .loc 2 57 9 view .LVU481 .LBB1385: .LBB1383: .LBB1379: .LBB1377: .LBB1375: .loc 2 57 9 view .LVU482 .loc 2 57 9 view .LVU483 movl %edi, %edx # cpu, cpu .LVL87: .loc 2 57 9 is_stmt 0 view .LVU484 That MOV in there is drowned in debugging information and latter makes it hard to follow the asm. And that DWARF info is not really needed for asm output staring. Disable the debug information generation which clutters the asm output unnecessarily: bsp_resume: # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) testq %rax, %rax # _2 je .L8 #, # arch/x86/kernel/cpu/common.c:1870: this_cpu->c_bsp_resume(&boot_cpu_data); movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax .L8: # arch/x86/kernel/cpu/common.c:1871: } rep ret .size bsp_resume, .-bsp_resume [ bp: write commit message. ] Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Borislav Petkov <bp@suse.de>
2019-02-10 13:51:00 +07:00
DEBUG_CFLAGS += -g
kbuild: Support split debug info v4 This is an alternative approach to lower the overhead of debug info (as we discussed a few days ago) gcc 4.7+ and newer binutils have a new "split debug info" debug info model where the debug info is only written once into central ".dwo" files. This avoids having to copy it around multiple times, from the object files to the final executable. It lowers the disk space requirements. In addition it defaults to compressed debug data. More details here: http://gcc.gnu.org/wiki/DebugFission This patch adds a new option to enable it. It has to be an option, because it'll undoubtedly break everyone's debuginfo packaging scheme. gdb/objdump/etc. all still work, if you have new enough versions. I don't see big compile wins (maybe a second or two faster or so), but the object dirs with debuginfo get significantly smaller. My standard kernel config (slightly bigger than defconfig) shrinks from 2.9G disk space to 1.1G objdir (with non reduced debuginfo). I presume if you are IO limited the compile time difference will be larger. Only problem I've seen so far is that it doesn't play well with older versions of ccache (apparently fixed, see https://bugzilla.samba.org/show_bug.cgi?id=10005) v2: various fixes from Dirk Gouders. Improve commit message slightly. v3: Fix clean rules and improve Kconfig slightly v4: Fix merge error in last version (Sam Ravnborg) Clarify description that it mainly helps disk size. Cc: Dirk Gouders <dirk@gouders.net> Signed-off-by: Andi Kleen <ak@linux.intel.com> Acked-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Michal Marek <mmarek@suse.cz>
2014-07-31 01:50:18 +07:00
endif
ifneq ($(LLVM_IAS),1)
KBUILD_AFLAGS += -Wa,-gdwarf-2
endif
ifdef CONFIG_DEBUG_INFO_DWARF4
DEBUG_CFLAGS += -gdwarf-4
endif
ifdef CONFIG_DEBUG_INFO_REDUCED
kbuild: Disable extra debugging info in .s output Modern gcc adds view assignments, reset assertion checking in .loc directives and a couple more additional debug markers, which clutters the asm output unnecessarily: For example: bsp_resume: .LFB3466: .loc 1 1868 1 is_stmt 1 view -0 .cfi_startproc .loc 1 1869 2 view .LVU73 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 14 is_stmt 0 view .LVU74 movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 5 view .LVU75 testq %rax, %rax # _2 je .L8 #, .loc 1 1870 3 is_stmt 1 view .LVU76 movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax or .loc 2 57 9 view .LVU478 .loc 2 57 9 view .LVU479 .loc 2 57 9 view .LVU480 .loc 2 57 9 view .LVU481 .LBB1385: .LBB1383: .LBB1379: .LBB1377: .LBB1375: .loc 2 57 9 view .LVU482 .loc 2 57 9 view .LVU483 movl %edi, %edx # cpu, cpu .LVL87: .loc 2 57 9 is_stmt 0 view .LVU484 That MOV in there is drowned in debugging information and latter makes it hard to follow the asm. And that DWARF info is not really needed for asm output staring. Disable the debug information generation which clutters the asm output unnecessarily: bsp_resume: # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) testq %rax, %rax # _2 je .L8 #, # arch/x86/kernel/cpu/common.c:1870: this_cpu->c_bsp_resume(&boot_cpu_data); movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax .L8: # arch/x86/kernel/cpu/common.c:1871: } rep ret .size bsp_resume, .-bsp_resume [ bp: write commit message. ] Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Borislav Petkov <bp@suse.de>
2019-02-10 13:51:00 +07:00
DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
Makefile: support compressed debug info As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size. All of the debug sections will have a `C` flag set. $ readelf -S <object file> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL For x86_64 defconfig + LLVM=1 (before): Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856 For x86_64 defconfig + LLVM=1 (after): Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776 Thanks to: Nick Clifton helped us to provide the minimal binutils version. Sedat Dilek found an increase in size of debug .deb package. Cc: Nick Clifton <nickc@redhat.com> Suggested-by: David Blaikie <blaikie@google.com> Reviewed-by: Fangrui Song <maskray@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-27 00:18:29 +07:00
ifdef CONFIG_DEBUG_INFO_COMPRESSED
DEBUG_CFLAGS += -gz=zlib
Makefile: Improve compressed debug info support detection Commit 10e68b02c861 ("Makefile: support compressed debug info") added support for compressed debug sections. Support is detected by checking - does the compiler support -gz=zlib - does the assembler support --compressed-debug-sections=zlib - does the linker support --compressed-debug-sections=zlib However, the gcc driver's support for this option is somewhat convoluted. The driver's builtin specs are set based on the version of binutils that it was configured with. It reports an error if the configure-time linker/assembler (i.e., not necessarily the actual assembler that will be run) do not support the option, but only if the assembler (or linker) is actually invoked when -gz=zlib is passed. The cc-option check in scripts/Kconfig.include does not invoke the assembler, so the gcc driver reports success even if it does not support the option being passed to the assembler. Because the as-option check passes the option directly to the assembler via -Wa,--compressed-debug-sections=zlib, the gcc driver does not see this option and will never report an error. Combined with an installed version of binutils that is more recent than the one the compiler was built with, it is possible for all three tests to succeed, yet an actual compilation with -gz=zlib to fail. Moreover, it is unnecessary to explicitly pass --compressed-debug-sections=zlib to the assembler via -Wa, since the driver will do that automatically when it supports -gz=zlib. Convert the as-option to just -gz=zlib, simplifying it as well as performing a better test of the gcc driver's capabilities. Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-12 05:03:39 +07:00
KBUILD_AFLAGS += -gz=zlib
Makefile: support compressed debug info As debug information gets larger and larger, it helps significantly save the size of vmlinux images to compress the information in the debug information sections. Note: this debug info is typically split off from the final compressed kernel image, which is why vmlinux is what's used in conjunction with GDB. Minimizing the debug info size should have no impact on boot times, or final compressed kernel image size. All of the debug sections will have a `C` flag set. $ readelf -S <object file> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \ vmlinux.gcc75.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +18 [ = ] 0 [Unmapped] -73.3% -114Ki [ = ] 0 .debug_aranges -76.2% -2.01Mi [ = ] 0 .debug_frame -73.6% -2.89Mi [ = ] 0 .debug_str -80.7% -4.66Mi [ = ] 0 .debug_abbrev -82.9% -4.88Mi [ = ] 0 .debug_ranges -70.5% -9.04Mi [ = ] 0 .debug_line -79.3% -10.9Mi [ = ] 0 .debug_loc -39.5% -88.6Mi [ = ] 0 .debug_info -18.2% -123Mi [ = ] 0 TOTAL $ bloaty vmlinux.clang11.compressed.dwarf4 -- \ vmlinux.clang11.uncompressed.dwarf4 FILE SIZE VM SIZE -------------- -------------- +0.0% +23 [ = ] 0 [Unmapped] -65.6% -871 [ = ] 0 .debug_aranges -77.4% -1.84Mi [ = ] 0 .debug_frame -82.9% -2.33Mi [ = ] 0 .debug_abbrev -73.1% -2.43Mi [ = ] 0 .debug_str -84.8% -3.07Mi [ = ] 0 .debug_ranges -65.9% -8.62Mi [ = ] 0 .debug_line -86.2% -40.0Mi [ = ] 0 .debug_loc -42.0% -64.1Mi [ = ] 0 .debug_info -22.1% -122Mi [ = ] 0 TOTAL For x86_64 defconfig + LLVM=1 (before): Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03 Maximum resident set size (kbytes): 43856 For x86_64 defconfig + LLVM=1 (after): Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52 Maximum resident set size (kbytes): 1566776 Thanks to: Nick Clifton helped us to provide the minimal binutils version. Sedat Dilek found an increase in size of debug .deb package. Cc: Nick Clifton <nickc@redhat.com> Suggested-by: David Blaikie <blaikie@google.com> Reviewed-by: Fangrui Song <maskray@google.com> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-27 00:18:29 +07:00
KBUILD_LDFLAGS += --compress-debug-sections=zlib
endif
endif # CONFIG_DEBUG_INFO
kbuild: Disable extra debugging info in .s output Modern gcc adds view assignments, reset assertion checking in .loc directives and a couple more additional debug markers, which clutters the asm output unnecessarily: For example: bsp_resume: .LFB3466: .loc 1 1868 1 is_stmt 1 view -0 .cfi_startproc .loc 1 1869 2 view .LVU73 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 14 is_stmt 0 view .LVU74 movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) .loc 1 1869 5 view .LVU75 testq %rax, %rax # _2 je .L8 #, .loc 1 1870 3 is_stmt 1 view .LVU76 movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax or .loc 2 57 9 view .LVU478 .loc 2 57 9 view .LVU479 .loc 2 57 9 view .LVU480 .loc 2 57 9 view .LVU481 .LBB1385: .LBB1383: .LBB1379: .LBB1377: .LBB1375: .loc 2 57 9 view .LVU482 .loc 2 57 9 view .LVU483 movl %edi, %edx # cpu, cpu .LVL87: .loc 2 57 9 is_stmt 0 view .LVU484 That MOV in there is drowned in debugging information and latter makes it hard to follow the asm. And that DWARF info is not really needed for asm output staring. Disable the debug information generation which clutters the asm output unnecessarily: bsp_resume: # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) movq this_cpu(%rip), %rax # this_cpu, this_cpu movq 64(%rax), %rax # this_cpu.94_1->c_bsp_resume, _2 # arch/x86/kernel/cpu/common.c:1869: if (this_cpu->c_bsp_resume) testq %rax, %rax # _2 je .L8 #, # arch/x86/kernel/cpu/common.c:1870: this_cpu->c_bsp_resume(&boot_cpu_data); movq $boot_cpu_data, %rdi #, jmp __x86_indirect_thunk_rax .L8: # arch/x86/kernel/cpu/common.c:1871: } rep ret .size bsp_resume, .-bsp_resume [ bp: write commit message. ] Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Borislav Petkov <bp@suse.de>
2019-02-10 13:51:00 +07:00
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
ifdef CONFIG_FUNCTION_TRACER
ifdef CONFIG_FTRACE_MCOUNT_RECORD
# gcc 5 supports generating the mcount tables directly
ifeq ($(call cc-option-yn,-mrecord-mcount),y)
CC_FLAGS_FTRACE += -mrecord-mcount
export CC_USING_RECORD_MCOUNT := 1
endif
ifdef CONFIG_HAVE_NOP_MCOUNT
ifeq ($(call cc-option-yn, -mnop-mcount),y)
CC_FLAGS_FTRACE += -mnop-mcount
CC_FLAGS_USING += -DCC_USING_NOP_MCOUNT
endif
endif
endif
ifdef CONFIG_HAVE_FENTRY
ifeq ($(call cc-option-yn, -mfentry),y)
CC_FLAGS_FTRACE += -mfentry
CC_FLAGS_USING += -DCC_USING_FENTRY
endif
endif
export CC_FLAGS_FTRACE
KBUILD_CFLAGS += $(CC_FLAGS_FTRACE) $(CC_FLAGS_USING)
KBUILD_AFLAGS += $(CC_FLAGS_USING)
ifdef CONFIG_DYNAMIC_FTRACE
ifdef CONFIG_HAVE_C_RECORDMCOUNT
BUILD_C_RECORDMCOUNT := y
export BUILD_C_RECORDMCOUNT
endif
endif
endif
# We trigger additional mismatches with less inlining
ifdef CONFIG_DEBUG_SECTION_MISMATCH
KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once)
endif
ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
LDFLAGS_vmlinux += --gc-sections
endif
ifdef CONFIG_SHADOW_CALL_STACK
CC_FLAGS_SCS := -fsanitize=shadow-call-stack
KBUILD_CFLAGS += $(CC_FLAGS_SCS)
export CC_FLAGS_SCS
endif
./Makefile: add debug option to enable function aligned on 32 bytes Recently 0day reported many strange performance changes (regression or improvement), in which there was no obvious relation between the culprit commit and the benchmark at the first look, and it causes people to doubt the test itself is wrong. Upon further check, many of these cases are caused by the change to the alignment of kernel text or data, as whole text/data of kernel are linked together, change in one domain may affect alignments of other domains. gcc has an option '-falign-functions=n' to force text aligned, and with that option enabled, some of those performance changes will be gone, like [1][2][3]. Add this option so that developers and 0day can easily find performance bump caused by text alignment change, as tracking these strange bump is quite time consuming. Though it can't help in other cases like data alignment changes like [4]. Following is some size data for v5.7 kernel built with a RHEL config used in 0day: text data bss dec filename 19738771 13292906 5554236 38585913 vmlinux.noalign 19758591 13297002 5529660 38585253 vmlinux.align32 Raw vmlinux size in bytes: v5.7 v5.7+align32 253950832 254018000 +0.02% Some benchmark data, most of them have no big change: * hackbench: [ -1.8%, +0.5%] * fsmark: [ -3.2%, +3.4%] # ext4/xfs/btrfs * kbuild: [ -2.0%, +0.9%] * will-it-scale: [ -0.5%, +1.8%] # mmap1/pagefault3 * netperf: - TCP_CRR [+16.6%, +97.4%] - TCP_RR [-18.5%, -1.8%] - TCP_STREAM [ -1.1%, +1.9%] [1] https://lore.kernel.org/lkml/20200114085637.GA29297@shao2-debian/ [2] https://lore.kernel.org/lkml/20200330011254.GA14393@feng-iot/ [3] https://lore.kernel.org/lkml/1d98d1f0-fe84-6df7-f5bd-f4cb2cdb7f45@intel.com/ [4] https://lore.kernel.org/lkml/20200205123216.GO12867@shao2-debian/ Signed-off-by: Feng Tang <feng.tang@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Andi Kleen <andi.kleen@intel.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Andy Shevchenko <andriy.shevchenko@intel.com> Link: http://lkml.kernel.org/r/1595475001-90945-1-git-send-email-feng.tang@intel.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-12 08:34:13 +07:00
ifdef CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_32B
KBUILD_CFLAGS += -falign-functions=32
endif
# arch Makefile may override CC so keep this after arch Makefile is included
NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include)
# warn about C99 declaration after statement
KBUILD_CFLAGS += -Wdeclaration-after-statement
# Variable Length Arrays (VLAs) should not be used anywhere in the kernel
KBUILD_CFLAGS += -Wvla
# disable pointer signed / unsigned warnings in gcc 4.0
KBUILD_CFLAGS += -Wno-pointer-sign
# disable stringop warnings in gcc 8+
KBUILD_CFLAGS += $(call cc-disable-warning, stringop-truncation)
# We'll want to enable this eventually, but it's not going away for 5.7 at least
KBUILD_CFLAGS += $(call cc-disable-warning, zero-length-bounds)
KBUILD_CFLAGS += $(call cc-disable-warning, array-bounds)
KBUILD_CFLAGS += $(call cc-disable-warning, stringop-overflow)
gcc-10: disable 'restrict' warning for now gcc-10 now warns about passing aliasing pointers to functions that take restricted pointers. That's actually a great warning, and if we ever start using 'restrict' in the kernel, it might be quite useful. But right now we don't, and it turns out that the only thing this warns about is an idiom where we have declared a few functions to be "printf-like" (which seems to make gcc pick up the restricted pointer thing), and then we print to the same buffer that we also use as an input. And people do that as an odd concatenation pattern, with code like this: #define sysfs_show_gen_prop(buffer, fmt, ...) \ snprintf(buffer, PAGE_SIZE, "%s"fmt, buffer, __VA_ARGS__) where we have 'buffer' as both the destination of the final result, and as the initial argument. Yes, it's a bit questionable. And outside of the kernel, people do have standard declarations like int snprintf( char *restrict buffer, size_t bufsz, const char *restrict format, ... ); where that output buffer is marked as a restrict pointer that cannot alias with any other arguments. But in the context of the kernel, that 'use snprintf() to concatenate to the end result' does work, and the pattern shows up in multiple places. And we have not marked our own version of snprintf() as taking restrict pointers, so the warning is incorrect for now, and gcc picks it up on its own. If we do start using 'restrict' in the kernel (and it might be a good idea if people find places where it matters), we'll need to figure out how to avoid this issue for snprintf and friends. But in the meantime, this warning is not useful. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-05-10 05:45:21 +07:00
# Another good warning that we'll want to enable eventually
KBUILD_CFLAGS += $(call cc-disable-warning, restrict)
# Enabled with W=2, disabled by default as noisy
KBUILD_CFLAGS += $(call cc-disable-warning, maybe-uninitialized)
# disable invalid "can't wrap" optimizations for signed / pointers
KBUILD_CFLAGS += -fno-strict-overflow
# Make sure -fstack-check isn't enabled (like gentoo apparently did)
KBUILD_CFLAGS += -fno-stack-check
# conserve stack if available
KBUILD_CFLAGS += $(call cc-option,-fconserve-stack)
# Prohibit date/time macros, which would make the build non-deterministic
KBUILD_CFLAGS += -Werror=date-time
# enforce correct pointer usage
KBUILD_CFLAGS += $(call cc-option,-Werror=incompatible-pointer-types)
# Require designated initializers for all marked structures
KBUILD_CFLAGS += $(call cc-option,-Werror=designated-init)
# change __FILE__ to the relative path from the srctree
KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srctree)/=)
# ensure -fcf-protection is disabled when using retpoline as it is
# incompatible with -mindirect-branch=thunk-extern
ifdef CONFIG_RETPOLINE
KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none)
endif
# include additional Makefiles when needed
include-y := scripts/Makefile.extrawarn
include-$(CONFIG_KASAN) += scripts/Makefile.kasan
include-$(CONFIG_KCSAN) += scripts/Makefile.kcsan
include-$(CONFIG_UBSAN) += scripts/Makefile.ubsan
include-$(CONFIG_KCOV) += scripts/Makefile.kcov
include-$(CONFIG_GCC_PLUGINS) += scripts/Makefile.gcc-plugins
include $(addprefix $(srctree)/, $(include-y))
# scripts/Makefile.gcc-plugins is intentionally included last.
# Do not add $(call cc-option,...) below this line. When you build the kernel
# from the clean source tree, the GCC plugins do not exist at this point.
# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments
KBUILD_CPPFLAGS += $(KCPPFLAGS)
KBUILD_AFLAGS += $(KAFLAGS)
KBUILD_CFLAGS += $(KCFLAGS)
KBUILD_LDFLAGS_MODULE += --build-id=sha1
LDFLAGS_vmlinux += --build-id=sha1
ifeq ($(CONFIG_STRIP_ASM_SYMS),y)
LDFLAGS_vmlinux += $(call ld-option, -X,)
endif
ifeq ($(CONFIG_RELR),y)
LDFLAGS_vmlinux += --pack-dyn-relocs=relr
endif
# We never want expected sections to be placed heuristically by the
# linker. All sections should be explicitly named in the linker script.
ifdef CONFIG_LD_ORPHAN_WARN
LDFLAGS_vmlinux += --orphan-handling=warn
endif
kbuild: add infrastructure to build userspace programs Kbuild supports the infrastructure to build host programs, but there was no support to build userspace programs for the target architecture (i.e. the same architecture as the kernel). Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154), but it was not merged. One problem at that time was, there was no good way to know whether $(CC) can link standalone programs. In fact, pre-built kernel.org toolchains [1] are often used for building the kernel, but they do not provide libc. Now, we can handle this cleanly because the compiler capability is evaluated at the Kconfig time. If $(CC) cannot link standalone programs, the relevant options are hidden by 'depends on CC_CAN_LINK'. The implementation just mimics scripts/Makefile.host The userspace programs are compiled with the same flags as the host programs. In addition, it uses -m32 or -m64 if it is found in $(KBUILD_CFLAGS). This new syntax has two usecases. - Sample programs Several userspace programs under samples/ include UAPI headers installed in usr/include. Most of them were previously built for the host architecture just to use the 'hostprogs' syntax. However, 'make headers' always works for the target architecture. This caused the arch mismatch in cross-compiling. To fix this distortion, sample code should be built for the target architecture. - Bpfilter net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper, and embeds it into the kernel. Currently, it overrides HOSTCC with CC to use the 'hostprogs' syntax. This hack should go away. [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Sam Ravnborg <sam@ravnborg.org>
2020-04-29 10:45:14 +07:00
# Align the bit size of userspace programs with the kernel
KBUILD_USERCFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CFLAGS))
kbuild: add infrastructure to build userspace programs Kbuild supports the infrastructure to build host programs, but there was no support to build userspace programs for the target architecture (i.e. the same architecture as the kernel). Sam Ravnborg worked on this in 2014 (https://lkml.org/lkml/2014/7/13/154), but it was not merged. One problem at that time was, there was no good way to know whether $(CC) can link standalone programs. In fact, pre-built kernel.org toolchains [1] are often used for building the kernel, but they do not provide libc. Now, we can handle this cleanly because the compiler capability is evaluated at the Kconfig time. If $(CC) cannot link standalone programs, the relevant options are hidden by 'depends on CC_CAN_LINK'. The implementation just mimics scripts/Makefile.host The userspace programs are compiled with the same flags as the host programs. In addition, it uses -m32 or -m64 if it is found in $(KBUILD_CFLAGS). This new syntax has two usecases. - Sample programs Several userspace programs under samples/ include UAPI headers installed in usr/include. Most of them were previously built for the host architecture just to use the 'hostprogs' syntax. However, 'make headers' always works for the target architecture. This caused the arch mismatch in cross-compiling. To fix this distortion, sample code should be built for the target architecture. - Bpfilter net/bpfilter/Makefile compiles bpfilter_umh as the user mode helper, and embeds it into the kernel. Currently, it overrides HOSTCC with CC to use the 'hostprogs' syntax. This hack should go away. [1]: https://mirrors.edge.kernel.org/pub/tools/crosstool/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Sam Ravnborg <sam@ravnborg.org>
2020-04-29 10:45:14 +07:00
# make the checker run with the right architecture
CHECKFLAGS += --arch=$(ARCH)
# insure the checker run with the right endianness
CHECKFLAGS += $(if $(CONFIG_CPU_BIG_ENDIAN),-mbig-endian,-mlittle-endian)
# the checker needs the correct machine size
CHECKFLAGS += $(if $(CONFIG_64BIT),-m64,-m32)
# Default kernel image to build when no specific target is given.
# KBUILD_IMAGE may be overruled on the command line or
# set in the environment
# Also any assignments in arch/$(ARCH)/Makefile take precedence over
# this default value
export KBUILD_IMAGE ?= vmlinux
#
# INSTALL_PATH specifies where to place the updated kernel and system map
# images. Default is /boot, but you can set it to other values
export INSTALL_PATH ?= /boot
kbuild: dtbs_install: new make target Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place. This commit adds a new 'dtbs_install' make target which copies all of the dtbs into the INSTALL_DTBS_PATH directory. INSTALL_DTBS_PATH can be set before calling make to change the default install directory. If not set then it defaults to: $INSTALL_PATH/dtbs/$KERNELRELEASE. This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time. v7: (reworked by Grant Likely) - Moved rules from arch/arm/Makefile to arch/arm/boot/dts/Makefile so that each dtb install could have a separate target and be reported as part of the make output. - Fixed dependency problem to ensure $KERNELRELEASE is calculated before attempting to install - Removed option to call external script. Copying the files should be sufficient and a build system can post-process the install directory. Despite the fact an external script is used for installing the kernel, I don't think that is a pattern that should be encouraged. I would rather see buildroot type tools post process the install directory to rename or move dtb files after installing to a staging directory. - Plus it is easy to add a hook after the fact without blocking the rest of this feature. - Move the helper targets into scripts/Makefile.lib with the rest of the common dtb rules Signed-off-by: Jason Cooper <jason@lakedaemon.net> Signed-off-by: Grant Likely <grant.likely@linaro.org> Cc: Michal Marek <mmarek@suse.cz> Cc: Russell King <linux@arm.linux.org.uk> Cc: Rob Herring <robh+dt@kernel.org>
2013-12-02 06:56:28 +07:00
#
# INSTALL_DTBS_PATH specifies a prefix for relocations required by build roots.
# Like INSTALL_MOD_PATH, it isn't defined in the Makefile, but can be passed as
# an argument if needed. Otherwise it defaults to the kernel install path
#
export INSTALL_DTBS_PATH ?= $(INSTALL_PATH)/dtbs/$(KERNELRELEASE)
#
# INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory
# relocations required by build roots. This is not defined in the
# makefile but the argument can be passed to make if needed.
#
MODLIB = $(INSTALL_MOD_PATH)/lib/modules/$(KERNELRELEASE)
export MODLIB
#
# INSTALL_MOD_STRIP, if defined, will cause modules to be
# stripped after they are installed. If INSTALL_MOD_STRIP is '1', then
# the default option --strip-debug will be used. Otherwise,
# INSTALL_MOD_STRIP value will be used as the options to the strip command.
ifdef INSTALL_MOD_STRIP
ifeq ($(INSTALL_MOD_STRIP),1)
mod_strip_cmd = $(STRIP) --strip-debug
else
mod_strip_cmd = $(STRIP) $(INSTALL_MOD_STRIP)
endif # INSTALL_MOD_STRIP=1
else
mod_strip_cmd = true
endif # INSTALL_MOD_STRIP
export mod_strip_cmd
kbuild: handle module compression while running 'make modules_install'. Since module-init-tools (gzip) and kmod (gzip and xz) support compressed modules, it could be useful to include a support for compressing modules right after having them installed. Doing this in kbuild instead of per distro can permit to make this kind of usage more generic. This patch add a Kconfig entry to "Enable loadable module support" menu and let you choose to compress using gzip (default) or xz. Both gzip and xz does not used any extra -[1-9] option since Andi Kleen and Rusty Russell prove no gain is made using them. gzip is called with -n argument to avoid storing original filename inside compressed file, that way we can save some more bytes. On a v3.16 kernel, 'make allmodconfig' generated 4680 modules for a total of 378MB (no strip, no sign, no compress), the following table shows observed disk space gain based on the allmodconfig .config : | time | +-------------+-----------------+ | manual .ko | make | size | percent | compression | modules_install | | gain +-------------+-----------------+------+-------- - | | 18.61s | 378M | GZIP | 3m16s | 3m37s | 102M | 73.41% XZ | 5m22s | 5m39s | 77M | 79.83% The gain for restricted environnement seems to be interesting while uncompress can be time consuming but happens only while loading a module, that is generally done only once. This is fully compatible with signed modules while the signed module is compressed. module-init-tools or kmod handles decompression and provide to other layer the uncompressed but signed payload. Reviewed-by: Willy Tarreau <w@1wt.eu> Signed-off-by: Bertrand Jacquin <beber@meleeweb.net> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-08-27 18:01:56 +07:00
# CONFIG_MODULE_COMPRESS, if defined, will cause module to be compressed
# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP
# or CONFIG_MODULE_COMPRESS_XZ.
mod_compress_cmd = true
ifdef CONFIG_MODULE_COMPRESS
ifdef CONFIG_MODULE_COMPRESS_GZIP
mod_compress_cmd = $(KGZIP) -n -f
kbuild: handle module compression while running 'make modules_install'. Since module-init-tools (gzip) and kmod (gzip and xz) support compressed modules, it could be useful to include a support for compressing modules right after having them installed. Doing this in kbuild instead of per distro can permit to make this kind of usage more generic. This patch add a Kconfig entry to "Enable loadable module support" menu and let you choose to compress using gzip (default) or xz. Both gzip and xz does not used any extra -[1-9] option since Andi Kleen and Rusty Russell prove no gain is made using them. gzip is called with -n argument to avoid storing original filename inside compressed file, that way we can save some more bytes. On a v3.16 kernel, 'make allmodconfig' generated 4680 modules for a total of 378MB (no strip, no sign, no compress), the following table shows observed disk space gain based on the allmodconfig .config : | time | +-------------+-----------------+ | manual .ko | make | size | percent | compression | modules_install | | gain +-------------+-----------------+------+-------- - | | 18.61s | 378M | GZIP | 3m16s | 3m37s | 102M | 73.41% XZ | 5m22s | 5m39s | 77M | 79.83% The gain for restricted environnement seems to be interesting while uncompress can be time consuming but happens only while loading a module, that is generally done only once. This is fully compatible with signed modules while the signed module is compressed. module-init-tools or kmod handles decompression and provide to other layer the uncompressed but signed payload. Reviewed-by: Willy Tarreau <w@1wt.eu> Signed-off-by: Bertrand Jacquin <beber@meleeweb.net> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-08-27 18:01:56 +07:00
endif # CONFIG_MODULE_COMPRESS_GZIP
ifdef CONFIG_MODULE_COMPRESS_XZ
mod_compress_cmd = $(XZ) -f
kbuild: handle module compression while running 'make modules_install'. Since module-init-tools (gzip) and kmod (gzip and xz) support compressed modules, it could be useful to include a support for compressing modules right after having them installed. Doing this in kbuild instead of per distro can permit to make this kind of usage more generic. This patch add a Kconfig entry to "Enable loadable module support" menu and let you choose to compress using gzip (default) or xz. Both gzip and xz does not used any extra -[1-9] option since Andi Kleen and Rusty Russell prove no gain is made using them. gzip is called with -n argument to avoid storing original filename inside compressed file, that way we can save some more bytes. On a v3.16 kernel, 'make allmodconfig' generated 4680 modules for a total of 378MB (no strip, no sign, no compress), the following table shows observed disk space gain based on the allmodconfig .config : | time | +-------------+-----------------+ | manual .ko | make | size | percent | compression | modules_install | | gain +-------------+-----------------+------+-------- - | | 18.61s | 378M | GZIP | 3m16s | 3m37s | 102M | 73.41% XZ | 5m22s | 5m39s | 77M | 79.83% The gain for restricted environnement seems to be interesting while uncompress can be time consuming but happens only while loading a module, that is generally done only once. This is fully compatible with signed modules while the signed module is compressed. module-init-tools or kmod handles decompression and provide to other layer the uncompressed but signed payload. Reviewed-by: Willy Tarreau <w@1wt.eu> Signed-off-by: Bertrand Jacquin <beber@meleeweb.net> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-08-27 18:01:56 +07:00
endif # CONFIG_MODULE_COMPRESS_XZ
endif # CONFIG_MODULE_COMPRESS
export mod_compress_cmd
ifdef CONFIG_MODULE_SIG_ALL
$(eval $(call config_filename,MODULE_SIG_KEY))
mod_sign_cmd = scripts/sign-file $(CONFIG_MODULE_SIG_HASH) $(MODULE_SIG_KEY_SRCPREFIX)$(CONFIG_MODULE_SIG_KEY) certs/signing_key.x509
else
mod_sign_cmd = true
endif
export mod_sign_cmd
HOST_LIBELF_LIBS = $(shell pkg-config libelf --libs 2>/dev/null || echo -lelf)
has_libelf = $(call try-run,\
echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOST_LIBELF_LIBS) -,1,0)
ifdef CONFIG_STACK_VALIDATION
ifeq ($(has_libelf),1)
objtool_target := tools/objtool FORCE
else
SKIP_STACK_VALIDATION := 1
export SKIP_STACK_VALIDATION
endif
endif
ifdef CONFIG_BPF
ifdef CONFIG_DEBUG_INFO_BTF
ifeq ($(has_libelf),1)
resolve_btfids_target := tools/bpf/resolve_btfids FORCE
else
ERROR_RESOLVE_BTFIDS := 1
endif
endif # CONFIG_DEBUG_INFO_BTF
endif # CONFIG_BPF
kbuild: mark prepare0 as PHONY to fix external module build Commit c3ff2a5193fa ("powerpc/32: add stack protector support") caused kernel panic on PowerPC when an external module is used with CONFIG_STACKPROTECTOR because the 'prepare' target was not executed for the external module build. Commit e07db28eea38 ("kbuild: fix single target build for external module") turned it into a build error because the 'prepare' target is now executed but the 'prepare0' target is missing for the external module build. External module on arm/arm64 with CONFIG_STACKPROTECTOR_PER_TASK is also broken in the same way. Move 'PHONY += prepare0' to the common place. GNU Make is fine with missing rule for phony targets. I also removed the comment which is wrong irrespective of this commit. I minimize the change so it can be easily backported to 4.20.x To fix v4.20, please backport e07db28eea38 ("kbuild: fix single target build for external module"), and then this commit. Link: https://bugzilla.kernel.org/show_bug.cgi?id=201891 Fixes: e07db28eea38 ("kbuild: fix single target build for external module") Fixes: c3ff2a5193fa ("powerpc/32: add stack protector support") Fixes: 189af4657186 ("ARM: smp: add support for per-task stack canaries") Fixes: 0a1213fa7432 ("arm64: enable per-task stack canaries") Cc: linux-stable <stable@vger.kernel.org> # v4.20 Reported-by: Samuel Holland <samuel@sholland.org> Reported-by: Alexey Kardashevskiy <aik@ozlabs.ru> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru>
2019-01-15 14:19:00 +07:00
PHONY += prepare0
kbuild: make single targets work more correctly Currently, the single target build directly descends into the directory of the target. For example, $ make foo/bar/baz.o ... directly descends into foo/bar/. On the other hand, the normal build usually descends one directory at a time, i.e. descends into foo/, and then foo/bar/. This difference causes some problems. [1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles The options in subdir-{as,cc}flags-y take effect in the current and its sub-directories. In other words, they are inherited downward. In the example above, the single target will miss subdir-{as,cc}flags-y if they are defined in foo/Makefile. [2] could be built in a different directory As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can handle files that are spread over several sub-directories. The build rule of foo/bar/baz.o may not necessarily be specified in foo/bar/Makefile. It might be specifies in foo/Makefile as follows: [foo/Makefile] obj-y := bar/baz.o This often happens when a module is so big that its source files are divided into sub-directories. In this case, there is no Makefile in the foo/bar/ directory, yet the single target descends into foo/bar/, then fails due to the missing Makefile. You can still do 'make foo/bar/' for partial building, but cannot do 'make foo/bar/baz.s'. I believe the single target '%.s' is a useful feature for inspecting the compiler output. Some modules work around this issue by putting an empty Makefile in every sub-directory. This commit fixes those problems by making the single target build descend in the same way as the normal build does. Another change is the single target build will observe the CONFIG options. Previously, it allowed users to build the foo.o even when the corresponding CONFIG_FOO is disabled: obj-$(CONFIG_FOO) += foo.o In the new behavior, the single target build will just fail and show "No rule to make target ..." (or "Nothing to be done for ..." if the stale object already exists, but cannot be updated). The disadvantage of this commit is the build speed. Now that the single target build visits every directory and parses lots of Makefiles, it is slower than before. (But, I hope it will not be too slow.) Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-14 22:19:18 +07:00
export MODORDER := $(extmod-prefix)modules.order
export MODULES_NSDEPS := $(extmod-prefix)modules.nsdeps
ifeq ($(KBUILD_EXTMOD),)
core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/
vmlinux-dirs := $(patsubst %/,%,$(filter %/, \
$(core-y) $(core-m) $(drivers-y) $(drivers-m) \
$(libs-y) $(libs-m)))
vmlinux-alldirs := $(sort $(vmlinux-dirs) Documentation \
$(patsubst %/,%,$(filter %/, $(core-) \
$(drivers-) $(libs-))))
subdir-modorder := $(addsuffix modules.order,$(filter %/, \
$(core-y) $(core-m) $(libs-y) $(libs-m) \
$(drivers-y) $(drivers-m)))
build-dirs := $(vmlinux-dirs)
clean-dirs := $(vmlinux-alldirs)
# Externally visible symbols (used by link-vmlinux.sh)
KBUILD_VMLINUX_OBJS := $(head-y) $(patsubst %/,%/built-in.a, $(core-y))
KBUILD_VMLINUX_OBJS += $(addsuffix built-in.a, $(filter %/, $(libs-y)))
kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y Kbuild supports not only obj-y but also lib-y to list objects linked to vmlinux. The difference between them is that all the objects from obj-y are forcibly linked to vmlinux, whereas the objects from lib-y are linked as needed; if there is no user of a lib-y object, it is not linked. lib-y is intended to list utility functions that may be called from all over the place (and may be unused at all), but it is a problem for EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need to keep exported symbols for the use from loadable modules. Commit 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably") worked around it by linking a dummy object, lib-ksyms.o, which contains references to all the symbols exported from lib.a in that directory. It uses the linker script command, EXTERN. Unfortunately, the meaning of EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does not work with LD=ld.lld (CBL issue #515). Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to get rid of it. At first, I was thinking of accumulating lib-y objects into obj-y (or even replacing lib-y with obj-y entirely), but the lib-y syntax is used beyond the ordinary use in lib/ and arch/*/lib/. Examples: - drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked into vmlinux in the own way (arm64), or linked to the decompressor (arm, x86). - arch/alpha/lib/Makefile builds lib.a which is linked not only to vmlinux, but also to bootloaders in arch/alpha/boot/Makefile. - arch/xtensa/boot/lib/Makefile builds lib.a for use from arch/xtensa/boot/boot-redboot/Makefile. One more thing, adding everything to obj-y would increase the vmlinux size of allnoconfig (or tinyconfig). For less impact, I tweaked the destination of lib.a at the top Makefile; when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS as before. The size impact for normal usecases is quite small since at lease one symbol in every lib-y object is eventually called by someone. In case you are intrested, here are the figures. x86_64_defconfig: text data bss dec hex filename 19566602 5422072 1589328 26578002 1958c52 vmlinux.before 19566932 5422104 1589328 26578364 1958dbc vmlinux.after The case with the biggest impact is allnoconfig + CONFIG_MODULES=y. ARCH=x86 allnoconfig + CONFIG_MODULES=y: text data bss dec hex filename 1175162 254740 1220608 2650510 28718e vmlinux.before 1177974 254836 1220608 2653418 287cea vmlinux.after Hopefully this is still not a big deal. The per-file trimming with the static library is not so effective after all. If fine-grained optimization is desired, some architectures support CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol basis. When LTO is supported in mainline, even better optimization will be possible. Link: https://github.com/ClangBuiltLinux/linux/issues/515 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reported-by: kbuild test robot <lkp@intel.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-03-12 05:37:25 +07:00
ifdef CONFIG_MODULES
KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y Kbuild supports not only obj-y but also lib-y to list objects linked to vmlinux. The difference between them is that all the objects from obj-y are forcibly linked to vmlinux, whereas the objects from lib-y are linked as needed; if there is no user of a lib-y object, it is not linked. lib-y is intended to list utility functions that may be called from all over the place (and may be unused at all), but it is a problem for EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need to keep exported symbols for the use from loadable modules. Commit 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably") worked around it by linking a dummy object, lib-ksyms.o, which contains references to all the symbols exported from lib.a in that directory. It uses the linker script command, EXTERN. Unfortunately, the meaning of EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does not work with LD=ld.lld (CBL issue #515). Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to get rid of it. At first, I was thinking of accumulating lib-y objects into obj-y (or even replacing lib-y with obj-y entirely), but the lib-y syntax is used beyond the ordinary use in lib/ and arch/*/lib/. Examples: - drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked into vmlinux in the own way (arm64), or linked to the decompressor (arm, x86). - arch/alpha/lib/Makefile builds lib.a which is linked not only to vmlinux, but also to bootloaders in arch/alpha/boot/Makefile. - arch/xtensa/boot/lib/Makefile builds lib.a for use from arch/xtensa/boot/boot-redboot/Makefile. One more thing, adding everything to obj-y would increase the vmlinux size of allnoconfig (or tinyconfig). For less impact, I tweaked the destination of lib.a at the top Makefile; when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS as before. The size impact for normal usecases is quite small since at lease one symbol in every lib-y object is eventually called by someone. In case you are intrested, here are the figures. x86_64_defconfig: text data bss dec hex filename 19566602 5422072 1589328 26578002 1958c52 vmlinux.before 19566932 5422104 1589328 26578364 1958dbc vmlinux.after The case with the biggest impact is allnoconfig + CONFIG_MODULES=y. ARCH=x86 allnoconfig + CONFIG_MODULES=y: text data bss dec hex filename 1175162 254740 1220608 2650510 28718e vmlinux.before 1177974 254836 1220608 2653418 287cea vmlinux.after Hopefully this is still not a big deal. The per-file trimming with the static library is not so effective after all. If fine-grained optimization is desired, some architectures support CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol basis. When LTO is supported in mainline, even better optimization will be possible. Link: https://github.com/ClangBuiltLinux/linux/issues/515 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reported-by: kbuild test robot <lkp@intel.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-03-12 05:37:25 +07:00
else
KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y Kbuild supports not only obj-y but also lib-y to list objects linked to vmlinux. The difference between them is that all the objects from obj-y are forcibly linked to vmlinux, whereas the objects from lib-y are linked as needed; if there is no user of a lib-y object, it is not linked. lib-y is intended to list utility functions that may be called from all over the place (and may be unused at all), but it is a problem for EXPORT_SYMBOL(). Even if there is no call-site in the vmlinux, we need to keep exported symbols for the use from loadable modules. Commit 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably") worked around it by linking a dummy object, lib-ksyms.o, which contains references to all the symbols exported from lib.a in that directory. It uses the linker script command, EXTERN. Unfortunately, the meaning of EXTERN of ld.lld is different from that of ld.bfd. Therefore, this does not work with LD=ld.lld (CBL issue #515). Anyway, the build rule of lib-ksyms.o is somewhat tricky. So, I want to get rid of it. At first, I was thinking of accumulating lib-y objects into obj-y (or even replacing lib-y with obj-y entirely), but the lib-y syntax is used beyond the ordinary use in lib/ and arch/*/lib/. Examples: - drivers/firmware/efi/libstub/Makefile builds lib.a, which is linked into vmlinux in the own way (arm64), or linked to the decompressor (arm, x86). - arch/alpha/lib/Makefile builds lib.a which is linked not only to vmlinux, but also to bootloaders in arch/alpha/boot/Makefile. - arch/xtensa/boot/lib/Makefile builds lib.a for use from arch/xtensa/boot/boot-redboot/Makefile. One more thing, adding everything to obj-y would increase the vmlinux size of allnoconfig (or tinyconfig). For less impact, I tweaked the destination of lib.a at the top Makefile; when CONFIG_MODULES=y, lib.a goes to KBUILD_VMLINUX_OBJS, which is forcibly linked to vmlinux, otherwise lib.a goes to KBUILD_VMLINUX_LIBS as before. The size impact for normal usecases is quite small since at lease one symbol in every lib-y object is eventually called by someone. In case you are intrested, here are the figures. x86_64_defconfig: text data bss dec hex filename 19566602 5422072 1589328 26578002 1958c52 vmlinux.before 19566932 5422104 1589328 26578364 1958dbc vmlinux.after The case with the biggest impact is allnoconfig + CONFIG_MODULES=y. ARCH=x86 allnoconfig + CONFIG_MODULES=y: text data bss dec hex filename 1175162 254740 1220608 2650510 28718e vmlinux.before 1177974 254836 1220608 2653418 287cea vmlinux.after Hopefully this is still not a big deal. The per-file trimming with the static library is not so effective after all. If fine-grained optimization is desired, some architectures support CONFIG_LD_DEAD_CODE_DATA_ELIMINATION, which trims dead code per-symbol basis. When LTO is supported in mainline, even better optimization will be possible. Link: https://github.com/ClangBuiltLinux/linux/issues/515 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reported-by: kbuild test robot <lkp@intel.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-03-12 05:37:25 +07:00
endif
KBUILD_VMLINUX_OBJS += $(patsubst %/,%/built-in.a, $(drivers-y))
export KBUILD_VMLINUX_OBJS KBUILD_VMLINUX_LIBS
export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds
# used by scripts/Makefile.package
export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) LICENSES arch include scripts tools)
vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_OBJS) $(KBUILD_VMLINUX_LIBS)
kbuild: link vmlinux only once for CONFIG_TRIM_UNUSED_KSYMS If CONFIG_TRIM_UNUSED_KSYMS is enabled and the kernel is built from a pristine state, the vmlinux is linked twice. [1] A user runs 'make' [2] First build with empty autoksyms.h [3] adjust_autoksyms.sh updates autoksyms.h and recurses 'make vmlinux' --------(begin sub-make)-------- [4] Second build with new autoksyms.h [5] link-vmlinux.sh is invoked because vmlinux is missing ---------(end sub-make)--------- [6] link-vmlinux.sh is invoked again despite vmlinux is up-to-date. The reason of [6] is probably because Make already decided to update vmlinux at the time of [2] because vmlinux was missing when Make built up the dependency graph. Because if_changed is implemented based on $?, this issue can be narrowed down to how Make handles $?. You can test it with the following simple code: [Test Makefile] A: B @echo newer prerequisite: $? cp B A B: C cp C B touch A [Result] $ rm -f A B $ touch C $ make cp C B touch A newer prerequisite: B cp B A Here, 'A' has been touched in the recipe of 'B'. So, the dependency 'A: B' has already been met before the recipe of 'A' is executed. However, Make does not notice the fact that the recipe of 'B' also updates 'A' as a side-effect. The situation is similar in this case; the vmlinux has actually been updated in the vmlinux_prereq target. Make cannot predict this, so judges the vmlinux is old. link-vmlinux.sh is costly, so it is better to not run it when unneeded. Split CONFIG_TRIM_UNUSED_KSYMS recursion to a dedicated target. The reason of commit 2441e78b1919 ("kbuild: better abstract vmlinux sequential prerequisites") was to cater to CONFIG_BUILD_DOCSRC, but it was later removed by commit 184892925118 ("samples: move blackfin gptimers-example from Documentation"). Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-16 14:37:15 +07:00
# Recurse until adjust_autoksyms.sh is satisfied
PHONY += autoksyms_recursive
ifdef CONFIG_TRIM_UNUSED_KSYMS
# For the kernel to actually contain only the needed exported symbols,
# we have to build modules as well to determine what those symbols are.
# (this can be evaluated only once include/config/auto.conf has been included)
KBUILD_MODULES := 1
autoksyms_recursive: descend modules.order
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/adjust_autoksyms.sh \
"$(MAKE) -f $(srctree)/Makefile vmlinux"
endif
kbuild: restore autoksyms.h touch to the top Makefile Commit d3fc425e819b ("kbuild: make sure autoksyms.h exists early") moved the code that touches autoksyms.h to scripts/kconfig/Makefile with obscure reason. From Nicolas' comment [1], he did not seem to be sure about the root cause. I guess I figured it out, so here is a fix-up I think is more correct. According to the error log in the original post [2], the build failed in scripts/mod/devicetable-offsets.c scripts/mod/Makefile is descended from scripts/Makefile, which is invoked from the top-level Makefile by the 'scripts' target. To build vmlinux and/or modules, Kbuild descend into $(vmlinux-dirs). This depends on 'prepare' and 'scripts' as follows: $(vmlinux-dirs): prepare scripts Because there is no dependency between 'prepare' and 'scripts', the parallel building can execute them simultaneously. 'prepare' depends on 'prepare1', which touched autoksyms.h, while 'scripts' descends into script/, then scripts/mod/, which needs <generated/autoksyms.h> if CONFIG_TRIM_UNUSED_KSYMS. It was the reason of the race. I am not happy to have unrelated code in the Kconfig Makefile, so getting it back to the top Makefile. I removed the standalone test target because I want to use it to create an empty autoksyms.h file. Here is a little improvement; unnecessary autoksyms.h is not created when CONFIG_TRIM_UNUSED_KSYMS is disabled. [1] https://lkml.org/lkml/2016/11/30/734 [2] https://lkml.org/lkml/2016/11/30/531 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-16 14:37:12 +07:00
autoksyms_h := $(if $(CONFIG_TRIM_UNUSED_KSYMS), include/generated/autoksyms.h)
quiet_cmd_autoksyms_h = GEN $@
cmd_autoksyms_h = mkdir -p $(dir $@); \
$(CONFIG_SHELL) $(srctree)/scripts/gen_autoksyms.sh $@
kbuild: restore autoksyms.h touch to the top Makefile Commit d3fc425e819b ("kbuild: make sure autoksyms.h exists early") moved the code that touches autoksyms.h to scripts/kconfig/Makefile with obscure reason. From Nicolas' comment [1], he did not seem to be sure about the root cause. I guess I figured it out, so here is a fix-up I think is more correct. According to the error log in the original post [2], the build failed in scripts/mod/devicetable-offsets.c scripts/mod/Makefile is descended from scripts/Makefile, which is invoked from the top-level Makefile by the 'scripts' target. To build vmlinux and/or modules, Kbuild descend into $(vmlinux-dirs). This depends on 'prepare' and 'scripts' as follows: $(vmlinux-dirs): prepare scripts Because there is no dependency between 'prepare' and 'scripts', the parallel building can execute them simultaneously. 'prepare' depends on 'prepare1', which touched autoksyms.h, while 'scripts' descends into script/, then scripts/mod/, which needs <generated/autoksyms.h> if CONFIG_TRIM_UNUSED_KSYMS. It was the reason of the race. I am not happy to have unrelated code in the Kconfig Makefile, so getting it back to the top Makefile. I removed the standalone test target because I want to use it to create an empty autoksyms.h file. Here is a little improvement; unnecessary autoksyms.h is not created when CONFIG_TRIM_UNUSED_KSYMS is disabled. [1] https://lkml.org/lkml/2016/11/30/734 [2] https://lkml.org/lkml/2016/11/30/531 Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-16 14:37:12 +07:00
$(autoksyms_h):
$(call cmd,autoksyms_h)
ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink)
# Final link of vmlinux with optional arch pass after final link
cmd_link-vmlinux = \
kbuild: do not export LDFLAGS_vmlinux When you clean the build tree for ARCH=arm, you may see the following error message from 'nm' command: $ make -j24 ARCH=arm clean CLEAN arch/arm/crypto CLEAN arch/arm/kernel CLEAN arch/arm/mach-at91 CLEAN arch/arm/mach-omap2 CLEAN arch/arm/vdso CLEAN certs CLEAN lib CLEAN usr CLEAN net/wireless CLEAN drivers/firmware/efi/libstub nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file /bin/sh: 1: arithmetic expression: expecting primary: " " CLEAN arch/arm/boot/compressed CLEAN drivers/scsi CLEAN drivers/tty/vt CLEAN arch/arm/boot CLEAN vmlinux.symvers modules.builtin modules.builtin.modinfo Even if you rerun the same command, the error message will not be shown despite vmlinux is already gone. To reproduce it, the parallel option -j is needed. Single thread cleaning always executes 'archclean', 'vmlinuxclean' in this order, so vmlinux still exists when arch/arm/boot/compressed/ is cleaned. Looking at arch/arm/boot/compressed/Makefile does not help understand the reason of the error message. Both KBSS_SZ and LDFLAGS_vmlinux are assigned with '=' operator, hence, they are not expanded unless used. Obviously, 'make clean' does not use them. In fact, the root cause exists in the top Makefile: export LDFLAGS_vmlinux Since LDFLAGS_vmlinux is an exported variable, LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is expanded when scripts/Makefile.clean has a command to execute. This is why the error message shows up only when there exist build artifacts in arch/arm/boot/compressed/. Adding 'unexport LDFLAGS_vmlinux' to arch/arm/boot/compressed/Makefile will fix it as far as ARCH=arm is concerned, but I think the proper fix is to get rid of 'export LDFLAGS_vmlinux' from the top Makefile. LDFLAGS_vmlinux in the top Makefile contains linker flags for the top vmlinux. LDFLAGS_vmlinux in arch/arm/boot/compressed/Makefile is for arch/arm/boot/compressed/vmlinux. They just happen to have the same variable name, but are used for different purposes. Stop shadowing LDFLAGS_vmlinux. This commit passes LDFLAGS_vmlinux to scripts/link-vmlinux.sh via a command line parameter instead of via an environment variable. LD and KBUILD_LDFLAGS are exported, but I did the same for consistency. Anyway, they must be included in cmd_link-vmlinux to allow if_changed to detect the changes in LD or KBUILD_LDFLAGS. The following Makefiles are not affected: arch/arm/boot/compressed/Makefile arch/h8300/boot/compressed/Makefile arch/nios2/boot/compressed/Makefile arch/parisc/boot/compressed/Makefile arch/s390/boot/compressed/Makefile arch/sh/boot/compressed/Makefile arch/sh/boot/romimage/Makefile arch/x86/boot/compressed/Makefile They use ':=' or '=' to clear the LDFLAGS_vmlinux inherited from the top Makefile. We need to take a closer look at the impact to unicore32 and xtensa. arch/unicore32/boot/compressed/Makefile only uses '+=' operator for LDFLAGS_vmlinux. So, the decompressor previously inherited the linker flags from the top Makefile. However, commit 70fac51feaf2 ("unicore32 additional architecture files: boot process") was merged before commit 1f2bfbd00e46 ("kbuild: link of vmlinux moved to a script"). So, I rather consider this is a bug fix of 1f2bfbd00e46. arch/xtensa/boot/boot-elf/Makefile is also affected, but this is also considered a fix for the same reason. It did not inherit LDFLAGS_vmlinux when commit 4bedea945451 ("[PATCH] xtensa: Architecture support for Tensilica Xtensa Part 2") was merged. I deleted $(LDFLAGS_vmlinux), which is now empty. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2020-07-02 02:29:36 +07:00
$(CONFIG_SHELL) $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)"; \
$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)
kbuild: link vmlinux only once for CONFIG_TRIM_UNUSED_KSYMS If CONFIG_TRIM_UNUSED_KSYMS is enabled and the kernel is built from a pristine state, the vmlinux is linked twice. [1] A user runs 'make' [2] First build with empty autoksyms.h [3] adjust_autoksyms.sh updates autoksyms.h and recurses 'make vmlinux' --------(begin sub-make)-------- [4] Second build with new autoksyms.h [5] link-vmlinux.sh is invoked because vmlinux is missing ---------(end sub-make)--------- [6] link-vmlinux.sh is invoked again despite vmlinux is up-to-date. The reason of [6] is probably because Make already decided to update vmlinux at the time of [2] because vmlinux was missing when Make built up the dependency graph. Because if_changed is implemented based on $?, this issue can be narrowed down to how Make handles $?. You can test it with the following simple code: [Test Makefile] A: B @echo newer prerequisite: $? cp B A B: C cp C B touch A [Result] $ rm -f A B $ touch C $ make cp C B touch A newer prerequisite: B cp B A Here, 'A' has been touched in the recipe of 'B'. So, the dependency 'A: B' has already been met before the recipe of 'A' is executed. However, Make does not notice the fact that the recipe of 'B' also updates 'A' as a side-effect. The situation is similar in this case; the vmlinux has actually been updated in the vmlinux_prereq target. Make cannot predict this, so judges the vmlinux is old. link-vmlinux.sh is costly, so it is better to not run it when unneeded. Split CONFIG_TRIM_UNUSED_KSYMS recursion to a dedicated target. The reason of commit 2441e78b1919 ("kbuild: better abstract vmlinux sequential prerequisites") was to cater to CONFIG_BUILD_DOCSRC, but it was later removed by commit 184892925118 ("samples: move blackfin gptimers-example from Documentation"). Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-16 14:37:15 +07:00
vmlinux: scripts/link-vmlinux.sh autoksyms_recursive $(vmlinux-deps) FORCE
+$(call if_changed,link-vmlinux)
kbuild: let fixdep directly write to .*.cmd files Currently, fixdep writes dependencies to .*.tmp, which is renamed to .*.cmd after everything succeeds. This is a very safe way to avoid corrupted .*.cmd files. The if_changed_dep has carried this safety mechanism since it was added in 2002. If fixdep fails for some reasons or a user terminates the build while fixdep is running, the incomplete output from the fixdep could be troublesome. This is my insight about some bad scenarios: [1] If the compiler succeeds to generate *.o file, but fixdep fails to write necessary dependencies to .*.cmd file, Make will miss to rebuild the object when headers or CONFIG options are changed. In this case, fixdep should not generate .*.cmd file at all so that 'arg-check' will surely trigger the rebuild of the object. [2] A partially constructed .*.cmd file may not be a syntactically correct makefile. The next time Make runs, it would include it, then fail to parse it. Once this happens, 'make clean' is be the only way to fix it. In fact, [1] is no longer a problem since commit 9c2af1c7377a ("kbuild: add .DELETE_ON_ERROR special target"). Make deletes a target file on any failure in its recipe. Because fixdep is a part of the recipe of *.o target, if it fails, the *.o is deleted anyway. However, I am a bit worried about the slight possibility of [2]. So, here is a solution. Let fixdep directly write to a .*.cmd file, but allow makefiles to include it only when its corresponding target exists. This effectively reverts commit 2982c953570b ("kbuild: remove redundant $(wildcard ...) for cmd_files calculation"), and commit 00d78ab2ba75 ("kbuild: remove dead code in cmd_files calculation in top Makefile") because now we must check the presence of targets. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-30 08:05:22 +07:00
targets := vmlinux
# The actual objects are generated when descending,
# make sure no implicit rule kicks in
$(sort $(vmlinux-deps) $(subdir-modorder)): descend ;
filechk_kernel.release = \
echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"
# Store (new) KERNELRELEASE string in include/config/kernel.release
include/config/kernel.release: FORCE
$(call filechk,kernel.release)
# Additional helpers built in scripts/
# Carefully list dependencies so we do not try to build scripts twice
# in parallel
PHONY += scripts
scripts: scripts_basic scripts_dtc
$(Q)$(MAKE) $(build)=$(@)
# Things we need to do before we recursively start building the kernel
# or the modules are listed in "prepare".
# A multi level approach is used. prepareN is processed before prepareN-1.
# archprepare is used in arch Makefiles and when processed asm symlink,
# version.h and scripts_basic is processed / created.
PHONY += prepare archprepare
archprepare: outputmakefile archheaders archscripts scripts include/config/kernel.release \
asm-generic $(version_h) $(autoksyms_h) include/generated/utsrelease.h \
include/generated/autoconf.h
kbuild: fix UML build error with CONFIG_GCC_PLUGINS UML fails to build with CONFIG_GCC_PLUGINS=y. $ make -s ARCH=um mrproper $ make -s ARCH=um allmodconfig $ make ARCH=um UPD include/generated/uapi/linux/version.h WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h WRAP arch/x86/include/generated/uapi/asm/poll.h WRAP arch/x86/include/generated/asm/dma-contiguous.h WRAP arch/x86/include/generated/asm/early_ioremap.h WRAP arch/x86/include/generated/asm/export.h WRAP arch/x86/include/generated/asm/mcs_spinlock.h WRAP arch/x86/include/generated/asm/mm-arch-hooks.h SYSTBL arch/x86/include/generated/asm/syscalls_32.h SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h SYSTBL arch/x86/include/generated/asm/syscalls_64.h SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h HOSTCC scripts/unifdef CC arch/x86/um/user-offsets.s cc1: error: cannot load plugin ./scripts/gcc-plugins/cyc_complexity_plugin.so ./scripts/gcc-plugins/cyc_complexity_plugin.so: cannot open shared object file: No such file or directory cc1: error: cannot load plugin ./scripts/gcc-plugins/structleak_plugin.so ./scripts/gcc-plugins/structleak_plugin.so: cannot open shared object file: No such file or directory cc1: error: cannot load plugin ./scripts/gcc-plugins/latent_entropy_plugin.so ./scripts/gcc-plugins/latent_entropy_plugin.so: cannot open shared object file: No such file or directory cc1: error: cannot load plugin ./scripts/gcc-plugins/randomize_layout_plugin.so ./scripts/gcc-plugins/randomize_layout_plugin.so: cannot open shared object file: No such file or directory make[1]: *** [scripts/Makefile.build;119: arch/x86/um/user-offsets.s] Error 1 make: *** [arch/um/Makefile;152: arch/x86/um/user-offsets.s] Error 2 Reorder the preparation stage (with cleanups) to make sure gcc-plugins is built before descending to arch/x86/um/. Fixes: 6b90bd4ba40b ("GCC plugin infrastructure") Reported-by: kbuild test robot <lkp@intel.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-29 09:58:50 +07:00
prepare0: archprepare
$(Q)$(MAKE) $(build)=scripts/mod
$(Q)$(MAKE) $(build)=.
# All the preparing..
prepare: prepare0 prepare-objtool prepare-resolve_btfids
# Support for using generic headers in asm-generic
asm-generic := -f $(srctree)/scripts/Makefile.asm-generic obj
PHONY += asm-generic uapi-asm-generic
asm-generic: uapi-asm-generic
$(Q)$(MAKE) $(asm-generic)=arch/$(SRCARCH)/include/generated/asm \
generic=include/asm-generic
uapi-asm-generic:
$(Q)$(MAKE) $(asm-generic)=arch/$(SRCARCH)/include/generated/uapi/asm \
generic=include/uapi/asm-generic
PHONY += prepare-objtool prepare-resolve_btfids
prepare-objtool: $(objtool_target)
ifeq ($(SKIP_STACK_VALIDATION),1)
ifdef CONFIG_UNWINDER_ORC
@echo "error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2
@false
else
@echo "warning: Cannot use CONFIG_STACK_VALIDATION=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2
endif
endif
prepare-resolve_btfids: $(resolve_btfids_target)
ifeq ($(ERROR_RESOLVE_BTFIDS),1)
@echo "error: Cannot resolve BTF IDs for CONFIG_DEBUG_INFO_BTF, please install libelf-dev, libelf-devel or elfutils-libelf-devel" >&2
@false
endif
# Generate some files
# ---------------------------------------------------------------------------
# KERNELRELEASE can change from a few different places, meaning version.h
# needs to be updated, so this check is forced on all builds
uts_len := 64
define filechk_utsrelease.h
if [ `echo -n "$(KERNELRELEASE)" | wc -c ` -gt $(uts_len) ]; then \
echo '"$(KERNELRELEASE)" exceeds $(uts_len) characters' >&2; \
exit 1; \
fi; \
echo \#define UTS_RELEASE \"$(KERNELRELEASE)\"
endef
define filechk_version.h
echo \#define LINUX_VERSION_CODE $(shell \
expr $(VERSION) \* 65536 + 0$(PATCHLEVEL) \* 256 + 0$(SUBLEVEL)); \
echo '#define KERNEL_VERSION(a,b,c) (((a) << 16) + ((b) << 8) + (c))'
endef
$(version_h): FORCE
$(call filechk,version.h)
$(Q)rm -f $(old_version_h)
include/generated/utsrelease.h: include/config/kernel.release FORCE
$(call filechk,utsrelease.h)
PHONY += headerdep
headerdep:
$(Q)find $(srctree)/include/ -name '*.h' | xargs --max-args 1 \
$(srctree)/scripts/headerdep.pl -I$(srctree)/include
# ---------------------------------------------------------------------------
# Kernel headers
#Default location for installed headers
export INSTALL_HDR_PATH = $(objtree)/usr
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
quiet_cmd_headers_install = INSTALL $(INSTALL_HDR_PATH)/include
cmd_headers_install = \
mkdir -p $(INSTALL_HDR_PATH); \
rsync -mrl --include='*/' --include='*\.h' --exclude='*' \
usr/include $(INSTALL_HDR_PATH)
PHONY += headers_install
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
headers_install: headers
$(call cmd,headers_install)
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
PHONY += archheaders archscripts
hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
PHONY += headers
headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
UAPI: Plumb the UAPI Kbuilds into the user header installation and checking Plumb the UAPI Kbuilds into the user header installation and checking system. As the headers are split the entries will be transferred across from the old Kbuild files to the UAPI Kbuild files. The changes made in this commit are: (1) Exported generated files (of which there are currently four) are moved to uapi/ directories under the appropriate generated/ directory, thus we get: include/generated/uapi/linux/version.h arch/x86/include/generated/uapi/asm/unistd_32.h arch/x86/include/generated/uapi/asm/unistd_64.h arch/x86/include/generated/uapi/asm/unistd_x32.h These paths were added to the build as -I flags in a previous patch. (2) scripts/Makefile.headersinst is now given the UAPI path to install from rather than the old path. It then determines the old path from that and includes that Kbuild also if it exists, thus permitting the headers to exist in either directory during the changeover. I also renamed the "install" variable to "installdir" as it refers to a directory not the install program. (3) scripts/headers_install.pl is altered to take a list of source file paths instead of just their names so that the makefile can tell it exactly where to find each file. For the moment, files can be obtained from one of four places for each output directory: .../include/uapi/foo/ .../include/generated/uapi/foo/ .../include/foo/ .../include/generated/foo/ The non-UAPI paths will be dropped later. Signed-off-by: David Howells <dhowells@redhat.com> Acked-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Acked-by: Dave Jones <davej@redhat.com>
2012-10-03 00:01:57 +07:00
$(error Headers not exportable for the $(SRCARCH) architecture))
$(Q)$(MAKE) $(hdr-inst)=include/uapi
$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi
# Deprecated. It is no-op now.
PHONY += headers_check
headers_check:
@:
ifdef CONFIG_HEADERS_INSTALL
kbuild: add 'headers' target to build up uapi headers in usr/include In Linux build system, build targets and installation targets are separated. Examples are: - 'make vmlinux' -> 'make install' - 'make modules' -> 'make modules_install' - 'make dtbs' -> 'make dtbs_install' - 'make vdso' -> 'make vdso_install' The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories. We have 'make headers_install' but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly. If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets. Anyway, I believe it makes sense to split the header installation into two stages. [1] 'make headers' Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include [2] 'make headers_install' Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include For the backward compatibility, 'headers_install' depends on 'headers'. Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH. Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains. Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to save an object and .*.cmd somewhere. The usr/include/ will be the work directory for that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04 17:14:02 +07:00
prepare: headers
endif
PHONY += scripts_unifdef
scripts_unifdef: scripts_basic
$(Q)$(MAKE) $(build)=scripts scripts/unifdef
# ---------------------------------------------------------------------------
# Kernel selftest
PHONY += kselftest
kselftest:
$(Q)$(MAKE) -C $(srctree)/tools/testing/selftests run_tests
kselftest-%: FORCE
$(Q)$(MAKE) -C $(srctree)/tools/testing/selftests $*
PHONY += kselftest-merge
kselftest-merge:
$(if $(wildcard $(objtree)/.config),, $(error No .config exists, config your kernel first!))
$(Q)find $(srctree)/tools/testing/selftests -name config | \
xargs $(srctree)/scripts/kconfig/merge_config.sh -m $(objtree)/.config
$(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
# ---------------------------------------------------------------------------
# Devicetree files
ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/boot/dts/),)
dtstree := arch/$(SRCARCH)/boot/dts
endif
ifneq ($(dtstree),)
%.dtb: include/config/kernel.release scripts_dtc
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
$(Q)$(MAKE) $(build)=$(dtstree) $(dtstree)/$@
PHONY += dtbs dtbs_install dtbs_check
kbuild: avoid concurrency issue in parallel building dtbs and dtbs_check 'make dtbs_check' checks the shecma in addition to building *.dtb files, in other words, 'make dtbs_check' is a super-set of 'make dtbs'. So, you do not have to do 'make dtbs dtbs_check', but I want to keep the build system as robust as possible in any use. Currently, 'dtbs' and 'dtbs_check' are independent of each other. In parallel building, two threads descend into arch/*/boot/dts/, one for dtbs and the other for dtbs_check, then end up with building the same DTB simultaneously. This commit fixes the concurrency issue. Otherwise, I see build errors like follows: $ make ARCH=arm64 defconfig $ make -j16 ARCH=arm64 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dtbs dtbs_check <snip> DTC arch/arm64/boot/dts/qcom/sdm845-cheza-r2.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905x-p212.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-evk.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb DTC arch/arm64/boot/dts/zte/zx296718-pcbox.dtb DTC arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1254-revA.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dtb DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-inx.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb CHECK arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml fixdep: error opening file: arch/arm64/boot/dts/allwinner/.sun50i-h6-orangepi-lite2.dtb.d: No such file or directory make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb] Error 2 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb' make[2]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-kd.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p231.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1275-revA.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-ddr4-evk.dtb fixdep: parse error; no targets found make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb] Error 1 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb' make[1]: *** [scripts/Makefile.build:505: arch/arm64/boot/dts/allwinner] Error 2 make[1]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/renesas/r8a77951-salvator-xs.dtb Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Rob Herring <robh@kernel.org>
2020-03-04 10:20:36 +07:00
dtbs: include/config/kernel.release scripts_dtc
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
$(Q)$(MAKE) $(build)=$(dtstree)
kbuild: avoid concurrency issue in parallel building dtbs and dtbs_check 'make dtbs_check' checks the shecma in addition to building *.dtb files, in other words, 'make dtbs_check' is a super-set of 'make dtbs'. So, you do not have to do 'make dtbs dtbs_check', but I want to keep the build system as robust as possible in any use. Currently, 'dtbs' and 'dtbs_check' are independent of each other. In parallel building, two threads descend into arch/*/boot/dts/, one for dtbs and the other for dtbs_check, then end up with building the same DTB simultaneously. This commit fixes the concurrency issue. Otherwise, I see build errors like follows: $ make ARCH=arm64 defconfig $ make -j16 ARCH=arm64 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dtbs dtbs_check <snip> DTC arch/arm64/boot/dts/qcom/sdm845-cheza-r2.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905x-p212.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-evk.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb DTC arch/arm64/boot/dts/zte/zx296718-pcbox.dtb DTC arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1254-revA.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dtb DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-inx.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb CHECK arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml fixdep: error opening file: arch/arm64/boot/dts/allwinner/.sun50i-h6-orangepi-lite2.dtb.d: No such file or directory make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb] Error 2 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb' make[2]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-kd.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p231.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1275-revA.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-ddr4-evk.dtb fixdep: parse error; no targets found make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb] Error 1 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb' make[1]: *** [scripts/Makefile.build:505: arch/arm64/boot/dts/allwinner] Error 2 make[1]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/renesas/r8a77951-salvator-xs.dtb Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Rob Herring <robh@kernel.org>
2020-03-04 10:20:36 +07:00
ifneq ($(filter dtbs_check, $(MAKECMDGOALS)),)
export CHECK_DTBS=y
kbuild: avoid concurrency issue in parallel building dtbs and dtbs_check 'make dtbs_check' checks the shecma in addition to building *.dtb files, in other words, 'make dtbs_check' is a super-set of 'make dtbs'. So, you do not have to do 'make dtbs dtbs_check', but I want to keep the build system as robust as possible in any use. Currently, 'dtbs' and 'dtbs_check' are independent of each other. In parallel building, two threads descend into arch/*/boot/dts/, one for dtbs and the other for dtbs_check, then end up with building the same DTB simultaneously. This commit fixes the concurrency issue. Otherwise, I see build errors like follows: $ make ARCH=arm64 defconfig $ make -j16 ARCH=arm64 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dtbs dtbs_check <snip> DTC arch/arm64/boot/dts/qcom/sdm845-cheza-r2.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905x-p212.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-evk.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb DTC arch/arm64/boot/dts/zte/zx296718-pcbox.dtb DTC arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p230.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1254-revA.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dtb DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-inx.dtb DTC arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb CHECK arch/arm64/boot/dts/altera/socfpga_stratix10_socdk.dt.yaml fixdep: error opening file: arch/arm64/boot/dts/allwinner/.sun50i-h6-orangepi-lite2.dtb.d: No such file or directory make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb] Error 2 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-lite2.dtb' make[2]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet-kd.dtb DTC arch/arm64/boot/dts/amlogic/meson-gxl-s905d-p231.dtb DTC arch/arm64/boot/dts/xilinx/zynqmp-zc1275-revA.dtb DTC arch/arm64/boot/dts/freescale/imx8mn-ddr4-evk.dtb fixdep: parse error; no targets found make[2]: *** [scripts/Makefile.lib:296: arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb] Error 1 make[2]: *** Deleting file 'arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-one-plus.dtb' make[1]: *** [scripts/Makefile.build:505: arch/arm64/boot/dts/allwinner] Error 2 make[1]: *** Waiting for unfinished jobs.... DTC arch/arm64/boot/dts/renesas/r8a77951-salvator-xs.dtb Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Rob Herring <robh@kernel.org>
2020-03-04 10:20:36 +07:00
dtbs: dt_binding_check
endif
dtbs_check: dtbs
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
dtbs_install:
$(Q)$(MAKE) $(dtbinst)=$(dtstree) dst=$(INSTALL_DTBS_PATH)
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
ifdef CONFIG_OF_EARLY_FLATTREE
all: dtbs
endif
endif
PHONY += scripts_dtc
scripts_dtc: scripts_basic
$(Q)$(MAKE) $(build)=scripts/dtc
ifneq ($(filter dt_binding_check, $(MAKECMDGOALS)),)
export CHECK_DT_BINDING=y
endif
PHONY += dt_binding_check
dt_binding_check: scripts_dtc
$(Q)$(MAKE) $(build)=Documentation/devicetree/bindings
# ---------------------------------------------------------------------------
# Modules
ifdef CONFIG_MODULES
# By default, build modules as well
all: modules
# When we're building modules with modversions, we need to consider
# the built-in objects during the descend as well, in order to
# make sure the checksums are up to date before we record them.
ifdef CONFIG_MODVERSIONS
KBUILD_BUILTIN := 1
endif
# Build modules
#
# A module can be listed more than once in obj-m resulting in
# duplicate lines in modules.order files. Those are removed
# using awk while concatenating to the final file.
PHONY += modules
modules: $(if $(KBUILD_BUILTIN),vmlinux) modules_check modules_prepare
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
PHONY += modules_check
modules_check: modules.order
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/modules-check.sh $<
cmd_modules_order = $(AWK) '!x[$$0]++' $(real-prereqs) > $@
modules.order: $(subdir-modorder) FORCE
$(call if_changed,modules_order)
targets += modules.order
# Target to prepare building external modules
PHONY += modules_prepare
modules_prepare: prepare
$(Q)$(MAKE) $(build)=scripts scripts/module.lds
# Target to install modules
PHONY += modules_install
modules_install: _modinst_ _modinst_post
PHONY += _modinst_
_modinst_:
@rm -rf $(MODLIB)/kernel
@rm -f $(MODLIB)/source
@mkdir -p $(MODLIB)/kernel
@ln -s $(abspath $(srctree)) $(MODLIB)/source
@if [ ! $(objtree) -ef $(MODLIB)/build ]; then \
rm -f $(MODLIB)/build ; \
ln -s $(CURDIR) $(MODLIB)/build ; \
fi
@sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order
@cp -f modules.builtin $(MODLIB)/
@cp -f $(objtree)/modules.builtin.modinfo $(MODLIB)/
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst
# This depmod is only for convenience to give the initial
# boot a modules.dep even before / is mounted read-write. However the
# boot script depmod is the master version.
PHONY += _modinst_post
_modinst_post: _modinst_
$(call cmd,depmod)
ifeq ($(CONFIG_MODULE_SIG), y)
PHONY += modules_sign
modules_sign:
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modsign
endif
else # CONFIG_MODULES
# Modules not configured
# ---------------------------------------------------------------------------
PHONY += modules modules_install
modules modules_install:
@echo >&2
@echo >&2 "The present kernel configuration has modules disabled."
@echo >&2 "Type 'make config' and enable loadable module support."
@echo >&2 "Then build a kernel with module support enabled."
@echo >&2
@exit 1
endif # CONFIG_MODULES
###
# Cleaning is done on three levels.
# make clean Delete most generated files
# Leave enough to build external modules
# make mrproper Delete the current configuration, and all generated files
# make distclean Remove editor backup files, patch leftover files and the like
# Directories & files removed with 'make clean'
CLEAN_FILES += include/ksym vmlinux.symvers \
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
modules.builtin modules.builtin.modinfo modules.nsdeps \
compile_commands.json
# Directories & files removed with 'make mrproper'
MRPROPER_FILES += include/config include/generated \
arch/$(SRCARCH)/include/generated .tmp_objdiff \
debian snap tar-install \
.config .config.old .version \
Module.symvers \
signing_key.pem signing_key.priv signing_key.x509 \
x509.genkey extra_certificates signing_key.x509.keyid \
signing_key.x509.signer vmlinux-gdb.py \
*.spec
# Directories & files removed with 'make distclean'
DISTCLEAN_FILES += tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS
# clean - Delete most, but leave enough to build external modules
#
clean: rm-files := $(CLEAN_FILES)
PHONY += archclean vmlinuxclean
vmlinuxclean:
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/link-vmlinux.sh clean
$(Q)$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) clean)
clean: archclean vmlinuxclean
# mrproper - Delete all generated files, including .config
#
mrproper: rm-files := $(wildcard $(MRPROPER_FILES))
mrproper-dirs := $(addprefix _mrproper_,scripts)
PHONY += $(mrproper-dirs) mrproper
$(mrproper-dirs):
$(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@)
mrproper: clean $(mrproper-dirs)
$(call cmd,rmfiles)
# distclean
#
distclean: rm-files := $(wildcard $(DISTCLEAN_FILES))
PHONY += distclean
distclean: mrproper
$(call cmd,rmfiles)
@find $(srctree) $(RCS_FIND_IGNORE) \
\( -name '*.orig' -o -name '*.rej' -o -name '*~' \
-o -name '*.bak' -o -name '#*#' -o -name '*%' \
-o -name 'core' \) \
-type f -print | xargs rm -f
# Packaging of the kernel to various formats
# ---------------------------------------------------------------------------
%src-pkg: FORCE
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.package $@
%pkg: include/config/kernel.release FORCE
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.package $@
# Brief documentation of the typical targets used
# ---------------------------------------------------------------------------
boards := $(wildcard $(srctree)/arch/$(SRCARCH)/configs/*_defconfig)
boards := $(sort $(notdir $(boards)))
board-dirs := $(dir $(wildcard $(srctree)/arch/$(SRCARCH)/configs/*/*_defconfig))
board-dirs := $(sort $(notdir $(board-dirs:/=)))
PHONY += help
help:
@echo 'Cleaning targets:'
@echo ' clean - Remove most generated files but keep the config and'
@echo ' enough build support to build external modules'
@echo ' mrproper - Remove all generated files + config + various backup files'
@echo ' distclean - mrproper + remove editor backup and patch files'
@echo ''
@echo 'Configuration targets:'
@$(MAKE) -f $(srctree)/scripts/kconfig/Makefile help
@echo ''
@echo 'Other generic targets:'
@echo ' all - Build all targets marked with [*]'
@echo '* vmlinux - Build the bare kernel'
@echo '* modules - Build all modules'
@echo ' modules_install - Install all modules to INSTALL_MOD_PATH (default: /)'
@echo ' dir/ - Build all files in dir and below'
@echo ' dir/file.[ois] - Build specified target only'
@echo ' dir/file.ll - Build the LLVM assembly file'
@echo ' (requires compiler support for LLVM assembly generation)'
@echo ' dir/file.lst - Build specified mixed source/assembly target only'
@echo ' (requires a recent binutils and recent build (System.map))'
@echo ' dir/file.ko - Build module including final link'
@echo ' modules_prepare - Set up for building external modules'
@echo ' tags/TAGS - Generate tags file for editors'
@echo ' cscope - Generate cscope index'
@echo ' gtags - Generate GNU GLOBAL index'
@echo ' kernelrelease - Output the release version string (use with make -s)'
@echo ' kernelversion - Output the version stored in Makefile (use with make -s)'
@echo ' image_name - Output the image name (use with make -s)'
@echo ' headers_install - Install sanitised kernel headers to INSTALL_HDR_PATH'; \
echo ' (default: $(INSTALL_HDR_PATH))'; \
echo ''
@echo 'Static analysers:'
@echo ' checkstack - Generate a list of stack hogs'
@echo ' versioncheck - Sanity check on version.h usage'
@echo ' includecheck - Check for duplicate included header files'
@echo ' export_report - List the usages of all exported symbols'
@echo ' headerdep - Detect inclusion cycles in headers'
@echo ' coccicheck - Check with Coccinelle'
Makefile: Add clang-tidy and static analyzer support to makefile This patch adds clang-tidy and the clang static-analyzer as make targets. The goal of this patch is to make static analysis tools usable and extendable by any developer or researcher who is familiar with basic c++. The current static analysis tools require intimate knowledge of the internal workings of the static analysis. Clang-tidy and the clang static analyzers expose an easy to use api and allow users unfamiliar with clang to write new checks with relative ease. ===Clang-tidy=== Clang-tidy is an easily extendable 'linter' that runs on the AST. Clang-tidy checks are easy to write and understand. A check consists of two parts, a matcher and a checker. The matcher is created using a domain specific language that acts on the AST (https://clang.llvm.org/docs/LibASTMatchersReference.html). When AST nodes are found by the matcher a callback is made to the checker. The checker can then execute additional checks and issue warnings. Here is an example clang-tidy check to report functions that have calls to local_irq_disable without calls to local_irq_enable and vice-versa. Functions flagged with __attribute((annotation("ignore_irq_balancing"))) are ignored for analysis. (https://reviews.llvm.org/D65828) ===Clang static analyzer=== The clang static analyzer is a more powerful static analysis tool that uses symbolic execution to find bugs. Currently there is a check that looks for potential security bugs from invalid uses of kmalloc and kfree. There are several more general purpose checks that are useful for the kernel. The clang static analyzer is well documented and designed to be extensible. (https://clang-analyzer.llvm.org/checker_dev_manual.html) (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf) The main draw of the clang tools is how accessible they are. The clang documentation is very nice and these tools are built specifically to be easily extendable by any developer. They provide an accessible method of bug-finding and research to people who are not overly familiar with the kernel codebase. Signed-off-by: Nathan Huckleberry <nhuck@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-22 21:56:18 +07:00
@echo ' clang-analyzer - Check with clang static analyzer'
@echo ' clang-tidy - Check with clang-tidy'
@echo ''
@echo 'Tools:'
@echo ' nsdeps - Generate missing symbol namespace dependencies'
@echo ''
@echo 'Kernel selftest:'
@echo ' kselftest - Build and run kernel selftest'
@echo ' Build, install, and boot kernel before'
@echo ' running kselftest on it'
@echo ' Run as root for full coverage'
@echo ' kselftest-all - Build kernel selftest'
@echo ' kselftest-install - Build and install kernel selftest'
@echo ' kselftest-clean - Remove all generated kselftest files'
@echo ' kselftest-merge - Merge all the config dependencies of'
@echo ' kselftest to existing .config.'
@echo ''
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
@$(if $(dtstree), \
echo 'Devicetree:'; \
echo '* dtbs - Build device tree blobs for enabled boards'; \
echo ' dtbs_install - Install dtbs to $(INSTALL_DTBS_PATH)'; \
echo ' dt_binding_check - Validate device tree binding documents'; \
echo ' dtbs_check - Validate device tree source files';\
kbuild: consolidate Devicetree dtb build rules There is nothing arch specific about building dtb files other than their location under /arch/*/boot/dts/. Keeping each arch aligned is a pain. The dependencies and supported targets are all slightly different. Also, a cross-compiler for each arch is needed, but really the host compiler preprocessor is perfectly fine for building dtbs. Move the build rules to a common location and remove the arch specific ones. This is done in a single step to avoid warnings about overriding rules. The build dependencies had been a mixture of 'scripts' and/or 'prepare'. These pull in several dependencies some of which need a target compiler (specifically devicetable-offsets.h) and aren't needed to build dtbs. All that is really needed is dtc, so adjust the dependencies to only be dtc. This change enables support 'dtbs_install' on some arches which were missing the target. Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Paul Burton <paul.burton@mips.com> Acked-by: Ley Foon Tan <ley.foon.tan@intel.com> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Michal Marek <michal.lkml@markovi.net> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Cc: Michal Simek <monstr@monstr.eu> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: James Hogan <jhogan@kernel.org> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Chris Zankel <chris@zankel.net> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: linux-kbuild@vger.kernel.org Cc: linux-snps-arc@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org Cc: uclinux-h8-devel@lists.sourceforge.jp Cc: linux-mips@linux-mips.org Cc: nios2-dev@lists.rocketboards.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-xtensa@linux-xtensa.org Signed-off-by: Rob Herring <robh@kernel.org>
2018-01-11 04:19:37 +07:00
echo '')
@echo 'Userspace tools targets:'
@echo ' use "make tools/help"'
@echo ' or "cd tools; make help"'
@echo ''
@echo 'Kernel packaging:'
@$(MAKE) -f $(srctree)/scripts/Makefile.package help
@echo ''
@echo 'Documentation targets:'
@$(MAKE) -f $(srctree)/Documentation/Makefile dochelp
@echo ''
@echo 'Architecture specific targets ($(SRCARCH)):'
@$(if $(archhelp),$(archhelp),\
echo ' No architecture specific help defined for $(SRCARCH)')
@echo ''
@$(if $(boards), \
$(foreach b, $(boards), \
printf " %-27s - Build for %s\\n" $(b) $(subst _defconfig,,$(b));) \
echo '')
@$(if $(board-dirs), \
$(foreach b, $(board-dirs), \
printf " %-16s - Show %s-specific targets\\n" help-$(b) $(b);) \
printf " %-16s - Show all of the above\\n" help-boards; \
echo '')
@echo ' make V=0|1 [targets] 0 => quiet build (default), 1 => verbose build'
@echo ' make V=2 [targets] 2 => give reason for rebuild of target'
@echo ' make O=dir [targets] Locate all output files in "dir", including .config'
@echo ' make C=1 [targets] Check re-compiled c source with $$CHECK'
@echo ' (sparse by default)'
@echo ' make C=2 [targets] Force check of all c source with $$CHECK'
@echo ' make RECORDMCOUNT_WARN=1 [targets] Warn about ignored mcount sections'
@echo ' make W=n [targets] Enable extra build checks, n=1,2,3 where'
@echo ' 1: warnings which may be relevant and do not occur too often'
@echo ' 2: warnings which occur quite often but may still be relevant'
@echo ' 3: more obscure warnings, can most likely be ignored'
@echo ' Multiple levels can be combined with W=12 or W=123'
@echo ''
@echo 'Execute "make" or "make all" to build all targets marked with [*] '
@echo 'For further info see the ./README file'
help-board-dirs := $(addprefix help-,$(board-dirs))
help-boards: $(help-board-dirs)
boards-per-dir = $(sort $(notdir $(wildcard $(srctree)/arch/$(SRCARCH)/configs/$*/*_defconfig)))
$(help-board-dirs): help-%:
@echo 'Architecture specific targets ($(SRCARCH) $*):'
@$(if $(boards-per-dir), \
$(foreach b, $(boards-per-dir), \
printf " %-24s - Build for %s\\n" $*/$(b) $(subst _defconfig,,$(b));) \
echo '')
# Documentation targets
# ---------------------------------------------------------------------------
DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \
linkcheckdocs dochelp refcheckdocs
Documentation/sphinx: add basic working Sphinx configuration and build Add basic configuration and makefile to build documentation from any .rst files under Documentation using Sphinx. For starters, there's just the placeholder index.rst. At the top level Makefile, hook Sphinx documentation targets alongside (but independent of) the DocBook toolchain, having both be run on the various 'make *docs' targets. All Sphinx processing is placed into Documentation/Makefile.sphinx. Both that and the Documentation/DocBook/Makefile are now expected to handle all the documentation targets, explicitly ignoring them if they're not relevant for that particular toolchain. The changes to the existing DocBook Makefile are kept minimal. There is graceful handling of missing Sphinx and rst2pdf (which is needed for pdf output) by checking for the tool and python module, respectively, with informative messages to the user. If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but otherwise gracefully fall back to the Sphinx default theme, with an informative message to the user, and slightly less pretty HTML output. Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available), epubdocs and xmldocs targets. The output documents are written into per output type subdirectories under Documentation/output. Finally, you can pass options to sphinx-build using the SPHINXBUILD make variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose output from Sphinx. This is based on the original work by Jonathan Corbet, but he probably wouldn't recognize this as his own anymore. Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-19 19:14:05 +07:00
PHONY += $(DOC_TARGETS)
$(DOC_TARGETS):
$(Q)$(MAKE) $(build)=Documentation $@
# Misc
# ---------------------------------------------------------------------------
PHONY += scripts_gdb
scripts_gdb: prepare0
$(Q)$(MAKE) $(build)=scripts/gdb
$(Q)ln -fsn $(abspath $(srctree)/scripts/gdb/vmlinux-gdb.py)
ifdef CONFIG_GDB_SCRIPTS
all: scripts_gdb
endif
else # KBUILD_EXTMOD
###
# External module support.
# When building external modules the kernel used as basis is considered
# read-only, and no consistency checks are made and the make
# system is not used on the basis kernel. If updates are required
# in the basis kernel ordinary make commands (without M=...) must
# be used.
#
# The following are the only valid targets when building external
# modules.
# make M=dir clean Delete all automatically generated files
# make M=dir modules Make all modules in specified dir
# make M=dir Same as 'make M=dir modules'
# make M=dir modules_install
# Install the modules built in the module directory
# Assumes install directory is already created
# We are always building only modules.
KBUILD_BUILTIN :=
KBUILD_MODULES := 1
build-dirs := $(KBUILD_EXTMOD)
PHONY += modules
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
modules: $(MODORDER)
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
$(MODORDER): descend
@:
PHONY += modules_install
modules_install: _emodinst_ _emodinst_post
install-dir := $(if $(INSTALL_MOD_DIR),$(INSTALL_MOD_DIR),extra)
PHONY += _emodinst_
_emodinst_:
$(Q)mkdir -p $(MODLIB)/$(install-dir)
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst
PHONY += _emodinst_post
_emodinst_post: _emodinst_
$(call cmd,depmod)
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
compile_commands.json: $(extmod-prefix)compile_commands.json
PHONY += compile_commands.json
clean-dirs := $(KBUILD_EXTMOD)
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \
$(KBUILD_EXTMOD)/compile_commands.json
PHONY += help
help:
@echo ' Building external modules.'
@echo ' Syntax: make -C path/to/kernel/src M=$$PWD target'
@echo ''
@echo ' modules - default target, build the module(s)'
@echo ' modules_install - install the module'
@echo ' clean - remove generated files in module directory only'
@echo ''
# no-op for external module builds
PHONY += prepare modules_prepare
endif # KBUILD_EXTMOD
# Single targets
# ---------------------------------------------------------------------------
# To build individual files in subdirectories, you can do like this:
#
# make foo/bar/baz.s
#
# The supported suffixes for single-target are listed in 'single-targets'
#
# To build only under specific subdirectories, you can do like this:
#
# make foo/bar/baz/
ifdef single-build
# .ko is special because modpost is needed
single-ko := $(sort $(filter %.ko, $(MAKECMDGOALS)))
single-no-ko := $(sort $(patsubst %.ko,%.mod, $(MAKECMDGOALS)))
$(single-ko): single_modpost
@:
$(single-no-ko): descend
@:
ifeq ($(KBUILD_EXTMOD),)
# For the single build of in-tree modules, use a temporary file to avoid
# the situation of modules_install installing an invalid modules.order.
MODORDER := .modules.tmp
endif
PHONY += single_modpost
single_modpost: $(single-no-ko) modules_prepare
$(Q){ $(foreach m, $(single-ko), echo $(extmod-prefix)$m;) } > $(MODORDER)
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
KBUILD_MODULES := 1
export KBUILD_SINGLE_TARGETS := $(addprefix $(extmod-prefix), $(single-no-ko))
# trim unrelated directories
build-dirs := $(foreach d, $(build-dirs), \
$(if $(filter $(d)/%, $(KBUILD_SINGLE_TARGETS)), $(d)))
endif
ifndef CONFIG_MODULES
KBUILD_MODULES :=
endif
# Handle descending into subdirectories listed in $(build-dirs)
# Preset locale variables to speed up the build process. Limit locale
# tweaks to this spot to avoid wrong language settings when running
# make menuconfig etc.
# Error messages still appears in the original language
PHONY += descend $(build-dirs)
descend: $(build-dirs)
$(build-dirs): prepare
$(Q)$(MAKE) $(build)=$@ \
single-build=$(if $(filter-out $@/, $(filter $@/%, $(KBUILD_SINGLE_TARGETS))),1) \
need-builtin=1 need-modorder=1
clean-dirs := $(addprefix _clean_, $(clean-dirs))
PHONY += $(clean-dirs) clean
$(clean-dirs):
$(Q)$(MAKE) $(clean)=$(patsubst _clean_%,%,$@)
clean: $(clean-dirs)
$(call cmd,rmfiles)
@find $(if $(KBUILD_EXTMOD), $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
\( -name '*.[aios]' -o -name '*.ko' -o -name '.*.cmd' \
-o -name '*.ko.*' \
-o -name '*.dtb' -o -name '*.dtb.S' -o -name '*.dt.yaml' \
-o -name '*.dwo' -o -name '*.lst' \
-o -name '*.su' -o -name '*.mod' \
-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
-o -name '*.lex.c' -o -name '*.tab.[ch]' \
-o -name '*.asn1.[ch]' \
-o -name '*.symtypes' -o -name 'modules.order' \
kbuild: create modules.builtin without Makefile.modbuiltin or tristate.conf Commit bc081dd6e9f6 ("kbuild: generate modules.builtin") added infrastructure to generate modules.builtin, the list of all builtin modules. Basically, it works like this: - Kconfig generates include/config/tristate.conf, the list of tristate CONFIG options with a value in a capital letter. - scripts/Makefile.modbuiltin makes Kbuild descend into directories to collect the information of builtin modules. I am not a big fan of it because Kbuild ends up with traversing the source tree twice. I am not sure how perfectly it should work, but this approach cannot avoid false positives; even if the relevant CONFIG option is tristate, some Makefiles forces obj-m to obj-y. Some examples are: arch/powerpc/platforms/powermac/Makefile: obj-$(CONFIG_NVRAM:m=y) += nvram.o net/ipv6/Makefile: obj-$(subst m,y,$(CONFIG_IPV6)) += inet6_hashtables.o net/netlabel/Makefile: obj-$(subst m,y,$(CONFIG_IPV6)) += netlabel_calipso.o Nobody has complained about (or noticed) it, so it is probably fine to have false positives in modules.builtin. This commit simplifies the implementation. Let's exploit the fact that every module has MODULE_LICENSE(). (modpost shows a warning if MODULE_LICENSE is missing. If so, 0-day bot would already have blocked such a module.) I added MODULE_FILE to <linux/module.h>. When the code is being compiled as builtin, it will be filled with the file path of the module, and collected into modules.builtin.info. Then, scripts/link-vmlinux.sh extracts the list of builtin modules out of it. This new approach fixes the false-positives above, but adds another type of false-positives; non-modular code may have MODULE_LICENSE() by mistake. This is not a big deal, it is just the code is always orphan. We can clean it up if we like. You can see cleanup examples by: $ git log --grep='make.* explicitly non-modular' To sum up, this commits deletes lots of code, but still produces almost equivalent results. Please note it does not increase the vmlinux size at all. As you can see in include/asm-generic/vmlinux.lds.h, the .modinfo section is discarded in the link stage. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2019-12-19 15:33:29 +07:00
-o -name '.tmp_*.o.*' \
GCC plugin infrastructure This patch allows to build the whole kernel with GCC plugins. It was ported from grsecurity/PaX. The infrastructure supports building out-of-tree modules and building in a separate directory. Cross-compilation is supported too. Currently the x86, arm, arm64 and uml architectures enable plugins. The directory of the gcc plugins is scripts/gcc-plugins. You can use a file or a directory there. The plugins compile with these options: * -fno-rtti: gcc is compiled with this option so the plugins must use it too * -fno-exceptions: this is inherited from gcc too * -fasynchronous-unwind-tables: this is inherited from gcc too * -ggdb: it is useful for debugging a plugin (better backtrace on internal errors) * -Wno-narrowing: to suppress warnings from gcc headers (ipa-utils.h) * -Wno-unused-variable: to suppress warnings from gcc headers (gcc_version variable, plugin-version.h) The infrastructure introduces a new Makefile target called gcc-plugins. It supports all gcc versions from 4.5 to 6.0. The scripts/gcc-plugin.sh script chooses the proper host compiler (gcc-4.7 can be built by either gcc or g++). This script also checks the availability of the included headers in scripts/gcc-plugins/gcc-common.h. The gcc-common.h header contains frequently included headers for GCC plugins and it has a compatibility layer for the supported gcc versions. The gcc-generate-*-pass.h headers automatically generate the registration structures for GIMPLE, SIMPLE_IPA, IPA and RTL passes. Note that 'make clean' keeps the *.so files (only the distclean or mrproper targets clean all) because they are needed for out-of-tree modules. Based on work created by the PaX Team. Signed-off-by: Emese Revfy <re.emese@gmail.com> Acked-by: Kees Cook <keescook@chromium.org> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-24 05:09:38 +07:00
-o -name '*.c.[012]*.*' \
-o -name '*.ll' \
-o -name '*.gcno' \) -type f -print | xargs rm -f
# Generate tags for editors
# ---------------------------------------------------------------------------
quiet_cmd_tags = GEN $@
cmd_tags = $(BASH) $(srctree)/scripts/tags.sh $@
tags TAGS cscope gtags: FORCE
$(call cmd,tags)
# Script to generate missing namespace dependencies
# ---------------------------------------------------------------------------
PHONY += nsdeps
nsdeps: export KBUILD_NSDEPS=1
nsdeps: modules
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/nsdeps
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
# Clang Tooling
# ---------------------------------------------------------------------------
quiet_cmd_gen_compile_commands = GEN $@
cmd_gen_compile_commands = $(PYTHON3) $< -a $(AR) -o $@ $(filter-out $<, $(real-prereqs))
Makefile: Add clang-tidy and static analyzer support to makefile This patch adds clang-tidy and the clang static-analyzer as make targets. The goal of this patch is to make static analysis tools usable and extendable by any developer or researcher who is familiar with basic c++. The current static analysis tools require intimate knowledge of the internal workings of the static analysis. Clang-tidy and the clang static analyzers expose an easy to use api and allow users unfamiliar with clang to write new checks with relative ease. ===Clang-tidy=== Clang-tidy is an easily extendable 'linter' that runs on the AST. Clang-tidy checks are easy to write and understand. A check consists of two parts, a matcher and a checker. The matcher is created using a domain specific language that acts on the AST (https://clang.llvm.org/docs/LibASTMatchersReference.html). When AST nodes are found by the matcher a callback is made to the checker. The checker can then execute additional checks and issue warnings. Here is an example clang-tidy check to report functions that have calls to local_irq_disable without calls to local_irq_enable and vice-versa. Functions flagged with __attribute((annotation("ignore_irq_balancing"))) are ignored for analysis. (https://reviews.llvm.org/D65828) ===Clang static analyzer=== The clang static analyzer is a more powerful static analysis tool that uses symbolic execution to find bugs. Currently there is a check that looks for potential security bugs from invalid uses of kmalloc and kfree. There are several more general purpose checks that are useful for the kernel. The clang static analyzer is well documented and designed to be extensible. (https://clang-analyzer.llvm.org/checker_dev_manual.html) (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf) The main draw of the clang tools is how accessible they are. The clang documentation is very nice and these tools are built specifically to be easily extendable by any developer. They provide an accessible method of bug-finding and research to people who are not overly familiar with the kernel codebase. Signed-off-by: Nathan Huckleberry <nhuck@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-22 21:56:18 +07:00
$(extmod-prefix)compile_commands.json: scripts/clang-tools/gen_compile_commands.py \
kbuild: wire up the build rule of compile_commands.json to Makefile Currently, you need to manually run scripts/gen_compile_commands.py to create compile_commands.json. It parses all the .*.cmd files found under the specified directory. If you rebuild the kernel over again without 'make clean', .*.cmd files from older builds will create stale entries in compile_commands.json. This commit wires up the compile_commands.json rule to Makefile, and makes it parse only the .*.cmd files involved in the current build. Pass $(KBUILD_VMLINUX_OBJS), $(KBUILD_VMLINUX_LIBS), and modules.order to the script. The objects or archives linked to vmlinux are listed in $(KBUILD_VMLINUX_OBJS) or $(KBUILD_VMLINUX_LIBS). All the modules are listed in modules.order. You can create compile_commands.json from Make: $ make -j$(nproc) CC=clang compile_commands.json You can also build vmlinux, modules, and compile_commands.json all together in a single command: $ make -j$(nproc) CC=clang all compile_commands.json It works for M= builds as well. In this case, compile_commands.json is created in the top directory of the external module. This is convenient, but it has a drawback; the coverage of the compile_commands.json is reduced because only the objects linked to vmlinux or modules are handled. For example, the following C files are not included in the compile_commands.json: - Decompressor source files (arch/*/boot/) - VDSO source files - C files used to generate intermediates (e.g. kernel/bounds.c) - Standalone host programs I think it is fine for most developers because our main interest is the kernel-space code. If you want to cover all the compiled C files, please build the kernel, then run the script manually as you did before: $ make clean # if you want to remove stale .cmd files [optional] $ make -j$(nproc) CC=clang $ scripts/gen_compile_commands.py Here is a note for out-of-tree builds. 'make compile_commands.json' works with O= option, but please notice compile_commands.json is created in the object tree instead of the source tree. Some people may want to have compile_commands.json in the source tree because Clang Tools searches for it through all parent paths of the first input source file. However, you cannot do this for O= builds. Kbuild should never generate any build artifact in the source tree when O= is given because the source tree might be read-only. Any write attempt to the source tree is monitored and the violation may be reported. See the commit log of 8ef14c2c41d9. So, the only possible way is to create compile_commands.json in the object tree, then specify '-p <build-path>' when you use clang-check, clang-tidy, etc. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nick Desaulniers <ndesaulniers@google.com>
2020-08-22 21:56:16 +07:00
$(if $(KBUILD_EXTMOD),,$(KBUILD_VMLINUX_OBJS) $(KBUILD_VMLINUX_LIBS)) \
$(if $(CONFIG_MODULES), $(MODORDER)) FORCE
$(call if_changed,gen_compile_commands)
targets += $(extmod-prefix)compile_commands.json
Makefile: Add clang-tidy and static analyzer support to makefile This patch adds clang-tidy and the clang static-analyzer as make targets. The goal of this patch is to make static analysis tools usable and extendable by any developer or researcher who is familiar with basic c++. The current static analysis tools require intimate knowledge of the internal workings of the static analysis. Clang-tidy and the clang static analyzers expose an easy to use api and allow users unfamiliar with clang to write new checks with relative ease. ===Clang-tidy=== Clang-tidy is an easily extendable 'linter' that runs on the AST. Clang-tidy checks are easy to write and understand. A check consists of two parts, a matcher and a checker. The matcher is created using a domain specific language that acts on the AST (https://clang.llvm.org/docs/LibASTMatchersReference.html). When AST nodes are found by the matcher a callback is made to the checker. The checker can then execute additional checks and issue warnings. Here is an example clang-tidy check to report functions that have calls to local_irq_disable without calls to local_irq_enable and vice-versa. Functions flagged with __attribute((annotation("ignore_irq_balancing"))) are ignored for analysis. (https://reviews.llvm.org/D65828) ===Clang static analyzer=== The clang static analyzer is a more powerful static analysis tool that uses symbolic execution to find bugs. Currently there is a check that looks for potential security bugs from invalid uses of kmalloc and kfree. There are several more general purpose checks that are useful for the kernel. The clang static analyzer is well documented and designed to be extensible. (https://clang-analyzer.llvm.org/checker_dev_manual.html) (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf) The main draw of the clang tools is how accessible they are. The clang documentation is very nice and these tools are built specifically to be easily extendable by any developer. They provide an accessible method of bug-finding and research to people who are not overly familiar with the kernel codebase. Signed-off-by: Nathan Huckleberry <nhuck@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-22 21:56:18 +07:00
PHONY += clang-tidy clang-analyzer
ifdef CONFIG_CC_IS_CLANG
quiet_cmd_clang_tools = CHECK $<
cmd_clang_tools = $(PYTHON3) $(srctree)/scripts/clang-tools/run-clang-tools.py $@ $<
clang-tidy clang-analyzer: $(extmod-prefix)compile_commands.json
$(call cmd,clang_tools)
else
clang-tidy clang-analyzer:
@echo "$@ requires CC=clang" >&2
@false
endif
# Scripts to check various things for consistency
# ---------------------------------------------------------------------------
scripts: remove namespace.pl namespace.pl is intended to help locate symbols which are defined but are not used externally. The goal is to avoid bloat of the namespace in the resulting kernel image. The script relies on object data, and only finds unused symbols for the configuration used to generate that object data. This results in a lot of false positive warnings such as symbols only used by a single architecture, or symbols which are used externally only under certain configurations. Running namespace.pl using allyesconfig, allmodconfig, and x86_64_defconfig yields the following results: * allmodconfig * 11122 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * allyesconfig * 10997 unique symbol names with no external reference * 1194 symbols listed as multiply defined * 214 symbols it can't resolve * x86_64_defconfig * 5757 unique symbol names with no external reference * 528 symbols listed as multiply defined * 154 symbols it can't resolve The script also has no way to easily limit the scope of the checks to a given subset of the kernel, such as only checking for symbols defined within a module or subsystem. Discussion on public mailing lists seems to indicate that many view the tool output as suspect or not very useful (see discussions at [1] and [2] for further context). As described by Masahiro Yamada at [2], namespace.pl provides 3 types of checks: listing multiply defined symbols, resolving external symbols, and warnings about symbols with no reference. The first category of issues is easily caught by the linker as any set of multiply defined symbols should fail to link. The second category of issues is also caught by linking, as undefined symbols would cause issues. Even with modules, these types of issues where a module relies on an external symbol are caught by modpost. The remaining category of issues reported is the list of symbols with no external reference, and is the primary motivation of this script. However, it ought to be clear from the above examples that the output is difficult to sort through. Even allyesconfig has ~10000 entries. The current submit-checklist indicates that patches ought to go through namespacecheck and fix any new issues arising. But that itself presents problems. As described at [1], many cases of reports are due to configuration where a function is used externally by some configuration settings. Prominent maintainers appear to dislike changes modify code such that symbols become static based on CONFIG_* flags ([3], and [4]) One possible solution is to adjust the advice and indicate that we only care about the output of namespacecheck on allyesconfig or allmodconfig builds... However, given the discussion at [2], I suspect that few people are actively using this tool. It doesn't have a maintainer in the MAINTAINERS flie, and it produces so many warnings for unused symbols that it is difficult to use effectively. Thus, I propose we simply remove it. [1] https://lore.kernel.org/netdev/20200708164812.384ae8ea@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/ [2] https://lore.kernel.org/lkml/20190129204319.15238-1-jacob.e.keller@intel.com/ [3] https://lore.kernel.org/netdev/20190828.154744.2058157956381129672.davem@davemloft.net/ [4] https://lore.kernel.org/netdev/20190827210928.576c5fef@cakuba.netronome.com/ Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Acked-by: Randy Dunlap <rdunlap@infradead.org> Acked-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-10-10 07:18:44 +07:00
PHONY += includecheck versioncheck coccicheck export_report
includecheck:
find $(srctree)/* $(RCS_FIND_IGNORE) \
-name '*.[hcS]' -type f -print | sort \
| xargs $(PERL) -w $(srctree)/scripts/checkincludes.pl
versioncheck:
find $(srctree)/* $(RCS_FIND_IGNORE) \
-name '*.[hcS]' -type f -print | sort \
| xargs $(PERL) -w $(srctree)/scripts/checkversion.pl
coccicheck:
$(Q)$(BASH) $(srctree)/scripts/$@
export_report:
$(PERL) $(srctree)/scripts/export_report.pl
PHONY += checkstack kernelrelease kernelversion image_name
# UML needs a little special treatment here. It wants to use the host
# toolchain, so needs $(SUBARCH) passed to checkstack.pl. Everyone
# else wants $(ARCH), including people doing cross-builds, which means
# that $(SUBARCH) doesn't work here.
ifeq ($(ARCH), um)
CHECKSTACK_ARCH := $(SUBARCH)
else
CHECKSTACK_ARCH := $(ARCH)
endif
checkstack:
$(OBJDUMP) -d vmlinux $$(find . -name '*.ko') | \
$(PERL) $(srctree)/scripts/checkstack.pl $(CHECKSTACK_ARCH)
kernelrelease:
@echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"
kernelversion:
@echo $(KERNELVERSION)
image_name:
@echo $(KBUILD_IMAGE)
# Clear a bunch of variables before executing the submake
ifeq ($(quiet),silent_)
tools_silent=s
endif
tools/: FORCE
tools: Honour the O= flag when tool build called from a higher Makefile Honour the O= flag that was passed to a higher level Makefile and then passed down as part of a tool build. To make this work, the top-level Makefile passes the original O= flag and subdir=tools to the tools/Makefile, and that in turn passes subdir=$(O)/$(subdir)/foodir when building tool foo in directory $(O)/$(subdir)/foodir (where the intervening slashes aren't added if an element is missing). For example, take perf. This is found in tools/perf/. Assume we're building into directory ~/zebra/, so we pass O=~/zebra to make. Dependening on where we run the build from, we see: make run in dir $(OUTPUT) dir ======================= ================== linux ~/zebra/tools/perf/ linux/tools ~/zebra/perf/ linux/tools/perf ~/zebra/ and if O= is not set, we get: make run in dir $(OUTPUT) dir ======================= ================== linux linux/tools/perf/ linux/tools linux/tools/perf/ linux/tools/perf linux/tools/perf/ The output directories are created by the descend function if they don't already exist. Signed-off-by: David Howells <dhowells@redhat.com> Cc: Borislav Petkov <bp@amd64.org> Cc: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/1378.1352379110@warthog.procyon.org.uk Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-11-06 04:02:08 +07:00
$(Q)mkdir -p $(objtree)/tools
$(Q)$(MAKE) LDFLAGS= MAKEFLAGS="$(tools_silent) $(filter --j% -j,$(MAKEFLAGS))" O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/
tools/%: FORCE
tools: Honour the O= flag when tool build called from a higher Makefile Honour the O= flag that was passed to a higher level Makefile and then passed down as part of a tool build. To make this work, the top-level Makefile passes the original O= flag and subdir=tools to the tools/Makefile, and that in turn passes subdir=$(O)/$(subdir)/foodir when building tool foo in directory $(O)/$(subdir)/foodir (where the intervening slashes aren't added if an element is missing). For example, take perf. This is found in tools/perf/. Assume we're building into directory ~/zebra/, so we pass O=~/zebra to make. Dependening on where we run the build from, we see: make run in dir $(OUTPUT) dir ======================= ================== linux ~/zebra/tools/perf/ linux/tools ~/zebra/perf/ linux/tools/perf ~/zebra/ and if O= is not set, we get: make run in dir $(OUTPUT) dir ======================= ================== linux linux/tools/perf/ linux/tools linux/tools/perf/ linux/tools/perf linux/tools/perf/ The output directories are created by the descend function if they don't already exist. Signed-off-by: David Howells <dhowells@redhat.com> Cc: Borislav Petkov <bp@amd64.org> Cc: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/1378.1352379110@warthog.procyon.org.uk Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-11-06 04:02:08 +07:00
$(Q)mkdir -p $(objtree)/tools
$(Q)$(MAKE) LDFLAGS= MAKEFLAGS="$(tools_silent) $(filter --j% -j,$(MAKEFLAGS))" O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ $*
quiet_cmd_rmfiles = $(if $(wildcard $(rm-files)),CLEAN $(wildcard $(rm-files)))
cmd_rmfiles = rm -rf $(rm-files)
# Run depmod only if we have System.map and depmod is executable
quiet_cmd_depmod = DEPMOD $(KERNELRELEASE)
cmd_depmod = $(CONFIG_SHELL) $(srctree)/scripts/depmod.sh $(DEPMOD) \
$(KERNELRELEASE)
kbuild: let fixdep directly write to .*.cmd files Currently, fixdep writes dependencies to .*.tmp, which is renamed to .*.cmd after everything succeeds. This is a very safe way to avoid corrupted .*.cmd files. The if_changed_dep has carried this safety mechanism since it was added in 2002. If fixdep fails for some reasons or a user terminates the build while fixdep is running, the incomplete output from the fixdep could be troublesome. This is my insight about some bad scenarios: [1] If the compiler succeeds to generate *.o file, but fixdep fails to write necessary dependencies to .*.cmd file, Make will miss to rebuild the object when headers or CONFIG options are changed. In this case, fixdep should not generate .*.cmd file at all so that 'arg-check' will surely trigger the rebuild of the object. [2] A partially constructed .*.cmd file may not be a syntactically correct makefile. The next time Make runs, it would include it, then fail to parse it. Once this happens, 'make clean' is be the only way to fix it. In fact, [1] is no longer a problem since commit 9c2af1c7377a ("kbuild: add .DELETE_ON_ERROR special target"). Make deletes a target file on any failure in its recipe. Because fixdep is a part of the recipe of *.o target, if it fails, the *.o is deleted anyway. However, I am a bit worried about the slight possibility of [2]. So, here is a solution. Let fixdep directly write to a .*.cmd file, but allow makefiles to include it only when its corresponding target exists. This effectively reverts commit 2982c953570b ("kbuild: remove redundant $(wildcard ...) for cmd_files calculation"), and commit 00d78ab2ba75 ("kbuild: remove dead code in cmd_files calculation in top Makefile") because now we must check the presence of targets. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-30 08:05:22 +07:00
# read saved command lines for existing targets
existing-targets := $(wildcard $(sort $(targets)))
-include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd)
endif # config-build
endif # mixed-build
endif # need-sub-make
PHONY += FORCE
FORCE:
# Declare the contents of the PHONY variable as phony. We keep that
# information in a variable so we can use it in if_changed and friends.
.PHONY: $(PHONY)