Currently, namespace_from_kstrtabns() relies on the fact that
namespace strings are recorded in the __ksymtab_strings section.
Actually, it is coded in include/linux/export.h, but modpost does
not need to hard-code the section name.
Elf_Sym::st_shndx holds the index of the relevant section. Using it is
a more portable way to get the namespace string.
Make namespace_from_kstrtabns() simply call sym_get_data(), and delete
the info->ksymtab_strings .
While I was here, I added more 'const' qualifiers to pointers.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The modpost, with the -d option given, generates per-module .ns_deps
files.
Kbuild generates per-module .mod files to carry module information.
This is convenient because Make handles multiple jobs in parallel
when the -j option is given.
On the other hand, the modpost always runs as a single thread.
I do not see a strong reason to produce separate .ns_deps files.
This commit changes the modpost to generate just one file,
modules.nsdeps, each line of which has the following format:
<module_name>: <list of missing namespaces>
Please note it contains *missing* namespaces instead of required ones.
So, modules.nsdeps is empty if the namespace dependency is all good.
This will work more efficiently because spatch will no longer process
already imported namespaces. I removed the '(if needed)' from the
nsdeps log since spatch is invoked only when needed.
This also solves the stale .ns_deps problem reported by Jessica Yu:
https://lkml.org/lkml/2019/10/28/467
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Jessica Yu <jeyu@kernel.org>
Acked-by: Jessica Yu <jeyu@kernel.org>
Reviewed-by: Matthias Maennich <maennich@google.com>
Tested-by: Matthias Maennich <maennich@google.com>
The introduction of Symbol Namespaces changed the naming schema of the
__ksymtab entries from __kysmtab__symbol to __ksymtab_NAMESPACE.symbol.
That caused some breakages in tools that depend on the name layout in
either the binaries(vmlinux,*.ko) or in System.map. E.g. kmod's depmod
would not be able to read System.map without a patch to support symbol
namespaces. A warning reported by depmod for namespaced symbols would
look like
depmod: WARNING: [...]/uas.ko needs unknown symbol usb_stor_adjust_quirks
In order to address this issue, revert to the original naming scheme and
rather read the __kstrtabns_<symbol> entries and their corresponding
values from __ksymtab_strings to update the namespace values for
symbols. After having read all symbols and handled them in
handle_modversions(), the symbols are created. In a second pass, read
the __kstrtabns_ entries and update the namespaces accordingly.
Fixes: 8651ec01da ("module: add support for symbol namespaces.")
Reported-by: Stefan Wahren <stefan.wahren@i2se.com>
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
This patch adds an option to modpost to generate a <module>.ns_deps file
per module, containing the namespace dependencies for that module.
E.g. if the linked module my-module.ko would depend on the symbol
myfunc.MY_NS in the namespace MY_NS, the my-module.ns_deps file created
by modpost would contain the entry MY_NS to express the namespace
dependency of my-module imposed by using the symbol myfunc.
These files can subsequently be used by static analysis tools (like
coccinelle scripts) to address issues with missing namespace imports. A
later patch of this series will introduce such a script 'nsdeps' and a
corresponding make target to automatically add missing
MODULE_IMPORT_NS() definitions to the module's sources. For that it uses
the information provided in the generated .ns_deps files.
Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
Add support for symbols that are exported into namespaces. For that,
extract any namespace suffix from the symbol name. In addition, emit a
warning whenever a module refers to an exported symbol without
explicitly importing the namespace that it is defined in. This patch
consistently adds the namespace suffix to symbol names exported into
Module.symvers.
Example warning emitted by modpost in case of the above violation:
WARNING: module ums-usbat uses symbol usb_stor_resume from namespace
USB_STORAGE, but does not import it.
Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Certain platforms (e. g. BSD-based ones) define some ELF constants
according to host. This patch fixes problems with cross-building
Linux kernel on these platforms (e. g. building ARM 32-bit version
on x86-64 host).
Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
LTO turns all global symbols effectively into statics. This
has the side effect that they all have a .NUMBER postfix to make
them unique. In modpost drop this postfix because it confuses
it.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1391846481-31491-8-git-send-email-ak@linux.intel.com
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Commit f02e8a6596 ("module: Sort exported symbols") sorts symbols
placing each of them in its own elf section. This sorting and merging
into the canonical sections are done by the linker.
Unfortunately modpost to generate Module.symvers file parses vmlinux.o
(which is not linked yet) and all modules object files (which aren't
linked yet). These aren't sanitized by the linker yet. That breaks
modpost that can't detect license properly for modules.
This patch makes modpost aware of the new exported symbols structure.
[ This above is a slightly corrected version of the explanation of the
problem, copied from commit 62a2635610 ("modpost: Fix modpost's
license checking V3"). That commit fixed the problem for module
object files, but not for vmlinux.o. This patch fixes modpost for
vmlinux.o. ]
Signed-off-by: Frank Rowand <frank.rowand@am.sony.com>
Signed-off-by: Alessio Igor Bogani <abogani@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Binutils 2.18.50 made a backwards-incompatible change in the way it
writes ELF objects with over 65280 sections, to improve conformance
with the ELF specification and interoperability with other ELF tools.
Specifically, it no longer adds 256 to section indices SHN_LORESERVE
and higher to skip over the reserved range SHN_LORESERVE through
SHN_HIRESERVE; those values are only considered special in the
st_shndx field, and not in other places where section indices are
stored. See:
http://sourceware.org/bugzilla/show_bug.cgi?id=5900http://groups.google.com/group/generic-abi/browse_thread/thread/e8bb63714b072e67/6c63738f12cc8a17
Signed-off-by: Anders Kaseorg <andersk@ksplice.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This patch makes modpost able to process object files with more than
64k sections. Needed for huge kernel builds (allyesconfig, for example)
with -ffunction-sections. 64k sections handling is covered, for example,
by this document:
"IA-64 gABI Proposal 74: Section Indexes"
http://www.codesourcery.com/public/cxx-abi/abi/prop-74-sindex.html
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Andi Kleen <andi@firstfloor.org>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Remove the unnecessary functions and variables.
Signed-off-by: Wenji Huang <wenji.huang@oracle.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The module alias support in the kernel have a consistency
check where it is checked that the size of a structure
in the kernel and on the build host are the same.
For cross builds this check does not make sense so detect
when we do cross builds and silently skip the check in these
situations.
This fixes a build bug for a wireless driver when cross building
for arm.
Acked-by: Michael Buesch <mb@bu3sch.de>
Tested-by: Gordon Farquharson <gordonfarquharson@gmail.com>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: stable@kernel.org
This adds some new magic in the MODPOST phase for CONFIG_MARKERS. Analogous
to the Module.symvers file, the build will now write a Module.markers file
when CONFIG_MARKERS=y is set. This file lists the name, defining module, and
format string of each marker, separated by \t characters. This simple text
file can be used by offline build procedures for instrumentation code,
analogous to how System.map and Module.symvers can be useful to have for
kernels other than the one you are running right now.
The strings are made easy to extract by having the __trace_mark macro define
the name and format together in a single array called __mstrtab_* in the
__markers_strings section. This is straightforward and reliable as long as
the marker structs are always defined by this macro. It is an unreasonable
amount of hairy work to extract the string pointers from the __markers section
structs, which entails handling a relocation type for every machine under the
sun.
Mathieu :
- Ran through checkpatch.pl
Signed-off-by: Roland McGrath <roland@redhat.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Cc: David Smith <dsmith@redhat.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The relocation record sometimes contained an address
which was not an exactly match for a symbol.
Implment some simple logic such that if there
is a symbol within 20 bytes of the address contained
in the relocation record then print the name of this
symbol.
With this change modpost could find symbol names
for the remaining .init.text symbols in my
allyesconfig build for x86_64.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
The Elfnn_Section is not available on all platforms,
noteworthy are cygwin.
Use the safe replacement _Half.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
On i386 and MIPS, warn_sec_mismatch() sometimes fails to show
usefull symbol name. This is because empty 'refsym' due to 0 r_addend
value. This patch is to adjust r_addend value, consulting with
apply_relocate() routine in kernel code.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
This reverts commit f892b7d480, which
totally broke the build on x86 with CONFIG_RELOCATABLE (which, as far as
I can tell, is the only case where it should even matter!) due to a
SIGSEGV in modpost.
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
On i386, ARM and MIPS, warn_sec_mismatch() sometimes fails to show
usefull symbol name. This is because empty 'refsym' due to 0 r_addend
value. This patch is to adjust r_addend value, consulting with
apply_relocate() routine in kernel code.
Without this patch:
MODPOST vmlinux
WARNING: init/built-in.o - Section mismatch: reference to .init.text: from .text between 'rest_init' (at offset 0xf4) and 'try_name'
WARNING: mm/built-in.o - Section mismatch: reference to .init.text: from .text between 'kmem_cache_create' (at offset 0x18a39) and 'cache_reap'
WARNING: mm/built-in.o - Section mismatch: reference to .init.text: from .text between 'kmem_cache_create' (at offset 0x18a6b) and 'cache_reap'
With this patch:
MODPOST vmlinux
WARNING: mm/built-in.o - Section mismatch: reference to .init.text:set_up_list3s from .text between 'kmem_cache_create' (at offset 0x18a39) and 'cache_reap'
WARNING: mm/built-in.o - Section mismatch: reference to .init.text:set_up_list3s from .text between 'kmem_cache_create' (at offset 0x18a6b) and 'cache_reap'
Now modpost can detect "kernel_init" name (and whitelist it) and show
"set_up_list3s" name.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Some of modpost's warnings are fatal, and some are not. Adopt the
compiler distinction between errors and warnings by calling merror()
for fatal diagnostics and warn() for non-fatal ones.
merror() was used as replacemtn for error() to avoid clash with glibc
Signed-off-by: Matthew Wilcox <matthew@wil.cx>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
We now have infrastructure in place to mark an EXPORTed symbol
as unused. So the natural next step is to warn during buildtime when
a module uses a symbol marked UNUSED.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Modules that uses GPL symbols can no longer be build with kbuild,
the build will fail during the modpost step.
When a GPL-incompatible module uses a EXPORT_SYMBOL_GPL_FUTURE symbol
then warn during modpost so author are actually notified.
The actual license compatibility check is shared with the kernel
to make sure it is in sync.
Patch originally from: Andreas Gruenbacher <agruen@suse.de> and
Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
This patch provides the ability to identify the export-type of each
exported symbols in Module.symvers.
NOTE: It updates the Module.symvers file with the additional
information as shown below.
0x0f8b92af platform_device_add_resources vmlinux EXPORT_SYMBOL_GPL
0xcf7efb2a ethtool_op_set_tx_csum vmlinux EXPORT_SYMBOL
Signed-off-by: Andreas Gruenbacher <agruen@suse.de>
Signed-off-by: Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Avantika Mathur <mathur@us.ibm.com>
Signed-off-by: Valdis Kletnieks <valdis.kletnieks@vt.edu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Here is an updated r_info layout fix. Please apply "check SHT_REL
sections" patch before this.
64bit mips has different r_info layout. This patch fixes modpost
segfault for 64bit little endian mips kernel.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
I found that modpost can not detect section mismatch on mips and i386. On
mips64, the modpost (with r_info layout fix) can detect it. The current
modpst only checks SHT_RELA section but I suppose SHT_REL section should be
checked also. This patch does not contain r_info layout fix. I'll post an
updated r_info layout fix on next mail.
Check SHT_REL sections as like as SHT_RELA sections to detect section
mismatch.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This reverts commit c8d8b837eb, which
caused problems for the x86 build. Quoth Sam:
"It was discussed on mips list but apparently the fix was bogus. I
will not have time to look into it so mips can carry this local fix
until we get a proper fix in mainline."
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
64bit mips has different r_info layout. This patch fixes modpost
segfault for 64bit little endian mips kernel.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Section mismatch is identified as references to .init*
sections from non .init sections. And likewise references
to .exit.* sections outside .exit sections.
.init.* sections are discarded after a module is initialized
and references to .init.* sections are oops candidates.
.exit.* sections are discarded when a module is built-in and
thus references to .exit are also oops candidates.
The checks were possible to do using 'make buildcheck' which
called the two perl scripts: reference_discarded.pl and
reference_init.pl. This patch just moves the same functionality
inside modpost and the scripts are then obsoleted.
They will though be kept for a while so users can do double
checks - but note that some .o files are skipped by the perl scripts
so result is not 1:1.
All credit for the concept goes to Keith Owens who implemented
the original perl scrips - this patch just moves it to modpost.
Compared to the perl script the implmentation in modpost will be run
for each kernel build - thus catching the error much sooner, but
the downside is that the individual .o file are not always identified.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
