When we are logging to stderr we are previously relying on libkmod
sending it to the default location in case we are not asked to log to
syslog. The problem is that modprobe may be used in scripts that don't
want to log to syslog (since they are not daemons, like scripts to
generate initrd) and then it's difficult to know where the message comes
from.
This patch treats only the messages coming from libkmod.
Once we read all we need from a module, unref it so any resource taken
by it (including the mmap to access the file in libkmod) will be
dropped. This drastically reduces the number of open file descriptors
and also the memory needed, with no performance penalties. Rather,
there's a small speedup of ~2.6%.
Running depmod in a laptop with 2973 modules and comparing the number of
open file descriptors for kmod-10, before and after the last patches to
depmod (caaf438cb6 and HEAD) we have:
Before: 2980 simultaneously open fds
After: 7 simultaneously open fds
kmod-10: 7 simultaneously open fds
So now we have the speedup of caching the file in kmod_module without
the drawback of increasing the number of open file descriptors.
In depmod_module_add() we already called kmod_module_get_name() and
copied the string to our struct. Use it instead of calling again and
again the libkmod function.
The overall goal is to coalesce the accesses to a file that is the
backend of a module. This commit addresses the calls to
kmod_module_get_get_dependency_symbols(). Calling it earlier, while we
are iterating the modules allows us to free the struct kmod of each
module much sooner. We are still not freeing it since there are other
places that must be refactored first.
There's a performance penalty of ~2.5% from previous commit.
The overall goal is to coalesce the accesses to a file that is the
backend of a module. This commit addresses the calls to
kmod_module_get_info(). Calling it earlier, while we are iterating the
modules allows us to free the struct kmod of each module much sooner. We
are still not freeing it since there are other places that must be
refactored first.
A nice side effect is that this commit reduces in ~33% the calls to
malloc(), giving a speedup of ~6% for cold caches (reproduced on only 1
laptop).
Before this commit the build system failed at late state with
non-helpful message when xsltproc was not available.
Making all in man
GEN depmod.d.5
/bin/sh: --nonet: command not found
make[2]: *** [depmod.d.5] Error 127
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
If we are accessing several times the modules and reading some sections
by sucessive calls to the functions below, we are incurring in a penalty
of having to open, parse the header and close the file. For each
function.
- kmod_module_get_info()
- kmod_module_get_versions()
- kmod_module_get_symbols()
- kmod_module_get_dependency_symbols()
These functions are particularly important to depmod. It calls all of
them, for each module. Moreover there's a huge bottleneck in the open
operation if we are using compression. Every time we open the module we
need to uncompress the file and after getting the information we need we
discard the result. This is clearly shown by profiling depmod with perf
(record + report), using compressed modules:
64.07% depmod libz.so.1.2.7 [.] 0x00000000000074b8 ◆
18.18% depmod libz.so.1.2.7 [.] crc32 ▒
2.42% depmod libz.so.1.2.7 [.] inflate ▒
1.17% depmod libc-2.16.so [.] __memcpy_ssse3_back ▒
0.96% depmod [kernel.kallsyms] [k] copy_user_generic_string ▒
0.89% depmod libc-2.16.so [.] __strcmp_sse42 ▒
0.82% depmod [kernel.kallsyms] [k] hrtimer_interrupt ▒
0.77% depmod libc-2.16.so [.] _int_malloc ▒
0.44% depmod kmod-nolib [.] kmod_elf_get_strings ▒
0.41% depmod kmod-nolib [.] kmod_elf_get_dependency_symbols ▒
0.37% depmod kmod-nolib [.] kmod_elf_get_section ▒
0.36% depmod kmod-nolib [.] kmod_elf_get_symbols
...
Average of running depmod 5 times, dropping caches between them, in a
slow spinning disk:
Before: 12.25 +- 0.20
After: 8.20 +- 0.21
m-i-t: 9.62 +- 0.27
So this patch leads to an improvement of ~33% over unpatched version,
ending up with 15% speedup over module-init-tools.
Although the hash table implementation allows passing a callback function
to free a value when it is removed from the hash table, hash_del() wasn't
freeing it if it was provided. Now it does.
As a bonus, it now checks if the callback is set in hash_add() as well.
This gives the test cases the ability to supply files that must be
checked after the test is run, rather than just checking stdout/stderr.
This is intended to be used with tools that generate files, like depmod.
It includes a poor's man implementation of a "check for differences in
files". Not really optimized, but it's simple enough and does what it
proposes to.
We now index the modules by uncompressed-relative-path instead of
relative-path. This is because the file modules.order, coming from
kernel, always comes with uncompressed paths. This fixes the issue of
not sorting the aliases correctly due to paths not matching when using
compressed modules.
This is a broken option that only leads to misery and incompatabilities
with other systems. Kbuild doesn't come close to supporting directories
other than /lib/modules with several targets simply failing without
hacky fixes. Simply remove the option and all traces of it, as it
doesn't make sense in today's world.
Clang doesn't treat unknown warnings flags as an error, but rather as a
warning. The result is that the detection for whic CFLAGS are supported
by this compiler will not work, since the compilation will succeed.
With this patch we now successfully detect clang doesn't support
-Wlogical-op, as opposed to previous behavior:
checking if clang supports flag -Wlogical-op in envvar CFLAGS... no
We use this macro only for LDFLAGS and CFLAGS, so it's safe to stash
-Werror there.
All tests should be in testsuite. The remaining tests in this directory
are not relevant enough to be ported. git log can be consulted if in
future we decide to put them in testsuite.
As discussed with Rusty Russel, it would be nice to remove the related
code from kernel. Deprecate its use on kmod, so people know they
shouldn't be using it.
This fixes a change in behavior regarding kmod and module-init-tools:
when trying to load a module by alias, we should check if it's
blacklisted, regardless of the command line arguments passed.
This was reported by "Dmitry V. Levin <ldv@altlinux.org>".
With this flag kmod_module_probe_insert_module() check if module is
blacklisted only if it's also an alias. This is needed in order to allow
blacklisting a module by name and effectively blacklisting all its
aliases as module-init-tools was doing.
Before this patch we could load pcspkr module as follows:
/etc/modprobe.d/test.conf:
alias yay pcspkr
blacklist pcspkr
$ modprobe yay
Now libkmod has support to blacklist "yay" because "pcspkr" is blacklisted.
Instead of replicating the same code several times, define and use macros for
the various types of wrapped functions in the testsuite's path.c LD_PRELOAD
wrapper.
Add various __xstat() variants and open64(), which are being used when enabling
large file support.
1) Embedded systems often don't want man pages on the
target; rather than pointlessly building them, then ignoring
the result, allow just not building them at all
2) When bootstrapping an operating systems, documentation is the
source of many cyclical dependencies, and allowing it to
be explicitly disabled is useful for earlier build passes.
