License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 21:07:57 +07:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0 */
|
2010-11-16 04:37:37 +07:00
|
|
|
#ifndef __KERNEL_PRINTK__
|
|
|
|
#define __KERNEL_PRINTK__
|
|
|
|
|
2013-04-30 06:17:19 +07:00
|
|
|
#include <stdarg.h>
|
2011-05-25 07:13:20 +07:00
|
|
|
#include <linux/init.h>
|
2012-07-31 04:40:11 +07:00
|
|
|
#include <linux/kern_levels.h>
|
2013-05-21 15:51:10 +07:00
|
|
|
#include <linux/linkage.h>
|
2014-01-24 06:54:16 +07:00
|
|
|
#include <linux/cache.h>
|
2011-05-25 07:13:20 +07:00
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
extern const char linux_banner[];
|
|
|
|
extern const char linux_proc_banner[];
|
|
|
|
|
2016-12-13 07:45:50 +07:00
|
|
|
#define PRINTK_MAX_SINGLE_HEADER_LEN 2
|
|
|
|
|
2012-07-31 04:40:09 +07:00
|
|
|
static inline int printk_get_level(const char *buffer)
|
|
|
|
{
|
2012-07-31 04:40:17 +07:00
|
|
|
if (buffer[0] == KERN_SOH_ASCII && buffer[1]) {
|
2012-07-31 04:40:09 +07:00
|
|
|
switch (buffer[1]) {
|
|
|
|
case '0' ... '7':
|
printk: reinstate KERN_CONT for printing continuation lines
Long long ago the kernel log buffer was a buffered stream of bytes, very
much like stdio in user space. It supported log levels by scanning the
stream and noticing the log level markers at the beginning of each line,
but if you wanted to print a partial line in multiple chunks, you just
did multiple printk() calls, and it just automatically worked.
Except when it didn't, and you had very confusing output when different
lines got all mixed up with each other. Then you got fragment lines
mixing with each other, or with non-fragment lines, because it was
traditionally impossible to tell whether a printk() call was a
continuation or not.
To at least help clarify the issue of continuation lines, we added a
KERN_CONT marker back in 2007 to mark continuation lines:
474925277671 ("printk: add KERN_CONT annotation").
That continuation marker was initially an empty string, and didn't
actuall make any semantic difference. But it at least made it possible
to annotate the source code, and have check-patch notice that a printk()
didn't need or want a log level marker, because it was a continuation of
a previous line.
To avoid the ambiguity between a continuation line that had that
KERN_CONT marker, and a printk with no level information at all, we then
in 2009 made KERN_CONT be a real log level marker which meant that we
could now reliably tell the difference between the two cases.
5fd29d6ccbc9 ("printk: clean up handling of log-levels and newlines")
and we could take advantage of that to make sure we didn't mix up
continuation lines with lines that just didn't have any loglevel at all.
Then, in 2012, the kernel log buffer was changed to be a "record" based
log, where each line was a record that has a loglevel and a timestamp.
You can see the beginning of that conversion in commits
e11fea92e13f ("kmsg: export printk records to the /dev/kmsg interface")
7ff9554bb578 ("printk: convert byte-buffer to variable-length record buffer")
with a number of follow-up commits to fix some painful fallout from that
conversion. Over all, it took a couple of months to sort out most of
it. But the upside was that you could have concurrent readers (and
writers) of the kernel log and not have lines with mixed output in them.
And one particular pain-point for the record-based kernel logging was
exactly the fragmentary lines that are generated in smaller chunks. In
order to still log them as one recrod, the continuation lines need to be
attached to the previous record properly.
However the explicit continuation record marker that is actually useful
for this exact case was actually removed in aroundm the same time by commit
61e99ab8e35a ("printk: remove the now unnecessary "C" annotation for KERN_CONT")
due to the incorrect belief that KERN_CONT wasn't meaningful. The
ambiguity between "is this a continuation line" or "is this a plain
printk with no log level information" was reintroduced, and in fact
became an even bigger pain point because there was now the whole
record-level merging of kernel messages going on.
This patch reinstates the KERN_CONT as a real non-empty string marker,
so that the ambiguity is fixed once again.
But it's not a plain revert of that original removal: in the four years
since we made KERN_CONT an empty string again, not only has the format
of the log level markers changed, we've also had some usage changes in
this area.
For example, some ACPI code seems to use KERN_CONT _together_ with a log
level, and now uses both the KERN_CONT marker and (for example) a
KERN_INFO marker to show that it's an informational continuation of a
line.
Which is actually not a bad idea - if the continuation line cannot be
attached to its predecessor, without the log level information we don't
know what log level to assign to it (and we traditionally just assigned
it the default loglevel). So having both a log level and the KERN_CONT
marker is not necessarily a bad idea, but it does mean that we need to
actually iterate over potentially multiple markers, rather than just a
single one.
Also, since KERN_CONT was still conceptually needed, and encouraged, but
didn't actually _do_ anything, we've also had the reverse problem:
rather than having too many annotations it has too few, and there is bit
rot with code that no longer marks the continuation lines with the
KERN_CONT marker.
So this patch not only re-instates the non-empty KERN_CONT marker, it
also fixes up the cases of bit-rot I noticed in my own logs.
There are probably other cases where KERN_CONT will be needed to be
added, either because it is new code that never dealt with the need for
KERN_CONT, or old code that has bitrotted without anybody noticing.
That said, we should strive to avoid the need for KERN_CONT. It does
result in real problems for logging, and should generally not be seen as
a good feature. If we some day can get rid of the feature entirely,
because nobody does any fragmented printk calls, that would be lovely.
But until that point, let's at mark the code that relies on the hacky
multi-fragment kernel printk's. Not only does it avoid the ambiguity,
it also annotates code as "maybe this would be good to fix some day".
(That said, particularly during single-threaded bootup, the downsides of
KERN_CONT are very limited. Things get much hairier when you have
multiple threads going on and user level reading and writing logs too).
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-09 10:32:40 +07:00
|
|
|
case 'c': /* KERN_CONT */
|
2012-07-31 04:40:09 +07:00
|
|
|
return buffer[1];
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline const char *printk_skip_level(const char *buffer)
|
|
|
|
{
|
2014-04-04 04:48:38 +07:00
|
|
|
if (printk_get_level(buffer))
|
|
|
|
return buffer + 2;
|
|
|
|
|
2012-07-31 04:40:09 +07:00
|
|
|
return buffer;
|
|
|
|
}
|
|
|
|
|
2016-12-13 07:45:47 +07:00
|
|
|
static inline const char *printk_skip_headers(const char *buffer)
|
|
|
|
{
|
|
|
|
while (printk_get_level(buffer))
|
|
|
|
buffer = printk_skip_level(buffer);
|
|
|
|
|
|
|
|
return buffer;
|
|
|
|
}
|
|
|
|
|
2015-06-26 05:01:24 +07:00
|
|
|
#define CONSOLE_EXT_LOG_MAX 8192
|
|
|
|
|
2014-06-05 06:11:46 +07:00
|
|
|
/* printk's without a loglevel use this.. */
|
2014-08-07 06:09:01 +07:00
|
|
|
#define MESSAGE_LOGLEVEL_DEFAULT CONFIG_MESSAGE_LOGLEVEL_DEFAULT
|
2014-06-05 06:11:46 +07:00
|
|
|
|
|
|
|
/* We show everything that is MORE important than this.. */
|
|
|
|
#define CONSOLE_LOGLEVEL_SILENT 0 /* Mum's the word */
|
|
|
|
#define CONSOLE_LOGLEVEL_MIN 1 /* Minimum loglevel we let people use */
|
|
|
|
#define CONSOLE_LOGLEVEL_DEBUG 10 /* issue debug messages */
|
|
|
|
#define CONSOLE_LOGLEVEL_MOTORMOUTH 15 /* You can't shut this one up */
|
|
|
|
|
2016-12-13 07:45:56 +07:00
|
|
|
/*
|
2018-06-19 18:57:26 +07:00
|
|
|
* Default used to be hard-coded at 7, quiet used to be hardcoded at 4,
|
|
|
|
* we're now allowing both to be set from kernel config.
|
2016-12-13 07:45:56 +07:00
|
|
|
*/
|
|
|
|
#define CONSOLE_LOGLEVEL_DEFAULT CONFIG_CONSOLE_LOGLEVEL_DEFAULT
|
2018-06-19 18:57:26 +07:00
|
|
|
#define CONSOLE_LOGLEVEL_QUIET CONFIG_CONSOLE_LOGLEVEL_QUIET
|
2016-12-13 07:45:56 +07:00
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
extern int console_printk[];
|
|
|
|
|
|
|
|
#define console_loglevel (console_printk[0])
|
|
|
|
#define default_message_loglevel (console_printk[1])
|
|
|
|
#define minimum_console_loglevel (console_printk[2])
|
|
|
|
#define default_console_loglevel (console_printk[3])
|
|
|
|
|
2011-01-13 07:59:43 +07:00
|
|
|
static inline void console_silent(void)
|
|
|
|
{
|
2014-06-05 06:11:46 +07:00
|
|
|
console_loglevel = CONSOLE_LOGLEVEL_SILENT;
|
2011-01-13 07:59:43 +07:00
|
|
|
}
|
|
|
|
|
|
|
|
static inline void console_verbose(void)
|
|
|
|
{
|
|
|
|
if (console_loglevel)
|
2014-06-05 06:11:46 +07:00
|
|
|
console_loglevel = CONSOLE_LOGLEVEL_MOTORMOUTH;
|
2011-01-13 07:59:43 +07:00
|
|
|
}
|
|
|
|
|
2016-08-03 04:04:07 +07:00
|
|
|
/* strlen("ratelimit") + 1 */
|
|
|
|
#define DEVKMSG_STR_MAX_SIZE 10
|
|
|
|
extern char devkmsg_log_str[];
|
|
|
|
struct ctl_table;
|
|
|
|
|
panic: avoid the extra noise dmesg
When kernel panic happens, it will first print the panic call stack,
then the ending msg like:
[ 35.743249] ---[ end Kernel panic - not syncing: Fatal exception
[ 35.749975] ------------[ cut here ]------------
The above message are very useful for debugging.
But if system is configured to not reboot on panic, say the
"panic_timeout" parameter equals 0, it will likely print out many noisy
message like WARN() call stack for each and every CPU except the panic
one, messages like below:
WARNING: CPU: 1 PID: 280 at kernel/sched/core.c:1198 set_task_cpu+0x183/0x190
Call Trace:
<IRQ>
try_to_wake_up
default_wake_function
autoremove_wake_function
__wake_up_common
__wake_up_common_lock
__wake_up
wake_up_klogd_work_func
irq_work_run_list
irq_work_tick
update_process_times
tick_sched_timer
__hrtimer_run_queues
hrtimer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
For people working in console mode, the screen will first show the panic
call stack, but immediately overridden by these noisy extra messages,
which makes debugging much more difficult, as the original context gets
lost on screen.
Also these noisy messages will confuse some users, as I have seen many bug
reporters posted the noisy message into bugzilla, instead of the real
panic call stack and context.
Adding a flag "suppress_printk" which gets set in panic() to avoid those
noisy messages, without changing current kernel behavior that both panic
blinking and sysrq magic key can work as is, suggested by Petr Mladek.
To verify this, make sure kernel is not configured to reboot on panic and
in console
# echo c > /proc/sysrq-trigger
to see if console only prints out the panic call stack.
Link: http://lkml.kernel.org/r/1551430186-24169-1-git-send-email-feng.tang@intel.com
Signed-off-by: Feng Tang <feng.tang@intel.com>
Suggested-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Acked-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Kees Cook <keescook@chromium.org>
Cc: Borislav Petkov <bp@suse.de>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jiri Slaby <jslaby@suse.com>
Cc: Sasha Levin <sashal@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-15 05:45:34 +07:00
|
|
|
extern int suppress_printk;
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
struct va_format {
|
|
|
|
const char *fmt;
|
|
|
|
va_list *va;
|
|
|
|
};
|
|
|
|
|
|
|
|
/*
|
|
|
|
* FW_BUG
|
|
|
|
* Add this to a message where you are sure the firmware is buggy or behaves
|
|
|
|
* really stupid or out of spec. Be aware that the responsible BIOS developer
|
|
|
|
* should be able to fix this issue or at least get a concrete idea of the
|
|
|
|
* problem by reading your message without the need of looking at the kernel
|
|
|
|
* code.
|
|
|
|
*
|
|
|
|
* Use it for definite and high priority BIOS bugs.
|
|
|
|
*
|
|
|
|
* FW_WARN
|
|
|
|
* Use it for not that clear (e.g. could the kernel messed up things already?)
|
|
|
|
* and medium priority BIOS bugs.
|
|
|
|
*
|
|
|
|
* FW_INFO
|
|
|
|
* Use this one if you want to tell the user or vendor about something
|
|
|
|
* suspicious, but generally harmless related to the firmware.
|
|
|
|
*
|
|
|
|
* Use it for information or very low priority BIOS bugs.
|
|
|
|
*/
|
|
|
|
#define FW_BUG "[Firmware Bug]: "
|
|
|
|
#define FW_WARN "[Firmware Warn]: "
|
|
|
|
#define FW_INFO "[Firmware Info]: "
|
|
|
|
|
|
|
|
/*
|
|
|
|
* HW_ERR
|
|
|
|
* Add this to a message for hardware errors, so that user can report
|
|
|
|
* it to hardware vendor instead of LKML or software vendor.
|
|
|
|
*/
|
|
|
|
#define HW_ERR "[Hardware Error]: "
|
|
|
|
|
2013-12-23 20:29:42 +07:00
|
|
|
/*
|
|
|
|
* DEPRECATED
|
|
|
|
* Add this to a message whenever you want to warn user space about the use
|
|
|
|
* of a deprecated aspect of an API so they can stop using it
|
|
|
|
*/
|
|
|
|
#define DEPRECATED "[Deprecated]: "
|
|
|
|
|
2011-01-13 07:59:45 +07:00
|
|
|
/*
|
|
|
|
* Dummy printk for disabled debugging statements to use whilst maintaining
|
2016-01-16 07:59:12 +07:00
|
|
|
* gcc's format checking.
|
2011-01-13 07:59:45 +07:00
|
|
|
*/
|
2016-07-05 05:31:26 +07:00
|
|
|
#define no_printk(fmt, ...) \
|
|
|
|
({ \
|
2017-09-17 22:01:44 +07:00
|
|
|
if (0) \
|
|
|
|
printk(fmt, ##__VA_ARGS__); \
|
2016-07-05 05:31:26 +07:00
|
|
|
0; \
|
|
|
|
})
|
2011-01-13 07:59:45 +07:00
|
|
|
|
2013-04-30 06:17:18 +07:00
|
|
|
#ifdef CONFIG_EARLY_PRINTK
|
2011-11-01 07:11:33 +07:00
|
|
|
extern asmlinkage __printf(1, 2)
|
2011-01-13 07:59:45 +07:00
|
|
|
void early_printk(const char *fmt, ...);
|
2013-04-30 06:17:18 +07:00
|
|
|
#else
|
|
|
|
static inline __printf(1, 2) __cold
|
|
|
|
void early_printk(const char *s, ...) { }
|
|
|
|
#endif
|
2011-01-13 07:59:45 +07:00
|
|
|
|
2016-05-21 07:00:33 +07:00
|
|
|
#ifdef CONFIG_PRINTK_NMI
|
|
|
|
extern void printk_nmi_enter(void);
|
|
|
|
extern void printk_nmi_exit(void);
|
2018-06-27 21:20:28 +07:00
|
|
|
extern void printk_nmi_direct_enter(void);
|
|
|
|
extern void printk_nmi_direct_exit(void);
|
2016-05-21 07:00:33 +07:00
|
|
|
#else
|
|
|
|
static inline void printk_nmi_enter(void) { }
|
|
|
|
static inline void printk_nmi_exit(void) { }
|
2018-06-27 21:20:28 +07:00
|
|
|
static inline void printk_nmi_direct_enter(void) { }
|
|
|
|
static inline void printk_nmi_direct_exit(void) { }
|
2016-05-21 07:00:33 +07:00
|
|
|
#endif /* PRINTK_NMI */
|
2014-11-21 21:16:58 +07:00
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
#ifdef CONFIG_PRINTK
|
2012-05-03 07:29:13 +07:00
|
|
|
asmlinkage __printf(5, 0)
|
|
|
|
int vprintk_emit(int facility, int level,
|
|
|
|
const char *dict, size_t dictlen,
|
|
|
|
const char *fmt, va_list args);
|
|
|
|
|
2011-11-01 07:11:33 +07:00
|
|
|
asmlinkage __printf(1, 0)
|
2011-01-13 07:59:45 +07:00
|
|
|
int vprintk(const char *fmt, va_list args);
|
2012-05-03 07:29:13 +07:00
|
|
|
|
2011-11-01 07:11:33 +07:00
|
|
|
asmlinkage __printf(1, 2) __cold
|
2011-01-13 07:59:45 +07:00
|
|
|
int printk(const char *fmt, ...);
|
2010-11-16 04:37:37 +07:00
|
|
|
|
2012-02-27 16:47:00 +07:00
|
|
|
/*
|
2014-06-05 06:11:40 +07:00
|
|
|
* Special printk facility for scheduler/timekeeping use only, _DO_NOT_USE_ !
|
2012-02-27 16:47:00 +07:00
|
|
|
*/
|
2014-06-05 06:11:40 +07:00
|
|
|
__printf(1, 2) __cold int printk_deferred(const char *fmt, ...);
|
2012-02-27 16:47:00 +07:00
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
/*
|
|
|
|
* Please don't use printk_ratelimit(), because it shares ratelimiting state
|
|
|
|
* with all other unrelated printk_ratelimit() callsites. Instead use
|
|
|
|
* printk_ratelimited() or plain old __ratelimit().
|
|
|
|
*/
|
|
|
|
extern int __printk_ratelimit(const char *func);
|
|
|
|
#define printk_ratelimit() __printk_ratelimit(__func__)
|
|
|
|
extern bool printk_timed_ratelimit(unsigned long *caller_jiffies,
|
|
|
|
unsigned int interval_msec);
|
|
|
|
|
|
|
|
extern int printk_delay_msec;
|
|
|
|
extern int dmesg_restrict;
|
|
|
|
|
2016-08-03 04:04:07 +07:00
|
|
|
extern int
|
|
|
|
devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, void __user *buf,
|
|
|
|
size_t *lenp, loff_t *ppos);
|
|
|
|
|
2013-03-23 05:04:39 +07:00
|
|
|
extern void wake_up_klogd(void);
|
|
|
|
|
2015-01-27 03:58:43 +07:00
|
|
|
char *log_buf_addr_get(void);
|
|
|
|
u32 log_buf_len_get(void);
|
crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE
Patch series "kexec/fadump: remove dependency with CONFIG_KEXEC and
reuse crashkernel parameter for fadump", v4.
Traditionally, kdump is used to save vmcore in case of a crash. Some
architectures like powerpc can save vmcore using architecture specific
support instead of kexec/kdump mechanism. Such architecture specific
support also needs to reserve memory, to be used by dump capture kernel.
crashkernel parameter can be a reused, for memory reservation, by such
architecture specific infrastructure.
This patchset removes dependency with CONFIG_KEXEC for crashkernel
parameter and vmcoreinfo related code as it can be reused without kexec
support. Also, crashkernel parameter is reused instead of
fadump_reserve_mem to reserve memory for fadump.
The first patch moves crashkernel parameter parsing and vmcoreinfo
related code under CONFIG_CRASH_CORE instead of CONFIG_KEXEC_CORE. The
second patch reuses the definitions of append_elf_note() & final_note()
functions under CONFIG_CRASH_CORE in IA64 arch code. The third patch
removes dependency on CONFIG_KEXEC for firmware-assisted dump (fadump)
in powerpc. The next patch reuses crashkernel parameter for reserving
memory for fadump, instead of the fadump_reserve_mem parameter. This
has the advantage of using all syntaxes crashkernel parameter supports,
for fadump as well. The last patch updates fadump kernel documentation
about use of crashkernel parameter.
This patch (of 5):
Traditionally, kdump is used to save vmcore in case of a crash. Some
architectures like powerpc can save vmcore using architecture specific
support instead of kexec/kdump mechanism. Such architecture specific
support also needs to reserve memory, to be used by dump capture kernel.
crashkernel parameter can be a reused, for memory reservation, by such
architecture specific infrastructure.
But currently, code related to vmcoreinfo and parsing of crashkernel
parameter is built under CONFIG_KEXEC_CORE. This patch introduces
CONFIG_CRASH_CORE and moves the above mentioned code under this config,
allowing code reuse without dependency on CONFIG_KEXEC. There is no
functional change with this patch.
Link: http://lkml.kernel.org/r/149035338104.6881.4550894432615189948.stgit@hbathini.in.ibm.com
Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-05-09 05:56:18 +07:00
|
|
|
void log_buf_vmcoreinfo_setup(void);
|
2011-05-25 07:13:20 +07:00
|
|
|
void __init setup_log_buf(int early);
|
2015-07-18 06:23:42 +07:00
|
|
|
__printf(1, 2) void dump_stack_set_arch_desc(const char *fmt, ...);
|
2013-05-01 05:27:12 +07:00
|
|
|
void dump_stack_print_info(const char *log_lvl);
|
dump_stack: unify debug information printed by show_regs()
show_regs() is inherently arch-dependent but it does make sense to print
generic debug information and some archs already do albeit in slightly
different forms. This patch introduces a generic function to print debug
information from show_regs() so that different archs print out the same
information and it's much easier to modify what's printed.
show_regs_print_info() prints out the same debug info as dump_stack()
does plus task and thread_info pointers.
* Archs which didn't print debug info now do.
alpha, arc, blackfin, c6x, cris, frv, h8300, hexagon, ia64, m32r,
metag, microblaze, mn10300, openrisc, parisc, score, sh64, sparc,
um, xtensa
* Already prints debug info. Replaced with show_regs_print_info().
The printed information is superset of what used to be there.
arm, arm64, avr32, mips, powerpc, sh32, tile, unicore32, x86
* s390 is special in that it used to print arch-specific information
along with generic debug info. Heiko and Martin think that the
arch-specific extra isn't worth keeping s390 specfic implementation.
Converted to use the generic version.
Note that now all archs print the debug info before actual register
dumps.
An example BUG() dump follows.
kernel BUG at /work/os/work/kernel/workqueue.c:4841!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.9.0-rc1-work+ #7
Hardware name: empty empty/S3992, BIOS 080011 10/26/2007
task: ffff88007c85e040 ti: ffff88007c860000 task.ti: ffff88007c860000
RIP: 0010:[<ffffffff8234a07e>] [<ffffffff8234a07e>] init_workqueues+0x4/0x6
RSP: 0000:ffff88007c861ec8 EFLAGS: 00010246
RAX: ffff88007c861fd8 RBX: ffffffff824466a8 RCX: 0000000000000001
RDX: 0000000000000046 RSI: 0000000000000001 RDI: ffffffff8234a07a
RBP: ffff88007c861ec8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffffffff8234a07a
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88015f7ff000 CR3: 00000000021f1000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88007c861ef8 ffffffff81000312 ffffffff824466a8 ffff88007c85e650
0000000000000003 0000000000000000 ffff88007c861f38 ffffffff82335e5d
ffff88007c862080 ffffffff8223d8c0 ffff88007c862080 ffffffff81c47760
Call Trace:
[<ffffffff81000312>] do_one_initcall+0x122/0x170
[<ffffffff82335e5d>] kernel_init_freeable+0x9b/0x1c8
[<ffffffff81c47760>] ? rest_init+0x140/0x140
[<ffffffff81c4776e>] kernel_init+0xe/0xf0
[<ffffffff81c6be9c>] ret_from_fork+0x7c/0xb0
[<ffffffff81c47760>] ? rest_init+0x140/0x140
...
v2: Typo fix in x86-32.
v3: CPU number dropped from show_regs_print_info() as
dump_stack_print_info() has been updated to print it. s390
specific implementation dropped as requested by s390 maintainers.
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
Cc: Mike Frysinger <vapier@gentoo.org>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Acked-by: Chris Metcalf <cmetcalf@tilera.com> [tile bits]
Acked-by: Richard Kuo <rkuo@codeaurora.org> [hexagon bits]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-05-01 05:27:17 +07:00
|
|
|
void show_regs_print_info(const char *log_lvl);
|
2018-02-13 14:28:34 +07:00
|
|
|
extern asmlinkage void dump_stack(void) __cold;
|
printk: introduce per-cpu safe_print seq buffer
This patch extends the idea of NMI per-cpu buffers to regions
that may cause recursive printk() calls and possible deadlocks.
Namely, printk() can't handle printk calls from schedule code
or printk() calls from lock debugging code (spin_dump() for instance);
because those may be called with `sem->lock' already taken or any
other `critical' locks (p->pi_lock, etc.). An example of deadlock
can be
vprintk_emit()
console_unlock()
up() << raw_spin_lock_irqsave(&sem->lock, flags);
wake_up_process()
try_to_wake_up()
ttwu_queue()
ttwu_activate()
activate_task()
enqueue_task()
enqueue_task_fair()
cfs_rq_of()
task_of()
WARN_ON_ONCE(!entity_is_task(se))
vprintk_emit()
console_trylock()
down_trylock()
raw_spin_lock_irqsave(&sem->lock, flags)
^^^^ deadlock
and some other cases.
Just like in NMI implementation, the solution uses a per-cpu
`printk_func' pointer to 'redirect' printk() calls to a 'safe'
callback, that store messages in a per-cpu buffer and flushes
them back to logbuf buffer later.
Usage example:
printk()
printk_safe_enter_irqsave(flags)
//
// any printk() call from here will endup in vprintk_safe(),
// that stores messages in a special per-CPU buffer.
//
printk_safe_exit_irqrestore(flags)
The 'redirection' mechanism, though, has been reworked, as suggested
by Petr Mladek. Instead of using a per-cpu @print_func callback we now
keep a per-cpu printk-context variable and call either default or nmi
vprintk function depending on its value. printk_nmi_entrer/exit and
printk_safe_enter/exit, thus, just set/celar corresponding bits in
printk-context functions.
The patch only adds printk_safe support, we don't use it yet.
Link: http://lkml.kernel.org/r/20161227141611.940-4-sergey.senozhatsky@gmail.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Tejun Heo <tj@kernel.org>
Cc: Calvin Owens <calvinowens@fb.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Hurley <peter@hurleysoftware.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2016-12-27 21:16:06 +07:00
|
|
|
extern void printk_safe_init(void);
|
|
|
|
extern void printk_safe_flush(void);
|
|
|
|
extern void printk_safe_flush_on_panic(void);
|
2010-11-16 04:37:37 +07:00
|
|
|
#else
|
2011-11-01 07:11:33 +07:00
|
|
|
static inline __printf(1, 0)
|
2011-01-13 07:59:45 +07:00
|
|
|
int vprintk(const char *s, va_list args)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
2011-11-01 07:11:33 +07:00
|
|
|
static inline __printf(1, 2) __cold
|
2011-01-13 07:59:45 +07:00
|
|
|
int printk(const char *s, ...)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
2012-02-27 16:47:00 +07:00
|
|
|
static inline __printf(1, 2) __cold
|
2014-06-05 06:11:40 +07:00
|
|
|
int printk_deferred(const char *s, ...)
|
2012-02-27 16:47:00 +07:00
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
2011-01-13 07:59:45 +07:00
|
|
|
static inline int printk_ratelimit(void)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
static inline bool printk_timed_ratelimit(unsigned long *caller_jiffies,
|
|
|
|
unsigned int interval_msec)
|
|
|
|
{
|
|
|
|
return false;
|
|
|
|
}
|
2010-11-16 04:37:37 +07:00
|
|
|
|
2013-03-23 05:04:39 +07:00
|
|
|
static inline void wake_up_klogd(void)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2015-01-27 03:58:43 +07:00
|
|
|
static inline char *log_buf_addr_get(void)
|
|
|
|
{
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline u32 log_buf_len_get(void)
|
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
crash: move crashkernel parsing and vmcore related code under CONFIG_CRASH_CORE
Patch series "kexec/fadump: remove dependency with CONFIG_KEXEC and
reuse crashkernel parameter for fadump", v4.
Traditionally, kdump is used to save vmcore in case of a crash. Some
architectures like powerpc can save vmcore using architecture specific
support instead of kexec/kdump mechanism. Such architecture specific
support also needs to reserve memory, to be used by dump capture kernel.
crashkernel parameter can be a reused, for memory reservation, by such
architecture specific infrastructure.
This patchset removes dependency with CONFIG_KEXEC for crashkernel
parameter and vmcoreinfo related code as it can be reused without kexec
support. Also, crashkernel parameter is reused instead of
fadump_reserve_mem to reserve memory for fadump.
The first patch moves crashkernel parameter parsing and vmcoreinfo
related code under CONFIG_CRASH_CORE instead of CONFIG_KEXEC_CORE. The
second patch reuses the definitions of append_elf_note() & final_note()
functions under CONFIG_CRASH_CORE in IA64 arch code. The third patch
removes dependency on CONFIG_KEXEC for firmware-assisted dump (fadump)
in powerpc. The next patch reuses crashkernel parameter for reserving
memory for fadump, instead of the fadump_reserve_mem parameter. This
has the advantage of using all syntaxes crashkernel parameter supports,
for fadump as well. The last patch updates fadump kernel documentation
about use of crashkernel parameter.
This patch (of 5):
Traditionally, kdump is used to save vmcore in case of a crash. Some
architectures like powerpc can save vmcore using architecture specific
support instead of kexec/kdump mechanism. Such architecture specific
support also needs to reserve memory, to be used by dump capture kernel.
crashkernel parameter can be a reused, for memory reservation, by such
architecture specific infrastructure.
But currently, code related to vmcoreinfo and parsing of crashkernel
parameter is built under CONFIG_KEXEC_CORE. This patch introduces
CONFIG_CRASH_CORE and moves the above mentioned code under this config,
allowing code reuse without dependency on CONFIG_KEXEC. There is no
functional change with this patch.
Link: http://lkml.kernel.org/r/149035338104.6881.4550894432615189948.stgit@hbathini.in.ibm.com
Signed-off-by: Hari Bathini <hbathini@linux.vnet.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-05-09 05:56:18 +07:00
|
|
|
static inline void log_buf_vmcoreinfo_setup(void)
|
2010-11-16 04:37:37 +07:00
|
|
|
{
|
|
|
|
}
|
2011-05-25 07:13:20 +07:00
|
|
|
|
|
|
|
static inline void setup_log_buf(int early)
|
|
|
|
{
|
|
|
|
}
|
2013-05-01 05:27:12 +07:00
|
|
|
|
2015-07-18 06:23:42 +07:00
|
|
|
static inline __printf(1, 2) void dump_stack_set_arch_desc(const char *fmt, ...)
|
2013-05-01 05:27:15 +07:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
2013-05-01 05:27:12 +07:00
|
|
|
static inline void dump_stack_print_info(const char *log_lvl)
|
|
|
|
{
|
|
|
|
}
|
dump_stack: unify debug information printed by show_regs()
show_regs() is inherently arch-dependent but it does make sense to print
generic debug information and some archs already do albeit in slightly
different forms. This patch introduces a generic function to print debug
information from show_regs() so that different archs print out the same
information and it's much easier to modify what's printed.
show_regs_print_info() prints out the same debug info as dump_stack()
does plus task and thread_info pointers.
* Archs which didn't print debug info now do.
alpha, arc, blackfin, c6x, cris, frv, h8300, hexagon, ia64, m32r,
metag, microblaze, mn10300, openrisc, parisc, score, sh64, sparc,
um, xtensa
* Already prints debug info. Replaced with show_regs_print_info().
The printed information is superset of what used to be there.
arm, arm64, avr32, mips, powerpc, sh32, tile, unicore32, x86
* s390 is special in that it used to print arch-specific information
along with generic debug info. Heiko and Martin think that the
arch-specific extra isn't worth keeping s390 specfic implementation.
Converted to use the generic version.
Note that now all archs print the debug info before actual register
dumps.
An example BUG() dump follows.
kernel BUG at /work/os/work/kernel/workqueue.c:4841!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.9.0-rc1-work+ #7
Hardware name: empty empty/S3992, BIOS 080011 10/26/2007
task: ffff88007c85e040 ti: ffff88007c860000 task.ti: ffff88007c860000
RIP: 0010:[<ffffffff8234a07e>] [<ffffffff8234a07e>] init_workqueues+0x4/0x6
RSP: 0000:ffff88007c861ec8 EFLAGS: 00010246
RAX: ffff88007c861fd8 RBX: ffffffff824466a8 RCX: 0000000000000001
RDX: 0000000000000046 RSI: 0000000000000001 RDI: ffffffff8234a07a
RBP: ffff88007c861ec8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffffffff8234a07a
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88015f7ff000 CR3: 00000000021f1000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff88007c861ef8 ffffffff81000312 ffffffff824466a8 ffff88007c85e650
0000000000000003 0000000000000000 ffff88007c861f38 ffffffff82335e5d
ffff88007c862080 ffffffff8223d8c0 ffff88007c862080 ffffffff81c47760
Call Trace:
[<ffffffff81000312>] do_one_initcall+0x122/0x170
[<ffffffff82335e5d>] kernel_init_freeable+0x9b/0x1c8
[<ffffffff81c47760>] ? rest_init+0x140/0x140
[<ffffffff81c4776e>] kernel_init+0xe/0xf0
[<ffffffff81c6be9c>] ret_from_fork+0x7c/0xb0
[<ffffffff81c47760>] ? rest_init+0x140/0x140
...
v2: Typo fix in x86-32.
v3: CPU number dropped from show_regs_print_info() as
dump_stack_print_info() has been updated to print it. s390
specific implementation dropped as requested by s390 maintainers.
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
Cc: Mike Frysinger <vapier@gentoo.org>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Acked-by: Chris Metcalf <cmetcalf@tilera.com> [tile bits]
Acked-by: Richard Kuo <rkuo@codeaurora.org> [hexagon bits]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-05-01 05:27:17 +07:00
|
|
|
|
|
|
|
static inline void show_regs_print_info(const char *log_lvl)
|
|
|
|
{
|
|
|
|
}
|
printk: introduce per-cpu safe_print seq buffer
This patch extends the idea of NMI per-cpu buffers to regions
that may cause recursive printk() calls and possible deadlocks.
Namely, printk() can't handle printk calls from schedule code
or printk() calls from lock debugging code (spin_dump() for instance);
because those may be called with `sem->lock' already taken or any
other `critical' locks (p->pi_lock, etc.). An example of deadlock
can be
vprintk_emit()
console_unlock()
up() << raw_spin_lock_irqsave(&sem->lock, flags);
wake_up_process()
try_to_wake_up()
ttwu_queue()
ttwu_activate()
activate_task()
enqueue_task()
enqueue_task_fair()
cfs_rq_of()
task_of()
WARN_ON_ONCE(!entity_is_task(se))
vprintk_emit()
console_trylock()
down_trylock()
raw_spin_lock_irqsave(&sem->lock, flags)
^^^^ deadlock
and some other cases.
Just like in NMI implementation, the solution uses a per-cpu
`printk_func' pointer to 'redirect' printk() calls to a 'safe'
callback, that store messages in a per-cpu buffer and flushes
them back to logbuf buffer later.
Usage example:
printk()
printk_safe_enter_irqsave(flags)
//
// any printk() call from here will endup in vprintk_safe(),
// that stores messages in a special per-CPU buffer.
//
printk_safe_exit_irqrestore(flags)
The 'redirection' mechanism, though, has been reworked, as suggested
by Petr Mladek. Instead of using a per-cpu @print_func callback we now
keep a per-cpu printk-context variable and call either default or nmi
vprintk function depending on its value. printk_nmi_entrer/exit and
printk_safe_enter/exit, thus, just set/celar corresponding bits in
printk-context functions.
The patch only adds printk_safe support, we don't use it yet.
Link: http://lkml.kernel.org/r/20161227141611.940-4-sergey.senozhatsky@gmail.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Tejun Heo <tj@kernel.org>
Cc: Calvin Owens <calvinowens@fb.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Hurley <peter@hurleysoftware.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2016-12-27 21:16:06 +07:00
|
|
|
|
2019-01-04 06:26:37 +07:00
|
|
|
static inline void dump_stack(void)
|
2018-02-13 14:28:34 +07:00
|
|
|
{
|
|
|
|
}
|
|
|
|
|
printk: introduce per-cpu safe_print seq buffer
This patch extends the idea of NMI per-cpu buffers to regions
that may cause recursive printk() calls and possible deadlocks.
Namely, printk() can't handle printk calls from schedule code
or printk() calls from lock debugging code (spin_dump() for instance);
because those may be called with `sem->lock' already taken or any
other `critical' locks (p->pi_lock, etc.). An example of deadlock
can be
vprintk_emit()
console_unlock()
up() << raw_spin_lock_irqsave(&sem->lock, flags);
wake_up_process()
try_to_wake_up()
ttwu_queue()
ttwu_activate()
activate_task()
enqueue_task()
enqueue_task_fair()
cfs_rq_of()
task_of()
WARN_ON_ONCE(!entity_is_task(se))
vprintk_emit()
console_trylock()
down_trylock()
raw_spin_lock_irqsave(&sem->lock, flags)
^^^^ deadlock
and some other cases.
Just like in NMI implementation, the solution uses a per-cpu
`printk_func' pointer to 'redirect' printk() calls to a 'safe'
callback, that store messages in a per-cpu buffer and flushes
them back to logbuf buffer later.
Usage example:
printk()
printk_safe_enter_irqsave(flags)
//
// any printk() call from here will endup in vprintk_safe(),
// that stores messages in a special per-CPU buffer.
//
printk_safe_exit_irqrestore(flags)
The 'redirection' mechanism, though, has been reworked, as suggested
by Petr Mladek. Instead of using a per-cpu @print_func callback we now
keep a per-cpu printk-context variable and call either default or nmi
vprintk function depending on its value. printk_nmi_entrer/exit and
printk_safe_enter/exit, thus, just set/celar corresponding bits in
printk-context functions.
The patch only adds printk_safe support, we don't use it yet.
Link: http://lkml.kernel.org/r/20161227141611.940-4-sergey.senozhatsky@gmail.com
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Tejun Heo <tj@kernel.org>
Cc: Calvin Owens <calvinowens@fb.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Hurley <peter@hurleysoftware.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2016-12-27 21:16:06 +07:00
|
|
|
static inline void printk_safe_init(void)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void printk_safe_flush(void)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void printk_safe_flush_on_panic(void)
|
|
|
|
{
|
|
|
|
}
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|
|
|
|
|
2017-11-13 23:50:59 +07:00
|
|
|
extern int kptr_restrict;
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
#ifndef pr_fmt
|
|
|
|
#define pr_fmt(fmt) fmt
|
|
|
|
#endif
|
|
|
|
|
2014-06-05 06:11:44 +07:00
|
|
|
/*
|
|
|
|
* These can be used to print at the various log levels.
|
|
|
|
* All of these will print unconditionally, although note that pr_debug()
|
|
|
|
* and other debug macros are compiled out unless either DEBUG is defined
|
|
|
|
* or CONFIG_DYNAMIC_DEBUG is set.
|
|
|
|
*/
|
2016-08-10 00:48:18 +07:00
|
|
|
#define pr_emerg(fmt, ...) \
|
|
|
|
printk(KERN_EMERG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_alert(fmt, ...) \
|
|
|
|
printk(KERN_ALERT pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_crit(fmt, ...) \
|
|
|
|
printk(KERN_CRIT pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_err(fmt, ...) \
|
|
|
|
printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_warning(fmt, ...) \
|
|
|
|
printk(KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_warn pr_warning
|
|
|
|
#define pr_notice(fmt, ...) \
|
|
|
|
printk(KERN_NOTICE pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_info(fmt, ...) \
|
|
|
|
printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
|
2015-04-16 06:16:59 +07:00
|
|
|
/*
|
|
|
|
* Like KERN_CONT, pr_cont() should only be used when continuing
|
|
|
|
* a line with no newline ('\n') enclosed. Otherwise it defaults
|
|
|
|
* back to KERN_DEFAULT.
|
|
|
|
*/
|
2010-11-16 04:37:37 +07:00
|
|
|
#define pr_cont(fmt, ...) \
|
|
|
|
printk(KERN_CONT fmt, ##__VA_ARGS__)
|
|
|
|
|
|
|
|
/* pr_devel() should produce zero code unless DEBUG is defined */
|
|
|
|
#ifdef DEBUG
|
|
|
|
#define pr_devel(fmt, ...) \
|
|
|
|
printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#else
|
|
|
|
#define pr_devel(fmt, ...) \
|
2011-01-13 07:59:45 +07:00
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|
|
|
|
|
2013-10-27 10:41:53 +07:00
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
/* If you are writing a driver, please use dev_dbg instead */
|
2011-12-20 05:11:18 +07:00
|
|
|
#if defined(CONFIG_DYNAMIC_DEBUG)
|
2016-08-03 04:03:47 +07:00
|
|
|
#include <linux/dynamic_debug.h>
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
/* dynamic_pr_debug() uses pr_fmt() internally so we don't need it here */
|
|
|
|
#define pr_debug(fmt, ...) \
|
|
|
|
dynamic_pr_debug(fmt, ##__VA_ARGS__)
|
2011-12-20 05:11:18 +07:00
|
|
|
#elif defined(DEBUG)
|
|
|
|
#define pr_debug(fmt, ...) \
|
|
|
|
printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
2010-11-16 04:37:37 +07:00
|
|
|
#else
|
|
|
|
#define pr_debug(fmt, ...) \
|
2011-01-13 07:59:45 +07:00
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|
|
|
|
|
2011-01-13 07:59:46 +07:00
|
|
|
/*
|
|
|
|
* Print a one-time message (analogous to WARN_ONCE() et al):
|
|
|
|
*/
|
|
|
|
|
|
|
|
#ifdef CONFIG_PRINTK
|
2014-01-24 06:54:16 +07:00
|
|
|
#define printk_once(fmt, ...) \
|
|
|
|
({ \
|
2019-04-13 09:11:31 +07:00
|
|
|
static bool __section(.data.once) __print_once; \
|
2016-07-05 05:31:26 +07:00
|
|
|
bool __ret_print_once = !__print_once; \
|
2014-01-24 06:54:16 +07:00
|
|
|
\
|
|
|
|
if (!__print_once) { \
|
|
|
|
__print_once = true; \
|
|
|
|
printk(fmt, ##__VA_ARGS__); \
|
|
|
|
} \
|
2016-07-05 05:31:26 +07:00
|
|
|
unlikely(__ret_print_once); \
|
2011-01-13 07:59:46 +07:00
|
|
|
})
|
2014-06-05 06:11:41 +07:00
|
|
|
#define printk_deferred_once(fmt, ...) \
|
|
|
|
({ \
|
2019-04-13 09:11:31 +07:00
|
|
|
static bool __section(.data.once) __print_once; \
|
2016-07-05 05:31:26 +07:00
|
|
|
bool __ret_print_once = !__print_once; \
|
2014-06-05 06:11:41 +07:00
|
|
|
\
|
|
|
|
if (!__print_once) { \
|
|
|
|
__print_once = true; \
|
|
|
|
printk_deferred(fmt, ##__VA_ARGS__); \
|
|
|
|
} \
|
2016-07-05 05:31:26 +07:00
|
|
|
unlikely(__ret_print_once); \
|
2014-06-05 06:11:41 +07:00
|
|
|
})
|
2011-01-13 07:59:46 +07:00
|
|
|
#else
|
2014-01-24 06:54:16 +07:00
|
|
|
#define printk_once(fmt, ...) \
|
2011-01-13 07:59:46 +07:00
|
|
|
no_printk(fmt, ##__VA_ARGS__)
|
2014-06-05 06:11:41 +07:00
|
|
|
#define printk_deferred_once(fmt, ...) \
|
|
|
|
no_printk(fmt, ##__VA_ARGS__)
|
2011-01-13 07:59:46 +07:00
|
|
|
#endif
|
|
|
|
|
|
|
|
#define pr_emerg_once(fmt, ...) \
|
|
|
|
printk_once(KERN_EMERG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_alert_once(fmt, ...) \
|
|
|
|
printk_once(KERN_ALERT pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_crit_once(fmt, ...) \
|
|
|
|
printk_once(KERN_CRIT pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_err_once(fmt, ...) \
|
|
|
|
printk_once(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_warn_once(fmt, ...) \
|
|
|
|
printk_once(KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_notice_once(fmt, ...) \
|
|
|
|
printk_once(KERN_NOTICE pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_info_once(fmt, ...) \
|
|
|
|
printk_once(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#define pr_cont_once(fmt, ...) \
|
|
|
|
printk_once(KERN_CONT pr_fmt(fmt), ##__VA_ARGS__)
|
2013-02-22 07:43:10 +07:00
|
|
|
|
|
|
|
#if defined(DEBUG)
|
|
|
|
#define pr_devel_once(fmt, ...) \
|
|
|
|
printk_once(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#else
|
|
|
|
#define pr_devel_once(fmt, ...) \
|
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#endif
|
|
|
|
|
2011-01-13 07:59:46 +07:00
|
|
|
/* If you are writing a driver, please use dev_dbg instead */
|
|
|
|
#if defined(DEBUG)
|
|
|
|
#define pr_debug_once(fmt, ...) \
|
|
|
|
printk_once(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#else
|
|
|
|
#define pr_debug_once(fmt, ...) \
|
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#endif
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
/*
|
|
|
|
* ratelimited messages with local ratelimit_state,
|
|
|
|
* no local ratelimit_state used in the !PRINTK case
|
|
|
|
*/
|
|
|
|
#ifdef CONFIG_PRINTK
|
2011-01-13 07:59:47 +07:00
|
|
|
#define printk_ratelimited(fmt, ...) \
|
|
|
|
({ \
|
2010-11-16 04:37:37 +07:00
|
|
|
static DEFINE_RATELIMIT_STATE(_rs, \
|
|
|
|
DEFAULT_RATELIMIT_INTERVAL, \
|
|
|
|
DEFAULT_RATELIMIT_BURST); \
|
|
|
|
\
|
|
|
|
if (__ratelimit(&_rs)) \
|
|
|
|
printk(fmt, ##__VA_ARGS__); \
|
|
|
|
})
|
|
|
|
#else
|
2011-01-13 07:59:47 +07:00
|
|
|
#define printk_ratelimited(fmt, ...) \
|
|
|
|
no_printk(fmt, ##__VA_ARGS__)
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|
|
|
|
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_emerg_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_EMERG pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_alert_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_ALERT pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_crit_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_CRIT pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_err_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_warn_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_notice_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_NOTICE pr_fmt(fmt), ##__VA_ARGS__)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_info_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
/* no pr_cont_ratelimited, don't do that... */
|
2013-02-22 07:43:10 +07:00
|
|
|
|
|
|
|
#if defined(DEBUG)
|
|
|
|
#define pr_devel_ratelimited(fmt, ...) \
|
|
|
|
printk_ratelimited(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#else
|
|
|
|
#define pr_devel_ratelimited(fmt, ...) \
|
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#endif
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
/* If you are writing a driver, please use dev_dbg instead */
|
2013-10-27 10:41:53 +07:00
|
|
|
#if defined(CONFIG_DYNAMIC_DEBUG)
|
|
|
|
/* descriptor check is first to prevent flooding with "callbacks suppressed" */
|
|
|
|
#define pr_debug_ratelimited(fmt, ...) \
|
|
|
|
do { \
|
|
|
|
static DEFINE_RATELIMIT_STATE(_rs, \
|
|
|
|
DEFAULT_RATELIMIT_INTERVAL, \
|
|
|
|
DEFAULT_RATELIMIT_BURST); \
|
2015-09-10 05:36:12 +07:00
|
|
|
DEFINE_DYNAMIC_DEBUG_METADATA(descriptor, pr_fmt(fmt)); \
|
2019-03-08 07:27:29 +07:00
|
|
|
if (DYNAMIC_DEBUG_BRANCH(descriptor) && \
|
2013-10-27 10:41:53 +07:00
|
|
|
__ratelimit(&_rs)) \
|
2015-09-10 05:36:12 +07:00
|
|
|
__dynamic_pr_debug(&descriptor, pr_fmt(fmt), ##__VA_ARGS__); \
|
2013-10-27 10:41:53 +07:00
|
|
|
} while (0)
|
|
|
|
#elif defined(DEBUG)
|
2011-01-13 07:59:47 +07:00
|
|
|
#define pr_debug_ratelimited(fmt, ...) \
|
2010-11-16 04:37:37 +07:00
|
|
|
printk_ratelimited(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
|
|
|
#else
|
|
|
|
#define pr_debug_ratelimited(fmt, ...) \
|
2011-01-13 07:59:45 +07:00
|
|
|
no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|
|
|
|
|
kmsg: export printk records to the /dev/kmsg interface
Support for multiple concurrent readers of /dev/kmsg, with read(),
seek(), poll() support. Output of message sequence numbers, to allow
userspace log consumers to reliably reconnect and reconstruct their
state at any given time. After open("/dev/kmsg"), read() always
returns *all* buffered records. If only future messages should be
read, SEEK_END can be used. In case records get overwritten while
/dev/kmsg is held open, or records get faster overwritten than they
are read, the next read() will return -EPIPE and the current reading
position gets updated to the next available record. The passed
sequence numbers allow the log consumer to calculate the amount of
lost messages.
[root@mop ~]# cat /dev/kmsg
5,0,0;Linux version 3.4.0-rc1+ (kay@mop) (gcc version 4.7.0 20120315 ...
6,159,423091;ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
7,160,424069;pci_root PNP0A03:00: host bridge window [io 0x0000-0x0cf7] (ignored)
SUBSYSTEM=acpi
DEVICE=+acpi:PNP0A03:00
6,339,5140900;NET: Registered protocol family 10
30,340,5690716;udevd[80]: starting version 181
6,341,6081421;FDC 0 is a S82078B
6,345,6154686;microcode: CPU0 sig=0x623, pf=0x0, revision=0x0
7,346,6156968;sr 1:0:0:0: Attached scsi CD-ROM sr0
SUBSYSTEM=scsi
DEVICE=+scsi:1:0:0:0
6,347,6289375;microcode: CPU1 sig=0x623, pf=0x0, revision=0x0
Cc: Karel Zak <kzak@redhat.com>
Tested-by: William Douglas <william.douglas@intel.com>
Signed-off-by: Kay Sievers <kay@vrfy.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2012-05-03 07:29:41 +07:00
|
|
|
extern const struct file_operations kmsg_fops;
|
|
|
|
|
2011-01-13 07:59:47 +07:00
|
|
|
enum {
|
|
|
|
DUMP_PREFIX_NONE,
|
|
|
|
DUMP_PREFIX_ADDRESS,
|
|
|
|
DUMP_PREFIX_OFFSET
|
|
|
|
};
|
2015-02-13 06:02:29 +07:00
|
|
|
extern int hex_dump_to_buffer(const void *buf, size_t len, int rowsize,
|
|
|
|
int groupsize, char *linebuf, size_t linebuflen,
|
|
|
|
bool ascii);
|
2011-01-13 07:59:47 +07:00
|
|
|
#ifdef CONFIG_PRINTK
|
|
|
|
extern void print_hex_dump(const char *level, const char *prefix_str,
|
|
|
|
int prefix_type, int rowsize, int groupsize,
|
|
|
|
const void *buf, size_t len, bool ascii);
|
|
|
|
#else
|
|
|
|
static inline void print_hex_dump(const char *level, const char *prefix_str,
|
|
|
|
int prefix_type, int rowsize, int groupsize,
|
|
|
|
const void *buf, size_t len, bool ascii)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
static inline void print_hex_dump_bytes(const char *prefix_str, int prefix_type,
|
|
|
|
const void *buf, size_t len)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
2012-12-06 04:48:27 +07:00
|
|
|
#if defined(CONFIG_DYNAMIC_DEBUG)
|
|
|
|
#define print_hex_dump_debug(prefix_str, prefix_type, rowsize, \
|
|
|
|
groupsize, buf, len, ascii) \
|
|
|
|
dynamic_hex_dump(prefix_str, prefix_type, rowsize, \
|
|
|
|
groupsize, buf, len, ascii)
|
2015-09-10 05:37:11 +07:00
|
|
|
#elif defined(DEBUG)
|
2012-12-06 04:48:27 +07:00
|
|
|
#define print_hex_dump_debug(prefix_str, prefix_type, rowsize, \
|
|
|
|
groupsize, buf, len, ascii) \
|
|
|
|
print_hex_dump(KERN_DEBUG, prefix_str, prefix_type, rowsize, \
|
|
|
|
groupsize, buf, len, ascii)
|
2015-09-10 05:37:11 +07:00
|
|
|
#else
|
|
|
|
static inline void print_hex_dump_debug(const char *prefix_str, int prefix_type,
|
|
|
|
int rowsize, int groupsize,
|
|
|
|
const void *buf, size_t len, bool ascii)
|
|
|
|
{
|
|
|
|
}
|
|
|
|
#endif
|
2012-12-06 04:48:27 +07:00
|
|
|
|
2019-09-26 06:46:29 +07:00
|
|
|
/**
|
|
|
|
* print_hex_dump_bytes - shorthand form of print_hex_dump() with default params
|
|
|
|
* @prefix_str: string to prefix each line with;
|
|
|
|
* caller supplies trailing spaces for alignment if desired
|
|
|
|
* @prefix_type: controls whether prefix of an offset, address, or none
|
|
|
|
* is printed (%DUMP_PREFIX_OFFSET, %DUMP_PREFIX_ADDRESS, %DUMP_PREFIX_NONE)
|
|
|
|
* @buf: data blob to dump
|
|
|
|
* @len: number of bytes in the @buf
|
|
|
|
*
|
|
|
|
* Calls print_hex_dump(), with log level of KERN_DEBUG,
|
|
|
|
* rowsize of 16, groupsize of 1, and ASCII output included.
|
|
|
|
*/
|
|
|
|
#define print_hex_dump_bytes(prefix_str, prefix_type, buf, len) \
|
|
|
|
print_hex_dump_debug(prefix_str, prefix_type, 16, 1, buf, len, true)
|
|
|
|
|
2010-11-16 04:37:37 +07:00
|
|
|
#endif
|