Commit Graph

80 Commits

Author SHA1 Message Date
Stefan Raspl
c012a0f267 tools/kvm_stat: re-animate display of dead guests
When filtering by guest (interactive commands 'p'/'g'), and the respective
guest was destroyed, detect when the guest is up again through the guest
name if possible.
I.e. when displaying events for a specific guest, it is not necessary
anymore to restart kvm_stat in case the guest is restarted.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
404517e408 tools/kvm_stat: indicate dead guests as such
For destroyed guests, kvm_stat essentially freezes with the last data
displayed. This is acceptable for users, in case they want to inspect the
final data. But it looks a bit irritating. Therefore, detect this situation
and display a respective indicator in the header.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
29c39f38e4 tools/kvm_stat: handle guest removals more gracefully
When running with the DebugFS provider, removal of a guest can result in a
negative CurAvg/s, which looks rather confusing.
If so, suppress the body refresh and print a message instead.
To reproduce, have at least one guest A completely booted. Then start
another guest B (which generates a huge amount of events), then destroy B.
On the next refresh, kvm_stat should display a whole lot of negative values
in the CurAvg/s column.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
0db8b31023 tools/kvm_stat: don't reset stats when setting PID filter for debugfs
When setting a PID filter in debugfs, we unnecessarily reset the
statistics, although there is no reason to do so. This behavior was
merely introduced with commit 9f114a03c6 "tools/kvm_stat: add
interactive command 'r'", most likely to mimic the behavior of
the tracepoints provider in this respect. However, there are plenty
of differences between the two providers, so there is no reason not
to take advantage of the possibility to filter by PID without
resetting the statistics.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
710ab11ad9 tools/kvm_stat: fix updates for dead guests
With pid filtering active, when a guest is removed e.g. via virsh shutdown,
successive updates produce garbage.
Therefore, we add code to detect this case and prevent further body updates.
Note that when displaying the help dialog via 'h' in this case, once we exit
we're stuck with the 'Collecting data...' message till we remove the filter.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
617c66b9f2 tools/kvm_stat: fix handling of invalid paths in debugfs provider
When filtering by guest, kvm_stat displays garbage when the guest is
destroyed - see sample output below.
We add code to remove the invalid paths from the providers, so at least
no more garbage is displayed.
Here's a sample output to illustrate:

  kvm statistics - pid 13986 (foo)

   Event                                         Total %Total CurAvg/s
   diagnose_258                                     -2    0.0        0
   deliver_program_interruption                     -3    0.0        0
   diagnose_308                                     -4    0.0        0
   halt_poll_invalid                               -91    0.0       -6
   deliver_service_signal                         -244    0.0      -16
   halt_successful_poll                           -250    0.1      -17
   exit_pei                                       -285    0.1      -19
   exit_external_request                          -312    0.1      -21
   diagnose_9c                                    -328    0.1      -22
   userspace_handled                              -713    0.1      -47
   halt_attempted_poll                            -939    0.2      -62
   deliver_emergency_signal                      -3126    0.6     -208
   halt_wakeup                                   -7199    1.5     -481
   exit_wait_state                               -7379    1.5     -493
   diagnose_500                                 -56499   11.5    -3757
   exit_null                                    -85491   17.4    -5685
   diagnose_44                                 -133300   27.1    -8874
   exit_instruction                            -195898   39.8   -13037
   Total                                       -492063

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Stefan Raspl
58f33cfe73 tools/kvm_stat: fix python3 issues
Python3 returns a float for a regular division - switch to a division
operator that returns an integer.
Furthermore, filters return a generator object instead of the actual
list - wrap result in yet another list, which makes it still work in
both, Python2 and 3.

Signed-off-by: Stefan Raspl <raspl@linux.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-08-30 17:15:12 +02:00
Cole Robinson
1f97e01a38 tools/kvm_stat: Remove unused function
Unused since added in 18e8f4100

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-and-tested-by: Stefan Raspl <stefan.raspl@linux.vnet.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-03-28 22:47:06 +02:00
Cole Robinson
0866c31bf3 tools/kvm_stat: Don't use deprecated file()
$ python3 tools/kvm/kvm_stat/kvm_stat
Traceback (most recent call last):
  File "tools/kvm/kvm_stat/kvm_stat", line 1668, in <module>
    main()
  File "tools/kvm/kvm_stat/kvm_stat", line 1639, in main
    assign_globals()
  File "tools/kvm/kvm_stat/kvm_stat", line 1618, in assign_globals
    for line in file('/proc/mounts'):
NameError: name 'file' is not defined

open() is the python3 way, and works on python2.6+

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-and-tested-by: Stefan Raspl <stefan.raspl@linux.vnet.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-03-28 22:47:06 +02:00
Cole Robinson
6ade1ae84c tools/kvm_stat: Fix python3 syntax
$ python3 tools/kvm/kvm_stat/kvm_stat
  File "tools/kvm/kvm_stat/kvm_stat", line 1137
    def sortkey((_k, v)):
                ^
SyntaxError: invalid syntax

Fix it in a way that's compatible with python2 and python3

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Tested-by: Stefan Raspl <stefan.raspl@linux.vnet.ibm.com>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2018-03-28 22:47:06 +02:00
Stefan Raspl
6789af030a tools/kvm_stat: print 'Total' line for multiple events only
The 'Total' line looks a bit weird when we have a single event only. This
can happen e.g. due to filters. Therefore suppress when there's only a
single event in the output.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:46 +01:00
Stefan Raspl
df72ecfc79 tools/kvm_stat: group child events indented after parent
We keep the current logic that sorts all events (parent and child), but
re-shuffle the events afterwards, grouping the children after the
respective parent. Note that the percentage column for child events
gives the percentage of the parent's total.
Since we rework the logic anyway, we modify the total average
calculation to use the raw numbers instead of the (rounded) averages.
Note that this can result in differing numbers (between total average
and the sum of the individual averages) due to rounding errors.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:45 +01:00
Stefan Raspl
18e8f4100e tools/kvm_stat: separate drilldown and fields filtering
Drilldown (i.e. toggle display of child trace events) was implemented by
overriding the fields filter. This resulted in inconsistencies: E.g. when
drilldown was not active, adding a filter that also matches child trace
events would not only filter fields according to the filter, but also add
in the child trace events matching the filter. E.g. on x86, setting
'kvm_userspace_exit' as the fields filter after startup would result in
display of kvm_userspace_exit(DCR), although that wasn't previously
present - not exactly what one would expect from a filter.
This patch addresses the issue by keeping drilldown and fields filter
separate. While at it, we also fix a PEP8 issue by adding a blank line
at one place (since we're in the area...).
We implement this by adding a framework that also allows to define a
taxonomy among the debugfs events to identify child trace events. I.e.
drilldown using 'x' can now also work with debugfs. A respective parent-
child relationship is only known for S390 at the moment, but could be
added adjusting other platforms' ARCH.dbg_is_child() methods
accordingly.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:44 +01:00
Stefan Raspl
516f1190a1 tools/kvm_stat: eliminate extra guest/pid selection dialog
We can do with a single dialog that takes both, pids and guest names.
Note that we keep both interactive commands, 'p' and 'g' for now, to
avoid confusion among users used to a specific key.

While at it, we improve on some minor glitches regarding curses usage,
e.g. cursor still visible when not supposed to be.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:44 +01:00
Stefan Raspl
c0e8c21eae tools/kvm_stat: mark private methods as such
Helps quite a bit reading the code when it's obvious when a method is
intended for internal use only.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:43 +01:00
Stefan Raspl
1fd6a708c8 tools/kvm_stat: fix debugfs handling
Te checks for debugfs assumed that debugfs is always mounted at
/sys/kernel/debug - which is likely, but not guaranteed. This is addressed
by checking /proc/mounts for the actual location.
Furthermore, when debugfs was mounted, but the kvm module not loaded, a
misleading error pointing towards debugfs not present was given.
To reproduce,
(a) run kvm_stat with debugfs mounted at a place different from
    /sys/kernel/debug
(b) run kvm_stat with debugfs mounted but kvm module not loaded

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:42 +01:00
Stefan Raspl
1cd8bfb1ed tools/kvm_stat: print error on invalid regex
Entering an invalid regular expression did not produce any indication of an
error so far.
To reproduce, press 'f' and enter 'foo(' (with an unescaped bracket).

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:41 +01:00
Stefan Raspl
3df33a0f34 tools/kvm_stat: fix crash when filtering out all non-child trace events
When we apply a filter that will only leave child trace events, we
receive a ZeroDivisionError when calculating the percentages.
In that case, provide percentages based on child events only.
To reproduce, run 'kvm_stat -f .*[\(].*'.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:41 +01:00
Marc Hartmayer
369d5a85bb tools/kvm_stat: avoid 'is' for equality checks
Use '==' for equality checks and 'is' when comparing identities.

An example where '==' and 'is' behave differently:
>>> a = 4242
>>> a == 4242
True
>>> a is 4242
False

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:40 +01:00
Marc Hartmayer
0eb578009a tools/kvm_stat: use a more pythonic way to iterate over dictionaries
If it's clear that the values of a dictionary will be used then use
the '.items()' method.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Tested-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
[Include fix for logging mode by Stefan Raspl]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:39 +01:00
Marc Hartmayer
006f1548ac tools/kvm_stat: use a namedtuple for storing the values
Use a namedtuple for storing the values as it allows to access the
fields of a tuple via names. This makes the overall code much easier
to read and to understand. Access by index is still possible as
before.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Tested-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:39 +01:00
Marc Hartmayer
faa312a543 tools/kvm_stat: simplify the sortkey function
The 'sortkey' function references a value in its enclosing
scope (closure). This is not common practice for a sort key function
so let's replace it. Additionally, the function 'sorted' has already a
parameter for reversing the result therefore the inversion of the
values is unneeded. The check for stats[x][1] is also superfluous as
it's ensured that this value is initialized with 0.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Tested-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2018-02-24 01:43:38 +01:00
Stefan Raspl
aa12f594f9 tools/kvm_stat: sort '-f help' output
Sort the fields returned by specifying '-f help' on the command line.
While at it, simplify the code a bit, indent the output and eliminate an
extra blank line at the beginning.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-21 13:03:32 +01:00
Stefan Raspl
cf656c7661 tools/kvm_stat: add line for totals
Add a line for the total number of events and current average at the
bottom of the body.
Note that both values exclude child trace events. I.e. if drilldown is
activated via interactive command 'x', only the totals are accounted, or
we'd be counting these twice (see previous commit "tools/kvm_stat: fix
child trace events accounting").

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:47 +01:00
Stefan Raspl
73fab6ffbd tools/kvm_stat: stop ignoring unhandled arguments
Unhandled arguments, which could easily include typos, are simply
ignored. We should be strict to avoid undetected typos.
To reproduce start kvm_stat with an extra argument, e.g.
'kvm_stat -d bnuh5ol' and note that this will actually work.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:46 +01:00
Stefan Raspl
822cfe3e48 tools/kvm_stat: suppress usage information on command line errors
Errors while parsing the '-g' command line argument result in display of
usage information prior to the error message. This is a bit confusing,
as the command line is syntactically correct.
To reproduce, run 'kvm_stat -g' and specify a non-existing or inactive
guest.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:46 +01:00
Stefan Raspl
08e20a6300 tools/kvm_stat: handle invalid regular expressions
Passing an invalid regular expression on the command line results in a
traceback. Note that interactive specification of invalid regular
expressions is not affected
To reproduce, run "kvm_stat -f '*'".

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:45 +01:00
Stefan Raspl
f3d11b0e86 tools/kvm_stat: add hint on '-f help' to man page
The man page update for this new functionality was omitted.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:44 +01:00
Stefan Raspl
fff8c9eb48 tools/kvm_stat: fix child trace events accounting
Child trace events were included in calculation of the overall total,
which is used for calculation of the percentages of the '%Total' column.
However, the parent trace envents' stats summarize the child trace
events, hence we'd incorrectly account for them twice, leading to
slightly wrong stats.
With this fix, we use the correct total. Consequently, the sum of the
child trace events' '%Total' column values is identical to the
respective value of the respective parent event. However, this also
means that the sum of the '%Total' column values will aggregate to more
than 100 percent.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:44 +01:00
Stefan Raspl
b74faa930d tools/kvm_stat: fix extra handling of 'help' with fields filter
Commit 67fbcd62f5 ("tools/kvm_stat: add '-f help' to get the available
event list") added support for '-f help'. However, the extra handling of
'help' will also take effect when 'help' is specified as a regex in
interactive mode via 'f'. This results in display of all events while
only those matching this regex should be shown.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:43 +01:00
Stefan Raspl
67c162b089 tools/kvm_stat: fix missing field update after filter change
When updating the fields filter, tracepoint events of fields previously
not visible were not enabled, as TracepointProvider.update_fields()
updated the member variable directly instead of using the setter, which
triggers the event enable/disable.
To reproduce, run 'kvm_stat -f kvm_exit', press 'c' to remove the
filter, and notice that no add'l fields that do not match the regex
'kvm_exit' will appear.
This issue was introduced by commit c469117df0 ("tools/kvm_stat:
simplify initializers").

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:42 +01:00
Stefan Raspl
faa0665041 tools/kvm_stat: fix drilldown in events-by-guests mode
When displaying debugfs events listed by guests, an attempt to switch to
reporting of stats for individual child trace events results in garbled
output. Reason is that when toggling drilldown, the update of the stats
doesn't honor when events are displayed by guests, as indicated by
Tui._display_guests.
To reproduce, run 'kvm_stat -d' and press 'b' followed by 'x'.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:42 +01:00
Stefan Raspl
19e8e54f43 tools/kvm_stat: fix command line option '-g'
Specifying a guest via '-g foo' always results in an error:
  $ kvm_stat -g foo
  Usage: kvm_stat [options]

  kvm_stat: error: Error while searching for guest "foo", use "-p" to
  specify a pid instead

Reason is that Tui.get_pid_from_gname() is not static, as it is supposed
to be.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Tested-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-12-14 09:25:41 +01:00
Linus Torvalds
974aa5630b First batch of KVM changes for 4.15
Common:
  - Python 3 support in kvm_stat
 
  - Accounting of slabs to kmemcg
 
 ARM:
  - Optimized arch timer handling for KVM/ARM
 
  - Improvements to the VGIC ITS code and introduction of an ITS reset
    ioctl
 
  - Unification of the 32-bit fault injection logic
 
  - More exact external abort matching logic
 
 PPC:
  - Support for running hashed page table (HPT) MMU mode on a host that
    is using the radix MMU mode;  single threaded mode on POWER 9 is
    added as a pre-requisite
 
  - Resolution of merge conflicts with the last second 4.14 HPT fixes
 
  - Fixes and cleanups
 
 s390:
  - Some initial preparation patches for exitless interrupts and crypto
 
  - New capability for AIS migration
 
  - Fixes
 
 x86:
  - Improved emulation of LAPIC timer mode changes, MCi_STATUS MSRs, and
    after-reset state
 
  - Refined dependencies for VMX features
 
  - Fixes for nested SMI injection
 
  - A lot of cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABCAAGBQJaDayXAAoJEED/6hsPKofo/3UH/3HvlcHt+ADTkCU1/iiKAs+i
 0zngIOXIxgHDnV0ww6bV+Znww0BzTYgKCAXX76z603jdpDwG/pzQQcbLDF5ZoJnD
 sQtF10gZinWaRsHlfbLqjrHGL2pGDHO1UKBKLJ0bAIyORPZBxs7i+VmrY/blnr9c
 0wsybJ8RbvwAxjsDL5jeX/z4NehPupmKUc4Lf0eZdSHwVOf9sjn+MP6jJ0r2JcIb
 D+zddPBiLStzN97t4gZpQsrlj3LKrDS+6hY+1TjSvlh+yHKFVFh58VhLm4DuDeb5
 bYOAlWJ/gAWEzfvr5Ld+Nd7SqWWn/14logPkQ4gcU4BI/neAOzk4c6hJfCHl1nk=
 =593n
 -----END PGP SIGNATURE-----

Merge tag 'kvm-4.15-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull KVM updates from Radim Krčmář:
 "First batch of KVM changes for 4.15

  Common:
   - Python 3 support in kvm_stat
   - Accounting of slabs to kmemcg

  ARM:
   - Optimized arch timer handling for KVM/ARM
   - Improvements to the VGIC ITS code and introduction of an ITS reset
     ioctl
   - Unification of the 32-bit fault injection logic
   - More exact external abort matching logic

  PPC:
   - Support for running hashed page table (HPT) MMU mode on a host that
     is using the radix MMU mode; single threaded mode on POWER 9 is
     added as a pre-requisite
   - Resolution of merge conflicts with the last second 4.14 HPT fixes
   - Fixes and cleanups

  s390:
   - Some initial preparation patches for exitless interrupts and crypto
   - New capability for AIS migration
   - Fixes

  x86:
   - Improved emulation of LAPIC timer mode changes, MCi_STATUS MSRs,
     and after-reset state
   - Refined dependencies for VMX features
   - Fixes for nested SMI injection
   - A lot of cleanups"

* tag 'kvm-4.15-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (89 commits)
  KVM: s390: provide a capability for AIS state migration
  KVM: s390: clear_io_irq() requests are not expected for adapter interrupts
  KVM: s390: abstract conversion between isc and enum irq_types
  KVM: s390: vsie: use common code functions for pinning
  KVM: s390: SIE considerations for AP Queue virtualization
  KVM: s390: document memory ordering for kvm_s390_vcpu_wakeup
  KVM: PPC: Book3S HV: Cosmetic post-merge cleanups
  KVM: arm/arm64: fix the incompatible matching for external abort
  KVM: arm/arm64: Unify 32bit fault injection
  KVM: arm/arm64: vgic-its: Implement KVM_DEV_ARM_ITS_CTRL_RESET
  KVM: arm/arm64: Document KVM_DEV_ARM_ITS_CTRL_RESET
  KVM: arm/arm64: vgic-its: Free caches when GITS_BASER Valid bit is cleared
  KVM: arm/arm64: vgic-its: New helper functions to free the caches
  KVM: arm/arm64: vgic-its: Remove kvm_its_unmap_device
  arm/arm64: KVM: Load the timer state when enabling the timer
  KVM: arm/arm64: Rework kvm_timer_should_fire
  KVM: arm/arm64: Get rid of kvm_timer_flush_hwstate
  KVM: arm/arm64: Avoid phys timer emulation in vcpu entry/exit
  KVM: arm/arm64: Move phys_timer_emulate function
  KVM: arm/arm64: Use kvm_arm_timer_set/get_reg for guest register traps
  ...
2017-11-16 13:00:24 -08:00
Greg Kroah-Hartman
b24413180f License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne.  Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed.  Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
 - Files considered eligible had to be source code files.
 - Make and config files were included as candidates if they contained >5
   lines of source
 - File already had some variant of a license header in it (even if <5
   lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

 - when both scanners couldn't find any license traces, file was
   considered to have no license information in it, and the top level
   COPYING file license applied.

   For non */uapi/* files that summary was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0                                              11139

   and resulted in the first patch in this series.

   If that file was a */uapi/* path one, it was "GPL-2.0 WITH
   Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0 WITH Linux-syscall-note                        930

   and resulted in the second patch in this series.

 - if a file had some form of licensing information in it, and was one
   of the */uapi/* ones, it was denoted with the Linux-syscall-note if
   any GPL family license was found in the file or had no licensing in
   it (per prior point).  Results summary:

   SPDX license identifier                            # files
   ---------------------------------------------------|------
   GPL-2.0 WITH Linux-syscall-note                       270
   GPL-2.0+ WITH Linux-syscall-note                      169
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
   LGPL-2.1+ WITH Linux-syscall-note                      15
   GPL-1.0+ WITH Linux-syscall-note                       14
   ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
   LGPL-2.0+ WITH Linux-syscall-note                       4
   LGPL-2.1 WITH Linux-syscall-note                        3
   ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
   ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1

   and that resulted in the third patch in this series.

 - when the two scanners agreed on the detected license(s), that became
   the concluded license(s).

 - when there was disagreement between the two scanners (one detected a
   license but the other didn't, or they both detected different
   licenses) a manual inspection of the file occurred.

 - In most cases a manual inspection of the information in the file
   resulted in a clear resolution of the license that should apply (and
   which scanner probably needed to revisit its heuristics).

 - When it was not immediately clear, the license identifier was
   confirmed with lawyers working with the Linux Foundation.

 - If there was any question as to the appropriate license identifier,
   the file was flagged for further research and to be revisited later
   in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.  The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
 - a full scancode scan run, collecting the matched texts, detected
   license ids and scores
 - reviewing anything where there was a license detected (about 500+
   files) to ensure that the applied SPDX license was correct
 - reviewing anything where there was no detection but the patch license
   was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
   SPDX license was correct

This produced a worksheet with 20 files needing minor correction.  This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg.  Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected.  This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.)  Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-02 11:10:55 +01:00
Jeremy Cline
9cc5fbbb8c tools/kvm_stat: Add Python 3 support to kvm_stat
Make kvm_stat support Python 3 by changing the use of "print" to a
function rather than a statement, switching from "iteritems" and
"iterkeys" (removed in Python 3) to "items" and "keys" respectively,
and decoding bytes to strings when dealing with text.

With this change, kvm_stat is usable with Python 2.6 and greater.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
2017-10-12 14:01:50 +02:00
Lin Ma
67fbcd62f5 tools/kvm_stat: add '-f help' to get the available event list
Signed-off-by: Lin Ma <lma@suse.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-07-26 19:04:53 +02:00
Lin Ma
efcb521943 tools/kvm_stat: use variables instead of hard paths in help output
Using variables instead of hard paths makes the requirements information
more accurate.

Signed-off-by: Lin Ma <lma@suse.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-07-26 19:04:52 +02:00
Stefan Raspl
5c1954d25d tools/kvm_stat: add new interactive command 'b'
Toggle display total number of events by guest (debugfs only).
When switching to display of events by guest, field filters remain
active. I.e. the number of events per guest reported considers only
events matching the filters. Likewise with pid/guest filtering.
Note that when switching to display of events by guest, DebugfsProvider
remains to collect data for events as it did before, but the read()
method summarizes the values by pid.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-27 16:44:50 +02:00
Stefan Raspl
ab7ef193fa tools/kvm_stat: add new command line switch '-i'
It might be handy to display the full history of event stats to compare
the current event distribution against any available historic data.
Since we have that available for debugfs, we offer a respective command
line option to display what's available.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-27 16:43:48 +02:00
Stefan Raspl
61f381bb7e tools/kvm_stat: fix error on interactive command 'g'
Fix an instance where print_all_gnames() is called without the mandatory
argument, resulting in a stack trace.
To reproduce, simply press 'g' in interactive mode.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-27 16:41:55 +02:00
Stefan Raspl
865279c53c tools/kvm_stat: display guest list in pid/guest selection screens
Display a (possibly inaccurate) list of all running guests. Note that we
leave a bit of extra room above the list for potential error messages.
Furthermore, we deliberately do not reject pids or guest names that are
not in our list, as we cannot rule out that our fuzzy approach might be
in error somehow.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:24:49 +02:00
Stefan Raspl
6667ae8f39 tools/kvm_stat: add new interactive command 'o'
Add new interactive command 'o' to toggle sorting by 'CurAvg/s' (default)
and 'Total' columns.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:24:48 +02:00
Stefan Raspl
64eefad2cd tools/kvm_stat: add new interactive command 's'
Add new command 's' to modify the update interval. Limited to a maximum of
25.5 sec and a minimum of 0.1 sec, since curses cannot handle longer
and shorter delays respectively.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:20:21 +02:00
Stefan Raspl
1fdea7b289 tools/kvm_stat: add new interactive command 'h'
Display interactive commands reference on 'h'.
While at it, sort interactive commands alphabetically in various places.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:17:59 +02:00
Stefan Raspl
38e89c37a1 tools/kvm_stat: rename 'Current' column to 'CurAvg/s'
'Current' can be misleading as it doesn't tell whether this is the amount
of events in the last interval or the current average per second.
Note that this necessitates widening the respective column by one more
character.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:17:37 +02:00
Stefan Raspl
f6d753102a tools/kvm_stat: make heading look a bit more like 'top'
Print header in standout font just like the 'top' command does.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:17:28 +02:00
Stefan Raspl
5725393764 tools/kvm_stat: display message indicating lack of events
Give users some indication on the reason why no data is displayed on the
screen yet.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:17:18 +02:00
Stefan Raspl
62d1b6cc24 tools/kvm_stat: show cursor in selection screens
Show the cursor in the interactive screens to specify pid, filter or guest
name as an orientation for the user.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:17:06 +02:00
Stefan Raspl
099a2dfc67 tools/kvm_stat: move functions to corresponding classes
Quite a few of the functions are used only in a single class. Moving
functions accordingly to improve the overall structure.
Furthermore, introduce a base class for the providers, which might also
come handy for future extensions.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-06-08 18:16:28 +02:00