Commit Graph

7321 Commits

Author SHA1 Message Date
Vasily Gorbik
2835c2ea95 s390/startup: avoid save_area_sync overflow
Currently we overflow save_area_sync and write over
save_area_async. Although this is not a real problem make
startup_pgm_check_handler consistent with late pgm check handler and
store [%r0,%r7] directly into gpregs_save_area.

Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-29 15:00:59 +02:00
Vasily Gorbik
1c7c83e8d2 s390: remove unused _swsusp_reset_dma
Since commit 394216275c ("s390: remove broken hibernate / power
management support") _swsusp_reset_dma is unused and could be safely
removed.

Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-29 15:00:58 +02:00
Vasily Gorbik
5c46f2768c s390/kaslr: correct and explain randomization base generation
Currently there are several minor problems with randomization base
generation code:

1. It might misbehave in low memory conditions. In particular there
might be enough space for the kernel on [0, block_sum] but after
	if (base < safe_addr)
		base = safe_addr;
it might not be enough anymore.

2. It does not correctly handle minimal address constraint. In condition
	if (base < safe_addr)
		base = safe_addr;
a synthetic value is compared with an address. If we have a memory
setup with memory holes due to offline memory regions, and safe_addr is
close to the end of the first online memory block - we might position
the kernel in invalid memory.

3. block_sum calculation logic contains off-by-one error. Let's say we
have a memory block in which the kernel fits perfectly
(end - start == kernel_size). In this case:
		if (end - start < kernel_size)
			continue;
		block_sum += end - start - kernel_size;
block_sum is not increased, while it is a valid kernel position.

So, address problems listed and explain algorithm used. Besides that
restructuring the code makes it possible to extend kernel positioning
algorithm further. Currently we pick position in between single
[min, max] range (min = safe_addr, max = memory_limit). In future we
can do that for multiple ranges as well (by calling
count_valid_kernel_positions for each range).

Reviewed-by: Philipp Rudo <prudo@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-29 15:00:58 +02:00
Vasily Gorbik
07a699bc43 s390/kaslr: avoid mixing valid random value and an error code
0 is a valid random value. To avoid mixing it with error code 0 as an
return code make get_random() take extra argument to output random
value and return an error code.

Reviewed-by: Philipp Rudo <prudo@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-29 15:00:58 +02:00
Sven Schnelle
ad5ceb33ee s390/stp: unify stp_work_mutex and clock_sync_mutex
No need to have two mutexes, and while at it rename it to
stp_mutex.

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:21 +02:00
Sven Schnelle
4fb53dde77 s390/stp: add sysfs file to show scheduled leap seconds
This patch introduces /sys/devices/system/stp/scheduled_leap_seconds,
which will contain either 0,0 if no leap second is scheduled, or
the UTC timestamp + leap second offset.

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:21 +02:00
Sven Schnelle
b2539aa0d7 s390/stp: add support for leap seconds
In the current implementation, leap seconds are only synchronized
during the bootup process when the STP clock is synced. If the Leap
second offset (LSO) changes the machine must be rebooted, which is
not desired. This patch adds the required code to handle Leap second
changes during runtime. If the Leap second changes, a Configuration
change machine check is triggered. The STP code than schedules a Leap
second insertion/deletion with do_adjtimex().

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:21 +02:00
Sven Schnelle
4ab79ed223 s390/stp: use u32 instead of unsigned int
In hardware-dependent headers using u32 is easier
to read and less error-prone.

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:21 +02:00
Sven Schnelle
bb7d066a09 s390/stp: use __packed
Use __packed instead of __attribute__((packed))

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:20 +02:00
Sven Schnelle
b3bd02495c s390/stp: add locking to sysfs functions
The sysfs function might race with stp_work_fn. To prevent that,
add the required locking. Another issue is that the sysfs functions
are checking the stp_online flag, but this flag just holds the user
setting whether STP is enabled. Add a flag to clock_sync_flag whether
stp_info holds valid data and use that instead.

Cc: stable@vger.kernel.org
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-26 15:51:20 +02:00
Harald Freudenberger
fa6999e326 s390/pkey: support CCA and EP11 secure ECC private keys
This patch extends the pkey kernel module to support CCA
and EP11 secure ECC (private) keys as source for deriving
ECC protected (private) keys.

There is yet another new ioctl to support this: PKEY_KBLOB2PROTK3
can handle all the old keys plus CCA and EP11 secure ECC keys.
For details see ioctl description in pkey.h.

The CPACF unit currently only supports a subset of 5
different ECC curves (P-256, P-384, P-521, ED25519, ED448) and
so only keys of this curve type can be transformed into
protected keys. However, the pkey and the cca/ep11 low level
functions do not check this but simple pass-through the key
blob to the firmware onto the crypto cards. So most likely
the failure will be a response carrying an error code
resulting in user space errno value EIO instead of EINVAL.

Deriving a protected key from an EP11 ECC secure key
requires a CEX7 in EP11 mode. Deriving a protected key from
an CCA ECC secure key requires a CEX7 in CCA mode.

Together with this new ioctl the ioctls for querying lists
of apqns (PKEY_APQNS4K and PKEY_APQNS4KT) have been extended
to support EP11 and CCA ECC secure key type and key blobs.

Together with this ioctl there comes a new struct ep11kblob_header
which is to be prepended onto the EP11 key blob. See details
in pkey.h for the fields in there. The older EP11 AES key blob
with some info stored in the (unused) session field is also
supported with this new ioctl.

Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-24 10:03:28 +02:00
Vasily Gorbik
5596c4c106 s390/sclp: remove unused sclp_early_printk_forced
This reverts commit 55a5542a54 ("s390/hibernate: fix error handling when
suspend cpu != resume cpu"). It added sclp_early_printk_force() which
is no longer used since commit 394216275c ("s390: remove broken
hibernate / power management support"). No hibernate - no problem.

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-21 08:08:44 +02:00
Vasily Gorbik
bcf1650c9b s390/boot: avoid unnecessary zeroing of .bss section
.bss section is a part of the decompressor's image now, linker fills it
with zeros already. No need do it with memset additionally.

Reviewed-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-21 08:07:44 +02:00
Liu Shixin
61f2e77489 s390/diag: convert to use DEFINE_SEQ_ATTRIBUTE macro
Use DEFINE_SEQ_ATTRIBUTE macro to simplify the code.

Signed-off-by: Liu Shixin <liushixin2@huawei.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-17 14:11:03 +02:00
Heiko Carstens
110a6dbb2e s390/uaccess: add HAVE_GET_KERNEL_NOFAULT support
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-17 14:11:03 +02:00
Heiko Carstens
fc3f61e1bc s390/dis: get rid of set_fs() usage
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-17 14:11:03 +02:00
Vasily Gorbik
c360c9a238 s390/kasan: support protvirt with 4-level paging
Currently the kernel crashes in Kasan instrumentation code if
CONFIG_KASAN_S390_4_LEVEL_PAGING is used on protected virtualization
capable machine where the ultravisor imposes addressing limitations on
the host and those limitations are lower then KASAN_SHADOW_OFFSET.

The problem is that Kasan has to know in advance where vmalloc/modules
areas would be. With protected virtualization enabled vmalloc/modules
areas are moved down to the ultravisor secure storage limit while kasan
still expects them at the very end of 4-level paging address space.

To fix that make Kasan recognize when protected virtualization is enabled
and predefine vmalloc/modules areas position which are compliant with
ultravisor secure storage limit.

Kasan shadow itself stays in place and might reside above that ultravisor
secure storage limit.

One slight difference compaired to a kernel without Kasan enabled is that
vmalloc/modules areas position is not reverted to default if ultravisor
initialization fails. It would still be below the ultravisor secure
storage limit.

Kernel layout with kasan, 4-level paging and protected virtualization
enabled (ultravisor secure storage limit is at 0x0000800000000000):
---[ vmemmap Area Start ]---
0x0000400000000000-0x0000400080000000
---[ vmemmap Area End ]---
---[ vmalloc Area Start ]---
0x00007fe000000000-0x00007fff80000000
---[ vmalloc Area End ]---
---[ Modules Area Start ]---
0x00007fff80000000-0x0000800000000000
---[ Modules Area End ]---
---[ Kasan Shadow Start ]---
0x0018000000000000-0x001c000000000000
---[ Kasan Shadow End ]---
0x001c000000000000-0x0020000000000000         1P PGD I

Kernel layout with kasan, 4-level paging and protected virtualization
disabled/unsupported:
---[ vmemmap Area Start ]---
0x0000400000000000-0x0000400060000000
---[ vmemmap Area End ]---
---[ Kasan Shadow Start ]---
0x0018000000000000-0x001c000000000000
---[ Kasan Shadow End ]---
---[ vmalloc Area Start ]---
0x001fffe000000000-0x001fffff80000000
---[ vmalloc Area End ]---
---[ Modules Area Start ]---
0x001fffff80000000-0x0020000000000000
---[ Modules Area End ]---

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:48 +02:00
Vasily Gorbik
c2314cb2dd s390/protvirt: support ultravisor without secure storage limit
Avoid potential crash due to lack of secure storage limit. Check that
max_sec_stor_addr is not 0 before adjusting vmalloc position.

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Vasily Gorbik
1d6671ae46 s390/protvirt: parse prot_virt option in the decompressor
To make early kernel address space layout definition possible parse
prot_virt option in the decompressor and pass it to the uncompressed
kernel. This enables kasan to take ultravisor secure storage limit into
consideration and pre-define vmalloc position correctly.

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Vasily Gorbik
8f78657c29 s390/kasan: avoid unnecessary moving of vmemmap
Currently vmemmap area is unconditionally moved beyond Kasan shadow
memory. When Kasan is not enabled vmemmap area position is calculated
in setup_memory_end() and depends on limiting factors like ultravisor
secure storage limit. Try to follow the same logic with Kasan enabled
as well and avoid unnecessary vmemmap area position changes unless it
really intersects with Kasan shadow.

Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Vasily Gorbik
ee4b2ce6d1 s390/mm,ptdump: sort markers
Kasan configuration options and size of physical memory present could
affect kernel memory layout. In particular vmemmap, vmalloc and modules
might come before kasan shadow or after it. To make ptdump correctly
output markers in the right order markers have to be sorted.

To preserve the original order of markers with the same start address
avoid using sort() from lib/sort.c (which is not stable sorting algorithm)
and sort markers in place.

Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Niklas Schnelle
4904e1941e s390/pci: add missing pci_iov.h include
this fixes a missing prototype compiler warning spotted by the kernel
test robot.

Fixes: abb95b7550 ("s390/pci: consolidate SR-IOV specific code")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Heiko Carstens
48111b4838 s390/mm,ptdump: add proper ifdefs
Use ifdefs instead of IS_ENABLED() to avoid compile error
for !PTDUMP_DEBUGFS:

arch/s390/mm/dump_pagetables.c: In function ‘pt_dump_init’:
arch/s390/mm/dump_pagetables.c:248:64: error: ‘ptdump_fops’ undeclared (first use in this function); did you mean ‘pidfd_fops’?
   debugfs_create_file("kernel_page_tables", 0400, NULL, NULL, &ptdump_fops);

Reported-by: Julian Wiedmann <jwi@linux.ibm.com>
Fixes: 08c8e685c7 ("s390: add ARCH_HAS_DEBUG_WX support")
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Alexander Egorenkov
980d5f9ab3 s390/boot: enable .bss section for compressed kernel
- Support static uninitialized variables in compressed kernel.
- Remove chkbss script
- Get rid of workarounds for not having .bss section

Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-16 14:08:47 +02:00
Janosch Frank
1a80b54d1c s390/uv: add destroy page call
We don't need to export pages if we destroy the VM configuration
afterwards anyway. Instead we can destroy the page which will zero it
and then make it accessible to the host.

Destroying is about twice as fast as the export.

Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Link: https://lore.kernel.org/kvm/20200907124700.10374-2-frankja@linux.ibm.com/
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Vasily Gorbik
e670e64af1 s390/mm,ptdump: add couple of additional markers
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
[hca@linux.ibm.com: add more markers, rename some markers]
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Vasily Gorbik
d411e3c674 s390/kasan: make shadow memory noexec
ARCH_HAS_DEBUG_WX feature support brought attention to the fact that
currently initial kasan shadow memory mapped without noexec flag. So fix that.

Temporary initial identity mapping is still created without noexec, but
it is replaced by properly set up paging later.

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Heiko Carstens
08c8e685c7 s390: add ARCH_HAS_DEBUG_WX support
Checks the whole kernel address space for W+X mappings. Note that
currently the first lowcore page unfortunately has to be mapped
W+X. Therefore this not reported as an insecure mapping.

For the very same reason the wording is also different to other
architectures if the test passes:

On s390 it is "no unexpected W+X pages found" instead of
"no W+X pages found".

Tested-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Heiko Carstens
6bf9a639e7 s390/mm,ptdump: make page table dumping seq_file optional
s390 version of ae5d1cf358 ("arm64: dump: Make the page table
dumping seq_file optional").

Tested-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Heiko Carstens
6c6687a444 s390/kprobes: make insn pages read-only
Make sure that kprobe insn pages are not writable anymore.

Tested-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Niklas Schnelle
c3b2c9064e s390/pci: remove clp_rescan_pci_devices_simple()
clp_rescan_pci_devices_simple() is neither simpler than
clp_scan_pci_devices() nor does it really scan PCI devices, in particular
it will neither add newly discovered devices nor remove those which
disappeared.
Instead it only refreshes PCI function handles and also
has just a single callsite in the same translation unit left which
in fact only refreshes one specific function handle identified by
a FID.

Clarify this by renaming the function and its helper to
clp_refresh_fh() respectvely __clp_refresh_fh() and make it take
a fid directly which saves us dealing with the NULL case which
updated all function handles but is not used anymore.
Furthermore since the only callsite is in the same translation unit
make it static.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:35 +02:00
Niklas Schnelle
809fcfaf92 s390/pci: remove clp_rescan_pci_devices()
there is only one call site of clp_rescan_pci_devices() and
all the function does is call zpci_remove_reserved_devices()
followed by a duplicating clp_scan_pci_devices().
So inline the single call as a call to zpci_remove_reserved_devices()
and clp_scan_pci_devices() and remove the function.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Niklas Schnelle
2bce60b503 s390/pci: remove unused function zpci_rescan()
the only caller of this was removed as part of the suspend/resume
removal so no need to keep this function around.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Niklas Schnelle
abb95b7550 s390/pci: consolidate SR-IOV specific code
currently we have multiple #ifdef CONFIG_PCI_IOV blocks spread over
different compliation units and headers, all dealing with SR-IOV
specific behavior.
This violates the style guide which discourages conditionally compiled
code blocks and hinders maintainability by speading SR-IOV functionality
over many files.

Let's move all of this into a conditionally compiled pci_iov.c file and
local header and prefix SR-IOV specific functions with zpci_iov_*.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Heiko Carstens
da1694ad9e s390/mm,ptdump: hold cpa mutex while walking for kernel page table dump
This is currently only preventing that outdated information is
provided to user space. A concurrent split of huge/large pages does
modify the kernel page tables, however either the huge/large mapping
is reported or the split area is being walked.

This "fixes" also only a potential future bug, since split pages could
also be merged again if page permissions are the same for larger
memory areas.

Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Heiko Carstens
36c2733c43 s390/mm,ptdump: hold memory hotplug lock while walking for kernel page table dump
This is the s390 variant of commit bf2b59f60e ("arm64/mm: Hold
memory hotplug lock while walking for kernel page table dump").

Right now this doesn't fix any real bug, however as soon as kvm
patches get merged which make use of memory remove we might end up
dereferencing/accessing freed page tables.

Therefore fix this potential bug already now.

Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Heiko Carstens
9d719d39aa s390/mm,ptdump: convert to generic page table dumper
Make use of generic ptdump infrastructure.

Reviewed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 11:38:34 +02:00
Julian Wiedmann
180a4c42e5 s390/qdio: always use dev_name() for device name in QIB
Passing a custom name from the device driver is nice - but in practice
it's only zfcp who has been using this. So we might as well hard-code
a naming scheme in the qdio layer, so that qeth also benefits from it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 10:30:07 +02:00
Niklas Schnelle
b02002cc4c s390/pci: Implement ioremap_wc/prot() with MIO
With our current support for the new MIO PCI instructions, write
combining/write back MMIO memory can be obtained via the pci_iomap_wc()
and pci_iomap_wc_range() functions.
This is achieved by using the write back address for a specific bar
as provided in clp_store_query_pci_fn()

These functions are however not widely used and instead drivers often
rely on ioremap_wc() and ioremap_prot(), which on other platforms enable
write combining using a PTE flag set through the pgrprot value.

While we do not have a write combining flag in the low order flag bits
of the PTE like x86_64 does, with MIO support, there is a write back bit
in the physical address (bit 1 on z15) and thus also the PTE.
Which bit is used to toggle write back and whether it is available at
all, is however not fixed in the architecture. Instead we get this
information from the CLP Store Logical Processor Characteristics for PCI
command. When the write back bit is not provided we fall back to the
existing behavior.

Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Reviewed-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 10:30:07 +02:00
Julian Wiedmann
4d4a3caaf3 s390/qdio: clean up QDR setup
__qdio_allocate_fill_qdr() is meant to set up one specific queue
descriptor in the QDR. But for this simple task, it gets passed a bunch
of global structs and offsets - and then navigates through the structs
to find its actual operands.

Clean up all the complicated pointer chasing & index calculation, and
just pass a descriptor and its associated queue struct.

While at it also add some virt_to_phys() translations, to clarify that
addresses in the QDR are meant to be absolute.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 10:30:07 +02:00
Sven Schnelle
4bf3ec384e s390: disable branch profiling for vdso
When branch profiling is enabled, if () gets annotated with code to
instrument the hit/miss ratio. This doesn't work for VDSO as we can't
access kernel code. Add -DDISABLE_BRANCH_PROFILING to fix this.

Reported-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-09-14 10:30:07 +02:00
Sven Schnelle
4bff8cb545 s390: convert to GENERIC_VDSO
Convert s390 to generic vDSO. There are a few special things on s390:

- vDSO can be called without a stack frame - glibc did this in the past.
  So we need to allocate a stackframe on our own.

- The former assembly code used stcke to get the TOD clock and applied
  time steering to it. We need to do the same in the new code. This is done
  in the architecture specific __arch_get_hw_counter function. The steering
  information is stored in an architecure specific area in the vDSO data.

- CPUCLOCK_VIRT is now handled with a syscall fallback, which might
  be slower/less accurate than the old implementation.

The getcpu() function stays as an assembly function because there is no
generic implementation and the code is just a few lines.

Performance number from my system do 100 mio gettimeofday() calls:

Plain syscall: 8.6s
Generic VDSO:  1.3s
old ASM VDSO:  1s

So it's a bit slower but still much faster than syscalls.

Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:21 +02:00
Heiko Carstens
98ad45fb58 s390/checksum: coding style changes
Add some coding style changes which hopefully make the code
look a bit less odd.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:20 +02:00
Heiko Carstens
612ad0785d s390/checksum: have consistent calculations
Use "|" instead of "+" within csum_fold() for consistency reasons,
like in the rest of the file.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:20 +02:00
Heiko Carstens
614b4f5d0f s390/checksum: make ip_fast_csum() faster
Convert ip_fast_csum() so it doesn't call csum_partial(), but instead
open code the checksum calculation. The problem with csum_partial() is
that it makes use of the cksm instruction, which has high startup
costs and therefore is only very fast if used on larger memory
regions.

IPv4 headers however are small in size (5-16 32-bit words). The open
coded variant calculates the checksum in ~30% of the time compared to
the old variant (z14, march=z196).

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:20 +02:00
Heiko Carstens
bb4644b14a s390/checksum: rewrite csum_tcpudp_nofold()
Rewrite csum_tcpudp_nofold() so that the generated code will not
contain branches. The old implementation was also optimized for
machines which came with "add logical with carry" instructions,
however the compiler doesn't generate them anymore. This is most
likely because those instructions are slower.

However with the old code the compiler generates a lot of branches,
which isn't too helpful usually. Therefore rewrite the code.

In a tight loop this doesn't make any difference since the branch
prediction unit does its job.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:20 +02:00
Heiko Carstens
b064904c50 s390/checksum: provide csum_ipv6_magic()
This implementation needs only ~30% of the time to calculate the
checksum compared to the generic variant. In addition the compiler
also generates only ~30% of the instructions compared to the generic
variant (on z14, compiled with march=z196).

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-08-26 18:47:20 +02:00
Niklas Schnelle
b97bf44f99 s390/pci: fix PF/VF linking on hot plug
Currently there are four places in which a PCI function is scanned
and made available to drivers:
 1. In pci_scan_root_bus() as part of the initial zbus
    creation.
 2. In zpci_bus_add_devices() when registering
    a device in configured state on a zbus that has already been
    scanned.
 3. When a function is already known to zPCI (in reserved/standby state)
    and configuration is triggered through firmware by PEC 0x301.
 4. When a device is already known to zPCI (in standby/reserved state)
    and configuration is triggered from within Linux using
    enable_slot().

The PF/VF linking step and setting of pdev->is_virtfn introduced with
commit e5794cf1a2 ("s390/pci: create links between PFs and VFs") was
only triggered for the second case, which is where VFs created through
sriov_numvfs usually land. However unlike some other platforms but like
POWER VFs can be individually enabled/disabled through
/sys/bus/pci/slots.

Fix this by doing VF setup as part of pcibios_bus_add_device() which is
called in all of the above cases.

Finally to remove the PF/VF links call the common code
pci_iov_remove_virtfn() function to remove linked VFs.
This takes care of the necessary sysfs cleanup.

Fixes: e5794cf1a2 ("s390/pci: create links between PFs and VFs")
Cc: <stable@vger.kernel.org> # 5.8: 2f0230b2f2: s390/pci: re-introduce zpci_remove_device()
Cc: <stable@vger.kernel.org> # 5.8
Acked-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2020-08-17 13:17:34 +02:00
Niklas Schnelle
2f0230b2f2 s390/pci: re-introduce zpci_remove_device()
For fixing the PF to VF link removal we need to perform some action on
every removal of a zdev from the common PCI subsystem.
So in preparation re-introduce zpci_remove_device() and use that instead
of directly calling the common code functions. This  was actually still
declared from earlier code but no longer implemented.

Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2020-08-17 13:17:25 +02:00
Niklas Schnelle
3cddb79afc s390/pci: fix zpci_bus_link_virtfn()
We were missing the pci_dev_put() for candidate PFs.  Furhtermore in
discussion with upstream it turns out that somewhat counterintuitively
some common code, in particular the vfio-pci driver, assumes that
pdev->is_virtfn always implies that pdev->physfn is set, i.e. that VFs
are always linked.
While POWER does seem to set pdev->is_virtfn even for unlinked functions
(see comments in arch/powerpc/kernel/eeh.c:eeh_debugfs_break_device())
for now just be safe and only set pdev->is_virtfn on linking.
Also make sure that we only search for parent PFs if the zbus is
multifunction and we thus know the devfn values supplied by firmware
come from the RID.

Fixes: e5794cf1a2 ("s390/pci: create links between PFs and VFs")
Cc: <stable@vger.kernel.org> # 5.8
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2020-08-17 13:17:18 +02:00