AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2024-11-24 02:30:52 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity

Remove Mysterious Macro Intended to Obscure Weird Behaviours (mmiowb())

Browse Source 
Remove mmiowb() from the kernel memory barrier API and instead, for
 architectures that need it, hide the barrier inside spin_unlock() when
 MMIO has been performed inside the critical section.
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAlzMFaUACgkQt6xw3ITB
 YzRICQgAiv7wF/yIbBhDOmCNCAKDO59chvFQWxXWdGk/aAB56kwKAMXJgLOvlMG/
 VRuuLyParTFQETC3jaxKgnO/1hb+PZLDt2Q2KqixtjIzBypKUPWvK2sf6THhSRF1
 GK0DBVUd1rCrWrR815+SPb8el4xXtdBzvAVB+Fx35PXVNpdRdqCkK+EQ6UnXGokm
 rXXHbnfsnquBDtmb4CR4r2beH+aNElXbdt0Kj8VcE5J7f7jTdW3z6Q9WFRvdKmK7
 yrsxXXB2w/EsWXOwFp0SLTV5+fgeGgTvv8uLjDw+SG6t0E0PebxjNAflT7dPrbYL
 WecjKC9WqBxrGY+4ew6YJP70ijLBCw==
 =aC8m
 -----END PGP SIGNATURE-----

Merge tag 'arm64-mmiowb' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux

Pull mmiowb removal from Will Deacon:
 "Remove Mysterious Macro Intended to Obscure Weird Behaviours (mmiowb())

  Remove mmiowb() from the kernel memory barrier API and instead, for
  architectures that need it, hide the barrier inside spin_unlock() when
  MMIO has been performed inside the critical section.

  The only relatively recent changes have been addressing review
  comments on the documentation, which is in a much better shape thanks
  to the efforts of Ben and Ingo.

  I was initially planning to split this into two pull requests so that
  you could run the coccinelle script yourself, however it's been plain
  sailing in linux-next so I've just included the whole lot here to keep
  things simple"

* tag 'arm64-mmiowb' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (23 commits)
  docs/memory-barriers.txt: Update I/O section to be clearer about CPU vs thread
  docs/memory-barriers.txt: Fix style, spacing and grammar in I/O section
  arch: Remove dummy mmiowb() definitions from arch code
  net/ethernet/silan/sc92031: Remove stale comment about mmiowb()
  i40iw: Redefine i40iw_mmiowb() to do nothing
  scsi/qla1280: Remove stale comment about mmiowb()
  drivers: Remove explicit invocations of mmiowb()
  drivers: Remove useless trailing comments from mmiowb() invocations
  Documentation: Kill all references to mmiowb()
  riscv/mmiowb: Hook up mmwiob() implementation to asm-generic code
  powerpc/mmiowb: Hook up mmwiob() implementation to asm-generic code
  ia64/mmiowb: Add unconditional mmiowb() to arch_spin_unlock()
  mips/mmiowb: Add unconditional mmiowb() to arch_spin_unlock()
  sh/mmiowb: Add unconditional mmiowb() to arch_spin_unlock()
  m68k/io: Remove useless definition of mmiowb()
  nds32/io: Remove useless definition of mmiowb()
  x86/io: Remove useless definition of mmiowb()
  arm64/io: Remove useless definition of mmiowb()
  ARM/io: Remove useless definition of mmiowb()
  mmiowb: Hook up mmiowb helpers to spinlocks and generic I/O accessors
  ...
This commit is contained in:
Linus Torvalds 2019-05-06 16:57:52 -07:00
commit dd4e5d6106
181 changed files with 343 additions and 828 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
Documentation/driver-api/device-io.rst
View File

@ -103,51 +103,6 @@ continuing execution::
ha->flags.ints_enabled = 0;
}
In addition to write posting, on some large multiprocessing systems
(e.g. SGI Challenge, Origin and Altix machines) posted writes won't be
strongly ordered coming from different CPUs. Thus it's important to
properly protect parts of your driver that do memory-mapped writes with
locks and use the :c:func:`mmiowb()` to make sure they arrive in the
order intended. Issuing a regular readX() will also ensure write ordering,
but should only be used when the
driver has to be sure that the write has actually arrived at the device
(not that it's simply ordered with respect to other writes), since a
full readX() is a relatively expensive operation.
Generally, one should use :c:func:`mmiowb()` prior to releasing a spinlock
that protects regions using :c:func:`writeb()` or similar functions that
aren't surrounded by readb() calls, which will ensure ordering
and flushing. The following pseudocode illustrates what might occur if
write ordering isn't guaranteed via :c:func:`mmiowb()` or one of the
readX() functions::
CPU A: spin_lock_irqsave(&dev_lock, flags)
CPU A: ...
CPU A: writel(newval, ring_ptr);
CPU A: spin_unlock_irqrestore(&dev_lock, flags)
...
CPU B: spin_lock_irqsave(&dev_lock, flags)
CPU B: writel(newval2, ring_ptr);
CPU B: ...
CPU B: spin_unlock_irqrestore(&dev_lock, flags)
In the case above, newval2 could be written to ring_ptr before newval.
Fixing it is easy though::
CPU A: spin_lock_irqsave(&dev_lock, flags)
CPU A: ...
CPU A: writel(newval, ring_ptr);
CPU A: mmiowb(); /* ensure no other writes beat us to the device */
CPU A: spin_unlock_irqrestore(&dev_lock, flags)
...
CPU B: spin_lock_irqsave(&dev_lock, flags)
CPU B: writel(newval2, ring_ptr);
CPU B: ...
CPU B: mmiowb();
CPU B: spin_unlock_irqrestore(&dev_lock, flags)
See tg3.c for a real world example of how to use :c:func:`mmiowb()`
PCI ordering rules also guarantee that PIO read responses arrive after any
outstanding DMA writes from that bus, since for some devices the result of
a readb() call may signal to the driver that a DMA transaction is

4
Documentation/driver-api/pci/p2pdma.rst
View File

@ -132,10 +132,6 @@ precludes passing these pages to userspace.
P2P memory is also technically IO memory but should never have any side
effects behind it. Thus, the order of loads and stores should not be important
and ioreadX(), iowriteX() and friends should not be necessary.
However, as the memory is not cache coherent, if access ever needs to
be protected by a spinlock then :c:func:`mmiowb()` must be used before
unlocking the lock. (See ACQUIRES VS I/O ACCESSES in
Documentation/memory-barriers.txt)
P2P DMA Support Library

249
Documentation/memory-barriers.txt
View File

@ -1937,21 +1937,6 @@ There are some more advanced barrier functions:
information on consistent memory.
MMIO WRITE BARRIER
------------------
The Linux kernel also has a special barrier for use with memory-mapped I/O
writes:
mmiowb();
This is a variation on the mandatory write barrier that causes writes to weakly
ordered I/O regions to be partially ordered. Its effects may go beyond the
CPU->Hardware interface and actually affect the hardware at some level.
See the subsection "Acquires vs I/O accesses" for more information.
===============================
IMPLICIT KERNEL MEMORY BARRIERS
===============================
@ -2317,75 +2302,6 @@ But it won't see any of:
*E, *F or *G following RELEASE Q
ACQUIRES VS I/O ACCESSES
------------------------
Under certain circumstances (especially involving NUMA), I/O accesses within
two spinlocked sections on two different CPUs may be seen as interleaved by the
PCI bridge, because the PCI bridge does not necessarily participate in the
cache-coherence protocol, and is therefore incapable of issuing the required
read memory barriers.
For example:
CPU 1 CPU 2
=============================== ===============================
spin_lock(Q)
writel(0, ADDR)
writel(1, DATA);
spin_unlock(Q);
spin_lock(Q);
writel(4, ADDR);
writel(5, DATA);
spin_unlock(Q);
may be seen by the PCI bridge as follows:
STORE *ADDR = 0, STORE *ADDR = 4, STORE *DATA = 1, STORE *DATA = 5
which would probably cause the hardware to malfunction.
What is necessary here is to intervene with an mmiowb() before dropping the
spinlock, for example:
CPU 1 CPU 2
=============================== ===============================
spin_lock(Q)
writel(0, ADDR)
writel(1, DATA);
mmiowb();
spin_unlock(Q);
spin_lock(Q);
writel(4, ADDR);
writel(5, DATA);
mmiowb();
spin_unlock(Q);
this will ensure that the two stores issued on CPU 1 appear at the PCI bridge
before either of the stores issued on CPU 2.
Furthermore, following a store by a load from the same device obviates the need
for the mmiowb(), because the load forces the store to complete before the load
is performed:
CPU 1 CPU 2
=============================== ===============================
spin_lock(Q)
writel(0, ADDR)
a = readl(DATA);
spin_unlock(Q);
spin_lock(Q);
writel(4, ADDR);
b = readl(DATA);
spin_unlock(Q);
See Documentation/driver-api/device-io.rst for more information.
=================================
WHERE ARE MEMORY BARRIERS NEEDED?
=================================
@ -2532,16 +2448,9 @@ the device to malfunction.
Inside of the Linux kernel, I/O should be done through the appropriate accessor
routines - such as inb() or writel() - which know how to make such accesses
appropriately sequential. While this, for the most part, renders the explicit
use of memory barriers unnecessary, there are a couple of situations where they
might be needed:
(1) On some systems, I/O stores are not strongly ordered across all CPUs, and
so for _all_ general drivers locks should be used and mmiowb() must be
issued prior to unlocking the critical section.
(2) If the accessor functions are used to refer to an I/O memory window with
relaxed memory access properties, then _mandatory_ memory barriers are
required to enforce ordering.
use of memory barriers unnecessary, if the accessor functions are used to refer
to an I/O memory window with relaxed memory access properties, then _mandatory_
memory barriers are required to enforce ordering.
See Documentation/driver-api/device-io.rst for more information.
@ -2586,8 +2495,7 @@ explicit barriers are used.
Normally this won't be a problem because the I/O accesses done inside such
sections will include synchronous load operations on strictly ordered I/O
registers that form implicit I/O barriers. If this isn't sufficient then an
mmiowb() may need to be used explicitly.
registers that form implicit I/O barriers.
A similar situation may occur between an interrupt routine and two routines
@ -2599,71 +2507,114 @@ likely, then interrupt-disabling locks should be used to guarantee ordering.
KERNEL I/O BARRIER EFFECTS
==========================
When accessing I/O memory, drivers should use the appropriate accessor
functions:
(*) inX(), outX():
These are intended to talk to I/O space rather than memory space, but
that's primarily a CPU-specific concept. The i386 and x86_64 processors
do indeed have special I/O space access cycles and instructions, but many
CPUs don't have such a concept.
The PCI bus, amongst others, defines an I/O space concept which - on such
CPUs as i386 and x86_64 - readily maps to the CPU's concept of I/O
space. However, it may also be mapped as a virtual I/O space in the CPU's
memory map, particularly on those CPUs that don't support alternate I/O
spaces.
Accesses to this space may be fully synchronous (as on i386), but
intermediary bridges (such as the PCI host bridge) may not fully honour
that.
They are guaranteed to be fully ordered with respect to each other.
They are not guaranteed to be fully ordered with respect to other types of
memory and I/O operation.
Interfacing with peripherals via I/O accesses is deeply architecture and device
specific. Therefore, drivers which are inherently non-portable may rely on
specific behaviours of their target systems in order to achieve synchronization
in the most lightweight manner possible. For drivers intending to be portable
between multiple architectures and bus implementations, the kernel offers a
series of accessor functions that provide various degrees of ordering
guarantees:
(*) readX(), writeX():
Whether these are guaranteed to be fully ordered and uncombined with
respect to each other on the issuing CPU depends on the characteristics
defined for the memory window through which they're accessing. On later
i386 architecture machines, for example, this is controlled by way of the
MTRR registers.
The readX() and writeX() MMIO accessors take a pointer to the
peripheral being accessed as an __iomem * parameter. For pointers
mapped with the default I/O attributes (e.g. those returned by
ioremap()), the ordering guarantees are as follows:
Ordinarily, these will be guaranteed to be fully ordered and uncombined,
provided they're not accessing a prefetchable device.
1. All readX() and writeX() accesses to the same peripheral are ordered
with respect to each other. This ensures that MMIO register accesses
by the same CPU thread to a particular device will arrive in program
order.
However, intermediary hardware (such as a PCI bridge) may indulge in
deferral if it so wishes; to flush a store, a load from the same location
is preferred[*], but a load from the same device or from configuration
space should suffice for PCI.
2. A writeX() issued by a CPU thread holding a spinlock is ordered
before a writeX() to the same peripheral from another CPU thread
issued after a later acquisition of the same spinlock. This ensures
that MMIO register writes to a particular device issued while holding
a spinlock will arrive in an order consistent with acquisitions of
the lock.
[*] NOTE! attempting to load from the same location as was written to may
cause a malfunction - consider the 16550 Rx/Tx serial registers for
example.
3. A writeX() by a CPU thread to the peripheral will first wait for the
completion of all prior writes to memory either issued by, or
propagated to, the same thread. This ensures that writes by the CPU
to an outbound DMA buffer allocated by dma_alloc_coherent() will be
visible to a DMA engine when the CPU writes to its MMIO control
register to trigger the transfer.
Used with prefetchable I/O memory, an mmiowb() barrier may be required to
force stores to be ordered.
4. A readX() by a CPU thread from the peripheral will complete before
any subsequent reads from memory by the same thread can begin. This
ensures that reads by the CPU from an incoming DMA buffer allocated
by dma_alloc_coherent() will not see stale data after reading from
the DMA engine's MMIO status register to establish that the DMA
transfer has completed.
Please refer to the PCI specification for more information on interactions
between PCI transactions.
5. A readX() by a CPU thread from the peripheral will complete before
any subsequent delay() loop can begin execution on the same thread.
This ensures that two MMIO register writes by the CPU to a peripheral
will arrive at least 1us apart if the first write is immediately read
back with readX() and udelay(1) is called prior to the second
writeX():
(*) readX_relaxed(), writeX_relaxed()
writel(42, DEVICE_REGISTER_0); // Arrives at the device...
readl(DEVICE_REGISTER_0);
udelay(1);
writel(42, DEVICE_REGISTER_1); // ...at least 1us before this.
These are similar to readX() and writeX(), but provide weaker memory
ordering guarantees. Specifically, they do not guarantee ordering with
respect to normal memory accesses (e.g. DMA buffers) nor do they guarantee
ordering with respect to LOCK or UNLOCK operations. If the latter is
required, an mmiowb() barrier can be used. Note that relaxed accesses to
the same peripheral are guaranteed to be ordered with respect to each
other.
The ordering properties of __iomem pointers obtained with non-default
attributes (e.g. those returned by ioremap_wc()) are specific to the
underlying architecture and therefore the guarantees listed above cannot
generally be relied upon for accesses to these types of mappings.
(*) ioreadX(), iowriteX()
(*) readX_relaxed(), writeX_relaxed():
These will perform appropriately for the type of access they're actually
doing, be it inX()/outX() or readX()/writeX().
These are similar to readX() and writeX(), but provide weaker memory
ordering guarantees. Specifically, they do not guarantee ordering with
respect to locking, normal memory accesses or delay() loops (i.e.
bullets 2-5 above) but they are still guaranteed to be ordered with
respect to other accesses from the same CPU thread to the same
peripheral when operating on __iomem pointers mapped with the default
I/O attributes.
(*) readsX(), writesX():
The readsX() and writesX() MMIO accessors are designed for accessing
register-based, memory-mapped FIFOs residing on peripherals that are not
capable of performing DMA. Consequently, they provide only the ordering
guarantees of readX_relaxed() and writeX_relaxed(), as documented above.
(*) inX(), outX():
The inX() and outX() accessors are intended to access legacy port-mapped
I/O peripherals, which may require special instructions on some
architectures (notably x86). The port number of the peripheral being
accessed is passed as an argument.
Since many CPU architectures ultimately access these peripherals via an
internal virtual memory mapping, the portable ordering guarantees
provided by inX() and outX() are the same as those provided by readX()
and writeX() respectively when accessing a mapping with the default I/O
attributes.
Device drivers may expect outX() to emit a non-posted write transaction
that waits for a completion response from the I/O peripheral before
returning. This is not guaranteed by all architectures and is therefore
not part of the portable ordering semantics.
(*) insX(), outsX():
As above, the insX() and outsX() accessors provide the same ordering
guarantees as readsX() and writesX() respectively when accessing a
mapping with the default I/O attributes.
(*) ioreadX(), iowriteX():
These will perform appropriately for the type of access they're actually
doing, be it inX()/outX() or readX()/writeX().
With the exception of the string accessors (insX(), outsX(), readsX() and
writesX()), all of the above assume that the underlying peripheral is
little-endian and will therefore perform byte-swapping operations on big-endian
architectures.
========================================

1
arch/alpha/include/asm/Kbuild
View File

@ -9,6 +9,7 @@ generic-y += irq_work.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += preempt.h
generic-y += sections.h
generic-y += trace_clock.h

2
arch/alpha/include/asm/io.h
View File

@ -513,8 +513,6 @@ extern inline void writeq(u64 b, volatile void __iomem *addr)
#define writel_relaxed(b, addr) __raw_writel(b, addr)
#define writeq_relaxed(b, addr) __raw_writeq(b, addr)
#define mmiowb()
/*
* String version of IO memory access ops:
*/

1
arch/arc/include/asm/Kbuild
View File

@ -16,6 +16,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += msi.h
generic-y += parport.h
generic-y += percpu.h

1
arch/arm/include/asm/Kbuild
View File

@ -9,6 +9,7 @@ generic-y += kdebug.h
generic-y += local.h
generic-y += local64.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += msi.h
generic-y += parport.h
generic-y += preempt.h

2
arch/arm/include/asm/io.h
View File

@ -281,8 +281,6 @@ extern void _memcpy_fromio(void *, const volatile void __iomem *, size_t);
extern void _memcpy_toio(volatile void __iomem *, const void *, size_t);
extern void _memset_io(volatile void __iomem *, int, size_t);
#define mmiowb()
/*
* Memory access primitives
* ------------------------

1
arch/arm64/include/asm/Kbuild
View File

@ -13,6 +13,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += msi.h
generic-y += qrwlock.h
generic-y += qspinlock.h

2
arch/arm64/include/asm/io.h
View File

@ -124,8 +124,6 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
#define __io_par(v) __iormb(v)
#define __iowmb() wmb()
#define mmiowb() do { } while (0)
/*
* Relaxed I/O memory access primitives. These follow the Device memory
* ordering rules but do not guarantee any ordering relative to Normal memory

1
arch/c6x/include/asm/Kbuild
View File

@ -23,6 +23,7 @@ generic-y += kvm_para.h
generic-y += local.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += mmu.h
generic-y += mmu_context.h
generic-y += pci.h

1
arch/csky/include/asm/Kbuild
View File

@ -28,6 +28,7 @@ generic-y += linkage.h
generic-y += local.h
generic-y += local64.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += module.h
generic-y += mutex.h
generic-y += pci.h

1
arch/h8300/include/asm/Kbuild
View File

@ -29,6 +29,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += mmu.h
generic-y += mmu_context.h
generic-y += module.h

1
arch/hexagon/include/asm/Kbuild
View File

@ -24,6 +24,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += pci.h
generic-y += percpu.h
generic-y += preempt.h

2
arch/hexagon/include/asm/io.h
View File

@ -184,8 +184,6 @@ static inline void writel(u32 data, volatile void __iomem *addr)
#define writew_relaxed __raw_writew
#define writel_relaxed __raw_writel
#define mmiowb()
/*
* Need an mtype somewhere in here, for cache type deals?
* This is probably too long for an inline.

17
arch/ia64/include/asm/io.h
View File

@ -113,20 +113,6 @@ extern int valid_mmap_phys_addr_range (unsigned long pfn, size_t count);
*/
#define __ia64_mf_a() ia64_mfa()
/**
* ___ia64_mmiowb - I/O write barrier
*
* Ensure ordering of I/O space writes. This will make sure that writes
* following the barrier will arrive after all previous writes. For most
* ia64 platforms, this is a simple 'mf.a' instruction.
*
* See Documentation/driver-api/device-io.rst for more information.
*/
static inline void ___ia64_mmiowb(void)
{
ia64_mfa();
}
static inline void*
__ia64_mk_io_addr (unsigned long port)
{
@ -161,7 +147,6 @@ __ia64_mk_io_addr (unsigned long port)
#define __ia64_writew ___ia64_writew
#define __ia64_writel ___ia64_writel
#define __ia64_writeq ___ia64_writeq
#define __ia64_mmiowb ___ia64_mmiowb
/*
* For the in/out routines, we need to do "mf.a" _after_ doing the I/O access to ensure
@ -296,7 +281,6 @@ __outsl (unsigned long port, const void *src, unsigned long count)
#define __outb platform_outb
#define __outw platform_outw
#define __outl platform_outl
#define __mmiowb platform_mmiowb
#define inb(p) __inb(p)
#define inw(p) __inw(p)
@ -310,7 +294,6 @@ __outsl (unsigned long port, const void *src, unsigned long count)
#define outsb(p,s,c) __outsb(p,s,c)
#define outsw(p,s,c) __outsw(p,s,c)
#define outsl(p,s,c) __outsl(p,s,c)
#define mmiowb() __mmiowb()
/*
* The address passed to these functions are ioremap()ped already.

25
arch/ia64/include/asm/mmiowb.h Normal file
View File

@ -0,0 +1,25 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_IA64_MMIOWB_H
#define _ASM_IA64_MMIOWB_H
#include <asm/machvec.h>
/**
* ___ia64_mmiowb - I/O write barrier
*
* Ensure ordering of I/O space writes. This will make sure that writes
* following the barrier will arrive after all previous writes. For most
* ia64 platforms, this is a simple 'mf.a' instruction.
*/
static inline void ___ia64_mmiowb(void)
{
ia64_mfa();
}
#define __ia64_mmiowb ___ia64_mmiowb
#define mmiowb() platform_mmiowb()
#include <asm-generic/mmiowb.h>
#endif /* _ASM_IA64_MMIOWB_H */

2
arch/ia64/include/asm/spinlock.h
View File

@ -73,6 +73,8 @@ static __always_inline void __ticket_spin_unlock(arch_spinlock_t *lock)
{
unsigned short *p = (unsigned short *)&lock->lock + 1, tmp;
/* This could be optimised with ARCH_HAS_MMIOWB */
mmiowb();
asm volatile ("ld2.bias %0=[%1]" : "=r"(tmp) : "r"(p));
WRITE_ONCE(*p, (tmp + 2) & ~1);
}

1
arch/m68k/include/asm/Kbuild
View File

@ -18,6 +18,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += percpu.h
generic-y += preempt.h
generic-y += sections.h

2
arch/m68k/include/asm/io_mm.h
View File

@ -377,8 +377,6 @@ static inline void isa_delay(void)
#define writesw(port, buf, nr) raw_outsw((port), (u16 *)(buf), (nr))
#define writesl(port, buf, nr) raw_outsl((port), (u32 *)(buf), (nr))
#define mmiowb()
#ifndef CONFIG_SUN3
#define IO_SPACE_LIMIT 0xffff
#else

1
arch/microblaze/include/asm/Kbuild
View File

@ -23,6 +23,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += parport.h
generic-y += percpu.h
generic-y += preempt.h

3
arch/mips/include/asm/io.h
View File

@ -102,9 +102,6 @@ static inline void set_io_port_base(unsigned long base)
#define iobarrier_w() wmb()
#define iobarrier_sync() iob()
/* Some callers use this older API instead. */
#define mmiowb() iobarrier_w()
/*
* virt_to_phys - map virtual addresses to physical
* @address: address to remap

11
arch/mips/include/asm/mmiowb.h Normal file
View File

@ -0,0 +1,11 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_MMIOWB_H
#define _ASM_MMIOWB_H
#include <asm/io.h>
#define mmiowb() iobarrier_w()
#include <asm-generic/mmiowb.h>
#endif /* _ASM_MMIOWB_H */

15
arch/mips/include/asm/spinlock.h
View File

@ -11,6 +11,21 @@
#include <asm/processor.h>
#include <asm/qrwlock.h>
#include <asm-generic/qspinlock_types.h>
#define queued_spin_unlock queued_spin_unlock
/**
* queued_spin_unlock - release a queued spinlock
* @lock : Pointer to queued spinlock structure
*/
static inline void queued_spin_unlock(struct qspinlock *lock)
{
/* This could be optimised with ARCH_HAS_MMIOWB */
mmiowb();
smp_store_release(&lock->locked, 0);
}
#include <asm/qspinlock.h>
#endif /* _ASM_SPINLOCK_H */

1
arch/nds32/include/asm/Kbuild
View File

@ -31,6 +31,7 @@ generic-y += limits.h
generic-y += local.h
generic-y += local64.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += parport.h
generic-y += pci.h
generic-y += percpu.h

2
arch/nds32/include/asm/io.h
View File

@ -55,8 +55,6 @@ static inline u32 __raw_readl(const volatile void __iomem *addr)
#define __iormb() rmb()
#define __iowmb() wmb()
#define mmiowb() __asm__ __volatile__ ("msync all" : : : "memory");
/*
* {read,write}{b,w,l,q}_relaxed() are like the regular version, but
* are not guaranteed to provide ordering against spinlocks or memory

1
arch/nios2/include/asm/Kbuild
View File

@ -27,6 +27,7 @@ generic-y += kvm_para.h
generic-y += local.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += module.h
generic-y += pci.h
generic-y += percpu.h

1
arch/openrisc/include/asm/Kbuild
View File

@ -24,6 +24,7 @@ generic-y += kvm_para.h
generic-y += local.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += module.h
generic-y += pci.h
generic-y += percpu.h

1
arch/parisc/include/asm/Kbuild
View File

@ -16,6 +16,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += percpu.h
generic-y += preempt.h
generic-y += seccomp.h

2
arch/parisc/include/asm/io.h
View File

@ -229,8 +229,6 @@ static inline void writeq(unsigned long long q, volatile void __iomem *addr)
#define writel_relaxed(l, addr) writel(l, addr)
#define writeq_relaxed(q, addr) writeq(q, addr)
#define mmiowb() do { } while (0)
void memset_io(volatile void __iomem *addr, unsigned char val, int count);
void memcpy_fromio(void *dst, const volatile void __iomem *src, int count);
void memcpy_toio(volatile void __iomem *dst, const void *src, int count);

1
arch/powerpc/Kconfig
View File

@ -125,6 +125,7 @@ config PPC
select ARCH_HAS_FORTIFY_SOURCE
select ARCH_HAS_GCOV_PROFILE_ALL
select ARCH_HAS_KCOV
select ARCH_HAS_MMIOWB if PPC64
select ARCH_HAS_PHYS_TO_DMA
select ARCH_HAS_PMEM_API if PPC64
select ARCH_HAS_PTE_SPECIAL

33
arch/powerpc/include/asm/io.h
View File

@ -34,14 +34,11 @@ extern struct pci_dev *isa_bridge_pcidev;
#include <asm/byteorder.h>
#include <asm/synch.h>
#include <asm/delay.h>
#include <asm/mmiowb.h>
#include <asm/mmu.h>
#include <asm/ppc_asm.h>
#include <asm/pgtable.h>
#ifdef CONFIG_PPC64
#include <asm/paca.h>
#endif
#define SIO_CONFIG_RA 0x398
#define SIO_CONFIG_RD 0x399
@ -107,12 +104,6 @@ extern bool isa_io_special;
*
*/
#ifdef CONFIG_PPC64
#define IO_SET_SYNC_FLAG() do { local_paca->io_sync = 1; } while(0)
#else
#define IO_SET_SYNC_FLAG()
#endif
#define DEF_MMIO_IN_X(name, size, insn) \
static inline u##size name(const volatile u##size __iomem *addr) \
{ \
@ -127,7 +118,7 @@ static inline void name(volatile u##size __iomem *addr, u##size val) \
{ \
__asm__ __volatile__("sync;"#insn" %1,%y0" \
: "=Z" (*addr) : "r" (val) : "memory"); \
IO_SET_SYNC_FLAG(); \
mmiowb_set_pending(); \
}
#define DEF_MMIO_IN_D(name, size, insn) \
@ -144,7 +135,7 @@ static inline void name(volatile u##size __iomem *addr, u##size val) \
{ \
__asm__ __volatile__("sync;"#insn"%U0%X0 %1,%0" \
: "=m" (*addr) : "r" (val) : "memory"); \
IO_SET_SYNC_FLAG(); \
mmiowb_set_pending(); \
}
DEF_MMIO_IN_D(in_8, 8, lbz);
@ -652,24 +643,6 @@ static inline void name at \
#include <asm-generic/iomap.h>
#ifdef CONFIG_PPC32
#define mmiowb()
#else
/*
* Enforce synchronisation of stores vs. spin_unlock
* (this does it explicitly, though our implementation of spin_unlock
* does it implicitely too)
*/
static inline void mmiowb(void)
{
unsigned long tmp;
__asm__ __volatile__("sync; li %0,0; stb %0,%1(13)"
: "=&r" (tmp) : "i" (offsetof(struct paca_struct, io_sync))
: "memory");
}
#endif /* !CONFIG_PPC32 */
static inline void iosync(void)
{
__asm__ __volatile__ ("sync" : : : "memory");

18
arch/powerpc/include/asm/mmiowb.h Normal file
View File

@ -0,0 +1,18 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_POWERPC_MMIOWB_H
#define _ASM_POWERPC_MMIOWB_H
#ifdef CONFIG_MMIOWB
#include <linux/compiler.h>
#include <asm/barrier.h>
#include <asm/paca.h>
#define arch_mmiowb_state() (&local_paca->mmiowb_state)
#define mmiowb() mb()
#endif /* CONFIG_MMIOWB */
#include <asm-generic/mmiowb.h>
#endif /* _ASM_POWERPC_MMIOWB_H */

6
arch/powerpc/include/asm/paca.h
View File

@ -34,6 +34,8 @@
#include <asm/cpuidle.h>
#include <asm/atomic.h>
#include <asm-generic/mmiowb_types.h>
register struct paca_struct *local_paca asm("r13");
#if defined(CONFIG_DEBUG_PREEMPT) && defined(CONFIG_SMP)
@ -171,7 +173,6 @@ struct paca_struct {
u16 trap_save; /* Used when bad stack is encountered */
u8 irq_soft_mask; /* mask for irq soft masking */
u8 irq_happened; /* irq happened while soft-disabled */
u8 io_sync; /* writel() needs spin_unlock sync */
u8 irq_work_pending; /* IRQ_WORK interrupt while soft-disable */
u8 nap_state_lost; /* NV GPR values lost in power7_idle */
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
@ -264,6 +265,9 @@ struct paca_struct {
#ifdef CONFIG_STACKPROTECTOR
unsigned long canary;
#endif
#ifdef CONFIG_MMIOWB
struct mmiowb_state mmiowb_state;
#endif
} ____cacheline_aligned;
extern void copy_mm_to_paca(struct mm_struct *mm);

17
arch/powerpc/include/asm/spinlock.h
View File

@ -39,19 +39,6 @@
#define LOCK_TOKEN 1
#endif
#if defined(CONFIG_PPC64) && defined(CONFIG_SMP)
#define CLEAR_IO_SYNC (get_paca()->io_sync = 0)
#define SYNC_IO do { \
if (unlikely(get_paca()->io_sync)) { \
mb(); \
get_paca()->io_sync = 0; \
} \
} while (0)
#else
#define CLEAR_IO_SYNC
#define SYNC_IO
#endif
#ifdef CONFIG_PPC_PSERIES
#define vcpu_is_preempted vcpu_is_preempted
static inline bool vcpu_is_preempted(int cpu)
@ -99,7 +86,6 @@ static inline unsigned long __arch_spin_trylock(arch_spinlock_t *lock)
static inline int arch_spin_trylock(arch_spinlock_t *lock)
{
CLEAR_IO_SYNC;
return __arch_spin_trylock(lock) == 0;
}
@ -130,7 +116,6 @@ extern void __rw_yield(arch_rwlock_t *lock);
static inline void arch_spin_lock(arch_spinlock_t *lock)
{
CLEAR_IO_SYNC;
while (1) {
if (likely(__arch_spin_trylock(lock) == 0))
break;
@ -148,7 +133,6 @@ void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags)
{
unsigned long flags_dis;
CLEAR_IO_SYNC;
while (1) {
if (likely(__arch_spin_trylock(lock) == 0))
break;
@ -167,7 +151,6 @@ void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags)
static inline void arch_spin_unlock(arch_spinlock_t *lock)
{
SYNC_IO;
__asm__ __volatile__("# arch_spin_unlock\n\t"
PPC_RELEASE_BARRIER: : :"memory");
lock->slock = 0;

5
arch/powerpc/xmon/xmon.c
View File

@ -2429,7 +2429,10 @@ static void dump_one_paca(int cpu)
DUMP(p, trap_save, "%#-*x");
DUMP(p, irq_soft_mask, "%#-*x");
DUMP(p, irq_happened, "%#-*x");
DUMP(p, io_sync, "%#-*x");
#ifdef CONFIG_MMIOWB
DUMP(p, mmiowb_state.nesting_count, "%#-*x");
DUMP(p, mmiowb_state.mmiowb_pending, "%#-*x");
#endif
DUMP(p, irq_work_pending, "%#-*x");
DUMP(p, nap_state_lost, "%#-*x");
DUMP(p, sprg_vdso, "%#-*llx");

1
arch/riscv/Kconfig
View File

@ -48,6 +48,7 @@ config RISCV
select RISCV_TIMER
select GENERIC_IRQ_MULTI_HANDLER
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_MMIOWB
select HAVE_EBPF_JIT if 64BIT
config MMU

15
arch/riscv/include/asm/io.h
View File

@ -20,6 +20,7 @@
#define _ASM_RISCV_IO_H
#include <linux/types.h>
#include <asm/mmiowb.h>
extern void __iomem *ioremap(phys_addr_t offset, unsigned long size);
@ -99,18 +100,6 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
}
#endif
/*
* FIXME: I'm flip-flopping on whether or not we should keep this or enforce
* the ordering with I/O on spinlocks like PowerPC does. The worry is that
* drivers won't get this correct, but I also don't want to introduce a fence
* into the lock code that otherwise only uses AMOs (and is essentially defined
* by the ISA to be correct). For now I'm leaving this here: "o,w" is
* sufficient to ensure that all writes to the device have completed before the
* write to the spinlock is allowed to commit. I surmised this from reading
* "ACQUIRES VS I/O ACCESSES" in memory-barriers.txt.
*/
#define mmiowb() __asm__ __volatile__ ("fence o,w" : : : "memory");
/*
* Unordered I/O memory access primitives. These are even more relaxed than
* the relaxed versions, as they don't even order accesses between successive
@ -165,7 +154,7 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
#define __io_br() do {} while (0)
#define __io_ar(v) __asm__ __volatile__ ("fence i,r" : : : "memory");
#define __io_bw() __asm__ __volatile__ ("fence w,o" : : : "memory");
#define __io_aw() do {} while (0)
#define __io_aw() mmiowb_set_pending()
#define readb(c) ({ u8 __v; __io_br(); __v = readb_cpu(c); __io_ar(__v); __v; })
#define readw(c) ({ u16 __v; __io_br(); __v = readw_cpu(c); __io_ar(__v); __v; })

14
arch/riscv/include/asm/mmiowb.h Normal file
View File

@ -0,0 +1,14 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_RISCV_MMIOWB_H
#define _ASM_RISCV_MMIOWB_H
/*
* "o,w" is sufficient to ensure that all writes to the device have completed
* before the write to the spinlock is allowed to commit.
*/
#define mmiowb() __asm__ __volatile__ ("fence o,w" : : : "memory");
#include <asm-generic/mmiowb.h>
#endif /* ASM_RISCV_MMIOWB_H */

1
arch/s390/include/asm/Kbuild
View File

@ -20,6 +20,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += trace_clock.h
generic-y += unaligned.h
generic-y += word-at-a-time.h

3
arch/sh/include/asm/io.h
View File

@ -229,9 +229,6 @@ __BUILD_IOPORT_STRING(q, u64)
#define IO_SPACE_LIMIT 0xffffffff
/* synco on SH-4A, otherwise a nop */
#define mmiowb() wmb()
/* We really want to try and get these to memcpy etc */
void memcpy_fromio(void *, const volatile void __iomem *, unsigned long);
void memcpy_toio(volatile void __iomem *, const void *, unsigned long);

12
arch/sh/include/asm/mmiowb.h Normal file
View File

@ -0,0 +1,12 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_SH_MMIOWB_H
#define __ASM_SH_MMIOWB_H
#include <asm/barrier.h>
/* synco on SH-4A, otherwise a nop */
#define mmiowb() wmb()
#include <asm-generic/mmiowb.h>
#endif /* __ASM_SH_MMIOWB_H */

2
arch/sh/include/asm/spinlock-llsc.h
View File

@ -47,6 +47,8 @@ static inline void arch_spin_unlock(arch_spinlock_t *lock)
{
unsigned long tmp;
/* This could be optimised with ARCH_HAS_MMIOWB */
mmiowb();
__asm__ __volatile__ (
"mov #1, %0 ! arch_spin_unlock \n\t"
"mov.l %0, @%1 \n\t"

1
arch/sparc/include/asm/Kbuild
View File

@ -15,6 +15,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += module.h
generic-y += msi.h
generic-y += preempt.h

2
arch/sparc/include/asm/io_64.h
View File

@ -396,8 +396,6 @@ static inline void memcpy_toio(volatile void __iomem *dst, const void *src,
}
}
#define mmiowb()
#ifdef __KERNEL__
/* On sparc64 we have the whole physical IO address space accessible

1
arch/um/include/asm/Kbuild
View File

@ -16,6 +16,7 @@ generic-y += irq_work.h
generic-y += kdebug.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += param.h
generic-y += pci.h
generic-y += percpu.h

1
arch/unicore32/include/asm/Kbuild
View File

@ -22,6 +22,7 @@ generic-y += kvm_para.h
generic-y += local.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += module.h
generic-y += parport.h
generic-y += percpu.h

1
arch/x86/include/asm/Kbuild
View File

@ -11,3 +11,4 @@ generic-y += early_ioremap.h
generic-y += export.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h

2
arch/x86/include/asm/io.h
View File

@ -90,8 +90,6 @@ build_mmio_write(__writel, "l", unsigned int, "r", )
#define __raw_writew __writew
#define __raw_writel __writel
#define mmiowb() barrier()
#ifdef CONFIG_X86_64
build_mmio_read(readq, "q", u64, "=r", :"memory")

1
arch/xtensa/include/asm/Kbuild
View File

@ -20,6 +20,7 @@ generic-y += local.h
generic-y += local64.h
generic-y += mcs_spinlock.h
generic-y += mm-arch-hooks.h
generic-y += mmiowb.h
generic-y += param.h
generic-y += percpu.h
generic-y += preempt.h

4
drivers/crypto/cavium/nitrox/nitrox_reqmgr.c
View File

@ -303,8 +303,6 @@ static void post_se_instr(struct nitrox_softreq *sr,
/* Ring doorbell with count 1 */
writeq(1, cmdq->dbell_csr_addr);
/* orders the doorbell rings */
mmiowb();
cmdq->write_idx = incr_index(idx, 1, ndev->qlen);
@ -599,8 +597,6 @@ void pkt_slc_resp_tasklet(unsigned long data)
* MSI-X interrupt generates if Completion count > Threshold
*/
writeq(slc_cnts.value, cmdq->compl_cnt_csr_addr);
/* order the writes */
mmiowb();
if (atomic_read(&cmdq->backlog_count))
schedule_work(&cmdq->backlog_qflush);

3
drivers/dma/txx9dmac.c
View File

@ -327,7 +327,6 @@ static void txx9dmac_reset_chan(struct txx9dmac_chan *dc)
channel_writel(dc, SAIR, 0);
channel_writel(dc, DAIR, 0);
channel_writel(dc, CCR, 0);
mmiowb();
}
/* Called with dc->lock held and bh disabled */
@ -954,7 +953,6 @@ static void txx9dmac_chain_dynamic(struct txx9dmac_chan *dc,
dma_sync_single_for_device(chan2parent(&dc->chan),
prev->txd.phys, ddev->descsize,
DMA_TO_DEVICE);
mmiowb();
if (!(channel_readl(dc, CSR) & TXX9_DMA_CSR_CHNEN) &&
channel_read_CHAR(dc) == prev->txd.phys)
/* Restart chain DMA */
@ -1080,7 +1078,6 @@ static void txx9dmac_free_chan_resources(struct dma_chan *chan)
static void txx9dmac_off(struct txx9dmac_dev *ddev)
{
dma_writel(ddev, MCR, 0);
mmiowb();
}
static int __init txx9dmac_chan_probe(struct platform_device *pdev)

1
drivers/firewire/ohci.c
View File

@ -2939,7 +2939,6 @@ static void set_multichannel_mask(struct fw_ohci *ohci, u64 channels)
reg_write(ohci, OHCI1394_IRMultiChanMaskLoClear, ~lo);
reg_write(ohci, OHCI1394_IRMultiChanMaskHiSet, hi);
reg_write(ohci, OHCI1394_IRMultiChanMaskLoSet, lo);
mmiowb();
ohci->mc_channels = channels;
}

10
drivers/gpu/drm/i915/intel_hdmi.c
View File

@ -182,7 +182,6 @@ static void g4x_write_infoframe(struct intel_encoder *encoder,
I915_WRITE(VIDEO_DIP_CTL, val);
mmiowb();
for (i = 0; i < len; i += 4) {
I915_WRITE(VIDEO_DIP_DATA, *data);
data++;
@ -190,7 +189,6 @@ static void g4x_write_infoframe(struct intel_encoder *encoder,
/* Write every possible data byte to force correct ECC calculation. */
for (; i < VIDEO_DIP_DATA_SIZE; i += 4)
I915_WRITE(VIDEO_DIP_DATA, 0);
mmiowb();
val |= g4x_infoframe_enable(type);
val &= ~VIDEO_DIP_FREQ_MASK;
@ -237,7 +235,6 @@ static void ibx_write_infoframe(struct intel_encoder *encoder,
I915_WRITE(reg, val);
mmiowb();
for (i = 0; i < len; i += 4) {
I915_WRITE(TVIDEO_DIP_DATA(intel_crtc->pipe), *data);
data++;
@ -245,7 +242,6 @@ static void ibx_write_infoframe(struct intel_encoder *encoder,
/* Write every possible data byte to force correct ECC calculation. */
for (; i < VIDEO_DIP_DATA_SIZE; i += 4)
I915_WRITE(TVIDEO_DIP_DATA(intel_crtc->pipe), 0);
mmiowb();
val |= g4x_infoframe_enable(type);
val &= ~VIDEO_DIP_FREQ_MASK;
@ -298,7 +294,6 @@ static void cpt_write_infoframe(struct intel_encoder *encoder,
I915_WRITE(reg, val);
mmiowb();
for (i = 0; i < len; i += 4) {
I915_WRITE(TVIDEO_DIP_DATA(intel_crtc->pipe), *data);
data++;
@ -306,7 +301,6 @@ static void cpt_write_infoframe(struct intel_encoder *encoder,
/* Write every possible data byte to force correct ECC calculation. */
for (; i < VIDEO_DIP_DATA_SIZE; i += 4)
I915_WRITE(TVIDEO_DIP_DATA(intel_crtc->pipe), 0);
mmiowb();
val |= g4x_infoframe_enable(type);
val &= ~VIDEO_DIP_FREQ_MASK;
@ -352,7 +346,6 @@ static void vlv_write_infoframe(struct intel_encoder *encoder,
I915_WRITE(reg, val);
mmiowb();
for (i = 0; i < len; i += 4) {
I915_WRITE(VLV_TVIDEO_DIP_DATA(intel_crtc->pipe), *data);
data++;
@ -360,7 +353,6 @@ static void vlv_write_infoframe(struct intel_encoder *encoder,
/* Write every possible data byte to force correct ECC calculation. */
for (; i < VIDEO_DIP_DATA_SIZE; i += 4)
I915_WRITE(VLV_TVIDEO_DIP_DATA(intel_crtc->pipe), 0);
mmiowb();
val |= g4x_infoframe_enable(type);
val &= ~VIDEO_DIP_FREQ_MASK;
@ -406,7 +398,6 @@ static void hsw_write_infoframe(struct intel_encoder *encoder,
val &= ~hsw_infoframe_enable(type);
I915_WRITE(ctl_reg, val);
mmiowb();
for (i = 0; i < len; i += 4) {
I915_WRITE(hsw_dip_data_reg(dev_priv, cpu_transcoder,
type, i >> 2), *data);
@ -416,7 +407,6 @@ static void hsw_write_infoframe(struct intel_encoder *encoder,
for (; i < data_size; i += 4)
I915_WRITE(hsw_dip_data_reg(dev_priv, cpu_transcoder,
type, i >> 2), 0);
mmiowb();
val |= hsw_infoframe_enable(type);
I915_WRITE(ctl_reg, val);

2
drivers/ide/tx4939ide.c
View File

@ -156,7 +156,6 @@ static u16 tx4939ide_check_error_ints(ide_hwif_t *hwif)
u16 sysctl = tx4939ide_readw(base, TX4939IDE_Sys_Ctl);
tx4939ide_writew(sysctl | 0x4000, base, TX4939IDE_Sys_Ctl);
mmiowb();
/* wait 12GBUSCLK (typ. 60ns @ GBUS200MHz, max 270ns) */
ndelay(270);
tx4939ide_writew(sysctl, base, TX4939IDE_Sys_Ctl);
@ -396,7 +395,6 @@ static void tx4939ide_init_hwif(ide_hwif_t *hwif)
/* Soft Reset */
tx4939ide_writew(0x8000, base, TX4939IDE_Sys_Ctl);
mmiowb();
/* at least 20 GBUSCLK (typ. 100ns @ GBUS200MHz, max 450ns) */
ndelay(450);
tx4939ide_writew(0x0000, base, TX4939IDE_Sys_Ctl);

3
drivers/infiniband/hw/hfi1/chip.c
View File

@ -8365,7 +8365,6 @@ static inline void clear_recv_intr(struct hfi1_ctxtdata *rcd)
struct hfi1_devdata *dd = rcd->dd;
u32 addr = CCE_INT_CLEAR + (8 * rcd->ireg);
mmiowb(); /* make sure everything before is written */
write_csr(dd, addr, rcd->imask);
/* force the above write on the chip and get a value back */
(void)read_csr(dd, addr);
@ -11803,12 +11802,10 @@ void update_usrhead(struct hfi1_ctxtdata *rcd, u32 hd, u32 updegr, u32 egrhd,
<< RCV_EGR_INDEX_HEAD_HEAD_SHIFT;
write_uctxt_csr(dd, ctxt, RCV_EGR_INDEX_HEAD, reg);
}
mmiowb();
reg = ((u64)rcv_intr_count << RCV_HDR_HEAD_COUNTER_SHIFT) |
(((u64)hd & RCV_HDR_HEAD_HEAD_MASK)
<< RCV_HDR_HEAD_HEAD_SHIFT);
write_uctxt_csr(dd, ctxt, RCV_HDR_HEAD, reg);
mmiowb();
}
u32 hdrqempty(struct hfi1_ctxtdata *rcd)

1
drivers/infiniband/hw/hfi1/pio.c
View File

@ -1578,7 +1578,6 @@ void hfi1_sc_wantpiobuf_intr(struct send_context *sc, u32 needint)
sc_del_credit_return_intr(sc);
trace_hfi1_wantpiointr(sc, needint, sc->credit_ctrl);
if (needint) {
mmiowb();
sc_return_credits(sc);
}
}

2
drivers/infiniband/hw/hns/hns_roce_hw_v1.c
View File

@ -1750,8 +1750,6 @@ static int hns_roce_v1_post_mbox(struct hns_roce_dev *hr_dev, u64 in_param,
writel(val, hcr + 5);
mmiowb();
return 0;
}

2
drivers/infiniband/hw/i40iw/i40iw_osdep.h
View File

@ -211,7 +211,7 @@ enum i40iw_status_code i40iw_hw_manage_vf_pble_bp(struct i40iw_device *iwdev,
struct i40iw_sc_vsi;
void i40iw_hw_stats_start_timer(struct i40iw_sc_vsi *vsi);
void i40iw_hw_stats_stop_timer(struct i40iw_sc_vsi *vsi);
#define i40iw_mmiowb() mmiowb()
#define i40iw_mmiowb() do { } while (0)
void i40iw_wr32(struct i40iw_hw *hw, u32 reg, u32 value);
u32 i40iw_rd32(struct i40iw_hw *hw, u32 reg);
#endif /* _I40IW_OSDEP_H_ */

6
drivers/infiniband/hw/mlx4/qp.c
View File

@ -3744,12 +3744,6 @@ static int _mlx4_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
writel_relaxed(qp->doorbell_qpn,
to_mdev(ibqp->device)->uar_map + MLX4_SEND_DOORBELL);
/*
* Make sure doorbells don't leak out of SQ spinlock
* and reach the HCA out of order.
*/
mmiowb();
stamp_send_wqe(qp, ind + qp->sq_spare_wqes - 1);
qp->sq_next_wqe = ind;

1
drivers/infiniband/hw/mlx5/qp.c
View File

@ -5126,7 +5126,6 @@ static int _mlx5_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
/* Make sure doorbells don't leak out of SQ spinlock
* and reach the HCA out of order.
*/
mmiowb();
bf->offset ^= bf->buf_size;
}

6
drivers/infiniband/hw/mthca/mthca_cmd.c
View File

@ -292,12 +292,6 @@ static int mthca_cmd_post(struct mthca_dev *dev,
err = mthca_cmd_post_hcr(dev, in_param, out_param, in_modifier,
op_modifier, op, token, event);
/*
* Make sure that our HCR writes don't get mixed in with
* writes from another CPU starting a FW command.
*/
mmiowb();
mutex_unlock(&dev->cmd.hcr_mutex);
return err;
}

5
drivers/infiniband/hw/mthca/mthca_cq.c
View File

@ -211,11 +211,6 @@ static inline void update_cons_index(struct mthca_dev *dev, struct mthca_cq *cq,
mthca_write64(MTHCA_TAVOR_CQ_DB_INC_CI | cq->cqn, incr - 1,
dev->kar + MTHCA_CQ_DOORBELL,
MTHCA_GET_DOORBELL_LOCK(&dev->doorbell_lock));
/*
* Make sure doorbells don't leak out of CQ spinlock
* and reach the HCA out of order:
*/
mmiowb();
}
}

17
drivers/infiniband/hw/mthca/mthca_qp.c
View File

@ -1809,11 +1809,6 @@ int mthca_tavor_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
(qp->qpn << 8) | size0,
dev->kar + MTHCA_SEND_DOORBELL,
MTHCA_GET_DOORBELL_LOCK(&dev->doorbell_lock));
/*
* Make sure doorbells don't leak out of SQ spinlock
* and reach the HCA out of order:
*/
mmiowb();
}
qp->sq.next_ind = ind;
@ -1924,12 +1919,6 @@ int mthca_tavor_post_receive(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
qp->rq.next_ind = ind;
qp->rq.head += nreq;
/*
* Make sure doorbells don't leak out of RQ spinlock and reach
* the HCA out of order:
*/
mmiowb();
spin_unlock_irqrestore(&qp->rq.lock, flags);
return err;
}
@ -2164,12 +2153,6 @@ int mthca_arbel_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
MTHCA_GET_DOORBELL_LOCK(&dev->doorbell_lock));
}
/*
* Make sure doorbells don't leak out of SQ spinlock and reach
* the HCA out of order:
*/
mmiowb();
spin_unlock_irqrestore(&qp->sq.lock, flags);
return err;
}

6
drivers/infiniband/hw/mthca/mthca_srq.c
View File

@ -570,12 +570,6 @@ int mthca_tavor_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
MTHCA_GET_DOORBELL_LOCK(&dev->doorbell_lock));
}
/*
* Make sure doorbells don't leak out of SRQ spinlock and
* reach the HCA out of order:
*/
mmiowb();
spin_unlock_irqrestore(&srq->lock, flags);
return err;
}

12
drivers/infiniband/hw/qedr/verbs.c
View File

@ -773,9 +773,6 @@ static void doorbell_cq(struct qedr_cq *cq, u32 cons, u8 flags)
cq->db.data.agg_flags = flags;
cq->db.data.value = cpu_to_le32(cons);
writeq(cq->db.raw, cq->db_addr);
/* Make sure write would stick */
mmiowb();
}
int qedr_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
@ -2084,8 +2081,6 @@ static int qedr_update_qp_state(struct qedr_dev *dev,
if (rdma_protocol_roce(&dev->ibdev, 1)) {
writel(qp->rq.db_data.raw, qp->rq.db);
/* Make sure write takes effect */
mmiowb();
}
break;
case QED_ROCE_QP_STATE_ERR:
@ -3502,9 +3497,6 @@ int qedr_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
smp_wmb();
writel(qp->sq.db_data.raw, qp->sq.db);
/* Make sure write sticks */
mmiowb();
spin_unlock_irqrestore(&qp->q_lock, flags);
return rc;
@ -3695,12 +3687,8 @@ int qedr_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
writel(qp->rq.db_data.raw, qp->rq.db);
/* Make sure write sticks */
mmiowb();
if (rdma_protocol_iwarp(&dev->ibdev, 1)) {
writel(qp->rq.iwarp_db2_data.raw, qp->rq.iwarp_db2);
mmiowb(); /* for second doorbell */
}
wr = wr->next;

4
drivers/infiniband/hw/qib/qib_iba6120.c
View File

@ -1884,7 +1884,6 @@ static void qib_6120_put_tid(struct qib_devdata *dd, u64 __iomem *tidptr,
qib_write_kreg(dd, kr_scratch, 0xfeeddeaf);
writel(pa, tidp32);
qib_write_kreg(dd, kr_scratch, 0xdeadbeef);
mmiowb();
spin_unlock_irqrestore(tidlockp, flags);
}
@ -1928,7 +1927,6 @@ static void qib_6120_put_tid_2(struct qib_devdata *dd, u64 __iomem *tidptr,
pa |= 2 << 29;
}
writel(pa, tidp32);
mmiowb();
}
@ -2053,9 +2051,7 @@ static void qib_update_6120_usrhead(struct qib_ctxtdata *rcd, u64 hd,
{
if (updegr)
qib_write_ureg(rcd->dd, ur_rcvegrindexhead, egrhd, rcd->ctxt);
mmiowb();
qib_write_ureg(rcd->dd, ur_rcvhdrhead, hd, rcd->ctxt);
mmiowb();
}
static u32 qib_6120_hdrqempty(struct qib_ctxtdata *rcd)

3
drivers/infiniband/hw/qib/qib_iba7220.c
View File

@ -2175,7 +2175,6 @@ static void qib_7220_put_tid(struct qib_devdata *dd, u64 __iomem *tidptr,
pa = chippa;
}
writeq(pa, tidptr);
mmiowb();
}
/**
@ -2704,9 +2703,7 @@ static void qib_update_7220_usrhead(struct qib_ctxtdata *rcd, u64 hd,
{
if (updegr)
qib_write_ureg(rcd->dd, ur_rcvegrindexhead, egrhd, rcd->ctxt);
mmiowb();
qib_write_ureg(rcd->dd, ur_rcvhdrhead, hd, rcd->ctxt);
mmiowb();
}
static u32 qib_7220_hdrqempty(struct qib_ctxtdata *rcd)

3
drivers/infiniband/hw/qib/qib_iba7322.c
View File

@ -3793,7 +3793,6 @@ static void qib_7322_put_tid(struct qib_devdata *dd, u64 __iomem *tidptr,
pa = chippa;
}
writeq(pa, tidptr);
mmiowb();
}
/**
@ -4440,10 +4439,8 @@ static void qib_update_7322_usrhead(struct qib_ctxtdata *rcd, u64 hd,
adjust_rcv_timeout(rcd, npkts);
if (updegr)
qib_write_ureg(rcd->dd, ur_rcvegrindexhead, egrhd, rcd->ctxt);
mmiowb();
qib_write_ureg(rcd->dd, ur_rcvhdrhead, hd, rcd->ctxt);
qib_write_ureg(rcd->dd, ur_rcvhdrhead, hd, rcd->ctxt);
mmiowb();
}
static u32 qib_7322_hdrqempty(struct qib_ctxtdata *rcd)

4
drivers/infiniband/hw/qib/qib_sd7220.c
View File

@ -1068,7 +1068,6 @@ static int qib_sd_setvals(struct qib_devdata *dd)
for (idx = 0; idx < NUM_DDS_REGS; ++idx) {
data = ((dds_reg_map & 0xF) << 4) | TX_FAST_ELT;
writeq(data, iaddr + idx);
mmiowb();
qib_read_kreg32(dd, kr_scratch);
dds_reg_map >>= 4;
for (midx = 0; midx < DDS_ROWS; ++midx) {
@ -1076,7 +1075,6 @@ static int qib_sd_setvals(struct qib_devdata *dd)
data = dds_init_vals[midx].reg_vals[idx];
writeq(data, daddr);
mmiowb();
qib_read_kreg32(dd, kr_scratch);
} /* End inner for (vals for this reg, each row) */
} /* end outer for (regs to be stored) */
@ -1098,13 +1096,11 @@ static int qib_sd_setvals(struct qib_devdata *dd)
didx = idx + min_idx;
/* Store the next RXEQ register address */
writeq(rxeq_init_vals[idx].rdesc, iaddr + didx);
mmiowb();
qib_read_kreg32(dd, kr_scratch);
/* Iterate through RXEQ values */
for (vidx = 0; vidx < 4; vidx++) {
data = rxeq_init_vals[idx].rdata[vidx];
writeq(data, taddr + (vidx << 6) + idx);
mmiowb();
qib_read_kreg32(dd, kr_scratch);
}
} /* end outer for (Reg-writes for RXEQ) */

8
drivers/media/pci/dt3155/dt3155.c
View File

@ -46,7 +46,6 @@ static int read_i2c_reg(void __iomem *addr, u8 index, u8 *data)
u32 tmp = index;
iowrite32((tmp << 17) | IIC_READ, addr + IIC_CSR2);
mmiowb();
udelay(45); /* wait at least 43 usec for NEW_CYCLE to clear */
if (ioread32(addr + IIC_CSR2) & NEW_CYCLE)
return -EIO; /* error: NEW_CYCLE not cleared */
@ -77,7 +76,6 @@ static int write_i2c_reg(void __iomem *addr, u8 index, u8 data)
u32 tmp = index;
iowrite32((tmp << 17) | IIC_WRITE | data, addr + IIC_CSR2);
mmiowb();
udelay(65); /* wait at least 63 usec for NEW_CYCLE to clear */
if (ioread32(addr + IIC_CSR2) & NEW_CYCLE)
return -EIO; /* error: NEW_CYCLE not cleared */
@ -104,7 +102,6 @@ static void write_i2c_reg_nowait(void __iomem *addr, u8 index, u8 data)
u32 tmp = index;
iowrite32((tmp << 17) | IIC_WRITE | data, addr + IIC_CSR2);
mmiowb();
}
/**
@ -264,7 +261,6 @@ static irqreturn_t dt3155_irq_handler_even(int irq, void *dev_id)
FLD_DN_ODD | FLD_DN_EVEN |
CAP_CONT_EVEN | CAP_CONT_ODD,
ipd->regs + CSR1);
mmiowb();
}
spin_lock(&ipd->lock);
@ -282,7 +278,6 @@ static irqreturn_t dt3155_irq_handler_even(int irq, void *dev_id)
iowrite32(dma_addr + ipd->width, ipd->regs + ODD_DMA_START);
iowrite32(ipd->width, ipd->regs + EVEN_DMA_STRIDE);
iowrite32(ipd->width, ipd->regs + ODD_DMA_STRIDE);
mmiowb();
}
/* enable interrupts, clear all irq flags */
@ -437,12 +432,10 @@ static int dt3155_init_board(struct dt3155_priv *pd)
/* resetting the adapter */
iowrite32(ADDR_ERR_ODD | ADDR_ERR_EVEN | FLD_CRPT_ODD | FLD_CRPT_EVEN |
FLD_DN_ODD | FLD_DN_EVEN, pd->regs + CSR1);
mmiowb();
msleep(20);
/* initializing adapter registers */
iowrite32(FIFO_EN | SRST, pd->regs + CSR1);
mmiowb();
iowrite32(0xEEEEEE01, pd->regs + EVEN_PIXEL_FMT);
iowrite32(0xEEEEEE01, pd->regs + ODD_PIXEL_FMT);
iowrite32(0x00000020, pd->regs + FIFO_TRIGER);
@ -454,7 +447,6 @@ static int dt3155_init_board(struct dt3155_priv *pd)
iowrite32(0, pd->regs + MASK_LENGTH);
iowrite32(0x0005007C, pd->regs + FIFO_FLAG_CNT);
iowrite32(0x01010101, pd->regs + IIC_CLK_DUR);
mmiowb();
/* verifying that we have a DT3155 board (not just a SAA7116 chip) */
read_i2c_reg(pd->regs, DT_ID, &tmp);

4
drivers/memstick/host/jmb38x_ms.c
View File

@ -644,7 +644,6 @@ static int jmb38x_ms_reset(struct jmb38x_ms_host *host)
writel(HOST_CONTROL_RESET_REQ | HOST_CONTROL_CLOCK_EN
| readl(host->addr + HOST_CONTROL),
host->addr + HOST_CONTROL);
mmiowb();
for (cnt = 0; cnt < 20; ++cnt) {
if (!(HOST_CONTROL_RESET_REQ
@ -659,7 +658,6 @@ static int jmb38x_ms_reset(struct jmb38x_ms_host *host)
writel(HOST_CONTROL_RESET | HOST_CONTROL_CLOCK_EN
| readl(host->addr + HOST_CONTROL),
host->addr + HOST_CONTROL);
mmiowb();
for (cnt = 0; cnt < 20; ++cnt) {
if (!(HOST_CONTROL_RESET
@ -672,7 +670,6 @@ static int jmb38x_ms_reset(struct jmb38x_ms_host *host)
return -EIO;
reset_ok:
mmiowb();
writel(INT_STATUS_ALL, host->addr + INT_SIGNAL_ENABLE);
writel(INT_STATUS_ALL, host->addr + INT_STATUS_ENABLE);
return 0;
@ -1009,7 +1006,6 @@ static void jmb38x_ms_remove(struct pci_dev *dev)
tasklet_kill(&host->notify);
writel(0, host->addr + INT_SIGNAL_ENABLE);
writel(0, host->addr + INT_STATUS_ENABLE);
mmiowb();
dev_dbg(&jm->pdev->dev, "interrupts off\n");
spin_lock_irqsave(&host->lock, flags);
if (host->req) {

2
drivers/misc/ioc4.c
View File

@ -156,7 +156,6 @@ ioc4_clock_calibrate(struct ioc4_driver_data *idd)
/* Reset to power-on state */
writel(0, &idd->idd_misc_regs->int_out.raw);
mmiowb();
/* Set up square wave */
int_out.raw = 0;
@ -164,7 +163,6 @@ ioc4_clock_calibrate(struct ioc4_driver_data *idd)
int_out.fields.mode = IOC4_INT_OUT_MODE_TOGGLE;
int_out.fields.diag = 0;
writel(int_out.raw, &idd->idd_misc_regs->int_out.raw);
mmiowb();
/* Check square wave period averaged over some number of cycles */
start = ktime_get_ns();

3
drivers/misc/mei/hw-me.c
View File

@ -350,9 +350,6 @@ static void mei_me_hw_reset_release(struct mei_device *dev)
hcsr |= H_IG;
hcsr &= ~H_RST;
mei_hcsr_set(dev, hcsr);
/* complete this write before we set host ready on another CPU */
mmiowb();
}
/**

1
drivers/misc/tifm_7xx1.c
View File

@ -403,7 +403,6 @@ static void tifm_7xx1_remove(struct pci_dev *dev)
fm->eject = tifm_7xx1_dummy_eject;
fm->has_ms_pif = tifm_7xx1_dummy_has_ms_pif;
writel(TIFM_IRQ_SETALL, fm->addr + FM_CLEAR_INTERRUPT_ENABLE);
mmiowb();
free_irq(dev->irq, fm);
tifm_remove_adapter(fm);

1
drivers/mmc/host/alcor.c
View File

@ -959,7 +959,6 @@ static void alcor_timeout_timer(struct work_struct *work)
alcor_request_complete(host, 0);
}
mmiowb();
mutex_unlock(&host->cmd_mutex);
}

13
drivers/mmc/host/sdhci.c
View File

@ -1807,7 +1807,6 @@ void sdhci_request(struct mmc_host *mmc, struct mmc_request *mrq)
sdhci_send_command(host, mrq->cmd);
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
EXPORT_SYMBOL_GPL(sdhci_request);
@ -2010,8 +2009,6 @@ void sdhci_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
*/
if (host->quirks & SDHCI_QUIRK_RESET_CMD_DATA_ON_IOS)
sdhci_do_reset(host, SDHCI_RESET_CMD | SDHCI_RESET_DATA);
mmiowb();
}
EXPORT_SYMBOL_GPL(sdhci_set_ios);
@ -2105,7 +2102,6 @@ static void sdhci_enable_sdio_irq_nolock(struct sdhci_host *host, int enable)
sdhci_writel(host, host->ier, SDHCI_INT_ENABLE);
sdhci_writel(host, host->ier, SDHCI_SIGNAL_ENABLE);
mmiowb();
}
}
@ -2353,7 +2349,6 @@ void sdhci_send_tuning(struct sdhci_host *host, u32 opcode)
host->tuning_done = 0;
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
/* Wait for Buffer Read Ready interrupt */
@ -2705,7 +2700,6 @@ static bool sdhci_request_done(struct sdhci_host *host)
host->mrqs_done[i] = NULL;
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
mmc_request_done(host->mmc, mrq);
@ -2739,7 +2733,6 @@ static void sdhci_timeout_timer(struct timer_list *t)
sdhci_finish_mrq(host, host->cmd->mrq);
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
@ -2770,7 +2763,6 @@ static void sdhci_timeout_data_timer(struct timer_list *t)
}
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
@ -3251,7 +3243,6 @@ int sdhci_resume_host(struct sdhci_host *host)
mmc->ops->set_ios(mmc, &mmc->ios);
} else {
sdhci_init(host, (host->mmc->pm_flags & MMC_PM_KEEP_POWER));
mmiowb();
}
if (host->irq_wake_enabled) {
@ -3391,7 +3382,6 @@ void sdhci_cqe_enable(struct mmc_host *mmc)
mmc_hostname(mmc), host->ier,
sdhci_readl(host, SDHCI_INT_STATUS));
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
EXPORT_SYMBOL_GPL(sdhci_cqe_enable);
@ -3416,7 +3406,6 @@ void sdhci_cqe_disable(struct mmc_host *mmc, bool recovery)
mmc_hostname(mmc), host->ier,
sdhci_readl(host, SDHCI_INT_STATUS));
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
EXPORT_SYMBOL_GPL(sdhci_cqe_disable);
@ -4255,8 +4244,6 @@ int __sdhci_add_host(struct sdhci_host *host)
goto unirq;
}
mmiowb();
ret = mmc_add_host(mmc);
if (ret)
goto unled;

3
drivers/mmc/host/tifm_sd.c
View File

@ -889,7 +889,6 @@ static int tifm_sd_initialize_host(struct tifm_sd *host)
struct tifm_dev *sock = host->dev;
writel(0, sock->addr + SOCK_MMCSD_INT_ENABLE);
mmiowb();
host->clk_div = 61;
host->clk_freq = 20000000;
writel(TIFM_MMCSD_RESET, sock->addr + SOCK_MMCSD_SYSTEM_CONTROL);
@ -940,7 +939,6 @@ static int tifm_sd_initialize_host(struct tifm_sd *host)
writel(TIFM_MMCSD_CERR | TIFM_MMCSD_BRS | TIFM_MMCSD_EOC
| TIFM_MMCSD_ERRMASK,
sock->addr + SOCK_MMCSD_INT_ENABLE);
mmiowb();
return 0;
}
@ -1005,7 +1003,6 @@ static void tifm_sd_remove(struct tifm_dev *sock)
spin_lock_irqsave(&sock->lock, flags);
host->eject = 1;
writel(0, sock->addr + SOCK_MMCSD_INT_ENABLE);
mmiowb();
spin_unlock_irqrestore(&sock->lock, flags);
tasklet_kill(&host->finish_tasklet);

10
drivers/mmc/host/via-sdmmc.c
View File

@ -686,7 +686,6 @@ static void via_sdc_request(struct mmc_host *mmc, struct mmc_request *mrq)
via_sdc_send_command(host, mrq->cmd);
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
@ -711,7 +710,6 @@ static void via_sdc_set_power(struct via_crdr_mmc_host *host,
gatt &= ~VIA_CRDR_PCICLKGATT_PAD_PWRON;
writeb(gatt, host->pcictrl_mmiobase + VIA_CRDR_PCICLKGATT);
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
via_pwron_sleep(host);
@ -770,7 +768,6 @@ static void via_sdc_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
if (readb(addrbase + VIA_CRDR_PCISDCCLK) != clock)
writeb(clock, addrbase + VIA_CRDR_PCISDCCLK);
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
if (ios->power_mode != MMC_POWER_OFF)
@ -830,7 +827,6 @@ static void via_reset_pcictrl(struct via_crdr_mmc_host *host)
via_restore_pcictrlreg(host);
via_restore_sdcreg(host);
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
}
@ -925,7 +921,6 @@ static irqreturn_t via_sdc_isr(int irq, void *dev_id)
result = IRQ_HANDLED;
mmiowb();
out:
spin_unlock(&sdhost->lock);
@ -960,7 +955,6 @@ static void via_sdc_timeout(struct timer_list *t)
}
}
mmiowb();
spin_unlock_irqrestore(&sdhost->lock, flags);
}
@ -1012,7 +1006,6 @@ static void via_sdc_card_detect(struct work_struct *work)
tasklet_schedule(&host->finish_tasklet);
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
via_reset_pcictrl(host);
@ -1020,7 +1013,6 @@ static void via_sdc_card_detect(struct work_struct *work)
spin_lock_irqsave(&host->lock, flags);
}
mmiowb();
spin_unlock_irqrestore(&host->lock, flags);
via_print_pcictrl(host);
@ -1188,7 +1180,6 @@ static void via_sd_remove(struct pci_dev *pcidev)
/* Disable generating further interrupts */
writeb(0x0, sdhost->pcictrl_mmiobase + VIA_CRDR_PCIINTCTRL);
mmiowb();
if (sdhost->mrq) {
pr_err("%s: Controller removed during "
@ -1197,7 +1188,6 @@ static void via_sd_remove(struct pci_dev *pcidev)
/* make sure all DMA is stopped */
writel(VIA_CRDR_DMACTRL_SFTRST,
sdhost->ddma_mmiobase + VIA_CRDR_DMACTRL);
mmiowb();
sdhost->mrq->cmd->error = -ENOMEDIUM;
if (sdhost->mrq->stop)
sdhost->mrq->stop->error = -ENOMEDIUM;

2
drivers/mtd/nand/raw/r852.c
View File

@ -45,7 +45,6 @@ static inline void r852_write_reg(struct r852_device *dev,
int address, uint8_t value)
{
writeb(value, dev->mmio + address);
mmiowb();
}
@ -61,7 +60,6 @@ static inline void r852_write_reg_dword(struct r852_device *dev,
int address, uint32_t value)
{
writel(cpu_to_le32(value), dev->mmio + address);
mmiowb();
}
/* returns pointer to our private structure */

1
drivers/mtd/nand/raw/txx9ndfmc.c
View File

@ -159,7 +159,6 @@ static void txx9ndfmc_cmd_ctrl(struct nand_chip *chip, int cmd,
if ((ctrl & NAND_CTRL_CHANGE) && cmd == NAND_CMD_NONE)
txx9ndfmc_write(dev, 0, TXX9_NDFDTR);
}
mmiowb();
}
static int txx9ndfmc_dev_ready(struct nand_chip *chip)

1
drivers/net/ethernet/aeroflex/greth.c
View File

@ -613,7 +613,6 @@ static irqreturn_t greth_interrupt(int irq, void *dev_id)
napi_schedule(&greth->napi);
}
mmiowb();
spin_unlock(&greth->devlock);
return retval;

4
drivers/net/ethernet/alacritech/slicoss.c
View File

@ -345,8 +345,6 @@ static void slic_set_rx_mode(struct net_device *dev)
if (sdev->promisc != set_promisc) {
sdev->promisc = set_promisc;
slic_configure_rcv(sdev);
/* make sure writes to receiver cant leak out of the lock */
mmiowb();
}
spin_unlock_bh(&sdev->link_lock);
}
@ -1461,8 +1459,6 @@ static netdev_tx_t slic_xmit(struct sk_buff *skb, struct net_device *dev)
if (slic_get_free_tx_descs(txq) < SLIC_MAX_REQ_TX_DESCS)
netif_stop_queue(dev);
/* make sure writes to io-memory cant leak out of tx queue lock */
mmiowb();
return NETDEV_TX_OK;
drop_skb:

1
drivers/net/ethernet/amazon/ena/ena_com.c
View File

@ -2016,7 +2016,6 @@ void ena_com_aenq_intr_handler(struct ena_com_dev *dev, void *data)
mb();
writel_relaxed((u32)aenq->head,
dev->reg_bar + ENA_REGS_AENQ_HEAD_DB_OFF);
mmiowb();
}
int ena_com_dev_reset(struct ena_com_dev *ena_dev,

1
drivers/net/ethernet/atheros/atlx/atl1.c
View File

@ -2439,7 +2439,6 @@ static netdev_tx_t atl1_xmit_frame(struct sk_buff *skb,
atl1_tx_map(adapter, skb, ptpd);
atl1_tx_queue(adapter, count, ptpd);
atl1_update_mailbox(adapter);
mmiowb();
return NETDEV_TX_OK;
}

1
drivers/net/ethernet/atheros/atlx/atl2.c
View File

@ -908,7 +908,6 @@ static netdev_tx_t atl2_xmit_frame(struct sk_buff *skb,
ATL2_WRITE_REGW(&adapter->hw, REG_MB_TXD_WR_IDX,
(adapter->txd_write_ptr >> 2));
mmiowb();
dev_consume_skb_any(skb);
return NETDEV_TX_OK;
}

4
drivers/net/ethernet/broadcom/bnx2.c
View File

@ -3305,8 +3305,6 @@ bnx2_rx_int(struct bnx2 *bp, struct bnx2_napi *bnapi, int budget)
BNX2_WR(bp, rxr->rx_bseq_addr, rxr->rx_prod_bseq);
mmiowb();
return rx_pkt;
}
@ -6723,8 +6721,6 @@ bnx2_start_xmit(struct sk_buff *skb, struct net_device *dev)
BNX2_WR16(bp, txr->tx_bidx_addr, prod);
BNX2_WR(bp, txr->tx_bseq_addr, txr->tx_prod_bseq);
mmiowb();
txr->tx_prod = prod;
if (unlikely(bnx2_tx_avail(bp, txr) <= MAX_SKB_FRAGS)) {

2
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
View File

@ -4166,8 +4166,6 @@ netdev_tx_t bnx2x_start_xmit(struct sk_buff *skb, struct net_device *dev)
DOORBELL_RELAXED(bp, txdata->cid, txdata->tx_db.raw);
mmiowb();
txdata->tx_bd_prod += nbd;
if (unlikely(bnx2x_tx_avail(bp, txdata) < MAX_DESC_PER_TX_PKT)) {

4
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
View File

@ -527,8 +527,6 @@ static inline void bnx2x_update_rx_prod(struct bnx2x *bp,
REG_WR_RELAXED(bp, fp->ustorm_rx_prods_offset + i * 4,
((u32 *)&rx_prods)[i]);
mmiowb(); /* keep prod updates ordered */
DP(NETIF_MSG_RX_STATUS,
"queue[%d]: wrote bd_prod %u cqe_prod %u sge_prod %u\n",
fp->index, bd_prod, rx_comp_prod, rx_sge_prod);
@ -653,7 +651,6 @@ static inline void bnx2x_igu_ack_sb_gen(struct bnx2x *bp, u8 igu_sb_id,
REG_WR(bp, igu_addr, cmd_data.sb_id_and_flags);
/* Make sure that ACK is written */
mmiowb();
barrier();
}
@ -674,7 +671,6 @@ static inline void bnx2x_hc_ack_sb(struct bnx2x *bp, u8 sb_id,
REG_WR(bp, hc_addr, (*(u32 *)&igu_ack));
/* Make sure that ACK is written */
mmiowb();
barrier();
}

1
drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
View File

@ -2623,7 +2623,6 @@ static int bnx2x_run_loopback(struct bnx2x *bp, int loopback_mode)
wmb();
DOORBELL_RELAXED(bp, txdata->cid, txdata->tx_db.raw);
mmiowb();
barrier();
num_pkts++;

29
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
View File

@ -869,9 +869,6 @@ static void bnx2x_hc_int_disable(struct bnx2x *bp)
"write %x to HC %d (addr 0x%x)\n",
val, port, addr);
/* flush all outstanding writes */
mmiowb();
REG_WR(bp, addr, val);
if (REG_RD(bp, addr) != val)
BNX2X_ERR("BUG! Proper val not read from IGU!\n");
@ -887,9 +884,6 @@ static void bnx2x_igu_int_disable(struct bnx2x *bp)
DP(NETIF_MSG_IFDOWN, "write %x to IGU\n", val);
/* flush all outstanding writes */
mmiowb();
REG_WR(bp, IGU_REG_PF_CONFIGURATION, val);
if (REG_RD(bp, IGU_REG_PF_CONFIGURATION) != val)
BNX2X_ERR("BUG! Proper val not read from IGU!\n");
@ -1595,7 +1589,6 @@ static void bnx2x_hc_int_enable(struct bnx2x *bp)
/*
* Ensure that HC_CONFIG is written before leading/trailing edge config
*/
mmiowb();
barrier();
if (!CHIP_IS_E1(bp)) {
@ -1611,9 +1604,6 @@ static void bnx2x_hc_int_enable(struct bnx2x *bp)
REG_WR(bp, HC_REG_TRAILING_EDGE_0 + port*8, val);
REG_WR(bp, HC_REG_LEADING_EDGE_0 + port*8, val);
}
/* Make sure that interrupts are indeed enabled from here on */
mmiowb();
}
static void bnx2x_igu_int_enable(struct bnx2x *bp)
@ -1674,9 +1664,6 @@ static void bnx2x_igu_int_enable(struct bnx2x *bp)
REG_WR(bp, IGU_REG_TRAILING_EDGE_LATCH, val);
REG_WR(bp, IGU_REG_LEADING_EDGE_LATCH, val);
/* Make sure that interrupts are indeed enabled from here on */
mmiowb();
}
void bnx2x_int_enable(struct bnx2x *bp)
@ -3833,7 +3820,6 @@ static void bnx2x_sp_prod_update(struct bnx2x *bp)
REG_WR16_RELAXED(bp, BAR_XSTRORM_INTMEM + XSTORM_SPQ_PROD_OFFSET(func),
bp->spq_prod_idx);
mmiowb();
}
/**
@ -5244,7 +5230,6 @@ static void bnx2x_update_eq_prod(struct bnx2x *bp, u16 prod)
{
/* No memory barriers */
storm_memset_eq_prod(bp, prod, BP_FUNC(bp));
mmiowb(); /* keep prod updates ordered */
}
static int bnx2x_cnic_handle_cfc_del(struct bnx2x *bp, u32 cid,
@ -6513,7 +6498,6 @@ void bnx2x_nic_init_cnic(struct bnx2x *bp)
/* flush all */
mb();
mmiowb();
}
void bnx2x_pre_irq_nic_init(struct bnx2x *bp)
@ -6553,7 +6537,6 @@ void bnx2x_post_irq_nic_init(struct bnx2x *bp, u32 load_code)
/* flush all before enabling interrupts */
mb();
mmiowb();
bnx2x_int_enable(bp);
@ -7775,12 +7758,10 @@ void bnx2x_igu_clear_sb_gen(struct bnx2x *bp, u8 func, u8 idu_sb_id, bool is_pf)
DP(NETIF_MSG_HW, "write 0x%08x to IGU(via GRC) addr 0x%x\n",
data, igu_addr_data);
REG_WR(bp, igu_addr_data, data);
mmiowb();
barrier();
DP(NETIF_MSG_HW, "write 0x%08x to IGU(via GRC) addr 0x%x\n",
ctl, igu_addr_ctl);
REG_WR(bp, igu_addr_ctl, ctl);
mmiowb();
barrier();
/* wait for clean up to finish */
@ -9550,7 +9531,6 @@ static void bnx2x_set_234_gates(struct bnx2x *bp, bool close)
DP(NETIF_MSG_HW | NETIF_MSG_IFUP, "%s gates #2, #3 and #4\n",
close ? "closing" : "opening");
mmiowb();
}
#define SHARED_MF_CLP_MAGIC 0x80000000 /* `magic' bit */
@ -9674,7 +9654,6 @@ static void bnx2x_pxp_prep(struct bnx2x *bp)
if (!CHIP_IS_E1(bp)) {
REG_WR(bp, PXP2_REG_RD_START_INIT, 0);
REG_WR(bp, PXP2_REG_RQ_RBC_DONE, 0);
mmiowb();
}
}
@ -9774,16 +9753,13 @@ static void bnx2x_process_kill_chip_reset(struct bnx2x *bp, bool global)
reset_mask1 & (~not_reset_mask1));
barrier();
mmiowb();
REG_WR(bp, GRCBASE_MISC + MISC_REGISTERS_RESET_REG_2_SET,
reset_mask2 & (~stay_reset2));
barrier();
mmiowb();
REG_WR(bp, GRCBASE_MISC + MISC_REGISTERS_RESET_REG_1_SET, reset_mask1);
mmiowb();
}
/**
@ -9867,9 +9843,6 @@ static int bnx2x_process_kill(struct bnx2x *bp, bool global)
REG_WR(bp, MISC_REG_UNPREPARED, 0);
barrier();
/* Make sure all is written to the chip before the reset */
mmiowb();
/* Wait for 1ms to empty GLUE and PCI-E core queues,
* PSWHST, GRC and PSWRD Tetris buffer.
*/
@ -14828,7 +14801,6 @@ static int bnx2x_drv_ctl(struct net_device *dev, struct drv_ctl_info *ctl)
if (rc)
break;
mmiowb();
barrier();
/* Start accepting on iSCSI L2 ring */
@ -14863,7 +14835,6 @@ static int bnx2x_drv_ctl(struct net_device *dev, struct drv_ctl_info *ctl)
if (!bnx2x_wait_sp_comp(bp, sp_bits))
BNX2X_ERR("rx_mode completion timed out!\n");
mmiowb();
barrier();
/* Unset iSCSI L2 MAC */

1
drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c
View File

@ -5039,7 +5039,6 @@ static inline int bnx2x_q_init(struct bnx2x *bp,
/* As no ramrod is sent, complete the command immediately */
o->complete_cmd(bp, o, BNX2X_Q_CMD_INIT);
mmiowb();
smp_mb();
return 0;

2
drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c
View File

@ -100,13 +100,11 @@ static void bnx2x_vf_igu_ack_sb(struct bnx2x *bp, struct bnx2x_virtf *vf,
DP(NETIF_MSG_HW, "write 0x%08x to IGU(via GRC) addr 0x%x\n",
cmd_data.sb_id_and_flags, igu_addr_data);
REG_WR(bp, igu_addr_data, cmd_data.sb_id_and_flags);
mmiowb();
barrier();
DP(NETIF_MSG_HW, "write 0x%08x to IGU(via GRC) addr 0x%x\n",
ctl, igu_addr_ctl);
REG_WR(bp, igu_addr_ctl, ctl);
mmiowb();
barrier();
}

4
drivers/net/ethernet/broadcom/bnx2x/bnx2x_vfpf.c
View File

@ -172,8 +172,6 @@ static int bnx2x_send_msg2pf(struct bnx2x *bp, u8 *done, dma_addr_t msg_mapping)
/* Trigger the PF FW */
writeb_relaxed(1, &zone_data->trigger.vf_pf_channel.addr_valid);
mmiowb();
/* Wait for PF to complete */
while ((tout >= 0) && (!*done)) {
msleep(interval);
@ -1179,7 +1177,6 @@ static void bnx2x_vf_mbx_resp_send_msg(struct bnx2x *bp,
/* ack the FW */
storm_memset_vf_mbx_ack(bp, vf->abs_vfid);
mmiowb();
/* copy the response header including status-done field,
* must be last dmae, must be after FW is acked
@ -2174,7 +2171,6 @@ static void bnx2x_vf_mbx_request(struct bnx2x *bp, struct bnx2x_virtf *vf,
*/
storm_memset_vf_mbx_ack(bp, vf->abs_vfid);
/* Firmware ack should be written before unlocking channel */
mmiowb();
bnx2x_unlock_vf_pf_channel(bp, vf, mbx->first_tlv.tl.type);
}
}

3
drivers/net/ethernet/broadcom/bnxt/bnxt.c
View File

@ -556,8 +556,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
tx_done:
mmiowb();
if (unlikely(bnxt_tx_avail(bp, txr) <= MAX_SKB_FRAGS + 1)) {
if (skb->xmit_more && !tx_buf->is_push)
bnxt_db_write(bp, &txr->tx_db, prod);
@ -2134,7 +2132,6 @@ static int bnxt_poll(struct napi_struct *napi, int budget)
&dim_sample);
net_dim(&cpr->dim, dim_sample);
}
mmiowb();
return work_done;
}

6
drivers/net/ethernet/broadcom/tg3.c
View File

@ -1073,7 +1073,6 @@ static void tg3_int_reenable(struct tg3_napi *tnapi)
struct tg3 *tp = tnapi->tp;
tw32_mailbox(tnapi->int_mbox, tnapi->last_tag << 24);
mmiowb();
/* When doing tagged status, this work check is unnecessary.
* The last_tag we write above tells the chip which piece of
@ -6999,7 +6998,6 @@ static int tg3_rx(struct tg3_napi *tnapi, int budget)
tw32_rx_mbox(TG3_RX_JMB_PROD_IDX_REG,
tpr->rx_jmb_prod_idx);
}
mmiowb();
} else if (work_mask) {
/* rx_std_buffers[] and rx_jmb_buffers[] entries must be
* updated before the producer indices can be updated.
@ -7210,8 +7208,6 @@ static int tg3_poll_work(struct tg3_napi *tnapi, int work_done, int budget)
tw32_rx_mbox(TG3_RX_JMB_PROD_IDX_REG,
dpr->rx_jmb_prod_idx);
mmiowb();
if (err)
tw32_f(HOSTCC_MODE, tp->coal_now);
}
@ -7278,7 +7274,6 @@ static int tg3_poll_msix(struct napi_struct *napi, int budget)
HOSTCC_MODE_ENABLE |
tnapi->coal_now);
}
mmiowb();
break;
}
}
@ -8159,7 +8154,6 @@ static netdev_tx_t tg3_start_xmit(struct sk_buff *skb, struct net_device *dev)
if (!skb->xmit_more || netif_xmit_stopped(txq)) {
/* Packets are ready, update Tx producer idx on card. */
tw32_tx_mbox(tnapi->prodmbox, entry);
mmiowb();
}
return NETDEV_TX_OK;

10
drivers/net/ethernet/cavium/liquidio/cn66xx_device.c
View File

@ -38,9 +38,6 @@ int lio_cn6xxx_soft_reset(struct octeon_device *oct)
lio_pci_readq(oct, CN6XXX_CIU_SOFT_RST);
lio_pci_writeq(oct, 1, CN6XXX_CIU_SOFT_RST);
/* make sure that the reset is written before starting timer */
mmiowb();
/* Wait for 10ms as Octeon resets. */
mdelay(100);
@ -487,9 +484,6 @@ void lio_cn6xxx_disable_interrupt(struct octeon_device *oct,
/* Disable Interrupts */
writeq(0, cn6xxx->intr_enb_reg64);
/* make sure interrupts are really disabled */
mmiowb();
}
static void lio_cn6xxx_get_pcie_qlmport(struct octeon_device *oct)
@ -555,10 +549,6 @@ static int lio_cn6xxx_process_droq_intr_regs(struct octeon_device *oct)
value &= ~(1 << oq_no);
octeon_write_csr(oct, reg, value);
/* Ensure that the enable register is written.
*/
mmiowb();
spin_unlock(&cn6xxx->lock_for_droq_int_enb_reg);
}
}

1
drivers/net/ethernet/cavium/liquidio/octeon_device.c
View File

@ -1449,7 +1449,6 @@ void lio_enable_irq(struct octeon_droq *droq, struct octeon_instr_queue *iq)
iq->pkt_in_done -= iq->pkts_processed;
iq->pkts_processed = 0;
/* this write needs to be flushed before we release the lock */
mmiowb();
spin_unlock_bh(&iq->lock);
oct = iq->oct_dev;
}

4
drivers/net/ethernet/cavium/liquidio/octeon_droq.c
View File

@ -513,8 +513,6 @@ int octeon_retry_droq_refill(struct octeon_droq *droq)
*/
wmb();
writel(desc_refilled, droq->pkts_credit_reg);
/* make sure mmio write completes */
mmiowb();
if (pkts_credit + desc_refilled >= CN23XX_SLI_DEF_BP)
reschedule = 0;
@ -712,8 +710,6 @@ octeon_droq_fast_process_packets(struct octeon_device *oct,
*/
wmb();
writel(desc_refilled, droq->pkts_credit_reg);
/* make sure mmio write completes */
mmiowb();
}
}
} /* for (each packet)... */

1
drivers/net/ethernet/cavium/liquidio/request_manager.c
View File

@ -278,7 +278,6 @@ ring_doorbell(struct octeon_device *oct, struct octeon_instr_queue *iq)
if (atomic_read(&oct->status) == OCT_DEV_RUNNING) {
writel(iq->fill_cnt, iq->doorbell_reg);
/* make sure doorbell write goes through */
mmiowb();
iq->fill_cnt = 0;
iq->last_db_time = jiffies;
return;

Some files were not shown because too many files have changed in this diff Show More