Commit Graph

35 Commits

Author SHA1 Message Date
Jean-Philippe Brucker
9aeb26cfc2 iommu/arm-smmu: Wire up map_sg for arm-smmu-v3
The map_sg callback is missing from arm_smmu_ops, but is required by
iommu.h. Similarly to most other IOMMU drivers, connect it to
default_iommu_map_sg.

Cc: <stable@vger.kernel.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-13 11:00:59 +02:00
Arnd Bergmann
287980e49f remove lots of IS_ERR_VALUE abuses
Most users of IS_ERR_VALUE() in the kernel are wrong, as they
pass an 'int' into a function that takes an 'unsigned long'
argument. This happens to work because the type is sign-extended
on 64-bit architectures before it gets converted into an
unsigned type.

However, anything that passes an 'unsigned short' or 'unsigned int'
argument into IS_ERR_VALUE() is guaranteed to be broken, as are
8-bit integers and types that are wider than 'unsigned long'.

Andrzej Hajda has already fixed a lot of the worst abusers that
were causing actual bugs, but it would be nice to prevent any
users that are not passing 'unsigned long' arguments.

This patch changes all users of IS_ERR_VALUE() that I could find
on 32-bit ARM randconfig builds and x86 allmodconfig. For the
moment, this doesn't change the definition of IS_ERR_VALUE()
because there are probably still architecture specific users
elsewhere.

Almost all the warnings I got are for files that are better off
using 'if (err)' or 'if (err < 0)'.
The only legitimate user I could find that we get a warning for
is the (32-bit only) freescale fman driver, so I did not remove
the IS_ERR_VALUE() there but changed the type to 'unsigned long'.
For 9pfs, I just worked around one user whose calling conventions
are so obscure that I did not dare change the behavior.

I was using this definition for testing:

 #define IS_ERR_VALUE(x) ((unsigned long*)NULL == (typeof (x)*)NULL && \
       unlikely((unsigned long long)(x) >= (unsigned long long)(typeof(x))-MAX_ERRNO))

which ends up making all 16-bit or wider types work correctly with
the most plausible interpretation of what IS_ERR_VALUE() was supposed
to return according to its users, but also causes a compile-time
warning for any users that do not pass an 'unsigned long' argument.

I suggested this approach earlier this year, but back then we ended
up deciding to just fix the users that are obviously broken. After
the initial warning that caused me to get involved in the discussion
(fs/gfs2/dir.c) showed up again in the mainline kernel, Linus
asked me to send the whole thing again.

[ Updated the 9p parts as per Al Viro  - Linus ]

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Andrzej Hajda <a.hajda@samsung.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: https://lkml.org/lkml/2016/1/7/363
Link: https://lkml.org/lkml/2016/5/27/486
Acked-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> # For nvmem part
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-27 15:26:11 -07:00
Robin Murphy
d546635731 iommu/arm-smmu: Use per-domain page sizes.
Now that we can accurately reflect the context format we choose for each
domain, do that instead of imposing the global lowest-common-denominator
restriction and potentially ending up with nothing. We currently have a
strict 1:1 correspondence between domains and context banks, so we don't
need to entertain the possibility of multiple formats _within_ a domain.

Signed-off-by: Will Deacon <will.deacon@arm.com>
[rm: split from original patch, added SMMUv3]
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 19:38:39 +02:00
Will Deacon
cbf8277ef4 iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass for now
Until all upstream devices have their DMA ops swizzled to point at the
SMMU, we need to treat the IOMMU_DOMAIN_DMA domain as bypass to avoid
putting devices into an empty address space when detaching from VFIO.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-02-18 15:02:44 +00:00
Will Deacon
bc7f2ce0a7 iommu/arm-smmu: Don't fail device attach if already attached to a domain
The ARM SMMU attach_dev implementations returns -EEXIST if the device
being attached is already attached to a domain. This doesn't play nicely
with the default domain, resulting in splats such as:

  WARNING: at drivers/iommu/iommu.c:1257
  Modules linked in:

  CPU: 3 PID: 1939 Comm: virtio-net-tx Tainted: G S              4.5.0-rc4+ #1
  Hardware name: FVP Base (DT)
  task: ffffffc87a9d0000 ti: ffffffc07a278000 task.ti: ffffffc07a278000
  PC is at __iommu_detach_group+0x68/0xe8
  LR is at __iommu_detach_group+0x48/0xe8

This patch fixes the problem by forcefully detaching the device from
its old domain, if present, when attaching to a new one. The unused
->detach_dev callback is also removed the iommu_ops structures.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-02-18 15:02:44 +00:00
Robin Murphy
9adb95949a iommu/arm-smmu: Support DMA-API domains
With DMA mapping ops provided by the iommu-dma code, only a minimal
contribution from the IOMMU driver is needed to create a suitable
DMA-API domain for them to use. Implement this for the ARM SMMUs.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-02-18 15:02:43 +00:00
Prem Mallappa
6380be0535 iommu/arm-smmu: Use STE.S1STALLD only when supported
It is ILLEGAL to set STE.S1STALLD to 1 if stage 1 is enabled and
either the stall or terminate models are not supported.

This patch fixes the STALLD check and ensures that we don't set STALLD
in the STE when it is not supported.

Signed-off-by: Prem Mallappa <pmallapp@broadcom.com>
[will: consistently use IDR0_STALL_MODEL_* prefix]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:36 +00:00
Prem Mallappa
324ba10823 iommu/arm-smmu: Fix write to GERRORN register
When acknowledging global errors, the GERRORN register should be written
with the original GERROR value so that active errors are toggled.

This patch fixed the driver to write the original GERROR value to
GERRORN, instead of an active error mask.

Signed-off-by: Prem Mallappa <pmallapp@broadcom.com>
[will: reworked use of active bits and fixed commit log]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:35 +00:00
Robin Murphy
75df138655 iommu/arm-smmu: Invalidate TLBs properly
When invalidating an IOVA range potentially spanning multiple pages,
such as when removing an entire intermediate-level table, we currently
only issue an invalidation for the first IOVA of that range. Since the
architecture specifies that address-based TLB maintenance operations
target a single entry, an SMMU could feasibly retain live entries for
subsequent pages within that unmapped range, which is not good.

Make sure we hit every possible entry by iterating over the whole range
at the granularity provided by the pagetable implementation.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
[will: added missing semicolons...]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:35 +00:00
Robin Murphy
06c610e8f3 iommu/io-pgtable: Indicate granule for TLB maintenance
IOMMU hardware with range-based TLB maintenance commands can work
happily with the iova and size arguments passed via the tlb_add_flush
callback, but for IOMMUs which require separate commands per entry in
the range, it is not straightforward to infer the necessary granularity
when it comes to issuing the actual commands.

Add an additional argument indicating the granularity for the benefit
of drivers needing to know, and update the ARM LPAE code appropriately
(for non-leaf invalidations we currently just assume the worst-case
page granularity rather than walking the table to check).

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:34 +00:00
Will Deacon
a0d5c04c60 iommu/arm-smmu: Handle unknown CERROR values gracefully
Whilst the architecture only defines a few of the possible CERROR values,
we should handle unknown values gracefully rather than go out of bounds
trying to print out an error description.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:33 +00:00
Peng Fan
9a4a9d8c34 iommu/arm-smmu: Correct group reference count
The basic flow for add a device:
 arm_smmu_add_device
        |->iommu_group_get_for_dev
            |->iommu_group_get
                     return group;  (1)
            |->ops->device_group : Init/increase reference count to/by 1.
            |->iommu_group_add_device : Increase reference count by 1.
		     return group   (2)
        |->return 0;

Since we are adding one device, the flow is (2) and the group reference
count will be increased by 2. So, we need to add iommu_group_put at the
end of arm_smmu_add_device to decrease the count by 1.

Also take the failure path into consideration when fail to add a device.

Signed-off-by: Peng Fan <van.freenix@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:33 +00:00
Will Deacon
a0eacd89e3 iommu/arm-smmu: Use incoming shareability attributes in bypass mode
When we initialise a bypass STE, we memset the structure to zero and
set the Valid and Config fields to indicate that the stream should
bypass the SMMU. Unfortunately, this results in an SHCFG field of 0
which means that the shareability of any incoming transactions is
overridden with non-shareable, leading to potential coherence problems
down the line.

This patch fixes the issue by initialising bypass STEs to use the
incoming shareability attributes. When translation is in effect at
either stage 1 or stage 2, the shareability is determined by the
page tables.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:33 +00:00
Will Deacon
04fa26c71b iommu/arm-smmu: Convert DMA buffer allocations to the managed API
The ARM SMMUv3 driver uses dma_{alloc,free}_coherent to manage its
queues and configuration data structures.

This patch converts the driver to the managed (dmam_*) API, so that
resources are freed automatically on device teardown. This greatly
simplifies the failure paths and allows us to remove a bunch of
handcrafted freeing code.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:32 +00:00
Will Deacon
89df3a96ba iommu/arm-smmu: Remove #define for non-existent PRIQ_0_OF field
PRIQ_0_OF has been removed from the SMMUv3 architecture, so remove its
corresponding (and unused) #define from the driver.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-12-17 12:05:32 +00:00
Joerg Roedel
b67ad2f7c7 Merge branches 'x86/vt-d', 'arm/omap', 'arm/smmu', 's390', 'core' and 'x86/amd' into next
Conflicts:
	drivers/iommu/amd_iommu_types.h
2015-11-02 20:03:34 +09:00
Joerg Roedel
af65993224 iommu/arm-smmu: Switch to device_group call-back
This converts the ARM SMMU and the SMMUv3 driver to use the
new device_group call-back.

Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22 00:00:49 +02:00
Marc Zyngier
166bdbd231 iommu/arm-smmu: Add support for MSI on SMMUv3
Despite being a platform device, the SMMUv3 is capable of signaling
interrupts using MSIs. Hook it into the platform MSI framework and
enjoy faults being reported in a new and exciting way.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
[will: tidied up the binding example and reworked most of the code]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-10-15 16:29:58 +01:00
Will Deacon
c0733a2cf3 iommu/arm-smmu: Fix error checking for ASID and VMID allocation
The bitmap allocator returns an int, which is one of the standard
negative values on failure. Rather than assigning this straight to a
u16 (like we do for the ASID and VMID callers), which means that we
won't detect failure correctly, use an int for the purposes of error
checking.

Cc: <stable@vger.kernel.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-10-15 16:29:47 +01:00
Will Deacon
941a802d93 iommu/arm-smmu: Use drvdata instead of maintaining smmu_devices list
Rather than keep a private list of struct arm_smmu_device and searching
this whenever we need to look up the correct SMMU instance, instead use
the drvdata field in the struct device to take care of the mapping for
us.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-10-14 13:43:50 +01:00
Will Deacon
1c27df1c0a iommu/arm-smmu: Use correct address mask for CMD_TLBI_S2_IPA
Stage-2 TLBI by IPA takes a 48-bit address field, as opposed to the
64-bit field used by the VA-based invalidation commands.

This patch re-jigs the SMMUv3 command construction code so that the
address field is correctly masked.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-09-22 17:36:05 +01:00
Will Deacon
f0c453dbcc iommu/arm-smmu: Ensure IAS is set correctly for AArch32-capable SMMUs
AArch32-capable SMMU implementations have a minimum IAS of 40 bits, so
ensure that is reflected in the stage-2 page table configuration.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-09-22 17:36:04 +01:00
Robin Murphy
857c88ca62 iommu/arm-smmu: Remove arm_smmu_flush_pgtable()
With the io-pgtable code now enforcing its own appropriate sync points,
the vestigial flush_pgtable callback becomes entirely redundant, so
remove it altogether.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-08-06 14:35:40 +01:00
Robin Murphy
bdc6d97347 iommu/arm-smmu: Clean up DMA API usage
With the correct DMA API calls now integrated into the io-pgtable code,
let that handle the flushing of non-coherent page table updates.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-08-06 14:35:39 +01:00
Will Deacon
85430968ae iommu/arm-smmu: Treat unknown OAS as 48-bit
A late change to the SMMUv3 architecture ensures that the OAS field
will be monotonically increasing, so we can assume that an unknown OAS
is at least 48-bit and use that, rather than fail the device probe.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-08-06 14:35:38 +01:00
Will Deacon
28c8b4045b iommu/arm-smmu: Limit 2-level strtab allocation for small SID sizes
If the StreamIDs in a system can all be resolved by a single level-2
stream table (i.e. SIDSIZE < SPLIT), then we currently get our maths
wrong and allocate the largest strtab we support, thanks to unsigned
overflow in our calculation.

This patch fixes the issue by checking the SIDSIZE explicitly when
calculating the size of our first-level stream table.

Reported-by: Matt Evans <matt.evans@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-31 11:38:25 +01:00
Marc Zyngier
ec11d63c67 iommu/arm-smmu: Fix MSI memory attributes to match specification
The MSI memory attributes in the SMMUv3 driver are from an older
revision of the spec, which doesn't match the current implementations.

Out with the old, in with the new.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-31 11:38:19 +01:00
Marc Zyngier
ccd6385dfb iommu/arm-smmu: Fix enabling of PRIQ interrupt
When an ARM SMMUv3 instance supports PRI, the driver registers
an interrupt handler, but fails to enable the generation of
such interrupt at the SMMU level.

This patches simply moves the enable flags to a variable that
gets updated by the PRI handling code before being written to the
SMMU register.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-31 11:38:14 +01:00
Zhen Lei
5e92946c39 iommu/arm-smmu: Skip the execution of CMD_PREFETCH_CONFIG
Hisilicon SMMUv3 devices treat CMD_PREFETCH_CONFIG as a illegal command,
execute it will trigger GERROR interrupt. Although the gerror code manage
to turn the prefetch into a SYNC, and the system can continue to run
normally, but it's ugly to print error information.

Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
[will: extended binding documentation]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-08 17:24:39 +01:00
Zhen Lei
e2f4c2330f iommu/arm-smmu: Enlarge STRTAB_L1_SZ_SHIFT to support larger sidsize
Because we will choose the minimum value between STRTAB_L1_SZ_SHIFT and
IDR1.SIDSIZE, so enlarge STRTAB_L1_SZ_SHIFT will not impact the platforms
whose IDR1.SIDSIZE is smaller than old STRTAB_L1_SZ_SHIFT value.

Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-08 17:24:39 +01:00
Zhen Lei
5d58c6207c iommu/arm-smmu: Fix the values of ARM64_TCR_{I,O}RGN0_SHIFT
The arm64 CPU architecture defines TCR[8:11] as holding the inner and
outer memory attributes for TTBR0.

This patch fixes the ARM SMMUv3 driver to pack these bits into the
context descriptor, rather than picking up the TTBR1 attributes as it
currently does.

Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-08 17:24:39 +01:00
Will Deacon
d2e88e7c08 iommu/arm-smmu: Fix LOG2SIZE setting for 2-level stream tables
STRTAB_BASE_CFG.LOG2SIZE should be set to log2(entries), where entries
is the *total* number of entries in the stream table, not just the first
level.

This patch fixes the register setting, which was previously being set to
the size of the l1 thanks to a multi-use "size" variable.

Reported-by: Zhen Lei <thunder.leizhen@huawei.com>
Tested-by: Zhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-08 17:24:38 +01:00
Zhen Lei
69146e7bfc iommu/arm-smmu: Fix the index calculation of strtab
The element size of cfg->strtab is just one DWORD, so we should use a
multiply operation instead of a shift when calculating the level 1
index.

Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-08 17:24:38 +01:00
Markus Elfring
a6e08fb2d2 iommu/arm-smmu: Delete an unnecessary check before the function call "free_io_pgtable_ops"
The free_io_pgtable_ops() function tests whether its argument is NULL
and then returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.

Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-29 21:57:32 +02:00
Will Deacon
48ec83bcbc iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices
Version three of the ARM SMMU architecture introduces significant
changes and improvements over previous versions of the specification,
necessitating a new driver in the Linux kernel.

The main change to the programming interface is that the majority of the
configuration data has been moved from MMIO registers to in-memory data
structures, with communication between the CPU and the SMMU being
mediated via in-memory circular queues.

This patch adds an initial driver for SMMUv3 to Linux. We currently
support pinned stage-1 (DMA) and stage-2 (KVM VFIO) mappings using the
generic IO-pgtable code.

Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-05-29 11:12:01 +02:00