A still unconfirmed hardware bug prevents the IPMMU microTLB 0 to be
flushed correctly, resulting in memory corruption. DMAC 0 channel 0 is
connected to microTLB 0 on currently supported platforms, so we can't
use it with the IPMMU. As the IOMMU API operates at the device level we
can't disable it selectively, so ignore channel 0 for now if the device
is part of an IOMMU group.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
When descriptor memory is accessed through an IOMMU the DMADAR register
isn't initialized automatically from the first descriptor at beginning
of transfer by the DMAC like it should. Initialize it manually with the
destination address of the first chunk.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
When wired to an IOMMU to access data, the DMAC accesses the hardware
descriptors through the IOMMU as well. We're using the DMA mapping API
to allocate the descriptors, but with a NULL device at the moment, which
prevents IOMMU mappings from being created. Fix this by passing the DMAC
device instead.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The error interrupt handler stops and reinitializes all channels. This
causes a crash for channels that have never been used, as their
descriptor lists are uninitialized. Fix it by initializing the
descriptor lists at probe time.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The rcar_dmac_desc_put() function is called in interrupt context and
must thus use spin_lock_irqsave() instead of spin_lock_irq().
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The desc variable is used uninitialized in the rcar_dmac_desc_get() and
rcar_dmac_xfer_chunk_get() functions if descriptors need to be
allocated. Fix it.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
If the atomic DMA coherent pool is too small, disable use of hardware
descriptor lists instead of crashing the system:
ERROR: 256 KiB atomic DMA coherent pool is too small!
Please increase it with coherent_pool= kernel parameter!
Unable to handle kernel NULL pointer dereference at virtual address 00000004
Internal error: Oops: a07 [#1] PREEMPT SMP ARM
PC is at rcar_dmac_chan_reinit+0x3c/0x160
LR is at _raw_spin_lock_irqsave+0x18/0x5c
[<802132c0>] (rcar_dmac_chan_reinit) from [<80214818>] (rcar_dmac_isr_error+0x84/0xa0)
[<80214818>] (rcar_dmac_isr_error) from [<80060484>] (handle_irq_event_percpu+0x50/0x150)
[<80060484>] (handle_irq_event_percpu) from [<800605c0>] (handle_irq_event+0x3c/0x5c)
[<800605c0>] (handle_irq_event) from [<8006350c>] (handle_fasteoi_irq+0xb8/0x198)
[<8006350c>] (handle_fasteoi_irq) from [<8005fdb0>] (generic_handle_irq+0x20/0x30)
[<8005fdb0>] (generic_handle_irq) from [<8000fcd0>] (handle_IRQ+0x50/0xc4)
[<8000fcd0>] (handle_IRQ) from [<800092cc>] (gic_handle_irq+0x28/0x5c)
[<800092cc>] (gic_handle_irq) from [<80012700>] (__irq_svc+0x40/0x70)
Kernel panic - not syncing: Fatal exception in interrupt
Signed-off-by: Jürg Billeter <j@bitron.ch>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Unlike DMA transfers descriptors that are preallocated and cached,
memory used to store hardware descriptors is allocated and freed with
the DMA coherent allocation API for every transfer. Besides degrading
performances, this creates a CMA stress test that seems to cause issues.
Running dmatest with the noverify option produces
[ 50.066539] alloc_contig_range test_pages_isolated(6b845, 6b846) failed
[ 50.235180] alloc_contig_range test_pages_isolated(6b848, 6b84e) failed
[ 52.964584] alloc_contig_range test_pages_isolated(6b847, 6b848) failed
[ 54.127113] alloc_contig_range test_pages_isolated(6b843, 6b844) failed
[ 56.270253] alloc_contig_range test_pages_isolated(6b84c, 6b850) failed
The root cause needs to be fixed, but in the meantime, as a workaround
and a performance improvement, cache hardware descriptors.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Tested-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
The DMAC supports hardware-based auto-configuration from descriptor
lists. This reduces the number of interrupts required for processing a
DMA transfer. Support that mode in the driver.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Tested-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
The DMAC is a general purpose multi-channel DMA controller that supports
both slave and memcpy transfers.
The driver currently supports the DMAC found in the r8a7790 and r8a7791
SoCs. Support for compatible DMA controllers (such as the audio DMAC)
will be added later.
Feature-wise, automatic hardware handling of descriptors chains isn't
supported yet. LPAE support is implemented.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Tested-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Tested-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
