Current code is violating the DMA Engine API by putting the submitted
requests directly into the HW queue. This causes queued transactions
to be started by another thread as soon as the first one finishes.
The DMA Engine document clearly states this.
"dmaengine_submit() will not start the DMA operation".
Move HW queuing of the requests into the issue_pending() routine
to comply with API requirements also create a new queued state for
temporarily holding the requests.
A descriptor goes through these transitions now.
free->prepared->queued->active->completed->free
as opposed to
free->prepared->active->completed->free
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Parameters like maximum read/write request size and the maximum
number of active transactions are currently configured in DT/ACPI.
This patch allows a user to override these to fine tune performance
for their application.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
We need to ensure that all DMAs and interrupts are cleared during
shutdown operation in order for kexec to start the next kernel clearly.
Otherwise, HW could be performing a DMA into random addresses in the
middle of second kernel start.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Once the channels are stopped, disable interrupts to make sure no new
HW interaction can happen.
Similarly, re-enable the interrupts only if we know that channel is
operational again.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
MODULE_DEVICE_TABLE is used by the kernel to determine which device driver
should be loaded for which platform device. MODULE_DEVICE_TABLE has been
only defined for the device-tree based platforms in the current code.
Defining it also for ACPI based platforms.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The 4.8-rc8 kernel is printing duplicate file entry warnings while removing
the HIDMA object. This is caused by stale sysfs entries remaining from the
previous execution.
_sysfs_warn_dup+0x5c/0x78
sysfs_add_file_mode_ns+0x13c/0x1c0
sysfs_create_file_ns+0x2c/0x40
device_create_file+0x54/0xa0
hidma_probe+0x7c8/0x808
Create hidma_sysfs_init and hidma_sysfs_uninit functions and call them from
the probe and remove path. To do proper clean up, adding the attrs object
to the device data structure to keep it around until remove call is made.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The newly added MSI support causes a harmless warning when MSI
is disabled:
drivers/dma/qcom/hidma.c:558:20: error: 'hidma_chirq_handler_msi' defined but not used [-Werror=unused-function]
This adds another #ifdef to match that around the users of the function.
Fixes: 1c0e3e82a7 ("dmaengine: qcom_hidma: add MSI support for interrupts")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Device node iterators put the previous value of the index variable, so an
explicit put causes a double put.
Generated by: scripts/coccinelle/iterators/device_node_continue.cocci
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The interrupts can now be delivered as platform MSI interrupts on newer
platforms. The code looks for a new OF and ACPI strings in order to enable
the functionality.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
When MSI interrupts are supported, error and the transfer interrupt can
come from multiple processor contexts.
Each error interrupt is an MSI interrupt. If the channel is disabled by
the first error interrupt, the remaining error interrupts will gracefully
return in the interrupt handler.
If an error is observed while servicing the completions in success case,
the posting of the completions will be aborted as soon as channel disabled
state is observed. The error interrupt handler will take it from there and
finish the remaining completions. We don't want to create multiple success
and error messages to be delivered to the client in mixed order.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Bring out the interrupt cause to the top level so that MSI interrupts
can be hooked at a later stage.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Getting ready for the MSI interrupts. The pending_tre_count is used
in the interrupt handler to make sure all outstanding requests are
serviced.
The driver will allocate 11 MSI interrupts. Each MSI interrupt can be
assigned to a different CPU. Then, we have a race condition for common
variables as they share the same interrupt handler with a different
cause bit and they can potentially be executed in parallel. Making this
variable atomic so that it can be updated from multiple processor
contexts.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
We try to consume as much successful transfers as possible. Now that we
support MSI interrupts, an error interrupt might be observed by another
processor while we are finishing the successful ones.
Try to abort successful processing if this is the case.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Introducing the hidma_ll_setup_irq function to set up the interrupt
type externally from the OS interface.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Configure the DMA bindings for the device tree based firmware.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Since 'commit acc29fb8f7 ("debugfs: ->d_parent is never NULL or
negative")', HIDMA object removal is no longer working. This is due to
redundant debugfs remove call in hidma_debug_uninit.
debugfs_remove_recursive(dmadev->debugfs);
debugfs_remove_recursive(dmadev->stats);
The first remove is for the directory. Second remove is for the file under
the directory. The directory remove makes file remove invalid.
Unable to handle kernel NULL pointer dereference at virtual address
[<ffff00000889f480>] down_write+0x18/0x68
[<ffff00000831c220>] debugfs_remove_recursive+0x50/0x1c0
[<ffff00000848e0a8>] hidma_debug_uninit+0x20/0x30
[<ffff00000848c5d8>] hidma_remove+0x48/0x98
[<ffff000008511b6c>] platform_drv_remove+0x24/0x68
[<ffff00000850fac8>] __device_release_driver+0x80/0x118
[<ffff00000850fb84>] device_release_driver+0x24/0x38
[<ffff00000850e928>] unbind_store+0xe8/0x110
[<ffff00000850dd30>] drv_attr_store+0x20/0x30
[<ffff000008253a48>] sysfs_kf_write+0x48/0x58
[<ffff000008252dd8>] kernfs_fop_write+0xb0/0x1d8
[<ffff0000081dab3c>] __vfs_write+0x1c/0x110
[<ffff0000081db940>] vfs_write+0xa0/0x1b8
[<ffff0000081dcd34>] SyS_write+0x44/0xa0
[<ffff000008082ef0>] el0_svc_naked+0x24/0x28
Removing the second line.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
When an error is observed, we try to disable the channel and prevent
further accesses from the client.
Depending on the type of error, transitioning into disabled state might
not be possible. Adding a check to make sure that HW is in enabled/running
state before the disable transition happens.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The HIDMA driver is capable of error detection. However, the error was
not being passed back to the client when tx_status API is called.
Changing the error handling behavior to follow this oder.
1. dmaengine asserts error interrupt
2. Driver receives and mark's the txn as error
3. Driver completes the txn and intimates the client. No further
submissions. Drop the locks before calling callback, as subsequent
processing by client maybe in callback thread.
4. Client invokes status and you can return error
5. On error, client calls terminate_all. You can reset channel, free all
descriptors in the active, pending and completed lists
6. Client prepares new txn and so on.
As part of this work, got rid of the reset in the interrupt handler when
an error happens and the HW is put into disabled state. The only way to
recover is for the client to terminate the channel.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Pass the DMA errors to the client by passing a result argument. The HW only
supports a generic error when something goes wrong. That's why, using
DMA_TRANS_ABORTED all the time.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
There is a race condition between data transfer callback and descriptor
free code. The callback routine may decide to clear the resources even
though the descriptor has not yet been freed.
Instead of calling the callback first and then releasing the memory,
this code is changing the order to return the descriptor back to the
free pool and then call the user provided callback.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This is in preperation of moving to a callback that provides results to the
callback for the transaction. The conversion will maintain current behavior
and the driver must convert to new callback mechanism at a later time in
order to receive results.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Use for_each_matching_node() macro instead of open coding it.
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Acked-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
drivers should ensure that tasklets are killed, so that they can't be
run after driver remove is executed
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Cc: Sinan Kaya <okaya@codeaurora.org>
drivers should ensure that tasklets are killed, so that they can't be
run after driver remove is executed
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Acked-by: Sinan Kaya <okaya@codeaurora.org>
In case of error, the function platform_device_register_full()
returns ERR_PTR() and never returns NULL. The NULL test in the
return value check should be replaced with IS_ERR().
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The bam_dma driver gained runtime PM support, but that causes build
warnings whenever CONFIG_PM is disabled:
drivers/dma/qcom/bam_dma.c:1324:12: error: 'bam_dma_runtime_resume' defined but not used [-Werror=unused-function]
static int bam_dma_runtime_resume(struct device *dev)
^~~~~~~~~~~~~~~~~~~~~~
drivers/dma/qcom/bam_dma.c:1315:12: error: 'bam_dma_runtime_suspend' defined but not used [-Werror=unused-function]
static int bam_dma_runtime_suspend(struct device *dev)
This removes the incomplete #ifdef guard and instead marks all
four PM functions as __maybe_unused, which avoids this kind of
warning.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Fixes: 7d2545599f ("dmaengine: qcom-bam-dma: Add pm_runtime support")
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Adds pm_runtime support for BAM DMA so that clock is enabled only
when there is a transaction going on to help save power.
Signed-off-by: Pramod Gurav <pramod.gurav@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
In order to create a relationship model between the channels and the
management object, we are adding support for object hierarchy to the
drivers. This patch simplifies the userspace application development.
We will not have to traverse different firmware paths based on device
tree or ACPI based kernels.
No matter what flavor of kernel is used, objects will be represented as
platform devices.
The new layout is as follows:
hidmam_10: hidma-mgmt@0x5A000000 {
compatible = "qcom,hidma-mgmt-1.0";
...
hidma_10: hidma@0x5a010000 {
compatible = "qcom,hidma-1.0";
...
}
}
The hidma_mgmt_init detects each instance of the hidma-mgmt-1.0 objects
in device tree and calls into the channel driver to create platform devices
for each child of the management object.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Add debugfs hooks for debugging the execution behavior of the DMA
channel. The debugfs hooks get initialized by the probe function and
uninitialized by the remove function.
A stats file is created in debugfs. The stats file will show the
information about each HIDMA channel as well as each asynchronous job
queued and completed at a given time.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This patch implements the hardware hooks for the HIDMA channel driver.
The main functions of interest are:
- hidma_ll_init
- hidma_ll_request
- hidma_ll_queue_request
- hidma_ll_hw_start
OS layer calls the hidma_ll_init function during probe to set up the
hardware. At this moment, the number of supported descriptors are also
given. On each request, a descriptor is allocated from the free pool and
filled in with the transfer parameters. Multiple requests can be queued
into the hardware via the OS interface. When client is ready for requests
to be executed, start method is called.
Completions are delivered via callbacks via tasklet.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
It seems that the define has not been with acurate name and
makes confusion while reading the code. The more acurate
name should be BAM_FIFO_SIZE.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Reviewed-by: Andy Gross <andy.gross@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The pipe fifo size register must instruct the bam hw
how many hw descriptors can be pushed to fifo. Currently
we instruct the hw with 32KBytes but wrap the tail in
bam_start_dma in BAM_P_EVNT_REG on 4095 i.e. 32760. This
leads to stalled transactions when the tail wraps.
Fix this by use the correct fifo size in BAM_P_FIFO_SIZES
register i.e. 32K - 8.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Some of the peripherals has bam which is controlled by remote
processor, thus the bam dma driver must avoid register writes
which initialise bam hw block. Those registers are protected
from xPU block and any writes to them will lead to secure
violation and system reboot.
Adding the contolled_remotely flag in bam driver to avoid
not permitted register writes in bam_init function.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Reviewed-by: Andy Gross <andy.gross@linaro.org>
Tested-by: Pramod Gurav <gpramod@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Currently we write BAM_IRQ_CLR register with zero even when no
BAM_IRQ occured. This write has some bad side effects when the
BAM instance is for the crypto engine. In case of crypto engine
some of the BAM registers are xPU protected and they cannot be
controlled by the driver.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Reviewed-by: Andy Gross <andy.gross@linaro.org>
Tested-by: Pramod Gurav <gpramod@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
This is smallish update with minor changes to core and new driver and usual
updates. Nothing super exciting here..
- We have made slave address as physical to enable driver to do the mapping.
- We now expose the maxburst for slave dma as new capability so clients can
know this and program accordingly
- addition of device synchronize callbacks on omap and edma.
- pl330 updates to support DMAFLUSHP for Rockchip platforms.
- Updates and improved sg handling in Xilinx VDMA driver.
- New hidma qualcomm dma driver, though some bits are still in progress
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJW6W4OAAoJEHwUBw8lI4NHIj0P/0UEXOn9Laj1dQ++3RuEHtJH
AvolC574yj/jdvhNNRAu3TBq214VDtVu+OEi6cAwybSMUOT0lbrSEI4a6K6iDIdH
QGfyz2PFDBMnNLqqNfeQulgB6YgoZ/7PXUOz9D+FX4wyM3poTBb9J2JI5okFuuJI
r4jmiZrXTZSmm2NTbG0QxHogoyvMDA59EB8cIgAUrl1rDssPkdvzU7ygW6qc5CMW
33tQFyz6Q74EI9ImPeYUkSf1zzT1va4uRce+3lEmLSvtOWG2pjOOZ1Vw89vtkyal
yX1eH06glVTQwpfV+fgmbjpn72Ftk+G6rqcB4aICSyN2dH7Gf4D+Dqjp1mdEHyFf
Oum5pWNPzJ97HoGLwxd8FEuA3ma3C0nC+nDl+ffNWLmIDGgeqFHSQaNBlf2S6y+a
VtGFJ0EaR//qIpwvPNfpJbkwjrEaEFdSYQcdpGcPPeTeOOpaLGkmJ/2kD2rpGSNC
iPh+G/h7sJYLFyiG7C6GeuWxShzSL+LpZqv0ks5i/QKmSNXWsvVQexAlBW43R385
uQkZSWOlzUwmGlTj9XUI2mUxhI73SgKt+WZ9wrJWvIThBHRwwSIln+72SzQ8d4ys
Smv3DkGt4gCxHmsV+G3nEIBlviECJn2KaaN450D6FVxgQ40yGV5gWAVX4yAWo2De
uMnQMDamjoajgbeanpbM
=wCCJ
-----END PGP SIGNATURE-----
Merge tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dma
Pull dmaengine updates from Vinod Koul:
"This is smallish update with minor changes to core and new driver and
usual updates. Nothing super exciting here..
- We have made slave address as physical to enable driver to do the
mapping.
- We now expose the maxburst for slave dma as new capability so
clients can know this and program accordingly
- addition of device synchronize callbacks on omap and edma.
- pl330 updates to support DMAFLUSHP for Rockchip platforms.
- Updates and improved sg handling in Xilinx VDMA driver.
- New hidma qualcomm dma driver, though some bits are still in
progress"
* tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (40 commits)
dmaengine: IOATDMA: revise channel reset workaround on CB3.3 platforms
dmaengine: add Qualcomm Technologies HIDMA channel driver
dmaengine: add Qualcomm Technologies HIDMA management driver
dmaengine: hidma: Add Device Tree binding
dmaengine: qcom_bam_dma: move to qcom directory
dmaengine: tegra: Move of_device_id table near to its user
dmaengine: xilinx_vdma: Remove unnecessary variable initializations
dmaengine: sirf: use __maybe_unused to hide pm functions
dmaengine: rcar-dmac: clear pertinence number of channels
dmaengine: sh: shdmac: don't open code of_device_get_match_data()
dmaengine: tegra: don't open code of_device_get_match_data()
dmaengine: qcom_bam_dma: Make driver work for BE
dmaengine: sun4i: support module autoloading
dma/mic_x100_dma: IS_ERR() vs PTR_ERR() typo
dmaengine: xilinx_vdma: Use readl_poll_timeout instead of do while loop's
dmaengine: xilinx_vdma: Simplify spin lock handling
dmaengine: xilinx_vdma: Fix issues with non-parking mode
dmaengine: xilinx_vdma: Improve SG engine handling
dmaengine: pl330: fix to support the burst mode
dmaengine: make slave address physical
...
This patch adds support for hidma engine. The driver consists of two
logical blocks. The DMA engine interface and the low-level interface.
The hardware only supports memcpy/memset and this driver only support
memcpy interface. HW and driver doesn't support slave interface.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
The Qualcomm Technologies HIDMA device has been designed to support
virtualization technology. The driver has been divided into two to follow
the hardware design.
1. HIDMA Management driver
2. HIDMA Channel driver
Each HIDMA HW consists of multiple channels. These channels share some set
of common parameters. These parameters are initialized by the management
driver during power up. Same management driver is used for monitoring the
execution of the channels. Management driver can change the performance
behavior dynamically such as bandwidth allocation and prioritization.
The management driver is executed in host context and is the main
management entity for all channels provided by the device.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Creating a QCOM directory for all QCOM DMA source files.
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Reviewed-by: Andy Gross <agross@codeaurora.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
