This patch adds Peripheral Page Request (PPR) failure processing
and reporting.
Bad address or pointer to a system memory block with inappropriate
read/write permission cause such PPR failure during a user queue
processing. PPR request handling is done by IOMMU driver notifying
AMDKFD module on PPR failure.
The process triggering a PPR failure will be notified by
appropriate event or SIGTERM signal will be sent to it.
v3:
- Change all bool fields in struct kfd_memory_exception_failure to
uint32_t
Signed-off-by: Alexey Skidanov <alexey.skidanov@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch adds the events module (kfd_events.c) and the interrupt
handle module for Kaveri (cik_event_interrupt.c).
The patch updates the interrupt_is_wanted(), so that it now calls the
interrupt isr function specific for the device that received the
interrupt. That function(implemented in cik_event_interrupt.c)
returns whether this interrupt is of interest to us or not.
The patch also updates the interrupt_wq(), so that it now calls the
device's specific wq function, which checks the interrupt source
and tries to signal relevant events.
v2:
Increase limit of signal events to 4096 per process
Remove bitfields from struct cik_ih_ring_entry
Rename radeon_kfd_event_mmap to kfd_event_mmap
Add debug prints to allocate_free_slot and allocate_signal_page
Make allocate_event_notification_slot return a correct value
Add warning prints to create_signal_event
Remove error print from IOCTL path
Reformatted debug prints in kfd_event_mmap
Map correct size (as received from mmap) in kfd_event_mmap
v3:
Reduce limit of signal events back to 256 per process
Fix allocation of kernel memory for signal events
Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
- AMDKFD_IOC_CREATE_EVENT:
Creates a new event of a specified type
- AMDKFD_IOC_DESTROY_EVENT:
Destroys an existing event
- AMDKFD_IOC_SET_EVENT:
Signal an existing event
- AMDKFD_IOC_RESET_EVENT:
Reset an existing event
- AMDKFD_IOC_WAIT_EVENTS:
Wait on event(s) until they are signaled
v2:
- Move the limit of the signal events to kfd_ioctl.h so it
can be used by userspace
v3:
- Change all bool fields in struct kfd_memory_exception_failure
to uint32_t
Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch adds the interrupt handling module, kfd_interrupt.c, and its
related members in different data structures to the amdkfd driver.
The amdkfd interrupt module maintains an internal interrupt ring
per amdkfd device. The internal interrupt ring contains interrupts
that needs further handling. The extra handling is deferred to
a later time through a workqueue.
There's no acknowledgment for the interrupts we use. The hardware
simply queues a new interrupt each time without waiting.
The fixed-size internal queue means that it's possible for us to lose
interrupts because we have no back-pressure to the hardware.
However, only interrupts that are "wanted" by amdkfd, are copied into
the amdkfd s/w interrupt ring, in order to minimize the chances
for overflow of the ring.
Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch creates a new structure for asic specific operations, instead
of using the existing structure of operations.
This is done to make the code flow more logic, readable and maintainable.
The change is done only to the device queue manager module at this point.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
kmalloc() returns a void pointer - no need to cast it in
drivers/gpu/drm/amd/amdkfd/kfd_process.c::kfd_process_destroy_delayed()
Signed-off-by: Firo Yang <firogm@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This backmerges 4.0-rc6 due to the recent fixes in rc5/6
- DP link rate refactoring from Ville
- byt/bsw rps tuning from Chris
- kerneldoc for the shrinker code
- more dynamic ppgtt pte work (Michel, Ben, ...)
- vlv dpll code refactoring to prep fro bxt (Imre)
- refactoring the sprite colorkey code (Ville)
- rotated ggtt view support from Tvrtko
- roll out struct drm_atomic_state to prep for atomic update (Ander)
* tag 'drm-intel-next-2015-03-27-merge' of git://anongit.freedesktop.org/drm-intel: (473 commits)
Linux 4.0-rc6
arm64: juno: Fix misleading name of UART reference clock
drm/i915: Update DRIVER_DATE to 20150327
drm/i915: Skip allocating shadow batch for 0-length batches
drm/i915: Handle error to get connector state when staging config
drm/i915: Compare GGTT view structs instead of types
drm/i915: fix simple_return.cocci warnings
drm/i915: Add module param to test the load detect code
drm/i915: Remove usage of encoder->new_crtc from clock computations
drm/i915: Don't look at staged config crtc when changing DRRS state
drm/i915: Convert intel_pipe_will_have_type() to using atomic state
drm/i915: Pass an atomic state to modeset_global_resources() functions
drm/i915: Add dynamic page trace events
drm/i915: Finish gen6/7 dynamic page table allocation
drm/i915: Remove unnecessary gen6_ppgtt_unmap_pages
drm/i915: Fix i915_dma_map_single positive error code
drm/i915: Prevent out of range pt in gen6_for_each_pde
drm/i915: fix definition of the DRM_IOCTL_I915_GET_SPRITE_COLORKEY ioctl
drm/i915: Rip out GET_SPRITE_COLORKEY ioctl
watchdog: imgpdc: Fix default heartbeat
...
The current code can only support one kgd instance. We have to
support multiple kgd instances in one system. i.e two amdgpu or two
radeon or one amdgpu + one radeon or more than two kgd instances.
Signed-off-by: Xihan Zhang <xihan.zhang@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Convert the timestamping in the amdkfd driver to use a timespec64 and 64bit
time accessors.
Although the existing code is completely safe beyond y2038 because it deals
with monotonic time, this patch is still needed in order to kill off all uses
of struct timespec.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
fence_wait_timeout() is an exported kernel symbol, so we should rename our
local function to something different.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch fixes the SDMA queue initialization, when running in non-HWS mode.
The first fix is to move the initialization of SDMA VM parameters before the
initialization of the SDMA MQD.
The second fix is to load the MQD to an HQD after the initialization of the MQD.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch adds a missing destruction of mqd, when destroying a kernel queue.
Without the destruction, there is a memory leakage when repeatedly creating and
destroying kernel queues.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
get_pipes_num() calls BUG_ON so we can't set it as inline because it produces a
warning as BUG_ON() uses static variables when it is expanded.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This patch fixes a bug in the initialization of the pipelines. The
init_pipelines() function was called with a constant value of 0 in the
first_pipe argument. This is an error because amdkfd doesn't handle pipe 0.
The correct way is to pass the value that get_first_pipe() returns as the
argument for first_pipe.
This bug appeared in 3.19 (first version with amdkfd) and it causes around 15%
drop in CPU performance of Kaveri (A10-7850).
v2: Don't set get_first_pipe() as inline because it calls BUG_ON()
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Cc: stable@vger.kernel.org
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Pull drm updates from Dave Airlie:
"This is the main drm pull, it has a shared branch with some alsa
crossover but everything should be acked by relevant people.
New drivers:
- ATMEL HLCDC driver
- designware HDMI core support (used in multiple SoCs).
core:
- lots more atomic modesetting work, properties and atomic ioctl
(hidden under option)
- bridge rework allows support for Samsung exynos chromebooks to
work finally.
- some more panels supported
i915:
- atomic plane update support
- DSI uses shared DSI infrastructure
- Skylake basic support is all merged now
- component framework used for i915/snd-hda interactions
- write-combine cpu memory mappings
- engine init code refactored
- full ppgtt enabled where execlists are enabled.
- cherryview rps/gpu turbo and pipe CRC support.
radeon:
- indirect draw support for evergreen/cayman
- SMC and manual fan control for SI/CI
- Displayport audio support
amdkfd:
- SDMA usermode queue support
- replace suballocator usage with more suitable one
- rework for allowing interfacing to more than radeon
nouveau:
- major renaming in prep for later splitting work
- merge arm platform driver into nouveau
- GK20A reclocking support
msm:
- conversion to atomic modesetting
- YUV support for mdp4/5
- eDP support
- hw cursor for mdp5
tegra:
- conversion to atomic modesetting
- better suspend/resume support for child devices
rcar-du:
- interlaced support
imx:
- move to using dw_hdmi shared support
- mode_fixup support
sti:
- DVO support
- HDMI infoframe support
exynos:
- refactoring and cleanup, removed lots of internal unnecessary
abstraction
- exynos7 DECON display controller support
Along with the usual bunch of fixes, cleanups etc"
* 'drm-next' of git://people.freedesktop.org/~airlied/linux: (724 commits)
drm/radeon: fix voltage setup on hawaii
drm/radeon/dp: Set EDP_CONFIGURATION_SET for bridge chips if necessary
drm/radeon: only enable kv/kb dpm interrupts once v3
drm/radeon: workaround for CP HW bug on CIK
drm/radeon: Don't try to enable write-combining without PAT
drm/radeon: use 0-255 rather than 0-100 for pwm fan range
drm/i915: Clamp efficient frequency to valid range
drm/i915: Really ignore long HPD pulses on eDP
drm/exynos: Add DECON driver
drm/i915: Correct the base value while updating LP_OUTPUT_HOLD in MIPI_PORT_CTRL
drm/i915: Insert a command barrier on BLT/BSD cache flushes
drm/i915: Drop vblank wait from intel_dp_link_down
drm/exynos: fix NULL pointer reference
drm/exynos: remove exynos_plane_dpms
drm/exynos: remove mode property of exynos crtc
drm/exynos: Remove exynos_plane_dpms() call with no effect
drm/i915: Squelch overzealous uncore reset WARN_ON
drm/i915: Take runtime pm reference on hangcheck_info
drm/i915: Correct the IOSF Dev_FN field for IOSF transfers
drm/exynos: fix DMA_ATTR_NO_KERNEL_MAPPING usage
...
This patch changes a BUG_ON() statement to pr_debug, in case the user tries to
update a non-existing queue.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch fixes a bug where the first_pipe index passed into init_pipelines()
was a #define instead of the value that is passed into amdkfd by radeon
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
This patch fixes a bug when calling to init_pipeline() interface.
The index that was passed to that function didn't take into account the
first_pipe value, which represents the first pipe index that is under amdkfd's
responsibility.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
CP microcode uses undocumented bits in this register to record queue
state information. The KFD zeroes these bits in update_mqd, when invoked
through the UPDATE_QUEUE ioctl, causing incoherent state when the ioctl
is used to successively unmap and map a queue.
Since the queue type cannot be changed in this path, move the MQD write
to init_mqd.
Signed-off-by: Jay Cornwall <jay.cornwall@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
dqm->queue_count tracks queues in the active state only. In a few
places this count is modified unconditionally, leading to an incorrect
value when the UPDATE_QUEUE ioctl is used to make a queue inactive.
Signed-off-by: Jay Cornwall <jay.cornwall@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This backmerges drm-fixes into drm-next mainly for the amdkfd
stuff, I'm not 100% confident, but it builds and the amdkfd
folks can fix anything up.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Conflicts:
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
This patch handles a case where amdkfd tries to destroy a queue but the queue
type is invalid.
This case occurs in non-HWS path.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Backmerge Linus tree after rc5 + drm-fixes went in.
There were a few amdkfd conflicts I wanted to avoid,
and Ben requested this for nouveau also.
Conflicts:
drivers/gpu/drm/amd/amdkfd/Makefile
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
drivers/gpu/drm/amd/amdkfd/kfd_priv.h
drivers/gpu/drm/amd/include/kgd_kfd_interface.h
drivers/gpu/drm/i915/intel_runtime_pm.c
drivers/gpu/drm/radeon/radeon_kfd.c
This patch replaces the two current amdkfd module parameters with a new one.
The current parameters that are being replaced are:
- Maximum number of HSA processes
- Maximum number of queues per process
The new parameter that replaces them is called "Maximum queues per device"
This replacement achieves two goals:
- Allows the user to have as many HSA processes as it wants (until
a maximum of 512 HSA processes in Kaveri).
- Removes the limitation the user had on maximum number of queues per HSA
process. E.g. the user can now have processes which only have one queue and
other processes which have hundreds of queues, while before the user
couldn't have more than 128 queues per process (as default).
The default value of the new parameter is 4096 (32 * 128, which were the
defaults of the old parameters). There is almost no additional GART memory
required for the default case. As a reminder, this amount of queues requires a
little bit below 4MB of GART memory.
v2:
In addition, This patch defines a new counter for queues accounting in the DQM
structure. This is done because the current counter only counts active queues
which allows the user to create more queues than the
max_num_of_queues_per_device module parameter allows.
However, we need the current counter for the runlist packet build process, so
the solution is to have a dedicated counter for this accounting.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
If the first queue created was failed on DQM then PQM should
unregister the process from DQM.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This patch completely removes the sync_with_hw() because it was broken and
actually there is no point of using it.
This function was used to:
- Make sure that the submitted packet to the HIQ (which is a kernel queue) was
read by the CP. However, it was discovered that the method this function used
to do that (checking wptr == rptr) is not consistent with how the actual CP
firmware works in all cases.
- Make sure that the queue is empty before issuing the next packet. To achieve
that, the function blocked amdkfd from continuing until the recently
submitted packet was consumed. However, the acquire_packet_buffer() already
checks if there is enough room for a new packet so calling sync_with_hw() is
redundant.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
In order not to occupy the current core and thus prevent the core from
servicing IOMMU PPR requests, this patch replaces the call in DQM to
cpu_relax() with a call to schedule().
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
This patch fixes a minor bug in allocate_hqd(), where the loop run from the
next-to-allocate pipe until the number of pipes.
This is wrong because we need to consider the possibility where
next-to-allocate pipe is not 0, and thus, the for-loop only checks part of the
pipes and doesn't wrap-around, as it supposed to do.
Therefore, we add another counting variable to make sure we go over all the
pipes, regardless of where we start to look at the first iteration of the loop.
This bug only affected non-HWS mode. In HWS mode, the CP fw is responsible for
allocating the HQD.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
The work queue couldn't reliably prevent the SW ring buffer from
overflowing, so dmesg was spammed by
kfd kfd: Interrupt ring overflow, dropping interrupt.
messages when running e.g. the Atlantis Substance demo from
https://wiki.unrealengine.com/Linux_Demos on Kaveri.
Since the SW ring buffer doesn't actually do anything at this point, just
remove it for now. When actual interrupt processing code is added to
amdkfd, it should try to do things immediately and only defer to work
queues when necessary.
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This patch changes kfd_ioctl() to be very similar to drm_ioctl().
The patch defines an array of amdkfd_ioctls, which maps IOCTL definition to the
ioctl function.
The kfd_ioctl() uses that mapping to call the appropriate ioctl function,
through a function pointer.
This patch also declares a new typedef for the ioctl function pointer.
v2: Renamed KFD_COMMAND_(START|END) to AMDKFD_...
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
This patch reformats the ioctl definitions in kfd_ioctl.h to be similar to the
drm ioctls definition style.
v2: Renamed KFD_COMMAND_(START|END) to AMDKFD_...
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
This patch moves the copy_to_user() and copy_from_user() calls from the
different ioctl functions in amdkfd to the general kfd_ioctl() function, as
this is a common code for all ioctls.
This was done according to example taken from drm_ioctl.c
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
This patch fixes a bug where deallocate_vmid() didn't actually unmap the
VMID<-->PASID mapping (in the registers).
That can cause undefined behavior.
This bug only occurs in non-HWS mode.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
This patch fixes a bug in DQM, where the MQD of a newly created compute queue
is not loaded to an HQD slot. As a result, the CP never reads packets from this
queue.
This bug happens only in non-HWS (hardware scheduling) mode. In HWS mode, the
CP is responsible of loading MQDs to HQDs slots.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Commit "amdkfd: use sizeof(long) granularity for the pasid bitmask" calculated
the number of longs it will need, but ended up allocating that number of
bytes rather than longs.
Fix that silly error and allocate the amount of data really required.
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
