Various bug fixes and improvements that accumulated over the last two
years.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Oded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Dword offsets are what KFD really needs and what amdgpu already returns.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Update the KGD to KFD interface to allow sharing pipes with queue
granularity instead of pipe granularity.
This allows for more interesting pipe/queue splits.
v2: fix overflow check for res.queue_mask
v3: fix shift overflow when setting res.queue_mask
v4: fix comment in is_pipeline_enabled()
v5: clamp res.queue_mask to the first MEC only
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Take ownership of pipe initialization away from KFD.
Note that hpd_eop_gpu_addr was already large enough to accomodate all
pipes.
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
radeon must load only after amdkfd's loading has been completed. If that
is not enforced, then radeon's call into amdkfd's functions will cause a
kernel BUG.
When radeon and amdkfd are built as kernel modules, that rule is
enforced by the kernel's modules loading mechanism. When radeon and
amdkfd are built inside the kernel image, that rule is enforced by
ordering in the drm Makefile (amdkfd before radeon).
Instead of using drm Makefile ordering, we can now use deferred
loading as amdkfd now returns -EPROBE_DEFER in kgd2kfd_init() when it is
not yet loaded.
This patch defers radeon loading by propagating -EPROBE_DEFER to the
kernel's drivers loading infrastructure. That will put radeon into the
pending drivers list (see description in dd.c). Once amdkfd is loaded,
a call to kgd2kfd_init() will return successfully and radeon will be
able to load.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Current dependencies between amdkfd and radeon/amdgpu force the loading
of amdkfd _before_ radeon and/or amdgpu are loaded. When all these kernel
drivers are built as modules, this ordering is enforced by the kernel
built-in mechanism of loading dependent modules.
However, there is no such mechanism in case where all these drivers are
compiled inside the kernel image (not as modules). The current way to
enforce loading of amdkfd before radeon/amdgpu, is to put amdkfd before
radeon/amdgpu in the drm Makefile, but that method is way too fragile.
In addition, there is no kernel mechanism to check whether a kernel
driver that is built inside the kernel image, has already been loaded.
To solve this, this patch adds to kfd_module.c a new static variable,
amdkfd_init_completed, that is set to 1 only when amdkfd's
module initialization function has been completed (successfully).
kgd2kfd_init(), which is the initialization function of the
kgd-->kfd interface, and which is the first function in amdkfd called by
radeon/amdgpu, will return successfully only if amdkfd_init_completed is
equal 1.
If amdkfd_init_completed is not equal to 1, kgd2kfd_init() will
return -EPROBE_DEFER to signal radeon/amdgpu they need to defer
their loading until amdkfd is loaded.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch splits the KGD_ENGINE_SDMA to KGD_ENGINE_SDMA1 and
KGD_ENGINE_SDMA2 to match CZ definitions.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch adds three new interfaces to kfd2kgd interface file of radeon.
The interfaces are:
- Check if a specific VMID has a valid PASID mapping
- Retrieve the PASID which is mapped to a specific VMID
- Issue a VMID invalidation request to the ATC
Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch adds new interface functions to the kfd2kgd interface file. The
new functions allow to perform H/W debugger operations by writing to GPU
registers.
Signed-off-by: Yair Shachar <yair.shachar@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This patch adds a new interface function to the kfd->kgd interface.
The function is kgd_init_interrupts() and its function is to
initialize a pipe's interrupts.
The function currently enables the timestamp interrupt and the
bad opcode interrupt.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
This backmerges 4.0-rc6 due to the recent fixes in rc5/6
- DP link rate refactoring from Ville
- byt/bsw rps tuning from Chris
- kerneldoc for the shrinker code
- more dynamic ppgtt pte work (Michel, Ben, ...)
- vlv dpll code refactoring to prep fro bxt (Imre)
- refactoring the sprite colorkey code (Ville)
- rotated ggtt view support from Tvrtko
- roll out struct drm_atomic_state to prep for atomic update (Ander)
* tag 'drm-intel-next-2015-03-27-merge' of git://anongit.freedesktop.org/drm-intel: (473 commits)
Linux 4.0-rc6
arm64: juno: Fix misleading name of UART reference clock
drm/i915: Update DRIVER_DATE to 20150327
drm/i915: Skip allocating shadow batch for 0-length batches
drm/i915: Handle error to get connector state when staging config
drm/i915: Compare GGTT view structs instead of types
drm/i915: fix simple_return.cocci warnings
drm/i915: Add module param to test the load detect code
drm/i915: Remove usage of encoder->new_crtc from clock computations
drm/i915: Don't look at staged config crtc when changing DRRS state
drm/i915: Convert intel_pipe_will_have_type() to using atomic state
drm/i915: Pass an atomic state to modeset_global_resources() functions
drm/i915: Add dynamic page trace events
drm/i915: Finish gen6/7 dynamic page table allocation
drm/i915: Remove unnecessary gen6_ppgtt_unmap_pages
drm/i915: Fix i915_dma_map_single positive error code
drm/i915: Prevent out of range pt in gen6_for_each_pde
drm/i915: fix definition of the DRM_IOCTL_I915_GET_SPRITE_COLORKEY ioctl
drm/i915: Rip out GET_SPRITE_COLORKEY ioctl
watchdog: imgpdc: Fix default heartbeat
...
The current code can only support one kgd instance. We have to
support multiple kgd instances in one system. i.e two amdgpu or two
radeon or one amdgpu + one radeon or more than two kgd instances.
Signed-off-by: Xihan Zhang <xihan.zhang@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
The current CP firmware can handle Usermode Queues only on MEC1.
To reflect this firmware change, this commit reduces number of compute pipelines
to 4 - 1, from 8 - 1 (the first pipeline is allocated for kgd).
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Cc: stable@vger.kernel.org
This backmerges drm-fixes into drm-next mainly for the amdkfd
stuff, I'm not 100% confident, but it builds and the amdkfd
folks can fix anything up.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Conflicts:
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
This patch fixes the behavior of kgd_init_pipeline in that this function
shouldn't automatically increase the pipe_id argument by 1 right at the start
of the function.
This is because the first_pipe value might not be always 1, and because a
proper interface function should not hide this info inside its implementation.
In other words, the calling function should provide the real pipe_id and not
count on kgd_init_pipeline to "fix" it.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Backmerge Linus tree after rc5 + drm-fixes went in.
There were a few amdkfd conflicts I wanted to avoid,
and Ben requested this for nouveau also.
Conflicts:
drivers/gpu/drm/amd/amdkfd/Makefile
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
drivers/gpu/drm/amd/amdkfd/kfd_priv.h
drivers/gpu/drm/amd/include/kgd_kfd_interface.h
drivers/gpu/drm/i915/intel_runtime_pm.c
drivers/gpu/drm/radeon/radeon_kfd.c
This patch fixes a bug in kgd_set_pasid_vmid_mapping(), where the function
only updated the ATC registers (IOMMU) with the new VMID <--> PASID mapping,
but didn't update the IH (Interrupt) registers.
The bug only occurs when using non-HWS mode. In HWS mode, the CP automatically
does the VMID <--> PASID mapping.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
If timeout happens, kgd_hqd_destroy() just returns -ETIME leaving queue
acquired.
It may cause a deadlock, so the patch proposes to release queue before return.
Found by Linux Driver Verification project (linuxtesting.org).
Signed-off-by: Alexey Khoroshilov <khoroshilov@ispras.ru>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This patch changes the radeon_kfd_init(), which is used to initialize the
interface between radeon and amdkfd, so the interface will be initialized only
if amdkfd was build, either as module or inside the kernel image.
In the modules case, the symbol_request() will be used (same as old code). In
the in-image compilation case, a direct call to kgd2kfd_init() will be done.
For other cases, radeon_kfd_init() will just return false.
This patch is necessary because in case of the following specific
configuration: kernel 32-bit, no modules support, random kernel base and no
hibernation, the symbol_request() doesn't work as expected - it doesn't return
NULL if the symbol doesn't exists - which makes the kernel panic.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
This patch implements a new interface that was added to the kfd-->kgd interface.
The new interface function retrieves the firmware version that is currently
in use by a specific engine. The firmware was uploaded to the engine by the
radeon driver.
v2: Returns the fw version of the specific engine, as passed into the function
by a new parameter
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch moves to radeon the initialization of compute vmid.
That initializations was done in kfd-->kgd interface, but doing it in radeon
as part of radeon's H/W initialization routines is more appropriate.
In addition, this simplifies the kfd-->kgd interface.
The patch removes the function from the interface file and from the interface
declaration file.
The function initializes memory apertures to fixed base/limit address and non
cached memory types.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch adds the implementation of the gtt interface functions.
The allocate function will allocate a single bo, pin and map it to kernel
memory. It will return the gpu address and cpu ptr as arguments.
v2:
The bulk of the allocations in the GART is for MQDs. MQDs represent active
user-mode queues, which are on the current runlist. It is important to
remember that active queues doesn't necessarily mean scheduled/running
queues, especially if there is over-subscription of queues or more than a
single HSA process.
Because the scheduling of the user-mode queues is done by the CP firmware,
amdkfd doesn't have any indication if the queue is scheduled or not. If the
CP will try to schedule a queue, and its MQD is not present, this will
probably stuck the CP permanently, as it will load garbage from the GART
(the address of the MQD is given to the CP inside the runlist packet).
In addition, there are a couple of small allocations which also should
always be pinned - runlist packets (2 packets) and HPDs. runlist packets can
be quite large, depending on number of processes and queues.
This new allocate function represents the short/mid-term solution of limiting
the total memory consumption to around 4MB by default.
The long-term solution is to create a mechanism through which radeon/ttm can
ask amdkfd to clear GART/VRAM memory due to memory pressure.
Then, amdkfd will preempt the running queues and wait until the memory pressure
is over. After that, amdkfd will reschedule the queues.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alexey Skidanov <Alexey.skidanov@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch implements the new SDMA interface functions. It also adds defines
and structures related to SDMA registers.
v2: Removed init_sdma_engines() from interface. Initialization is done in
radeon.
v3:
- Removed unused defines.
- Added SDMA_ prefix to defines that didn't have them.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This patch adds the interface between the radeon driver and the amdkfd driver.
The interface implementation is contained in radeon_kfd.c and radeon_kfd.h.
The interface itself is represented by a pointer to struct
kfd_dev. The pointer is located inside radeon_device structure.
All the register accesses that amdkfd need are done using this interface. This
allows us to avoid direct register accesses in amdkfd proper, while also
avoiding locking between amdkfd and radeon.
The single exception is the doorbells that are used in both of the drivers.
However, because they are located in separate pci bar pages, the danger of
sharing registers between the drivers is minimal.
Having said that, we are planning to move the doorbells as well to radeon.
v3:
Add interface for sa manager init and fini. The init function will allocate a
buffer on system memory and pin it to the GART address space via the radeon sa
manager.
All mappings of buffers to GART address space are done via the radeon sa
manager. The interface of allocate memory will use the radeon sa manager to sub
allocate from the single buffer that was allocated during the init function.
Change lower_32/upper_32 calls to use linux macros
Add documentation for the interface
v4:
Change ptr field type in kgd_mem from uint32_t* to void* to match to type that
is returned by radeon_sa_bo_cpu_addr
v5:
Change format of mqd structure to work with latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime.
Move generic kfd-->kgd interface and other generic kgd definitions to a generic
header file that will be used by AMD's radeon and amdgpu drivers
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
