Commit Graph

148 Commits

Author SHA1 Message Date
Christian König
0fc8683e56 drm/amdgpu: allocate GTT space for shadow VM page tables
We need to access those with the system domain.

Fixes fallout from only allocating GTT space on demand.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-09-28 16:16:21 -04:00
Christian König
d7a4ac667e drm/amdgpu: fix addr handling in amdgpu_vm_bo_update_mapping
Otherwise we will look at the wrong place in the IB when GART
mappings are split into smaller updates.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Tom StDenis <Tom.StDenis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-09-27 13:00:49 -04:00
Christian König
c855e25090 drm/amdgpu: bind GTT on demand
We don't really need the GTT table any more most of the time. So bind it
only on demand.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-09-14 15:10:30 -04:00
Christian König
99e124f402 drm/amdgpu: cleanup amdgpu_vm_bo_update params
Make it more obvious what we are doing here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-22 13:47:24 -04:00
Chunming Zhou
4c7e885506 drm/amdgpu: update pt shadow while updating pt V2
V2:
move shadow parameter to amdgpu_pte_update_params.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: minutemaidpark@hotmail.com
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-22 13:47:20 -04:00
Chunming Zhou
6557e3d294 drm/amdgpu: update pd shadow while updating pd V2
V2:
Checking if shadow is valid.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-22 13:47:19 -04:00
Christian König
afef8b8f99 drm/amdgpu: add function pointer to the pte_update_params
Remember what function to call while planning the commands instead
of figuring it our later on.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:44:45 -04:00
Christian König
96105e5375 drm/amdgpu: stop splitting PTE commands into smaller ones
It doesn't make much sense to create bigger commands first which we then need
to split into smaller one again. Just make sure the commands we create aren't
to big in the first place.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:44:34 -04:00
Christian König
dc157c6daa drm/amdgpu: remove AMDGPU_VM_NO_FLUSH define
Not used any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:44:20 -04:00
Christian König
de9ea7bd36 drm/amdgpu: cleanup the write_pte implementations
We don't need the gart mapping handling here any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:44:10 -04:00
Christian König
b7fc2cbd5e drm/amdgpu: remove pages_addr handling from the VM code
Not needed any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:43:59 -04:00
Christian König
b0456f9306 drm/amdgpu: write PTEs directly into the IB.
Write the PTEs at the end of the IB instead of directly into the SDMA commands.
This can save quite some CPU cycles building the entries.

This doesn't change the DW estimation because PTEs where embedded into the IB
before as well. It just moves them to the end of the IB.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:43:44 -04:00
Chunming Zhou
1baa439fb2 drm/amdgpu: allocate shadow for pd/pt bo V2
The pd/pt shadow bo will be used to backup page table, when gpu reset
happens, we can restore the page table by them.
V2:
Free shadow bo.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-16 10:42:53 -04:00
Christian König
e2b84e4be3 drm/amdgpu: use more than 64KB fragment size if possible
We align to 64KB, but when userspace aligns even more we can easily use more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-10 14:06:08 -04:00
Christian König
92696dd52e drm/amdgpu: flip frag_ptes and update_pts
We can add the fragment params before we split the update for the page tables.
That should save a few CPU cycles for larger updates.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-10 14:05:58 -04:00
Christian König
27c5f36fe1 drm/amdgpu: add adev to the pte_update_params
No need to carry that forward as a separate parameter.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-10 14:05:49 -04:00
Christian König
29efc4f5df drm/amdgpu: rename amdgpu_vm_update_params
Well those are actually page table entry parameters.
This also makes the variable names used a bit shorter.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-10 14:05:39 -04:00
Christian König
1303c73c61 drm/amdgpu: cleanup VM fragment defines
We can actually do way more than just the 64KB we currently used as default.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-08-10 14:05:30 -04:00
Christian König
0c0fdf1439 drm/amdgpu: trace need_flush in grab_vm as well
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-14 16:46:08 -04:00
Christian König
090b767e97 drm/amdgpu: check flush fence context instead of same ring v2
Otherwise we can run into the following situation:

1. Process A grabs ID 1 for ring 0.
2. Process B grabs ID 1 for ring 0.
3. Process A grabs ID 1 for ring 1.
4. Process A tries to reuse ID1 for ring 0 but things he doesn't need to flush.

v2: check the context of the flush fence instead of messing with the owner field.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-14 16:46:07 -04:00
Chunming Zhou
aa1c890008 drm/amdgpu: recovery hw jobs when gpu reset V3
V3: directly use pd_addr.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 15:06:17 -04:00
Chunming Zhou
192b7dcbb5 drm/amdgpu: abstract amdgpu_vm_is_gpu_reset
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 15:06:16 -04:00
Chunming Zhou
fd53be302f drm/amdgpu: add a bool to specify if needing vm flush V2
which avoids job->vm_pd_addr be changed.

V2: pass job structure to amdgpu_vm_grab_id and amdgpu_vm_flush directly.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 15:06:16 -04:00
Chunming Zhou
b46b8a877c drm/amdgpu: must update page table after gpu reset
Record the gpu reset count in vmid to identify if gpu reset happened.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 15:06:03 -04:00
Chunming Zhou
6adb0513a4 drm/amdgpu: must update page table after gpu reset
Record the gpu reset count in vmid to identify if gpu reset happened.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 15:06:02 -04:00
Christian König
5a712a8727 drm/amdgpu: validate VM PTs only on eviction
We don't need to validate them again if the eviction counter didn't changed.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:54:49 -04:00
Alex Deucher
93dcc37d2c drm/amdgpu: disable compute pipeline sync workaround when using fixed fw
No need to stall the pipe when we are using firmware with the
fix.

Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:54:46 -04:00
Christian König
281d144ddd drm/amdgpu: save the PD addr before scheduling the job
When we pipeline evictions the page directory could already be
moving somewhere else when grab_id is called.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:54:41 -04:00
Christian König
3cabaa54e7 drm/amdgpu: sync to buffer moves before VM updates
Otherwise we could update the VM page tables while the move is only scheduled.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:54:38 -04:00
Alex Xie
2171849768 drm/amdgpu: Initialize the variables in a straight-forward way
Initialize the variable in a straight-forward way instead of
hiding the initialization inside the loop. This can also
reduce one function call.

Signed-off-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:27 -04:00
Alex Xie
3a6f8e0c72 drm/amdgpu: Add comment to describe the purpose of one difficult if statement
Use == instead of != in the if statement to make code easier understood

Signed-off-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:26 -04:00
Alex Xie
677131a16d drm/amdgpu: Change some variable names to make code easier understood
Add comment to describe some variables otherwise.

Signed-off-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:26 -04:00
Christian König
3dab83bee1 drm/amdgpu: remove now unnecessary checks
vm_flush() now comes directly after vm_grab_id().

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:24 -04:00
Christian König
1fbb2e9299 drm/amdgpu: use a fence array for VMID management
Just wait for any fence to become available, instead
of waiting for the last entry of the LRU.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:23 -04:00
Christian König
8d76001e00 drm/amdgpu: reuse VMIDs assigned to a VM only if there is also a free one
This fixes a fairness problem with the GPU scheduler. VM having lot of
jobs could previously starve VM with less jobs.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:23 -04:00
Christian König
36fd7c5c3c drm/amdgpu: prefer VMIDs idle on the current ring
Prefer to use a VMIDs which are idle on the ring we want to submit to. This
also removes bubbling idle VMIDs up on the LRU, which is actually not
beneficial.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:22 -04:00
Christian König
3542023896 drm/amdgpu: add optional ring to amdgpu_sync_is_idle
Check if the sync object is idle depending on the ring a submission works with.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:21 -04:00
Harish Kasiviswanathan
f4833c4fd4 drm/amdgpu: Encapsulate some VM table update parameters (v2)
Bundle some VM table parameters into amdgpu_vm_update_params structure,
so that number of function parameters can be reduced. Only structural
change, no logic change.

v2: agd: squash in fix from Harish

Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 13:53:54 -04:00
Christian König
b1c8a81fdd drm/amdgpu: remove define for reserved client ID
Just set it to zero instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:20 -04:00
Christian König
79aa03893d drm/amdgpu: remove owner cleanup v2
The client ID is now unique, so no need to resert the owner fields any more.

v2: remove unused variables as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:19 -04:00
Christian König
0ea54b9b6c drm/amdgpu: make the VMID owner always 64bit
Otherwise we could (in theory) run into problems on 32bit systems.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:18 -04:00
Chunming Zhou
fe70766487 drm/amdgpu: add pipeline sync for compute job
hardware ring is async processed, the job is executed in parallel.
In some case, this will result vm fault, like jobs with different vmids.

This works around a CPC hw issue which will eventually be fixed in fw.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:16 -04:00
Monk Liu
c5637837ba drm/amdgpu: keep vm in job instead of ib (v2)
ib.vm is a legacy way to get vm, after scheduler
implemented vm should be get from job, and all ibs
from one job share the same vm, no need to keep ib.vm
just move vm field to job.

this patch as well add job as paramter to ib_schedule
so it can get vm from job->vm.

v2: agd: sqaush in:
drm/amdgpu: check if ring emit_vm_flush exists in vm flush

No vm flush on engines that don't support VM.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=95195

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:16 -04:00
Chunming Zhou
1f207f81e3 drm/amdgpu: make vmid owner be client_id
Using the pointer is not adequate.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:15 -04:00
Chunming Zhou
031e2983e8 drm/amdgpu: add client id for every vm
This adds a unique id for each vm client so we can
properly track them.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:14 -04:00
Chunming Zhou
444066b915 drm/amdgpu: fix wrong release of vmid owner
The release of the vmid owner was not handled
correctly.  We need to take the lock and walk
the lru list.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:14 -04:00
Chunming Zhou
178d7cb8d5 drm/amdgpu: fix error checking when reuse vmid on same ring
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:22:58 -04:00
Chunming Zhou
68befebee4 drm/amdgpu: only update last_flush when vmid doesn't have other new owner
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:22:55 -04:00
Christian König
794f50b95d drm/amdgpu: reuse VMIDs already assigned to a process
If we don't need to flush we can easily use another VMID
already assigned to the process.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:30 -04:00
Christian König
41d9eb2c5a drm/amdgpu: add a fence after the VM flush
This way we can track when the flush is done.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:24 -04:00