Cleans up and consolidates all of the per-asic logic.
v2: squash in "drm/amdgpu: fix NULL err for sriov detect" (Chunming)
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
And also provide the level for which we need a PDE.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Remove the superflous .debugfs_init callback and register all files in
amdgpu_device.c in just one function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Handle dynamic offsets correctly in static arrays.
Acked-by: Christian Konig <christian.koenig@amd.com>
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The base offsets of the IP blocks may change across
asics even though the relative register offsets
are the same for an IP. Handle this dynamically.
Acked-by: Christian Konig <christian.koenig@amd.com>
Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This moves and renames the AMDGPU scheduler to a common location in DRM
in order to facilitate re-use by other drivers. This is mostly a straight
forward rename with no code changes.
One notable exception is the function to_drm_sched_fence(), which is no
longer a inline header function to avoid the need to export the
drm_sched_fence_ops_scheduled and drm_sched_fence_ops_finished structures.
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead mark fence as explicit in it's amdgpu_sync_entry.
v2:
Fix use after free bug and add new parameter description.
Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
NULL pointer is because original logic will step into
set_pde_pte() even after the gart.ptr is freed due to
there are twice gart_unbind() on all gart area.
also, there are other minor fixes:
1,since gart_init only create dummy page, the corresponding
gart_fini shouldn't do more like unbinding all GART, this is
unnecessary because in driver fini stage all GART unbinding
had already been done during each IP's SW_FINI (GMC's
SW_FINI is the last one called), so remove the step
for the GART unbinding in gart_fini().
2,gart_fini() is already invoked during each GMC IP's gart_fini
routine,e.g. gmc_vx_0_gart_fini(), so no need to manually
call it during ttm_fini().
3,amdgpu_gem_force_release() should be put ahead of
amdgpu_vm_manager_fini()
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Try to resize BAR0 to let CPU access all of VRAM.
v2: rebased, style cleanups, disable mem decode before resize,
handle gmc_v9 as well, round size up to power of two.
v3: handle gmc_v6 as well, release and reassign all BARs in the driver.
v4: rename new function to amdgpu_device_resize_fb_bar,
reenable mem decoding only if all resources are assigned.
v5: reorder resource release, return -ENODEV instead of BUG_ON().
v6: squash in rebase fix
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The GTT manager handles the GART address space anyway, so it is
completely pointless to keep the same information around twice.
v2: rebased
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
since now gpu reset is unified with gpu_recover
for both bare-metal and SR-IOV:
1)rename in_sriov_reset to in_gpu_reset
2)move lock_reset from adev->virt to adev
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
1,new imple names amdgpu_gpu_recover which gives more hint
on what it does compared with gpu_reset
2,gpu_recover unify bare-metal and SR-IOV, only the asic reset
part is implemented differently
3,gpu_recover will increase hang job karma and mark its entity/context
as guilty if exceeds limit
V2:
4,in scheduler main routine the job from guilty context will be immedialy
fake signaled after it poped from queue and its fence be set with
"-ECANCELED" error
5,in scheduler recovery routine all jobs from the guilty entity would be
dropped
6,in run_job() routine the real IB submission would be skipped if @skip parameter
equales true or there was VRAM lost occured.
V3:
7,replace deprecated gpu reset, use new gpu recover
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
move powerplay and amdgpu shared structures
and definitions to kgd_pp_interface.h. This
is the interface between the base driver
and powerplay.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Clean up the interface.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
reset_counter marks the reset counter number once the context
is created, shouldn't be changed due to query.
To keep U/K interface on the ctx_query and keep ctx's reset_counter
logic compatible with GPU RESET feature, now use another var named
"reset_counter_query" to replace the original checked & updated in
amdgpu_ctx_query.
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
this way the real interested guilty is connected to entity->guilty
pointer, and we can use entity->pointer later in gpu recovery procedure
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Chunming Zhou <David1.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This matches the corresponding UAPI fields. Treating the ring index as
signed could result in accessing random unrelated memory if the MSB was
set.
Fixes: effd924d2f ("drm/amdgpu: untie user ring ids from kernel ring
ids v6")
Cc: stable@vger.kernel.org
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
With the enablement of VCN Dec and Enc from user space, User space queries
kernel for the IP information, if HW has UVD/VCE, the info comes from these
IP blocks, but this could end up mis-interpret for VCN when they are in the
union, the other way same when HW with VCN block.
Signed-off-by: Leo Liu <leo.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fixes: 95d0906f85 ("drm/amdgpu: add initial vcn support and decode tests")
Cc: stable@vger.kernel.org
Reviewed-and-Tested-by: Michel Dänzer <michel.daenzer@amd.com>
-----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJaDlaqAAoJEAx081l5xIa+VB8P/3tl1kg6gONXBHA89t4aoyaM
uKyLy2D8//9RCPupnI2nOablbcdXzmZYE5gsLGHcN5G/cf9qHksslqo6P/8cjfIC
lOz+2AxzFGTP9s6M0jyE7l4Dlk53Chd+7yOTJfm322BUuAZW7nSjWGglkO6rW6RR
JRyNwIoRLX62nAkD769R9QTh8sh2P7pWvXKUSRtMQVWRRI0fICvUFuqyBbEFjJZN
4GGkqM5bA6GU+z1W91iqkXoPWz34Zejch7cLBM5pXiZsgXOuzl4V/RwxdKZlWVrf
9oA9357yKvvvb1bkNRgjNqLLHdOxQUomv1k2RxCbvX2xUecOCTKXKb4/X+AurZEI
ENfSejTbzj+mP18CI1IsvsQolkighP1xxqjH3zmSu+bS0ivWBywbpDUVN969qKrV
9kHigMwxxX5YCWGoLswhZ+6OsPm5R2FRKg10QVQAlARjye4Q7ssP+l+KRRP8rvkc
D4rZiLBMuIDersRhW3ylEym8gXqSO2BoBJZS3+ECSzweIhvwziNgY0q6lpFxfzJa
fzjW/mfK/uucEshoZrxJVRAEiWwtULvi1KVnTpQ/lm254maj4mOy6atqs7rmdAKK
Jetfg+Z0Fb+805fHeS2dk/E855qwmTCsBf+TA4hGrxoW3EHB3yNLH1j4MSUxK8es
6SpuEv7hzeyCiK0QJcSH
=0JS4
-----END PGP SIGNATURE-----
Merge tag 'drm-for-v4.15-amd-dc' of git://people.freedesktop.org/~airlied/linux
Pull amdgpu DC display code for Vega from Dave Airlie:
"This is the pull request for the AMD DC (display code) layer which is
a requirement to program the display engines on the new Vega and Raven
based GPUs. It also contains support for all amdgpu supported GPUs
(CIK, VI, Polaris), which has to be enabled. It is also a kms atomic
modesetting compatible driver (unlike the current in-tree display
code).
I've kept it separate from drm-next because it may have some things
that cause you to reject it.
Background story:
AMD have an internal team creating a shared OS codebase for display at
hw bring up time using information from their hardware teams. This
process doesn't lead to the most Linux friendly/looking code but we
have worked together on cleaning a lot of it up and dealing with
sparse/smatch/checkpatch, and having their team internally adhere to
Linux coding standards.
This tree is a complete history rebased since they started opening it,
we decided not to squash it down as the history may have some value.
Some of the commits therefore might not reach kernel standards, and we
are steadily training people in AMD to better write commit msgs.
There is a major bunch of generated bandwidth calculation and
verification code that comes from their hardware team. On Vega and
before this is float calculations, on Raven (DCN10) this is double
based. They do the required things to do FP in the kernel, and I could
understand this might raise some issues. Rewriting the bandwidth would
be a major undertaken in reverification, it's non-trivial to work out
if a display can handle the complete set of mode information thrown at
it.
Future story:
There is a TODO list with this, and it address most of the remaining
things that would be nice to refine/remove. The DCN10 code is still
under development internally and they push out a lot of patches quite
regularly and are supporting this code base with their display team. I
think we've reached the point where keeping it out of tree is going to
motivate distributions to start carrying the code, so I'd prefer we
get it in tree. I think this code is slightly better than STAGING
quality but not massively so, I'd really like to see that float/double
magic gone and fixed point used, but AMD don't seem to think the
accuracy and revalidation of the code is worth the effort"
* tag 'drm-for-v4.15-amd-dc' of git://people.freedesktop.org/~airlied/linux: (1110 commits)
drm/amd/display: fix MST link training fail division by 0
drm/amd/display: Fix formatting for null pointer dereference fix
drm/amd/display: Remove dangling planes on dc commit state
drm/amd/display: add flip_immediate to commit update for stream
drm/amd/display: Miss register MST encoder cbs
drm/amd/display: Fix warnings on S3 resume
drm/amd/display: use num_timing_generator instead of pipe_count
drm/amd/display: use configurable FBC option in dm
drm/amd/display: fix AZ clock not enabled before program AZ endpoint
amdgpu/dm: Don't use DRM_ERROR in amdgpu_dm_atomic_check
amd/display: Fix potential null dereference in dce_calcs.c
amdgpu/dm: Remove unused forward declaration
drm/amdgpu: Remove unused dc_stream from amdgpu_crtc
amdgpu/dc: Fix double unlock in amdgpu_dm_commit_planes
amdgpu/dc: Fix missing null checks in amdgpu_dm.c
amdgpu/dc: Fix potential null dereferences in amdgpu_dm.c
amdgpu/dc: fix more indentation warnings
amdgpu/dc: handle allocation failures in dc_commit_planes_to_stream.
amdgpu/dc: fix indentation warning from smatch.
amdgpu/dc: fix non-ansi function decls.
...
Register accessing is performed when IRQ is disabled. Never sleep in
this function.
Known issue: dead sleep in many use cases of index/data registers.
v2:
- wrap polling fence functions.
- don't trigger IRQ for polling in case of wrongly fence signal.
v3:
- handle wrap round gracefully.
- add comments for polling function
v4:
- don't return negative timeout confused with error code
Signed-off-by: pding <Pixel.Ding@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
with current WB usage we only use 57 slots, so 512
is extreamly sufficient, and reduce to 512 can
make WB fit into one page.
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of per device track the VRAM lost per context and return ECANCELED
instead of ENODEV.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of reading the current counter from fpriv.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Helps avoiding deadlock during GPU reset.
Added mutex to amdgpu_ctx to preserve order of fences on a ring.
v2:
Put waiting logic in a function in a seperate function in amdgpu_ctx.c
Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Introduce amdgpu_ctx_priority_override(). A mechanism to override a
context's priority.
An override can be terminated by setting the override to
AMD_SCHED_PRIORITY_UNSET.
v2: change refcounted interface for a direct set
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Programming CP_HQD_QUEUE_PRIORITY enables a queue to take priority over
other queues on the same pipe. Multiple queues on a pipe are timesliced
so this gives us full precedence over other queues.
Programming CP_HQD_PIPE_PRIORITY changes the SPI_ARB_PRIORITY of the
wave as follows:
0x2: CS_H
0x1: CS_M
0x0: CS_L
The SPI block will then dispatch work according to the policy set by
SPI_ARB_PRIORITY. In the current policy CS_H is higher priority than
gfx.
In order to prevent getting stuck in loops of resources bouncing between
GFX and high priority compute and introducing further latency, we
statically reserve a portion of the pipe.
v2: fix srbm_select to ring->queue and use ring->funcs->type
v3: use AMD_SCHED_PRIORITY_* instead of AMDGPU_CTX_PRIORITY_*
v4: switch int to enum amd_sched_priority
v5: corresponding changes for srbm_lock
v6: change CU reservation to PIPE_PERCENT allocation
v7: use kiq instead of MMIO
v8: back to MMIO, and make the implementation sleep safe.
v9: corresponding changes for splitting HIGH into _HW/_SW
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
SR-IOV need to reserve a piece of shared VRAM at the exact place
to exchange data betweem PF and VF. The start address and size of
the shared mem are passed to guest through VBIOS structure
VRAM_UsageByFirmware.
VRAM_UsageByFirmware is a general feature in VBIOS, it indicates
that VBIOS need to reserve a piece of memory on the VRAM.
Because the mem address is specified. Reserve it early in
amdgpu_ttm_init to make sure that it can monoplize the space.
Signed-off-by: Horace Chen <horace.chen@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Initial pull request for DC support. We've completed a substantial amount of
the cleanup and restructuring in our TODO. There are a few additional
cleanups that we are continuing to work on, but I don't think there are any
showstoppers remaining. We've tried to maintain most of the history for bisect
purposes. Harry made sure all the commits build. We've enabled DC for vega10
and Raven. Pre-vega10 parts can be enabled via module parameter (amdgpu.dc=1),
but are not enabled by default at this point until we get further testing
upstream.
This code provides atomic modesetting support for DCE8 (CIK), DCE10 (Tonga,
Fiji), DCE11 (CZ, ST, Polaris), DCE12 (vega10), and DCN1 (RV) including
HDMI and DP audio, DP MST, and many other advanced display features.
+
Latest cleanups for DC from you and Harry. Note that there is some
flickering on some older asics with this branch due to a regression in powerplay
that has already been fixed and will be included in my next non-DC pull request
next week.
* 'drm-next-4.15-dc' of git://people.freedesktop.org/~agd5f/linux: (897 commits)
amdgpu/dc: use kref for dc_state.
amdgpu/dc: convert dc_sink to kref.
amdgpu/dc: convert dc_stream_state to kref.
amdgpu/dc: use kref for dc_plane_state.
amdgpu/dc: convert dc_gamma to kref reference counting.
amdgpu/dc: convert dc_transfer to use a kref.
amdgpu/dc: kill a bunch of dead code.
amdgpu/dc: set a bunch of functions to static.
amdgpu/dc: kill some deadcode in dc core.
amdgpu/dc: fix indentation on a couple of returns.
amdgpu/dm: don't use after free.
amdgpu/dc: kfree already checks for NULL.
amdgpu/dc: fix a bunch of misc whitespace.
amdgpu/dc: drop hw_sequencer_types.h
amdgpu/dc: drop dce110_types.h
amdgpu/dc: use kernel ilog2 for log_2.
amdgpu/dc: don't memset after kzalloc.
amdgpu/dc: inline dal grph object id functions.
amdgpu/dc: inline dml_round_to_multiple
amdgpu/dc: rename bios get_image symbol to something more searchable.
...
for being able to convert an amdgpu fence into one of the handles.
Mesa will use this.
Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu not care powerplay or dpm is enabled.
just check ip functions and pp functions
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Useful for testing the effects of multipipe compute without recompiling.
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
We want to make DC less chatty but still allow bug reporters to
provide more detailed logs.
Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Supported DCE versions: 8.0, 10.0, 11.0, 11.2
v2: rebase against 4.11
Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
v2: drop hdp invalidate/flush.
v3: honor pgoff during prime mmap. Add a barrier after cpu access.
v4: drop begin/end_cpu_access() for now, revisit later.
Signed-off-by: Samuel Li <Samuel.Li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use it to replace the hard coded value in amdgpu_vm_bo_update_mapping().
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When max_bytes is not 8 bytes aligned and bo size is larger than
max_bytes, the last 8 bytes in a ttm node may be left unchanged.
For example, on pre SDMA 4.0, max_bytes = 0x1fffff, and the bo size
is 0x200000, the problem will happen.
In order to fix the problem, we separately store the max nums of
PTEs/PDEs a single operation can set in amdgpu_vm_pte_funcs
structure, rather than inferring it from bytes limit of SDMA
constant fill, i.e. fill_max_bytes.
Together with the fix, we replace the hard code value "10" in
amdgpu_vm_bo_update_mapping() with the corresponding values from
structure amdgpu_vm_pte_funcs.
Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
GPU reset will require all hw doing hw_init thus
ucode_init_bo will be invoked again, which lead to
memory leak
skip the fw_buf allocation during sriov gpu reset to avoid
memory leak.
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
otherwise a gpu hang will make application couldn't be killed
under timedout=0 mode
v2:
Fix memoryleak job/job->s_fence issue
unlock mn
remove the ERROR msg after waiting being interrupted
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
currently in_reset is only used in sriov gpu reset, and it
will be used for other non-gfx hw component later, like
PSP, so move it from gfx to adev and rename to in_sriov_reset
make more sense.
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
fix checkpatch.pl WARNING:
Prefer 'unsigned int' to bare use of 'unsigned'
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
To filter out high-frequency interrupts that can be safely ignored.
v2: squash in trivial typo fix for si (Alex)
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Just some cleanup.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Just some cleanup.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This is quite controversial because it adds another lock which is held during
page table updates, but I don't see much other option.
v2: allow multiple updates to be in flight at the same time
v3: simplify the patch, take the read side only once
v4: correctly fix rebase conflict
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When we need to find the mapping we need sysvm access anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead take the callback lock during the final parts of CS.
This should solve the last remaining locking order problems with BO reservations.
v2: rebase, make dummy functions static inline
v3: add one more missing inline and comments
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of moving them in the MMU notifier move them during CS.
v2: still mark pages as accessed/dirty
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> (v1)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead use a counter to figure out if we need to set new pages or not.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Move calling put_page into the unpopulate callback. Otherwise we mess up the pages
reference count when it is unbound multiple times.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add the IOCTL interface so that applications can allocate per VM BOs.
Still WIP since not all corner cases are tested yet, but this reduces average
CS overhead for 10K BOs from 21ms down to 48us.
v2: add some extra checks, remove the WIP tag
v3: rename new flag to AMDGPU_GEM_CREATE_VM_ALWAYS_VALID
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
