Commit Graph

166 Commits

Author SHA1 Message Date
Christian König
3dab83bee1 drm/amdgpu: remove now unnecessary checks
vm_flush() now comes directly after vm_grab_id().

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:24 -04:00
Christian König
1fbb2e9299 drm/amdgpu: use a fence array for VMID management
Just wait for any fence to become available, instead
of waiting for the last entry of the LRU.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:23 -04:00
Christian König
8d76001e00 drm/amdgpu: reuse VMIDs assigned to a VM only if there is also a free one
This fixes a fairness problem with the GPU scheduler. VM having lot of
jobs could previously starve VM with less jobs.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:23 -04:00
Christian König
36fd7c5c3c drm/amdgpu: prefer VMIDs idle on the current ring
Prefer to use a VMIDs which are idle on the ring we want to submit to. This
also removes bubbling idle VMIDs up on the LRU, which is actually not
beneficial.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:22 -04:00
Christian König
3542023896 drm/amdgpu: add optional ring to amdgpu_sync_is_idle
Check if the sync object is idle depending on the ring a submission works with.

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-07-07 14:51:21 -04:00
Harish Kasiviswanathan
f4833c4fd4 drm/amdgpu: Encapsulate some VM table update parameters (v2)
Bundle some VM table parameters into amdgpu_vm_update_params structure,
so that number of function parameters can be reduced. Only structural
change, no logic change.

v2: agd: squash in fix from Harish

Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 13:53:54 -04:00
Christian König
b1c8a81fdd drm/amdgpu: remove define for reserved client ID
Just set it to zero instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:20 -04:00
Christian König
79aa03893d drm/amdgpu: remove owner cleanup v2
The client ID is now unique, so no need to resert the owner fields any more.

v2: remove unused variables as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:19 -04:00
Christian König
0ea54b9b6c drm/amdgpu: make the VMID owner always 64bit
Otherwise we could (in theory) run into problems on 32bit systems.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:18 -04:00
Chunming Zhou
fe70766487 drm/amdgpu: add pipeline sync for compute job
hardware ring is async processed, the job is executed in parallel.
In some case, this will result vm fault, like jobs with different vmids.

This works around a CPC hw issue which will eventually be fixed in fw.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:16 -04:00
Monk Liu
c5637837ba drm/amdgpu: keep vm in job instead of ib (v2)
ib.vm is a legacy way to get vm, after scheduler
implemented vm should be get from job, and all ibs
from one job share the same vm, no need to keep ib.vm
just move vm field to job.

this patch as well add job as paramter to ib_schedule
so it can get vm from job->vm.

v2: agd: sqaush in:
drm/amdgpu: check if ring emit_vm_flush exists in vm flush

No vm flush on engines that don't support VM.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=95195

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:16 -04:00
Chunming Zhou
1f207f81e3 drm/amdgpu: make vmid owner be client_id
Using the pointer is not adequate.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:15 -04:00
Chunming Zhou
031e2983e8 drm/amdgpu: add client id for every vm
This adds a unique id for each vm client so we can
properly track them.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:14 -04:00
Chunming Zhou
444066b915 drm/amdgpu: fix wrong release of vmid owner
The release of the vmid owner was not handled
correctly.  We need to take the lock and walk
the lru list.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-11 12:31:14 -04:00
Chunming Zhou
178d7cb8d5 drm/amdgpu: fix error checking when reuse vmid on same ring
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:22:58 -04:00
Chunming Zhou
68befebee4 drm/amdgpu: only update last_flush when vmid doesn't have other new owner
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:22:55 -04:00
Christian König
794f50b95d drm/amdgpu: reuse VMIDs already assigned to a process
If we don't need to flush we can easily use another VMID
already assigned to the process.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:30 -04:00
Christian König
41d9eb2c5a drm/amdgpu: add a fence after the VM flush
This way we can track when the flush is done.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:24 -04:00
Christian König
832a902f94 drm/amdgpu: use a sync object for VMID fences v2
v2: rebase & cleanup

This way we can store more than one fence as user for each VMID.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:16 -04:00
Christian König
bcb1ba35a8 drm/amdgpu: merge VM manager and VM context ID structure
No need to have two of them any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-04 20:19:13 -04:00
Christian König
8358dceed9 drm/amdgpu: use BO pages instead of GART array
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-02 15:26:56 -04:00
Christian König
fa3ab3c7ba drm/amdgpu: change parameter passing in the VM code
Make it more flexible by passing src and page addresses
directly instead of the structures they contain.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-02 15:26:56 -04:00
Chunming Zhou
8e9fbeb522 drm/amdgpu: improve vmid assigment V2
V2: the signaled items on the LRU maintain their order

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-05-02 15:04:11 -04:00
Christian König
32b41ac21f drm/amdgpu: Revert "add mutex for ba_va->valids/invalids"
Not needed any more because we need to protect the elements on the list anyway.

This reverts commit 38bf516c75b4ef0f5c716e05fa9baab7c52d6c39.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-09 13:04:02 -05:00
Christian König
2025021582 drm/amdgpu: Revert "add lock for interval tree in vm"
Not needed any more because we need to protect the elements on the list anyway.

This reverts commit fe237ed7efec8ac147a4572fdf81173a7f8ddda7.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-09 13:04:02 -05:00
Christian König
e17841b975 drm/amdgpu: Revert "add spin lock to protect freed list in vm (v3)"
Not needed any more because we need to protect the elements on the list anyway.

This reverts commit dae6ecf9e6c9b677e577826c3ac665c6dd9c490b.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-09 13:04:01 -05:00
Felix Kuehling
fb29b57c34 drm/amdgpu: Fix two bugs in amdgpu_vm_bo_split_mapping
Off-by-one: last is inclusive, so the maximum is start + max_size - 1
Wrong unit: addr is in bytes, max_size is in pages

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-09 13:04:00 -05:00
Christian König
2f568dbd6b drm/amdgpu: move get_user_pages out of amdgpu_ttm_tt_pin_userptr v6
That avoids lock inversion between the BO reservation lock
and the anon_vma lock.

v2:
* Changed amdgpu_bo_list_entry.user_pages to an array of pointers
* Lock mmap_sem only for get_user_pages
* Added invalidation of unbound userpointer BOs
* Fixed memory leak and page reference leak

v3 (chk):
* Revert locking mmap_sem only for_get user_pages
* Revert adding invalidation of unbound userpointer BOs
* Sanitize and fix error handling

v4 (chk):
* Init userpages pointer everywhere.
* Fix error handling when get_user_pages() fails.
* Add invalidation of unbound userpointer BOs again.

v5 (chk):
* Add maximum number of tries.

v6 (chk):
* Fix error handling when we run out of tries.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> (v4)
Acked-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-08 11:01:50 -05:00
Christian König
d564a06e1c drm/amdgpu: if a GDS switch is needed emit a pipeline sync as well
Otherwise we might change the GDS settings while they are still in use.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2016-03-08 11:01:50 -05:00
Christian König
b8c7b39ec1 drm/amdgpu: split pipeline sync and vm flush
This allows us to use the pipeline sync for other tasks as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2016-03-08 11:01:49 -05:00
Christian König
971fe9a941 drm/amdgpu: switch the GDS only on demand v2
Switching the GDS space to often seems to be problematic.

This patch together with the following can avoid VM faults on context switch.

v2: extend commit message a bit

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
2016-03-08 11:01:49 -05:00
Christian König
cffadc83c7 drm/amdgpu: move the GDS switch into vm flush as well
After all it's an operation on the VMID.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2016-03-08 11:01:48 -05:00
Christian König
a8bd1bec7c drm/amdgpu: sync to the active user on reusing a VMID
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-03-08 11:01:45 -05:00
Christian König
22073fe764 drm/amdgpu: trace the pd_addr in vm_grab_id as well
Makes matching it to the flushes much easier.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-29 11:33:59 -05:00
Christian König
4ff37a83f1 drm/amdgpu: fix VM faults caused by vm_grab_id() v4
The owner must be per ring as long as we don't
support sharing VMIDs per process. Also move the
assigned VMID and page directory address into the
IB structure.

v3: assign the VMID to all IBs, not just the first one.
v4: use correct pointer for owner

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-29 11:33:46 -05:00
Christian König
2d55e45a03 drm/amdgpu: use SDMA round robin for VM updates v3
Distribute the load on both rings.

v2: use a loop for the initialization
v3: agd: rebase on upstream

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-12 15:38:16 -05:00
Christian König
2bd9ccfa75 drm/amdgpu: use per VM entity for page table updates (v2)
Updates from different VMs can be processed independently.

v2: agd: rebase on upstream

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-12 15:35:16 -05:00
Christian König
e86f9ceee1 drm/amdgpu: move sync into job object
No need to keep that for every IB.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:24 -05:00
Christian König
d71518b5aa drm/amdgpu: cleanup in kernel job submission
Add a job_alloc_with_ib helper and proper job submission.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:22 -05:00
Christian König
b07c60c065 drm/amdgpu: move ring from IBs into job
We can't submit to multiple rings at the same time anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:20 -05:00
Christian König
9e5d53094c drm/amdgpu: make pad_ib a ring function v3
The padding depends on the firmware version and we need that for BO moves as
well, not only for VM updates.

v2: new approach of making pad_ib a ring function
v3: fix typo in macro name

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:20 -05:00
Christian König
b6ea2f37a2 drm/amdgpu: fix size estimation for clear IB
We only need a few dw here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:13 -05:00
Christian König
31f6c1fedb drm/amdgpu: optimize amdgpu_vm_update_ptes a bit
Don't calculate the end address multiple times.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:07 -05:00
Christian König
8843dbbbf3 drm/amdgpu: cleanup comments in VM code
Neither the global nor the local mutex exists any more and
amdgpu doesn't support cayman.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:07 -05:00
Christian König
a1e08d3b83 drm/amdgpu: optimize VM fencing
No need to fence every page table, just the page directory is enough.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:06 -05:00
Christian König
a14faa6573 drm/amdgpu: split VM mappings into smaller operations (v3)
If we can't copy entries from the GTT or fill them with one command split
up the mapping operation into multiple ones.

v2: agd: rebase on upstream
v3: squash in Christian's fix

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:05 -05:00
Christian König
9ab2146289 drm/amdgpu: use BOs GART instance for mapping addresses v4
That allows the VM code to use GART BOs from other driver instances.

v2: don't use copy optimization for foreign GARTs, that won't work.
v3: some more comment cleanups
v4: agd: rebase on upstream

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:05 -05:00
Christian König
b07c9d2a73 drm/amdgpu: move more logic into amdgpu_vm_map_gart v3
No need to duplicate that code over and over again. Also stop using the
flags to determine if we need to map the addresses.

v2: constify the pages_addr
v3: rebased, fix typo in commit message

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:04 -05:00
Christian König
599f434817 drm/amdgpu: remove nonsense IB size checks
Those are just leftovers from the time we wrote the VM
updates directly to the ring.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:17:04 -05:00
Christian König
a9a78b329a drm/amdgpu: use a global LRU list for VMIDs
With the scheduler enabled managing per ring LRUs don't
make much sense any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:16:58 -05:00
Christian König
94dd0a4ae0 drm/amdgpu: merge vm_grab_id and vm_fence v2
No need for an extra function any more.

v2: comment cleanups

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:16:57 -05:00
Christian König
165e4e07c2 drm/amdgpu: add VM pointer to id trace
Because of the scheduler all traces come from the same thread now and
can't be distincted otherwise.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:16:56 -05:00
Chunming Zhou
cadf97b196 drm/amdgpu: clean up non-scheduler code path (v2)
Non-scheduler code is longer supported.

v2: agd: rebased on upstream

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Ken Wang  <Qingqing.Wang@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:16:50 -05:00
Christian König
1ea863fd73 drm/amdgpu: keep the prefered/allowed domains in the BO
Stop copying that to the bo list entry, it doesn't change anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-02-10 14:16:47 -05:00
Michel Dänzer
9571e1d840 drm/amdgpu: Use drm_calloc_large for VM page_tables array
It can be big, depending on the VM address space size, which is tunable
via the vm_size module parameter.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93721
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
2016-01-19 12:25:09 -05:00
Christian König
eceb8a1562 drm/amdgpu: move VM page tables to the LRU end on CS v2
This makes it less likely to run into an ENOMEM because
VM page tables are evicted last.

v2: move the BOs in the LRU tail after validation

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2016-01-13 12:22:17 -05:00
Dave Airlie
fd3e14ffbd Merge branch 'drm-next-4.5' of git://people.freedesktop.org/~agd5f/linux into drm-next
[airlied: fixup build problems on arm - added errno.h include]
* 'drm-next-4.5' of git://people.freedesktop.org/~agd5f/linux: (152 commits)
  amd/powerplay: fix copy paste typo in hardwaremanager.c
  amd/powerplay: disable powerplay by default initially
  amd/powerplay: don't enable ucode fan control if vbios has no fan table
  drm/amd/powerplay: show gpu load when print gpu performance for Cz. (v2)
  drm/amd/powerplay: check whether need to enable thermal control. (v2)
  drm/amd/powerplay: add point check to avoid NULL point hang.
  drm/amdgpu/powerplay: Program a calculated value as Deep Sleep clock.
  drm/amd/powerplay: Don't return an error if fan table is missing
  drm/powerplay/hwmgr: log errors in tonga_hwmgr_backend_init
  drm/powerplay: add debugging output to processpptables.c
  drm/powerplay: add debugging output to tonga_processpptables.c
  amd/powerplay: Add structures required to report configuration change
  amd/powerplay: Fix get dal power level
  amd\powerplay Implement get dal power level
  drm/amd/powerplay: display gpu load when print performance for tonga.
  drm/amdgpu/powerplay: enable sysfs and debugfs interfaces late
  drm/amd/powerplay: move shared function of vi to hwmgr. (v2)
  drm/amd/powerplay: check whether enable dpm in powerplay.
  drm/amd/powerplay: fix bug that dpm funcs in debugfs/sysfs missing.
  drm/amd/powerplay: fix boolreturn.cocci warnings
  ...
2015-12-23 14:15:26 +10:00
Felix Kuehling
005ae95e6e drm/amdgpu: Fix off-by-one errors in amdgpu_vm_bo_map
eaddr is sometimes treated as the last address inside the address
range, and sometimes as the first address outside the range. This
was resulting in errors when a test filled up the entire address
space. Make it consistent to always be the last address within the
range.

Signed-off-by: Felix.Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org
2015-12-21 16:39:14 -05:00
Christian König
ee1782c3f2 drm/amdgpu: keep the PTs validation list in the VM v2
This avoids allocating it on the fly.

v2: fix grammar in comment

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-12-18 17:29:46 -05:00
Christian König
56467ebfb2 drm/amdgpu: split VM PD and PT handling during CS
This way we avoid the extra allocation for the page directory entry.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-12-18 17:29:45 -05:00
Christian König
3c0eea6c35 drm/amdgpu: put VM page tables directly into duplicates list
They share the reservation object with the page directory anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-12-18 17:29:45 -05:00
jimqu
81d75a30c6 drm/amdgpu: add spin lock to protect freed list in vm (v2)
there is a protection fault about freed list when OCL test.
add a spin lock to protect it.

v2: drop changes in vm_fini

Signed-off-by: JimQu <jim.qu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-12-04 13:08:13 -05:00
jimqu
9c4153b1ee drm/amdgpu: add spin lock to protect freed list in vm (v2)
there is a protection fault about freed list when OCL test.
add a spin lock to protect it.

v2: drop changes in vm_fini

Signed-off-by: JimQu <jim.qu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-12-04 12:23:38 -05:00
Christian König
82b9c55b1e drm/amdgpu: fix VM page table reference counting
We use the reservation object of the page directory for the page tables as
well, because of this the page directory should be freed last. Ensure that
by keeping a reference from the page tables to the directory.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2015-11-30 14:41:33 -05:00
Chunming Zhou
e98c1b0de6 drm/amdgpu: remove vm->mutex
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-11-20 18:22:28 -05:00
Chunming Zhou
69b576a1bc drm/amdgpu: add mutex for ba_va->valids/invalids
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-11-20 18:22:23 -05:00
Chunming Zhou
49b02b180a drm/amdgpu: reserve/unreserve objects out of map/unmap operations
Change-Id: Id6514f2fb6e002437fdbe99353d5d35f4ac736c7
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-11-18 11:41:20 -05:00
Chunming Zhou
ef9f0a83d6 drm/amdgpu: move bo_reserve out of amdgpu_vm_clear_bo
Change-Id: Ifbb0c06680494bfa04d0be5e5941d31ae2e5ef28
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-11-18 11:41:02 -05:00
Chunming Zhou
c25867dfab drm/amdgpu: add lock for interval tree in vm
Change-Id: I62b892a22af37b32e6b4aefca80a25cf45426ed2
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-11-18 11:40:55 -05:00
Christian König
1c16c0a7b2 drm/amdgpu: keep the owner for VMIDs
We don't need the last VM use any more, keep the owner directly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <davdi1.zhou@amd.com>
2015-11-18 11:40:37 -05:00
Christian König
ea89f8c9e8 drm/amdgpu: move VM manager clean into the VM code again
It's not a good idea to duplicate that code.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <davdi1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-11-18 11:40:27 -05:00
Christian König
16ae42feb0 drm/amdgpu: use common fence for amdgpu_vm_fence
Just cleanup the function parameters.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-11-16 11:05:49 -05:00
Christian König
b56c22853e drm/amdgpu: use fence_is_later() for vm_flush as well v2
v2: remove superfluous check

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-11-16 11:05:48 -05:00
Dave Airlie
793423ffcb Merge branch 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux into drm-next
- Updated register headers for GFX 8.1 for Stoney
- Add some new CZ revisions
- minor pageflip optimizations
- Fencing clean up
- Warning fix
- More fence cleanup
- oops fix
- Fiji fixes

* 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux: (29 commits)
  drm/amdgpu: group together common fence implementation
  drm/amdgpu: remove AMDGPU_FENCE_OWNER_MOVE
  drm/amdgpu: remove now unused fence functions
  drm/amdgpu: fix fence fallback check
  drm/amdgpu: fix stoping the scheduler timeout
  drm/amdgpu: cleanup on error in amdgpu_cs_ioctl()
  drm/amdgpu: update Fiji's Golden setting
  drm/amdgpu: update Fiji's rev id
  drm/amdgpu: extract common code in vi_common_early_init
  drm/amd/scheduler: don't oops on failure to load
  drm/amdgpu: don't oops on failure to load (v2)
  drm/amdgpu: don't VT switch on suspend
  drm/amdgpu: Make amdgpu_mn functions inline
  drm/amdgpu: remove amdgpu_fence_ref/unref
  drm/amdgpu: use common fence for sync
  drm/amdgpu: use the new fence_is_later
  drm/amdgpu: use common fences for VMID management v2
  drm/amdgpu: move ring_from_fence to common code
  drm/amdgpu: switch to common fence_wait_any_timeout v2
  drm/amdgpu: remove unneeded fence functions
  ...
2015-11-05 10:57:25 +10:00
Christian König
d52832986a drm/amdgpu: use common fences for VMID management v2
v2: add missing NULL check.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-10-30 01:55:12 -04:00
Dave Airlie
f1a04d8258 Merge branch 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux into drm-next
More amdgpu and radeon stuff for drm-next.  Stoney support is the big change.
The rest is just bug fixes and code cleanups.  The Stoney stuff is pretty
low impact with respect to existing chips.

* 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux:
  drm/amdgpu: change VM size default to 64GB
  drm/amdgpu: add Stoney pci ids
  drm/amdgpu: update the core VI support for Stoney
  drm/amdgpu: add VCE support for Stoney (v2)
  drm/amdgpu: add UVD support for Stoney
  drm/amdgpu: add GFX support for Stoney (v2)
  drm/amdgpu: add SDMA support for Stoney (v2)
  drm/amdgpu: add DCE support for Stoney
  drm/amdgpu: Update SMC/DPM for Stoney
  drm/amdgpu: add GMC support for Stoney
  drm/amdgpu: add Stoney chip family
  drm/amdgpu: fix the broken vm->mutex V2
  drm/amdgpu: remove the unnecessary parameter adev for amdgpu_fence_wait_any()
  drm/amdgpu: remove the exclusive lock
  drm/amdgpu: remove old lockup detection infrastructure
  drm: fix trivial typos
  drm/amdgpu/dce: simplify suspend/resume
  drm/amdgpu/gfx8: set TC_WB_ACTION_EN in RELEASE_MEM packet
  drm/radeon: Use rdev->gem.mutex to protect hyperz/cmask owners
2015-10-30 09:48:28 +10:00
Chunming Zhou
f48b2659f5 drm/amdgpu: fix the broken vm->mutex V2
fix the vm->mutex and ww_mutex confilcts.
vm->mutex is always token first, then ww_mutex.

V2: remove unneccessary checking for pt bo.

Change-Id: Iea56e183752c02831126d06d2f5b7a474a6e4743
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-10-21 11:35:14 -04:00
Dave Airlie
48f87dd146 Merge commit '06d1ee32a4d25356a710b49d5e95dbdd68bdf505' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into drm-next
Backmerge the drm-fixes pull from Linus's tree into drm-next.

This is to fix some conflicts and make future pulls cleaner
2015-10-16 10:25:28 +10:00
Christian König
d6c10f6b81 drm/amdgpu: add VM CS mapping trace point
Output all VM mappings a command submission uses.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-10-14 16:16:41 -04:00
Christian König
39ff8449c5 drm/amdgpu: also trace already allocated VMIDs
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-10-14 16:16:38 -04:00
Sudip Mukherjee
7a574557e6 drm/amdgpu: fix memory leak in amdgpu_vm_update_page_directory
If amdgpu_ib_get() fails we returned the error code but we missed
freeing ib.

Cc: "Christian König" <christian.koenig@amd.com>
Cc: Jammy Zhou <Jammy.Zhou@amd.com>
Cc: Chunming Zhou <david1.zhou@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "monk.liu" <monk.liu@amd.com>
Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
2015-10-08 12:18:23 -04:00
Junwei Zhang
4c7eb91cae drm/amdgpu: refine the job naming for amdgpu_job and amdgpu_sched_job
Use consistent naming across functions.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
2015-09-23 17:23:36 -04:00
Christian König
bf60efd353 drm/amdgpu: use only one reservation object for each VM v2
Reduces the locking and fencing overhead.

v2: add comment why we need the duplicates list in the GEM op.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
2015-09-23 17:23:35 -04:00
Christian König
72d7668b5b drm/amdgpu: export reservation_object from dmabuf to ttm (v2)
Adds an extra argument to amdgpu_bo_create, which is only used in amdgpu_prime.c.

Port of radeon commit 831b6966a6.

v2: fix up kfd.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-09-23 17:23:34 -04:00
Christian König
b7d698d7fd drm/amdgpu: fix overflow on 32bit systems
mem->start is a long, so this can overflow on 32bit systems.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Cc: stable@vger.kernel.org
2015-09-23 17:23:33 -04:00
Alex Deucher
857d913d05 drm/amdgpu: be explicit about cpu vram access for driver BOs (v2)
For kernel driver BOs, be explicit about whether we need
vram access up front.  This avoids unecessary migrations and
avoids using visible vram for buffers were it's not needed.

v2: line wrap fixes

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2015-09-03 10:29:32 -04:00
Christian König
a60c42320b drm/amdgpu: use PT for VM sync on unmap
Instead of the array which is used for ID management.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-09-02 12:19:54 -04:00
Chunming Zhou
c7ae72c01b drm/amdgpu: use IB for copy buffer of eviction
This aids handling buffers moves with the scheduler.

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
2015-08-26 17:50:42 -04:00
Chunming Zhou
3c62338c26 drm/amdgpu: fix last_vm_update fence is not effetive for sched fence
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
2015-08-25 10:53:05 -04:00
Chunming Zhou
bb977d3711 drm/amdgpu: abstract amdgpu_job for scheduler
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
2015-08-20 17:00:35 -04:00
Bas Nieuwenhuizen
05906dec7d drm/amdgpu: wait on page directory changes. v2
Pagetables can be moved and therefore the page directory update can be necessary
for the current cs even if none of the the bo's are moved. In that scenario
there is no fence between the sdma0 and gfx ring, so we add one.

v2 (chk): rebased

Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-08-17 16:51:23 -04:00
monk.liu
6d1d0ef743 drm/amdgpu: fix duplicated mapping invoke bug
fix the bug that there is duplicated bo_update_mapping issued

Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2015-08-17 16:51:20 -04:00
Chunming Zhou
281b422301 drm/amdgpu: add reference for **fence
fix fence is released when pass to **fence sometimes.
add reference for it.

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
2015-08-17 16:51:17 -04:00
Chunming Zhou
4af9f07ccd drm/amdgpu: use kernel submit helper in vm
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
2015-08-17 16:51:07 -04:00
Christian König
91404fb208 drm/amdgpu: merge amd_sched_entity and amd_context_entity v2
Avoiding a couple of casts.

v2: rename c_entity to entity as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
2015-08-17 16:51:05 -04:00
Chunming Zhou
80de5913cf Revert "drm/amdgpu: return new seq_no for amd_sched_push_job"
This reverts commit d1d33da8eb86b8ca41dd9ed95738030df5267b95.

Reviewed-by: Christian K?nig <christian.koenig@amd.com>

Conflicts:
	drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
	drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
2015-08-17 16:51:02 -04:00
Christian König
47f38501f1 drm/amdgpu: cleanup amdgpu_ctx inti/fini v2
Cleanup the kernel context handling.

v2: rebased

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
2015-08-17 16:51:02 -04:00
Jammy Zhou
ea199cc9f8 drm/amdgpu: return new seq_no for amd_sched_push_job
It is clean to update last_queued_v_seq in the scheduler module

Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-08-17 16:50:56 -04:00
Chunming Zhou
bb1e38a4be drm/amdgpu: use kernel fence for last_pt_update
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
2015-08-17 16:50:51 -04:00
Chunming Zhou
e40a31159b drm/amdgpu: use kernel fence diretly in amdgpu_bo_fence
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
2015-08-17 16:50:50 -04:00