Commit Graph

55 Commits

Author SHA1 Message Date
Dave Hansen
1b2ee1266e mm/core: Do not enforce PKEY permissions on remote mm access
We try to enforce protection keys in software the same way that we
do in hardware.  (See long example below).

But, we only want to do this when accessing our *own* process's
memory.  If GDB set PKRU[6].AD=1 (disable access to PKEY 6), then
tried to PTRACE_POKE a target process which just happened to have
some mprotect_pkey(pkey=6) memory, we do *not* want to deny the
debugger access to that memory.  PKRU is fundamentally a
thread-local structure and we do not want to enforce it on access
to _another_ thread's data.

This gets especially tricky when we have workqueues or other
delayed-work mechanisms that might run in a random process's context.
We can check that we only enforce pkeys when operating on our *own* mm,
but delayed work gets performed when a random user context is active.
We might end up with a situation where a delayed-work gup fails when
running randomly under its "own" task but succeeds when running under
another process.  We want to avoid that.

To avoid that, we use the new GUP flag: FOLL_REMOTE and add a
fault flag: FAULT_FLAG_REMOTE.  They indicate that we are
walking an mm which is not guranteed to be the same as
current->mm and should not be subject to protection key
enforcement.

Thanks to Jerome Glisse for pointing out this scenario.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boaz Harrosh <boaz@plexistor.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Chinner <dchinner@redhat.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Dominik Dingel <dingel@linux.vnet.ibm.com>
Cc: Dominik Vogt <vogt@linux.vnet.ibm.com>
Cc: Eric B Munson <emunson@akamai.com>
Cc: Geliang Tang <geliangtang@163.com>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jason Low <jason.low2@hp.com>
Cc: Jerome Marchand <jmarchan@redhat.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Matthew Wilcox <willy@linux.intel.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mikulas Patocka <mpatocka@redhat.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Shachar Raindel <raindel@mellanox.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Xie XiuQi <xiexiuqi@huawei.com>
Cc: iommu@lists.linux-foundation.org
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-s390@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-02-18 19:46:28 +01:00
Julia Lawall
759ce23b62 iommu/amd: Constify mmu_notifier_ops structures
This mmu_notifier_ops structure is never modified, so declare it as
const, like the other mmu_notifier_ops structures.

Done with the help of Coccinelle.

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-14 15:44:29 +01:00
Joerg Roedel
492e74594e iommu/amd: Cleanup error handling in do_fault()
Get rid of the three error paths that look the same and move
error handling to a single place.

Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Acked-By: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-14 15:39:41 +01:00
Joerg Roedel
43c0ea20de iommu/amd: Correctly set flags for handle_mm_fault call
Instead of just checking for a write access, calculate the
flags that are passed to handle_mm_fault() more precisly and
use the pre-defined macros.

Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Acked-By: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-14 15:39:34 +01:00
Joerg Roedel
7b5cc1a9c9 iommu/amd: Do proper access checking before calling handle_mm_fault()
The handle_mm_fault function expects the caller to do the
access checks. Not doing so and calling the function with
wrong permissions is a bug (catched by a BUG_ON).
So fix this bug by adding proper access checking to the io
page-fault code in the AMD IOMMUv2 driver.

Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Acked-By: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-14 15:37:47 +01:00
Jay Cornwall
d14f6fced5 iommu/amd: Fix BUG when faulting a PROT_NONE VMA
handle_mm_fault indirectly triggers a BUG in do_numa_page
when given a VMA without read/write/execute access. Check
this condition in do_fault.

do_fault -> handle_mm_fault -> handle_pte_fault -> do_numa_page

  mm/memory.c
  3147  static int do_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
  ....
  3159  /* A PROT_NONE fault should not end up here */
  3160  BUG_ON(!(vma->vm_flags & (VM_READ | VM_EXEC | VM_WRITE)));

Signed-off-by: Jay Cornwall <jay@jcornwall.me>
Cc: <stable@vger.kernel.org> # v4.1+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-15 16:13:07 +02:00
Joerg Roedel
23d3a98c13 iommu/amd: Use BUG_ON instead of if () BUG()
Found by a coccicheck script.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-08-13 19:49:14 +02:00
Joerg Roedel
55c99a4dc5 iommu/amd: Use iommu_attach_group()
Since the conversion to default domains the
iommu_attach_device function only works for devices with
their own group. But this isn't always true for current
IOMMUv2 capable devices, so use iommu_attach_group instead.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-07-30 10:28:48 +02:00
Oded Gabbay
1bf1b431d9 iommu/amd: Fix bug in put_pasid_state_wait
This patch fixes a bug in put_pasid_state_wait that appeared in kernel 4.0
The bug is that pasid_state->count wasn't decremented before entering the
wait_event. Thus, the condition in wait_event will never be true.

The fix is to decrement (atomically) the pasid_state->count before the
wait_event.

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Cc: stable@vger.kernel.org #v4.0
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-05-04 13:20:22 +02:00
Dan Carpenter
940f700dfb iommu/amd: Small cleanup in mn_release()
"pasid_state->device_state" and "dev_state" are the same, but it's nicer
to use dev_state consistently.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-04 12:58:59 +01:00
Joerg Roedel
a20cc76b9e Merge branches 'arm/renesas', 'arm/smmu', 'arm/omap', 'ppc/pamu', 'x86/amd' and 'core' into next
Conflicts:
	drivers/iommu/Kconfig
	drivers/iommu/Makefile
2015-02-04 16:53:44 +01:00
Joerg Roedel
63ce3ae889 iommu: Update my email address
The AMD address is dead for a long time already, replace it
with a working one.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-02-04 16:14:34 +01:00
Joerg Roedel
a1bec062c9 iommu/amd: Use wait_event in put_pasid_state_wait
Now that I learned about possible spurious wakeups this
place needs fixing too. Replace the self-coded sleep variant
with the generic wait_event() helper.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-02-04 15:50:51 +01:00
Peter Zijlstra
91f65facba iommu/amd: Fix amd_iommu_free_device()
put_device_state_wait() doesn't loop on the condition and a spurious
wakeup will have it free the device state even though there might still
be references out to it.

Fix this by using 'normal' wait primitives.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-02-04 15:33:36 +01:00
Linus Torvalds
988adfdffd Merge branch 'drm-next' of git://people.freedesktop.org/~airlied/linux
Pull drm updates from Dave Airlie:
 "Highlights:

   - AMD KFD driver merge

     This is the AMD HSA interface for exposing a lowlevel interface for
     GPGPU use.  They have an open source userspace built on top of this
     interface, and the code looks as good as it was going to get out of
     tree.

   - Initial atomic modesetting work

     The need for an atomic modesetting interface to allow userspace to
     try and send a complete set of modesetting state to the driver has
     arisen, and been suffering from neglect this past year.  No more,
     the start of the common code and changes for msm driver to use it
     are in this tree.  Ongoing work to get the userspace ioctl finished
     and the code clean will probably wait until next kernel.

   - DisplayID 1.3 and tiled monitor exposed to userspace.

     Tiled monitor property is now exposed for userspace to make use of.

   - Rockchip drm driver merged.

   - imx gpu driver moved out of staging

  Other stuff:

   - core:
        panel - MIPI DSI + new panels.
        expose suggested x/y properties for virtual GPUs

   - i915:
        Initial Skylake (SKL) support
        gen3/4 reset work
        start of dri1/ums removal
        infoframe tracking
        fixes for lots of things.

   - nouveau:
        tegra k1 voltage support
        GM204 modesetting support
        GT21x memory reclocking work

   - radeon:
        CI dpm fixes
        GPUVM improvements
        Initial DPM fan control

   - rcar-du:
        HDMI support added
        removed some support for old boards
        slave encoder driver for Analog Devices adv7511

   - exynos:
        Exynos4415 SoC support

   - msm:
        a4xx gpu support
        atomic helper conversion

   - tegra:
        iommu support
        universal plane support
        ganged-mode DSI support

   - sti:
        HDMI i2c improvements

   - vmwgfx:
        some late fixes.

   - qxl:
        use suggested x/y properties"

* 'drm-next' of git://people.freedesktop.org/~airlied/linux: (969 commits)
  drm: sti: fix module compilation issue
  drm/i915: save/restore GMBUS freq across suspend/resume on gen4
  drm: sti: correctly cleanup CRTC and planes
  drm: sti: add HQVDP plane
  drm: sti: add cursor plane
  drm: sti: enable auxiliary CRTC
  drm: sti: fix delay in VTG programming
  drm: sti: prepare sti_tvout to support auxiliary crtc
  drm: sti: use drm_crtc_vblank_{on/off} instead of drm_vblank_{on/off}
  drm: sti: fix hdmi avi infoframe
  drm: sti: remove event lock while disabling vblank
  drm: sti: simplify gdp code
  drm: sti: clear all mixer control
  drm: sti: remove gpio for HDMI hot plug detection
  drm: sti: allow to change hdmi ddc i2c adapter
  drm/doc: Document drm_add_modes_noedid() usage
  drm/i915: Remove '& 0xffff' from the mask given to WA_REG()
  drm/i915: Invert the mask and val arguments in wa_add() and WA_REG()
  drm: Zero out DRM object memory upon cleanup
  drm/i915/bdw: Fix the write setting up the WIZ hashing mode
  ...
2014-12-15 15:52:01 -08:00
Linus Torvalds
78a45c6f06 Merge branch 'akpm' (second patch-bomb from Andrew)
Merge second patchbomb from Andrew Morton:
 - the rest of MM
 - misc fs fixes
 - add execveat() syscall
 - new ratelimit feature for fault-injection
 - decompressor updates
 - ipc/ updates
 - fallocate feature creep
 - fsnotify cleanups
 - a few other misc things

* emailed patches from Andrew Morton <akpm@linux-foundation.org>: (99 commits)
  cgroups: Documentation: fix trivial typos and wrong paragraph numberings
  parisc: percpu: update comments referring to __get_cpu_var
  percpu: update local_ops.txt to reflect this_cpu operations
  percpu: remove __get_cpu_var and __raw_get_cpu_var macros
  fsnotify: remove destroy_list from fsnotify_mark
  fsnotify: unify inode and mount marks handling
  fallocate: create FAN_MODIFY and IN_MODIFY events
  mm/cma: make kmemleak ignore CMA regions
  slub: fix cpuset check in get_any_partial
  slab: fix cpuset check in fallback_alloc
  shmdt: use i_size_read() instead of ->i_size
  ipc/shm.c: fix overly aggressive shmdt() when calls span multiple segments
  ipc/msg: increase MSGMNI, remove scaling
  ipc/sem.c: increase SEMMSL, SEMMNI, SEMOPM
  ipc/sem.c: change memory barrier in sem_lock() to smp_rmb()
  lib/decompress.c: consistency of compress formats for kernel image
  decompress_bunzip2: off by one in get_next_block()
  usr/Kconfig: make initrd compression algorithm selection not expert
  fault-inject: add ratelimit option
  ratelimit: add initialization macro
  ...
2014-12-13 13:00:36 -08:00
Jesse Barnes
9dc00f4c4f iommu/amd: use handle_mm_fault directly
This could be useful for debug in the future if we want to track
major/minor faults more closely, and also avoids the put_page trick we
used with gup.

In order to do this, we also track the task struct in the PASID state
structure.  This lets us update the appropriate task stats after the fault
has been handled, and may aid with debug in the future as well.

Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Tested-by: Oded Gabbay <oded.gabbay@amd.com>
Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-12-13 12:42:47 -08:00
Oded Gabbay
1c51099a42 iommu/amd: Fix accounting of device_state
This patch fixes a bug in the accounting of the
device_state.  In the current code, the device_state was put
(decremented) too many times, which sometimes lead to the
driver getting stuck permanently in put_device_state_wait().
That happen because the device_state->count would go below
zero, which is never supposed to happen.

The root cause is that the device_state was decremented in
put_pasid_state() and put_pasid_state_wait() but also in all
the functions that call those functions. Therefore, the
device_state was decremented twice in each of these code
paths.

The fix is to decouple the device_state accounting from the
pasid_state accounting - remove the call to
put_device_state() from the put_pasid_state() and the
put_pasid_state_wait())

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-12 14:58:33 +01:00
Andres Lagar-Cavilla
5712846808 kvm: Fix page ageing bugs
1. We were calling clear_flush_young_notify in unmap_one, but we are
within an mmu notifier invalidate range scope. The spte exists no more
(due to range_start) and the accessed bit info has already been
propagated (due to kvm_pfn_set_accessed). Simply call
clear_flush_young.

2. We clear_flush_young on a primary MMU PMD, but this may be mapped
as a collection of PTEs by the secondary MMU (e.g. during log-dirty).
This required expanding the interface of the clear_flush_young mmu
notifier, so a lot of code has been trivially touched.

3. In the absence of shadow_accessed_mask (e.g. EPT A bit), we emulate
the access bit by blowing the spte. This requires proper synchronizing
with MMU notifier consumers, like every other removal of spte's does.

Signed-off-by: Andres Lagar-Cavilla <andreslc@google.com>
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-09-24 14:07:58 +02:00
Joerg Roedel
daff2f9c9a iommu/amd: Fix 2 typos in comments
amd_iommu_pasid_bind -> amd_iommu_bind_pasid

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-30 16:07:50 +02:00
Joerg Roedel
75058a302b iommu/amd: Fix device_state reference counting
The references to the device state are not dropped
everywhere. This might cause a dead-lock in
amd_iommu_free_device(). Fix it.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-30 16:07:50 +02:00
Joerg Roedel
8301da53fb iommu/amd: Remove change_pte mmu_notifier call-back
All calls to this call-back are wrapped with
mmu_notifer_invalidate_range_start()/end(), making this
notifier pretty useless, so remove it.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-30 16:07:49 +02:00
Joerg Roedel
fcaa960608 iommu/amd: Don't set pasid_state->mm to NULL in unbind_pasid
With calling te mmu_notifier_register function we hold a
reference to the mm_struct that needs to be released in
mmu_notifier_unregister. This is true even if the notifier
was already unregistered from exit_mmap and the .release
call-back has already run.

So make sure we call mmu_notifier_unregister unconditionally
in amd_iommu_unbind_pasid.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-30 16:07:49 +02:00
Joerg Roedel
d9e1611e73 iommu/amd: Don't call the inv_ctx_cb when pasid is not set up
On the error path of amd_iommu_bind_pasid() we call
mmu_notifier_unregister() for cleanup. This calls
mn_release() which calls the users inv_ctx_cb function if
one is available. Since the pasid is not set up yet there is
nothing the user can to tear down in this call-back. So
don't call inv_ctx_cb on the error path of
amd_iommu_unbind_pasid() and make life of the users simpler.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:37:03 +02:00
Joerg Roedel
dba3838d7a iommu/amd: Don't hold a reference to task_struct
Since we are only caring about the lifetime of the mm_struct
and not the task we can't safely keep a reference to it. The
reference is also not needed anymore, so remove that code
entirely.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:58 +02:00
Joerg Roedel
f0aac63b87 iommu/amd: Don't hold a reference to mm_struct
With mmu_notifiers we don't need to hold a reference to the
mm_struct during the time the pasid is bound to a device. We
can rely on the .mn_release call back to inform us when the
mm_struct goes away.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:52 +02:00
Joerg Roedel
53d340ef4a iommu/amd: Add pasid_state->invalid flag
This is used to signal the ppr_notifer function that no more
faults should be processes on this pasid_state. This way we
can keep the pasid_state safely in the state-table so that
it can be freed in the amd_iommu_unbind_pasid() function.

This allows us to not hold a reference to the mm_struct
during the whole pasid-binding-time.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:46 +02:00
Joerg Roedel
dc88db7ee9 iommu/amd: Drop pasid_state reference in ppr_notifer error path
In case we are not able to allocate a fault structure a
reference to the pasid_state will be leaked. Fix that by
dropping the reference in the error path in case we hold
one.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:40 +02:00
Joerg Roedel
61feb43897 iommu/amd: Get rid of __unbind_pasid
Unbind_pasid is only called from mn_release which already
has the pasid_state. Use this to simplify the unbind_pasid
path and get rid of __unbind_pasid.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:34 +02:00
Joerg Roedel
c5db16ad6c iommu/amd: Don't free pasid_state in mn_release path
The mmu_notifier state is part of pasid_state so it can't be
freed in the mn_release path. Free the pasid_state after
mmu_notifer_unregister has completed.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:27 +02:00
Joerg Roedel
caf8a518d7 iommu/amd: Don't call mmu_notifer_unregister in __unbind_pasid
This function is called only in the mn_release() path, so
there is no need to unregister the mmu_notifer here.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10 15:36:22 +02:00
Joerg Roedel
ff6d0cce38 iommu/amd: Fix typo in amd_iommu_v2 driver
Fix typo in a comment.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-10 15:36:17 +02:00
Alexey Skidanov
b00675b867 iommu/amd: Fix for pasid initialization
The pasid wasn't properly initialized before caling to invalid PPR calback

Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-09 16:02:41 +02:00
Joerg Roedel
d73a6d722a iommu/amd: Fix small race between invalidate_range_end/start
Commit e79df31 introduced mmu_notifer_count to protect
against parallel mmu_notifier_invalidate_range_start/end
calls. The patch left a small race condition when
invalidate_range_end() races with a new
invalidate_range_start() the empty page-table may be
reverted leading to stale TLB entries in the IOMMU and the
device. Use a spin_lock instead of just an atomic variable
to eliminate the race.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-06-20 16:14:22 +02:00
Joerg Roedel
e79df31c60 iommu/amd: Handle parallel invalidate_range_start/end calls correctly
Add a counter to the pasid_state so that we do not restore
the original page-table before all invalidate_range_start
to invalidate_range_end sections have finished.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-05-26 11:28:14 +02:00
Joerg Roedel
9163b90135 iommu/amd: Remove IOMMUv2 pasid_state_list
This list was only used for the task_exit notifier function.
Now that it is gone we can remove it.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26 11:28:14 +02:00
Joerg Roedel
a40d4c67d7 iommu/amd: Implement mmu_notifier_release call-back
Since mmu_notifier call-backs can sleep (because they use
SRCU now) we can use them to tear down PASID mappings. This
allows us to finally remove the hack to use the task_exit
notifier from oprofile to get notified when a process dies.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26 11:28:13 +02:00
Joerg Roedel
741669c765 iommu/amd: Convert IOMMUv2 state_table into state_list
The state_table consumes 512kb of memory and is only sparsly
populated. Convert it into a list to save memory. There
should be no measurable performance impact.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26 11:28:12 +02:00
Joerg Roedel
b87d2d7c0b iommu/amd: Don't access IOMMUv2 state_table directly
This is a preparation for converting the state_table into a
state_list.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26 11:28:12 +02:00
Jay Cornwall
4378d99295 iommu/amd: Take mmap_sem when calling get_user_pages
get_user_pages requires caller to hold a read lock on mmap_sem.

Signed-off-by: Jay Cornwall <jay.cornwall@amd.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-05-13 12:33:11 +02:00
Oded Gabbay
a015c1e926 iommu/amd: fix accounting of device_state
This patch fixes a bug in the accounting of the device_state.
In the current code, the device_state was put (decremented) too many times,
which sometimes lead to the driver getting stuck permanently in
put_device_state_wait(). That happen because the device_state->count would go
below zero, which is never supposed to happen.

The root cause is that the device_state was decremented in put_pasid_state()
and put_pasid_state_wait() but also in all the functions that call those
functions. Therefore, the device_state was decremented twice in each of these
code paths.

The fix is to decouple the device_state accounting from the pasid_state
accounting - remove the call to put_device_state() from the
put_pasid_state() and the put_pasid_state_wait())

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-10 10:57:36 +02:00
Joerg Roedel
e7cc3dd48c iommu/amd: use new invalidate_range mmu-notifier
Make use of the new invalidate_range mmu_notifier call-back and remove the
old logic of assigning an empty page-table between invalidate_range_start
and invalidate_range_end.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Rik van Riel <riel@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Johannes Weiner <jweiner@redhat.com>
Cc: Jay Cornwall <Jay.Cornwall@amd.com>
Cc: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-13 13:46:09 +11:00
Linus Torvalds
97027da6ad IOMMU Updates for Linux v3.6-rc1
The most important part of these updates is the IOMMU groups code
 enhancement written by Alex Williamson. It abstracts the problem that a
 given hardware IOMMU can't isolate any given device from any other
 device (e.g. 32 bit PCI devices can't usually be isolated). Devices that
 can't be isolated are grouped together. This code is required for the
 upcoming VFIO framework.
 
 Another IOMMU-API change written by be is the introduction of domain
 attributes. This makes it easier to handle GART-like IOMMUs with the
 IOMMU-API because now the start-address and the size of the domain
 address space can be queried.
 
 Besides that there are a few cleanups and fixes for the NVidia Tegra
 IOMMU drivers and the reworked init-code for the AMD IOMMU. The later is
 from my patch-set to support interrupt remapping. The rest of this
 patch-set requires x86 changes which are not mergabe yet. So full
 support for interrupt remapping with AMD IOMMUs will come in a future
 merge window.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJQDV/MAAoJECvwRC2XARrjSDcP+gJbtSHDMyZ71zyfQfAZcxJt
 rTqLbdZRtIjrjgtKSEDp8u5Bo5TK9dAYoZVuJMOZewFzwI/fSfbRsWp1PU0I88Fr
 ZzM+/o1N9MLvf1e3kRVOzNzUfku+jTQgUBD4txsbtQzc/IeGHe9qP1Bqzs/xg4Pk
 SjWu7pLNYxaER10z76nRodNn6zGjsc7GFdOW8cJu2HOAHhisIAR291jSQgd6Rz9r
 zWqSTsXIEzYt2CtU3G2/tFJ554Mp8v5F80gHo+0Ldw8aNxlD6nGtbqGNt+KI8qTv
 MUL8KJ0TNms9CZdti1CSlSNp51VgJi2GaWKCaDAkYuuER2IbC/8Yp/p2DIIA0GNp
 HpziIs+dauZPWfZHc6oU7lJAClGAG4MUx7CysVIOzl7ML/Bf4mjYv0faGf5YQfyE
 weOR+OPPIWDUwgjzHKMAboA4ijkE/v+EKjOaN/S9rEqFEMKC99fwGkf9wUcpZTne
 8lzdI2JrgYNDWMVNYlomeLD4lBAbxb/QsnRUa33igjr0MclvMDkp5HaO631Z1+Zx
 be2z8Rl1CtMwS4qeaOXoeaoNWHU26+oJRZNtCGi/Fw4aKqYXP1dnE/m0GtqEP9Yi
 +CU2rKbZn3j0+ZcQjCQop8FREPrZ2/Uaji70b6G7WZ2ApcqBxzBffpbMKOmd6T1D
 HIzGh0fpdYNDuwn6Txit
 =MbAC
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v3.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "The most important part of these updates is the IOMMU groups code
  enhancement written by Alex Williamson.  It abstracts the problem that
  a given hardware IOMMU can't isolate any given device from any other
  device (e.g.  32 bit PCI devices can't usually be isolated).  Devices
  that can't be isolated are grouped together.  This code is required
  for the upcoming VFIO framework.

  Another IOMMU-API change written by me is the introduction of domain
  attributes.  This makes it easier to handle GART-like IOMMUs with the
  IOMMU-API because now the start-address and the size of the domain
  address space can be queried.

  Besides that there are a few cleanups and fixes for the NVidia Tegra
  IOMMU drivers and the reworked init-code for the AMD IOMMU.  The
  latter is from my patch-set to support interrupt remapping.  The rest
  of this patch-set requires x86 changes which are not mergabe yet.  So
  full support for interrupt remapping with AMD IOMMUs will come in a
  future merge window."

* tag 'iommu-updates-v3.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (33 commits)
  iommu/amd: Fix hotplug with iommu=pt
  iommu/amd: Add missing spin_lock initialization
  iommu/amd: Convert iommu initialization to state machine
  iommu/amd: Introduce amd_iommu_init_dma routine
  iommu/amd: Move unmap_flush message to amd_iommu_init_dma_ops()
  iommu/amd: Split enable_iommus() routine
  iommu/amd: Introduce early_amd_iommu_init routine
  iommu/amd: Move informational prinks out of iommu_enable
  iommu/amd: Split out PCI related parts of IOMMU initialization
  iommu/amd: Use acpi_get_table instead of acpi_table_parse
  iommu/amd: Fix sparse warnings
  iommu/tegra: Don't call alloc_pdir with as->lock
  iommu/tegra: smmu: Fix unsleepable memory allocation at alloc_pdir()
  iommu/tegra: smmu: Remove unnecessary sanity check at alloc_pdir()
  iommu/exynos: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/tegra: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/msm: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/omap: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/vt-d: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/amd: Implement DOMAIN_ATTR_GEOMETRY attribute
  ...
2012-07-24 16:24:11 -07:00
Masanari Iida
07db04098d iommu: Fix typo in iommu
Correct spelling typo in debug messages and comments
in drivers/iommu.

Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2012-07-24 12:58:49 +02:00
Joerg Roedel
395e51f18d Merge branches 'iommu/fixes', 'x86/amd', 'groups', 'arm/tegra' and 'api/domain-attr' into next
Conflicts:
	drivers/iommu/iommu.c
	include/linux/iommu.h
2012-07-23 12:17:00 +02:00
Joerg Roedel
2c13d47a1a iommu/amd: Add missing spin_lock initialization
Add missing spin_lock initialization in
amd_iommu_bind_pasid() function and make lockdep happy
again.

Cc: stable@vger.kernel.org # >= v3.3
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-07-19 10:56:10 +02:00
Joerg Roedel
98f1ad2582 iommu/amd: Fix sparse warnings
A few sparse warnings fire in drivers/iommu/amd_iommu_init.c.
Fix most of them with this patch. Also fix the sparse
warnings in drivers/iommu/irq_remapping.c while at it.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-07-17 12:14:55 +02:00
Joerg Roedel
474d567db3 iommu/amd: Don't initialize IOMMUv2 resources when not required
Add a check to the init-path of the AMD IOMMUv2 driver if
the hardware is available in the system. Only allocate all
the resources if it is really available.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-03-15 12:46:40 +01:00
Joerg Roedel
bc21662f72 iommu/amd: Add invalidate-context call-back
This call-back is invoked when the task that is bound to a
pasid is about to exit. The driver can use it to shutdown
all context related to that context in a safe way.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-15 11:15:39 +01:00
Joerg Roedel
175d614673 iommu/amd: Add invalid_ppr callback
This callback can be used to change the PRI response code
sent to a device when a PPR fault fails.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-14 12:09:39 +01:00