linux/drivers/infiniband/core
Leon Romanovsky 1ff5325c3c RDMA/uverbs: Fix circular locking dependency
Avoid circular locking dependency by calling
to uobj_alloc_commit() outside of xrcd_tree_mutex lock.

======================================================
WARNING: possible circular locking dependency detected
4.15.0+ #87 Not tainted
------------------------------------------------------
syzkaller401056/269 is trying to acquire lock:
 (&uverbs_dev->xrcd_tree_mutex){+.+.}, at: [<000000006c12d2cd>] uverbs_free_xrcd+0xd2/0x360

but task is already holding lock:
 (&ucontext->uobjects_lock){+.+.}, at: [<00000000da010f09>] uverbs_cleanup_ucontext+0x168/0x730

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #1 (&ucontext->uobjects_lock){+.+.}:
       __mutex_lock+0x111/0x1720
       rdma_alloc_commit_uobject+0x22c/0x600
       ib_uverbs_open_xrcd+0x61a/0xdd0
       ib_uverbs_write+0x7f9/0xef0
       __vfs_write+0x10d/0x700
       vfs_write+0x1b0/0x550
       SyS_write+0xc7/0x1a0
       entry_SYSCALL_64_fastpath+0x1e/0x8b

-> #0 (&uverbs_dev->xrcd_tree_mutex){+.+.}:
       lock_acquire+0x19d/0x440
       __mutex_lock+0x111/0x1720
       uverbs_free_xrcd+0xd2/0x360
       remove_commit_idr_uobject+0x6d/0x110
       uverbs_cleanup_ucontext+0x2f0/0x730
       ib_uverbs_cleanup_ucontext.constprop.3+0x52/0x120
       ib_uverbs_close+0xf2/0x570
       __fput+0x2cd/0x8d0
       task_work_run+0xec/0x1d0
       do_exit+0x6a1/0x1520
       do_group_exit+0xe8/0x380
       SyS_exit_group+0x1e/0x20
       entry_SYSCALL_64_fastpath+0x1e/0x8b

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ucontext->uobjects_lock);
                               lock(&uverbs_dev->xrcd_tree_mutex);
                               lock(&ucontext->uobjects_lock);
  lock(&uverbs_dev->xrcd_tree_mutex);

 *** DEADLOCK ***

3 locks held by syzkaller401056/269:
 #0:  (&file->cleanup_mutex){+.+.}, at: [<00000000c9f0c252>] ib_uverbs_close+0xac/0x570
 #1:  (&ucontext->cleanup_rwsem){++++}, at: [<00000000b6994d49>] uverbs_cleanup_ucontext+0xf6/0x730
 #2:  (&ucontext->uobjects_lock){+.+.}, at: [<00000000da010f09>] uverbs_cleanup_ucontext+0x168/0x730

stack backtrace:
CPU: 0 PID: 269 Comm: syzkaller401056 Not tainted 4.15.0+ #87
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
Call Trace:
 dump_stack+0xde/0x164
 ? dma_virt_map_sg+0x22c/0x22c
 ? uverbs_cleanup_ucontext+0x168/0x730
 ? console_unlock+0x502/0xbd0
 print_circular_bug.isra.24+0x35e/0x396
 ? print_circular_bug_header+0x12e/0x12e
 ? find_usage_backwards+0x30/0x30
 ? entry_SYSCALL_64_fastpath+0x1e/0x8b
 validate_chain.isra.28+0x25d1/0x40c0
 ? check_usage+0xb70/0xb70
 ? graph_lock+0x160/0x160
 ? find_usage_backwards+0x30/0x30
 ? cyc2ns_read_end+0x10/0x10
 ? print_irqtrace_events+0x280/0x280
 ? __lock_acquire+0x93d/0x1630
 __lock_acquire+0x93d/0x1630
 lock_acquire+0x19d/0x440
 ? uverbs_free_xrcd+0xd2/0x360
 __mutex_lock+0x111/0x1720
 ? uverbs_free_xrcd+0xd2/0x360
 ? uverbs_free_xrcd+0xd2/0x360
 ? __mutex_lock+0x828/0x1720
 ? mutex_lock_io_nested+0x1550/0x1550
 ? uverbs_cleanup_ucontext+0x168/0x730
 ? __lock_acquire+0x9a9/0x1630
 ? mutex_lock_io_nested+0x1550/0x1550
 ? uverbs_cleanup_ucontext+0xf6/0x730
 ? lock_contended+0x11a0/0x11a0
 ? uverbs_free_xrcd+0xd2/0x360
 uverbs_free_xrcd+0xd2/0x360
 remove_commit_idr_uobject+0x6d/0x110
 uverbs_cleanup_ucontext+0x2f0/0x730
 ? sched_clock_cpu+0x18/0x200
 ? uverbs_close_fd+0x1c0/0x1c0
 ib_uverbs_cleanup_ucontext.constprop.3+0x52/0x120
 ib_uverbs_close+0xf2/0x570
 ? ib_uverbs_remove_one+0xb50/0xb50
 ? ib_uverbs_remove_one+0xb50/0xb50
 __fput+0x2cd/0x8d0
 task_work_run+0xec/0x1d0
 do_exit+0x6a1/0x1520
 ? fsnotify_first_mark+0x220/0x220
 ? exit_notify+0x9f0/0x9f0
 ? entry_SYSCALL_64_fastpath+0x5/0x8b
 ? entry_SYSCALL_64_fastpath+0x5/0x8b
 ? trace_hardirqs_on_thunk+0x1a/0x1c
 ? time_hardirqs_on+0x27/0x670
 ? time_hardirqs_off+0x27/0x490
 ? syscall_return_slowpath+0x6c/0x460
 ? entry_SYSCALL_64_fastpath+0x5/0x8b
 do_group_exit+0xe8/0x380
 SyS_exit_group+0x1e/0x20
 entry_SYSCALL_64_fastpath+0x1e/0x8b
RIP: 0033:0x431ce9

Cc: syzkaller <syzkaller@googlegroups.com>
Cc: <stable@vger.kernel.org> # 4.11
Fixes: fd3c7904db ("IB/core: Change idr objects to use the new schema")
Reported-by: Noa Osherovich <noaos@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2018-02-15 15:31:27 -07:00
..
addr.c RDMA/core: Avoid copying ifindex twice 2017-12-18 15:37:10 -07:00
agent.c
agent.h
cache.c {net, IB}/mlx5: Manage port association for multiport RoCE 2018-01-08 11:42:22 -07:00
cgroup.c
cm_msgs.h
cm.c RDMA/cm: Fix access to uninitialized variable 2018-01-28 14:07:16 -07:00
cma_configfs.c IB/cma: use strlcpy() instead of strncpy() 2018-01-15 15:33:21 -07:00
cma.c RDMA/cma: Use existing netif_is_bond_master function 2018-01-28 14:07:16 -07:00
core_priv.h RDMA/core: Add resource tracking for create and destroy QPs 2018-01-29 20:21:39 -07:00
cq.c RDMA/core: Add resource tracking for create and destroy CQs 2018-01-29 20:21:40 -07:00
device.c RDMA/restrack: Add general infrastructure to track RDMA resources 2018-01-29 20:21:39 -07:00
fmr_pool.c infiniband: fix core/fmr_pool.c kernel-doc notation 2018-01-10 22:00:34 -07:00
iwcm.c RDMA/netlink: Fix general protection fault 2017-12-07 15:28:07 -05:00
iwcm.h
iwpm_msg.c RDMA/iwpm: Properly mark end of NL messages 2017-09-29 11:32:42 -04:00
iwpm_util.c RDMA/iwpm: Fix uninitialized error code in iwpm_send_mapinfo() 2017-12-13 10:55:49 -07:00
iwpm_util.h
mad_priv.h
mad_rmpp.c
mad_rmpp.h
mad.c drivers: infiniband: remove duplicate includes 2017-12-22 09:39:35 -07:00
Makefile RDMA/restrack: Add general infrastructure to track RDMA resources 2018-01-29 20:21:39 -07:00
mr_pool.c
multicast.c
netlink.c RDMA/netlink: Simplify code of autoload modules 2018-01-02 13:36:57 -07:00
nldev.c RDMA/nldev: missing error code in nldev_res_get_doit() 2018-02-01 15:24:32 -07:00
opa_smi.h
packer.c
rdma_core.c IB/uverbs: Fix unbalanced unlock on error path for rdma_explicit_destroy 2018-02-15 15:31:26 -07:00
rdma_core.h IB/core: Add new ioctl interface 2017-08-31 08:35:09 -04:00
restrack.c RDMA/restrack: Remove unimplemented XRCD object 2018-02-15 14:59:44 -07:00
roce_gid_mgmt.c {net, IB}/mlx5: Manage port association for multiport RoCE 2018-01-08 11:42:22 -07:00
rw.c IB/core: remove redundant check on prot_sg_cnt 2017-10-10 10:49:45 -04:00
sa_query.c IB/SA: Check dlid before SA agent queries for ClassPortInfo 2017-12-22 13:33:30 -07:00
sa.h
security.c Merge branch 'from-rc' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git 2017-12-27 21:50:46 -07:00
smi.c
smi.h
sysfs.c IB/core: Fix two kernel warnings triggered by rxe registration 2018-01-03 17:26:59 -07:00
ucm.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
ucma.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
ud_header.c
umem_odp.c RDMA/umem: Avoid partial declaration of non-static function 2017-11-10 13:02:12 -05:00
umem.c IB/umem: Fix use of npages/nmap fields 2017-12-18 15:37:06 -07:00
user_mad.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
uverbs_cmd.c RDMA/uverbs: Fix circular locking dependency 2018-02-15 15:31:27 -07:00
uverbs_ioctl_merge.c IB/uverbs: Fix method merging in uverbs_ioctl_merge 2018-02-15 14:59:45 -07:00
uverbs_ioctl.c IB/uverbs: Fix possible oops with duplicate ioctl attributes 2018-02-15 14:59:46 -07:00
uverbs_main.c RDMA/uverbs: Protect from command mask overflow 2018-02-15 15:31:26 -07:00
uverbs_marshall.c IB/core: Convert OPA AH to IB for Extended LIDs only 2017-11-13 15:53:57 -05:00
uverbs_std_types.c IB/uverbs: Use u64_to_user_ptr() not a union 2018-02-15 14:59:45 -07:00
uverbs.h IB/uverbs: Allow CQ moderation with modify CQ 2017-11-13 16:59:22 -05:00
verbs.c RDMA/core: Add resource tracking for create and destroy PDs 2018-01-29 20:21:40 -07:00