linux/drivers/infiniband/ulp/ipoib
Feras Daoud 3e31a490e0 IB/ipoib: Fix deadlock between ipoib_stop and mcast join flow
Before calling ipoib_stop, rtnl_lock should be taken, then
the flow clears the IPOIB_FLAG_ADMIN_UP and IPOIB_FLAG_OPER_UP
flags, and waits for mcast completion if IPOIB_MCAST_FLAG_BUSY
is set.

On the other hand, the flow of multicast join task initializes
a mcast completion, sets the IPOIB_MCAST_FLAG_BUSY and calls
ipoib_mcast_join. If IPOIB_FLAG_OPER_UP flag is not set, this
call returns EINVAL without setting the mcast completion and
leads to a deadlock.

    ipoib_stop                          |
        |                               |
    clear_bit(IPOIB_FLAG_ADMIN_UP)      |
        |                               |
    Context Switch                      |
        |                       ipoib_mcast_join_task
        |                               |
        |                       spin_lock_irq(lock)
        |                               |
        |                       init_completion(mcast)
        |                               |
        |                       set_bit(IPOIB_MCAST_FLAG_BUSY)
        |                               |
        |                       Context Switch
        |                               |
    clear_bit(IPOIB_FLAG_OPER_UP)       |
        |                               |
    spin_lock_irqsave(lock)             |
        |                               |
    Context Switch                      |
        |                       ipoib_mcast_join
        |                       return (-EINVAL)
        |                               |
        |                       spin_unlock_irq(lock)
        |                               |
        |                       Context Switch
        |                               |
    ipoib_mcast_dev_flush               |
    wait_for_completion(mcast)          |

ipoib_stop will wait for mcast completion for ever, and will
not release the rtnl_lock. As a result panic occurs with the
following trace:

    [13441.639268] Call Trace:
    [13441.640150]  [<ffffffff8168b579>] schedule+0x29/0x70
    [13441.641038]  [<ffffffff81688fc9>] schedule_timeout+0x239/0x2d0
    [13441.641914]  [<ffffffff810bc017>] ? complete+0x47/0x50
    [13441.642765]  [<ffffffff810a690d>] ? flush_workqueue_prep_pwqs+0x16d/0x200
    [13441.643580]  [<ffffffff8168b956>] wait_for_completion+0x116/0x170
    [13441.644434]  [<ffffffff810c4ec0>] ? wake_up_state+0x20/0x20
    [13441.645293]  [<ffffffffa05af170>] ipoib_mcast_dev_flush+0x150/0x190 [ib_ipoib]
    [13441.646159]  [<ffffffffa05ac967>] ipoib_ib_dev_down+0x37/0x60 [ib_ipoib]
    [13441.647013]  [<ffffffffa05a4805>] ipoib_stop+0x75/0x150 [ib_ipoib]

Fixes: 08bc327629 ("IB/ipoib: fix for rare multicast join race condition")
Signed-off-by: Feras Daoud <ferasda@mellanox.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Signed-off-by: Doug Ledford <dledford@redhat.com>
2017-04-21 11:45:55 -04:00
..
ipoib_cm.c IB/IPoIB: Use defined function for netdev_priv function 2017-04-20 15:19:44 -04:00
ipoib_ethtool.c IB/IPoIB: Use defined function for netdev_priv function 2017-04-20 15:19:44 -04:00
ipoib_fs.c IB/IPoIB: Use defined function for netdev_priv function 2017-04-20 15:19:44 -04:00
ipoib_ib.c IB/ipoib: Update broadcast object if PKey value was changed in index 0 2017-04-21 11:45:55 -04:00
ipoib_main.c IB/IPoIB: Support acceleration options callbacks 2017-04-20 15:19:44 -04:00
ipoib_multicast.c IB/ipoib: Fix deadlock between ipoib_stop and mcast join flow 2017-04-21 11:45:55 -04:00
ipoib_netlink.c IB/IPoIB: Support acceleration options callbacks 2017-04-20 15:19:44 -04:00
ipoib_verbs.c IB/IPoIB: Support acceleration options callbacks 2017-04-20 15:19:44 -04:00
ipoib_vlan.c IB/IPoIB: Support acceleration options callbacks 2017-04-20 15:19:44 -04:00
ipoib.h IB/IPoIB: Support acceleration options callbacks 2017-04-20 15:19:44 -04:00
Kconfig kconfig: rename CONFIG_EMBEDDED to CONFIG_EXPERT 2011-01-20 17:02:05 -08:00
Makefile IB/ipoib: Add rtnl_link_ops support 2012-09-20 16:49:17 -04:00