leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity

IB/mad: Ensure fairness in ib_mad_completion_handler

Browse Source 
It was found that when a process was rapidly sending MADs other processes could
be hung in their unregister calls.

This would happen when process A was injecting packets fast enough that the
single threaded workqueue was never exiting ib_mad_completion_handler.
Therefore when process B called flush_workqueue via the unregister call it
would hang until process A stopped sending MADs.

The fix is to periodically reschedule ib_mad_completion_handler after
processing a large number of completions.  The number of completions chosen was
decided based on the defaults for the recv queue size.  However, it was kept
fixed such that increasing those queue sizes would not adversely affect
fairness in the future.

Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Dean Luick <dean.luick@intel.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>
This commit is contained in:
Dean Luick 2015-12-10 16:52:30 -05:00 committed by Doug Ledford
parent 051f263098
commit 0d6ed314de
1 changed files with 18 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
drivers/infiniband/core/mad.c
View File

@ -61,6 +61,18 @@ MODULE_PARM_DESC(send_queue_size, "Size of send queue in number of work requests
module_param_named(recv_queue_size, mad_recvq_size, int, 0444);
MODULE_PARM_DESC(recv_queue_size, "Size of receive queue in number of work requests");
/*
* Define a limit on the number of completions which will be processed by the
* worker thread in a single work item. This ensures that other work items
* (potentially from other users) are processed fairly.
*
* The number of completions was derived from the default queue sizes above.
* We use a value which is double the larger of the 2 queues (receive @ 512)
* but keep it fixed such that an increase in that value does not introduce
* unfairness.
*/
#define MAD_COMPLETION_PROC_LIMIT 1024
static struct list_head ib_mad_port_list;
static u32 ib_mad_client_id = 0;
@ -2555,6 +2567,7 @@ static void ib_mad_completion_handler(struct work_struct *work)
{
struct ib_mad_port_private *port_priv;
struct ib_wc wc;
int count = 0;
port_priv = container_of(work, struct ib_mad_port_private, work);
ib_req_notify_cq(port_priv->cq, IB_CQ_NEXT_COMP);
@ -2574,6 +2587,11 @@ static void ib_mad_completion_handler(struct work_struct *work)
}
} else
mad_error_handler(port_priv, &wc);
if (++count > MAD_COMPLETION_PROC_LIMIT) {
queue_work(port_priv->wq, &port_priv->work);
break;
}
}
}