linux/drivers/infiniband/core
Chuck Lever 3e5901cbfc RDMA/core: Trace points for diagnosing completion queue issues
Sample trace events:

   kworker/u29:0-300   [007]   120.042217: cq_alloc:             cq.id=4 nr_cqe=161 comp_vector=2 poll_ctx=WORKQUEUE
          <idle>-0     [002]   120.056292: cq_schedule:          cq.id=4
    kworker/2:1H-482   [002]   120.056402: cq_process:           cq.id=4 wake-up took 109 [us] from interrupt
    kworker/2:1H-482   [002]   120.056407: cq_poll:              cq.id=4 requested 16, returned 1
          <idle>-0     [002]   120.067503: cq_schedule:          cq.id=4
    kworker/2:1H-482   [002]   120.067537: cq_process:           cq.id=4 wake-up took 34 [us] from interrupt
    kworker/2:1H-482   [002]   120.067541: cq_poll:              cq.id=4 requested 16, returned 1
          <idle>-0     [002]   120.067657: cq_schedule:          cq.id=4
    kworker/2:1H-482   [002]   120.067672: cq_process:           cq.id=4 wake-up took 15 [us] from interrupt
    kworker/2:1H-482   [002]   120.067674: cq_poll:              cq.id=4 requested 16, returned 1

 ...

         systemd-1     [002]   122.392653: cq_schedule:          cq.id=4
    kworker/2:1H-482   [002]   122.392688: cq_process:           cq.id=4 wake-up took 35 [us] from interrupt
    kworker/2:1H-482   [002]   122.392693: cq_poll:              cq.id=4 requested 16, returned 16
    kworker/2:1H-482   [002]   122.392836: cq_poll:              cq.id=4 requested 16, returned 16
    kworker/2:1H-482   [002]   122.392970: cq_poll:              cq.id=4 requested 16, returned 16
    kworker/2:1H-482   [002]   122.393083: cq_poll:              cq.id=4 requested 16, returned 16
    kworker/2:1H-482   [002]   122.393195: cq_poll:              cq.id=4 requested 16, returned 3

Several features to note in this output:
 - The WCE count and context type are reported at allocation time
 - The CPU and kworker for each CQ is evident
 - The CQ's restracker ID is tagged on each trace event
 - CQ poll scheduling latency is measured
 - Details about how often single completions occur versus multiple
   completions are evident
 - The cost of the ULP's completion handler is recorded

Link: https://lore.kernel.org/r/20191218201815.30584.3481.stgit@manet.1015granger.net
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-by: Parav Pandit <parav@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2020-01-07 16:10:53 -04:00
..
addr.c RDMA/netlink: Do not always generate an ACK for some netlink operations 2020-01-03 16:02:32 -04:00
agent.c RDMA: Mark if destroy address handle is in a sleepable context 2018-12-19 16:28:03 -07:00
agent.h
cache.c IB/core: Do not notify GID change event of an unregistered device 2019-10-22 16:58:22 -03:00
cgroup.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 288 2019-06-05 17:36:37 +02:00
cm_msgs.h RDMA/cm: Delete unused CM LAP functions 2020-01-03 21:06:21 -04:00
cm.c RDMA/cm: Delete unused CM ARP functions 2020-01-03 21:07:21 -04:00
cma_configfs.c RDMA/core: Annotate destroy of mutex to ensure that it is released as unlocked 2019-07-25 12:07:14 -03:00
cma_priv.h IB/cma: Define option to set ack timeout and pack tos_set 2019-02-08 16:14:21 -07:00
cma_trace.c RDMA/cma: Add trace points in RDMA Connection Manager 2020-01-07 16:10:53 -04:00
cma_trace.h RDMA/cma: Add trace points in RDMA Connection Manager 2020-01-07 16:10:53 -04:00
cma.c RDMA/cma: Add trace points in RDMA Connection Manager 2020-01-07 16:10:53 -04:00
core_priv.h RDMA subsystem updates for 5.5 2019-11-27 10:17:28 -08:00
counters.c RDMA/counter: Prevent auto-binding a QP which are not tracked with res 2019-12-12 15:38:15 -05:00
cq.c RDMA/core: Trace points for diagnosing completion queue issues 2020-01-07 16:10:53 -04:00
device.c hmm related patches for 5.5 2019-11-30 10:33:14 -08:00
fmr_pool.c RDMA: Delete DEBUG code 2019-08-20 13:27:53 -04:00
ib_core_uverbs.c IB/core: Introduce rdma_user_mmap_entry_insert_range() API 2019-12-12 16:55:36 -05:00
iwcm.c RDMA/iwcm: move iw_rem_ref() calls out of spinlock 2019-10-18 14:40:01 -04:00
iwcm.h
iwpm_msg.c RDMA/iwpm: Delete unnecessary checks before the macro call "dev_kfree_skb" 2019-08-27 13:09:23 -03:00
iwpm_util.c RDMA/iwpm: Delete unnecessary checks before the macro call "dev_kfree_skb" 2019-08-27 13:09:23 -03:00
iwpm_util.h infiniband: fix core/ipwm_util.h kernel-doc warnings 2019-10-22 14:45:31 -03:00
mad_priv.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
mad_rmpp.c RDMA: Mark if destroy address handle is in a sleepable context 2018-12-19 16:28:03 -07:00
mad_rmpp.h
mad.c RDMA: Change MAD processing function to remove extra casting and parameter 2019-11-12 20:20:15 -04:00
Makefile RDMA/core: Trace points for diagnosing completion queue issues 2020-01-07 16:10:53 -04:00
mr_pool.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
multicast.c IB/core, ipoib: Do not overreact to SM LID change event 2019-05-07 16:06:03 -03:00
netlink.c IB/core: Avoid deadlock during netlink message handling 2019-10-24 20:49:37 -03:00
nldev.c RDMA subsystem updates for 5.5 2019-11-27 10:17:28 -08:00
opa_smi.h RDMA: Start use ib_device_ops 2018-12-12 07:40:16 -07:00
packer.c
rdma_core.c RDMA/core: Create mmap database and cookie helper functions 2019-11-06 13:08:00 -04:00
rdma_core.h RDMA/core: Clear out the udata before error unwind 2019-05-27 14:35:26 -03:00
restrack.c RDMA/restrack: Remove PID namespace support 2019-10-23 15:58:31 -03:00
restrack.h RDMA/restrack: Remove PID namespace support 2019-10-23 15:58:31 -03:00
roce_gid_mgmt.c drivers: use in_dev_for_each_ifa_rtnl/rcu 2019-06-02 18:06:26 -07:00
rw.c RDMA/rw: Support threshold for registration vs scattering to local pages 2019-10-22 14:26:52 -03:00
sa_query.c RDMA/netlink: Do not always generate an ACK for some netlink operations 2020-01-03 16:02:32 -04:00
sa.h RDMA/core: Annotate timeout as unsigned long 2018-10-16 13:34:01 -04:00
security.c IB/core: Fix wrong iterating on ports 2019-10-04 15:50:27 -03:00
smi.c
smi.h RDMA: Start use ib_device_ops 2018-12-12 07:40:16 -07:00
sysfs.c RDMA: Change MAD processing function to remove extra casting and parameter 2019-11-12 20:20:15 -04:00
trace.c RDMA/core: Trace points for diagnosing completion queue issues 2020-01-07 16:10:53 -04:00
ucma.c RDMA: Report available cdevs through RDMA_NLDEV_CMD_GET_CHARDEV 2019-06-18 22:44:08 -04:00
ud_header.c
umem_odp.c IB/core: Fix ODP with IB_ACCESS_HUGETLB handling 2020-01-03 17:00:14 -04:00
umem.c IB/umem: remove the dmasync argument to ib_umem_get 2019-11-17 10:37:00 -04:00
user_mad.c Merge branch 'odp_fixes' into rdma.git for-next 2019-08-21 14:10:36 -03:00
uverbs_cmd.c RDMA/core: Create mmap database and cookie helper functions 2019-11-06 13:08:00 -04:00
uverbs_ioctl.c RDMA/uverbs: Add a check for uverbs_attr_get to uverbs_copy_to_struct_or_zero 2019-10-22 16:10:03 -03:00
uverbs_main.c compat_ioctl: remove most of fs/compat_ioctl.c 2019-12-01 13:46:15 -08:00
uverbs_marshall.c
uverbs_std_types_counters.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_cq.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
uverbs_std_types_device.c IB/uverbs: Fix ioctl query port to consider device disassociation 2019-01-25 11:58:06 -07:00
uverbs_std_types_dm.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_flow_action.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_mr.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
uverbs_std_types.c IB: Remove 'uobject->context' dependency in object destroy APIs 2019-04-01 14:59:35 -03:00
uverbs_uapi.c RDMA: Move driver_id into struct ib_device_ops 2019-06-10 16:56:02 -03:00
uverbs.h RDMA/uverbs: Prevent potential underflow 2019-10-22 15:05:36 -03:00
verbs.c RDMA/core: Trace points for diagnosing completion queue issues 2020-01-07 16:10:53 -04:00