leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
da6629793a
linux/drivers/infiniband/core
History
Yamin Friedman da6629793a RDMA/core: Provide RDMA DIM support for ULPs 
Added the interface in the infiniband driver that applies the rdma_dim
adaptive moderation. There is now a special function for allocating an
ib_cq that uses rdma_dim.

Performance improvement (ConnectX-5 100GbE, x86) running FIO benchmark over
NVMf between two equal end-hosts with 56 cores across a Mellanox switch
using null_blk device:

READS without DIM:
blk size | BW       | IOPS | 99th percentile latency  | 99.99th latency
512B     | 3.8GiB/s | 7.7M | 1401  usec               | 2442  usec
4k       | 7.0GiB/s | 1.8M | 4817  usec               | 6587  usec
64k      | 10.7GiB/s| 175k | 9896  usec               | 10028 usec

IO WRITES without DIM:
blk size | BW       | IOPS | 99th percentile latency  | 99.99th latency
512B     | 3.6GiB/s | 7.5M | 1434  usec               | 2474  usec
4k       | 6.3GiB/s | 1.6M | 938   usec               | 1221  usec
64k      | 10.7GiB/s| 175k | 8979  usec               | 12780 usec

IO READS with DIM:
blk size | BW       | IOPS | 99th percentile latency  | 99.99th latency
512B     | 4GiB/s   | 8.2M | 816    usec              | 889   usec
4k       | 10.1GiB/s| 2.65M| 3359   usec              | 5080  usec
64k      | 10.7GiB/s| 175k | 9896   usec              | 10028 usec

IO WRITES with DIM:
blk size | BW       | IOPS  | 99th percentile latency | 99.99th latency
512B     | 3.9GiB/s | 8.1M  | 799   usec              | 922   usec
4k       | 9.6GiB/s | 2.5M  | 717   usec              | 1004  usec
64k      | 10.7GiB/s| 176k  | 8586  usec              | 12256 usec

The rdma_dim algorithm was designed to measure the effectiveness of
moderation on the flow in a general way and thus should be appropriate
for all RDMA storage protocols.

rdma_dim is configured to be the default option based on performance
improvement seen after extensive tests.

Signed-off-by: Yamin Friedman <yaminf@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-07-08 16:37:22 -03:00
..
addr.c 5.2 Merge Window second pull request 2019-05-14 20:56:31 -07:00
agent.c RDMA: Mark if destroy address handle is in a sleepable context 2018-12-19 16:28:03 -07:00
agent.h
cache.c IB/core, ipoib: Do not overreact to SM LID change event 2019-05-07 16:06:03 -03:00
cgroup.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 288 2019-06-05 17:36:37 +02:00
cm_msgs.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
cm.c IB/cm: Reduce dependency on gid attribute ndev check 2019-05-03 11:10:02 -03:00
cma_configfs.c RDMA/cma: Move cma module specific functions to cma_priv.h 2018-11-22 11:57:33 -07:00
cma_priv.h IB/cma: Define option to set ack timeout and pack tos_set 2019-02-08 16:14:21 -07:00
cma.c RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev 2019-05-03 11:10:03 -03:00
core_priv.h RDMA: Add NLDEV_GET_CHARDEV to allow char dev discovery and autoload 2019-06-18 22:41:05 -04:00
counters.c RDMA/nldev: Allow get counter mode through RDMA netlink 2019-07-05 10:22:55 -03:00
cq.c RDMA/core: Provide RDMA DIM support for ULPs 2019-07-08 16:37:22 -03:00
device.c RDMA/netlink: Implement counter dumpit calback 2019-07-05 10:22:55 -03:00
fmr_pool.c RDMA: Start use ib_device_ops 2018-12-12 07:40:16 -07:00
iwcm.c RDMA: Get rid of iw_cm_verbs 2019-05-03 10:56:56 -03:00
iwcm.h
iwpm_msg.c RDMA/iwpm: move kdoc comments to functions 2019-02-05 15:40:41 -07:00
iwpm_util.c netlink: make validation more configurable for future strictness 2019-04-27 17:07:21 -04:00
iwpm_util.h RDMA/IWPM: Support no port mapping requirements 2019-02-04 16:26:02 -07:00
mad_priv.h RDMA: Use __packed annotation instead of __attribute__ ((packed)) 2019-03-25 21:14:12 -03:00
mad_rmpp.c RDMA: Mark if destroy address handle is in a sleepable context 2018-12-19 16:28:03 -07:00
mad_rmpp.h
mad.c IB/MAD: Add SMP details to MAD tracing 2019-03-27 15:52:01 -03:00
Makefile RDMA/counter: Add set/clear per-port auto mode support 2019-07-05 10:22:54 -03:00
mr_pool.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
multicast.c IB/core, ipoib: Do not overreact to SM LID change event 2019-05-07 16:06:03 -03:00
netlink.c RDMA/cma: Remove CM_ID statistics provided by rdma-cm module 2019-02-05 15:30:33 -07:00
nldev.c RDMA/nldev: Allow get default counter statistics through RDMA netlink 2019-07-05 10:22:55 -03:00
opa_smi.h RDMA: Start use ib_device_ops 2018-12-12 07:40:16 -07:00
packer.c
rdma_core.c uverbs: Convert idr to XArray 2019-04-25 12:27:11 -03:00
rdma_core.h RDMA/core: Clear out the udata before error unwind 2019-05-27 14:35:26 -03:00
restrack.c RDMA/restrack: Make is_visible_in_pid_ns() as an API 2019-07-05 10:22:54 -03:00
restrack.h RDMA/restrack: Make is_visible_in_pid_ns() as an API 2019-07-05 10:22:54 -03:00
roce_gid_mgmt.c IB/core: Fix oops in netdev_next_upper_dev_rcu() 2018-12-12 12:14:49 -05:00
rw.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
sa_query.c 5.2 Merge Window pull request 2019-05-09 09:02:46 -07:00
sa.h RDMA/core: Annotate timeout as unsigned long 2018-10-16 13:34:01 -04:00
security.c RDMA/device: Consolidate ib_device per_port data into one place 2019-02-19 10:13:39 -07:00
smi.c
smi.h RDMA: Start use ib_device_ops 2018-12-12 07:40:16 -07:00
sysfs.c RDMA/nldev: Allow get default counter statistics through RDMA netlink 2019-07-05 10:22:55 -03:00
ucma.c RDMA: Report available cdevs through RDMA_NLDEV_CMD_GET_CHARDEV 2019-06-18 22:44:08 -04:00
ud_header.c
umem_odp.c RDMA/odp: Do not leak dma maps when working with huge pages 2019-06-20 21:52:47 -04:00
umem.c RDMA: Check umem pointer validity prior to release 2019-06-20 15:17:59 -04:00
user_mad.c docs: infiniband: convert docs to ReST and rename to *.rst 2019-06-25 10:23:09 -03:00
uverbs_cmd.c RDMA/uverbs: remove redundant assignment to variable ret 2019-07-04 14:06:47 -03:00
uverbs_ioctl.c RDMA/uverbs: Initialize udata struct on destroy flows 2019-05-02 17:07:02 -03:00
uverbs_main.c RDMA: Report available cdevs through RDMA_NLDEV_CMD_GET_CHARDEV 2019-06-18 22:44:08 -04:00
uverbs_marshall.c IB/cm: Replace members of sa_path_rec with 'struct sgid_attr *' 2018-06-25 14:19:57 -06:00
uverbs_std_types_counters.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_cq.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
uverbs_std_types_device.c IB/uverbs: Fix ioctl query port to consider device disassociation 2019-01-25 11:58:06 -07:00
uverbs_std_types_dm.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_flow_action.c IB: When attrs.udata/ufile is available use that instead of uobject 2019-04-08 13:05:25 -03:00
uverbs_std_types_mr.c Linux 5.2-rc6 2019-06-28 21:18:23 -03:00
uverbs_std_types.c IB: Remove 'uobject->context' dependency in object destroy APIs 2019-04-01 14:59:35 -03:00
uverbs_uapi.c RDMA: Move driver_id into struct ib_device_ops 2019-06-10 16:56:02 -03:00
uverbs.h uverbs: Convert idr to XArray 2019-04-25 12:27:11 -03:00
verbs.c RDMA/counter: Add "auto" configuration mode support 2019-07-05 10:22:54 -03:00