Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-24 21:21:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
78c91ae2c6
linux/include/net
History
Philo Lu 78c91ae2c6 ipv4/udp: Add 4-tuple hash for connected socket 
Currently, the udp_table has two hash table, the port hash and portaddr
hash. Usually for UDP servers, all sockets have the same local port and
addr, so they are all on the same hash slot within a reuseport group.

In some applications, UDP servers use connect() to manage clients. In
particular, when firstly receiving from an unseen 4 tuple, a new socket
is created and connect()ed to the remote addr:port, and then the fd is
used exclusively by the client.

Once there are connected sks in a reuseport group, udp has to score all
sks in the same hash2 slot to find the best match. This could be
inefficient with a large number of connections, resulting in high
softirq overhead.

To solve the problem, this patch implement 4-tuple hash for connected
udp sockets. During connect(), hash4 slot is updated, as well as a
corresponding counter, hash4_cnt, in hslot2. In __udp4_lib_lookup(),
hslot4 will be searched firstly if the counter is non-zero. Otherwise,
hslot2 is used like before. Note that only connected sockets enter this
hash4 path, while un-connected ones are not affected.

hlist_nulls is used for hash4, because we probably move to another hslot
wrongly when lookup with concurrent rehash. Then we check nulls at the
list end to see if we should restart lookup. Because udp does not use
SLAB_TYPESAFE_BY_RCU, we don't need to touch sk_refcnt when lookup.

Stress test results (with 1 cpu fully used) are shown below, in pps:
(1) _un-connected_ socket as server
    [a] w/o hash4: 1,825176
    [b] w/  hash4: 1,831750 (+0.36%)

(2) 500 _connected_ sockets as server
    [c] w/o hash4:   290860 (only 16% of [a])
    [d] w/  hash4: 1,889658 (+3.1% compared with [b])

With hash4, compute_score is skipped when lookup, so [d] is slightly
better than [b].

Co-developed-by: Cambda Zhu <cambda@linux.alibaba.com>
Signed-off-by: Cambda Zhu <cambda@linux.alibaba.com>
Co-developed-by: Fred Chen <fred.cc@alibaba-inc.com>
Signed-off-by: Fred Chen <fred.cc@alibaba-inc.com>
Co-developed-by: Yubing Qiu <yubing.qiuyubing@alibaba-inc.com>
Signed-off-by: Yubing Qiu <yubing.qiuyubing@alibaba-inc.com>
Signed-off-by: Philo Lu <lulie@linux.alibaba.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2024-11-18 11:56:21 +00:00
..
9p netfs, 9p: Implement helpers for new write code 2024-05-01 18:07:37 +01:00
bluetooth Bluetooth: MGMT: Add initial implementation of MGMT_OP_HCI_CMD_SYNC 2024-11-14 15:41:31 -05:00
caif caif: Remove unused cfsrvl_getphyid 2024-10-08 15:33:49 -07:00
iucv s390/iucv: Correct spelling in iucv.h 2024-08-26 09:37:22 -07:00
libeth libeth: add Tx buffer completion helpers 2024-09-09 13:15:37 -07:00
mana net: mana: Enable debugfs files for MANA device 2024-10-09 13:42:04 +01:00
netfilter netfilter: nf_tables: allocate element update information dynamically 2024-11-14 13:05:49 +01:00
netns xfrm: Add an inbound percpu state cache. 2024-10-29 11:56:18 +01:00
nfc net: nfc: Propagate ISO14443 type A target ATS to userspace via netlink 2024-11-07 10:21:58 +01:00
page_pool memory-provider: dmabuf devmem memory provider 2024-09-11 20:44:31 -07:00
phonet phonet: Convert phonet_routes.lock to spinlock_t. 2024-10-24 16:03:40 +02:00
sctp sctp: Correct spelling in headers 2024-08-26 09:37:23 -07:00
tc_act net/sched: Retire ipt action 2024-01-02 12:41:16 +00:00
6lowpan.h
act_api.h net/sched: act_api: unexport tcf_action_dump_1() 2024-10-23 11:43:47 +02:00
addrconf.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
af_ieee802154.h
af_rxrpc.h rxrpc, afs: Allow afs to pin rxrpc_peer objects 2023-12-24 15:22:50 +00:00
af_unix.h af_unix: Remove U_LOCK_GC_LISTENER. 2024-06-25 11:10:18 +02:00
af_vsock.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-08-15 17:18:52 -07:00
ah.h
amt.h
arp.h neighbour: switch to standard rcu, instead of rcu_bh 2023-03-21 21:32:18 -07:00
atmclip.h
ax25.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-05-14 10:53:19 -07:00
ax88796.h
bareudp.h
bond_3ad.h bonding: Correct spelling in headers 2024-08-26 09:37:22 -07:00
bond_alb.h bonding: Correct spelling in headers 2024-08-26 09:37:22 -07:00
bond_options.h bonding: add ns target multicast address to slave device 2024-11-14 11:16:28 +01:00
bonding.h bonding: change ipsec_lock from spin lock to mutex 2024-08-27 13:11:37 -07:00
bpf_sk_storage.h
busy_poll.h net: Add control functions for irq suspension 2024-11-11 18:45:06 -08:00
calipso.h move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
cfg80211-wext.h
cfg80211.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-10-31 18:10:07 -07:00
cfg802154.h mac802154: fix llsec key resources release in mac802154_llsec_key_del 2024-03-06 21:01:26 +01:00
checksum.h net: checksum: Move from32to16() to generic header 2024-10-30 15:29:59 +01:00
cipso_ipv4.h move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
cls_cgroup.h
codel_impl.h
codel_qdisc.h
codel.h codel: fix kernel-doc notation warnings 2023-07-14 20:39:29 -07:00
compat.h
datalink.h net: datalink: Remove unused declarations 2023-07-27 17:17:32 -07:00
dcbevent.h
dcbnl.h net: dcb: add helper functions to retrieve PCP and DSCP rewrite maps 2023-01-20 09:33:22 +00:00
devlink.h devlink: remove unused devlink_resource_register() 2024-10-29 16:52:57 -07:00
dropreason-core.h net: ip: make ip_mkroute_input/__mkroute_input return drop reasons 2024-11-12 11:24:51 +01:00
dropreason.h net: openvswitch: add last-action drop reason 2023-08-14 08:01:06 +01:00
dsa_stubs.h net: dsa: Use conduit and user terms 2023-10-24 13:08:14 -07:00
dsa.h net: dsa: remove obsolete phylink dsa_switch operations 2024-10-07 16:23:10 -07:00
dscp.h net: add IEEE 802.1q specific helpers 2024-05-08 10:35:09 +01:00
dsfield.h
dst_cache.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
dst_metadata.h net: Silence false field-spanning write warning in metadata_dst memcpy 2024-08-20 15:22:17 -07:00
dst_ops.h net: fix __dst_negative_advice() race 2024-05-29 17:34:49 -07:00
dst.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
eee.h net: simplify eeecfg_mac_can_tx_lpi 2024-11-13 18:49:50 -08:00
erspan.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
esp.h
espintcp.h inet: preserve const qualifier in inet_csk() 2024-04-01 21:27:08 -07:00
ethoc.h
failover.h
fib_notifier.h net: do not acquire rtnl in fib_seq_sum() 2024-10-11 15:35:05 -07:00
fib_rules.h fib: rules: use READ_ONCE()/WRITE_ONCE() on ops->fib_rules_seq 2024-10-11 15:35:05 -07:00
firewire.h
flow_dissector.h flow_dissector: cleanup FLOW_DISSECTOR_KEY_ENC_FLAGS 2024-07-15 09:14:39 -07:00
flow_offload.h net: sched: propagate "skip_sw" flag to struct flow_cls_common_offload 2024-10-30 17:33:53 -07:00
flow.h inet: shrink struct flowi_common 2023-11-02 09:31:02 +01:00
fou.h bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs 2023-04-12 16:40:39 -07:00
fq_impl.h
fq.h net: fq: Remove unused typedef fq_flow_get_default_t 2023-08-08 15:58:23 -07:00
garp.h
gen_stats.h
genetlink.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-10-21 09:14:18 +02:00
geneve.h
gre.h ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
gro_cells.h
gro.h net: gro: fix napi_gro_cb zeroed alignment 2024-05-14 10:49:50 -07:00
gso.h net: move gso declarations and functions to their own files 2023-06-10 00:11:41 -07:00
gtp.h gtp: properly parse extension headers 2024-05-07 01:35:55 +02:00
gue.h
handshake.h net/handshake: Add helpers for parsing incoming TLS Alerts 2023-07-28 14:07:59 -07:00
hotdata.h net: move sysctl_mem_pcpu_rsv to net_hotdata 2024-04-30 18:46:52 -07:00
hwbm.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
icmp.h
ieee8021q.h net: add IEEE 802.1q specific helpers 2024-05-08 10:35:09 +01:00
ieee80211_radiotap.h wireless fixes for v6.12-rc5 2024-10-25 10:44:41 +01:00
ieee802154_netdev.h mac802154: Handle association requests from peers 2023-11-20 11:43:03 +01:00
if_inet6.h ipv6: anycast: complete RCU handling of struct ifacaddr6 2024-02-26 18:40:34 -08:00
ife.h
inet6_connection_sock.h
inet6_hashtables.h inet6: constify 'struct net' parameter of various lookup helpers 2024-08-05 16:27:26 -07:00
inet_common.h net: change proto and proto_ops accept type 2024-05-13 18:19:09 -06:00
inet_connection_sock.h tcp: add a fast path in tcp_delack_timer() 2024-10-04 15:34:40 -07:00
inet_dscp.h
inet_ecn.h
inet_frag.h net: Rename mono_delivery_time to tstamp_type for scalabilty 2024-05-23 14:14:23 -07:00
inet_hashtables.h inet: constify 'struct net' parameter of various lookup helpers 2024-08-05 16:22:45 -07:00
inet_sock.h net: add TIME_WAIT logic to sk_to_full_sk() 2024-10-14 17:39:36 -07:00
inet_timewait_sock.h tcp: remove volatile qualifier on tw_substate 2024-08-28 17:08:16 -07:00
inetpeer.h
ioam6.h net: ioam6: multicast event 2024-02-28 11:19:41 +00:00
ip6_checksum.h
ip6_fib.h ipv6: use READ_ONCE()/WRITE_ONCE() on fib6_table->fib_seq 2024-10-11 15:35:05 -07:00
ip6_route.h ipv6: fix source address selection with route leak 2024-07-14 07:34:16 -07:00
ip6_tunnel.h ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
ip_fib.h net: ip: make fib_validate_source() support drop reasons 2024-11-12 11:24:50 +01:00
ip_tunnels.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-10-31 18:10:07 -07:00
ip_vs.h ipvs: Correct spelling in comments 2023-04-22 01:39:41 +02:00
ip.h ipv4: Switch inet_addr_hash() to less predictable hash. 2024-10-23 13:17:35 +02:00
ipcomp.h
ipconfig.h
ipv6_frag.h
ipv6_stubs.h ipv6: udp: constify 'struct net' parameter of socket lookups 2024-08-05 16:27:26 -07:00
ipv6.h ipv6: Correct spelling in ipv6.h 2024-08-26 09:37:22 -07:00
iw_handler.h Revert "wifi: cfg80211: unexport wireless_nlevent_flush()" 2024-10-09 08:53:01 +02:00
kcm.h kcm: Serialise kcm_sendmsg() for the same socket. 2024-08-19 18:36:12 -07:00
l3mdev.h net: Remove likely from l3mdev_master_ifindex_by_index 2024-10-10 11:57:34 +02:00
lag.h
lapb.h
llc_c_ac.h net: llc: Remove unused function declarations 2023-08-04 15:33:17 -07:00
llc_c_ev.h net: llc: Remove unused function declarations 2023-08-04 15:33:17 -07:00
llc_c_st.h llc: Constify struct llc_conn_state_trans 2024-07-15 08:51:01 -07:00
llc_conn.h llc: Check netns in llc_estab_match() and llc_listener_match(). 2023-07-20 10:46:28 +02:00
llc_if.h
llc_pdu.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
llc_s_ac.h
llc_s_ev.h
llc_s_st.h llc: Constify struct llc_sap_state_trans 2024-07-15 08:51:19 -07:00
llc_sap.h
llc.h
lwtunnel.h lwt: Check LWTUNNEL_XMIT_CONTINUE strictly 2023-08-18 16:05:26 +02:00
mac80211.h wifi: mac80211: fix description of ieee80211_set_active_links() for new sequence 2024-11-07 14:38:01 +01:00
mac802154.h move asm/unaligned.h to linux/unaligned.h 2024-10-02 17:23:23 -04:00
macsec.h macsec: Enable devices to advertise whether they update sk_buff md_dst during offloads 2024-04-25 08:20:54 -07:00
mctp.h net: mctp: Expose transport binding identifier via IFLA attribute 2024-11-09 09:04:54 -08:00
mctpdevice.h net: mctp: Expose transport binding identifier via IFLA attribute 2024-11-09 09:04:54 -08:00
mip6.h
mld.h
mpls_iptunnel.h
mpls.h
mptcp.h mptcp: fallback to TCP after SYN+MPC drops 2024-09-11 15:57:50 -07:00
mrp.h
ncsi.h
ndisc.h ipv6: eliminate ndisc_ops_is_useropt() 2024-08-12 17:23:57 -07:00
neighbour_tables.h neighbour: Create netdev->neighbour association 2024-11-09 13:22:57 -08:00
neighbour.h neighbour: Create netdev->neighbour association 2024-11-09 13:22:57 -08:00
net_debug.h socket: Print pf->create() when it does not clear sock->sk on failure. 2024-10-29 16:31:23 -07:00
net_failover.h
net_namespace.h rtnetlink: Add per-netns RTNL. 2024-10-08 15:16:59 +02:00
net_ratelimit.h
net_shaper.h net-shapers: implement NL get operation 2024-10-10 08:30:22 -07:00
net_trackers.h
netdev_queues.h net: qstat: extend kdoc about get_base_stats 2024-06-01 15:11:52 -07:00
netdev_rx_queue.h memory-provider: fix compilation issue without SYSFS 2024-09-12 21:00:26 -07:00
netevent.h
netkit.h bpf, netkit: Add indirect call wrapper for fetching peer dev 2023-11-20 10:15:16 -08:00
netlabel.h netlabel: document doi_remove field of struct netlbl_calipso_ops 2024-10-31 19:00:47 -07:00
netlink.h net: netlink: add nla_get_*_default() accessors 2024-11-11 10:32:06 -08:00
netmem.h page_pool: devmem support 2024-09-11 20:44:31 -07:00
netprio_cgroup.h
netrom.h
nexthop.h net: nexthop: Increase weight to u16 2024-08-12 17:50:34 -07:00
nl802154.h ieee802154: Correct spelling in nl802154.h 2024-08-30 22:30:55 +02:00
nsh.h net: NSH: fix kernel-doc notation warning 2023-07-14 20:39:29 -07:00
p8022.h net: 802: Remove unused function declarations 2023-08-04 15:33:50 -07:00
pfcp.h pfcp: always set pfcp metadata 2024-04-01 10:49:28 +01:00
pie.h pie: fix kernel-doc notation warning 2023-07-14 20:39:30 -07:00
ping.h net/ipv4: ping_group_range: allow GID from 2147483648 to 4294967294 2023-06-02 09:55:22 +01:00
pkt_cls.h net: sched: propagate "skip_sw" flag to struct flow_cls_common_offload 2024-10-30 17:33:53 -07:00
pkt_sched.h net/sched: Add helper macros with module names 2024-02-02 10:57:55 -08:00
pptp.h
proto_memory.h net: move sysctl_mem_pcpu_rsv to net_hotdata 2024-04-30 18:46:52 -07:00
protocol.h ipv6: move tcp_ipv6_hash_secret and udp_ipv6_hash_secret to net_hotdata 2024-03-07 21:12:43 -08:00
psample.h net: psample: fix flag being set in wrong skb 2024-07-11 18:11:31 -07:00
psnap.h
raw.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2023-04-06 12:01:20 -07:00
rawv6.h ipv6: raw: constify raw_v6_match() socket argument 2023-03-17 08:56:37 +00:00
red.h net: sched: Correct spelling in headers 2024-08-26 09:37:23 -07:00
regulatory.h net: Correct spelling in headers 2024-08-26 09:37:23 -07:00
request_sock.h tcp: move reqsk_alloc() to inet_connection_sock.c 2024-06-06 15:18:04 +02:00
rose.h
route.h net: ip: make ip_route_use_hint() return drop reasons 2024-11-12 11:24:51 +01:00
rpl.h ipv6: rpl: Remove pskb(_may)?_pull() in ipv6_rpl_srh_rcv(). 2023-06-19 11:32:58 -07:00
rps.h net: rps: add rps_input_queue_head_add() helper 2024-04-01 11:28:32 +01:00
rsi_91x.h rsi: remove kernel-doc comment marker 2023-07-14 20:39:30 -07:00
rstreason.h tcp: rstreason: introduce SK_RST_REASON_TCP_DISCONNECT_WITH_DATA for active reset 2024-08-07 10:24:46 +01:00
rtnetlink.h rtnetlink: Register rtnl_dellink() and rtnl_setlink() with RTNL_FLAG_DOIT_PERNET_WIP. 2024-11-11 17:26:52 -08:00
rtnh.h
sch_generic.h net/sched: accept TCA_STAB only for root qdisc 2024-10-08 15:38:56 -07:00
scm.h af_unix: Add dead flag to struct scm_fp_list. 2024-05-10 18:52:45 -07:00
secure_seq.h
seg6_hmac.h ipv6: sr: restruct ifdefines 2024-05-30 18:29:38 -07:00
seg6_local.h seg6: Use nested-BH locking for seg6_bpf_srh_states. 2024-06-24 16:41:23 -07:00
seg6.h ipv6: sr: restruct ifdefines 2024-05-30 18:29:38 -07:00
selftests.h
slhc_vj.h
smc.h net/smc: add operations to merge sndbuf with peer DMB 2024-04-30 13:24:48 +02:00
snmp.h
sock_reuseport.h net: core: annotate socks of struct sock_reuseport with __counted_by 2024-08-02 17:16:59 -07:00
sock.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-10-25 09:08:22 +02:00
Space.h net: appletalk: remove cops support 2023-10-04 11:49:20 -07:00
stp.h
strparser.h
switchdev.h net: bridge: switchdev: Skip MDB replays of deferred events on offload 2024-02-16 09:36:37 +00:00
tc_wrapper.h Merge branch 'x86/bugs' into x86/core, to pick up pending changes before dependent patches 2024-02-14 10:49:37 +01:00
tcp_ao.h net/tcp: Add missing lockdep annotations for TCP-AO hlist traversals 2024-11-03 12:10:11 -08:00
tcp_states.h tcp: Dump bound-only sockets in inet_diag. 2023-12-04 14:45:26 -08:00
tcp.h tcp: add more warn of socket in tcp_send_loss_probe() 2024-10-30 13:26:55 +00:00
tcx.h bpf: Fix too early release of tcx_entry 2024-07-08 14:07:31 -07:00
timewait_sock.h tcp: get rid of twsk_unique() 2024-05-09 20:25:55 -07:00
tipc.h
tls_prot.h net/tls: Add TLS Alert definitions 2023-07-28 14:07:59 -07:00
tls_toe.h
tls.h bpf: Add sk_is_inet and IS_ICSK check in tls_sw_has_ctx_tx/rx 2024-11-06 11:08:56 -08:00
transp_v6.h inet6: Remove unused function declaration udpv6_connect() 2023-08-01 15:06:27 -07:00
tso.h
tun_proto.h
udp_tunnel.h ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
udp.h ipv4/udp: Add 4-tuple hash for connected socket 2024-11-18 11:56:21 +00:00
udplite.h udplite: fix various data-races 2023-09-14 16:16:36 +02:00
vsock_addr.h
vxlan.h vxlan: add support for flowlabel inherit 2023-11-16 22:33:31 +00:00
wext.h
x25.h x25: Correct spelling in x25.h 2024-08-26 09:37:23 -07:00
x25device.h
xdp_priv.h
xdp_sock_drv.h xsk: s/free_list_node/list_node/ 2024-10-14 17:22:59 +02:00
xdp_sock.h net: Optimize xdp_do_flush() with bpf_net_context infos. 2024-07-02 15:26:57 +02:00
xdp.h net, xdp: Correct grammar 2023-12-14 16:38:59 +01:00
xfrm.h xfrm: Convert struct xfrm_dst_lookup_params -> tos to dscp_t. 2024-11-06 12:42:51 +01:00
xsk_buff_pool.h xsk: Carry a copy of xdp_zc_max_segs within xsk_buff_pool 2024-10-14 17:23:30 +02:00