Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-02 10:11:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
9115e8cd2a
linux/include/net
History
Eric Dumazet 9115e8cd2a net: reorganize struct sock for better data locality 
Group fields used in TX path, and keep some cache lines mostly read
to permit sharing among cpus.

Gained two 4 bytes holes on 64bit arches.

Added a place holder for tcp tsq_flags, next to sk_wmem_alloc
to speed up tcp_wfree() in the following patch.

I have not added ____cacheline_aligned_in_smp, this might be done later.
I prefer doing this once inet and tcp/udp sockets reorg is also done.

Tested with both TCP and UDP.

UDP receiver performance under flood increased by ~20 % :
Accessing sk_filter/sk_wq/sk_napi_id no longer stalls because sk_drops
was moved away from a critical cache line, now mostly read and shared.

	/* --- cacheline 4 boundary (256 bytes) --- */
	unsigned int               sk_napi_id;           /* 0x100   0x4 */
	int                        sk_rcvbuf;            /* 0x104   0x4 */
	struct sk_filter *         sk_filter;            /* 0x108   0x8 */
	union {
		struct socket_wq * sk_wq;                /*         0x8 */
		struct socket_wq * sk_wq_raw;            /*         0x8 */
	};                                               /* 0x110   0x8 */
	struct xfrm_policy *       sk_policy[2];         /* 0x118  0x10 */
	struct dst_entry *         sk_rx_dst;            /* 0x128   0x8 */
	struct dst_entry *         sk_dst_cache;         /* 0x130   0x8 */
	atomic_t                   sk_omem_alloc;        /* 0x138   0x4 */
	int                        sk_sndbuf;            /* 0x13c   0x4 */
	/* --- cacheline 5 boundary (320 bytes) --- */
	int                        sk_wmem_queued;       /* 0x140   0x4 */
	atomic_t                   sk_wmem_alloc;        /* 0x144   0x4 */
	long unsigned int          sk_tsq_flags;         /* 0x148   0x8 */
	struct sk_buff *           sk_send_head;         /* 0x150   0x8 */
	struct sk_buff_head        sk_write_queue;       /* 0x158  0x18 */
	__s32                      sk_peek_off;          /* 0x170   0x4 */
	int                        sk_write_pending;     /* 0x174   0x4 */
	long int                   sk_sndtimeo;          /* 0x178   0x8 */

Signed-off-by: Eric Dumazet <edumazet@google.com>
Tested-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-12-05 13:32:24 -05:00
..
9p
bluetooth Bluetooth: Fix using the correct source address type 2016-11-22 22:50:46 +01:00
caif
irda
iucv s390/iucv: do not use arrays as argument 2015-09-21 16:03:04 -07:00
netfilter Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
netns ipv4: fib: Allow for consistent FIB dumping 2016-12-03 19:29:35 -05:00
nfc NFC: digital: Add support for NFC DEP Response Waiting Time 2016-07-11 02:01:14 +02:00
phonet sock: struct proto hash function may error 2016-02-11 03:54:14 -05:00
sctp sctp: use new rhlist interface on sctp transport rhashtable 2016-11-16 23:22:17 -05:00
tc_act net/sched: act_tunnel_key: add helper inlines to access tcf_tunnel_key 2016-11-09 13:41:53 -05:00
6lowpan.h 6lowpan: add 802.15.4 short addr slaac 2016-06-15 20:41:22 -07:00
act_api.h net/sched: act_mirred: Add new tc_action_ops get_dev() 2016-12-02 13:28:36 -05:00
addrconf.h ipv6: fix a potential deadlock in do_ipv6_setsockopt() 2016-10-21 11:29:02 -04:00
af_ieee802154.h ieee802154: af_ieee802154: fix typo in comment. 2015-09-17 13:20:05 +02:00
af_rxrpc.h rxrpc: Rewrite the data and ack handling code 2016-09-08 11:10:12 +01:00
af_unix.h af_unix: split 'u->readlock' into two: 'iolock' and 'bindlock' 2016-09-04 13:29:29 -07:00
af_vsock.h VSOCK: Introduce virtio_vsock_common.ko 2016-08-02 02:57:29 +03:00
ah.h
arp.h
atmclip.h
ax25.h ax25: Stop using sock->sk_protinfo. 2015-06-28 16:55:44 -07:00
ax88796.h
bond_3ad.h bonding: 3ad: apply ad_actor settings changes immediately 2016-02-09 04:45:49 -05:00
bond_alb.h
bond_options.h bonding: convert num_grat_arp to the new bonding option API 2015-07-27 01:05:24 -07:00
bonding.h netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
busy_poll.h udp: enable busy polling for all sockets 2016-11-18 10:44:31 -05:00
calipso.h calipso: Add a label cache. 2016-06-27 15:06:17 -04:00
cfg80211-wext.h
cfg80211.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-10-30 12:42:58 -04:00
cfg802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
checksum.h csum: Update csum_block_add to use rotate instead of byteswap 2016-03-13 15:01:00 -04:00
cipso_ipv4.h
cls_cgroup.h cls_cgroup: get sk_classid only from full sockets 2016-04-19 20:09:25 -04:00
codel_impl.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
codel_qdisc.h net_sched: fq_codel: cache skb->truesize into skb->cb 2016-06-25 12:19:35 -04:00
codel.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
compat.h packet: compat support for sock_fprog 2016-06-09 23:41:03 -07:00
datalink.h
dcbevent.h
dcbnl.h
devlink.h devlink: Add E-Switch inline mode control 2016-11-24 16:01:14 -05:00
dn_dev.h
dn_fib.h
dn_neigh.h netfilter: Pass net into okfn 2015-09-17 17:18:37 -07:00
dn_nsp.h
dn_route.h
dn.h
dsa.h net: dsa: add port fast ageing 2016-09-23 08:38:50 -04:00
dsfield.h
dst_cache.h net: add dst_cache support 2016-02-16 20:21:48 -05:00
dst_metadata.h net/dst: Add dst port to dst_metadata utility functions 2016-11-09 13:41:54 -05:00
dst_ops.h ipv4, ipv6: Pass net into __ip_local_out and __ip6_local_out 2015-10-08 04:27:02 -07:00
dst.h route: move lwtunnel state to a single place 2016-04-25 16:20:09 -04:00
esp.h
ethoc.h net/ethoc: support big-endian register layout 2015-09-23 15:33:15 -07:00
fib_rules.h net: core: add UID to flows, rules, and routes 2016-11-04 14:45:23 -04:00
firewire.h
flow_dissector.h net/sched: cls_flower: Add UDP port to tunnel parameters 2016-11-09 13:41:54 -05:00
flow.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
flowcache.h
fou.h fou: Add encap ops for IPv6 tunnels 2016-05-20 18:03:16 -04:00
fq_impl.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
fq.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
garp.h
gen_stats.h net: sched: do not acquire qdisc spinlock in qdisc/class stats dump 2016-06-07 16:37:14 -07:00
genetlink.h genetlink: Make family a signed integer. 2016-11-13 12:14:59 -05:00
geneve.h net: Remove deprecated tunnel specific UDP offload functions 2016-06-17 20:23:32 -07:00
gre.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-08-18 01:17:32 -04:00
gro_cells.h gro_cells: mark napi struct as not busy poll candidates 2016-11-15 22:27:27 -05:00
gtp.h gtp: #define #define _GTP_H_ and not #define _GTP_H 2016-07-25 17:55:43 -07:00
gue.h
hwbm.h net: add a hardware buffer management helper API 2016-03-14 12:19:46 -04:00
icmp.h net: snmp: kill STATS_BH macros 2016-04-27 22:48:25 -04:00
ieee80211_radiotap.h wireless: radiotap: fix timestamp sampling position values 2016-10-19 12:11:36 +02:00
ieee802154_netdev.h mac802154: constify ieee802154_llsec_ops structure 2016-01-04 20:40:41 +01:00
if_inet6.h ipv6 addrconf: Implemented enhanced DAD (RFC7527) 2016-12-03 23:21:37 -05:00
ila.h ila: Add generic ILA translation facility 2015-12-15 23:25:20 -05:00
inet6_connection_sock.h ipv6: remove unused in6_addr struct 2016-03-22 15:45:44 -04:00
inet6_hashtables.h tcp/dccp: do not touch listener sk_refcnt under synflood 2016-04-04 22:11:20 -04:00
inet_common.h ip4ip6: Support for GSO/GRO 2016-05-20 18:03:17 -04:00
inet_connection_sock.h tcp/dccp: drop SYN packets if accept queue is full 2016-10-29 15:09:21 -04:00
inet_ecn.h ipv6: suppress sparse warnings in IP6_ECN_set_ce() 2016-08-13 15:08:00 -07:00
inet_frag.h ipv4: namespacify ip fragment max dist sysctl knob 2016-02-16 20:42:54 -05:00
inet_hashtables.h tcp/dccp: do not touch listener sk_refcnt under synflood 2016-04-04 22:11:20 -04:00
inet_sock.h ipv4: add IP_RECVFRAGSIZE cmsg 2016-11-03 15:41:11 -04:00
inet_timewait_sock.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-10-20 06:08:27 -07:00
inetpeer.h inet: tcp: fix inetpeer_set_addr_v4() 2015-12-16 00:14:12 -05:00
ip6_checksum.h ipv6: Pass proto to csum_ipv6_magic as __u8 instead of unsigned short 2016-03-13 23:55:13 -04:00
ip6_fib.h net: ipv6: Fix processing of RAs in presence of VRF 2016-10-27 16:30:52 -04:00
ip6_route.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
ip6_tunnel.h ip6_tunnel: Clear IP6CB in ip6tunnel_xmit() 2016-11-02 15:18:36 -04:00
ip_fib.h ipv4: fib: Replay events when registering FIB notifier 2016-12-03 19:29:35 -05:00
ip_tunnels.h netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
ip_vs.h ipvs: update real-server binding of outgoing connections in SIP-pe 2016-06-06 09:47:25 +09:00
ip.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-11-15 10:54:36 -05:00
ipcomp.h
ipconfig.h
ipv6.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
ipx.h
iw_handler.h cfg80211/wext: fix message ordering 2016-01-29 17:13:43 +01:00
kcm.h kcm: Use stream parser 2016-08-17 19:36:23 -04:00
l3mdev.h net: ipv4: Do not drop to make_route if oif is l3mdev 2016-10-13 12:05:26 -04:00
lapb.h
lib80211.h
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h net: Pass kern from net_proto_family.create to sk_alloc 2015-05-11 10:50:17 -04:00
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h
lwtunnel.h lwtunnel: subtract tunnel headroom from mtu on output redirect 2016-11-16 17:01:15 -05:00
mac80211.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-10-30 12:42:58 -04:00
mac802154.h ieee802154: cleanup WARN_ON for fc fetch 2016-07-08 13:23:12 +02:00
mip6.h
mld.h
mpls_iptunnel.h mpls: multipath route support 2015-10-23 06:26:42 -07:00
mpls.h openvswitch: use mpls_hdr 2016-10-03 02:00:22 -04:00
mrp.h
ncsi.h net/ncsi: Introduce ncsi_stop_dev() 2016-10-04 02:11:51 -04:00
ndisc.h ipv6 addrconf: Implemented enhanced DAD (RFC7527) 2016-12-03 23:21:37 -05:00
neighbour.h net: add explicit logging and stat for neighbour table overflow 2015-08-10 13:46:21 -07:00
net_namespace.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-11-22 13:27:16 -05:00
net_ratelimit.h
netevent.h neigh: Send a notification when DELAY_PROBE_TIME changes 2016-07-05 09:06:29 -07:00
netlabel.h netlabel: Implement CALIPSO config functions for SMACK. 2016-06-27 15:06:18 -04:00
netlink.h netlink: 2-clause nla_ok() 2016-12-03 15:54:02 -05:00
netprio_cgroup.h net: wrap sock->sk_cgrp_prioidx and ->sk_classid inside a struct 2015-12-08 22:02:33 -05:00
netrom.h
nexthop.h
nl802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
p8022.h
ping.h net: ping: make ping_v6_sendmsg static 2016-03-23 22:09:58 -04:00
pkt_cls.h net/sched: cls_flower: Add offload support using egress Hardware device 2016-12-02 13:28:37 -05:00
pkt_sched.h net: make default TX queue length a defined constant 2016-11-07 20:15:55 -05:00
pptp.h pptp: Refactor the struct and macros of PPTP codes 2016-08-15 10:55:53 -07:00
protocol.h udp: Remove udp_offloads 2016-04-07 16:53:30 -04:00
psnap.h
raw.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
rawv6.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
red.h
regulatory.h
request_sock.h inet: reqsk_alloc() needs to take care of dead listeners 2016-04-04 22:11:19 -04:00
rose.h
route.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
rtnetlink.h net: rtnetlink: add support for the IFLA_STATS_LINK_XSTATS_SLAVE attribute 2016-06-30 06:15:04 -04:00
sch_generic.h sched: add and use qdisc_skb_head helpers 2016-09-19 01:47:18 -04:00
scm.h unix: correctly track in-flight fds in sending process user_struct 2016-02-08 10:30:42 -05:00
secure_seq.h tcp: randomize tcp timestamp offsets for each connection 2016-12-02 12:49:59 -05:00
seg6_hmac.h ipv6: sr: add core files for SR HMAC support 2016-11-09 20:40:06 -05:00
seg6.h ipv6: sr: add core files for SR HMAC support 2016-11-09 20:40:06 -05:00
slhc_vj.h
snmp.h net: snmp: fix 64bit stats on 32bit arches 2016-04-28 11:49:45 -04:00
sock_reuseport.h soreuseport: fix NULL ptr dereference SO_REUSEPORT after bind 2016-01-19 14:44:23 -05:00
sock.h net: reorganize struct sock for better data locality 2016-12-05 13:32:24 -05:00
Space.h
stp.h
strparser.h kcm: Remove TCP specific references from kcm and strparser 2016-08-28 23:32:41 -04:00
switchdev.h switchdev: remove FIB offload infrastructure 2016-09-28 04:48:00 -04:00
tcp_states.h
tcp.h tcp: randomize tcp timestamp offsets for each connection 2016-12-02 12:49:59 -05:00
timewait_sock.h inet: remove BUG_ON() in twsk_destructor() 2015-07-09 15:12:20 -07:00
transp_v6.h ipv6: add new struct ipcm6_cookie 2016-05-03 16:08:14 -04:00
tso.h net: tso: add support for IPv6 2015-10-26 22:24:22 -07:00
udp_tunnel.h vxlan: Add new UDP encapsulation offload type for VXLAN-GPE 2016-06-17 20:23:32 -07:00
udp.h udplite: fix NULL pointer dereference 2016-11-15 11:59:38 -05:00
udplite.h udplite: fix NULL pointer dereference 2016-11-15 11:59:38 -05:00
vsock_addr.h
vxlan.h vxlan: remove unsed vxlan_dev_dst_port() 2016-11-15 12:16:13 -05:00
wext.h
wimax.h
x25.h
x25device.h
xfrm.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-09-23 06:46:57 -04:00