Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-02 02:01:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
2b85a34e91
linux/include/net
History
Eric Dumazet 2b85a34e91 net: No more expensive sock_hold()/sock_put() on each tx 
One of the problem with sock memory accounting is it uses
a pair of sock_hold()/sock_put() for each transmitted packet.

This slows down bidirectional flows because the receive path
also needs to take a refcount on socket and might use a different
cpu than transmit path or transmit completion path. So these
two atomic operations also trigger cache line bounces.

We can see this in tx or tx/rx workloads (media gateways for example),
where sock_wfree() can be in top five functions in profiles.

We use this sock_hold()/sock_put() so that sock freeing
is delayed until all tx packets are completed.

As we also update sk_wmem_alloc, we could offset sk_wmem_alloc
by one unit at init time, until sk_free() is called.
Once sk_free() is called, we atomic_dec_and_test(sk_wmem_alloc)
to decrement initial offset and atomicaly check if any packets
are in flight.

skb_set_owner_w() doesnt call sock_hold() anymore

sock_wfree() doesnt call sock_put() anymore, but check if sk_wmem_alloc
reached 0 to perform the final freeing.

Drawback is that a skb->truesize error could lead to unfreeable sockets, or
even worse, prematurely calling __sk_free() on a live socket.

Nice speedups on SMP. tbench for example, going from 2691 MB/s to 2711 MB/s
on my 8 cpu dev machine, even if tbench was not really hitting sk_refcnt
contention point. 5 % speedup on a UDP transmit workload (depends
on number of flows), lowering TX completion cpu usage.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2009-06-11 02:55:43 -07:00
..
9p net/9p: handle correctly interrupted 9P requests 2009-04-05 16:54:53 -05:00
bluetooth bluetooth: Kill skb_frags_no(), unused. 2009-06-08 16:16:56 -07:00
ieee802154 net: add NL802154 interface for configuration of 802.15.4 devices 2009-06-09 05:25:33 -07:00
irda irda: Add irda_skb_cb qdisc related padding 2008-12-17 15:44:58 -08:00
iucv af_iucv: New socket option for setting IUCV MSGLIMITs 2009-04-23 04:04:38 -07:00
netfilter netfilter: nf_nat: add support for persistent mappings 2009-04-16 18:33:01 +02:00
netns netfilter: nf_conntrack: use SLAB_DESTROY_BY_RCU and get rid of call_rcu() 2009-03-25 21:05:46 +01:00
phonet Phonet: use per-namespace devices list 2009-01-26 21:03:35 -08:00
sctp sctp: support non-blocking version of the new sctp_connectx() API 2009-06-03 09:14:47 -04:00
tc_act pkt_action: add new action skbedit 2008-09-12 16:30:20 -07:00
tipc tipc: Remove unneeded parameter to tipc_createport_raw() 2008-07-14 22:42:19 -07:00
act_api.h
addrconf.h ipv6: Fix conflict resolutions during ipv6 binding 2009-03-24 19:49:11 -07:00
af_rxrpc.h
af_unix.h net: Fix soft lockups/OOM issues w/ unix garbage collector 2008-11-26 15:32:27 -08:00
ah.h
arp.h
atmclip.h clip: convert to internal network_device_stats 2009-01-21 14:01:59 -08:00
ax25.h
ax88796.h ax88796: Add method to take MAC from platform data 2009-03-24 23:32:03 -07:00
cfg80211.h cfg80211: add rfkill support 2009-06-03 14:06:14 -04:00
checksum.h include/net net/ - csum_partial - remove unnecessary casts 2008-11-19 15:44:53 -08:00
cipso_ipv4.h netlabel: Label incoming TCP connections correctly in SELinux 2009-03-28 15:01:36 +11:00
compat.h net: Use standard structures for generic socket address structures. 2008-07-19 22:35:47 -07:00
datalink.h
dcbnl.h net: fix DCB setstate to return success/failure 2008-12-21 20:09:50 -08:00
dn_dev.h
dn_fib.h decnet: remove private wrappers of endian helpers 2008-11-27 00:12:47 -08:00
dn_neigh.h
dn_nsp.h
dn_route.h
dn.h decnet: compile fix for removal of byteorder wrapper 2008-11-27 23:04:13 -08:00
dsa.h dsa: add switch chip cascading support 2009-03-21 19:06:54 -07:00
dsfield.h
dst.h net: skb->dst accessors 2009-06-03 02:51:04 -07:00
esp.h
ethoc.h net: Add support for the OpenCores 10/100 Mbps Ethernet MAC. 2009-03-27 00:16:21 -07:00
fib_rules.h net: Remove unused parameter from fill method in fib_rules_ops. 2009-05-20 17:26:23 -07:00
flow.h netns xfrm: lookup in netns 2008-11-25 17:35:18 -08:00
garp.h vlan: Add GVRP support 2008-07-05 21:26:57 -07:00
gen_stats.h pkt_sched: gen_estimator: Optimize gen_estimator_active() 2008-11-26 15:24:32 -08:00
genetlink.h genetlink: Introduce genl_register_family_with_ops() 2009-05-21 16:50:22 -07:00
icmp.h mib: put icmpmsg statistics on struct net 2008-07-18 04:04:22 -07:00
ieee80211_radiotap.h wireless: radiotap updates 2009-03-27 20:12:52 -04:00
if_inet6.h ipv6: reorder struct inet6_ifaddr to remove padding on 64 bit builds 2009-03-21 13:29:05 -07:00
inet6_connection_sock.h
inet6_hashtables.h net: skb->dst accessors 2009-06-03 02:51:04 -07:00
inet_common.h
inet_connection_sock.h net: more #ifdef CONFIG_COMPAT 2008-08-28 02:53:51 -07:00
inet_ecn.h net: replace __constant_{endian} uses in net headers 2009-02-14 22:58:35 -08:00
inet_frag.h inet fragments: fix sparse warning: context imbalance 2009-02-26 23:13:35 -08:00
inet_hashtables.h net: skb->dst accessors 2009-06-03 02:51:04 -07:00
inet_sock.h ipv4: New multicast-all socket option 2009-06-02 00:45:24 -07:00
inet_timewait_sock.h net: Convert TCP & DCCP hash tables to use RCU / hlist_nulls 2008-11-16 19:40:17 -08:00
inetpeer.h net: remove CVS keywords 2008-06-11 21:00:38 -07:00
ip6_checksum.h
ip6_fib.h
ip6_route.h net: skb->dst accessors 2009-06-03 02:51:04 -07:00
ip6_tunnel.h net: remove CVS keywords 2008-06-11 21:00:38 -07:00
ip_fib.h ipv4: remove unused macro (FIB_RES_RESET) from ip_fib.h. 2009-05-03 14:19:51 -07:00
ip_vs.h net: replace __constant_{endian} uses in net headers 2009-02-14 22:58:35 -08:00
ip.h snmp: add missing counters for RFC 4293 2009-04-27 02:45:02 -07:00
ipcomp.h ipsec: ipcomp - Merge IPComp implementations 2008-07-25 02:54:40 -07:00
ipconfig.h net: remove CVS keywords 2008-06-11 21:00:38 -07:00
ipip.h sit: stateless autoconf for isatap 2009-05-19 16:02:02 -07:00
ipv6.h snmp: add missing counters for RFC 4293 2009-04-27 02:45:02 -07:00
ipx.h net: replace __constant_{endian} uses in net headers 2009-02-14 22:58:35 -08:00
iw_handler.h wext: Emit event stream entries correctly when compat. 2008-06-16 18:50:49 -07:00
lapb.h
lib80211.h wireless: missing include in lib80211.h 2008-11-21 11:42:55 -05:00
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h
mac80211.h mac80211: do not pass PS frames out of mac80211 again 2009-06-10 13:28:37 -04:00
mip6.h
ndisc.h ipv6: Fix sporadic sendmsg -EINVAL when sending to multicast groups. 2009-01-04 16:04:39 -08:00
neighbour.h net: Cleanup of neighbour code 2008-11-12 00:54:54 -08:00
net_namespace.h netns: Remove net_alive 2009-03-03 01:14:27 -08:00
netdma.h net_dma: convert to dma_find_channel 2009-01-06 11:38:15 -07:00
netevent.h
netlabel.h netlabel: Cleanup the Smack/NetLabel code to fix incoming TCP connections 2009-03-28 15:01:37 +11:00
netlink.h netlink: add nla_policy_len() 2009-03-25 18:26:30 +01:00
netrom.h netrom: convert to internal net_device_stats 2009-01-21 14:02:01 -08:00
nexthop.h
p8022.h
pkt_cls.h ematch: simpler tcf_em_unregister() 2008-11-16 23:01:49 -08:00
pkt_sched.h pkt_sched: Change PSCHED_SHIFT from 10 to 6 2009-06-09 05:25:30 -07:00
protocol.h ipv6: Add GRO support 2009-01-08 10:40:57 -08:00
psnap.h snap: use const for descriptor 2009-03-21 19:06:50 -07:00
raw.h
rawv6.h
red.h
regulatory.h cfg80211: clean up includes 2009-04-22 16:57:17 -04:00
request_sock.h net: Fix memory leak in the proto_register function 2008-11-21 16:45:22 -08:00
rose.h rose: improving AX25 routing frames via ROSE network 2008-06-17 17:08:32 -07:00
route.h net: skb->rtable accessor 2009-06-03 02:51:02 -07:00
rtnetlink.h [RTNL]: Introduce the rtnl_kill_links helper. 2008-04-16 00:46:52 -07:00
sch_generic.h net: reorder struct Qdisc for better SMP performance 2009-03-20 01:33:32 -07:00
scm.h Merge branch 'master' into next 2008-11-14 11:29:12 +11:00
slhc_vj.h
snmp.h snmp: add missing counters for RFC 4293 2009-04-27 02:45:02 -07:00
sock.h net: No more expensive sock_hold()/sock_put() on each tx 2009-06-11 02:55:43 -07:00
stp.h net: Add STP demux layer 2008-07-05 21:25:39 -07:00
tcp_states.h
tcp.h Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2009-05-08 02:48:30 -07:00
timewait_sock.h net: Fix memory leak in the proto_register function 2008-11-21 16:45:22 -08:00
transp_v6.h net: replace __constant_{endian} uses in net headers 2009-02-14 22:58:35 -08:00
udp.h ipv6: Fix NULL pointer dereference with time-wait sockets 2009-04-11 01:53:06 -07:00
udplite.h udp: introduce struct udp_table and multiple spinlocks 2008-10-29 01:41:45 -07:00
wext.h wext: Dispatch and handle compat ioctls entirely in net/wireless/wext.c 2008-06-16 18:32:46 -07:00
wimax.h rfkill: rewrite 2009-06-03 14:06:13 -04:00
x25.h
x25device.h
xfrm.h net: skb->dst accessors 2009-06-03 02:51:04 -07:00