linux/include/net
Florian Westphal ec30d78c14 xfrm: add xdst pcpu cache
retain last used xfrm_dst in a pcpu cache.
On next request, reuse this dst if the policies are the same.

The cache will not help with strict RR workloads as there is no hit.

The cache packet-path part is reasonably small, the notifier part is
needed so we do not add long hangs when a device is dismantled but some
pcpu xdst still holds a reference, there are also calls to the flush
operation when userspace deletes SAs so modules can be removed
(there is no hit.

We need to run the dst_release on the correct cpu to avoid races with
packet path.  This is done by adding a work_struct for each cpu and then
doing the actual test/release on each affected cpu via schedule_work_on().

Test results using 4 network namespaces and null encryption:

ns1           ns2          -> ns3           -> ns4
netperf -> xfrm/null enc   -> xfrm/null dec -> netserver

what                    TCP_STREAM      UDP_STREAM      UDP_RR
Flow cache:             14644.61        294.35          327231.64
No flow cache:		14349.81	242.64		202301.72
Pcpu cache:		14629.70	292.21		205595.22

UDP tests used 64byte packets, tests ran for one minute each,
value is average over ten iterations.

'Flow cache' is 'net-next', 'No flow cache' is net-next plus this
series but without this patch.

Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-18 11:13:41 -07:00
..
9p 9p: constify ->d_name handling 2017-01-12 04:01:17 -05:00
bluetooth Bluetooth: Set LE Default PHY preferences 2017-05-18 13:52:49 +02:00
caif
irda scripts/spelling.txt: add "overide" pattern and fix typo instances 2017-03-09 17:01:09 -08:00
iucv s390/iucv: do not use arrays as argument 2015-09-21 16:03:04 -07:00
netfilter net: convert nf_bridge_info.use from atomic_t to refcount_t 2017-07-01 07:39:07 -07:00
netns xfrm: remove flow cache 2017-07-18 11:13:41 -07:00
nfc NFC: Add nfc_dbg() macro 2017-04-05 10:15:20 +02:00
phonet sock: struct proto hash function may error 2016-02-11 03:54:14 -05:00
sctp sctp: remove the typedef sctp_hmac_algo_param_t 2017-07-16 20:52:14 -07:00
tc_act net: sched: introduce helper to identify gact trap action 2017-06-06 12:45:23 -04:00
6lowpan.h 6lowpan: Fix IID format for Bluetooth 2017-04-12 22:02:36 +02:00
act_api.h net: sched: add termination action to allow goto chain 2017-05-17 15:22:13 -04:00
addrconf.h net, ipv6: convert inet6_ifaddr.refcnt from atomic_t to refcount_t 2017-07-04 01:29:04 -07:00
af_ieee802154.h ieee802154: af_ieee802154: fix typo in comment. 2015-09-17 13:20:05 +02:00
af_rxrpc.h rxrpc: Provide a cmsg to specify the amount of Tx data for a call 2017-06-07 17:15:46 +01:00
af_unix.h net/unix: drop obsolete fd-recursion limits 2017-07-17 08:57:59 -07:00
af_vsock.h VSOCK: Add vsockmon tap functions 2017-04-24 12:35:56 -04:00
ah.h
arp.h net: convert neighbour.refcnt from atomic_t to refcount_t 2017-07-01 07:39:07 -07:00
atmclip.h
ax25.h net, ax25: convert ax25_cb.refcount from atomic_t to refcount_t 2017-07-04 22:35:19 +01:00
ax88796.h
bond_3ad.h bonding: 3ad: apply ad_actor settings changes immediately 2016-02-09 04:45:49 -05:00
bond_alb.h
bond_options.h bonding: Prevent duplicate userspace notification 2017-05-27 18:51:41 -04:00
bonding.h bonding: fix wq initialization for links created via netlink 2017-04-21 15:28:37 -04:00
busy_poll.h net: Commonize busy polling code to focus on napi_id instead of socket 2017-03-24 20:49:31 -07:00
calipso.h net, calipso: convert calipso_doi.refcount from atomic_t to refcount_t 2017-07-04 22:35:16 +01:00
cfg80211-wext.h
cfg80211.h nl80211: add authorized flag to ROAM event 2017-06-13 11:04:37 +02:00
cfg802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
checksum.h csum: eliminate sparse warning in remcsum_unadjust() 2017-01-20 12:12:13 -05:00
cipso_ipv4.h net, ipv4: convert cipso_v4_doi.refcount from atomic_t to refcount_t 2017-07-04 01:29:04 -07:00
cls_cgroup.h cls_cgroup: get sk_classid only from full sockets 2016-04-19 20:09:25 -04:00
codel_impl.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
codel_qdisc.h net_sched: fq_codel: cache skb->truesize into skb->cb 2016-06-25 12:19:35 -04:00
codel.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
compat.h packet: compat support for sock_fprog 2016-06-09 23:41:03 -07:00
datalink.h
dcbevent.h
dcbnl.h
devlink.h net/devlink: Add E-Switch encapsulation control 2017-04-22 20:26:37 +03:00
dn_dev.h
dn_fib.h net, decnet: convert dn_fib_info.fib_clntref from atomic_t to refcount_t 2017-07-04 22:35:15 +01:00
dn_neigh.h netfilter: Pass net into okfn 2015-09-17 17:18:37 -07:00
dn_nsp.h
dn_route.h
dn.h
dsa.h net: dsa: Associate slave network device with CPU port 2017-06-13 16:35:03 -04:00
dsfield.h
dst_cache.h net: add dst_cache support 2016-02-16 20:21:48 -05:00
dst_metadata.h net: store port/representator id in metadata_dst 2017-06-25 11:42:01 -04:00
dst_ops.h net: add confirm_neigh method to dst_ops 2017-02-07 13:07:46 -05:00
dst.h net: add debug atomic_inc_not_zero() in dst_hold() 2017-06-17 22:54:01 -04:00
esp.h esp6: Reorganize esp_output 2017-04-14 10:06:42 +02:00
ethoc.h net/ethoc: support big-endian register layout 2015-09-23 15:33:15 -07:00
fib_rules.h net: convert fib_rule.refcnt from atomic_t to refcount_t 2017-07-01 07:39:09 -07:00
firewire.h
flow_dissector.h net/flow_dissector: add support for dissection of misc ip header fields 2017-06-04 18:12:23 -04:00
flow.h xfrm: remove flow cache 2017-07-18 11:13:41 -07:00
fou.h fou: Add encap ops for IPv6 tunnels 2016-05-20 18:03:16 -04:00
fq_impl.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
fq.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
garp.h
gen_stats.h net_sched: gen_estimator: complete rewrite of rate estimators 2016-12-05 15:21:59 -05:00
genetlink.h genetlink: remove ops_list from genetlink header. 2017-06-05 10:54:55 -04:00
geneve.h net: Remove deprecated tunnel specific UDP offload functions 2016-06-17 20:23:32 -07:00
gre.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-08-18 01:17:32 -04:00
gro_cells.h gro_cells: move to net/core/gro_cells.c 2017-02-08 14:38:18 -05:00
gtp.h gtp: #define #define _GTP_H_ and not #define _GTP_H 2016-07-25 17:55:43 -07:00
gue.h
hwbm.h net: add a hardware buffer management helper API 2016-03-14 12:19:46 -04:00
icmp.h net: snmp: kill STATS_BH macros 2016-04-27 22:48:25 -04:00
ieee80211_radiotap.h wireless: radiotap: rewrite the radiotap header file 2017-01-25 16:00:33 +01:00
ieee802154_netdev.h mac802154: constify ieee802154_llsec_ops structure 2016-01-04 20:40:41 +01:00
if_inet6.h net, ipv6: convert ifacaddr6.aca_refcnt from atomic_t to refcount_t 2017-07-04 01:29:04 -07:00
ife.h net: Introduce ife encapsulation module 2017-02-03 15:16:45 -05:00
ila.h ila: Add generic ILA translation facility 2015-12-15 23:25:20 -05:00
inet6_connection_sock.h inet: drop ->bind_conflict 2017-01-18 13:04:28 -05:00
inet6_hashtables.h tcp/dccp: do not touch listener sk_refcnt under synflood 2016-04-04 22:11:20 -04:00
inet_common.h net: Work around lockdep limitation in sockets that use sockets 2017-03-09 18:23:27 -08:00
inet_connection_sock.h tcp: ULP infrastructure 2017-06-15 12:12:40 -04:00
inet_ecn.h ipv6: suppress sparse warnings in IP6_ECN_set_ce() 2016-08-13 15:08:00 -07:00
inet_frag.h Merge branch 'for-4.13' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu 2017-07-06 08:59:41 -07:00
inet_hashtables.h net: make sk_ehashfn() static 2017-07-03 03:29:14 -07:00
inet_sock.h net/tcp-fastopen: Add new API support 2017-01-25 14:04:38 -05:00
inet_timewait_sock.h ipv4: Namespaceify tcp_tw_recycle and tcp_max_tw_buckets knob 2016-12-29 11:38:31 -05:00
inetpeer.h inetpeer: remove AVL implementation in favor of RB tree 2017-07-17 08:59:01 -07:00
ip6_checksum.h ipv6: Pass proto to csum_ipv6_magic as __u8 instead of unsigned short 2016-03-13 23:55:13 -04:00
ip6_fib.h net: remove DST_NOCACHE flag 2017-06-17 22:54:01 -04:00
ip6_route.h ip6: fix PMTU discovery when using /127 subnets 2017-07-16 16:36:01 -07:00
ip6_tunnel.h ip6_tunnel: Allow policy-based routing through tunnels 2017-04-21 13:21:30 -04:00
ip_fib.h net, ipv4: convert fib_info.fib_clntref from atomic_t to refcount_t 2017-07-04 01:29:04 -07:00
ip_tunnels.h ip_tunnel: Allow policy-based routing through tunnels 2017-04-21 13:21:31 -04:00
ip_vs.h ipvs: remove unused function ip_vs_set_state_timeout 2017-04-28 12:00:10 +02:00
ip.h net: ipv4: Refine the ipv4_default_advmss 2017-04-13 13:19:48 -04:00
ipcomp.h
ipconfig.h
ipv6.h net, ipv6: convert ipv6_txoptions.refcnt from atomic_t to refcount_t 2017-07-04 01:29:03 -07:00
ipx.h net, ipx: convert ipx_route.refcnt from atomic_t to refcount_t 2017-07-04 22:35:17 +01:00
iw_handler.h wext: uninline stream addition functions 2017-01-13 09:38:42 +01:00
kcm.h kcm: Use stream parser 2016-08-17 19:36:23 -04:00
l3mdev.h net: ipv4: Do not drop to make_route if oif is l3mdev 2016-10-13 12:05:26 -04:00
lapb.h net, lapb: convert lapb_cb.refcnt from atomic_t to refcount_t 2017-07-04 22:35:16 +01:00
lib80211.h
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h net, llc: convert llc_sap.refcnt from atomic_t to refcount_t 2017-07-04 22:35:15 +01:00
lwtunnel.h net: add extack arg to lwtunnel build state 2017-05-30 11:55:32 -04:00
mac80211.h mac80211: manage RX BA session offload without SKB queue 2017-06-08 14:16:29 +02:00
mac802154.h ieee802154: cleanup WARN_ON for fc fetch 2016-07-08 13:23:12 +02:00
mip6.h
mld.h
mpls_iptunnel.h net: mpls: Increase max number of labels for lwt encap 2017-04-01 20:21:44 -07:00
mpls.h openvswitch: use mpls_hdr 2016-10-03 02:00:22 -04:00
mrp.h
ncsi.h net/ncsi: Introduce ncsi_stop_dev() 2016-10-04 02:11:51 -04:00
ndisc.h net: convert neighbour.refcnt from atomic_t to refcount_t 2017-07-01 07:39:07 -07:00
neighbour.h net: convert neigh_params.refcnt from atomic_t to refcount_t 2017-07-01 07:39:07 -07:00
net_namespace.h net: convert net.passive from atomic_t to refcount_t 2017-07-01 07:39:09 -07:00
net_ratelimit.h
netevent.h neigh: Send a notification when DELAY_PROBE_TIME changes 2016-07-05 09:06:29 -07:00
netlabel.h net: convert netlbl_lsm_cache.refcount from atomic_t to refcount_t 2017-07-01 07:39:09 -07:00
netlink.h netlink: pass extended ACK struct to parsing functions 2017-04-13 13:58:22 -04:00
netprio_cgroup.h net: wrap sock->sk_cgrp_prioidx and ->sk_classid inside a struct 2015-12-08 22:02:33 -05:00
netrom.h net, netrom: convert nr_node.refcount from atomic_t to refcount_t 2017-07-04 22:35:17 +01:00
nexthop.h
nl802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
p8022.h
ping.h net: ping: make ping_v6_sendmsg static 2016-03-23 22:09:58 -04:00
pkt_cls.h sched: add helper for updating statistics on all actions 2017-05-31 17:58:13 -04:00
pkt_sched.h net: sched: move tc_classify function to cls_api.c 2017-05-17 15:22:13 -04:00
pptp.h pptp: Refactor the struct and macros of PPTP codes 2016-08-15 10:55:53 -07:00
protocol.h net: Add sysctl to toggle early demux for tcp and udp 2017-03-24 13:17:07 -07:00
psample.h net: Introduce psample, a new genetlink channel for packet sampling 2017-01-24 13:44:28 -05:00
psnap.h
raw.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
rawv6.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
red.h ktime: Get rid of the union 2016-12-25 17:21:22 +01:00
regulatory.h
request_sock.h net: convert sock.sk_refcnt from atomic_t to refcount_t 2017-07-01 07:39:08 -07:00
rose.h
route.h ipv4: call dst_hold_safe() properly 2017-06-17 22:54:00 -04:00
rtnetlink.h net: add netlink_ext_ack argument to rtnl_link_ops.slave_validate 2017-06-26 23:13:22 -04:00
sch_generic.h net, sched: convert Qdisc.refcnt from atomic_t to refcount_t 2017-07-04 22:35:16 +01:00
scm.h sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h> 2017-03-02 08:42:31 +01:00
secure_seq.h tcp: Namespaceify sysctl_tcp_timestamps 2017-06-08 10:53:29 -04:00
seg6_hmac.h ipv6: sr: add core files for SR HMAC support 2016-11-09 20:40:06 -05:00
seg6.h ipv6: sr: add core files for SR HMAC support 2016-11-09 20:40:06 -05:00
slhc_vj.h
smc.h smc: netlink interface for SMC sockets 2017-01-09 16:07:41 -05:00
snmp.h net: snmp: fix 64bit stats on 32bit arches 2016-04-28 11:49:45 -04:00
sock_reuseport.h soreuseport: fix NULL ptr dereference SO_REUSEPORT after bind 2016-01-19 14:44:23 -05:00
sock.h socket: add documentation for missing elements 2017-07-12 14:39:43 -07:00
Space.h
stp.h
strparser.h kcm: Remove TCP specific references from kcm and strparser 2016-08-28 23:32:41 -04:00
switchdev.h net: switchdev: add SET_SWITCHDEV_OPS helper 2017-07-01 08:51:32 -07:00
tcp_states.h
tcp.h bpf: Add support for changing congestion control 2017-07-01 16:15:14 -07:00
timewait_sock.h
tls.h tls: kernel TLS support 2017-06-15 12:12:40 -04:00
transp_v6.h ipv6: add new struct ipcm6_cookie 2016-05-03 16:08:14 -04:00
tso.h net: tso: add support for IPv6 2015-10-26 22:24:22 -07:00
udp_tunnel.h vxlan: Add new UDP encapsulation offload type for VXLAN-GPE 2016-06-17 20:23:32 -07:00
udp.h udp: move scratch area helpers into the include file 2017-06-27 15:43:56 -04:00
udplite.h udp: use a separate rx queue for packet reception 2017-05-16 15:41:29 -04:00
vsock_addr.h
vxlan.h net, vxlan: convert vxlan_sock.refcnt from atomic_t to refcount_t 2017-07-04 22:35:15 +01:00
wext.h dev_ioctl: copy only the smaller struct iwreq for wext 2017-06-14 13:52:44 +02:00
wimax.h
x25.h net, x25: convert x25_neigh.refcnt from atomic_t to refcount_t 2017-07-04 22:35:18 +01:00
x25device.h
xfrm.h xfrm: add xdst pcpu cache 2017-07-18 11:13:41 -07:00