linux/net/core
Alexander Lobakin 39806b96c8 page_pool: try direct bulk recycling
Now that the checks for direct recycling possibility live inside the
Page Pool core, reuse them when performing bulk recycling.
page_pool_put_page_bulk() can be called from process context as well,
page_pool_napi_local() takes care of this at the very beginning.
Under high .ndo_xdp_xmit() traffic load, the win is 2-3% Pps assuming
the sending driver uses xdp_return_frame_bulk() on Tx completion.

Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Link: https://lore.kernel.org/r/20240329165507.3240110-3-aleksander.lobakin@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-02 18:13:49 -07:00
..
bpf_sk_storage.c netlink: introduce type-checking attribute iteration 2024-03-29 15:06:02 -07:00
datagram.c net: remove skb_free_datagram_locked() 2024-03-26 15:37:24 +01:00
dev_addr_lists_test.c net: fill in MODULE_DESCRIPTION()s under net/core 2023-10-28 11:29:27 +01:00
dev_addr_lists.c
dev_ioctl.c net: partial revert of the "Make timestamping selectable: series 2023-11-18 18:42:37 -08:00
dev.c net: rps: add rps_input_queue_head_add() helper 2024-04-01 11:28:32 +01:00
dev.h net: move dev_xmit_recursion() helpers to net/core/dev.h 2024-04-01 11:28:31 +01:00
drop_monitor.c genetlink: Use internal flags for multicast groups 2023-12-29 08:43:59 +00:00
dst_cache.c
dst.c net: dst: Make dst_destroy() static and return void. 2024-02-06 11:45:53 +01:00
failover.c net: failover: use IFF_NO_ADDRCONF flag to prevent ipv6 addrconf 2022-12-12 15:18:25 -08:00
fib_notifier.c
fib_rules.c fib: rules: remove repeated assignment in fib_nl2rule 2024-01-07 15:16:19 +00:00
filter.c ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
flow_dissector.c ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
flow_offload.c tc: flower: Enable offload support IPSEC SPI field. 2023-08-02 10:09:32 +01:00
gen_estimator.c treewide: Convert del_timer*() to timer_shutdown*() 2022-12-25 13:38:09 -08:00
gen_stats.c
gro_cells.c net: move netdev_max_backlog to net_hotdata 2024-03-07 21:12:42 -08:00
gro.c net: introduce struct net_hotdata 2024-03-07 21:12:41 -08:00
gso.c net: introduce struct net_hotdata 2024-03-07 21:12:41 -08:00
hotdata.c net: move dev_rx_weight to net_hotdata 2024-03-07 21:12:42 -08:00
hwbm.c
link_watch.c net: add netdev_set_operstate() helper 2024-02-14 11:20:13 +00:00
lwt_bpf.c lwt: Fix return values of BPF xmit ops 2023-08-18 16:05:26 +02:00
lwtunnel.c
Makefile net: net_test: add tests for IP tunnel flags conversion helpers 2024-04-01 10:49:28 +01:00
neighbour.c neighbour: Don't let neigh_forced_gc() disable preemption for long 2023-12-08 10:37:43 +00:00
net_namespace.c net: use synchronize_rcu_expedited in cleanup_net() 2024-02-12 12:17:03 +00:00
net_test.c pfcp: always set pfcp metadata 2024-04-01 10:49:28 +01:00
net-procfs.c net: make softnet_data.dropped an atomic_t 2024-04-01 11:28:32 +01:00
net-sysfs.c net: dqs: add NIC stall detector based on BQL 2024-03-08 10:23:26 +00:00
net-sysfs.h
net-traces.c udp6: add a missing call into udp_fail_queue_rcv_skb tracepoint 2023-07-07 09:16:52 +01:00
netclassid_cgroup.c cgroup, netclassid: on modifying netclassid in cgroup, only consider the main process. 2023-10-16 16:36:53 -07:00
netdev-genl-gen.c netdev: add per-queue statistics 2024-03-07 21:13:25 -08:00
netdev-genl-gen.h netdev: add per-queue statistics 2024-03-07 21:13:25 -08:00
netdev-genl.c netdev: add queue stat for alloc failures 2024-03-07 21:13:26 -08:00
netevent.c
netpoll.c netpoll: allocate netdev tracker right away 2023-06-15 08:21:11 +01:00
netprio_cgroup.c
of_net.c net: Explicitly include correct DT includes 2023-07-27 20:33:16 -07:00
page_pool_priv.h net: page_pool: report when page pool was destroyed 2023-11-28 15:48:39 +01:00
page_pool_user.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-03-07 10:29:36 -08:00
page_pool.c page_pool: try direct bulk recycling 2024-04-02 18:13:49 -07:00
pktgen.c net: pktgen: Use wait_event_freezable_timeout() for freezable kthread 2023-12-27 14:34:52 +00:00
ptp_classifier.c
request_sock.c tcp: make sure init the accept_queue's spinlocks once 2024-01-19 21:13:25 -08:00
rtnetlink.c netlink: introduce type-checking attribute iteration 2024-03-29 15:06:02 -07:00
scm.c af_unix: Link struct unix_edge when queuing skb. 2024-03-29 08:27:50 -07:00
secure_seq.c
selftests.c net: fill in MODULE_DESCRIPTION()s under net/core 2023-10-28 11:29:27 +01:00
skbuff.c page_pool: check for PP direct cache locality later 2024-04-02 18:13:49 -07:00
skmsg.c bpf, sockmap: Fix NULL pointer dereference in sk_psock_verdict_data_ready() 2024-02-21 17:15:23 +01:00
sock_destructor.h
sock_diag.c sock_diag: remove sock_diag_mutex 2024-01-23 15:13:55 +01:00
sock_map.c bpf: syzkaller found null ptr deref in unix_bpf proto add 2023-12-13 16:32:28 -08:00
sock_reuseport.c
sock.c net: add sk_wake_async_rcu() helper 2024-03-29 15:03:11 -07:00
stream.c net: Return error from sk_stream_wait_connect() if sk_wait_event() fails 2023-12-15 10:48:51 +00:00
sysctl_net_core.c net: move rps_sock_flow_table to net_hotdata 2024-03-07 21:12:43 -08:00
timestamping.c net: partial revert of the "Make timestamping selectable: series 2023-11-18 18:42:37 -08:00
tso.c net: tso: inline tso_count_descs() 2022-12-12 15:04:39 -08:00
utils.c
xdp.c net: move skbuff_cache(s) to net_hotdata 2024-03-07 21:12:42 -08:00