linux/net/core
Hagen Paul Pfeifer 01f2f3f6ef net: optimize Berkeley Packet Filter (BPF) processing
Gcc is currenlty not in the ability to optimize the switch statement in
sk_run_filter() because of dense case labels. This patch replace the
OR'd labels with ordered sequenced case labels. The sk_chk_filter()
function is modified to patch/replace the original OPCODES in a
ordered but equivalent form. gcc is now in the ability to transform the
switch statement in sk_run_filter into a jump table of complexity O(1).

Until this patch gcc generates a sequence of conditional branches (O(n) of 567
byte .text segment size (arch x86_64):

7ff: 8b 06                 mov    (%rsi),%eax
801: 66 83 f8 35           cmp    $0x35,%ax
805: 0f 84 d0 02 00 00     je     adb <sk_run_filter+0x31d>
80b: 0f 87 07 01 00 00     ja     918 <sk_run_filter+0x15a>
811: 66 83 f8 15           cmp    $0x15,%ax
815: 0f 84 c5 02 00 00     je     ae0 <sk_run_filter+0x322>
81b: 77 73                 ja     890 <sk_run_filter+0xd2>
81d: 66 83 f8 04           cmp    $0x4,%ax
821: 0f 84 17 02 00 00     je     a3e <sk_run_filter+0x280>
827: 77 29                 ja     852 <sk_run_filter+0x94>
829: 66 83 f8 01           cmp    $0x1,%ax
[...]

With the modification the compiler translate the switch statement into
the following jump table fragment:

7ff: 66 83 3e 2c           cmpw   $0x2c,(%rsi)
803: 0f 87 1f 02 00 00     ja     a28 <sk_run_filter+0x26a>
809: 0f b7 06              movzwl (%rsi),%eax
80c: ff 24 c5 00 00 00 00  jmpq   *0x0(,%rax,8)
813: 44 89 e3              mov    %r12d,%ebx
816: e9 43 03 00 00        jmpq   b5e <sk_run_filter+0x3a0>
81b: 41 89 dc              mov    %ebx,%r12d
81e: e9 3b 03 00 00        jmpq   b5e <sk_run_filter+0x3a0>

Furthermore, I reordered the instructions to reduce cache line misses by
order the most common instruction to the start.

Signed-off-by: Hagen Paul Pfeifer <hagen@jauu.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-06-25 21:33:12 -07:00
..
datagram.c net: fix lock_sock_bh/unlock_sock_bh 2010-05-27 00:30:53 -07:00
dev_addr_lists.c net: include linux/proc_fs.h in dev_addr_lists.c 2010-04-07 16:46:36 -07:00
dev.c net: consolidate netif_needs_gso() checks 2010-06-23 12:58:41 -07:00
drop_monitor.c tracing: Let tracepoints have data passed to tracepoint callbacks 2010-05-14 09:50:34 -04:00
dst.c dst: don't inline dst_ifdown 2010-04-13 03:32:44 -07:00
ethtool.c net: Remove unnecessary semicolons after switch statements 2010-05-17 17:44:35 -07:00
fib_rules.c net: fib_rules: mark arguments to fib_rules_register const and __net_initdata 2010-04-26 16:02:04 +02:00
filter.c net: optimize Berkeley Packet Filter (BPF) processing 2010-06-25 21:33:12 -07:00
flow.c flow: delayed deletion of flow cache entries 2010-04-07 03:43:20 -07:00
gen_estimator.c pkt_sched: gen_kill_estimator() rcu fixes 2010-06-11 18:37:08 -07:00
gen_stats.c pkt_sched: gen_estimator: Dont report fake rate estimators 2009-10-07 01:07:42 -07:00
iovec.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
kmap_skb.h [PATCH] severing skbuff.h -> highmem.h 2006-12-04 02:00:29 -05:00
link_watch.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
Makefile net: convert multicast list to list_head 2010-04-03 14:22:15 -07:00
neighbour.c net: fix __neigh_event_send() 2010-05-28 01:57:16 -07:00
net_namespace.c net: disallow to use net_assign_generic externally 2010-04-27 15:49:02 -07:00
net-sysfs.c net: Enable 64-bit net device statistics on 32-bit architectures 2010-06-12 15:51:22 -07:00
net-sysfs.h net: Expose all network devices in a namespaces in sysfs 2010-05-21 09:37:34 -07:00
net-traces.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
netevent.c [NET]: net/core/netevent.c should #include <net/netevent.h> 2007-07-05 17:40:27 -07:00
netpoll.c net: fix "netpoll: Allow netpoll_setup/cleanup recursion" 2010-06-24 20:33:04 -07:00
pktgen.c pktgen: increasing transmission granularity 2010-06-11 18:37:09 -07:00
request_sock.c net: convert BUG_TRAP to generic WARN_ON 2008-07-25 21:43:18 -07:00
rtnetlink.c net: Enable 64-bit net device statistics on 32-bit architectures 2010-06-12 15:51:22 -07:00
scm.c scm: Capture the full credentials of the scm sender. 2010-06-16 14:55:56 -07:00
skbuff.c net: rxhash already set in __copy_skb_header 2010-06-13 17:16:54 -07:00
sock.c net: Export cred_to_ucred to modules. 2010-06-16 16:18:25 -07:00
stream.c net: sock_def_readable() and friends RCU conversion 2010-05-01 15:00:15 -07:00
sysctl_net_core.c net: Consistent skb timestamping 2010-05-15 23:57:10 -07:00
user_dma.c net/core/user_dma.c: Use frag list abstraction interfaces. 2009-06-09 00:19:10 -07:00
utils.c printk: Remove ratelimit.h from kernel.h 2009-09-22 16:18:09 +02:00