Andrey Ignatov
0640c77c46
bpf: Avoid retpoline for bpf_for_each_map_elem
Similarly to 09772d92cd ("bpf: avoid retpoline for
lookup/update/delete calls on maps") and 84430d4232 ("bpf, verifier:
avoid retpoline for map push/pop/peek operation") avoid indirect call
while calling bpf_for_each_map_elem.
Before (a program fragment):
; if (rules_map) {
142: (15) if r4 == 0x0 goto pc+8
143: (bf) r3 = r10
; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0);
144: (07) r3 += -24
145: (bf) r1 = r4
146: (18) r2 = subprog[+5]
148: (b7) r4 = 0
149: (85) call bpf_for_each_map_elem#143680 <-- indirect call via
helper
After (same program fragment):
; if (rules_map) {
142: (15) if r4 == 0x0 goto pc+8
143: (bf) r3 = r10
; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0);
144: (07) r3 += -24
145: (bf) r1 = r4
146: (18) r2 = subprog[+5]
148: (b7) r4 = 0
149: (85) call bpf_for_each_array_elem#170336 <-- direct call
On a benchmark that calls bpf_for_each_map_elem() once and does many
other things (mostly checking fields in skb) with CONFIG_RETPOLINE=y it
makes program faster.
Before:
============================================================================
Benchmark.cpp time/iter iters/s
============================================================================
IngressMatchByRemoteEndpoint 80.78ns 12.38M
IngressMatchByRemoteIP 80.66ns 12.40M
IngressMatchByRemotePort 80.87ns 12.37M
After:
============================================================================
Benchmark.cpp time/iter iters/s
============================================================================
IngressMatchByRemoteEndpoint 73.49ns 13.61M
IngressMatchByRemoteIP 71.48ns 13.99M
IngressMatchByRemotePort 70.39ns 14.21M
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20211006001838.75607-1-rdna@fb.com
2021-10-05 19:22:33 -07:00
..
2021-10-05 19:22:33 -07:00
2021-09-28 09:29:19 +02:00
2021-05-07 00:26:34 -07:00
2021-09-07 12:08:04 -07:00
2021-09-17 11:54:48 -07:00
2021-09-22 10:24:01 -04:00
2021-09-23 11:19:49 -07:00
2021-06-22 11:07:18 -07:00
2021-09-24 14:11:04 +02:00
2021-09-02 13:00:15 -07:00
2021-08-19 12:00:24 +02:00
2021-09-15 17:49:16 +02:00
2021-08-30 19:25:42 +02:00
2021-09-14 13:23:22 -07:00
2021-08-30 14:26:36 -07:00
2021-09-12 11:37:41 -07:00
2021-09-23 11:53:51 +02:00
2021-10-01 19:58:02 -07:00
2021-05-02 00:43:35 +09:00
2021-09-08 11:50:26 -07:00
2021-05-07 00:26:33 -07:00
2021-02-23 10:25:27 -05:00
2021-08-24 18:52:36 -04:00
2020-12-03 14:58:35 +01:00
2021-03-22 15:23:32 -04:00
2021-07-19 22:38:24 -04:00
2021-03-22 15:23:32 -04:00
2021-06-10 20:59:05 -04:00
2021-01-24 14:27:16 +01:00
2021-08-11 13:11:12 -07:00
2021-09-08 15:32:35 -07:00
2021-08-16 18:55:32 +02:00
2021-08-10 18:31:32 +02:00
2021-07-08 11:48:22 -07:00
2021-08-23 16:13:04 -05:00
2021-05-12 11:43:25 +02:00
2021-08-23 13:10:37 -06:00
2020-12-15 22:46:19 -08:00
2021-09-08 18:45:53 -07:00
2021-06-18 11:43:08 +02:00
2021-09-03 23:00:22 +02:00
2021-05-27 04:01:50 +09:00
2021-02-26 09:41:03 -08:00
2021-07-02 12:08:10 -07:00
2021-06-10 10:00:08 +02:00
2021-07-05 10:46:20 +02:00
2021-07-08 11:48:22 -07:00
2020-12-15 19:36:48 -08:00
2021-08-17 17:50:51 +02:00
2021-06-28 22:43:05 +02:00
2021-08-30 16:36:10 +02:00
2021-05-07 00:26:32 -07:00
2021-02-22 12:33:26 +00:00
2021-09-08 15:32:34 -07:00
2021-05-07 00:26:33 -07:00
2021-07-11 11:06:09 -07:00
2021-06-29 17:29:11 -07:00
2021-05-02 00:43:35 +09:00
2021-01-19 10:24:45 +01:00
2021-01-19 10:24:45 +01:00
2021-07-19 11:57:48 +02:00
2021-08-16 18:55:32 +02:00
2021-09-03 09:58:12 -07:00
2021-08-27 16:30:18 +08:00
2021-08-30 16:36:10 +02:00
2021-08-16 14:42:22 +02:00
2021-09-03 09:58:12 -07:00
2021-08-10 12:53:07 +02:00
2021-09-08 11:50:26 -07:00
2021-06-18 11:43:09 +02:00
2021-06-21 13:08:36 +01:00
2020-12-15 22:46:18 -08:00
2020-11-25 18:52:35 +01:00
2021-05-14 19:41:32 -07:00
2021-09-22 10:24:01 -04:00
2021-07-27 11:39:30 -07:00
2020-12-01 10:30:28 +00:00
2021-09-01 14:52:05 -07:00
2021-09-03 10:08:28 -07:00
2021-08-11 14:47:16 +02:00
2021-08-10 14:57:42 +02:00
2021-08-10 22:50:07 +02:00
2021-07-05 10:46:33 +02:00
2021-03-23 16:01:58 +01:00
2021-09-08 15:32:35 -07:00
2021-09-08 12:55:35 -07:00
2021-06-23 16:41:24 -06:00
2021-09-03 10:08:28 -07:00
2021-04-30 11:20:42 -07:00
2020-11-16 08:08:54 -08:00
2021-08-10 10:48:07 -07:00
2021-08-16 11:39:51 -04:00
2021-08-09 15:45:02 -05:00
2021-05-07 00:26:34 -07:00
2021-05-09 13:07:03 -07:00
2021-09-03 09:58:12 -07:00
2021-09-08 11:50:27 -07:00
2021-07-03 11:41:14 -07:00
2021-01-26 11:16:34 +00:00
2021-06-29 10:53:46 -07:00
2021-08-17 07:49:10 -10:00
2021-08-17 07:49:10 -10:00