leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

io_uring: optimise io_free_batch_list()

Browse Source 
Delay reading the next node in io_free_batch_list(), allows the compiler
to load the value a bit later improving register spilling in some cases.
With gcc 11.1 it helped to move @task_refs variable from the stack to a
register and optimises out a couple of per request instructions.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/cc9fdfb6f72a4e8bc9918a5e9f2d97869a263ae4.1633373302.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Pavel Begunkov 2021-10-04 20:02:55 +01:00 committed by Jens Axboe
parent c072481ded
commit c1e53a6988
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
fs/io_uring.c
View File

@ -2280,9 +2280,10 @@ static void io_free_batch_list(struct io_ring_ctx *ctx,
struct io_kiocb *req = container_of(node, struct io_kiocb, struct io_kiocb *req = container_of(node, struct io_kiocb,
comp_list); comp_list);
if (!req_ref_put_and_test(req)) {
node = req->comp_list.next; node = req->comp_list.next;
if (!req_ref_put_and_test(req))
continue; continue;
}
io_queue_next(req); io_queue_next(req);
io_dismantle_req(req); io_dismantle_req(req);
@ -2294,6 +2295,7 @@ static void io_free_batch_list(struct io_ring_ctx *ctx,
task_refs = 0; task_refs = 0;
} }
task_refs++; task_refs++;
node = req->comp_list.next;
wq_stack_add_head(&req->comp_list, &ctx->submit_state.free_list); wq_stack_add_head(&req->comp_list, &ctx->submit_state.free_list);
} while (node); } while (node);