linux/io_uring
Dylan Yudaken 9b8c54755a io_uring: add io_aux_cqe which allows deferred completion
Use the just introduced deferred post cqe completion state when possible
in io_aux_cqe. If not possible fallback to io_post_aux_cqe.

This introduces a complication because of allow_overflow. For deferred
completions we cannot know without locking the completion_lock if it will
overflow (and even if we locked it, another post could sneak in and cause
this cqe to be in overflow).
However since overflow protection is mostly a best effort defence in depth
to prevent infinite loops of CQEs for poll, just checking the overflow bit
is going to be good enough and will result in at most 16 (array size of
deferred cqes) overflows.

Suggested-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Dylan Yudaken <dylany@meta.com>
Link: https://lore.kernel.org/r/20221124093559.3780686-6-dylany@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-11-25 06:10:04 -07:00
..
advise.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
advise.h
alloc_cache.h
cancel.c io_uring: add IORING_SETUP_DEFER_TASKRUN 2022-09-21 10:30:42 -06:00
cancel.h
epoll.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
epoll.h
fdinfo.c io_uring: fix fdinfo sqe offsets calculation 2022-10-12 16:30:56 -06:00
fdinfo.h
filetable.c
filetable.h io_uring: kill hot path fixed file bitmap debug checks 2022-10-16 17:07:53 -06:00
fs.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
fs.h
io_uring.c io_uring: add io_aux_cqe which allows deferred completion 2022-11-25 06:10:04 -07:00
io_uring.h io_uring: add io_aux_cqe which allows deferred completion 2022-11-25 06:10:04 -07:00
io-wq.c io-wq: Fix memory leak in worker creation 2022-10-20 05:48:59 -07:00
io-wq.h
kbuf.c io_uring: iopoll protect complete_post 2022-11-23 10:45:31 -07:00
kbuf.h io_uring: allow buffer recycling in READV 2022-09-21 10:30:43 -06:00
Makefile
msg_ring.c io_uring/msg_ring: Fix NULL pointer dereference in io_msg_send_fd() 2022-10-19 12:33:33 -07:00
msg_ring.h
net.c io_uring: add io_aux_cqe which allows deferred completion 2022-11-25 06:10:04 -07:00
net.h io_uring/net: zerocopy sendmsg 2022-09-21 13:15:02 -06:00
nop.c
nop.h
notif.c io_uring/net: move mm accounting to a slower path 2022-11-21 07:38:31 -07:00
notif.h io_uring: move zc reporting from the hot path 2022-11-21 07:38:31 -07:00
opdef.c io_uring/opdef: remove 'audit_skip' from SENDMSG_ZC 2022-10-12 16:30:56 -06:00
opdef.h io_uring: add custom opcode hooks on fail 2022-09-21 13:15:02 -06:00
openclose.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
openclose.h
poll.c io_uring: add io_aux_cqe which allows deferred completion 2022-11-25 06:10:04 -07:00
poll.h
refs.h
rsrc.c io_uring: do not always force run task_work in io_uring_register 2022-11-21 07:38:54 -07:00
rsrc.h io_uring: remove FFS_SCM 2022-10-16 17:07:12 -06:00
rw.c io_uring: remove iopoll spinlock 2022-11-23 10:47:07 -07:00
rw.h io_uring/rw: don't lose partial IO result on fail 2022-09-21 13:15:02 -06:00
slist.h
splice.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
splice.h
sqpoll.c
sqpoll.h
statx.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
statx.h
sync.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
sync.h
tctx.c io_uring: remove io_register_submitter 2022-10-07 12:25:30 -06:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: remove io_req_tw_post_queue 2022-11-23 10:44:00 -07:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring: iopoll protect complete_post 2022-11-23 10:45:31 -07:00
uring_cmd.h
xattr.c __io_setxattr(): constify path 2022-09-01 17:39:05 -04:00
xattr.h