Jens Axboe 595e52284d io_uring/poll: don't enable lazy wake for POLLEXCLUSIVE ... There are a few quirks around using lazy wake for poll unconditionally, and one of them is related the EPOLLEXCLUSIVE. Those may trigger exclusive wakeups, which wake a limited number of entries in the wait queue. If that wake number is less than the number of entries someone is waiting for (and that someone is also using DEFER_TASKRUN), then we can get stuck waiting for more entries while we should be processing the ones we already got. If we're doing exclusive poll waits, flag the request as not being compatible with lazy wakeups. Reported-by: Pavel Begunkov <asml.silence@gmail.com> Fixes: 6ce4a93dbb ("io_uring/poll: use IOU_F_TWQ_LAZY_WAKE for wakeups") Signed-off-by: Jens Axboe <axboe@kernel.dk>		2023-12-13 08:58:15 -07:00
..
advise.c	io_uring: always go async for unsupported fadvise flags	2023-01-29 15:18:26 -07:00
advise.h
alloc_cache.h	io_uring/rsrc: consolidate node caching	2023-04-12 12:09:41 -06:00
cancel.c	io_uring: use fget/fput consistently	2023-11-28 11:56:29 -07:00
cancel.h	io_uring: add support for futex wake and wait	2023-09-29 02:36:57 -06:00
epoll.c	io_uring: undeprecate epoll_ctl support	2023-05-26 20:22:41 -06:00
epoll.h
fdinfo.c	io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval	2023-11-15 06:35:46 -07:00
fdinfo.h
filetable.c	io_uring: add helpers to decode the fixed file file_ptr	2023-06-20 09:36:22 -06:00
filetable.h	io_uring: add helpers to decode the fixed file file_ptr	2023-06-20 09:36:22 -06:00
fs.c	io_uring/fs: consider link->flags when getting path for LINKAT	2023-11-20 09:01:42 -07:00
fs.h
futex.c	io_uring: add support for vectored futex waits	2023-09-29 02:37:08 -06:00
futex.h	io_uring: add support for vectored futex waits	2023-09-29 02:37:08 -06:00
io_uring.c	io_uring: fix mutex_unlock with unreferenced ctx	2023-12-03 19:09:28 -07:00
io_uring.h	io_uring: enable io_mem_alloc/free to be used in other parts	2023-11-27 20:53:52 -07:00
io-wq.c	io-wq: fully initialize wqe before calling cpuhp_state_add_instance_nocalls()	2023-10-05 14:11:18 -06:00
io-wq.h	io_uring: break out of iowq iopoll on teardown	2023-09-07 09:02:27 -06:00
kbuf.c	io_uring/kbuf: check for buffer list readiness after NULL check	2023-12-05 07:02:13 -07:00
kbuf.h	io_uring: free io_buffer_list entries via RCU	2023-11-28 11:45:02 -07:00
Makefile	io_uring: add support for futex wake and wait	2023-09-29 02:36:57 -06:00
msg_ring.c	io_uring: use io_file_from_index in io_msg_grab_file	2023-06-20 09:36:22 -06:00
msg_ring.h	io_uring: get rid of double locking	2022-12-07 06:47:13 -07:00
net.c	io_uring/net: ensure socket is marked connected on connect retry	2023-11-03 13:25:50 -06:00
net.h	io_uring: Add KASAN support for alloc_caches	2023-04-03 07:16:14 -06:00
nop.c
nop.h
notif.c	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
notif.h	io_uring/notif: add constant for ubuf_info flags	2023-04-15 14:21:04 -06:00
opdef.c	io_uring/rw: add separate prep handler for fixed read/write	2023-11-06 07:43:16 -07:00
opdef.h	io_uring/rw: mark readv/writev as vectored in the opcode definition	2023-09-21 12:00:46 -06:00
openclose.c	io_uring: use files_lookup_fd_locked()	2023-10-19 11:02:49 +02:00
openclose.h
poll.c	io_uring/poll: don't enable lazy wake for POLLEXCLUSIVE	2023-12-13 08:58:15 -07:00
poll.h	io_uring: avoid indirect function calls for the hottest task_work	2023-06-02 08:55:37 -06:00
refs.h
rsrc.c	io_uring: fix off-by one bvec index	2023-11-20 15:21:38 -07:00
rsrc.h	io_uring/af_unix: disable sending io_uring over sockets	2023-12-07 10:35:19 -07:00
rw.c	io_uring: do not clamp read length for multishot read	2023-11-06 13:41:58 -07:00
rw.h	io_uring/rw: add separate prep handler for fixed read/write	2023-11-06 07:43:16 -07:00
slist.h	io_uring: silence variable ‘prev’ set but not used warning	2023-03-09 10:10:58 -07:00
splice.c	io_uring/splice: use fput() directly	2023-08-10 10:24:25 -06:00
splice.h
sqpoll.c	io_uring/fdinfo: remove need for sqpoll lock for thread/pid retrieval	2023-11-15 06:35:46 -07:00
sqpoll.h	io_uring/sqpoll: fix io-wq affinity when IORING_SETUP_SQPOLL is used	2023-08-16 13:40:28 -06:00
statx.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
statx.h
sync.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
sync.h
tctx.c	io_uring: Add io_uring_setup flag to pre-register ring fd and never install it	2023-05-16 08:06:00 -06:00
tctx.h	io_uring: simplify __io_uring_add_tctx_node	2022-10-07 12:25:30 -06:00
timeout.c	io_uring: never overflow io_aux_cqe	2023-08-11 10:42:57 -06:00
timeout.h	io_uring: remove unused return from io_disarm_next	2022-09-21 13:15:01 -06:00
uring_cmd.c	io_uring/cmd: Introduce SOCKET_URING_OP_SETSOCKOPT	2023-10-19 16:42:03 -06:00
uring_cmd.h	io_uring: Remove unnecessary BUILD_BUG_ON	2023-05-04 08:19:05 -06:00
waitid.c	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
waitid.h	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
xattr.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
xattr.h