linux/io_uring
Pavel Begunkov dfbbfbf191 io_uring: introduce concept of memory regions
We've got a good number of mappings we share with the userspace, that
includes the main rings, provided buffer rings, upcoming rings for
zerocopy rx and more. All of them duplicate user argument parsing and
some internal details as well (page pinnning, huge page optimisations,
mmap'ing, etc.)

Introduce a notion of regions. For userspace for now it's just a new
structure called struct io_uring_region_desc which is supposed to
parameterise all such mapping / queue creations. A region either
represents a user provided chunk of memory, in which case the user_addr
field should point to it, or a request for the kernel to allocate the
memory, in which case the user would need to mmap it after using the
offset returned in the mmap_offset field. With a uniform userspace API
we can avoid additional boiler plate code and apply future optimisation
to all of them at once.

Internally, there is a new structure struct io_mapped_region holding all
relevant runtime information and some helpers to work with it. This
patch limits it to user provided regions.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/0e6fe25818dfbaebd1bd90b870a6cac503fe1a24.1731689588.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2024-11-15 09:58:34 -07:00
..
advise.c io_uring/advise: support 64-bit lengths 2024-06-16 14:54:55 -06:00
advise.h
alloc_cache.h io_uring/alloc_cache: switch to array based caching 2024-04-15 08:10:25 -06:00
cancel.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
cancel.h io_uring/cancel: get rid of init_hash_table() helper 2024-10-29 13:43:27 -06:00
epoll.c io_uring: undeprecate epoll_ctl support 2023-05-26 20:22:41 -06:00
epoll.h
eventfd.c io_uring/eventfd: move ctx->evfd_last_cq_tail into io_ev_fd 2024-10-29 13:43:26 -06:00
eventfd.h io_uring/eventfd: move eventfd handling to separate file 2024-06-16 14:54:55 -06:00
fdinfo.c io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
fdinfo.h
filetable.c io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers 2024-11-07 15:24:33 -07:00
filetable.h io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers 2024-11-07 15:24:33 -07:00
fs.c io_uring/fs: consider link->flags when getting path for LINKAT 2023-11-20 09:01:42 -07:00
fs.h
futex.c io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
futex.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
io_uring.c io_uring: temporarily disable registered waits 2024-11-15 09:58:34 -07:00
io_uring.h io_uring: avoid normal tw intermediate fallback 2024-11-06 13:55:38 -07:00
io-wq.c io_uring/io-wq: inherit cpuset of cgroup in io worker 2024-09-11 07:27:56 -06:00
io-wq.h io_uring/io-wq: make io_wq_work flags atomic 2024-06-16 14:54:55 -06:00
kbuf.c for-6.12/io_uring-20240913 2024-09-16 13:29:00 +02:00
kbuf.h io_uring/kbuf: add support for incremental buffer consumption 2024-08-29 08:44:58 -06:00
Makefile io_uring: add GCOV_PROFILE_URING Kconfig option 2024-08-30 10:52:02 -06:00
memmap.c io_uring: introduce concept of memory regions 2024-11-15 09:58:34 -07:00
memmap.h io_uring: introduce concept of memory regions 2024-11-15 09:58:34 -07:00
msg_ring.c switch io_msg_ring() to CLASS(fd) 2024-11-15 09:55:54 -07:00
msg_ring.h io_uring/msg_ring: add support for sending a sync message 2024-10-29 13:43:26 -06:00
napi.c io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
napi.h io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
net.c io_uring/rsrc: add & apply io_req_assign_buf_node() 2024-11-07 15:24:33 -07:00
net.h io_uring: Introduce IORING_OP_LISTEN 2024-06-19 07:57:21 -06:00
nop.c io_uring/rsrc: add & apply io_req_assign_buf_node() 2024-11-07 15:24:33 -07:00
nop.h
notif.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
notif.h io_uring/notif: implement notification stacking 2024-04-22 19:31:18 -06:00
opdef.c io_uring/splice: open code 2nd direct file assignment 2024-10-29 13:43:28 -06:00
opdef.h io_uring: Fix probe of disabled operations 2024-06-19 08:58:00 -06:00
openclose.c io_uring: enable audit and restrict cred override for IORING_OP_FIXED_FD_INSTALL 2024-01-23 15:25:14 -07:00
openclose.h io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL 2023-12-12 07:42:57 -07:00
poll.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
poll.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
refs.h io_uring: kill dead code in io_req_complete_post 2024-04-15 08:10:26 -06:00
register.c io_uring: temporarily disable registered waits 2024-11-15 09:58:34 -07:00
register.h io_uring: temporarily disable registered waits 2024-11-15 09:58:34 -07:00
rsrc.c io_uring/rsrc: remove '->ctx_ptr' of 'struct io_rsrc_node' 2024-11-07 15:24:33 -07:00
rsrc.h io_uring/rsrc: add & apply io_req_assign_buf_node() 2024-11-07 15:24:33 -07:00
rw.c io_uring/rsrc: add & apply io_req_assign_buf_node() 2024-11-07 15:24:33 -07:00
rw.h io_uring/alloc_cache: switch to array based caching 2024-04-15 08:10:25 -06:00
slist.h io_uring: silence variable ‘prev’ set but not used warning 2023-03-09 10:10:58 -07:00
splice.c io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers 2024-11-07 15:24:33 -07:00
splice.h io_uring/splice: open code 2nd direct file assignment 2024-10-29 13:43:28 -06:00
sqpoll.c io_uring/sqpoll: wait on sqd->wait for thread parking 2024-10-29 13:43:27 -06:00
sqpoll.h io_uring/sqpoll: statistics of the true utilization of sq threads 2024-03-01 06:28:19 -07:00
statx.c vfs: retire user_path_at_empty and drop empty arg from getname_flags 2024-06-05 17:03:57 +02:00
statx.h
sync.c io_uring: for requests that require async, force it 2023-01-29 15:18:26 -07:00
sync.h
tctx.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
timeout.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
truncate.c io_uring: add support for ftruncate 2024-02-09 09:04:39 -07:00
truncate.h io_uring: add support for ftruncate 2024-02-09 09:04:39 -07:00
uring_cmd.c io_uring/uring_cmd: fix buffer index retrieval 2024-11-11 08:11:37 -07:00
uring_cmd.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
waitid.c io_uring: move struct io_kiocb from task_struct to io_uring_task 2024-11-06 13:55:38 -07:00
waitid.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
xattr.c vfs: retire user_path_at_empty and drop empty arg from getname_flags 2024-06-05 17:03:57 +02:00
xattr.h