linux/fs
Linus Torvalds 9ccce092fc orangefs: implement orangefs_readahead
mm/readahead.c/read_pages was quite a bit different back
 when I put my open-coded readahead logic into orangefs_readpage.
 It seemed to work as designed then, it is a trainwreck now.
 
 This patch implements orangefs_readahead using new xarray
 and readahead_expand features that have just been pulled and
 removes all my open-coded readahead logic.
 
 This patch results in an extreme read performance improvement,
 these sample numbers are from my test VM:
 
 Here's an example of what's upstream in
 5.11.8-200.fc33.x86_64:
 
 30+0 records in
 30+0 records out
 125829120 bytes (126 MB, 120 MiB) copied, 5.77943 s, 21.8 MB/s
 
 And here's this version of orangefs_readahead on top of
 5.12.0-rc4:
 
 30+0 records in
 30+0 records out
 125829120 bytes (126 MB, 120 MiB) copied, 0.325919 s, 386 MB/s
 
 There are four xfstest regressions with this patch. David Howells
 and Matthew Wilcox have been helping me work with this code. One
 of the regressions has gone away with the most recent version of
 their code that I'm using. I hope this patch can be
 pulled even though there are still a few regressions, and that
 we can try to get them resolved during the RC period.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEIGSFVdO6eop9nER2z0QOqevODb4FAmCPCUsACgkQz0QOqevO
 Db77DQ/7B8V7RPlQ8C6HJlSuCED67W9isCG5CdzGobVafBrirbUusanQJRhjrIZO
 Voy0NYsR/rsM3K1tNk9AE7rlbT4UQibeUXwFVcVjBvtyXBiTgjbROc2AP4pjxAWu
 erH2McMEbrYjgrevwR/PKxyD8wS6vTX2InnI4yvlkbfEz04u/KkTSu0oN4UCU/8u
 8/drWDTIgZz6wffb1RpMFsCP77tfVWIWlRlH39u9OTe4fhPMug8jN+uOBrfyYxdp
 snJWznyeSYCQ4q/KkPkjfSUTDmx3+E1WeSHMNviHfwENdbcUAojk2O9wepBwJhQn
 r0DFU2yM+132oRkWO1DF7If1FRfvcmHjE4bmlLBSg+xgKOKpdMCs7Nf+s1Sji+w/
 8xTAPWzdqBeW6z4nIncvZPtjtes3979mJ/Jm/f4GLonAQB6yPJcIzA8gl5EEgXI3
 20pAt2JNCgCHVhHQso5fkLINlpND/cwlbOEOjyrNXIoJJngGDRo9FQ/osGBaLv5i
 n3XWC41lYnX9nqJ2FuVLBuZ+Jv1k5XSQualpyGGVTFaYp/jZVbjUOgJk7QPNsWl7
 9cUZAMVdDW6y7z1aZ2bu5y7VFIkPe4nfZNqrgXX+YySq0uOTrQBegkQRp1pu3t8m
 P3P9lVqcrn/kw+FASZborq921Njw+YDHvZuYfrnbF7J0sUL0fu4=
 =09Vm
 -----END PGP SIGNATURE-----

Merge tag 'for-linus-5.13-ofs-1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux

Pull orangefs updates from Mike Marshall:
 "orangefs: implement orangefs_readahead

  mm/readahead.c/read_pages was quite a bit different back when I put my
  open-coded readahead logic into orangefs_readpage. That logic seemed
  to work as designed back then, it is a trainwreck now.

  This implements orangefs_readahead using the new xarray and
  readahead_expand features and removes all my open-coded readahead
  logic.

  This results in an extreme read performance improvement, these sample
  numbers are from my test VM:

  Here's an example of what's upstream in
  5.11.8-200.fc33.x86_64:

     30+0 records in
     30+0 records out
     125829120 bytes (126 MB, 120 MiB) copied, 5.77943 s, 21.8 MB/s

  And here's this version of orangefs_readahead on top of 5.12.0-rc4:

     30+0 records in
     30+0 records out
     125829120 bytes (126 MB, 120 MiB) copied, 0.325919 s, 386 MB/s

  There are four xfstest regressions with this patch. David Howells and
  Matthew Wilcox have been helping me work with this code"

* tag 'for-linus-5.13-ofs-1' of git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux:
  orangefs: leave files in the page cache for a few micro seconds at least
  Orangef: implement orangefs_readahead.
2021-05-02 14:13:46 -07:00
..
9p 9p: missing chunk of "fs/9p: Don't update file type when updating file attributes" 2021-03-12 22:15:22 -05:00
adfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
affs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
afs afs: Fix speculative status fetches 2021-05-01 11:55:36 -07:00
autofs autofs: should_expire() argument is guaranteed to be positive 2021-03-24 14:14:27 -04:00
befs fs/befs: Delete obsolete TODO file 2021-03-30 16:54:49 -07:00
bfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
btrfs Merge branch 'miklos.fileattr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 11:18:24 -07:00
cachefiles fscache, cachefiles: Add alternate API to use kiocb for read/write to cache 2021-04-23 10:14:32 +01:00
ceph ceph: don't allow type or device number to change on non-I_NEW inodes 2021-03-08 10:19:37 -05:00
cifs Merge branch 'work.inode-type-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 10:57:42 -07:00
coda coda: fix reference counting in coda_file_mmap error path 2021-04-23 14:42:39 -07:00
configfs configfs: fix a use-after-free in __configfs_open_file 2021-03-11 12:13:48 +01:00
cramfs cramfs: use %pD instead of messing with file_dentry()->d_name 2021-01-05 23:02:47 -05:00
crypto Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2021-04-26 08:51:23 -07:00
debugfs Scheduler updates for this cycle are: 2021-04-28 13:33:57 -07:00
devpts
dlm fs: dlm: fix missing unlock on error in accept_from_sock() 2021-03-29 13:28:18 -05:00
ecryptfs Merge branch 'work.ecryptfs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:05:54 -07:00
efivarfs efivars: convert to fileattr 2021-04-12 15:04:29 +02:00
efs [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
erofs erofs: enable big pcluster feature 2021-04-10 03:20:19 +08:00
exfat exfat: speed up iterate/lookup by fixing start point of traversing cluster chain 2021-04-27 20:45:07 +09:00
exportfs exportfs: Add a function to return the raw output from fh_to_dentry() 2020-12-09 09:39:38 -05:00
ext2 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
ext4 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
f2fs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
fat idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
freevxfs
fscache fscache, cachefiles: Add alternate API to use kiocb for read/write to cache 2021-04-23 10:14:32 +01:00
fuse Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
gfs2 Changes in gfs2: 2021-04-29 10:33:35 -07:00
hfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hfsplus hfsplus: convert to fileattr 2021-04-12 15:04:29 +02:00
hostfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
hpfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hugetlbfs hugetlbfs: remove unneeded return value of hugetlb_vmtruncate() 2021-02-24 13:38:35 -08:00
iomap iomap: use filemap_range_needs_writeback() for O_DIRECT reads 2021-04-30 11:20:36 -07:00
isofs isofs: handle large user and group ID 2021-02-03 19:05:52 +01:00
jbd2 ext4: fix debug format string warning 2021-04-09 23:32:16 -04:00
jffs2 fs/jffs2: Delete obsolete TODO file 2021-03-30 16:54:49 -07:00
jfs jfs: convert to fileattr 2021-04-12 15:04:29 +02:00
kernfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
lockd SUNRPC: Make trace_svc_process() display the RPC procedure symbolically 2021-01-25 09:36:23 -05:00
minix fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
netfs netfs: Miscellaneous fixes 2021-04-26 23:23:41 +01:00
nfs selinux/stable-5.13 PR 20210426 2021-04-27 13:42:11 -07:00
nfs_common NFSD: Add an xdr_stream-based encoder for NFSv2/3 ACLs 2021-03-22 10:19:00 -04:00
nfsd Merge branch 'work.inode-type-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 10:57:42 -07:00
nilfs2 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-05-02 09:14:01 -07:00
nls
notify fanotify_user: use upper_32_bits() to verify mask 2021-03-25 15:33:45 +01:00
ntfs ntfs: check for valid standard information attribute 2021-02-24 13:38:26 -08:00
ocfs2 ocfs2/dlm: remove unused function 2021-04-30 11:20:35 -07:00
omfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
openpromfs openpromfs: don't do unlock_new_inode() until the new inode is set up 2021-03-12 22:15:22 -05:00
orangefs orangefs: leave files in the page cache for a few micro seconds at least 2021-04-29 08:06:05 -04:00
overlayfs overlayfs update for 5.13 2021-04-30 15:17:08 -07:00
proc Kbuild updates for v5.13 2021-04-29 14:24:39 -07:00
pstore printk changes for 5.13 2021-04-27 18:09:44 -07:00
qnx4 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
qnx6 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
quota quota: report warning limits for realtime space quotas 2021-03-18 17:03:54 +01:00
ramfs ramfs: support O_TMPFILE 2021-02-24 13:38:26 -08:00
reiserfs \n 2021-04-29 10:51:29 -07:00
romfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-10-24 12:26:05 -07:00
squashfs squashfs: fix xattr id and id lookup sanity checks 2021-03-25 09:22:55 -07:00
sysfs sysfs: Support zapping of binary attr mmaps 2021-01-12 14:26:31 +01:00
sysv fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
tracefs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
ubifs Merge branch 'miklos.fileattr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-04-27 11:18:24 -07:00
udf useful constants: struct qstr for ".." 2021-04-15 22:36:45 -04:00
ufs useful constants: struct qstr for ".." 2021-04-15 22:36:45 -04:00
unicode unicode: Add utf8_casefold_hash 2020-09-10 14:03:31 -07:00
vboxsf vboxsf: don't allow to change the inode type 2021-03-12 22:15:00 -05:00
verity fsverity: relax build time dependency on CRYPTO_SHA256 2021-04-22 17:31:32 +10:00
xfs New code for 5.13: 2021-04-29 10:43:51 -07:00
zonefs \n 2021-04-29 11:06:13 -07:00
aio.c Revert "mremap: don't allow MREMAP_DONTUNMAP on special_mappings and aio" 2021-04-30 11:20:39 -07:00
anon_inodes.c fs: anon_inodes: rephrase to appropriate kernel-doc 2021-01-15 12:17:25 -05:00
attr.c ima: handle idmapped mounts 2021-01-24 14:27:20 +01:00
bad_inode.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
binfmt_aout.c
binfmt_elf_fdpic.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
binfmt_elf.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
binfmt_em86.c
binfmt_flat.c binfmt_flat: revert "binfmt_flat: don't offset the data start" 2020-08-24 08:49:13 +10:00
binfmt_misc.c binfmt_misc: fix possible deadlock in bm_register_write 2021-03-13 11:27:30 -08:00
binfmt_script.c
block_dev.c for-5.13/drivers-2021-04-27 2021-04-28 14:39:37 -07:00
buffer.c buffer: a small optimization in grow_buffers 2021-03-22 11:30:03 -04:00
char_dev.c
compat_binfmt_elf.c get rid of COMPAT_ELF_EXEC_PAGESIZE 2021-01-06 08:42:51 -05:00
coredump.c coredump: don't bother with do_truncate() 2021-03-08 10:21:11 -05:00
d_path.c constify dentry argument of dentry_path()/dentry_path_raw() 2021-03-21 11:43:58 -04:00
dax.c mm: provide a saner PTE walking API for modules 2021-02-09 07:05:44 -05:00
dcache.c useful constants: struct qstr for ".." 2021-04-15 22:36:45 -04:00
direct-io.c fs: direct-io: fix missing sdio->boundary 2021-04-09 14:54:23 -07:00
drop_caches.c
eventfd.c eventfd: Export eventfd_ctx_do_read() 2020-11-15 09:49:10 -05:00
eventpoll.c fs: eventpoll: fix comments & kernel-doc notation 2021-03-06 17:36:53 -07:00
exec.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
fcntl.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
fhandle.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
file_table.c epoll: take epitem list out of struct file 2020-10-25 20:02:08 -04:00
file.c file: fix close_range() for unshare+cloexec 2021-04-02 14:11:10 +02:00
filesystems.c
fs_context.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
fs_parser.c vfs: fs_parser: clean up kernel-doc warnings 2021-04-30 11:20:35 -07:00
fs_pin.c
fs_struct.c
fs_types.c
fs-writeback.c fs: improve comments for writeback_single_inode() 2021-01-13 17:26:50 +01:00
fsopen.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
init.c init: handle idmapped mounts 2021-01-24 14:27:19 +01:00
inode.c fs.idmapped.helpers.v5.13 2021-04-27 12:49:42 -07:00
internal.h idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
io_uring.c for-5.13/io_uring-2021-04-27 2021-04-28 14:56:09 -07:00
io-wq.c io-wq: remove unused io_wqe_need_worker() function 2021-04-20 11:24:22 -06:00
io-wq.h io-wq: eliminate the need for a manager thread 2021-04-11 17:42:00 -06:00
ioctl.c vfs: add fileattr ops 2021-04-12 15:04:23 +02:00
Kconfig netfs: Provide readahead and readpage netfs helpers 2021-04-23 10:14:32 +01:00
Kconfig.binfmt Merge branch 'work.elf-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-21 09:29:23 -08:00
kernel_read_file.c fs/kernel_file_read: Add "offset" arg for partial reads 2020-10-05 13:37:04 +02:00
libfs.c libfs: fix kernel-doc for mnt_userns 2021-03-23 11:20:25 +01:00
locks.c File locking fixes for v5.13 2021-04-26 13:24:39 -07:00
Makefile netfs: Provide readahead and readpage netfs helpers 2021-04-23 10:14:32 +01:00
mbcache.c
mount.h mount: make {lock,unlock}_mount_hash() static 2021-01-24 14:29:34 +01:00
mpage.c block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
namei.c fs.idmapped.helpers.v5.13 2021-04-27 12:49:42 -07:00
namespace.c fs/namespace: correct/improve kernel-doc notation 2021-03-31 14:22:55 -06:00
no-block.c
nsfs.c
open.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
pipe.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
pnode.c
pnode.h mount: fix mounting of detached mounts onto targets that reside on shared mounts 2021-03-08 15:18:43 +01:00
posix_acl.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
proc_namespace.c fs: introduce MOUNT_ATTR_IDMAP 2021-01-24 14:43:45 +01:00
read_write.c teach sendfile(2) to handle send-to-pipe directly 2021-01-25 23:29:36 -05:00
readdir.c readdir: make sure to verify directory entry for legacy interfaces too 2021-04-17 11:39:49 -07:00
remap_range.c ioctl: handle idmapped mounts 2021-01-24 14:27:19 +01:00
select.c kernel, fs: Introduce and use set_restart_fn() and arch_set_restart_data() 2021-03-16 22:13:10 +01:00
seq_file.c seq_file: Add a seq_bprintf function 2021-04-27 15:50:15 -07:00
signalfd.c signal: Introduce TRAP_PERF si_code and si_perf to siginfo 2021-04-16 16:32:41 +02:00
splice.c for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
stack.c
stat.c fs: fix reporting supported extra file attributes for statx() 2021-04-17 23:03:50 -04:00
statfs.c s390,alpha: switch to 64-bit ino_t 2021-02-13 17:17:53 +01:00
super.c fs,security: Add sb_delete hook 2021-04-22 12:22:11 -07:00
sync.c
timerfd.c
userfaultfd.c userfaultfd: use secure anon inodes for userfaultfd 2021-01-14 17:40:57 -05:00
utimes.c utimes: handle idmapped mounts 2021-01-24 14:27:18 +01:00
xattr.c xattr: fix kernel-doc for mnt_userns and vfs xattr helpers 2021-03-23 11:20:26 +01:00