leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity
196e402adf
linux/fs
History
Harshad Shirwadkar 196e402adf ext4: improve cr 0 / cr 1 group scanning 
Instead of traversing through groups linearly, scan groups in specific
orders at cr 0 and cr 1. At cr 0, we want to find groups that have the
largest free order >= the order of the request. So, with this patch,
we maintain lists for each possible order and insert each group into a
list based on the largest free order in its buddy bitmap. During cr 0
allocation, we traverse these lists in the increasing order of largest
free orders. This allows us to find a group with the best available cr
0 match in constant time. If nothing can be found, we fallback to cr 1
immediately.

At CR1, the story is slightly different. We want to traverse in the
order of increasing average fragment size. For CR1, we maintain a rb
tree of groupinfos which is sorted by average fragment size. Instead
of traversing linearly, at CR1, we traverse in the order of increasing
average fragment size, starting at the most optimal group. This brings
down cr 1 search complexity to log(num groups).

For cr >= 2, we just perform the linear search as before. Also, in
case of lock contention, we intermittently fallback to linear search
even in CR 0 and CR 1 cases. This allows us to proceed during the
allocation path even in case of high contention.

There is an opportunity to do optimization at CR2 too. That's because
at CR2 we only consider groups where bb_free counter (number of free
blocks) is greater than the request extent size. That's left as future
work.

All the changes introduced in this patch are protected under a new
mount option "mb_optimize_scan".

With this patchset, following experiment was performed:

Created a highly fragmented disk of size 65TB. The disk had no
contiguous 2M regions. Following command was run consecutively for 3
times:

time dd if=/dev/urandom of=file bs=2M count=10

Here are the results with and without cr 0/1 optimizations introduced
in this patch:

|---------+------------------------------+---------------------------|
|         | Without CR 0/1 Optimizations | With CR 0/1 Optimizations |
|---------+------------------------------+---------------------------|
| 1st run | 5m1.871s                     | 2m47.642s                 |
| 2nd run | 2m28.390s                    | 0m0.611s                  |
| 3rd run | 2m26.530s                    | 0m1.255s                  |
|---------+------------------------------+---------------------------|

Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Link: https://lore.kernel.org/r/20210401172129.189766-6-harshadshirwadkar@gmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-04-09 11:34:59 -04:00
..
9p Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-27 08:07:12 -08:00
adfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
affs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
afs afs: Stop listxattr() from listing "afs.*" attributes 2021-03-15 17:09:54 +00:00
autofs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
befs [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
bfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
btrfs for-5.12-rc3-tag 2021-03-18 13:38:42 -07:00
cachefiles idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
ceph idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
cifs cifs: fix allocation size on newly created files 2021-03-19 11:51:31 -05:00
coda fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
configfs configfs: fix a use-after-free in __configfs_open_file 2021-03-11 12:13:48 +01:00
cramfs cramfs: use %pD instead of messing with file_dentry()->d_name 2021-01-05 23:02:47 -05:00
crypto block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
debugfs Driver core / debugfs update for 5.12-rc1 2021-02-24 10:13:55 -08:00
devpts
dlm fs: dlm: check on existing node address 2020-11-10 12:14:20 -06:00
ecryptfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
efivarfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
efs [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
erofs Change since last update: 2021-03-13 12:26:22 -08:00
exfat idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
exportfs exportfs: Add a function to return the raw output from fh_to_dentry() 2020-12-09 09:39:38 -05:00
ext2 fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
ext4 ext4: improve cr 0 / cr 1 group scanning 2021-04-09 11:34:59 -04:00
f2fs block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
fat idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
freevxfs
fscache
fuse fuse: 32-bit user space ioctl compat for fuse device 2021-03-16 15:20:16 +01:00
gfs2 block-5.12-2021-03-12-v2 2021-03-12 13:25:49 -08:00
hfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hfsplus idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
hostfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
hpfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
hugetlbfs hugetlbfs: remove unneeded return value of hugetlb_vmtruncate() 2021-02-24 13:38:35 -08:00
iomap Merge branch 'iomap-5.12-fixes' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux 2021-03-18 10:37:30 -07:00
isofs isofs: handle large user and group ID 2021-02-03 19:05:52 +01:00
jbd2 block: use an on-stack bio in blkdev_issue_flush 2021-01-27 09:51:48 -07:00
jffs2 idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
jfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-27 08:07:12 -08:00
kernfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
lockd SUNRPC: Make trace_svc_process() display the RPC procedure symbolically 2021-01-25 09:36:23 -05:00
minix fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
nfs nfs: we don't support removing system.nfs4_acl 2021-03-11 13:17:42 -05:00
nfs_common NFSv4_2: SSC helper should use its own config. 2021-01-28 10:55:37 -05:00
nfsd NFSD: fix error handling in NFSv4.0 callbacks 2021-03-11 10:58:49 -05:00
nilfs2 block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
nls
notify idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
ntfs ntfs: check for valid standard information attribute 2021-02-24 13:38:26 -08:00
ocfs2 ocfs2: simplify the calculation of variables 2021-02-24 13:38:26 -08:00
omfs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
openpromfs
orangefs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
overlayfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
proc mm: use is_cow_mapping() across tree where proper 2021-03-13 11:27:30 -08:00
pstore pstore fixes for v5.12-rc2 2021-03-05 17:21:25 -08:00
qnx4 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
qnx6 [PATCH] reduce boilerplate in fsid handling 2020-09-18 16:45:50 -04:00
quota quota: Fix memory leak when handling corrupted quota file 2021-01-05 14:42:18 +01:00
ramfs ramfs: support O_TMPFILE 2021-02-24 13:38:26 -08:00
reiserfs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
romfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-10-24 12:26:05 -07:00
squashfs block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
sysfs sysfs: Support zapping of binary attr mmaps 2021-01-12 14:26:31 +01:00
sysv fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
tracefs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
ubifs idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
udf idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
ufs fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
unicode unicode: Add utf8_casefold_hash 2020-09-10 14:03:31 -07:00
vboxsf fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
verity idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
xfs xfs: also reject BULKSTAT_SINGLE in a mount user namespace 2021-03-15 08:50:41 -07:00
zonefs zonefs: fix to update .i_wr_refcnt correctly in zonefs_open_zone() 2021-03-17 08:56:50 +09:00
aio.c Merge branch 'akpm' (patches from Andrew) 2020-12-15 12:53:37 -08:00
anon_inodes.c fs: anon_inodes: rephrase to appropriate kernel-doc 2021-01-15 12:17:25 -05:00
attr.c ima: handle idmapped mounts 2021-01-24 14:27:20 +01:00
bad_inode.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
binfmt_aout.c
binfmt_elf_fdpic.c Merge branch 'parisc-5.12-1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux 2021-02-21 13:20:41 -08:00
binfmt_elf.c Merge branch 'parisc-5.12-1' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux 2021-02-21 13:20:41 -08:00
binfmt_em86.c
binfmt_flat.c binfmt_flat: revert "binfmt_flat: don't offset the data start" 2020-08-24 08:49:13 +10:00
binfmt_misc.c binfmt_misc: fix possible deadlock in bm_register_write 2021-03-13 11:27:30 -08:00
binfmt_script.c
block_dev.c block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
buffer.c fs: buffer: use raw page_memcg() on locked page 2021-02-24 13:38:30 -08:00
char_dev.c
compat_binfmt_elf.c get rid of COMPAT_ELF_EXEC_PAGESIZE 2021-01-06 08:42:51 -05:00
coredump.c fs/coredump: use kmap_local_page() 2021-02-26 09:41:05 -08:00
d_path.c fs: fix NULL dereference due to data race in prepend_path() 2020-10-14 14:54:45 -07:00
dax.c mm: provide a saner PTE walking API for modules 2021-02-09 07:05:44 -05:00
dcache.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
direct-io.c block-5.12-2021-02-27 2021-02-28 11:23:38 -08:00
drop_caches.c
eventfd.c eventfd: Export eventfd_ctx_do_read() 2020-11-15 09:49:10 -05:00
eventpoll.c kcmp: Support selection of SYS_kcmp without CHECKPOINT_RESTORE 2021-02-16 09:59:41 +01:00
exec.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
fcntl.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
fhandle.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
file_table.c epoll: take epitem list out of struct file 2020-10-25 20:02:08 -04:00
file.c fs: provide locked helper variant of close_fd_get_file() 2021-02-01 10:02:42 -07:00
filesystems.c
fs_context.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
fs_parser.c fs_parse: mark fs_param_bad_value() as static 2020-10-13 18:38:27 -07:00
fs_pin.c
fs_struct.c vfs: Use sequence counter with associated spinlock 2020-07-29 16:14:27 +02:00
fs_types.c
fs-writeback.c fs: improve comments for writeback_single_inode() 2021-01-13 17:26:50 +01:00
fsopen.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
init.c init: handle idmapped mounts 2021-01-24 14:27:19 +01:00
inode.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-27 08:07:12 -08:00
internal.h idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
io_uring.c io_uring: call req_set_fail_links() on short send[msg]()/recv[msg]() with MSG_WAITALL 2021-03-21 09:41:14 -06:00
io-wq.c io-wq: ensure task is running before processing task_work 2021-03-21 09:41:14 -06:00
io-wq.h io_uring: remove structures from include/linux/io_uring.h 2021-03-18 09:44:35 -06:00
ioctl.c fs: remove ksys_ioctl 2020-07-31 08:16:01 +02:00
Kconfig s390,alpha: make TMPFS_INODE64 available again 2021-03-08 10:46:30 +01:00
Kconfig.binfmt Merge branch 'work.elf-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-21 09:29:23 -08:00
kernel_read_file.c fs/kernel_file_read: Add "offset" arg for partial reads 2020-10-05 13:37:04 +02:00
libfs.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
locks.c Revert "nfsd4: a client's own opens needn't prevent delegations" 2021-03-09 10:37:34 -05:00
Makefile fs: Remove dcookies support 2021-01-29 10:06:46 +05:30
mbcache.c
mount.h mount: make {lock,unlock}_mount_hash() static 2021-01-24 14:29:34 +01:00
mpage.c block: rename BIO_MAX_PAGES to BIO_MAX_VECS 2021-03-11 07:47:48 -07:00
namei.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
namespace.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2021-02-27 08:07:12 -08:00
no-block.c
nsfs.c
open.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
pipe.c fs: delete repeated words in comments 2021-02-24 13:38:26 -08:00
pnode.c
pnode.h mount: fix mounting of detached mounts onto targets that reside on shared mounts 2021-03-08 15:18:43 +01:00
posix_acl.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
proc_namespace.c fs: introduce MOUNT_ATTR_IDMAP 2021-01-24 14:43:45 +01:00
read_write.c teach sendfile(2) to handle send-to-pipe directly 2021-01-25 23:29:36 -05:00
readdir.c fs: remove ksys_getdents64 2020-07-31 08:16:00 +02:00
remap_range.c ioctl: handle idmapped mounts 2021-01-24 14:27:19 +01:00
select.c kernel, fs: Introduce and use set_restart_fn() and arch_set_restart_data() 2021-03-16 22:13:10 +01:00
seq_file.c fs: fix kernel-doc markups 2021-01-21 14:06:00 -07:00
signalfd.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
splice.c for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
stack.c
stat.c fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
statfs.c s390,alpha: switch to 64-bit ino_t 2021-02-13 17:17:53 +01:00
super.c It has been a relatively quiet cycle in docsland. 2021-02-22 10:57:46 -08:00
sync.c
timerfd.c
userfaultfd.c userfaultfd: use secure anon inodes for userfaultfd 2021-01-14 17:40:57 -05:00
utimes.c utimes: handle idmapped mounts 2021-01-24 14:27:18 +01:00
xattr.c namei: handle idmapped mounts in may_*() helpers 2021-01-24 14:27:17 +01:00