linux/fs
Linus Torvalds 720261cfc7 bcachefs changes for 6.11-rc1 (version 2)
Additional fixes on top of the original 6.11 pull request:
 - undefined behaviour fixes, originally noted as breaking userspace LTO
   builds
 - fix a spurious warning in fsck_err, reported by Marcin
 - fix an integer overflow on trans->nr_updates, also reported by Marcin;
   this broke during deletion of highly fragmented indirect extents
 - Add comments for lockdep functions
 
 ======
 
 - Metadata version 1.8: Stripe sectors accounting, BCH_DATA_unstriped
 
 This splits out the accounting of dirty sectors and stripe sectors in
 alloc keys; this lets us see stripe buckets that still have unstriped
 data in them.
 
 This is needed for ensuring that erasure coding is working correctly, as
 well as completing stripe creation after a crash.
 
 - Metadata version 1.9: Disk accounting rewrite
 
 The previous disk accounting scheme relied heavily on percpu counters
 that were also sharded by outstanding journal buffer; it was fast but
 not extensible or scalable, and meant that all accounting counters were
 recorded in every journal entry.
 
 The new disk accounting scheme stores accounting as normal btree keys;
 updates are deltas until they are flushed by the btree write buffer.
 
 This means we have no practical limit on the number of counters, and a
 new tagged union format that's easy to extend.
 
 We now have counters for compression type/ratio, per-snapshot-id usage,
 per-btree-id usage, and pending rebalance work.
 
 - Self healing on read IO/checksum error
 
 data is now automatically rewritten if we get a read error and then a
 successful retry
 
 - Mount API conversion (thanks to Thomas Bertschinger)
 
 - Better lockdep coverage
 
 Previously, btree node locks were tracked individually by lockdep, like
 any other lock. But we may take _many_ btree node locks simultaneously,
 we easily blow through the limit of 48 locks that lockdep can track,
 leading to lockdep turning itself off.
 
 Tracking each btree node lock individually isn't really necessary since
 we have our own cycle detector for deadlock avoidance and centralized
 tracking of btree node locks, so we now have a single lockdep_map in
 btree_trans for "any btree nodes are locked".
 
 - some more small incremental work towards online check_allocations
 
 - lots more debugging improvements, fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEKnAFLkS8Qha+jvQrE6szbY3KbnYFAmaZmVYACgkQE6szbY3K
 bnayLBAAr/RB75neEVzNqzE/qLoz9HBfPs7NrGNZg3bLzie9BvcEKGf3VUs2mm83
 6qN/bCJRBhd2vdMcFvIrYYqvD+F2qFFFrBjTzY20toReCwO6q5o8Ihfzv+uSu1qn
 Lg/6AbwwDwHPgSLcFM6yVfNsNBpOZ4tru7xble5/89RGp5uhbU38tsFwkUcngN/t
 4GWjtUYC+rFZaJSbt+wtGtM++nSURhMu1rxbR3MnVLT3JNW0wiG+FnRymCRQOYwx
 eslI6wP3JArTKMvACJqXTDivmjUPNdvsJ26LW6b5KBLi411OgV219ek0mJlkc5gl
 lOOZ5LPmPqn0BxsIdqOd+/OGOpvYkfWEyDj6/46K8KKFt+i++vCTzqIeL06HGQFS
 oCZajoHseLTlWPZTnoi3/KPWkKSJyaROrvFPSHT5/9sJfeeFt88hNSMP9g4+S4GI
 QSXK70GzEVjxr7YzUZwZUmRGWHt7YcS/qkTKJZRkLwUbr2BnrKEJhOa0t/i3RopN
 glFP/hP3dObTcWUF4xH90htfD2E/wHbP7nPwNCL6367n18Uj4TJD09vtM8xHsXSR
 YXECCjfskVDHAQJw/aV5jF1NpaeSW6g/bILi8tQhvRpfzLLvsaVGxA1xp5v+VZEQ
 w3WBepPofEE1EaVfKNeHCiE7STAnSVbWYWTatmjPdqVCyT7C9S0=
 =tnhh
 -----END PGP SIGNATURE-----

Merge tag 'bcachefs-2024-07-18.2' of https://evilpiepirate.org/git/bcachefs

Pull bcachefs updates from Kent Overstreet:

 - Metadata version 1.8: Stripe sectors accounting, BCH_DATA_unstriped

   This splits out the accounting of dirty sectors and stripe sectors in
   alloc keys; this lets us see stripe buckets that still have unstriped
   data in them.

   This is needed for ensuring that erasure coding is working correctly,
   as well as completing stripe creation after a crash.

 - Metadata version 1.9: Disk accounting rewrite

   The previous disk accounting scheme relied heavily on percpu counters
   that were also sharded by outstanding journal buffer; it was fast but
   not extensible or scalable, and meant that all accounting counters
   were recorded in every journal entry.

   The new disk accounting scheme stores accounting as normal btree
   keys; updates are deltas until they are flushed by the btree write
   buffer.

   This means we have no practical limit on the number of counters, and
   a new tagged union format that's easy to extend.

   We now have counters for compression type/ratio, per-snapshot-id
   usage, per-btree-id usage, and pending rebalance work.

 - Self healing on read IO/checksum error

   Data is now automatically rewritten if we get a read error and then a
   successful retry

 - Mount API conversion (thanks to Thomas Bertschinger)

 - Better lockdep coverage

   Previously, btree node locks were tracked individually by lockdep,
   like any other lock. But we may take _many_ btree node locks
   simultaneously, we easily blow through the limit of 48 locks that
   lockdep can track, leading to lockdep turning itself off.

   Tracking each btree node lock individually isn't really necessary
   since we have our own cycle detector for deadlock avoidance and
   centralized tracking of btree node locks, so we now have a single
   lockdep_map in btree_trans for "any btree nodes are locked".

 - Some more small incremental work towards online check_allocations

 - Lots more debugging improvements

 - Fixes, including:
    - undefined behaviour fixes, originally noted as breaking userspace
      LTO builds
    - fix a spurious warning in fsck_err, reported by Marcin
    - fix an integer overflow on trans->nr_updates, also reported by
      Marcin; this broke during deletion of highly fragmented indirect
      extents

* tag 'bcachefs-2024-07-18.2' of https://evilpiepirate.org/git/bcachefs: (120 commits)
  lockdep: Add comments for lockdep_set_no{validate,track}_class()
  bcachefs: Fix integer overflow on trans->nr_updates
  bcachefs: silence silly kdoc warning
  bcachefs: Fix fsck warning about btree_trans not passed to fsck error
  bcachefs: Add an error message for insufficient rw journal devs
  bcachefs: varint: Avoid left-shift of a negative value
  bcachefs: darray: Don't pass NULL to memcpy()
  bcachefs: Kill bch2_assert_btree_nodes_not_locked()
  bcachefs: Rename BCH_WRITE_DONE -> BCH_WRITE_SUBMITTED
  bcachefs: __bch2_read(): call trans_begin() on every loop iter
  bcachefs: show none if label is not set
  bcachefs: drop packed, aligned from bkey_inode_buf
  bcachefs: btree node scan: fall back to comparing by journal seq
  bcachefs: Add lockdep support for btree node locks
  lockdep: lockdep_set_notrack_class()
  bcachefs: Improve copygc_wait_to_text()
  bcachefs: Convert clock code to u64s
  bcachefs: Improve startup message
  bcachefs: Self healing on read IO error
  bcachefs: Make read_only a mount option again, but hidden
  ...
2024-07-18 17:27:43 -07:00
..
9p Two fixes headed to stable trees: 2024-05-29 09:25:15 -07:00
adfs fs/adfs: add MODULE_DESCRIPTION 2024-07-18 09:50:08 +02:00
affs affs: struct slink_front: Replace 1-element array with flexible array 2024-07-11 16:14:26 +02:00
afs afs: Convert comma to semicolon 2024-07-02 21:23:00 +02:00
autofs vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
bcachefs bcachefs changes for 6.11-rc1 (version 2) 2024-07-18 17:27:43 -07:00
befs befs: Convert befs_symlink_read_folio() to use folio_end_read() 2024-05-31 12:31:39 +02:00
bfs mm, slab: remove last vestiges of SLAB_MEM_SPREAD 2024-03-12 20:32:19 -07:00
btrfs for-6.11-tag 2024-07-17 12:38:04 -07:00
cachefiles Merge patch series "cachefiles: random bugfixes" 2024-07-05 18:40:40 +02:00
ceph We have a series from Xiubo that adds support for additional access 2024-05-25 14:23:58 -07:00
coda coda: Convert coda_symlink_filler() to use folio_end_read() 2024-05-31 12:31:39 +02:00
configfs fs/configfs: Add a callback to determine attribute visibility 2024-06-17 20:42:57 +02:00
cramfs vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
crypto The usual shower of singleton fixes and minor series all over MM, 2024-05-19 09:21:03 -07:00
debugfs vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
devpts
dlm dlm: add rcu_barrier before destroy kmem cache 2024-06-13 12:48:46 -05:00
ecryptfs hardening updates for 6.10-rc1 2024-05-13 14:14:05 -07:00
efivarfs efivarfs: Convert to new uid/gid option parsing helpers 2024-07-02 06:21:18 +02:00
efs vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
erofs erofs: silence uninitialized variable warning in z_erofs_scan_folio() 2024-07-13 12:47:34 +08:00
exfat Description for this pull request: 2024-07-17 12:53:47 -07:00
exportfs fhandle: relax open_by_handle_at() permission checks 2024-05-28 15:57:23 +02:00
ext2 ext2: Verify bitmap and itable block numbers before using them 2024-06-26 12:54:11 +02:00
ext4 Many cleanups and bug fixes in ext4, especially for the fast commit 2024-07-18 17:03:42 -07:00
f2fs vfs-6.11.casefold 2024-07-15 11:18:49 -07:00
fat vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
freevxfs freevxfs: Convert freevxfs to the new mount API. 2024-03-26 09:04:53 +01:00
fuse vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
gfs2 gfs2: Clean up glock demote logic 2024-07-09 10:40:03 +02:00
hfs vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
hfsplus vfs-6.11.misc 2024-07-15 10:52:51 -07:00
hostfs vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
hpfs vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
hugetlbfs hugetlbfs: Convert to new uid/gid option parsing helpers 2024-07-02 06:21:19 +02:00
iomap vfs-6.11.iomap 2024-07-15 13:28:14 -07:00
isofs \n 2024-07-17 13:11:42 -07:00
jbd2 jbd2: increase maximum transaction size 2024-07-08 23:59:37 -04:00
jffs2 jffs2: Remove calls to set/clear the folio error flag 2024-05-31 12:31:41 +02:00
jfs jfs: xattr: fix buffer overflow for invalid xattr 2024-06-04 18:09:03 +02:00
kernfs kernfs: mount: Remove unnecessary ‘NULL’ values from knparent 2024-05-04 19:02:39 +02:00
lockd lockd: Use *-y instead of *-objs in Makefile 2024-07-08 14:10:03 -04:00
minix vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
netfs vfs-6.10-rc8.fixes 2024-07-11 09:03:28 -07:00
nfs NFS Client Updates for Linux 6.11 2024-07-18 17:17:30 -07:00
nfs_common fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfsd NFSD 6.11 Release Notes 2024-07-17 12:00:49 -07:00
nilfs2 nilfs2: fix kernel bug on rename operation of broken directory 2024-07-03 22:40:38 -07:00
nls fs: nls: add missing MODULE_DESCRIPTION() macros 2024-06-03 16:37:07 +02:00
notify fsnotify: clear PARENT_WATCHED flags lazily 2024-06-05 09:52:38 +02:00
ntfs3 ntfs3: Convert to new uid/gid option parsing helpers 2024-07-02 06:21:19 +02:00
ocfs2 ocfs2: fix DIO failure due to insufficient transaction credits 2024-06-24 20:52:10 -07:00
omfs
openpromfs openpromfs: add missing MODULE_DESCRIPTION() macro 2024-06-20 09:46:01 +02:00
orangefs orangefs: Remove calls to set/clear the error flag 2024-05-31 12:31:41 +02:00
overlayfs ovl: fix encoding fid for lower only root 2024-06-14 10:30:40 +02:00
proc sysctl changes for 6.11-rc1 2024-07-16 14:24:29 -07:00
pstore memblock: updates for 6.11-rc1 2024-07-18 14:48:11 -07:00
qnx4 qnx4: add MODULE_DESCRIPTION() 2024-05-28 11:52:53 +02:00
qnx6 qnx6: add MODULE_DESCRIPTION() 2024-05-28 11:52:49 +02:00
quota vfs: replace WARN(down_read_trylock, ...) abuse with proper asserts 2024-06-03 15:45:47 +02:00
ramfs mm: switch mm->get_unmapped_area() to a flag 2024-04-25 20:56:25 -07:00
reiserfs reiserfs: Remove call to folio_set_error() 2024-05-31 12:31:41 +02:00
romfs romfs: Convert romfs_read_folio() to use a folio 2024-05-31 12:31:42 +02:00
smb vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
squashfs Mainly singleton patches, documented in their respective changelogs. 2024-05-19 14:02:03 -07:00
sysfs Merge 6.9-rc5 into driver-core-next 2024-04-23 13:27:43 +02:00
sysv fs: sysv: add MODULE_DESCRIPTION() 2024-05-28 11:52:45 +02:00
tracefs tracefs: Convert to new uid/gid option parsing helpers 2024-07-02 06:21:20 +02:00
ubifs This pull request contains updates for UBI and UBIFS: 2024-03-21 15:09:29 -07:00
udf udf: prevent integer overflow in udf_bitmap_free_blocks() 2024-06-26 12:54:11 +02:00
ufs ufs: Remove call to set the folio error flag 2024-05-31 12:31:42 +02:00
unicode kbuild: use $(src) instead of $(srctree)/$(src) for source directory 2024-05-10 04:34:52 +09:00
vboxsf vfs-6.11.mount.api 2024-07-15 11:31:32 -07:00
verity bpf: treewide: Align kfunc signatures to prog point-of-view 2024-06-12 11:01:31 -07:00
xfs New code for 6.11: 2024-07-17 12:57:48 -07:00
zonefs zonefs: enable support for large folios 2024-06-11 11:22:57 +09:00
aio.c fs: Initial atomic write support 2024-06-20 15:19:17 -06:00
anon_inodes.c fs: Create anon_inode_getfile_fmode() 2024-04-26 10:33:05 +02:00
attr.c fs: Export in_group_or_capable() 2024-06-25 11:15:48 +02:00
backing-file.c ovl: implement tmpfile 2024-05-02 20:35:57 +02:00
bad_inode.c
binfmt_elf_fdpic.c fs: don't block i_writecount during exec 2024-06-03 15:52:10 +02:00
binfmt_elf_test.c
binfmt_elf.c execve updates for v6.11-rc1 2024-07-16 12:59:20 -07:00
binfmt_flat.c
binfmt_misc.c vfs-6.11.module.description 2024-07-15 11:14:59 -07:00
binfmt_script.c fs: binfmt: add missing MODULE_DESCRIPTION() macros 2024-05-28 12:06:51 +02:00
buffer.c Many cleanups and bug fixes in ext4, especially for the fast commit 2024-07-18 17:03:42 -07:00
char_dev.c
compat_binfmt_elf.c
coredump.c virtio: features, fixes, cleanups 2024-05-23 12:04:36 -07:00
d_path.c
dax.c dax: use huge_zero_folio 2024-04-25 20:56:20 -07:00
dcache.c vfs-6.11.misc 2024-07-15 10:52:51 -07:00
direct-io.c fs/direct-io: remove redundant assignment to variable retval 2024-04-11 10:21:24 +02:00
drop_caches.c
eventfd.c eventfd: strictly check the count parameter of eventfd_write to avoid inputting illegal strings 2024-02-08 10:12:26 +01:00
eventpoll.c epoll: be better about file lifetimes 2024-05-05 14:00:48 -07:00
exec_test.c exec: Avoid pathological argc, envc, and bprm->p values 2024-07-13 21:31:58 -07:00
exec.c execve updates for v6.11-rc1 2024-07-16 12:59:20 -07:00
fcntl.c fcntl: add F_DUPFD_QUERY fcntl() 2024-05-10 08:26:31 +02:00
fhandle.c fhandle: relax open_by_handle_at() permission checks 2024-05-28 15:57:23 +02:00
file_table.c lsm/stable-6.9 PR 20240312 2024-03-12 20:03:34 -07:00
file.c fs/file: fix the check in find_next_fd() 2024-05-30 09:11:47 +02:00
filesystems.c
fs_context.c
fs_parser.c fs_parse: add uid & gid option option parsing helpers 2024-07-02 06:20:49 +02:00
fs_pin.c
fs_struct.c
fs_types.c
fs-writeback.c fs/writeback: remove unnecessary return in writeback_inodes_sb 2024-04-05 15:53:45 +02:00
fsopen.c vfs: retire user_path_at_empty and drop empty arg from getname_flags 2024-06-05 17:03:57 +02:00
init.c
inode.c vfs-6.11.inode 2024-07-15 11:39:44 -07:00
internal.h vfs-6.11.pidfs 2024-07-15 12:34:01 -07:00
ioctl.c fs/ioctl: Add a comment to keep the logic in sync with LSM policies 2024-05-13 06:58:35 +02:00
Kconfig - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
Kconfig.binfmt exec: Add KUnit test for bprm_stack_limits() 2024-06-19 13:13:55 -07:00
kernel_read_file.c
libfs.c libfs: Introduce case-insensitive string comparison helper 2024-06-07 17:00:44 +02:00
locks.c vfs-6.10-rc8.fixes 2024-07-11 09:03:28 -07:00
Makefile vfs-6.9.pidfd 2024-03-11 10:21:06 -07:00
mbcache.c vfs: remove SLAB_MEM_SPREAD flag usage 2024-02-27 11:21:31 +01:00
mnt_idmapping.c fs/mnt_idmapping.c: Return -EINVAL when no map is written 2024-02-08 10:12:37 +01:00
mount.h vfs-6.11.mount 2024-07-15 11:54:04 -07:00
mpage.c buffer: Remove calls to set and clear the folio error flag 2024-05-31 12:31:43 +02:00
namei.c vfs-6.11.misc 2024-07-15 10:52:51 -07:00
namespace.c vfs-6.11.mount 2024-07-15 11:54:04 -07:00
nsfs.c nsfs: use cleanup guard 2024-07-18 09:50:08 +02:00
open.c vfs-6.11.misc 2024-07-15 10:52:51 -07:00
pidfs.c pidfs: allow retrieval of namespace file descriptors 2024-06-28 10:37:29 +02:00
pipe.c
pnode.c
pnode.h
posix_acl.c lsm/stable-6.9 PR 20240312 2024-03-12 20:03:34 -07:00
proc_namespace.c fs: rename show_mnt_opts -> show_vfsmnt_opts 2024-06-28 14:36:43 +02:00
read_write.c fs: Initial atomic write support 2024-06-20 15:19:17 -06:00
readdir.c readdir: Add missing quote in macro comment 2024-06-03 15:49:26 +02:00
remap_range.c vfs: export remap and write check helpers 2024-04-15 14:54:13 -07:00
select.c fs/select: rework stack allocation hack for clang 2024-02-20 09:23:52 +01:00
seq_file.c seq_file: Simplify __seq_puts() 2024-05-02 16:28:20 +02:00
signalfd.c signalfd: drop an obsolete comment 2024-05-24 13:34:07 +02:00
splice.c remove call_{read,write}_iter() functions 2024-04-15 16:03:25 -04:00
stack.c
stat.c for-6.11/block-20240710 2024-07-15 14:20:22 -07:00
statfs.c
super.c fs: don't misleadingly warn during thaw operations 2024-06-18 16:20:47 +02:00
sync.c
sysctls.c
timerfd.c timerfd: convert to ->read_iter() 2024-04-10 16:23:02 -06:00
userfaultfd.c Fix userfaultfd_api to return EINVAL as expected 2024-07-03 22:40:36 -07:00
utimes.c
xattr.c evm: Move to LSM infrastructure 2024-02-15 23:43:47 -05:00