leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
c495dcd6fb
linux/fs/btrfs
History
Omar Sandoval c495dcd6fb btrfs: don't prematurely free work in run_ordered_work() 
We hit the following very strange deadlock on a system with Btrfs on a
loop device backed by another Btrfs filesystem:

1. The top (loop device) filesystem queues an async_cow work item from
   cow_file_range_async(). We'll call this work X.
2. Worker thread A starts work X (normal_work_helper()).
3. Worker thread A executes the ordered work for the top filesystem
   (run_ordered_work()).
4. Worker thread A finishes the ordered work for work X and frees X
   (work->ordered_free()).
5. Worker thread A executes another ordered work and gets blocked on I/O
   to the bottom filesystem (still in run_ordered_work()).
6. Meanwhile, the bottom filesystem allocates and queues an async_cow
   work item which happens to be the recently-freed X.
7. The workqueue code sees that X is already being executed by worker
   thread A, so it schedules X to be executed _after_ worker thread A
   finishes (see the find_worker_executing_work() call in
   process_one_work()).

Now, the top filesystem is waiting for I/O on the bottom filesystem, but
the bottom filesystem is waiting for the top filesystem to finish, so we
deadlock.

This happens because we are breaking the workqueue assumption that a
work item cannot be recycled while it still depends on other work. Fix
it by waiting to free the work item until we are done with all of the
related ordered work.

P.S.:

One might ask why the workqueue code doesn't try to detect a recycled
work item. It actually does try by checking whether the work item has
the same work function (find_worker_executing_work()), but in our case
the function is the same. This is the only key that the workqueue code
has available to compare, short of adding an additional, layer-violating
"custom key". Considering that we're the only ones that have ever hit
this, we should just play by the rules.

Unfortunately, we haven't been able to create a minimal reproducer other
than our full container setup using a compress-force=zstd filesystem on
top of another compress-force=zstd filesystem.

Suggested-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 12:46:48 +01:00
..
tests Btrfs: fix selftests failure due to uninitialized i_mode in test inodes 2019-09-24 14:45:02 +02:00
acl.c btrfs: cleanup btrfs_setxattr_trans and drop transaction parameter 2019-04-29 19:02:44 +02:00
async-thread.c btrfs: don't prematurely free work in run_ordered_work() 2019-11-18 12:46:48 +01:00
async-thread.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
backref.c Btrfs: fix deadlock between fiemap and transaction commits 2019-07-30 18:25:12 +02:00
backref.h btrfs: fiemap: preallocate ulists for btrfs_check_shared 2019-07-01 13:34:53 +02:00
block-group.c btrfs: block-group: Fix a memory leak due to missing btrfs_put_block_group() 2019-10-11 21:27:51 +02:00
block-group.h btrfs: move struct io_ctl to free-space-cache.h 2019-09-09 14:59:15 +02:00
block-rsv.c btrfs: use btrfs_try_granting_tickets in update_global_rsv 2019-09-09 14:59:19 +02:00
block-rsv.h btrfs: migrate the global_block_rsv helpers to block-rsv.c 2019-07-02 12:30:55 +02:00
btrfs_inode.h btrfs: remove assumption about csum type form btrfs_print_data_csum_error() 2019-07-01 13:35:02 +02:00
check-integrity.c btrfs: reduce stack usage for btrfsic_process_written_block 2019-09-09 14:58:58 +02:00
check-integrity.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
compression.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00
compression.h btrfs: compression: replace set_level callbacks by a common helper 2019-09-09 14:59:11 +02:00
ctree.c btrfs: ctree: Remove stray comment of setting up path lock 2019-11-18 12:46:46 +01:00
ctree.h btrfs: move extent_io_tree defs to their own header 2019-11-18 12:46:47 +01:00
delalloc-space.c Btrfs: fix qgroup double free after failure to reserve metadata for delalloc 2019-10-17 20:13:44 +02:00
delalloc-space.h btrfs: migrate the delalloc space stuff to it's own home 2019-07-04 17:26:17 +02:00
delayed-inode.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00
delayed-inode.h Btrfs: delayed-inode: use rb_first_cached for ins_root and del_root 2018-10-15 17:23:33 +02:00
delayed-ref.c btrfs: rename btrfs_space_info_add_old_bytes 2019-09-09 14:59:18 +02:00
delayed-ref.h btrfs: migrate the delayed refs rsv code 2019-07-04 17:26:17 +02:00
dev-replace.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00
dev-replace.h btrfs: get fs_info from trans in btrfs_run_dev_replace 2019-04-29 19:02:43 +02:00
dir-item.c btrfs: remove unused parameter fs_info from btrfs_extend_item 2019-04-29 19:02:50 +02:00
disk-io.c btrfs: don't needlessly create extent-refs kernel thread 2019-10-15 15:43:29 +02:00
disk-io.h btrfs: Make reada_tree_block_flagged private 2019-09-09 14:59:11 +02:00
export.c btrfs: Remove 'objectid' member from struct btrfs_root 2018-10-15 17:23:25 +02:00
export.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
extent_io.c btrfs: move the failrec tree stuff into extent-io-tree.h 2019-11-18 12:46:47 +01:00
extent_io.h btrfs: move the failrec tree stuff into extent-io-tree.h 2019-11-18 12:46:47 +01:00
extent_map.c btrfs: assert extent map tree lock in add_extent_mapping 2019-09-09 14:59:00 +02:00
extent_map.h btrfs: Remove impossible condition from mergable_maps 2019-02-25 14:13:21 +01:00
extent-io-tree.h btrfs: move the failrec tree stuff into extent-io-tree.h 2019-11-18 12:46:47 +01:00
extent-tree.c btrfs: refactor the ticket wakeup code 2019-09-09 14:59:18 +02:00
file-item.c btrfs: directly call into crypto framework for checksumming 2019-07-01 13:35:02 +02:00
file.c btrfs: simplify inode locking for RWF_NOWAIT 2019-11-18 12:46:45 +01:00
free-space-cache.c btrfs: stop clearing EXTENT_DIRTY in inode I/O tree 2019-09-09 14:59:17 +02:00
free-space-cache.h btrfs: move struct io_ctl to free-space-cache.h 2019-09-09 14:59:15 +02:00
free-space-tree.c btrfs: move basic block_group definitions to their own header 2019-09-09 14:59:03 +02:00
free-space-tree.h btrfs: move basic block_group definitions to their own header 2019-09-09 14:59:03 +02:00
inode-item.c btrfs: Make btrfs_find_name_in_ext_backref return struct btrfs_inode_extref 2019-09-09 14:59:16 +02:00
inode-map.c btrfs: qgroup: Always free PREALLOC META reserve in btrfs_delalloc_release_extents() 2019-10-15 18:50:07 +02:00
inode-map.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
inode.c Btrfs: fix log context list corruption after rename exchange operation 2019-11-11 19:46:02 +01:00
ioctl.c btrfs: un-deprecate ioctls START_SYNC and WAIT_SYNC 2019-11-04 21:42:01 +01:00
Kconfig btrfs: Fix build error while LIBCRC32C is module 2019-07-17 17:03:30 +02:00
locking.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00
locking.h btrfs: Remove unused locking functions 2019-09-09 14:58:59 +02:00
lzo.c btrfs: compression: replace set_level callbacks by a common helper 2019-09-09 14:59:11 +02:00
Makefile btrfs: migrate the block group lookup code 2019-09-09 14:59:04 +02:00
misc.h btrfs: move math functions to misc.h 2019-09-09 14:59:15 +02:00
ordered-data.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00
ordered-data.h btrfs: don't assume ordered sums to be 4 bytes 2019-07-01 13:35:00 +02:00
orphan.c btrfs: replace GPL boilerplate by SPDX -- sources 2018-04-12 16:29:51 +02:00
print-tree.c btrfs: switch extent_buffer write_locks from atomic to int 2019-07-02 12:30:47 +02:00
print-tree.h btrfs: print-tree: debugging output enhancement 2018-04-20 19:18:16 +02:00
props.c btrfs: rename the btrfs_calc_*_metadata_size helpers 2019-09-09 14:59:13 +02:00
props.h btrfs: delete unused function btrfs_set_prop_trans 2019-04-29 19:02:54 +02:00
qgroup.c btrfs: get rid of unnecessary memset() of work item 2019-11-18 12:46:47 +01:00
qgroup.h btrfs: qgroup: Move reserved data accounting from btrfs_delayed_ref_head to btrfs_qgroup_extent_record 2019-02-25 14:13:39 +01:00
raid56.c btrfs: move private raid56 definitions from ctree.h 2019-09-09 14:59:15 +02:00
raid56.h btrfs: constify map parameter for nr_parity_stripes and nr_data_stripes 2019-07-01 13:34:58 +02:00
rcu-string.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
reada.c btrfs: Make reada_tree_block_flagged private 2019-09-09 14:59:11 +02:00
ref-verify.c btrfs: fix uninitialized ret in ref-verify 2019-10-03 15:00:56 +02:00
ref-verify.h btrfs: ref-verify: Use btrfs_ref to refactor btrfs_ref_tree_mod() 2019-04-29 19:02:49 +02:00
relocation.c btrfs: qgroup: Always free PREALLOC META reserve in btrfs_delalloc_release_extents() 2019-10-15 18:50:07 +02:00
root-tree.c btrfs: rename the btrfs_calc_*_metadata_size helpers 2019-09-09 14:59:13 +02:00
scrub.c btrfs: move basic block_group definitions to their own header 2019-09-09 14:59:03 +02:00
send.c btrfs: silence maybe-uninitialized warning in clone_range 2019-10-08 13:14:55 +02:00
send.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
space-info.c Btrfs: fix race leading to metadata space leak after task received signal 2019-10-25 19:11:34 +02:00
space-info.h btrfs: rename btrfs_space_info_add_old_bytes 2019-09-09 14:59:18 +02:00
struct-funcs.c btrfs: tie extent buffer and it's token together 2019-09-09 14:59:16 +02:00
super.c btrfs: separate out the extent io init function 2019-11-18 12:46:47 +01:00
sysfs.c btrfs: sysfs: move helper macros to sysfs.c 2019-09-09 14:59:08 +02:00
sysfs.h btrfs: sysfs: move helper macros to sysfs.c 2019-09-09 14:59:08 +02:00
transaction.c Btrfs: make btrfs_wait_extents() static 2019-11-18 12:46:45 +01:00
transaction.h Btrfs: make btrfs_wait_extents() static 2019-11-18 12:46:45 +01:00
tree-checker.c btrfs: tree-checker: Add check for INODE_REF 2019-11-18 12:46:46 +01:00
tree-checker.h btrfs: get fs_info from eb in btrfs_check_chunk_valid 2019-04-29 19:02:39 +02:00
tree-defrag.c btrfs: open code now trivial btrfs_set_lock_blocking 2019-02-25 14:13:27 +01:00
tree-log.c for-5.4-rc2-tag 2019-10-10 08:30:51 -07:00
tree-log.h btrfs: get fs_info from trans in btrfs_set_log_full_commit 2019-04-29 19:02:41 +02:00
ulist.c btrfs: replace GPL boilerplate by SPDX -- sources 2018-04-12 16:29:51 +02:00
ulist.h btrfs: replace GPL boilerplate by SPDX -- headers 2018-04-12 16:29:46 +02:00
uuid-tree.c btrfs: remove unused parameter fs_info from btrfs_extend_item 2019-04-29 19:02:50 +02:00
volumes.c btrfs: Consider system chunk array size for new SYSTEM chunks 2019-10-25 19:11:34 +02:00
volumes.h btrfs: reset device stat using btrfs_dev_stat_set 2019-09-09 14:59:06 +02:00
xattr.c Btrfs: fix failure to persist compression property xattr deletion on fsync 2019-06-17 16:37:17 +02:00
xattr.h btrfs: cleanup btrfs_setxattr_trans and drop transaction parameter 2019-04-29 19:02:44 +02:00
zlib.c btrfs: compression: replace set_level callbacks by a common helper 2019-09-09 14:59:11 +02:00
zstd.c btrfs: move cond_wake_up functions out of ctree 2019-09-09 14:59:15 +02:00