linux/fs/btrfs
Filipe Manana e4545de5b0 Btrfs: fix fsync data loss after append write
If we do an append write to a file (which increases its inode's i_size)
that does not have the flag BTRFS_INODE_NEEDS_FULL_SYNC set in its inode,
and the previous transaction added a new hard link to the file, which sets
the flag BTRFS_INODE_COPY_EVERYTHING in the file's inode, and then fsync
the file, the inode's new i_size isn't logged. This has the consequence
that after the fsync log is replayed, the file size remains what it was
before the append write operation, which means users/applications will
not be able to read the data that was successsfully fsync'ed before.

This happens because neither the inode item nor the delayed inode get
their i_size updated when the append write is made - doing so would
require starting a transaction in the buffered write path, something that
we do not do intentionally for performance reasons.

Fix this by making sure that when the flag BTRFS_INODE_COPY_EVERYTHING is
set the inode is logged with its current i_size (log the in-memory inode
into the log tree).

This issue is not a recent regression and is easy to reproduce with the
following test case for fstests:

  seq=`basename $0`
  seqres=$RESULT_DIR/$seq
  echo "QA output created by $seq"

  here=`pwd`
  tmp=/tmp/$$
  status=1	# failure is the default!

  _cleanup()
  {
          _cleanup_flakey
          rm -f $tmp.*
  }
  trap "_cleanup; exit \$status" 0 1 2 3 15

  # get standard environment, filters and checks
  . ./common/rc
  . ./common/filter
  . ./common/dmflakey

  # real QA test starts here
  _supported_fs generic
  _supported_os Linux
  _need_to_be_root
  _require_scratch
  _require_dm_flakey
  _require_metadata_journaling $SCRATCH_DEV

  _crash_and_mount()
  {
          # Simulate a crash/power loss.
          _load_flakey_table $FLAKEY_DROP_WRITES
          _unmount_flakey
          # Allow writes again and mount. This makes the fs replay its fsync log.
          _load_flakey_table $FLAKEY_ALLOW_WRITES
          _mount_flakey
  }

  rm -f $seqres.full

  _scratch_mkfs >> $seqres.full 2>&1
  _init_flakey
  _mount_flakey

  # Create the test file with some initial data and then fsync it.
  # The fsync here is only needed to trigger the issue in btrfs, as it causes the
  # the flag BTRFS_INODE_NEEDS_FULL_SYNC to be removed from the btrfs inode.
  $XFS_IO_PROG -f -c "pwrite -S 0xaa 0 32k" \
                  -c "fsync" \
                  $SCRATCH_MNT/foo | _filter_xfs_io
  sync

  # Add a hard link to our file.
  # On btrfs this sets the flag BTRFS_INODE_COPY_EVERYTHING on the btrfs inode,
  # which is a necessary condition to trigger the issue.
  ln $SCRATCH_MNT/foo $SCRATCH_MNT/bar

  # Sync the filesystem to force a commit of the current btrfs transaction, this
  # is a necessary condition to trigger the bug on btrfs.
  sync

  # Now append more data to our file, increasing its size, and fsync the file.
  # In btrfs because the inode flag BTRFS_INODE_COPY_EVERYTHING was set and the
  # write path did not update the inode item in the btree nor the delayed inode
  # item (in memory struture) in the current transaction (created by the fsync
  # handler), the fsync did not record the inode's new i_size in the fsync
  # log/journal. This made the data unavailable after the fsync log/journal is
  # replayed.
  $XFS_IO_PROG -c "pwrite -S 0xbb 32K 32K" \
               -c "fsync" \
               $SCRATCH_MNT/foo | _filter_xfs_io

  echo "File content after fsync and before crash:"
  od -t x1 $SCRATCH_MNT/foo

  _crash_and_mount

  echo "File content after crash and log replay:"
  od -t x1 $SCRATCH_MNT/foo

  status=0
  exit

The expected file output before and after the crash/power failure expects the
appended data to be available, which is:

  0000000 aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa
  *
  0100000 bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb
  *
  0200000

Cc: stable@vger.kernel.org
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <clm@fb.com>
2015-06-30 14:36:47 -07:00
..
tests btrfs: qgroup: Switch self test to extent-oriented qgroup mechanism. 2015-06-10 09:26:05 -07:00
acl.c btrfs: remove useless ACL check 2014-06-09 17:20:42 -07:00
async-thread.c btrfs: Fix lockdep warning of wr_ctx->wr_lock in scrub_free_wr_ctx() 2015-06-10 07:04:52 -07:00
async-thread.h btrfs: Fix lockdep warning of wr_ctx->wr_lock in scrub_free_wr_ctx() 2015-06-10 07:04:52 -07:00
backref.c btrfs: backref: Add special time_seq == (u64)-1 case for 2015-06-10 09:25:45 -07:00
backref.h btrfs: cleanup, remove inode_item_info helper 2015-01-14 19:23:47 +01:00
btrfs_inode.h Btrfs: fix metadata inconsistencies after directory fsync 2015-03-26 17:56:23 -07:00
check-integrity.c Merge branch 'cleanups-post-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux into for-linus-4.1 2015-03-25 10:52:48 -07:00
check-integrity.h block: submit_bio_wait() conversions 2013-11-24 16:33:41 -07:00
compression.c Merge branch 'cleanups-post-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux into for-linus-4.1 2015-03-25 10:52:48 -07:00
compression.h btrfs: constify structs with op functions or static definitions 2015-02-16 18:48:44 +01:00
ctree.c Btrfs: fix up read_tree_block to return proper error 2015-06-03 04:03:08 -07:00
ctree.h Btrfs: fix race between balance and unused block group deletion 2015-06-30 14:36:46 -07:00
delayed-inode.c Btrfs: fill ->last_trans for delayed inode in btrfs_fill_inode. 2015-04-26 06:27:03 -07:00
delayed-inode.h Btrfs: introduce the delayed inode ref deletion for the single link inode 2014-01-28 13:20:09 -08:00
delayed-ref.c btrfs: delayed-ref: double free in btrfs_add_delayed_tree_ref() 2015-06-24 12:28:03 -07:00
delayed-ref.h btrfs: qgroup: Add the ability to skip given qgroup for old/new_roots. 2015-06-10 09:26:23 -07:00
dev-replace.c Btrfs: sysfs: add support to show replacing target in the sysfs 2015-06-19 14:03:54 +02:00
dev-replace.h
dir-item.c Btrfs: make xattr replace operations atomic 2014-11-20 17:20:07 -08:00
disk-io.c Btrfs: fix crash on close_ctree() if cleaner starts new transaction 2015-06-30 14:36:46 -07:00
disk-io.h Btrfs: disk-io: replace root args iff only fs_info used 2015-02-16 18:48:43 +01:00
export.c VFS: normal filesystems (and lustre): d_inode() annotations 2015-04-15 15:06:57 -04:00
export.h
extent_io.c Btrfs: set UNWRITTEN for prealloc'ed extents in fiemap 2015-06-03 04:03:03 -07:00
extent_io.h btrfs: constify structs with op functions or static definitions 2015-02-16 18:48:44 +01:00
extent_map.c Btrfs: do not move em to modified list when unpinning 2014-11-21 11:59:54 -08:00
extent_map.h Btrfs: fix NULL pointer crash when running balance and scrub concurrently 2014-06-19 14:20:55 -07:00
extent-tree.c Btrfs: fix race between balance and unused block group deletion 2015-06-30 14:36:46 -07:00
extent-tree.h btrfs: qgroup: Add new qgroup calculation function 2015-06-10 09:25:49 -07:00
file-item.c Merge branch 'cleanups-post-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux into for-linus-4.1 2015-03-25 10:52:48 -07:00
file.c Btrfs: avoid syncing log in the fast fsync path when not necessary 2015-06-10 07:02:43 -07:00
free-space-cache.c Btrfs: fix mutex unlock without prior lock on space cache truncation 2015-06-02 19:34:34 -07:00
free-space-cache.h Btrfs: allow block group cache writeout outside critical section in commit 2015-04-10 14:07:22 -07:00
hash.c btrfs: LLVMLinux: Remove VLAIS 2014-10-14 10:51:22 +02:00
hash.h Btrfs: fix btrfs boot when compiled as built-in 2014-01-28 13:20:31 -08:00
inode-item.c Btrfs: fix fsync log replay for inodes with a mix of regular refs and extrefs 2015-01-21 18:02:05 -08:00
inode-map.c Btrfs: fix race between caching kthread and returning inode to inode cache 2015-06-30 14:36:46 -07:00
inode-map.h
inode.c Btrfs: fix hang during inode eviction due to concurrent readahead 2015-06-03 04:03:09 -07:00
ioctl.c btrfs: Handle unaligned length in extent_same 2015-06-10 07:02:50 -07:00
Kconfig rcu: Make SRCU optional by using CONFIG_SRCU 2015-01-06 11:04:29 -08:00
locking.c btrfs: fix lockups from btrfs_clear_path_blocking 2014-11-19 10:34:35 -08:00
locking.h btrfs: fix lockups from btrfs_clear_path_blocking 2014-11-19 10:34:35 -08:00
lzo.c btrfs: constify structs with op functions or static definitions 2015-02-16 18:48:44 +01:00
Makefile Btrfs: add sanity tests for new qgroup accounting code 2014-06-09 17:20:49 -07:00
math.h btrfs: cleanup 64bit/32bit divs, compile time constants 2015-03-03 17:23:57 +01:00
ordered-data.c Btrfs: don't attach unnecessary extents to transaction on fsync 2015-06-10 07:02:44 -07:00
ordered-data.h Btrfs: avoid syncing log in the fast fsync path when not necessary 2015-06-10 07:02:43 -07:00
orphan.c btrfs: kill the key type accessor helpers 2014-09-17 13:37:12 -07:00
print-tree.c btrfs: remove parameter blocksize from read_tree_block 2014-10-02 17:14:50 +02:00
print-tree.h
props.c btrfs: constify structs with op functions or static definitions 2015-02-16 18:48:44 +01:00
props.h Btrfs: add support for inode properties 2014-01-28 13:20:24 -08:00
qgroup.c btrfs: qgroup: allow user to clear the limitation on qgroup 2015-06-30 13:20:00 -07:00
qgroup.h btrfs: qgroup: Cleanup the old ref_node-oriented mechanism. 2015-06-10 09:26:11 -07:00
raid56.c Merge branch 'cleanups-post-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux into for-linus-4.1 2015-03-25 10:52:48 -07:00
raid56.h Btrfs: Make raid_map array be inlined in btrfs_bio structure 2015-01-21 18:06:47 -08:00
rcu-string.h
reada.c Btrfs: add ref_count and free function for btrfs_bio 2015-01-21 18:06:48 -08:00
relocation.c Btrfs: fix up read_tree_block to return proper error 2015-06-03 04:03:08 -07:00
root-tree.c Btrfs: use bitfield instead of integer data type for the some variants in btrfs_root 2014-06-09 17:20:40 -07:00
scrub.c btrfs: add error handling for scrub_workers_get() 2015-06-30 13:20:03 -07:00
send.c Btrfs: use received_uuid of parent during send 2015-06-12 13:20:38 -07:00
send.h
struct-funcs.c
super.c Btrfs: show subvol= and subvolid= in /proc/mounts 2015-06-03 04:03:02 -07:00
sysfs.c Btrfs: Check if kobject is initialized before put 2015-06-22 14:43:31 +02:00
sysfs.h Btrfs: sysfs: btrfs_sysfs_remove_fsid() make it non static 2015-05-27 12:27:22 +02:00
transaction.c btrfs: qgroup: Make snapshot accounting work with new extent-oriented 2015-06-10 09:26:29 -07:00
transaction.h btrfs: qgroup: Add the ability to skip given qgroup for old/new_roots. 2015-06-10 09:26:23 -07:00
tree-defrag.c btrfs: let tree defrag work in SSD mode 2015-06-02 19:34:33 -07:00
tree-log.c Btrfs: fix fsync data loss after append write 2015-06-30 14:36:47 -07:00
tree-log.h Btrfs: fix metadata inconsistencies after directory fsync 2015-03-26 17:56:23 -07:00
ulist.c btrfs: ulist: Add ulist_del() function. 2015-06-10 09:26:17 -07:00
ulist.h btrfs: ulist: Add ulist_del() function. 2015-06-10 09:26:17 -07:00
uuid-tree.c Btrfs: make btrfs_search_forward return with nodes unlocked 2014-09-17 13:38:02 -07:00
volumes.c Btrfs: fix race between balance and unused block group deletion 2015-06-30 14:36:46 -07:00
volumes.h Btrfs: sysfs: add pointer to access fs_info from fs_devices 2015-05-27 12:27:21 +02:00
xattr.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2015-04-26 17:22:07 -07:00
xattr.h btrfs: use generic posix ACL infrastructure 2014-01-25 23:58:18 -05:00
zlib.c btrfs: constify structs with op functions or static definitions 2015-02-16 18:48:44 +01:00