linux/fs/xfs/libxfs
Brian Foster f8f2835a9c xfs: defer agfl block frees when dfops is available
The AGFL fixup code executes before every block allocation/free and
rectifies the AGFL based on the current, dynamic allocation
requirements of the fs. The AGFL must hold a minimum number of
blocks to satisfy a worst case split of the free space btrees caused
by the impending allocation operation. The AGFL is also updated to
maintain the implicit requirement for a minimum number of free slots
to satisfy a worst case join of the free space btrees.

Since the AGFL caches individual blocks, AGFL reduction typically
involves multiple, single block frees. We've had reports of
transaction overrun problems during certain workloads that boil down
to AGFL reduction freeing multiple blocks and consuming more space
in the log than was reserved for the transaction.

Since the objective of freeing AGFL blocks is to ensure free AGFL
free slots are available for the upcoming allocation, one way to
address this problem is to release surplus blocks from the AGFL
immediately but defer the free of those blocks (similar to how
file-mapped blocks are unmapped from the file in one transaction and
freed via a deferred operation) until the transaction is rolled.
This turns AGFL reduction into an operation with predictable log
reservation consumption.

Add the capability to defer AGFL block frees when a deferred ops
list is available to the AGFL fixup code. Add a dfops pointer to the
transaction to carry dfops through various contexts to the allocator
context. Deferring AGFL frees is  conditional behavior based on
whether the transaction pointer is populated. The long term
objective is to reuse the transaction pointer to clean up all
unrelated callchains that pass dfops on the stack along with a
transaction and in doing so, consistently defer AGFL blocks from the
allocator.

A bit of customization is required to handle deferred completion
processing because AGFL blocks are accounted against a per-ag
reservation pool and AGFL blocks are not inserted into the extent
busy list when freed (they are inserted when used and released back
to the AGFL). Reuse the majority of the existing deferred extent
free infrastructure and customize it appropriately to handle AGFL
blocks.

Note that this patch only adds infrastructure. It does not change
behavior because no callers have been updated to pass ->t_agfl_dfops
into the allocation code.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
2018-05-09 10:04:02 -07:00
..
xfs_ag_resv.c xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_ag_resv.h xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_alloc_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_alloc_btree.h
xfs_alloc.c xfs: defer agfl block frees when dfops is available 2018-05-09 10:04:02 -07:00
xfs_alloc.h xfs: create agfl block free helper function 2018-05-09 10:04:01 -07:00
xfs_attr_leaf.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_attr_leaf.h xfs: create structure verifier function for shortform xattrs 2018-01-08 10:54:46 -08:00
xfs_attr_remote.c xfs: make xfs_buf_incore out of line 2018-05-09 10:04:00 -07:00
xfs_attr_remote.h
xfs_attr_sf.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_attr.c xfs: don't fail when converting shortform attr to long form during ATTR_REPLACE 2018-04-17 19:10:15 -07:00
xfs_bit.c libxfs: Optimize the loop for xfs_bitmap_empty 2016-01-04 16:10:19 +11:00
xfs_bit.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_bmap_btree.c Cleanup old XFS_BTREE_* traces 2018-03-11 20:27:55 -07:00
xfs_bmap_btree.h xfs: refactor bmap record validation 2018-03-23 18:05:07 -07:00
xfs_bmap.c xfs: set format back to extents if xfs_bmap_extents_to_btree 2018-04-17 17:10:17 -07:00
xfs_bmap.h xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_btree.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_btree.h xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_cksum.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_da_btree.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_da_btree.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_da_format.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_da_format.h xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_defer.c xfs: add the ability to join a held buffer to a defer_ops 2017-12-14 09:17:35 -08:00
xfs_defer.h xfs: defer agfl block frees when dfops is available 2018-05-09 10:04:02 -07:00
xfs_dir2_block.c xfs: sanity-check the unused space before trying to use it 2018-03-23 18:05:07 -07:00
xfs_dir2_data.c xfs: sanity-check the unused space before trying to use it 2018-03-23 18:05:07 -07:00
xfs_dir2_leaf.c xfs: sanity-check the unused space before trying to use it 2018-03-23 18:05:07 -07:00
xfs_dir2_node.c xfs: sanity-check the unused space before trying to use it 2018-03-23 18:05:07 -07:00
xfs_dir2_priv.h xfs: refactor short form directory structure verifier function 2018-01-08 10:54:46 -08:00
xfs_dir2_sf.c xfs: directory scrubber must walk through data block to offset 2018-01-17 21:00:46 -08:00
xfs_dir2.c xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_dir2.h xfs: sanity-check the unused space before trying to use it 2018-03-23 18:05:07 -07:00
xfs_dquot_buf.c xfs: print specific dqblk that failed verifiers 2018-05-09 10:04:01 -07:00
xfs_errortag.h xfs: move error injection tags into their own file 2017-11-01 15:03:16 -07:00
xfs_format.h xfs: convert XFS_AGFL_SIZE to a helper function 2018-03-11 20:27:56 -07:00
xfs_fs.h xfs: check sb_agblocks and sb_agblklog when validating superblock 2018-01-17 21:00:47 -08:00
xfs_ialloc_btree.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_ialloc_btree.h xfs: use per-AG reservations for the finobt 2017-01-25 07:49:35 -08:00
xfs_ialloc.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_ialloc.h xfs: add scrub cross-referencing helpers for the inode btrees 2018-01-17 21:00:44 -08:00
xfs_iext_tree.c xfs: move xfs_iext_insert tracepoint to report useful information 2017-12-14 09:20:11 -08:00
xfs_inode_buf.c xfs: enhance dinode verifier 2018-04-17 17:10:17 -07:00
xfs_inode_buf.h xfs: move inode extent size hint validation to libxfs 2018-03-23 18:05:08 -07:00
xfs_inode_fork.c xfs: refactor inode verifier error logging 2018-03-23 18:05:07 -07:00
xfs_inode_fork.h xfs: provide a centralized method for verifying inline fork data 2018-01-08 10:54:47 -08:00
xfs_log_format.h xfs: fix type usage 2017-11-16 12:06:45 -08:00
xfs_log_recover.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_log_rlimit.c xfs: trace log reservations at mount time 2018-01-08 10:54:47 -08:00
xfs_quota_defs.h xfs: add full xfs_dqblk verifier 2018-05-09 10:04:01 -07:00
xfs_refcount_btree.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_refcount_btree.h xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_refcount.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_refcount.h xfs: add scrub cross-referencing helpers for the refcount btrees 2018-01-17 21:00:44 -08:00
xfs_rmap_btree.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_rmap_btree.h xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_rmap.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_rmap.h xfs: add scrub cross-referencing helpers for the rmap btrees 2018-01-17 21:00:44 -08:00
xfs_rtbitmap.c xfs: cross-reference the realtime bitmap 2018-01-17 21:00:46 -08:00
xfs_sb.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_sb.h xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_shared.h xfs: trace log reservations at mount time 2018-01-08 10:54:47 -08:00
xfs_symlink_remote.c Split buffer's b_fspriv field 2018-01-29 07:27:22 -08:00
xfs_trans_resv.c xfs: non-scrub - remove unused function parameters 2018-04-09 10:23:42 -07:00
xfs_trans_resv.h xfs: increase log reservations for reflink 2016-10-05 16:26:29 -07:00
xfs_trans_space.h xfs: reserve enough blocks to handle btree splits when remapping 2017-05-03 13:21:40 -07:00
xfs_types.h xfs: move xfs_bmbt_irec and xfs_exntst_t to xfs_types.h 2017-11-06 11:53:41 -08:00