linux_dsm_epyc7002/fs/xfs
Brian Foster 0ab32086d0 xfs: account only rmapbt-used blocks against rmapbt perag res
The rmapbt perag metadata reservation reserves blocks for the
reverse mapping btree (rmapbt). Since the rmapbt uses blocks from
the agfl and perag accounting is updated as blocks are allocated
from the allocation btrees, the reservation actually accounts blocks
as they are allocated to (or freed from) the agfl rather than the
rmapbt itself.

While this works for blocks that are eventually used for the rmapbt,
not all agfl blocks are destined for the rmapbt. Blocks that are
allocated to the agfl (and thus "reserved" for the rmapbt) but then
used by another structure leads to a growing inconsistency over time
between the runtime tracking of rmapbt usage vs. actual rmapbt
usage. Since the runtime tracking thinks all agfl blocks are rmapbt
blocks, it essentially believes that less future reservation is
required to satisfy the rmapbt than what is actually necessary.

The inconsistency is rectified across mount cycles because the perag
reservation is initialized based on the actual rmapbt usage at mount
time. The problem, however, is that the excessive drain of the
reservation at runtime opens a window to allocate blocks for other
purposes that might be required for the rmapbt on a subsequent
mount. This problem can be demonstrated by a simple test that runs
an allocation workload to consume agfl blocks over time and then
observe the difference in the agfl reservation requirement across an
unmount/mount cycle:

  mount ...: xfs_ag_resv_init: ... resv 3193 ask 3194 len 3194
  ...
  ...      : xfs_ag_resv_alloc_extent: ... resv 2957 ask 3194 len 1
  umount...: xfs_ag_resv_free: ... resv 2956 ask 3194 len 0
  mount ...: xfs_ag_resv_init: ... resv 3052 ask 3194 len 3194

As the above tracepoints show, the reservation requirement reduces
from 3194 blocks to 2956 blocks as the workload runs.  Without any
other changes in the filesystem, the same reservation requirement
jumps from 2956 to 3052 blocks over a umount/mount cycle.

To address this divergence, update the RMAPBT reservation to account
blocks used for the rmapbt only rather than all blocks filled into
the agfl. This patch makes several high-level changes toward that
end:

1.) Reintroduce an AGFL reservation type to serve as an accounting
    no-op for blocks allocated to (or freed from) the AGFL.
2.) Invoke RMAPBT usage accounting from the actual rmapbt block
    allocation path rather than the AGFL allocation path.

The first change is required because agfl blocks are considered free
blocks throughout their lifetime. The perag reservation subsystem is
invoked unconditionally by the allocation subsystem, so we need a
way to tell the perag subsystem (via the allocation subsystem) to
not make any accounting changes for blocks filled into the AGFL.

The second change causes the in-core RMAPBT reservation usage
accounting to remain consistent with the on-disk state at all times
and eliminates the risk of leaving the rmapbt reservation
underfilled.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
2018-03-11 20:27:57 -07:00
..
libxfs xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
scrub xfs: convert XFS_AGFL_SIZE to a helper function 2018-03-11 20:27:56 -07:00
Kconfig fs/*/Kconfig: drop links to 404-compliant http://acl.bestbits.at 2018-01-01 12:45:37 -07:00
kmem.c xfs: fall back to vmalloc when allocation log vector buffers 2018-03-11 20:27:55 -07:00
kmem.h xfs: fall back to vmalloc when allocation log vector buffers 2018-03-11 20:27:55 -07:00
Makefile xfs: use a b+tree for the in-core extent list 2017-11-06 11:53:41 -08:00
mrlock.h
xfs_acl.c xfs: don't change inode mode if ACL update fails 2017-10-11 10:21:06 -07:00
xfs_acl.h xfs: Don't clear SGID when inheriting ACLs 2017-06-27 18:23:21 -07:00
xfs_aops.c xfs: fix transaction allocation deadlock in IO path 2018-03-11 20:27:55 -07:00
xfs_aops.h xfs: perform dax_device lookup at mount 2017-08-31 09:31:47 -07:00
xfs_attr_inactive.c xfs: fail if xattr inactivation hits a hole 2017-10-26 15:38:22 -07:00
xfs_attr_list.c xfs: remove u_int* type usage 2017-11-09 15:50:29 -08:00
xfs_attr.h xfs: scrub extended attributes 2017-10-26 15:38:26 -07:00
xfs_bmap_item.c xfs: log recovery should replay deferred ops in order 2017-11-27 09:34:08 -08:00
xfs_bmap_item.h xfs: log recovery should replay deferred ops in order 2017-11-27 09:34:08 -08:00
xfs_bmap_util.c xfs: account format bouncing into rmapbt swapext tx reservation 2018-03-11 20:27:57 -07:00
xfs_bmap_util.h xfs: simplify the xfs_getbmap interface 2017-10-26 15:38:20 -07:00
xfs_buf_item.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_buf_item.h Use list_head infra-structure for buffer's log items list 2018-01-29 07:27:22 -08:00
xfs_buf.c xfs: Correctly invert xfs_buftarg LRU isolation logic 2018-03-11 20:27:56 -07:00
xfs_buf.h Use list_head infra-structure for buffer's log items list 2018-01-29 07:27:22 -08:00
xfs_dir2_readdir.c xfs: directory scrubber must walk through data block to offset 2018-01-17 21:00:46 -08:00
xfs_discard.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_discard.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_dquot_item.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_dquot_item.h
xfs_dquot.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_dquot.h
xfs_error.c xfs: refactor inode verifier corruption error printing 2018-01-29 07:27:22 -08:00
xfs_error.h xfs: refactor inode verifier corruption error printing 2018-01-29 07:27:22 -08:00
xfs_export.c xfs: abstract block export operations from nfsd layouts 2016-07-15 15:31:29 -04:00
xfs_export.h
xfs_extent_busy.c xfs: fix len comparison in xfs_extent_busy_trim 2017-02-16 17:20:12 -08:00
xfs_extent_busy.h xfs: improve handling of busy extents in the low-level allocator 2017-02-09 10:50:25 -08:00
xfs_extfree_item.c xfs: always honor OWN_UNKNOWN rmap removal requests 2017-12-21 08:48:38 -08:00
xfs_extfree_item.h xfs: refactor redo intent item processing 2016-08-03 11:23:49 +10:00
xfs_file.c dax: pass detailed error code from dax_iomap_fault() 2018-01-07 16:38:43 -05:00
xfs_filestream.c Merge branch 'xfs-4.9-log-recovery-fixes' into for-next 2016-10-03 09:56:28 +11:00
xfs_filestream.h
xfs_fsmap.c xfs: move two more RT specific functions into CONFIG_XFS_RT 2017-10-16 12:26:50 -07:00
xfs_fsmap.h xfs: implement the GETFSMAP ioctl 2017-04-03 15:18:17 -07:00
xfs_fsops.c xfs: convert XFS_AGFL_SIZE to a helper function 2018-03-11 20:27:56 -07:00
xfs_fsops.h xfs: hoist xfs_fs_geometry to libxfs 2018-01-08 10:54:48 -08:00
xfs_globals.c xfs: define fatal assert build time tunable 2017-06-19 08:59:10 -07:00
xfs_icache.c New in this version: 2018-01-31 10:18:00 -08:00
xfs_icache.h xfs: remove leftover CoW reservations when remounting ro 2017-12-21 08:47:32 -08:00
xfs_icreate_item.c fs: xfs: xfs_icreate_item: constify xfs_item_ops structure 2016-11-28 14:57:42 +11:00
xfs_icreate_item.h
xfs_inode_item.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_inode_item.h xfs: remove inode log format typedef 2017-11-01 15:03:16 -07:00
xfs_inode.c xfs: check for cow blocks before trying to clear them 2018-03-11 20:27:56 -07:00
xfs_inode.h xfs: allow xfs_lock_two_inodes to take different EXCL/SHARED modes 2018-01-29 07:27:23 -08:00
xfs_ioctl32.c xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_ioctl32.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_ioctl.c xfs: refactor the geometry structure filling function 2018-01-08 10:54:48 -08:00
xfs_ioctl.h xfs: remove u_int* type usage 2017-11-09 15:50:29 -08:00
xfs_iomap.c xfs: don't block on the ilock for RWF_NOWAIT 2018-03-01 14:12:45 -08:00
xfs_iomap.h xfs: update i_size after unwritten conversion in dio completion 2017-09-26 10:55:19 -07:00
xfs_iops.c xfs: implement the lazytime mount option 2018-03-11 20:27:55 -07:00
xfs_iops.h xfs: Propagate dentry down to inode_change_ok() 2016-09-22 10:56:19 +02:00
xfs_itable.c xfs: remove if_rdev 2017-10-26 15:38:27 -07:00
xfs_itable.h xfs: create inode pointer verifiers 2017-10-26 15:38:23 -07:00
xfs_linux.h xfs: use %px for data pointers when debugging 2018-01-12 14:09:08 -08:00
xfs_log_cil.c xfs: fall back to vmalloc when allocation log vector buffers 2018-03-11 20:27:55 -07:00
xfs_log_priv.h locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() 2017-10-25 11:01:08 +02:00
xfs_log_recover.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_log.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_log.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_message.c xfs: define bug_on_assert debug mode sysfs tunable 2017-06-19 08:59:10 -07:00
xfs_message.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_mount.c xfs: remove unused m_dmevmask from xfs_mount struct 2018-03-11 20:27:55 -07:00
xfs_mount.h xfs: account only rmapbt-used blocks against rmapbt perag res 2018-03-11 20:27:57 -07:00
xfs_mru_cache.c
xfs_mru_cache.h
xfs_ondisk.h xfs: Don't log uninitialised fields in inode structures 2017-10-11 10:21:06 -07:00
xfs_pnfs.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_pnfs.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_qm_bhv.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_qm_syscalls.c xfs: wait on new inodes during quotaoff dquot release 2017-04-28 08:11:08 -07:00
xfs_qm.c xfs: use %px for data pointers when debugging 2018-01-12 14:09:08 -08:00
xfs_qm.h
xfs_quota.h
xfs_quotaops.c VFS: Convert sb->s_flags & MS_RDONLY to sb_rdonly(sb) 2017-07-17 08:45:34 +01:00
xfs_refcount_item.c xfs: reserve blocks for refcount / rmap log item recovery 2018-02-22 14:41:25 -08:00
xfs_refcount_item.h xfs: log recovery should replay deferred ops in order 2017-11-27 09:34:08 -08:00
xfs_reflink.c xfs: rename agfl perag res type to rmapbt 2018-03-11 20:27:57 -07:00
xfs_reflink.h xfs: separate function to check if inode shares extents 2017-06-19 14:11:35 -07:00
xfs_rmap_item.c xfs: reserve blocks for refcount / rmap log item recovery 2018-02-22 14:41:25 -08:00
xfs_rmap_item.h xfs: convert RUI log formats to use variable length arrays 2016-09-19 10:24:27 +10:00
xfs_rtalloc.c xfs: remove the ip argument to xfs_defer_finish 2017-09-01 10:55:30 -07:00
xfs_rtalloc.h xfs: cross-reference the realtime bitmap 2018-01-17 21:00:46 -08:00
xfs_stats.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_stats.h xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_super.c xfs: check for cow blocks before trying to clear them 2018-03-11 20:27:56 -07:00
xfs_super.h xfs: add scrub to XFS_BUILD_OPTIONS 2018-02-01 21:06:15 -08:00
xfs_symlink.c xfs: remove "no-allocation" reservations for file creations 2017-12-08 17:51:05 -08:00
xfs_symlink.h xfs: allow reading of already-locked remote symbolic link 2017-06-20 10:45:22 -07:00
xfs_sysctl.c xfs: garbage collect old cowextsz reservations 2016-10-05 16:26:28 -07:00
xfs_sysctl.h xfs: define bug_on_assert debug mode sysfs tunable 2017-06-19 08:59:10 -07:00
xfs_sysfs.c xfs: replace log_badcrc_factor knob with error injection tag 2017-06-27 18:23:21 -07:00
xfs_sysfs.h xfs: configurable error behavior via sysfs 2016-05-18 10:58:51 +10:00
xfs_trace.c fs: xfs: remove duplicate includes 2017-12-08 17:51:05 -08:00
xfs_trace.h xfs: use %pS printk format for direct instruction addresses 2018-01-12 14:09:08 -08:00
xfs_trans_ail.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_trans_bmap.c xfs: try to avoid blowing out the transaction reservation when bunmaping a shared extent 2017-06-19 08:59:10 -07:00
xfs_trans_buf.c xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_trans_dquot.c
xfs_trans_extfree.c xfs: set up per-AG free space reservations 2016-09-19 10:30:52 +10:00
xfs_trans_inode.c xfs: implement the lazytime mount option 2018-03-11 20:27:55 -07:00
xfs_trans_priv.h xfs: Rename xa_ elements to ail_ 2018-03-11 20:27:56 -07:00
xfs_trans_refcount.c xfs: connect refcount adjust functions to upper layers 2016-10-03 09:11:22 -07:00
xfs_trans_rmap.c xfs: remove double-underscore integer types 2017-06-19 14:11:33 -07:00
xfs_trans.c xfs: shutdown if block allocation overruns tx reservation 2018-03-11 20:27:57 -07:00
xfs_trans.h Use list_head infra-structure for buffer's log items list 2018-01-29 07:27:22 -08:00
xfs_xattr.c xfs: several xattr functions can be void 2016-12-05 12:32:14 +11:00
xfs.h xfs: always define STATIC to static noinline 2017-11-06 11:53:58 -08:00