• Tejun Heo's avatar
    blkcg: introduce blkg_stat and blkg_rwstat · edcb0722
    Tejun Heo authored
    blkcg uses u64_stats_sync to avoid reading wrong u64 statistic values
    on 32bit archs and some stat counters have subtypes to distinguish
    read/writes and sync/async IOs.  The stat code paths are confusing and
    involve a lot of going back and forth between blkcg core and specific
    policy implementations, and synchronization and subtype handling are
    open coded in blkcg core.
    This patch introduces struct blkg_stat and blkg_rwstat which, with
    accompanying operations, encapsulate stat updating and accessing with
    proper synchronization.
    blkg_stat is simple u64 counter with 64bit read-access protection.
    blkg_rwstat is the one with rw and [a]sync subcounters and takes @rw
    flags to distinguish IO subtypes (%REQ_WRITE and %REQ_SYNC) and
    replaces stat_sub_type indexed arrays.
    All counters in blkio_group_stats and blkio_group_stats_cpu are
    replaced with either blkg_stat or blkg_rwstat along with all users.
    This does add one u64_stats_sync per counter and increase stats_sync
    operations but they're empty/noops on 64bit archs and blkcg doesn't
    have too many counters, especially with DEBUG_BLK_CGROUP off.
    While the currently resulting code isn't necessarily simpler at the
    moment, this will enable further clean up of blkcg stats code.
    - blkg_stat_add() replaces blkio_add_stat() and
      blkio_check_and_dec_stat().  Note that BUG_ON() on underflow in the
      latter function no longer exists.  It's *way* better to have
      underflowed stat counters than oopsing.
    - blkio_group_stats->dequeue is now a proper u64 stat counter instead
      of ulong.
    - reset_stats() updated to clear each stat counters individually and
    - Some functions reconstruct rw flags from direction and sync
      booleans.  This will be removed by future patches.
    Signed-off-by: default avatarTejun Heo <tj@kernel.org>