• Waiman Long's avatar
    locking/qspinlock: Use a simple write to grab the lock · 2c83e8e9
    Waiman Long authored
    Currently, atomic_cmpxchg() is used to get the lock. However, this
    is not really necessary if there is more than one task in the queue
    and the queue head don't need to reset the tail code. For that case,
    a simple write to set the lock bit is enough as the queue head will
    be the only one eligible to get the lock as long as it checks that
    both the lock and pending bits are not set. The current pending bit
    waiting code will ensure that the bit will not be set as soon as the
    tail code in the lock is set.
    With that change, the are some slight improvement in the performance
    of the queued spinlock in the 5M loop micro-benchmark run on a 4-socket
    Westere-EX machine as shown in the tables below.
    		[Standalone/Embedded - same node]
      # of tasks	Before patch	After patch	%Change
      ----------	-----------	----------	-------
           3	 2324/2321	2248/2265	 -3%/-2%
           4	 2890/2896	2819/2831	 -2%/-2%
           5	 3611/3595	3522/3512	 -2%/-2%
           6	 4281/4276	4173/4160	 -3%/-3%
           7	 5018/5001	4875/4861	 -3%/-3%
           8	 5759/5750	5563/5568	 -3%/-3%
    		[Standalone/Embedded - different nodes]
      # of tasks	Before patch	After patch	%Change
      ----------	-----------	----------	-------
           3	12242/12237	12087/12093	 -1%/-1%
           4	10688/10696	10507/10521	 -2%/-2%
    It was also found that this change produced a much bigger performance
    improvement in the newer IvyBridge-EX chip and was essentially to close
    the performance gap between the ticket spinlock and queued spinlock.
    The disk workload of the AIM7 benchmark was run on a 4-socket
    Westmere-EX machine with both ext4 and xfs RAM disks at 3000 users
    on a 3.14 based kernel. The results of the test runs were:
                    AIM7 XFS Disk Test
      kernel                 JPM    Real Time   Sys Time    Usr Time
      -----                  ---    ---------   --------    --------
      ticketlock            5678233    3.17       96.61       5.81
      qspinlock             5750799    3.13       94.83       5.97
                    AIM7 EXT4 Disk Test
      kernel                 JPM    Real Time   Sys Time    Usr Time
      -----                  ---    ---------   --------    --------
      ticketlock            1114551   16.15      509.72       7.11
      qspinlock             2184466    8.24      232.99       6.01
    The ext4 filesystem run had a much higher spinlock contention than
    the xfs filesystem run.
    The "ebizzy -m" test was also run with the following results:
      kernel               records/s  Real Time   Sys Time    Usr Time
      -----                ---------  ---------   --------    --------
      ticketlock             2075       10.00      216.35       3.49
      qspinlock              3023       10.00      198.20       4.80
    Signed-off-by: default avatarWaiman Long <Waiman.Long@hp.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Daniel J Blueman <daniel@numascale.com>
    Cc: David Vrabel <david.vrabel@citrix.com>
    Cc: Douglas Hatch <doug.hatch@hp.com>
    Cc: H. Peter Anvin <hpa@zytor.com>
    Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Oleg Nesterov <oleg@redhat.com>
    Cc: Paolo Bonzini <paolo.bonzini@gmail.com>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Scott J Norton <scott.norton@hp.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: virtualization@lists.linux-foundation.org
    Cc: xen-devel@lists.xenproject.org
    Link: http://lkml.kernel.org/r/1429901803-29771-7-git-send-email-Waiman.Long@hp.com
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
qspinlock.c 10.6 KB