• Waiman Long's avatar
    locking/qspinlock: Prefetch the next node cacheline · 81b55986
    Waiman Long authored
    A queue head CPU, after acquiring the lock, will have to notify
    the next CPU in the wait queue that it has became the new queue
    head. This involves loading a new cacheline from the MCS node of the
    next CPU. That operation can be expensive and add to the latency of
    locking operation.
    This patch addes code to optmistically prefetch the next MCS node
    cacheline if the next pointer is defined and it has been spinning
    for the MCS lock for a while. This reduces the locking latency and
    improves the system throughput.
    The performance change will depend on whether the prefetch overhead
    can be hidden within the latency of the lock spin loop. On really
    short critical section, there may not be performance gain at all. With
    longer critical section, however, it was found to have a performance
    boost of 5-10% over a range of different queue depths with a spinlock
    loop microbenchmark.
    Signed-off-by: default avatarWaiman Long <Waiman.Long@hpe.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Davidlohr Bueso <dave@stgolabs.net>
    Cc: Douglas Hatch <doug.hatch@hpe.com>
    Cc: H. Peter Anvin <hpa@zytor.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Scott J Norton <scott.norton@hpe.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Link: http://lkml.kernel.org/r/1447114167-47185-3-git-send-email-Waiman.Long@hpe.com
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>
qspinlock.c 13.1 KB