• Peter Zijlstra's avatar
    locking/mutex: Rework mutex::owner · 3ca0ff57
    Peter Zijlstra authored
    The current mutex implementation has an atomic lock word and a
    non-atomic owner field.
    This disparity leads to a number of issues with the current mutex code
    as it means that we can have a locked mutex without an explicit owner
    (because the owner field has not been set, or already cleared).
    This leads to a number of weird corner cases, esp. between the
    optimistic spinning and debug code. Where the optimistic spinning
    code needs the owner field updated inside the lock region, the debug
    code is more relaxed because the whole lock is serialized by the
    Also, the spinning code itself has a few corner cases where we need to
    deal with a held lock without an owner field.
    Furthermore, it becomes even more of a problem when trying to fix
    starvation cases in the current code. We end up stacking special case
    on special case.
    To solve this rework the basic mutex implementation to be a single
    atomic word that contains the owner and uses the low bits for extra
    This matches how PI futexes and rt_mutex already work. By having the
    owner an integral part of the lock state a lot of the problems
    dissapear and we get a better option to deal with starvation cases,
    direct owner handoff.
    Changing the basic mutex does however invalidate all the arch specific
    mutex code; this patch leaves that unused in-place, a later patch will
    remove that.
    Tested-by: default avatarJason Low <jason.low2@hpe.com>
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Reviewed-by: default avatarWill Deacon <will.deacon@arm.com>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: default avatarIngo Molnar <mingo@kernel.org>