linux/include/asm-generic/bitops
Hector Martin 415d832497 locking/atomic: Make test_and_*_bit() ordered on failure
These operations are documented as always ordered in
include/asm-generic/bitops/instrumented-atomic.h, and producer-consumer
type use cases where one side needs to ensure a flag is left pending
after some shared data was updated rely on this ordering, even in the
failure case.

This is the case with the workqueue code, which currently suffers from a
reproducible ordering violation on Apple M1 platforms (which are
notoriously out-of-order) that ends up causing the TTY layer to fail to
deliver data to userspace properly under the right conditions.  This
change fixes that bug.

Change the documentation to restrict the "no order on failure" story to
the _lock() variant (for which it makes sense), and remove the
early-exit from the generic implementation, which is what causes the
missing barrier semantics in that case.  Without this, the remaining
atomic op is fully ordered (including on ARM64 LSE, as of recent
versions of the architecture spec).

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: stable@vger.kernel.org
Fixes: e986a0d6cb ("locking/atomics, asm-generic/bitops/atomic.h: Rewrite using atomic_*() APIs")
Fixes: 61e02392d3 ("locking/atomic/bitops: Document and clarify ordering semantics for failed test_and_{}_bit()")
Signed-off-by: Hector Martin <marcan@marcan.st>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2022-08-16 09:19:43 -07:00
..
__ffs.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
__fls.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
arch_hweight.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
atomic.h locking/atomic: Make test_and_*_bit() ordered on failure 2022-08-16 09:19:43 -07:00
builtin-__ffs.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
builtin-__fls.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
builtin-ffs.h asm-generic: ffs: Drop bogus reference to ffz location 2021-08-11 11:38:10 +02:00
builtin-fls.h fls: change parameter to unsigned int 2019-01-04 13:13:46 -08:00
const_hweight.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ext2-atomic-setbit.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ext2-atomic.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ffs.h asm-generic: ffs: Drop bogus reference to ffz location 2021-08-11 11:38:10 +02:00
ffz.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fls64.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fls.h fls: change parameter to unsigned int 2019-01-04 13:13:46 -08:00
generic-non-atomic.h bitops: define const_*() versions of the non-atomics 2022-06-30 19:52:41 -07:00
hweight.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
instrumented-atomic.h asm-generic/bitops: Always inline all bit manipulation helpers 2022-01-25 22:30:28 +01:00
instrumented-lock.h asm-generic/bitops, kcsan: Add instrumentation for barriers 2021-12-09 16:42:28 -08:00
instrumented-non-atomic.h bitops: wrap non-atomic bitops with a transparent macro 2022-06-30 19:52:41 -07:00
le.h bitops: move find_bit_*_le functions from le.h to find.h 2022-01-15 08:47:30 -08:00
lock.h locking/atomic: add generic arch_*() bitops 2021-07-16 18:46:45 +02:00
non-atomic.h bitops: unify non-atomic bitops prototypes across architectures 2022-06-30 19:52:41 -07:00
non-instrumented-non-atomic.h bitops: wrap non-atomic bitops with a transparent macro 2022-06-30 19:52:41 -07:00
sched.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00