mirror of
https://github.com/torvalds/linux.git
synced 2024-12-03 17:41:22 +00:00
415d832497
These operations are documented as always ordered in include/asm-generic/bitops/instrumented-atomic.h, and producer-consumer type use cases where one side needs to ensure a flag is left pending after some shared data was updated rely on this ordering, even in the failure case. This is the case with the workqueue code, which currently suffers from a reproducible ordering violation on Apple M1 platforms (which are notoriously out-of-order) that ends up causing the TTY layer to fail to deliver data to userspace properly under the right conditions. This change fixes that bug. Change the documentation to restrict the "no order on failure" story to the _lock() variant (for which it makes sense), and remove the early-exit from the generic implementation, which is what causes the missing barrier semantics in that case. Without this, the remaining atomic op is fully ordered (including on ARM64 LSE, as of recent versions of the architecture spec). Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Cc: stable@vger.kernel.org Fixes:e986a0d6cb("locking/atomics, asm-generic/bitops/atomic.h: Rewrite using atomic_*() APIs") Fixes:61e02392d3("locking/atomic/bitops: Document and clarify ordering semantics for failed test_and_{}_bit()") Signed-off-by: Hector Martin <marcan@marcan.st> Acked-by: Will Deacon <will@kernel.org> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
72 lines
1.5 KiB
Plaintext
72 lines
1.5 KiB
Plaintext
=============
|
|
Atomic bitops
|
|
=============
|
|
|
|
While our bitmap_{}() functions are non-atomic, we have a number of operations
|
|
operating on single bits in a bitmap that are atomic.
|
|
|
|
|
|
API
|
|
---
|
|
|
|
The single bit operations are:
|
|
|
|
Non-RMW ops:
|
|
|
|
test_bit()
|
|
|
|
RMW atomic operations without return value:
|
|
|
|
{set,clear,change}_bit()
|
|
clear_bit_unlock()
|
|
|
|
RMW atomic operations with return value:
|
|
|
|
test_and_{set,clear,change}_bit()
|
|
test_and_set_bit_lock()
|
|
|
|
Barriers:
|
|
|
|
smp_mb__{before,after}_atomic()
|
|
|
|
|
|
All RMW atomic operations have a '__' prefixed variant which is non-atomic.
|
|
|
|
|
|
SEMANTICS
|
|
---------
|
|
|
|
Non-atomic ops:
|
|
|
|
In particular __clear_bit_unlock() suffers the same issue as atomic_set(),
|
|
which is why the generic version maps to clear_bit_unlock(), see atomic_t.txt.
|
|
|
|
|
|
RMW ops:
|
|
|
|
The test_and_{}_bit() operations return the original value of the bit.
|
|
|
|
|
|
ORDERING
|
|
--------
|
|
|
|
Like with atomic_t, the rule of thumb is:
|
|
|
|
- non-RMW operations are unordered;
|
|
|
|
- RMW operations that have no return value are unordered;
|
|
|
|
- RMW operations that have a return value are fully ordered.
|
|
|
|
- RMW operations that are conditional are unordered on FAILURE,
|
|
otherwise the above rules apply. In the case of test_and_set_bit_lock(),
|
|
if the bit in memory is unchanged by the operation then it is deemed to have
|
|
failed.
|
|
|
|
Except for a successful test_and_set_bit_lock() which has ACQUIRE semantics and
|
|
clear_bit_unlock() which has RELEASE semantics.
|
|
|
|
Since a platform only has a single means of achieving atomic operations
|
|
the same barriers as for atomic_t are used, see atomic_t.txt.
|
|
|