linux/include
akpm@osdl.org 838cd153a5 [PATCH] N32 sigset and __COMPAT_ENDIAN_SWAP__
I'm testing glibc on MIPS64, little-endian, N32, O32 and N64 multilibs.

Among the NPTL test failures seen are some arising from sigsuspend problems
for N32: it blocks the wrong signals, so SIGCANCEL (SIGRTMIN) is blocked
despite glibc's carefully excluding it from sets of signals to block.
Specifically, testing suggests it blocks signal N^32 instead of signal N,
so (in the example tested) blocking SIGUSR1 (17) blocks signal 49 instead.

glibc's sigset_t uses an array of unsigned long, as does the kernel.
In both cases, signal N+1 is represented as
(1UL << (N % (8 * sizeof (unsigned long)))) in word number
(N / (8 * sizeof (unsigned long))).

Thus the N32 glibc uses an array of 32-bit words and the N64 kernel uses an
array of 64-bit words.  For little-endian, the layout is the same, with
signals 1-32 in the first 4 bytes, signals 33-64 in the second, etc.; for
big-endian, userspace has that layout while in the kernel each 8 bytes have
the two halves swapped from the userspace layout.

The N32 sigsuspend syscall uses sigset_from_compat to convert the userspace
sigset to kernel format.  If __COMPAT_ENDIAN_SWAP__ is *not* set, this uses
logic of the form

  set->sig[0] = compat->sig[0] | (((long)compat->sig[1]) << 32 )

to convert the userspace sigset to a kernel one.  This looks correct to me
for both big and little endian, given that in userspace compat->sig[1] will
represent signals 33-64, and so will the high 32 bits of set->sig[0] in the
kernel.  If however __COMPAT_ENDIAN_SWAP__ *is* set, as it is for
__MIPSEL__, it uses

  set->sig[0] = compat->sig[1] | (((long)compat->sig[0]) << 32 );

which seems incorrect for both big and little endian, and would
explain the observed symptoms.

This code is the only use of __COMPAT_ENDIAN_SWAP__, so if incorrect
then that macro serves no purpose, in which case something like the
following patch would seem appropriate to remove it.

Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-06-25 10:01:15 -07:00
..
acpi Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6 2006-06-23 07:52:36 -07:00
asm-alpha [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-arm [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-arm26 [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-cris Remove unneeded _syscallX macros from user view in asm-*/unistd.h 2006-04-29 01:51:47 +01:00
asm-frv [PATCH] frv: clean frv unistd.h 2006-06-23 07:42:55 -07:00
asm-generic [PATCH] introduce WARN_ON_ONCE(cond) 2006-06-25 10:01:11 -07:00
asm-h8300 [PATCH] adjust handle_IRR_event() return type 2006-06-23 07:43:08 -07:00
asm-i386 [PATCH] Make copy_from_user_inatomic NOT zero the tail on i386 2006-06-25 10:01:09 -07:00
asm-ia64 [PATCH] Define __raw_get_cpu_var and use it 2006-06-25 10:01:01 -07:00
asm-m32r [PATCH] vgacon: make VGA_MAP_MEM take size, remove extra use 2006-06-22 15:05:58 -07:00
asm-m68k [PATCH] m68k: convert VME irq code 2006-06-25 10:00:58 -07:00
asm-m68knommu [PATCH] adjust handle_IRR_event() return type 2006-06-23 07:43:08 -07:00
asm-mips [PATCH] N32 sigset and __COMPAT_ENDIAN_SWAP__ 2006-06-25 10:01:15 -07:00
asm-parisc [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-powerpc [PATCH] Define __raw_get_cpu_var and use it 2006-06-25 10:01:01 -07:00
asm-ppc [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-s390 [PATCH] Define __raw_get_cpu_var and use it 2006-06-25 10:01:01 -07:00
asm-sh [PATCH] random: remove SA_SAMPLE_RANDOM from floppy driver 2006-06-25 10:01:00 -07:00
asm-sh64 Remove unneeded _syscallX macros from user view in asm-*/unistd.h 2006-04-29 01:51:47 +01:00
asm-sparc [SBUS]: Rewrite and plug into of_device framework. 2006-06-23 23:15:50 -07:00
asm-sparc64 [PATCH] Define __raw_get_cpu_var and use it 2006-06-25 10:01:01 -07:00
asm-um Merge git://git.infradead.org/hdrcleanup-2.6 2006-06-20 15:10:08 -07:00
asm-v850 [PATCH] adjust handle_IRR_event() return type 2006-06-23 07:43:08 -07:00
asm-x86_64 [PATCH] Define __raw_get_cpu_var and use it 2006-06-25 10:01:01 -07:00
asm-xtensa [PATCH] xtensa: remove verify_area macros 2006-06-23 07:43:02 -07:00
keys
linux [PATCH] Get rid of /proc/sys/proc 2006-06-25 10:01:15 -07:00
math-emu
media Revert include/media changes: Mauro says those ioctls are only used in-kernel(!) 2006-05-06 19:47:57 +01:00
mtd Merge git://git.infradead.org/hdrcleanup-2.6 2006-06-20 15:10:08 -07:00
net [NET]: fix net-core kernel-doc 2006-06-23 02:07:42 -07:00
pcmcia Don't include linux/config.h from anywhere else in include/ 2006-04-26 12:56:16 +01:00
rdma IB/uverbs: Don't serialize with ib_uverbs_idr_mutex 2006-06-17 20:44:49 -07:00
rxrpc
scsi Merge branch 'master' into upstream 2006-06-22 22:11:56 -04:00
sound [ALSA] version 1.0.12rc1 2006-06-22 21:35:11 +02:00
video Don't include linux/config.h from anywhere else in include/ 2006-04-26 12:56:16 +01:00