linux/drivers/char
Aristeu Sergio Rozanski Filho 5a39e8c6d6 [PATCH] tty_io: fix race in master pty close/slave pty close path
This patch fixes a possible race that leads to double freeing an idr index.
 When the master begin to close, release_dev() is called and then
pty_close() is called:

        if (tty->driver->close)
                tty->driver->close(tty, filp);

This is done without helding any locks other than BKL.  Inside pty_close(),
being a master close, the devpts entry will be removed:

#ifdef CONFIG_UNIX98_PTYS
                if (tty->driver == ptm_driver)
                        devpts_pty_kill(tty->index);
#endif

But devpts_pty_kill() will call get_node() that may sleep while waiting for
&devpts_root->d_inode->i_sem.  When this happens and the slave is being
opened, tty_open() just found the driver and index:

        driver = get_tty_driver(device, &index);
        if (!driver) {
                mutex_unlock(&tty_mutex);
                return -ENODEV;
        }

This part of the code is already protected under tty_mute.  The problem is
that the slave close already got an index.  Then init_dev() is called and
blocks waiting for the same &devpts_root->d_inode->i_sem.

When the master close resumes, it removes the devpts entry, and the
relation between idr index and the tty is gone.  The master then sleeps
waiting for the tty_mutex on release_dev().

Slave open resumes and found no tty for that index.  As result, a NULL tty
is returned and init_dev() doesn't flow to fast_track:

        /* check whether we're reopening an existing tty */
        if (driver->flags & TTY_DRIVER_DEVPTS_MEM) {
                tty = devpts_get_tty(idx);
                if (tty && driver->subtype == PTY_TYPE_MASTER)
                        tty = tty->link;
        } else {
                tty = driver->ttys[idx];
        }
        if (tty) goto fast_track;

The result of this, is that a new tty will be created and init_dev() returns
sucessfull. After returning, tty_mutex is dropped and master close may resume.

Master close finds it's the only use and both sides are closing, then releases
the tty and the index. At this point, the idr index is free, but slave still
has it.

Slave open then calls pty_open() and finds that tty->link->count is 0,
because there's no master and returns error.  Then tty_open() calls
release_dev() which executes without any warning, as it was a case of last
slave close when the master is already closed (master->count == 0,
slave->count == 1).  The tty is then released with the already released idr
index.

This normally would only issue a warning on idr_remove() but in case of a
customer's critical application, it's never too simple:

thread1: opens master, gets index X
thread1: begin closing master
thread2: begin opening slave with index X
thread1: finishes closing master, index X released
thread3: opens master, gets index X, just released
thread2: fails opening slave, releases index X         <----
thread4: opens master, gets index X, init_dev() then find an already in use
	 and healthy tty and fails

If no more indexes are released, ptmx_open() will keep failing, as the
first free index available is X, and it will make init_dev() fail because
you're trying to "reopen a master" which isn't valid.

The patch notices when this race happens and make init_dev() fail
imediately.  The init_dev() function is called with tty_mutex held, so it's
safe to continue with tty till the end of function because release_dev()
won't make any further changes without grabbing the tty_mutex.

Without the patch, on some machines it's possible get easily idr warnings
like this one:

idr_remove called for id=15 which is not allocated.
 [<c02555b9>] idr_remove+0x139/0x170
 [<c02a1b62>] release_mem+0x182/0x230
 [<c02a28e7>] release_dev+0x4b7/0x700
 [<c02a0ea7>] tty_ldisc_enable+0x27/0x30
 [<c02a1e64>] init_dev+0x254/0x580
 [<c02a0d64>] check_tty_count+0x14/0xb0
 [<c02a4f05>] tty_open+0x1c5/0x340
 [<c02a4d40>] tty_open+0x0/0x340
 [<c017388f>] chrdev_open+0xaf/0x180
 [<c017c2ac>] open_namei+0x8c/0x760
 [<c01737e0>] chrdev_open+0x0/0x180
 [<c0167bc9>] __dentry_open+0xc9/0x210
 [<c0167e2c>] do_filp_open+0x5c/0x70
 [<c0167a91>] get_unused_fd+0x61/0xd0
 [<c0167e93>] do_sys_open+0x53/0x100
 [<c0167f97>] sys_open+0x27/0x30
 [<c010303b>] syscall_call+0x7/0xb

using this test application available on:
 http://www.ruivo.org/~aris/pty_sodomizer.c

Signed-off-by: Aristeu Sergio Rozanski Filho <aris@ruivo.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Chuck Ebbert <cebbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-03-01 14:53:39 -08:00
..
agp [AGPGART] fix compile errors 2007-02-27 00:36:00 -05:00
drm [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
hw_random [PATCH] Char: use more PCI_DEVICE macro 2007-02-12 09:48:27 -08:00
ip2 Fix bogus 'inline' in drivers/char/ip2/i2lib.c 2007-02-21 11:18:26 -08:00
ipmi [PATCH] sysctl: remove insert_at_head from register_sysctl 2007-02-14 08:09:59 -08:00
mwave [PATCH] mwave: interesting flags savings 2007-02-20 17:10:14 -08:00
pcmcia [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
rio rio: typo in bitwise AND expression. 2007-02-17 18:57:09 +01:00
tpm [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
watchdog [WATCHDOG] machzwd warning fix 2007-02-17 22:42:15 +01:00
.gitignore [MIPS] Remove IT8172-based platforms, ITE 8172G and Globespan IVR support. 2006-10-03 17:59:17 +01:00
amiserial.c [PATCH] CHAR-Amiserial: turn local_save_flags() + local_irq_disable() into local_irq_save() 2007-02-11 11:18:07 -08:00
apm-emulation.c [APM] Add shared version of APM emulation 2007-02-09 17:08:57 +00:00
applicom.c IRQ: Maintain regs pointer globally rather than passing to IRQ handlers 2006-10-05 15:10:12 +01:00
applicom.h
briq_panel.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
cd1865.h
ChangeLog
consolemap.c [PATCH] getting rid of all casts of k[cmz]alloc() calls 2006-12-13 09:05:58 -08:00
cp437.uni
cs5535_gpio.c [PATCH] struct path: convert char-drivers 2006-12-08 08:28:44 -08:00
cyclades.c [PATCH] Char: cyclades, use pci_device_id 2007-02-12 09:48:28 -08:00
decserial.c [PATCH] dz: Fixes to make it work 2006-12-07 08:39:41 -08:00
defkeymap.c_shipped
defkeymap.map
digi1.h
digi.h
digiFep1.h
digiPCI.h
ds1286.c [PATCH] proper flags type of spin_lock_irqsave() 2006-10-01 00:39:21 -07:00
ds1302.c [PATCH] DS1302: local_irq_disable() is redundant after local_irq_save() 2007-02-12 09:48:30 -08:00
ds1620.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
dsp56k.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
dtlk.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
ec3104_keyb.c IRQ: Maintain regs pointer globally rather than passing to IRQ handlers 2006-10-05 15:10:12 +01:00
efirtc.c
epca.c [PATCH] Char: tty_wakeup cleanup 2007-02-11 10:51:26 -08:00
epca.h [PATCH] char: kill unneeded memsets 2006-10-04 07:55:13 -07:00
epcaconfig.h
esp.c [PATCH] tty: switch to ktermios 2006-12-08 08:28:57 -08:00
generic_nvram.c [PATCH] mark struct file_operations const 3 2007-02-12 09:48:45 -08:00
generic_serial.c [PATCH] Char: tty_wakeup cleanup 2007-02-11 10:51:26 -08:00
genrtc.c WorkStruct: make allyesconfig 2006-11-22 14:57:56 +00:00
hangcheck-timer.c [PATCH] time: x86_64: convert x86_64 to use GENERIC_TIME 2007-02-16 08:14:00 -08:00
hpet.c [PATCH] sysctl: remove insert_at_head from register_sysctl 2007-02-14 08:09:59 -08:00
hvc_beat.c [POWERPC] Celleb: hypervisor console driver 2007-02-07 14:03:21 +11:00
hvc_console.c [PATCH] Make hvc_console.c compile on non-powerpc: Remove NO_IRQ 2007-02-26 12:58:33 -08:00
hvc_console.h
hvc_iseries.c Merge branch 'master' of git://oak/home/sfr/kernels/iseries/work 2006-10-04 15:02:27 +10:00
hvc_rtas.c
hvc_vio.c
hvcs.c [PATCH] tty: switch to ktermios 2006-12-08 08:28:57 -08:00
hvsi.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
i8k.c
ip27-rtc.c
isicom.c [PATCH] Char: tty_wakeup cleanup 2007-02-11 10:51:26 -08:00
istallion.c [PATCH] Char: tty_wakeup cleanup 2007-02-11 10:51:26 -08:00
Kconfig [PATCH] Char: mxser, obsolete old, nonexperimental new 2007-02-11 10:51:28 -08:00
keyboard.c [PATCH] Fix SAK_work workqueue initialization. 2007-02-13 16:07:36 -08:00
lcd.c [MIPS] Add MTD device support for Cobalt 2007-02-20 17:11:55 +00:00
lcd.h [MIPS] Add MTD device support for Cobalt 2007-02-20 17:11:55 +00:00
lp.c [PATCH] getting rid of all casts of k[cmz]alloc() calls 2006-12-13 09:05:58 -08:00
Makefile [APM] Add shared version of APM emulation 2007-02-09 17:08:57 +00:00
mbcs.c [PATCH] mark struct file_operations const 3 2007-02-12 09:48:45 -08:00
mbcs.h
mem.c Revert "[PATCH] Fix up mmap_kmem" 2007-01-22 08:53:24 -08:00
misc.c [PATCH] Correct misc_register return code handling in several drivers 2006-12-07 08:39:35 -08:00
mmtimer.c [PATCH] Correct misc_register return code handling in several drivers 2006-12-07 08:39:35 -08:00
moxa.c [PATCH] Char: moxa, pci probing 2007-02-11 10:51:30 -08:00
mspec.c [PATCH] mark struct file_operations const 3 2007-02-12 09:48:45 -08:00
mxser_new.c [PATCH] Char: mxser_new, fix sparse warning 2007-02-11 10:51:29 -08:00
mxser_new.h [PATCH] Char: mxser_new, upgrade to 1.9.15 2007-02-11 10:51:29 -08:00
mxser.c [PATCH] mxser: remove useless fields 2007-02-11 11:18:06 -08:00
mxser.h [PATCH] mxser: remove ambiguous redefinition of INIT_WORK 2007-02-11 10:51:25 -08:00
n_hdlc.c
n_r3964.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
n_tty.c [PATCH] tty: update the tty layer to work with struct pid 2007-02-12 09:48:32 -08:00
nsc_gpio.c [PATCH] struct path: convert char-drivers 2006-12-08 08:28:44 -08:00
nvram.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
nwbutton.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
nwbutton.h IRQ: Maintain regs pointer globally rather than passing to IRQ handlers 2006-10-05 15:10:12 +01:00
nwflash.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
pc8736x_gpio.c [PATCH] drivers/char/pc8736x_gpio.c: remove unused static functions 2006-09-29 09:18:05 -07:00
ppdev.c [PATCH] struct path: convert char-drivers 2006-12-08 08:28:44 -08:00
pty.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
random.c [PATCH] mark struct file_operations const 3 2007-02-12 09:48:45 -08:00
raw.c [PATCH] raw: don't allow the creation of a raw device with minor number 0 2007-02-11 10:51:34 -08:00
riscom8_reg.h
riscom8.c [PATCH] Char: tty_wakeup cleanup 2007-02-11 10:51:26 -08:00
riscom8.h
rocket_int.h
rocket.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
rocket.h
rtc.c [PATCH] sysctl: remove insert_at_head from register_sysctl 2007-02-14 08:09:59 -08:00
scc.h
scx200_gpio.c [PATCH] scx200_gpio export cleanups 2006-09-29 09:18:06 -07:00
selection.c [PATCH] tty locking on resize 2006-09-29 09:18:12 -07:00
ser_a2232.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
ser_a2232.h
ser_a2232fw.ax
ser_a2232fw.h
serial167.c [PATCH] Char: serial167, cleanup 2007-02-11 10:51:28 -08:00
snsc_event.c IRQ: Maintain regs pointer globally rather than passing to IRQ handlers 2006-10-05 15:10:12 +01:00
snsc.c IRQ: Maintain regs pointer globally rather than passing to IRQ handlers 2006-10-05 15:10:12 +01:00
snsc.h
sonypi.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
specialix_io8.h
specialix.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
stallion.c [PATCH] Char: stallion, use dynamic dev 2006-12-08 08:28:59 -08:00
sx.c [PATCH] sx: fix non-PCI build 2006-12-13 09:05:49 -08:00
sx.h [PATCH] Char: sx, request regions 2006-12-08 08:28:59 -08:00
sxboards.h
sxwindow.h
synclink_gt.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
synclink.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
synclinkmp.c [PATCH] Char: timers cleanup 2007-02-12 09:48:30 -08:00
sysrq.c [PATCH] Add SysRq-Q to print timer_list debug info 2007-02-16 08:13:59 -08:00
tb0219.c [PATCH] struct path: convert char-drivers 2006-12-08 08:28:44 -08:00
tipar.c [PATCH] struct path: convert char-drivers 2006-12-08 08:28:44 -08:00
tlclk.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
toshiba.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
tty_io.c [PATCH] tty_io: fix race in master pty close/slave pty close path 2007-03-01 14:53:39 -08:00
tty_ioctl.c [PATCH] tty: improve encode_baud_rate logic 2007-02-11 10:51:32 -08:00
vc_screen.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
viocons.c [POWERPC] iSeries: fix viocons init 2006-12-20 16:37:48 +11:00
viotape.c [PATCH] mark struct file_operations const 3 2007-02-12 09:48:45 -08:00
vme_scc.c [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
vr41xx_giu.c [MIPS] Vr41xx: Fix after GENERIC_HARDIRQS_NO__DO_IRQ change 2007-01-23 18:26:47 +00:00
vt_ioctl.c [PATCH] vt: refactor console SAK processing 2007-02-11 10:51:24 -08:00
vt.c [PATCH] Fix SAK_work workqueue initialization. 2007-02-13 16:07:36 -08:00