linux/drivers/video/console
Johannes Weiner 0035fe00f7 fbcon: don't use vc_resize() on initialization
Catalin and kmemleak spotted a leak of a VC screen buffer in
vc_allocate() due to the following chain of events:

	vc_allocate()
	  visual_init(init=1)
	    vc->vc_sw->con_init(init=1)
              fbcon_init()
	        vc_resize()
	          vc->screen_buf = kmalloc()
	  vc->screen_buf = kmalloc()

The common way for the VC drivers is to set the screen dimension
parameters manually in the init case and only call vc_resize() for
!init - which allocates a screen buffer according to the new
dimensions.

fbcon instead would do vc_resize() unconditionally and afterwards set
the dimensions manually (again) for !init - i.e. completely upside
down.  The vc_resize() allocated buffer would then get lost by
vc_allocate() allocating a fresh one.

Use vc_resize() only for actual resizing to close the leak.

Set the dimensions manually only in initialization mode to remove the
redundant setting in resize mode.

The kmemleak trace from Catalin:

unreferenced object 0xde158000 (size 12288):
  comm "Xorg", pid 1439, jiffies 4294961016
  hex dump (first 32 bytes):
    20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00   . . . . . . . .
    20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00   . . . . . . . .
  backtrace:
    [<c006f74b>] __save_stack_trace+0x17/0x1c
    [<c006f81d>] create_object+0xcd/0x188
    [<c01f5457>] kmemleak_alloc+0x1b/0x3c
    [<c006e303>] __kmalloc+0xdb/0xe8
    [<c012cc4b>] vc_do_resize+0x73/0x1e0
    [<c012cdf1>] vc_resize+0x15/0x18
    [<c011afc1>] fbcon_init+0x1f9/0x2b8
    [<c0129e87>] visual_init+0x9f/0xdc
    [<c012aff3>] vc_allocate+0x7f/0xfc
    [<c012b087>] con_open+0x17/0x80
    [<c0120e43>] tty_open+0x1f7/0x2e4
    [<c0072fa1>] chrdev_open+0x101/0x118
    [<c006ffad>] __dentry_open+0x105/0x1cc
    [<c00700fd>] nameidata_to_filp+0x2d/0x38
    [<c00788cd>] do_filp_open+0x2c1/0x54c
    [<c006fdff>] do_sys_open+0x3b/0xb4

Reported-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Tested-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Krzysztof Helt <krzysztof.h1@poczta.fm>
Tested-by: Dave Young <hidave.darkstar@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-08-07 10:39:56 -07:00
..
.gitignore sparc: Ignore drivers/video/console/promcon_tbl.c conmakehash generated file 2008-07-30 16:49:52 -07:00
bitblit.c fbcon: fix color generation for monochrome framebuffer 2008-02-06 10:41:18 -08:00
dummycon.c Remove magic macros for screen_info structure members 2007-10-16 22:57:17 -07:00
fbcon_ccw.c fbcon: fix color generation for monochrome framebuffer 2008-02-06 10:41:18 -08:00
fbcon_cw.c fbcon: fix color generation for monochrome framebuffer 2008-02-06 10:41:18 -08:00
fbcon_rotate.c
fbcon_rotate.h fbcon: fix rotate upside down crash 2009-08-07 10:39:55 -07:00
fbcon_ud.c fbcon: fix color generation for monochrome framebuffer 2008-02-06 10:41:18 -08:00
fbcon.c fbcon: don't use vc_resize() on initialization 2009-08-07 10:39:56 -07:00
fbcon.h fbcon: fix monochrome color value calculation 2008-10-02 15:53:13 -07:00
font_6x11.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_7x14.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_8x8.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_8x16.c export font_vga_8x16 2007-10-16 09:43:19 -07:00
font_10x18.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_acorn_8x8.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_mini_4x6.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_pearl_8x8.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_sun8x16.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
font_sun12x22.c fbcon: Convert struct font_desc to use ISO C initializers 2007-10-16 09:43:17 -07:00
fonts.c cleanup after APUS removal 2008-02-06 10:41:01 -08:00
Kconfig x86 setup: remove remnants of CONFIG_VIDEO_SELECT (read: vga=) 2008-09-03 14:12:39 -07:00
Makefile
mdacon.c Revert "vt: fix background color on line feed" 2008-10-14 12:12:02 -07:00
newport_con.c netport_con.c: fix build errors and warnings 2007-10-18 14:37:17 -07:00
prom.uni
promcon.c
softcursor.c fbdev: change asm/uaccess.h to linux/uaccess.h 2007-10-16 09:43:17 -07:00
sticon.c Revert "vt: fix background color on line feed" 2008-10-14 12:12:02 -07:00
sticore.c parisc: sticore.c - check return values 2009-08-02 15:26:51 +02:00
tileblit.c fbcon: fix color generation for monochrome framebuffer 2008-02-06 10:41:18 -08:00
vgacon.c vgacon: use slab allocator instead of the bootmem allocator 2009-06-11 19:33:34 +03:00