Ingo recently introduced a great speedup for allocating new mmaps using the
free_area_cache pointer which boosts the specweb SSL benchmark by 4-5% and
causes huge performance increases in thread creation.
The downside of this patch is that it does lead to fragmentation in the
mmap-ed areas (visible via /proc/self/maps), such that some applications
that work fine under 2.4 kernels quickly run out of memory on any 2.6
kernel.
The problem is twofold:
1) the free_area_cache is used to continue a search for memory where
the last search ended. Before the change new areas were always
searched from the base address on.
So now new small areas are cluttering holes of all sizes
throughout the whole mmap-able region whereas before small holes
tended to close holes near the base leaving holes far from the base
large and available for larger requests.
2) the free_area_cache also is set to the location of the last
munmap-ed area so in scenarios where we allocate e.g. five regions of
1K each, then free regions 4 2 3 in this order the next request for 1K
will be placed in the position of the old region 3, whereas before we
appended it to the still active region 1, placing it at the location
of the old region 2. Before we had 1 free region of 2K, now we only
get two free regions of 1K -> fragmentation.
The patch addresses thes issues by introducing yet another cache descriptor
cached_hole_size that contains the largest known hole size below the
current free_area_cache. If a new request comes in the size is compared
against the cached_hole_size and if the request can be filled with a hole
below free_area_cache the search is started from the base instead.
The results look promising: Whereas 2.6.12-rc4 fragments quickly and my
(earlier posted) leakme.c test program terminates after 50000+ iterations
with 96 distinct and fragmented maps in /proc/self/maps it performs nicely
(as expected) with thread creation, Ingo's test_str02 with 20000 threads
requires 0.7s system time.
Taking out Ingo's patch (un-patch available per request) by basically
deleting all mentions of free_area_cache from the kernel and starting the
search for new memory always at the respective bases we observe: leakme
terminates successfully with 11 distinctive hardly fragmented areas in
/proc/self/maps but thread creating is gringdingly slow: 30+s(!) system
time for Ingo's test_str02 with 20000 threads.
Now - drumroll ;-) the appended patch works fine with leakme: it ends with
only 7 distinct areas in /proc/self/maps and also thread creation seems
sufficiently fast with 0.71s for 20000 threads.
Signed-off-by: Wolfgang Wander <wwc@rentec.com>
Credit-to: "Richard Purdie" <rpurdie@rpsys.net>
Signed-off-by: Ken Chen <kenneth.w.chen@intel.com>
Acked-by: Ingo Molnar <mingo@elte.hu> (partly)
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Patch from Bellido Nicolas
Core support for AAEC-2000 based platforms.
This is an updated version of the previous patch, and takes
into account Russell's comments.
AAED-2000 default configuration will follow as soon
as some problems with the bootloader are sorted out...
Signed-off-by: Nicolas Bellido
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
flush_dcache_page() did nothing for these caches, but since they
suffer from I/D cache coherency issues, we need to ensure that data
is written back to RAM.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
Not that there might be many of them on the planet, but at least RMK
apparently has one.
Signed-off-by: Nicolas Pitre
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
The ARM copypage changes in 2.6.12-rc4-git1 removed the preempt locking
from the copypage functions which broke the XScale implementation.
This patch fixes the locking on XScale and removes the now unneeded
minicache code.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Checked-by: Richard Purdie
Patch from Nicolas Pitre
Not all ARMv6 processors implement the TLS register.
Signed-off-by: Nicolas Pitre
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Move the locking for copy_user_page() and clear_user_page() into
the implementations which require locking. For simple memcpy/
memset based implementations, the locking is extra overhead which
is not necessary, and prevents preemption occuring.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Add pmd_off() and pmd_off_k() to obtain the pmd pointer for a
virtual address, and use them throughout the mm initialisation.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Patch from Nicolas Pitre
This better express things, and should cover RMK's weird SMP toys.
Signed-off-by: Nicolas Pitre
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from George G. Davis
This patch is required for kernel XIP support on ARMv6 machines. It ensures that the access permission bits for kernel XIP section descriptors are APX=1 and AP[1:0]=01, which is Kernel read-only/User no access permissions. Prior to this change, kernel XIP section descriptor access permissions were set to Kernel no access/User no access on ARMv6 machines and the kernel would therefore hang upon entry to userspace when set_fs(USER_DS) was executed.
Signed-off-by: Steve Longerbeam
Signed-off-by: George G. Davis
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from Nicolas Pitre
This patch entirely reworks the kernel assistance for NPTL on ARM.
In particular this provides an efficient way to retrieve the TLS
value and perform atomic operations without any instruction emulation
nor special system call. This even allows for pre ARMv6 binaries to
be forward compatible with SMP systems without any penalty.
The problematic and performance critical operations are performed
through segment of kernel provided user code reachable from user space
at a fixed address in kernel memory. Those fixed entry points are
within the vector page so we basically get it for free as no extra
memory page is required and nothing else may be mapped at that
location anyway.
This is different from (but doesn't preclude) a full blown VDSO
implementation, however a VDSO would prevent some assembly tricks with
constants that allows for efficient branching to those code segments.
And since those code segments only use a few cycles before returning to
user code, the overhead of a VDSO far call would add a significant
overhead to such minimalistic operations.
The ARM_NR_set_tls syscall also changed number. This is done for two
reasons:
1) this patch changes the way the TLS value was previously meant to be
retrieved, therefore we ensure whatever library using the old way
gets fixed (they only exist in private tree at the moment since the
NPTL work is still progressing).
2) the previous number was allocated in a range causing an undefined
instruction trap on kernels not supporting that syscall and it was
determined that allocating it in a range returning -ENOSYS would be
much nicer for libraries trying to determine if the feature is
present or not.
Signed-off-by: Nicolas Pitre
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Patch from George G. Davis
As noted in http://www.arm.com/linux/patch-2.6.9-arm1.gz, the "Faulty SWP instruction on 1136 doesn't set bit 11 in DFSR." So the v6_early_abort handler does not report the correct rd/wr direction for the SWP instruction which may result in SEGVS or hangs. In order to work around this problem, this patch merely updates the fix contained in the ARM Ltd. patch to use the macroised abort handler fixups.
Signed-off-by: George G. Davis
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
)
From: Russell King <rmk+lkml@arm.linux.org.uk>
Oddly, max_low_pfn/max_pfn end up being the number of pages in the system,
rather than the maximum PFN on ARM. This doesn't seem to cause any problems,
so just add a note about it.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
)
From: Russell King <rmk+lkml@arm.linux.org.uk>
ARM wasn't raising a SIGBUS with a siginfo structure. Fix
__do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
for the sigbus path to use this.
We need to prevent the siginfo code being called if we do not have a user
space context to call it, so consolidate the "user_mode()" tests.
Thanks to Ian Campbell who spotted this oversight.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
