2012-05-22 02:50:07 +00:00
|
|
|
/*
|
2008-03-19 17:26:14 +00:00
|
|
|
* x86 SMP booting functions
|
|
|
|
*
|
2009-01-05 14:08:04 +00:00
|
|
|
* (c) 1995 Alan Cox, Building #3 <alan@lxorguk.ukuu.org.uk>
|
2009-01-31 01:03:42 +00:00
|
|
|
* (c) 1998, 1999, 2000, 2009 Ingo Molnar <mingo@redhat.com>
|
2008-03-19 17:26:14 +00:00
|
|
|
* Copyright 2001 Andi Kleen, SuSE Labs.
|
|
|
|
*
|
|
|
|
* Much of the core SMP work is based on previous work by Thomas Radke, to
|
|
|
|
* whom a great many thanks are extended.
|
|
|
|
*
|
|
|
|
* Thanks to Intel for making available several different Pentium,
|
|
|
|
* Pentium Pro and Pentium-II/Xeon MP machines.
|
|
|
|
* Original development of Linux SMP code supported by Caldera.
|
|
|
|
*
|
|
|
|
* This code is released under the GNU General Public License version 2 or
|
|
|
|
* later.
|
|
|
|
*
|
|
|
|
* Fixes
|
|
|
|
* Felix Koop : NR_CPUS used properly
|
|
|
|
* Jose Renau : Handle single CPU case.
|
|
|
|
* Alan Cox : By repeated request 8) - Total BogoMIPS report.
|
|
|
|
* Greg Wright : Fix for kernel stacks panic.
|
|
|
|
* Erich Boleyn : MP v1.4 and additional changes.
|
|
|
|
* Matthias Sattler : Changes for 2.1 kernel map.
|
|
|
|
* Michel Lespinasse : Changes for 2.1 kernel map.
|
|
|
|
* Michael Chastain : Change trampoline.S to gnu as.
|
|
|
|
* Alan Cox : Dumb bug: 'B' step PPro's are fine
|
|
|
|
* Ingo Molnar : Added APIC timers, based on code
|
|
|
|
* from Jose Renau
|
|
|
|
* Ingo Molnar : various cleanups and rewrites
|
|
|
|
* Tigran Aivazian : fixed "0.00 in /proc/uptime on SMP" bug.
|
|
|
|
* Maciej W. Rozycki : Bits for genuine 82489DX APICs
|
|
|
|
* Andi Kleen : Changed for SMP boot into long mode.
|
|
|
|
* Martin J. Bligh : Added support for multi-quad systems
|
|
|
|
* Dave Jones : Report invalid combinations of Athlon CPUs.
|
|
|
|
* Rusty Russell : Hacked into shape for new "hotplug" boot process.
|
|
|
|
* Andi Kleen : Converted to new state machine.
|
|
|
|
* Ashok Raj : CPU hotplug support
|
|
|
|
* Glauber Costa : i386 and x86_64 integration
|
|
|
|
*/
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
|
|
|
|
|
2008-03-03 17:12:42 +00:00
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/smp.h>
|
2016-07-14 00:18:56 +00:00
|
|
|
#include <linux/export.h>
|
2008-03-03 17:13:03 +00:00
|
|
|
#include <linux/sched.h>
|
2017-02-01 15:36:40 +00:00
|
|
|
#include <linux/sched/topology.h>
|
2017-02-08 17:51:36 +00:00
|
|
|
#include <linux/sched/hotplug.h>
|
2017-02-08 17:51:37 +00:00
|
|
|
#include <linux/sched/task_stack.h>
|
2008-03-03 17:13:07 +00:00
|
|
|
#include <linux/percpu.h>
|
2008-03-03 17:13:12 +00:00
|
|
|
#include <linux/bootmem.h>
|
2008-03-19 17:25:59 +00:00
|
|
|
#include <linux/err.h>
|
|
|
|
#include <linux/nmi.h>
|
2009-09-02 01:25:07 +00:00
|
|
|
#include <linux/tboot.h>
|
2009-09-17 14:36:43 +00:00
|
|
|
#include <linux/stackprotector.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 08:04:11 +00:00
|
|
|
#include <linux/gfp.h>
|
2012-03-13 18:55:09 +00:00
|
|
|
#include <linux/cpuidle.h>
|
2008-03-03 17:13:07 +00:00
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
#include <asm/acpi.h>
|
2008-03-19 17:25:59 +00:00
|
|
|
#include <asm/desc.h>
|
2008-03-03 17:13:07 +00:00
|
|
|
#include <asm/nmi.h>
|
|
|
|
#include <asm/irq.h>
|
2012-05-08 18:22:28 +00:00
|
|
|
#include <asm/realmode.h>
|
2008-03-03 17:13:07 +00:00
|
|
|
#include <asm/cpu.h>
|
|
|
|
#include <asm/numa.h>
|
2008-03-19 17:25:59 +00:00
|
|
|
#include <asm/pgtable.h>
|
|
|
|
#include <asm/tlbflush.h>
|
|
|
|
#include <asm/mtrr.h>
|
2010-09-17 22:39:11 +00:00
|
|
|
#include <asm/mwait.h>
|
2009-02-17 12:58:15 +00:00
|
|
|
#include <asm/apic.h>
|
2011-02-22 14:38:05 +00:00
|
|
|
#include <asm/io_apic.h>
|
2015-04-24 00:54:44 +00:00
|
|
|
#include <asm/fpu/internal.h>
|
x86: fix wakeup_cpu with numaq/es7000, v2
Impact: fix secondary-CPU wakeup/init path with numaq and es7000
While looking at wakeup_secondary_cpu for WAKE_SECONDARY_VIA_NMI:
|#ifdef WAKE_SECONDARY_VIA_NMI
|/*
| * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
| * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
| * won't ... remember to clear down the APIC, etc later.
| */
|static int __devinit
|wakeup_secondary_cpu(int logical_apicid, unsigned long start_eip)
|{
| unsigned long send_status, accept_status = 0;
| int maxlvt;
|...
| if (APIC_INTEGRATED(apic_version[phys_apicid])) {
| maxlvt = lapic_get_maxlvt();
I noticed that there is no warning about undefined phys_apicid...
because WAKE_SECONDARY_VIA_NMI and WAKE_SECONDARY_VIA_INIT can not be
defined at the same time. So NUMAQ is using wrong wakeup_secondary_cpu.
WAKE_SECONDARY_VIA_NMI, WAKE_SECONDARY_VIA_INIT and
WAKE_SECONDARY_VIA_MIP are variants of a weird and fragile
preprocessor-driven "HAL" mechanisms to specify the kind of secondary-CPU
wakeup strategy a given x86 kernel will use.
The vast majority of systems want to use INIT for secondary wakeup - NUMAQ
uses an NMI, (old-style-) ES7000 uses 'MIP' (a firmware driven in-memory
flag to let secondaries continue).
So convert these mechanisms to x86_quirks and add a
->wakeup_secondary_cpu() method to specify the rare exception
to the sane default.
Extend genapic accordingly as well, for 32-bit.
While looking further, I noticed that functions in wakecup.h for numaq
and es7000 are different to the default in mach_wakecpu.h - but smpboot.c
will only use default mach_wakecpu.h with smphook.h.
So we need to add mach_wakecpu.h for mach_generic, to properly support
numaq and es7000, and vectorize the following SMP init methods:
int trampoline_phys_low;
int trampoline_phys_high;
void (*wait_for_init_deassert)(atomic_t *deassert);
void (*smp_callin_clear_local_apic)(void);
void (*store_NMI_vector)(unsigned short *high, unsigned short *low);
void (*restore_NMI_vector)(unsigned short *high, unsigned short *low);
void (*inquire_remote_apic)(int apicid);
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-11-16 11:12:49 +00:00
|
|
|
#include <asm/setup.h>
|
2009-01-21 08:26:06 +00:00
|
|
|
#include <asm/uv/uv.h>
|
2008-03-19 17:25:59 +00:00
|
|
|
#include <linux/mc146818rtc.h>
|
2009-11-09 19:27:04 +00:00
|
|
|
#include <asm/i8259.h>
|
x86: Improve the printout of the SMP bootup CPU table
As the new x86 CPU bootup printout format code maintainer, I am
taking immediate action to improve and clean (and thus indulge
my OCD) the reporting of the cores when coming up online.
Fix padding to a right-hand alignment, cleanup code and bind
reporting width to the max number of supported CPUs on the
system, like this:
[ 0.074509] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.644008] smpboot: Booting Node 1, Processors: #8 #9 #10 #11 #12 #13 #14 #15 OK
[ 1.245006] smpboot: Booting Node 2, Processors: #16 #17 #18 #19 #20 #21 #22 #23 OK
[ 1.864005] smpboot: Booting Node 3, Processors: #24 #25 #26 #27 #28 #29 #30 #31 OK
[ 2.489005] smpboot: Booting Node 4, Processors: #32 #33 #34 #35 #36 #37 #38 #39 OK
[ 3.093005] smpboot: Booting Node 5, Processors: #40 #41 #42 #43 #44 #45 #46 #47 OK
[ 3.698005] smpboot: Booting Node 6, Processors: #48 #49 #50 #51 #52 #53 #54 #55 OK
[ 4.304005] smpboot: Booting Node 7, Processors: #56 #57 #58 #59 #60 #61 #62 #63 OK
[ 4.961413] Brought up 64 CPUs
and this:
[ 0.072367] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.686329] Brought up 8 CPUs
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Libin <huawei.libin@huawei.com>
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Link: http://lkml.kernel.org/r/20130927143554.GF4422@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-27 14:35:54 +00:00
|
|
|
#include <asm/misc.h>
|
2017-09-06 17:36:24 +00:00
|
|
|
#include <asm/qspinlock.h>
|
2012-05-08 18:22:28 +00:00
|
|
|
|
2008-03-03 17:12:58 +00:00
|
|
|
/* Number of siblings per CPU package */
|
|
|
|
int smp_num_siblings = 1;
|
|
|
|
EXPORT_SYMBOL(smp_num_siblings);
|
|
|
|
|
|
|
|
/* Last level cache ID of each logical CPU */
|
x86: Add read_mostly declaration/definition to variables from smp.h
Add "read-mostly" qualifier to the following variables in
smp.h:
- cpu_sibling_map
- cpu_core_map
- cpu_llc_shared_map
- cpu_llc_id
- cpu_number
- x86_cpu_to_apicid
- x86_bios_cpu_apicid
- x86_cpu_to_logical_apicid
As long as all the variables above are only written during the
initialization, this change is meant to prevent the false
sharing. More specifically, on vSMP Foundation platform
x86_cpu_to_apicid shared the same internode_cache_line with
frequently written lapic_events.
From the analysis of the first 33 per_cpu variables out of 219
(memories they describe, to be more specific) the 8 have read_mostly
nature (tlb_vector_offset, cpu_loops_per_jiffy, xen_debug_irq, etc.)
and 25 are frequently written (irq_stack_union, gdt_page,
exception_stacks, idt_desc, etc.).
Assuming that the spread of the rest of the per_cpu variables is
similar, identifying the read mostly memories will make more sense
in terms of long-term code maintenance comparing to identifying
frequently written memories.
Signed-off-by: Vlad Zolotarov <vlad@scalemp.com>
Acked-by: Shai Fultheim <shai@scalemp.com>
Cc: Shai Fultheim (Shai@ScaleMP.com) <Shai@scalemp.com>
Cc: ido@wizery.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1719258.EYKzE4Zbq5@vlad
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2012-06-11 09:56:52 +00:00
|
|
|
DEFINE_PER_CPU_READ_MOSTLY(u16, cpu_llc_id) = BAD_APICID;
|
2008-03-03 17:12:58 +00:00
|
|
|
|
|
|
|
/* representing HT siblings of each logical CPU */
|
x86: Add read_mostly declaration/definition to variables from smp.h
Add "read-mostly" qualifier to the following variables in
smp.h:
- cpu_sibling_map
- cpu_core_map
- cpu_llc_shared_map
- cpu_llc_id
- cpu_number
- x86_cpu_to_apicid
- x86_bios_cpu_apicid
- x86_cpu_to_logical_apicid
As long as all the variables above are only written during the
initialization, this change is meant to prevent the false
sharing. More specifically, on vSMP Foundation platform
x86_cpu_to_apicid shared the same internode_cache_line with
frequently written lapic_events.
From the analysis of the first 33 per_cpu variables out of 219
(memories they describe, to be more specific) the 8 have read_mostly
nature (tlb_vector_offset, cpu_loops_per_jiffy, xen_debug_irq, etc.)
and 25 are frequently written (irq_stack_union, gdt_page,
exception_stacks, idt_desc, etc.).
Assuming that the spread of the rest of the per_cpu variables is
similar, identifying the read mostly memories will make more sense
in terms of long-term code maintenance comparing to identifying
frequently written memories.
Signed-off-by: Vlad Zolotarov <vlad@scalemp.com>
Acked-by: Shai Fultheim <shai@scalemp.com>
Cc: Shai Fultheim (Shai@ScaleMP.com) <Shai@scalemp.com>
Cc: ido@wizery.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1719258.EYKzE4Zbq5@vlad
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2012-06-11 09:56:52 +00:00
|
|
|
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
|
2008-03-03 17:12:58 +00:00
|
|
|
EXPORT_PER_CPU_SYMBOL(cpu_sibling_map);
|
|
|
|
|
|
|
|
/* representing HT and core siblings of each logical CPU */
|
x86: Add read_mostly declaration/definition to variables from smp.h
Add "read-mostly" qualifier to the following variables in
smp.h:
- cpu_sibling_map
- cpu_core_map
- cpu_llc_shared_map
- cpu_llc_id
- cpu_number
- x86_cpu_to_apicid
- x86_bios_cpu_apicid
- x86_cpu_to_logical_apicid
As long as all the variables above are only written during the
initialization, this change is meant to prevent the false
sharing. More specifically, on vSMP Foundation platform
x86_cpu_to_apicid shared the same internode_cache_line with
frequently written lapic_events.
From the analysis of the first 33 per_cpu variables out of 219
(memories they describe, to be more specific) the 8 have read_mostly
nature (tlb_vector_offset, cpu_loops_per_jiffy, xen_debug_irq, etc.)
and 25 are frequently written (irq_stack_union, gdt_page,
exception_stacks, idt_desc, etc.).
Assuming that the spread of the rest of the per_cpu variables is
similar, identifying the read mostly memories will make more sense
in terms of long-term code maintenance comparing to identifying
frequently written memories.
Signed-off-by: Vlad Zolotarov <vlad@scalemp.com>
Acked-by: Shai Fultheim <shai@scalemp.com>
Cc: Shai Fultheim (Shai@ScaleMP.com) <Shai@scalemp.com>
Cc: ido@wizery.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1719258.EYKzE4Zbq5@vlad
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2012-06-11 09:56:52 +00:00
|
|
|
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_map);
|
2008-03-03 17:12:58 +00:00
|
|
|
EXPORT_PER_CPU_SYMBOL(cpu_core_map);
|
|
|
|
|
x86: Add read_mostly declaration/definition to variables from smp.h
Add "read-mostly" qualifier to the following variables in
smp.h:
- cpu_sibling_map
- cpu_core_map
- cpu_llc_shared_map
- cpu_llc_id
- cpu_number
- x86_cpu_to_apicid
- x86_bios_cpu_apicid
- x86_cpu_to_logical_apicid
As long as all the variables above are only written during the
initialization, this change is meant to prevent the false
sharing. More specifically, on vSMP Foundation platform
x86_cpu_to_apicid shared the same internode_cache_line with
frequently written lapic_events.
From the analysis of the first 33 per_cpu variables out of 219
(memories they describe, to be more specific) the 8 have read_mostly
nature (tlb_vector_offset, cpu_loops_per_jiffy, xen_debug_irq, etc.)
and 25 are frequently written (irq_stack_union, gdt_page,
exception_stacks, idt_desc, etc.).
Assuming that the spread of the rest of the per_cpu variables is
similar, identifying the read mostly memories will make more sense
in terms of long-term code maintenance comparing to identifying
frequently written memories.
Signed-off-by: Vlad Zolotarov <vlad@scalemp.com>
Acked-by: Shai Fultheim <shai@scalemp.com>
Cc: Shai Fultheim (Shai@ScaleMP.com) <Shai@scalemp.com>
Cc: ido@wizery.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1719258.EYKzE4Zbq5@vlad
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2012-06-11 09:56:52 +00:00
|
|
|
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map);
|
2011-01-21 23:29:44 +00:00
|
|
|
|
2008-03-03 17:12:58 +00:00
|
|
|
/* Per CPU bogomips and other parameters */
|
2014-11-04 08:26:42 +00:00
|
|
|
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
|
2008-03-03 17:12:58 +00:00
|
|
|
EXPORT_PER_CPU_SYMBOL(cpu_info);
|
2008-03-03 17:13:02 +00:00
|
|
|
|
2016-02-22 22:19:15 +00:00
|
|
|
/* Logical package management. We might want to allocate that dynamically */
|
|
|
|
unsigned int __max_logical_packages __read_mostly;
|
|
|
|
EXPORT_SYMBOL(__max_logical_packages);
|
x86/smp: Fix __max_logical_packages value setup
Frank reported kernel panic when he disabled several cores in BIOS
via following option:
Core Disable Bitmap(Hex) [0]
with number 0xFFE, which leaves 16 CPUs in system (out of 48).
The kernel panic below goes along with following messages:
smpboot: Max logical packages: 2^M
smpboot: APIC(0) Converting physical 0 to logical package 0^M
smpboot: APIC(20) Converting physical 1 to logical package 1^M
smpboot: APIC(40) Package 2 exceeds logical package map^M
smpboot: CPU 8 APICId 40 disabled^M
smpboot: APIC(60) Package 3 exceeds logical package map^M
smpboot: CPU 12 APICId 60 disabled^M
...
general protection fault: 0000 [#1] SMP^M
Modules linked in:^M
CPU: 15 PID: 1 Comm: swapper/0 Not tainted 4.7.0-rc5+ #1^M
Hardware name: SGI UV300/UV300, BIOS SGI UV 300 series BIOS 05/25/2016^M
task: ffff8801673e0000 ti: ffff8801673ac000 task.ti: ffff8801673ac000^M
RIP: 0010:[<ffffffff81014d54>] [<ffffffff81014d54>] uncore_change_context+0xd4/0x180^M
...
[<ffffffff810158ac>] uncore_event_init_cpu+0x6c/0x70^M
[<ffffffff81d8c91c>] intel_uncore_init+0x1c2/0x2dd^M
[<ffffffff81d8c75a>] ? uncore_cpu_setup+0x17/0x17^M
[<ffffffff81002190>] do_one_initcall+0x50/0x190^M
[<ffffffff810ab193>] ? parse_args+0x293/0x480^M
[<ffffffff81d87365>] kernel_init_freeable+0x1a5/0x249^M
[<ffffffff81d86a35>] ? set_debug_rodata+0x12/0x12^M
[<ffffffff816dc19e>] kernel_init+0xe/0x110^M
[<ffffffff816e93bf>] ret_from_fork+0x1f/0x40^M
[<ffffffff816dc190>] ? rest_init+0x80/0x80^M
The reason for the panic is wrong value of __max_logical_packages,
which lets logical_package_map uninitialized and the uncore code
relying on this map being properly initialized (maybe we should
add some safety checks there as well).
The __max_logical_packages is computed as:
DIV_ROUND_UP(total_cpus, ncpus);
- ncpus being number of cores
With above BIOS setup we get total_cpus == 16 which set
__max_logical_packages to 2 (ncpus is 12).
Once topology_update_package_map processes CPU with logical
pkg over 2 we display above messages and fail to initialize
the physical_to_logical_pkg map, which makes the uncore code
crash.
The fix is to remove logical_package_map bitmap completely
and keep and update the logical_packages number instead.
After we enumerate all the present CPUs, we check if the
enumerated logical packages count is within its computed
maximum from BIOS data.
If it's not the case, we set this maximum to the new enumerated
value and freeze any new addition of logical packages.
The freeze is because lot of init code like uncore/rapl/cqm
depends on having maximum logical package value set to allocate
their data, so we can't change it later on.
Prarit Bhargava tested the patch and confirms that it solves
the problem:
From dmidecode:
Core Count: 24
Core Enabled: 24
Thread Count: 48
Orig kernel boot log:
[ 0.464981] smpboot: Max logical packages: 19
[ 0.469861] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.477261] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.484760] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.492258] smpboot: APIC(c0) Converting physical 3 to logical package 3
1. nr_cpus=8, should stop enumerating in package 0:
[ 0.533664] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.539596] smpboot: Max logical packages: 19
2. max_cpus=8, should still enumerate all packages:
[ 0.526494] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.532428] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.538456] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.544486] smpboot: APIC(c0) Converting physical 3 to logical package 3
[ 0.550524] smpboot: Max logical packages: 19
3. nr_cpus=49 ( 2 socket + 1 core on 3rd socket), should stop enumerating in
package 2:
[ 0.521378] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.527314] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.533345] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.539368] smpboot: Max logical packages: 19
4. maxcpus=49, should still enumerate all packages:
[ 0.525591] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.531525] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.537547] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.543579] smpboot: APIC(c0) Converting physical 3 to logical package 3
[ 0.549624] smpboot: Max logical packages: 19
5. kdump (nr_cpus=1) works as well.
Reported-by: Frank Ramsay <framsay@redhat.com>
Tested-by: Prarit Bhargava <prarit@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Reviewed-by: Prarit Bhargava <prarit@redhat.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20160815101700.GA30090@krava
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-08-15 10:17:00 +00:00
|
|
|
static unsigned int logical_packages __read_mostly;
|
2016-02-22 22:19:15 +00:00
|
|
|
|
2016-05-20 00:09:55 +00:00
|
|
|
/* Maximum number of SMT threads on any online core */
|
2017-12-04 16:45:21 +00:00
|
|
|
int __read_mostly __max_smt_threads = 1;
|
2016-05-20 00:09:55 +00:00
|
|
|
|
2016-11-22 20:23:54 +00:00
|
|
|
/* Flag to indicate if a complete sched domain rebuild is required */
|
|
|
|
bool x86_topology_update;
|
|
|
|
|
|
|
|
int arch_update_cpu_topology(void)
|
|
|
|
{
|
|
|
|
int retval = x86_topology_update;
|
|
|
|
|
|
|
|
x86_topology_update = false;
|
|
|
|
return retval;
|
|
|
|
}
|
|
|
|
|
2015-01-15 21:22:29 +00:00
|
|
|
static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
|
|
|
|
{
|
|
|
|
unsigned long flags;
|
|
|
|
|
|
|
|
spin_lock_irqsave(&rtc_lock, flags);
|
|
|
|
CMOS_WRITE(0xa, 0xf);
|
|
|
|
spin_unlock_irqrestore(&rtc_lock, flags);
|
|
|
|
*((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_HIGH)) =
|
|
|
|
start_eip >> 4;
|
|
|
|
*((volatile unsigned short *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) =
|
|
|
|
start_eip & 0xf;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void smpboot_restore_warm_reset_vector(void)
|
|
|
|
{
|
|
|
|
unsigned long flags;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Paranoid: Set warm reset code and vector here back
|
|
|
|
* to default values.
|
|
|
|
*/
|
|
|
|
spin_lock_irqsave(&rtc_lock, flags);
|
|
|
|
CMOS_WRITE(0, 0xf);
|
|
|
|
spin_unlock_irqrestore(&rtc_lock, flags);
|
|
|
|
|
|
|
|
*((volatile u32 *)phys_to_virt(TRAMPOLINE_PHYS_LOW)) = 0;
|
|
|
|
}
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
2012-11-13 19:32:41 +00:00
|
|
|
* Report back to the Boot Processor during boot time or to the caller processor
|
|
|
|
* during CPU online.
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static void smp_callin(void)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
|
|
|
int cpuid, phys_id;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If waken up by an INIT in an 82489DX configuration
|
2015-08-16 15:45:48 +00:00
|
|
|
* cpu_callout_mask guarantees we don't get here before
|
|
|
|
* an INIT_deassert IPI reaches our local APIC, so it is
|
|
|
|
* now safe to touch our local APIC.
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
2012-11-14 12:36:53 +00:00
|
|
|
cpuid = smp_processor_id();
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* (This works even if the APIC is not enabled.)
|
|
|
|
*/
|
2008-07-12 01:44:16 +00:00
|
|
|
phys_id = read_apic_id();
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* the boot CPU has finished the init stage and is spinning
|
|
|
|
* on callin_map until we finish. We are free to set up this
|
|
|
|
* CPU, first the APIC. (this is probably redundant on most
|
|
|
|
* boards)
|
|
|
|
*/
|
2015-01-15 21:22:40 +00:00
|
|
|
apic_ap_setup();
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2011-11-15 23:33:56 +00:00
|
|
|
/*
|
|
|
|
* Save our processor parameters. Note: this information
|
|
|
|
* is needed for clock calibration.
|
|
|
|
*/
|
|
|
|
smp_store_cpu_info(cpuid);
|
|
|
|
|
2017-10-28 00:11:00 +00:00
|
|
|
/*
|
|
|
|
* The topology information must be up to date before
|
|
|
|
* calibrate_delay() and notify_cpu_starting().
|
|
|
|
*/
|
|
|
|
set_cpu_sibling_map(raw_smp_processor_id());
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
|
|
|
* Get our bogomips.
|
2011-11-15 23:33:56 +00:00
|
|
|
* Update loops_per_jiffy in cpu_data. Previous call to
|
|
|
|
* smp_store_cpu_info() stored a value that is close but not as
|
|
|
|
* accurate as the value just calculated.
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
|
|
|
calibrate_delay();
|
2011-11-15 23:33:56 +00:00
|
|
|
cpu_data(cpuid).loops_per_jiffy = loops_per_jiffy;
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Stack at about %p\n", &cpuid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2010-11-18 10:47:31 +00:00
|
|
|
wmb();
|
|
|
|
|
2010-03-23 18:30:52 +00:00
|
|
|
notify_cpu_starting(cpuid);
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
|
|
|
* Allow the master to continue.
|
|
|
|
*/
|
2009-01-04 13:18:03 +00:00
|
|
|
cpumask_set_cpu(cpuid, cpu_callin_mask);
|
2008-03-19 17:25:59 +00:00
|
|
|
}
|
|
|
|
|
2012-11-14 12:36:53 +00:00
|
|
|
static int cpu0_logical_apicid;
|
|
|
|
static int enable_start_cpu0;
|
2008-03-19 17:26:00 +00:00
|
|
|
/*
|
|
|
|
* Activate a secondary processor.
|
|
|
|
*/
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static void notrace start_secondary(void *unused)
|
2008-03-19 17:26:00 +00:00
|
|
|
{
|
|
|
|
/*
|
2017-09-11 00:48:27 +00:00
|
|
|
* Don't put *anything* except direct CPU state initialization
|
|
|
|
* before cpu_init(), SMP booting is too fragile that we want to
|
|
|
|
* limit the things done here to the most necessary things.
|
2008-03-19 17:26:00 +00:00
|
|
|
*/
|
2017-09-11 00:48:27 +00:00
|
|
|
if (boot_cpu_has(X86_FEATURE_PCID))
|
|
|
|
__write_cr4(__read_cr4() | X86_CR4_PCIDE);
|
2012-11-14 12:36:53 +00:00
|
|
|
|
x86-32: Separate 1:1 pagetables from swapper_pg_dir
This patch fixes machine crashes which occur when heavily exercising the
CPU hotplug codepaths on a 32-bit kernel. These crashes are caused by
AMD Erratum 383 and result in a fatal machine check exception. Here's
the scenario:
1. On 32-bit, the swapper_pg_dir page table is used as the initial page
table for booting a secondary CPU.
2. To make this work, swapper_pg_dir needs a direct mapping of physical
memory in it (the low mappings). By adding those low, large page (2M)
mappings (PAE kernel), we create the necessary conditions for Erratum
383 to occur.
3. Other CPUs which do not participate in the off- and onlining game may
use swapper_pg_dir while the low mappings are present (when leave_mm is
called). For all steps below, the CPU referred to is a CPU that is using
swapper_pg_dir, and not the CPU which is being onlined.
4. The presence of the low mappings in swapper_pg_dir can result
in TLB entries for addresses below __PAGE_OFFSET to be established
speculatively. These TLB entries are marked global and large.
5. When the CPU with such TLB entry switches to another page table, this
TLB entry remains because it is global.
6. The process then generates an access to an address covered by the
above TLB entry but there is a permission mismatch - the TLB entry
covers a large global page not accessible to userspace.
7. Due to this permission mismatch a new 4kb, user TLB entry gets
established. Further, Erratum 383 provides for a small window of time
where both TLB entries are present. This results in an uncorrectable
machine check exception signalling a TLB multimatch which panics the
machine.
There are two ways to fix this issue:
1. Always do a global TLB flush when a new cr3 is loaded and the
old page table was swapper_pg_dir. I consider this a hack hard
to understand and with performance implications
2. Do not use swapper_pg_dir to boot secondary CPUs like 64-bit
does.
This patch implements solution 2. It introduces a trampoline_pg_dir
which has the same layout as swapper_pg_dir with low_mappings. This page
table is used as the initial page table of the booting CPU. Later in the
bringup process, it switches to swapper_pg_dir and does a global TLB
flush. This fixes the crashes in our test cases.
-v2: switch to swapper_pg_dir right after entering start_secondary() so
that we are able to access percpu data which might not be mapped in the
trampoline page table.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
LKML-Reference: <20100816123833.GB28147@aftab>
Signed-off-by: Borislav Petkov <borislav.petkov@amd.com>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2010-08-16 12:38:33 +00:00
|
|
|
#ifdef CONFIG_X86_32
|
2010-08-28 13:58:33 +00:00
|
|
|
/* switch away from the initial page table */
|
x86-32: Separate 1:1 pagetables from swapper_pg_dir
This patch fixes machine crashes which occur when heavily exercising the
CPU hotplug codepaths on a 32-bit kernel. These crashes are caused by
AMD Erratum 383 and result in a fatal machine check exception. Here's
the scenario:
1. On 32-bit, the swapper_pg_dir page table is used as the initial page
table for booting a secondary CPU.
2. To make this work, swapper_pg_dir needs a direct mapping of physical
memory in it (the low mappings). By adding those low, large page (2M)
mappings (PAE kernel), we create the necessary conditions for Erratum
383 to occur.
3. Other CPUs which do not participate in the off- and onlining game may
use swapper_pg_dir while the low mappings are present (when leave_mm is
called). For all steps below, the CPU referred to is a CPU that is using
swapper_pg_dir, and not the CPU which is being onlined.
4. The presence of the low mappings in swapper_pg_dir can result
in TLB entries for addresses below __PAGE_OFFSET to be established
speculatively. These TLB entries are marked global and large.
5. When the CPU with such TLB entry switches to another page table, this
TLB entry remains because it is global.
6. The process then generates an access to an address covered by the
above TLB entry but there is a permission mismatch - the TLB entry
covers a large global page not accessible to userspace.
7. Due to this permission mismatch a new 4kb, user TLB entry gets
established. Further, Erratum 383 provides for a small window of time
where both TLB entries are present. This results in an uncorrectable
machine check exception signalling a TLB multimatch which panics the
machine.
There are two ways to fix this issue:
1. Always do a global TLB flush when a new cr3 is loaded and the
old page table was swapper_pg_dir. I consider this a hack hard
to understand and with performance implications
2. Do not use swapper_pg_dir to boot secondary CPUs like 64-bit
does.
This patch implements solution 2. It introduces a trampoline_pg_dir
which has the same layout as swapper_pg_dir with low_mappings. This page
table is used as the initial page table of the booting CPU. Later in the
bringup process, it switches to swapper_pg_dir and does a global TLB
flush. This fixes the crashes in our test cases.
-v2: switch to swapper_pg_dir right after entering start_secondary() so
that we are able to access percpu data which might not be mapped in the
trampoline page table.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
LKML-Reference: <20100816123833.GB28147@aftab>
Signed-off-by: Borislav Petkov <borislav.petkov@amd.com>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2010-08-16 12:38:33 +00:00
|
|
|
load_cr3(swapper_pg_dir);
|
|
|
|
__flush_tlb_all();
|
|
|
|
#endif
|
2017-11-27 14:21:39 +00:00
|
|
|
load_current_idt();
|
2017-09-17 16:03:51 +00:00
|
|
|
cpu_init();
|
|
|
|
x86_cpuinit.early_percpu_clock_init();
|
|
|
|
preempt_disable();
|
|
|
|
smp_callin();
|
|
|
|
|
|
|
|
enable_start_cpu0 = 0;
|
|
|
|
|
2008-03-19 17:26:00 +00:00
|
|
|
/* otherwise gcc will move up smp_processor_id before the cpu_init */
|
|
|
|
barrier();
|
|
|
|
/*
|
2017-10-03 09:47:27 +00:00
|
|
|
* Check TSC synchronization with the boot CPU:
|
2008-03-19 17:26:00 +00:00
|
|
|
*/
|
|
|
|
check_tsc_sync_target();
|
|
|
|
|
|
|
|
/*
|
2017-09-13 21:29:40 +00:00
|
|
|
* Lock vector_lock, set CPU online and bring the vector
|
|
|
|
* allocator online. Online must be set with vector_lock held
|
|
|
|
* to prevent a concurrent irq setup/teardown from seeing a
|
|
|
|
* half valid vector space.
|
2008-03-19 17:26:00 +00:00
|
|
|
*/
|
2008-08-09 22:09:02 +00:00
|
|
|
lock_vector_lock();
|
2009-01-04 13:18:03 +00:00
|
|
|
set_cpu_online(smp_processor_id(), true);
|
2017-09-13 21:29:40 +00:00
|
|
|
lapic_online();
|
2008-08-09 22:09:02 +00:00
|
|
|
unlock_vector_lock();
|
2015-02-25 19:42:15 +00:00
|
|
|
cpu_set_state_online(smp_processor_id());
|
2010-02-26 16:49:12 +00:00
|
|
|
x86_platform.nmi_init();
|
2008-03-19 17:26:00 +00:00
|
|
|
|
2008-09-07 09:29:58 +00:00
|
|
|
/* enable local interrupts */
|
|
|
|
local_irq_enable();
|
|
|
|
|
2009-09-17 14:36:43 +00:00
|
|
|
/* to prevent fake stack check failure in clock setup */
|
|
|
|
boot_init_stack_canary();
|
2008-09-07 09:29:58 +00:00
|
|
|
|
2009-08-19 10:35:53 +00:00
|
|
|
x86_cpuinit.setup_percpu_clockev();
|
2008-03-19 17:26:00 +00:00
|
|
|
|
|
|
|
wmb();
|
2016-02-26 18:43:40 +00:00
|
|
|
cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
|
2008-03-19 17:26:00 +00:00
|
|
|
}
|
|
|
|
|
2017-11-14 12:42:56 +00:00
|
|
|
/**
|
|
|
|
* topology_phys_to_logical_pkg - Map a physical package id to a logical
|
|
|
|
*
|
|
|
|
* Returns logical package id or -1 if not found
|
|
|
|
*/
|
|
|
|
int topology_phys_to_logical_pkg(unsigned int phys_pkg)
|
|
|
|
{
|
|
|
|
int cpu;
|
|
|
|
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
struct cpuinfo_x86 *c = &cpu_data(cpu);
|
|
|
|
|
|
|
|
if (c->initialized && c->phys_proc_id == phys_pkg)
|
|
|
|
return c->logical_proc_id;
|
|
|
|
}
|
|
|
|
return -1;
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL(topology_phys_to_logical_pkg);
|
|
|
|
|
2016-12-12 10:04:53 +00:00
|
|
|
/**
|
|
|
|
* topology_update_package_map - Update the physical to logical package map
|
|
|
|
* @pkg: The physical package id as retrieved via CPUID
|
|
|
|
* @cpu: The cpu for which this is updated
|
|
|
|
*/
|
|
|
|
int topology_update_package_map(unsigned int pkg, unsigned int cpu)
|
2016-02-22 22:19:15 +00:00
|
|
|
{
|
2017-11-14 12:42:56 +00:00
|
|
|
int new;
|
2016-02-22 22:19:15 +00:00
|
|
|
|
2017-11-14 12:42:56 +00:00
|
|
|
/* Already available somewhere? */
|
|
|
|
new = topology_phys_to_logical_pkg(pkg);
|
|
|
|
if (new >= 0)
|
2016-02-22 22:19:15 +00:00
|
|
|
goto found;
|
|
|
|
|
x86/smp: Fix __max_logical_packages value setup
Frank reported kernel panic when he disabled several cores in BIOS
via following option:
Core Disable Bitmap(Hex) [0]
with number 0xFFE, which leaves 16 CPUs in system (out of 48).
The kernel panic below goes along with following messages:
smpboot: Max logical packages: 2^M
smpboot: APIC(0) Converting physical 0 to logical package 0^M
smpboot: APIC(20) Converting physical 1 to logical package 1^M
smpboot: APIC(40) Package 2 exceeds logical package map^M
smpboot: CPU 8 APICId 40 disabled^M
smpboot: APIC(60) Package 3 exceeds logical package map^M
smpboot: CPU 12 APICId 60 disabled^M
...
general protection fault: 0000 [#1] SMP^M
Modules linked in:^M
CPU: 15 PID: 1 Comm: swapper/0 Not tainted 4.7.0-rc5+ #1^M
Hardware name: SGI UV300/UV300, BIOS SGI UV 300 series BIOS 05/25/2016^M
task: ffff8801673e0000 ti: ffff8801673ac000 task.ti: ffff8801673ac000^M
RIP: 0010:[<ffffffff81014d54>] [<ffffffff81014d54>] uncore_change_context+0xd4/0x180^M
...
[<ffffffff810158ac>] uncore_event_init_cpu+0x6c/0x70^M
[<ffffffff81d8c91c>] intel_uncore_init+0x1c2/0x2dd^M
[<ffffffff81d8c75a>] ? uncore_cpu_setup+0x17/0x17^M
[<ffffffff81002190>] do_one_initcall+0x50/0x190^M
[<ffffffff810ab193>] ? parse_args+0x293/0x480^M
[<ffffffff81d87365>] kernel_init_freeable+0x1a5/0x249^M
[<ffffffff81d86a35>] ? set_debug_rodata+0x12/0x12^M
[<ffffffff816dc19e>] kernel_init+0xe/0x110^M
[<ffffffff816e93bf>] ret_from_fork+0x1f/0x40^M
[<ffffffff816dc190>] ? rest_init+0x80/0x80^M
The reason for the panic is wrong value of __max_logical_packages,
which lets logical_package_map uninitialized and the uncore code
relying on this map being properly initialized (maybe we should
add some safety checks there as well).
The __max_logical_packages is computed as:
DIV_ROUND_UP(total_cpus, ncpus);
- ncpus being number of cores
With above BIOS setup we get total_cpus == 16 which set
__max_logical_packages to 2 (ncpus is 12).
Once topology_update_package_map processes CPU with logical
pkg over 2 we display above messages and fail to initialize
the physical_to_logical_pkg map, which makes the uncore code
crash.
The fix is to remove logical_package_map bitmap completely
and keep and update the logical_packages number instead.
After we enumerate all the present CPUs, we check if the
enumerated logical packages count is within its computed
maximum from BIOS data.
If it's not the case, we set this maximum to the new enumerated
value and freeze any new addition of logical packages.
The freeze is because lot of init code like uncore/rapl/cqm
depends on having maximum logical package value set to allocate
their data, so we can't change it later on.
Prarit Bhargava tested the patch and confirms that it solves
the problem:
From dmidecode:
Core Count: 24
Core Enabled: 24
Thread Count: 48
Orig kernel boot log:
[ 0.464981] smpboot: Max logical packages: 19
[ 0.469861] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.477261] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.484760] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.492258] smpboot: APIC(c0) Converting physical 3 to logical package 3
1. nr_cpus=8, should stop enumerating in package 0:
[ 0.533664] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.539596] smpboot: Max logical packages: 19
2. max_cpus=8, should still enumerate all packages:
[ 0.526494] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.532428] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.538456] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.544486] smpboot: APIC(c0) Converting physical 3 to logical package 3
[ 0.550524] smpboot: Max logical packages: 19
3. nr_cpus=49 ( 2 socket + 1 core on 3rd socket), should stop enumerating in
package 2:
[ 0.521378] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.527314] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.533345] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.539368] smpboot: Max logical packages: 19
4. maxcpus=49, should still enumerate all packages:
[ 0.525591] smpboot: APIC(0) Converting physical 0 to logical package 0
[ 0.531525] smpboot: APIC(40) Converting physical 1 to logical package 1
[ 0.537547] smpboot: APIC(80) Converting physical 2 to logical package 2
[ 0.543579] smpboot: APIC(c0) Converting physical 3 to logical package 3
[ 0.549624] smpboot: Max logical packages: 19
5. kdump (nr_cpus=1) works as well.
Reported-by: Frank Ramsay <framsay@redhat.com>
Tested-by: Prarit Bhargava <prarit@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Reviewed-by: Prarit Bhargava <prarit@redhat.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20160815101700.GA30090@krava
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-08-15 10:17:00 +00:00
|
|
|
new = logical_packages++;
|
2016-12-12 10:04:53 +00:00
|
|
|
if (new != pkg) {
|
|
|
|
pr_info("CPU %u Converting physical %u to logical package %u\n",
|
|
|
|
cpu, pkg, new);
|
|
|
|
}
|
2016-02-22 22:19:15 +00:00
|
|
|
found:
|
2017-11-14 12:42:56 +00:00
|
|
|
cpu_data(cpu).logical_proc_id = new;
|
2016-02-22 22:19:15 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-11-13 19:32:41 +00:00
|
|
|
void __init smp_store_boot_cpu_info(void)
|
|
|
|
{
|
|
|
|
int id = 0; /* CPU 0 */
|
|
|
|
struct cpuinfo_x86 *c = &cpu_data(id);
|
|
|
|
|
|
|
|
*c = boot_cpu_data;
|
|
|
|
c->cpu_index = id;
|
2017-11-14 12:42:57 +00:00
|
|
|
topology_update_package_map(c->phys_proc_id, id);
|
2017-11-14 12:42:56 +00:00
|
|
|
c->initialized = true;
|
2012-11-13 19:32:41 +00:00
|
|
|
}
|
|
|
|
|
2008-03-19 17:25:05 +00:00
|
|
|
/*
|
|
|
|
* The bootstrap kernel entry code has set these up. Save them for
|
|
|
|
* a given CPU
|
|
|
|
*/
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
void smp_store_cpu_info(int id)
|
2008-03-19 17:25:05 +00:00
|
|
|
{
|
|
|
|
struct cpuinfo_x86 *c = &cpu_data(id);
|
|
|
|
|
2017-11-14 12:42:56 +00:00
|
|
|
/* Copy boot_cpu_data only on the first bringup */
|
|
|
|
if (!c->initialized)
|
|
|
|
*c = boot_cpu_data;
|
2008-03-19 17:25:05 +00:00
|
|
|
c->cpu_index = id;
|
2012-11-13 19:32:41 +00:00
|
|
|
/*
|
|
|
|
* During boot time, CPU0 has this setup already. Save the info when
|
|
|
|
* bringing up AP or offlined CPU0.
|
|
|
|
*/
|
|
|
|
identify_secondary_cpu(c);
|
2017-11-14 12:42:56 +00:00
|
|
|
c->initialized = true;
|
2008-03-19 17:25:05 +00:00
|
|
|
}
|
|
|
|
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
static bool
|
|
|
|
topology_same_node(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
|
|
|
|
{
|
|
|
|
int cpu1 = c->cpu_index, cpu2 = o->cpu_index;
|
|
|
|
|
|
|
|
return (cpu_to_node(cpu1) == cpu_to_node(cpu2));
|
|
|
|
}
|
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static bool
|
2012-05-11 11:05:59 +00:00
|
|
|
topology_sane(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o, const char *name)
|
2010-09-30 12:41:56 +00:00
|
|
|
{
|
2012-05-11 11:05:59 +00:00
|
|
|
int cpu1 = c->cpu_index, cpu2 = o->cpu_index;
|
|
|
|
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
return !WARN_ONCE(!topology_same_node(c, o),
|
2012-05-11 11:05:59 +00:00
|
|
|
"sched: CPU #%d's %s-sibling CPU #%d is not on the same node! "
|
|
|
|
"[node: %d != %d]. Ignoring dependency.\n",
|
|
|
|
cpu1, name, cpu2, cpu_to_node(cpu1), cpu_to_node(cpu2));
|
|
|
|
}
|
|
|
|
|
2015-05-26 13:11:35 +00:00
|
|
|
#define link_mask(mfunc, c1, c2) \
|
2012-05-11 11:05:59 +00:00
|
|
|
do { \
|
2015-05-26 13:11:35 +00:00
|
|
|
cpumask_set_cpu((c1), mfunc(c2)); \
|
|
|
|
cpumask_set_cpu((c2), mfunc(c1)); \
|
2012-05-11 11:05:59 +00:00
|
|
|
} while (0)
|
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static bool match_smt(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
|
2012-05-11 11:05:59 +00:00
|
|
|
{
|
2015-12-07 09:39:41 +00:00
|
|
|
if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
|
2012-05-11 11:05:59 +00:00
|
|
|
int cpu1 = c->cpu_index, cpu2 = o->cpu_index;
|
|
|
|
|
|
|
|
if (c->phys_proc_id == o->phys_proc_id &&
|
2017-02-05 10:50:21 +00:00
|
|
|
per_cpu(cpu_llc_id, cpu1) == per_cpu(cpu_llc_id, cpu2)) {
|
|
|
|
if (c->cpu_core_id == o->cpu_core_id)
|
|
|
|
return topology_sane(c, o, "smt");
|
|
|
|
|
|
|
|
if ((c->cu_id != 0xff) &&
|
|
|
|
(o->cu_id != 0xff) &&
|
|
|
|
(c->cu_id == o->cu_id))
|
|
|
|
return topology_sane(c, o, "smt");
|
|
|
|
}
|
2012-05-11 11:05:59 +00:00
|
|
|
|
|
|
|
} else if (c->phys_proc_id == o->phys_proc_id &&
|
|
|
|
c->cpu_core_id == o->cpu_core_id) {
|
|
|
|
return topology_sane(c, o, "smt");
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static bool match_llc(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
|
2012-05-11 11:05:59 +00:00
|
|
|
{
|
|
|
|
int cpu1 = c->cpu_index, cpu2 = o->cpu_index;
|
|
|
|
|
|
|
|
if (per_cpu(cpu_llc_id, cpu1) != BAD_APICID &&
|
|
|
|
per_cpu(cpu_llc_id, cpu1) == per_cpu(cpu_llc_id, cpu2))
|
|
|
|
return topology_sane(c, o, "llc");
|
|
|
|
|
|
|
|
return false;
|
2010-09-30 12:41:56 +00:00
|
|
|
}
|
|
|
|
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
/*
|
|
|
|
* Unlike the other levels, we do not enforce keeping a
|
|
|
|
* multicore group inside a NUMA node. If this happens, we will
|
|
|
|
* discard the MC level of the topology later.
|
|
|
|
*/
|
|
|
|
static bool match_die(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
|
2012-05-11 11:05:59 +00:00
|
|
|
{
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
if (c->phys_proc_id == o->phys_proc_id)
|
|
|
|
return true;
|
2012-05-11 11:05:59 +00:00
|
|
|
return false;
|
|
|
|
}
|
2008-03-19 17:25:05 +00:00
|
|
|
|
2016-11-22 20:23:57 +00:00
|
|
|
#if defined(CONFIG_SCHED_SMT) || defined(CONFIG_SCHED_MC)
|
|
|
|
static inline int x86_sched_itmt_flags(void)
|
|
|
|
{
|
|
|
|
return sysctl_sched_itmt_enabled ? SD_ASYM_PACKING : 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef CONFIG_SCHED_MC
|
|
|
|
static int x86_core_flags(void)
|
|
|
|
{
|
|
|
|
return cpu_core_flags() | x86_sched_itmt_flags();
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_SCHED_SMT
|
|
|
|
static int x86_smt_flags(void)
|
|
|
|
{
|
|
|
|
return cpu_smt_flags() | x86_sched_itmt_flags();
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
#endif
|
|
|
|
|
2016-09-21 19:19:03 +00:00
|
|
|
static struct sched_domain_topology_level x86_numa_in_package_topology[] = {
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
#ifdef CONFIG_SCHED_SMT
|
2016-11-22 20:23:57 +00:00
|
|
|
{ cpu_smt_mask, x86_smt_flags, SD_INIT_NAME(SMT) },
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_SCHED_MC
|
2016-11-22 20:23:57 +00:00
|
|
|
{ cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC) },
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
#endif
|
|
|
|
{ NULL, },
|
|
|
|
};
|
2016-09-21 19:19:03 +00:00
|
|
|
|
|
|
|
static struct sched_domain_topology_level x86_topology[] = {
|
|
|
|
#ifdef CONFIG_SCHED_SMT
|
2016-11-22 20:23:57 +00:00
|
|
|
{ cpu_smt_mask, x86_smt_flags, SD_INIT_NAME(SMT) },
|
2016-09-21 19:19:03 +00:00
|
|
|
#endif
|
|
|
|
#ifdef CONFIG_SCHED_MC
|
2016-11-22 20:23:57 +00:00
|
|
|
{ cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC) },
|
2016-09-21 19:19:03 +00:00
|
|
|
#endif
|
|
|
|
{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
|
|
|
|
{ NULL, },
|
|
|
|
};
|
|
|
|
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
/*
|
2016-09-21 19:19:03 +00:00
|
|
|
* Set if a package/die has multiple NUMA nodes inside.
|
|
|
|
* AMD Magny-Cours and Intel Cluster-on-Die have this.
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
*/
|
2016-09-21 19:19:03 +00:00
|
|
|
static bool x86_has_numa_in_package;
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
void set_cpu_sibling_map(int cpu)
|
2008-03-03 17:13:02 +00:00
|
|
|
{
|
2012-05-11 11:05:59 +00:00
|
|
|
bool has_smt = smp_num_siblings > 1;
|
2013-05-29 12:48:15 +00:00
|
|
|
bool has_mp = has_smt || boot_cpu_data.x86_max_cores > 1;
|
2008-03-03 17:13:02 +00:00
|
|
|
struct cpuinfo_x86 *c = &cpu_data(cpu);
|
2012-05-11 11:05:59 +00:00
|
|
|
struct cpuinfo_x86 *o;
|
2016-05-20 00:09:55 +00:00
|
|
|
int i, threads;
|
2008-03-03 17:13:02 +00:00
|
|
|
|
2009-01-04 13:18:03 +00:00
|
|
|
cpumask_set_cpu(cpu, cpu_sibling_setup_mask);
|
2008-03-03 17:13:02 +00:00
|
|
|
|
2013-05-29 12:48:15 +00:00
|
|
|
if (!has_mp) {
|
2015-05-26 13:11:35 +00:00
|
|
|
cpumask_set_cpu(cpu, topology_sibling_cpumask(cpu));
|
2012-05-11 11:05:59 +00:00
|
|
|
cpumask_set_cpu(cpu, cpu_llc_shared_mask(cpu));
|
2015-05-26 13:11:35 +00:00
|
|
|
cpumask_set_cpu(cpu, topology_core_cpumask(cpu));
|
2008-03-03 17:13:02 +00:00
|
|
|
c->booted_cores = 1;
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2009-01-04 13:18:03 +00:00
|
|
|
for_each_cpu(i, cpu_sibling_setup_mask) {
|
2012-05-11 11:05:59 +00:00
|
|
|
o = &cpu_data(i);
|
|
|
|
|
|
|
|
if ((i == cpu) || (has_smt && match_smt(c, o)))
|
2015-05-26 13:11:35 +00:00
|
|
|
link_mask(topology_sibling_cpumask, cpu, i);
|
2012-05-11 11:05:59 +00:00
|
|
|
|
2013-05-29 12:48:15 +00:00
|
|
|
if ((i == cpu) || (has_mp && match_llc(c, o)))
|
2015-05-26 13:11:35 +00:00
|
|
|
link_mask(cpu_llc_shared_mask, cpu, i);
|
2012-05-11 11:05:59 +00:00
|
|
|
|
2012-05-31 07:37:38 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* This needs a separate iteration over the cpus because we rely on all
|
2015-05-26 13:11:35 +00:00
|
|
|
* topology_sibling_cpumask links to be set-up.
|
2012-05-31 07:37:38 +00:00
|
|
|
*/
|
|
|
|
for_each_cpu(i, cpu_sibling_setup_mask) {
|
|
|
|
o = &cpu_data(i);
|
|
|
|
|
x86, sched: Add new topology for multi-NUMA-node CPUs
I'm getting the spew below when booting with Haswell (Xeon
E5-2699 v3) CPUs and the "Cluster-on-Die" (CoD) feature enabled
in the BIOS. It seems similar to the issue that some folks from
AMD ran in to on their systems and addressed in this commit:
161270fc1f9d ("x86/smp: Fix topology checks on AMD MCM CPUs")
Both these Intel and AMD systems break an assumption which is
being enforced by topology_sane(): a socket may not contain more
than one NUMA node.
AMD special-cased their system by looking for a cpuid flag. The
Intel mode is dependent on BIOS options and I do not know of a
way which it is enumerated other than the tables being parsed
during the CPU bringup process. In other words, we have to trust
the ACPI tables <shudder>.
This detects the situation where a NUMA node occurs at a place in
the middle of the "CPU" sched domains. It replaces the default
topology with one that relies on the NUMA information from the
firmware (SRAT table) for all levels of sched domains above the
hyperthreads.
This also fixes a sysfs bug. We used to freak out when we saw
the "mc" group cross a node boundary, so we stopped building the
MC group. MC gets exported as the 'core_siblings_list' in
/sys/devices/system/cpu/cpu*/topology/ and this caused CPUs with
the same 'physical_package_id' to not be listed together in
'core_siblings_list'. This violates a statement from
Documentation/ABI/testing/sysfs-devices-system-cpu:
core_siblings: internal kernel map of cpu#'s hardware threads
within the same physical_package_id.
core_siblings_list: human-readable list of the logical CPU
numbers within the same physical_package_id as cpu#.
The sysfs effects here cause an issue with the hwloc tool where
it gets confused and thinks there are more sockets than are
physically present.
Before this patch, there are two packages:
# cd /sys/devices/system/cpu/
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
But 4 _sets_ of core siblings:
# cat cpu*/topology/core_siblings_list | sort | uniq -c
9 0-8
9 18-26
9 27-35
9 9-17
After this set, there are only 2 sets of core siblings, which
is what we expect for a 2-socket system.
# cat cpu*/topology/physical_package_id | sort | uniq -c
18 0
18 1
# cat cpu*/topology/core_siblings_list | sort | uniq -c
18 0-17
18 18-35
Example spew:
...
NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
#2 #3 #4 #5 #6 #7 #8
.... node #1, CPUs: #9
------------[ cut here ]------------
WARNING: CPU: 9 PID: 0 at /home/ak/hle/linux-hle-2.6/arch/x86/kernel/smpboot.c:306 topology_sane.isra.2+0x74/0x90()
sched: CPU #9's mc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
Modules linked in:
CPU: 9 PID: 0 Comm: swapper/9 Not tainted 3.17.0-rc1-00293-g8e01c4d-dirty #631
Hardware name: Intel Corporation S2600WTT/S2600WTT, BIOS GRNDSDP1.86B.0036.R05.1407140519 07/14/2014
0000000000000009 ffff88046ddabe00 ffffffff8172e485 ffff88046ddabe48
ffff88046ddabe38 ffffffff8109691d 000000000000b001 0000000000000009
ffff88086fc12580 000000000000b020 0000000000000009 ffff88046ddabe98
Call Trace:
[<ffffffff8172e485>] dump_stack+0x45/0x56
[<ffffffff8109691d>] warn_slowpath_common+0x7d/0xa0
[<ffffffff8109698c>] warn_slowpath_fmt+0x4c/0x50
[<ffffffff81074f94>] topology_sane.isra.2+0x74/0x90
[<ffffffff8107530e>] set_cpu_sibling_map+0x31e/0x4f0
[<ffffffff8107568d>] start_secondary+0x1ad/0x240
---[ end trace 3fe5f587a9fcde61 ]---
#10 #11 #12 #13 #14 #15 #16 #17
.... node #2, CPUs: #18 #19 #20 #21 #22 #23 #24 #25 #26
.... node #3, CPUs: #27 #28 #29 #30 #31 #32 #33 #34 #35
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
[ Added LLC domain and s/match_mc/match_die/ ]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: brice.goglin@gmail.com
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/20140918193334.C065EBCE@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-18 19:33:34 +00:00
|
|
|
if ((i == cpu) || (has_mp && match_die(c, o))) {
|
2015-05-26 13:11:35 +00:00
|
|
|
link_mask(topology_core_cpumask, cpu, i);
|
2012-05-11 11:05:59 +00:00
|
|
|
|
2008-03-03 17:13:02 +00:00
|
|
|
/*
|
|
|
|
* Does this new cpu bringup a new core?
|
|
|
|
*/
|
2015-05-26 13:11:35 +00:00
|
|
|
if (cpumask_weight(
|
|
|
|
topology_sibling_cpumask(cpu)) == 1) {
|
2008-03-03 17:13:02 +00:00
|
|
|
/*
|
|
|
|
* for each core in package, increment
|
|
|
|
* the booted_cores for this new cpu
|
|
|
|
*/
|
2015-05-26 13:11:35 +00:00
|
|
|
if (cpumask_first(
|
|
|
|
topology_sibling_cpumask(i)) == i)
|
2008-03-03 17:13:02 +00:00
|
|
|
c->booted_cores++;
|
|
|
|
/*
|
|
|
|
* increment the core count for all
|
|
|
|
* the other cpus in this package
|
|
|
|
*/
|
|
|
|
if (i != cpu)
|
|
|
|
cpu_data(i).booted_cores++;
|
|
|
|
} else if (i != cpu && !c->booted_cores)
|
|
|
|
c->booted_cores = cpu_data(i).booted_cores;
|
|
|
|
}
|
2014-09-30 21:45:46 +00:00
|
|
|
if (match_die(c, o) && !topology_same_node(c, o))
|
2016-09-21 19:19:03 +00:00
|
|
|
x86_has_numa_in_package = true;
|
2008-03-03 17:13:02 +00:00
|
|
|
}
|
2016-05-20 00:09:55 +00:00
|
|
|
|
|
|
|
threads = cpumask_weight(topology_sibling_cpumask(cpu));
|
|
|
|
if (threads > __max_smt_threads)
|
|
|
|
__max_smt_threads = threads;
|
2008-03-03 17:13:02 +00:00
|
|
|
}
|
|
|
|
|
2008-03-03 17:13:03 +00:00
|
|
|
/* maps the cpu to the sched domain representing multi-core */
|
2008-12-26 11:53:41 +00:00
|
|
|
const struct cpumask *cpu_coregroup_mask(int cpu)
|
2008-03-03 17:13:03 +00:00
|
|
|
{
|
2012-05-29 14:39:09 +00:00
|
|
|
return cpu_llc_shared_mask(cpu);
|
2008-12-26 11:53:41 +00:00
|
|
|
}
|
|
|
|
|
2008-04-23 11:20:56 +00:00
|
|
|
static void impress_friends(void)
|
2008-03-19 17:25:27 +00:00
|
|
|
{
|
|
|
|
int cpu;
|
|
|
|
unsigned long bogosum = 0;
|
|
|
|
/*
|
|
|
|
* Allow the user to impress friends.
|
|
|
|
*/
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Before bogomips\n");
|
2008-03-19 17:25:27 +00:00
|
|
|
for_each_possible_cpu(cpu)
|
2009-01-04 13:18:03 +00:00
|
|
|
if (cpumask_test_cpu(cpu, cpu_callout_mask))
|
2008-03-19 17:25:27 +00:00
|
|
|
bogosum += cpu_data(cpu).loops_per_jiffy;
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_info("Total of %d processors activated (%lu.%02lu BogoMIPS)\n",
|
2008-03-19 17:25:29 +00:00
|
|
|
num_online_cpus(),
|
2008-03-19 17:25:27 +00:00
|
|
|
bogosum/(500000/HZ),
|
|
|
|
(bogosum/(5000/HZ))%100);
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Before bogocount - setting activated=1\n");
|
2008-03-19 17:25:27 +00:00
|
|
|
}
|
|
|
|
|
x86: fix wakeup_cpu with numaq/es7000, v2
Impact: fix secondary-CPU wakeup/init path with numaq and es7000
While looking at wakeup_secondary_cpu for WAKE_SECONDARY_VIA_NMI:
|#ifdef WAKE_SECONDARY_VIA_NMI
|/*
| * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
| * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
| * won't ... remember to clear down the APIC, etc later.
| */
|static int __devinit
|wakeup_secondary_cpu(int logical_apicid, unsigned long start_eip)
|{
| unsigned long send_status, accept_status = 0;
| int maxlvt;
|...
| if (APIC_INTEGRATED(apic_version[phys_apicid])) {
| maxlvt = lapic_get_maxlvt();
I noticed that there is no warning about undefined phys_apicid...
because WAKE_SECONDARY_VIA_NMI and WAKE_SECONDARY_VIA_INIT can not be
defined at the same time. So NUMAQ is using wrong wakeup_secondary_cpu.
WAKE_SECONDARY_VIA_NMI, WAKE_SECONDARY_VIA_INIT and
WAKE_SECONDARY_VIA_MIP are variants of a weird and fragile
preprocessor-driven "HAL" mechanisms to specify the kind of secondary-CPU
wakeup strategy a given x86 kernel will use.
The vast majority of systems want to use INIT for secondary wakeup - NUMAQ
uses an NMI, (old-style-) ES7000 uses 'MIP' (a firmware driven in-memory
flag to let secondaries continue).
So convert these mechanisms to x86_quirks and add a
->wakeup_secondary_cpu() method to specify the rare exception
to the sane default.
Extend genapic accordingly as well, for 32-bit.
While looking further, I noticed that functions in wakecup.h for numaq
and es7000 are different to the default in mach_wakecpu.h - but smpboot.c
will only use default mach_wakecpu.h with smphook.h.
So we need to add mach_wakecpu.h for mach_generic, to properly support
numaq and es7000, and vectorize the following SMP init methods:
int trampoline_phys_low;
int trampoline_phys_high;
void (*wait_for_init_deassert)(atomic_t *deassert);
void (*smp_callin_clear_local_apic)(void);
void (*store_NMI_vector)(unsigned short *high, unsigned short *low);
void (*restore_NMI_vector)(unsigned short *high, unsigned short *low);
void (*inquire_remote_apic)(int apicid);
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-11-16 11:12:49 +00:00
|
|
|
void __inquire_remote_apic(int apicid)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
|
|
|
unsigned i, regs[] = { APIC_ID >> 4, APIC_LVR >> 4, APIC_SPIV >> 4 };
|
2011-07-01 01:10:53 +00:00
|
|
|
const char * const names[] = { "ID", "VERSION", "SPIV" };
|
2008-03-19 17:25:59 +00:00
|
|
|
int timeout;
|
|
|
|
u32 status;
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_info("Inquiring remote APIC 0x%x...\n", apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
for (i = 0; i < ARRAY_SIZE(regs); i++) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_info("... APIC 0x%x %s: ", apicid, names[i]);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Wait for idle.
|
|
|
|
*/
|
|
|
|
status = safe_apic_wait_icr_idle();
|
|
|
|
if (status)
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_cont("a previous APIC delivery may have failed\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-10 18:16:49 +00:00
|
|
|
apic_icr_write(APIC_DM_REMRD | regs[i], apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
timeout = 0;
|
|
|
|
do {
|
|
|
|
udelay(100);
|
|
|
|
status = apic_read(APIC_ICR) & APIC_ICR_RR_MASK;
|
|
|
|
} while (status == APIC_ICR_RR_INPROG && timeout++ < 1000);
|
|
|
|
|
|
|
|
switch (status) {
|
|
|
|
case APIC_ICR_RR_VALID:
|
|
|
|
status = apic_read(APIC_RRR);
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_cont("%08x\n", status);
|
2008-03-19 17:25:59 +00:00
|
|
|
break;
|
|
|
|
default:
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_cont("failed\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-05-11 21:27:09 +00:00
|
|
|
/*
|
|
|
|
* The Multiprocessor Specification 1.4 (1997) example code suggests
|
|
|
|
* that there should be a 10ms delay between the BSP asserting INIT
|
|
|
|
* and de-asserting INIT, when starting a remote processor.
|
|
|
|
* But that slows boot and resume on modern processors, which include
|
|
|
|
* many cores and don't require that delay.
|
|
|
|
*
|
|
|
|
* Cmdline "init_cpu_udelay=" is available to over-ride this delay.
|
2015-05-11 21:27:10 +00:00
|
|
|
* Modern processor families are quirked to remove the delay entirely.
|
2015-05-11 21:27:09 +00:00
|
|
|
*/
|
|
|
|
#define UDELAY_10MS_DEFAULT 10000
|
|
|
|
|
2015-11-22 23:16:15 +00:00
|
|
|
static unsigned int init_udelay = UINT_MAX;
|
2015-05-11 21:27:09 +00:00
|
|
|
|
|
|
|
static int __init cpu_init_udelay(char *str)
|
|
|
|
{
|
|
|
|
get_option(&str, &init_udelay);
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
early_param("cpu_init_udelay", cpu_init_udelay);
|
|
|
|
|
2015-05-11 21:27:10 +00:00
|
|
|
static void __init smp_quirk_init_udelay(void)
|
|
|
|
{
|
|
|
|
/* if cmdline changed it from default, leave it alone */
|
2015-11-22 23:16:15 +00:00
|
|
|
if (init_udelay != UINT_MAX)
|
2015-05-11 21:27:10 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
/* if modern processor, use no delay */
|
|
|
|
if (((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) && (boot_cpu_data.x86 == 6)) ||
|
2015-11-22 23:16:15 +00:00
|
|
|
((boot_cpu_data.x86_vendor == X86_VENDOR_AMD) && (boot_cpu_data.x86 >= 0xF))) {
|
2015-05-11 21:27:10 +00:00
|
|
|
init_udelay = 0;
|
2015-11-22 23:16:15 +00:00
|
|
|
return;
|
|
|
|
}
|
2015-10-16 04:14:28 +00:00
|
|
|
/* else, use legacy delay */
|
|
|
|
init_udelay = UDELAY_10MS_DEFAULT;
|
2015-05-11 21:27:10 +00:00
|
|
|
}
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
|
|
|
* Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
|
|
|
|
* INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
|
|
|
|
* won't ... remember to clear down the APIC, etc later.
|
|
|
|
*/
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
int
|
2012-11-14 12:36:53 +00:00
|
|
|
wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
|
|
|
unsigned long send_status, accept_status = 0;
|
|
|
|
int maxlvt;
|
|
|
|
|
|
|
|
/* Target chip */
|
|
|
|
/* Boot on the stack */
|
|
|
|
/* Kick the second */
|
2012-11-14 12:36:53 +00:00
|
|
|
apic_icr_write(APIC_DM_NMI | apic->dest_logical, apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Waiting for send to finish...\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
send_status = safe_apic_wait_icr_idle();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Give the other CPU some time to accept the IPI.
|
|
|
|
*/
|
|
|
|
udelay(200);
|
2016-09-13 18:12:32 +00:00
|
|
|
if (APIC_INTEGRATED(boot_cpu_apic_version)) {
|
2008-09-14 17:58:49 +00:00
|
|
|
maxlvt = lapic_get_maxlvt();
|
|
|
|
if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
|
|
|
|
apic_write(APIC_ESR, 0);
|
|
|
|
accept_status = (apic_read(APIC_ESR) & 0xEF);
|
|
|
|
}
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("NMI sent\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
if (send_status)
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_err("APIC never delivered???\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
if (accept_status)
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_err("APIC delivery error (%lx)\n", accept_status);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
return (send_status | accept_status);
|
|
|
|
}
|
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static int
|
x86: fix wakeup_cpu with numaq/es7000, v2
Impact: fix secondary-CPU wakeup/init path with numaq and es7000
While looking at wakeup_secondary_cpu for WAKE_SECONDARY_VIA_NMI:
|#ifdef WAKE_SECONDARY_VIA_NMI
|/*
| * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
| * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
| * won't ... remember to clear down the APIC, etc later.
| */
|static int __devinit
|wakeup_secondary_cpu(int logical_apicid, unsigned long start_eip)
|{
| unsigned long send_status, accept_status = 0;
| int maxlvt;
|...
| if (APIC_INTEGRATED(apic_version[phys_apicid])) {
| maxlvt = lapic_get_maxlvt();
I noticed that there is no warning about undefined phys_apicid...
because WAKE_SECONDARY_VIA_NMI and WAKE_SECONDARY_VIA_INIT can not be
defined at the same time. So NUMAQ is using wrong wakeup_secondary_cpu.
WAKE_SECONDARY_VIA_NMI, WAKE_SECONDARY_VIA_INIT and
WAKE_SECONDARY_VIA_MIP are variants of a weird and fragile
preprocessor-driven "HAL" mechanisms to specify the kind of secondary-CPU
wakeup strategy a given x86 kernel will use.
The vast majority of systems want to use INIT for secondary wakeup - NUMAQ
uses an NMI, (old-style-) ES7000 uses 'MIP' (a firmware driven in-memory
flag to let secondaries continue).
So convert these mechanisms to x86_quirks and add a
->wakeup_secondary_cpu() method to specify the rare exception
to the sane default.
Extend genapic accordingly as well, for 32-bit.
While looking further, I noticed that functions in wakecup.h for numaq
and es7000 are different to the default in mach_wakecpu.h - but smpboot.c
will only use default mach_wakecpu.h with smphook.h.
So we need to add mach_wakecpu.h for mach_generic, to properly support
numaq and es7000, and vectorize the following SMP init methods:
int trampoline_phys_low;
int trampoline_phys_high;
void (*wait_for_init_deassert)(atomic_t *deassert);
void (*smp_callin_clear_local_apic)(void);
void (*store_NMI_vector)(unsigned short *high, unsigned short *low);
void (*restore_NMI_vector)(unsigned short *high, unsigned short *low);
void (*inquire_remote_apic)(int apicid);
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-11-16 11:12:49 +00:00
|
|
|
wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
2015-05-04 09:42:34 +00:00
|
|
|
unsigned long send_status = 0, accept_status = 0;
|
2008-03-19 17:25:59 +00:00
|
|
|
int maxlvt, num_starts, j;
|
|
|
|
|
x86: APIC: remove apic_write_around(); use alternatives
Use alternatives to select the workaround for the 11AP Pentium erratum
for the affected steppings on the fly rather than build time. Remove the
X86_GOOD_APIC configuration option and replace all the calls to
apic_write_around() with plain apic_write(), protecting accesses to the
ESR as appropriate due to the 3AP Pentium erratum. Remove
apic_read_around() and all its invocations altogether as not needed.
Remove apic_write_atomic() and all its implementing backends. The use of
ASM_OUTPUT2() is not strictly needed for input constraints, but I have
used it for readability's sake.
I had the feeling no one else was brave enough to do it, so I went ahead
and here it is. Verified by checking the generated assembly and tested
with both a 32-bit and a 64-bit configuration, also with the 11AP
"feature" forced on and verified with gdb on /proc/kcore to work as
expected (as an 11AP machines are quite hard to get hands on these days).
Some script complained about the use of "volatile", but apic_write() needs
it for the same reason and is effectively a replacement for writel(), so I
have disregarded it.
I am not sure what the policy wrt defconfig files is, they are generated
and there is risk of a conflict resulting from an unrelated change, so I
have left changes to them out. The option will get removed from them at
the next run.
Some testing with machines other than mine will be needed to avoid some
stupid mistake, but despite its volume, the change is not really that
intrusive, so I am fairly confident that because it works for me, it will
everywhere.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-07-16 18:15:30 +00:00
|
|
|
maxlvt = lapic_get_maxlvt();
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
|
|
|
* Be paranoid about clearing APIC errors.
|
|
|
|
*/
|
2016-09-13 18:12:32 +00:00
|
|
|
if (APIC_INTEGRATED(boot_cpu_apic_version)) {
|
x86: APIC: remove apic_write_around(); use alternatives
Use alternatives to select the workaround for the 11AP Pentium erratum
for the affected steppings on the fly rather than build time. Remove the
X86_GOOD_APIC configuration option and replace all the calls to
apic_write_around() with plain apic_write(), protecting accesses to the
ESR as appropriate due to the 3AP Pentium erratum. Remove
apic_read_around() and all its invocations altogether as not needed.
Remove apic_write_atomic() and all its implementing backends. The use of
ASM_OUTPUT2() is not strictly needed for input constraints, but I have
used it for readability's sake.
I had the feeling no one else was brave enough to do it, so I went ahead
and here it is. Verified by checking the generated assembly and tested
with both a 32-bit and a 64-bit configuration, also with the 11AP
"feature" forced on and verified with gdb on /proc/kcore to work as
expected (as an 11AP machines are quite hard to get hands on these days).
Some script complained about the use of "volatile", but apic_write() needs
it for the same reason and is effectively a replacement for writel(), so I
have disregarded it.
I am not sure what the policy wrt defconfig files is, they are generated
and there is risk of a conflict resulting from an unrelated change, so I
have left changes to them out. The option will get removed from them at
the next run.
Some testing with machines other than mine will be needed to avoid some
stupid mistake, but despite its volume, the change is not really that
intrusive, so I am fairly confident that because it works for me, it will
everywhere.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-07-16 18:15:30 +00:00
|
|
|
if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
|
|
|
|
apic_write(APIC_ESR, 0);
|
2008-03-19 17:25:59 +00:00
|
|
|
apic_read(APIC_ESR);
|
|
|
|
}
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Asserting INIT\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Turn INIT on target chip
|
|
|
|
*/
|
|
|
|
/*
|
|
|
|
* Send IPI
|
|
|
|
*/
|
2008-07-10 18:16:49 +00:00
|
|
|
apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT,
|
|
|
|
phys_apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Waiting for send to finish...\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
send_status = safe_apic_wait_icr_idle();
|
|
|
|
|
x86/smp/boot: Fix legacy SMP bootup slow-boot bug
So while testing kernels using tools/kvm/ (kvmtool) I noticed that it
booted super slow:
[ 0.142991] Performance Events: no PMU driver, software events only.
[ 0.149265] x86: Booting SMP configuration:
[ 0.149765] .... node #0, CPUs: #1
[ 0.148304] kvm-clock: cpu 1, msr 2:1bfe9041, secondary cpu clock
[ 10.158813] KVM setup async PF for cpu 1
[ 10.159000] #2
[ 10.159000] kvm-stealtime: cpu 1, msr 211a4d400
[ 10.158829] kvm-clock: cpu 2, msr 2:1bfe9081, secondary cpu clock
[ 20.167805] KVM setup async PF for cpu 2
[ 20.168000] #3
[ 20.168000] kvm-stealtime: cpu 2, msr 211a8d400
[ 20.167818] kvm-clock: cpu 3, msr 2:1bfe90c1, secondary cpu clock
[ 30.176902] KVM setup async PF for cpu 3
[ 30.177000] #4
[ 30.177000] kvm-stealtime: cpu 3, msr 211acd400
One CPU booted up per 10 seconds. With 120 CPUs that takes a while.
Bisection pinpointed this commit:
853b160aaafb ("Revert f5d6a52f5111 ("x86/smpboot: Skip delays during SMP initialization similar to Xen")")
But that commit just restores previous behavior, so it cannot cause the
problem. After some head scratching it turns out that these two commits:
1a744cb356c5 ("x86/smp/boot: Remove 10ms delay from cpu_up() on modern processors")
d68921f9bd14 ("x86/smp/boot: Add cmdline "cpu_init_udelay=N" to specify cpu_up() delay")
added the following code to smpboot.c:
- mdelay(10);
+ mdelay(init_udelay);
Note the mismatch in the units: the delay is called 'udelay' and is set
to microseconds - while the function used here is actually 'mdelay',
which counts in milliseconds ...
So the delay for legacy systems is off by a factor of 1,000, so instead
of 10 msecs we waited for 10 seconds ...
The reason bisection pointed to 853b160aaafb was that 853b160aaafb removed
a (broken) boot-time speedup patch, which masked the factor 1,000 bug.
Fix it by using udelay(). This fixes my bootup problems.
Cc: Len Brown <len.brown@intel.com>
Cc: Alan Cox <alan@linux.intel.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jan H. Schönherr <jschoenh@amazon.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-05-18 10:05:13 +00:00
|
|
|
udelay(init_udelay);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Deasserting INIT\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/* Target chip */
|
|
|
|
/* Send IPI */
|
2008-07-10 18:16:49 +00:00
|
|
|
apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT, phys_apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Waiting for send to finish...\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
send_status = safe_apic_wait_icr_idle();
|
|
|
|
|
|
|
|
mb();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Should we send STARTUP IPIs ?
|
|
|
|
*
|
|
|
|
* Determine this based on the APIC version.
|
|
|
|
* If we don't have an integrated APIC, don't send the STARTUP IPIs.
|
|
|
|
*/
|
2016-09-13 18:12:32 +00:00
|
|
|
if (APIC_INTEGRATED(boot_cpu_apic_version))
|
2008-03-19 17:25:59 +00:00
|
|
|
num_starts = 2;
|
|
|
|
else
|
|
|
|
num_starts = 0;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Run STARTUP IPI loop.
|
|
|
|
*/
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("#startup loops: %d\n", num_starts);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
for (j = 1; j <= num_starts; j++) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Sending STARTUP #%d\n", j);
|
x86: APIC: remove apic_write_around(); use alternatives
Use alternatives to select the workaround for the 11AP Pentium erratum
for the affected steppings on the fly rather than build time. Remove the
X86_GOOD_APIC configuration option and replace all the calls to
apic_write_around() with plain apic_write(), protecting accesses to the
ESR as appropriate due to the 3AP Pentium erratum. Remove
apic_read_around() and all its invocations altogether as not needed.
Remove apic_write_atomic() and all its implementing backends. The use of
ASM_OUTPUT2() is not strictly needed for input constraints, but I have
used it for readability's sake.
I had the feeling no one else was brave enough to do it, so I went ahead
and here it is. Verified by checking the generated assembly and tested
with both a 32-bit and a 64-bit configuration, also with the 11AP
"feature" forced on and verified with gdb on /proc/kcore to work as
expected (as an 11AP machines are quite hard to get hands on these days).
Some script complained about the use of "volatile", but apic_write() needs
it for the same reason and is effectively a replacement for writel(), so I
have disregarded it.
I am not sure what the policy wrt defconfig files is, they are generated
and there is risk of a conflict resulting from an unrelated change, so I
have left changes to them out. The option will get removed from them at
the next run.
Some testing with machines other than mine will be needed to avoid some
stupid mistake, but despite its volume, the change is not really that
intrusive, so I am fairly confident that because it works for me, it will
everywhere.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-07-16 18:15:30 +00:00
|
|
|
if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
|
|
|
|
apic_write(APIC_ESR, 0);
|
2008-03-19 17:25:59 +00:00
|
|
|
apic_read(APIC_ESR);
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("After apic_write\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* STARTUP IPI
|
|
|
|
*/
|
|
|
|
|
|
|
|
/* Target chip */
|
|
|
|
/* Boot on the stack */
|
|
|
|
/* Kick the second */
|
2008-07-10 18:16:49 +00:00
|
|
|
apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12),
|
|
|
|
phys_apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Give the other CPU some time to accept the IPI.
|
|
|
|
*/
|
2015-10-16 04:14:29 +00:00
|
|
|
if (init_udelay == 0)
|
|
|
|
udelay(10);
|
|
|
|
else
|
2015-08-16 15:45:47 +00:00
|
|
|
udelay(300);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("Startup point 1\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Waiting for send to finish...\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
send_status = safe_apic_wait_icr_idle();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Give the other CPU some time to accept the IPI.
|
|
|
|
*/
|
2015-10-16 04:14:29 +00:00
|
|
|
if (init_udelay == 0)
|
|
|
|
udelay(10);
|
|
|
|
else
|
2015-08-16 15:45:47 +00:00
|
|
|
udelay(200);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
x86: APIC: remove apic_write_around(); use alternatives
Use alternatives to select the workaround for the 11AP Pentium erratum
for the affected steppings on the fly rather than build time. Remove the
X86_GOOD_APIC configuration option and replace all the calls to
apic_write_around() with plain apic_write(), protecting accesses to the
ESR as appropriate due to the 3AP Pentium erratum. Remove
apic_read_around() and all its invocations altogether as not needed.
Remove apic_write_atomic() and all its implementing backends. The use of
ASM_OUTPUT2() is not strictly needed for input constraints, but I have
used it for readability's sake.
I had the feeling no one else was brave enough to do it, so I went ahead
and here it is. Verified by checking the generated assembly and tested
with both a 32-bit and a 64-bit configuration, also with the 11AP
"feature" forced on and verified with gdb on /proc/kcore to work as
expected (as an 11AP machines are quite hard to get hands on these days).
Some script complained about the use of "volatile", but apic_write() needs
it for the same reason and is effectively a replacement for writel(), so I
have disregarded it.
I am not sure what the policy wrt defconfig files is, they are generated
and there is risk of a conflict resulting from an unrelated change, so I
have left changes to them out. The option will get removed from them at
the next run.
Some testing with machines other than mine will be needed to avoid some
stupid mistake, but despite its volume, the change is not really that
intrusive, so I am fairly confident that because it works for me, it will
everywhere.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-07-16 18:15:30 +00:00
|
|
|
if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
|
2008-03-19 17:25:59 +00:00
|
|
|
apic_write(APIC_ESR, 0);
|
|
|
|
accept_status = (apic_read(APIC_ESR) & 0xEF);
|
|
|
|
if (send_status || accept_status)
|
|
|
|
break;
|
|
|
|
}
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_debug("After Startup\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
if (send_status)
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_err("APIC never delivered???\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
if (accept_status)
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_err("APIC delivery error (%lx)\n", accept_status);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
return (send_status | accept_status);
|
|
|
|
}
|
|
|
|
|
2009-12-11 01:19:36 +00:00
|
|
|
/* reduce the number of lines printed when booting a large cpu count system */
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static void announce_cpu(int cpu, int apicid)
|
2009-12-11 01:19:36 +00:00
|
|
|
{
|
|
|
|
static int current_node = -1;
|
2010-06-01 19:04:55 +00:00
|
|
|
int node = early_cpu_to_node(cpu);
|
x86/boot: Further compress CPUs bootup message
Turn it into (for example):
[ 0.073380] x86: Booting SMP configuration:
[ 0.074005] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
[ 0.603005] .... node #1, CPUs: #8 #9 #10 #11 #12 #13 #14 #15
[ 1.200005] .... node #2, CPUs: #16 #17 #18 #19 #20 #21 #22 #23
[ 1.796005] .... node #3, CPUs: #24 #25 #26 #27 #28 #29 #30 #31
[ 2.393005] .... node #4, CPUs: #32 #33 #34 #35 #36 #37 #38 #39
[ 2.996005] .... node #5, CPUs: #40 #41 #42 #43 #44 #45 #46 #47
[ 3.600005] .... node #6, CPUs: #48 #49 #50 #51 #52 #53 #54 #55
[ 4.202005] .... node #7, CPUs: #56 #57 #58 #59 #60 #61 #62 #63
[ 4.811005] .... node #8, CPUs: #64 #65 #66 #67 #68 #69 #70 #71
[ 5.421006] .... node #9, CPUs: #72 #73 #74 #75 #76 #77 #78 #79
[ 6.032005] .... node #10, CPUs: #80 #81 #82 #83 #84 #85 #86 #87
[ 6.648006] .... node #11, CPUs: #88 #89 #90 #91 #92 #93 #94 #95
[ 7.262005] .... node #12, CPUs: #96 #97 #98 #99 #100 #101 #102 #103
[ 7.865005] .... node #13, CPUs: #104 #105 #106 #107 #108 #109 #110 #111
[ 8.466005] .... node #14, CPUs: #112 #113 #114 #115 #116 #117 #118 #119
[ 9.073006] .... node #15, CPUs: #120 #121 #122 #123 #124 #125 #126 #127
[ 9.679901] x86: Booted up 16 nodes, 128 CPUs
and drop useless elements.
Change num_digits() to hpa's division-avoiding, cell-phone-typed
version which he went at great lengths and pains to submit on a
Saturday evening.
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: huawei.libin@huawei.com
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20130930095624.GB16383@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-30 09:56:24 +00:00
|
|
|
static int width, node_width;
|
x86: Improve the printout of the SMP bootup CPU table
As the new x86 CPU bootup printout format code maintainer, I am
taking immediate action to improve and clean (and thus indulge
my OCD) the reporting of the cores when coming up online.
Fix padding to a right-hand alignment, cleanup code and bind
reporting width to the max number of supported CPUs on the
system, like this:
[ 0.074509] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.644008] smpboot: Booting Node 1, Processors: #8 #9 #10 #11 #12 #13 #14 #15 OK
[ 1.245006] smpboot: Booting Node 2, Processors: #16 #17 #18 #19 #20 #21 #22 #23 OK
[ 1.864005] smpboot: Booting Node 3, Processors: #24 #25 #26 #27 #28 #29 #30 #31 OK
[ 2.489005] smpboot: Booting Node 4, Processors: #32 #33 #34 #35 #36 #37 #38 #39 OK
[ 3.093005] smpboot: Booting Node 5, Processors: #40 #41 #42 #43 #44 #45 #46 #47 OK
[ 3.698005] smpboot: Booting Node 6, Processors: #48 #49 #50 #51 #52 #53 #54 #55 OK
[ 4.304005] smpboot: Booting Node 7, Processors: #56 #57 #58 #59 #60 #61 #62 #63 OK
[ 4.961413] Brought up 64 CPUs
and this:
[ 0.072367] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.686329] Brought up 8 CPUs
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Libin <huawei.libin@huawei.com>
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Link: http://lkml.kernel.org/r/20130927143554.GF4422@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-27 14:35:54 +00:00
|
|
|
|
|
|
|
if (!width)
|
|
|
|
width = num_digits(num_possible_cpus()) + 1; /* + '#' sign */
|
2009-12-11 01:19:36 +00:00
|
|
|
|
x86/boot: Further compress CPUs bootup message
Turn it into (for example):
[ 0.073380] x86: Booting SMP configuration:
[ 0.074005] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
[ 0.603005] .... node #1, CPUs: #8 #9 #10 #11 #12 #13 #14 #15
[ 1.200005] .... node #2, CPUs: #16 #17 #18 #19 #20 #21 #22 #23
[ 1.796005] .... node #3, CPUs: #24 #25 #26 #27 #28 #29 #30 #31
[ 2.393005] .... node #4, CPUs: #32 #33 #34 #35 #36 #37 #38 #39
[ 2.996005] .... node #5, CPUs: #40 #41 #42 #43 #44 #45 #46 #47
[ 3.600005] .... node #6, CPUs: #48 #49 #50 #51 #52 #53 #54 #55
[ 4.202005] .... node #7, CPUs: #56 #57 #58 #59 #60 #61 #62 #63
[ 4.811005] .... node #8, CPUs: #64 #65 #66 #67 #68 #69 #70 #71
[ 5.421006] .... node #9, CPUs: #72 #73 #74 #75 #76 #77 #78 #79
[ 6.032005] .... node #10, CPUs: #80 #81 #82 #83 #84 #85 #86 #87
[ 6.648006] .... node #11, CPUs: #88 #89 #90 #91 #92 #93 #94 #95
[ 7.262005] .... node #12, CPUs: #96 #97 #98 #99 #100 #101 #102 #103
[ 7.865005] .... node #13, CPUs: #104 #105 #106 #107 #108 #109 #110 #111
[ 8.466005] .... node #14, CPUs: #112 #113 #114 #115 #116 #117 #118 #119
[ 9.073006] .... node #15, CPUs: #120 #121 #122 #123 #124 #125 #126 #127
[ 9.679901] x86: Booted up 16 nodes, 128 CPUs
and drop useless elements.
Change num_digits() to hpa's division-avoiding, cell-phone-typed
version which he went at great lengths and pains to submit on a
Saturday evening.
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: huawei.libin@huawei.com
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20130930095624.GB16383@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-30 09:56:24 +00:00
|
|
|
if (!node_width)
|
|
|
|
node_width = num_digits(num_possible_nodes()) + 1; /* + '#' */
|
|
|
|
|
|
|
|
if (cpu == 1)
|
|
|
|
printk(KERN_INFO "x86: Booting SMP configuration:\n");
|
|
|
|
|
2017-05-16 18:42:35 +00:00
|
|
|
if (system_state < SYSTEM_RUNNING) {
|
2009-12-11 01:19:36 +00:00
|
|
|
if (node != current_node) {
|
|
|
|
if (current_node > (-1))
|
x86/boot: Further compress CPUs bootup message
Turn it into (for example):
[ 0.073380] x86: Booting SMP configuration:
[ 0.074005] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
[ 0.603005] .... node #1, CPUs: #8 #9 #10 #11 #12 #13 #14 #15
[ 1.200005] .... node #2, CPUs: #16 #17 #18 #19 #20 #21 #22 #23
[ 1.796005] .... node #3, CPUs: #24 #25 #26 #27 #28 #29 #30 #31
[ 2.393005] .... node #4, CPUs: #32 #33 #34 #35 #36 #37 #38 #39
[ 2.996005] .... node #5, CPUs: #40 #41 #42 #43 #44 #45 #46 #47
[ 3.600005] .... node #6, CPUs: #48 #49 #50 #51 #52 #53 #54 #55
[ 4.202005] .... node #7, CPUs: #56 #57 #58 #59 #60 #61 #62 #63
[ 4.811005] .... node #8, CPUs: #64 #65 #66 #67 #68 #69 #70 #71
[ 5.421006] .... node #9, CPUs: #72 #73 #74 #75 #76 #77 #78 #79
[ 6.032005] .... node #10, CPUs: #80 #81 #82 #83 #84 #85 #86 #87
[ 6.648006] .... node #11, CPUs: #88 #89 #90 #91 #92 #93 #94 #95
[ 7.262005] .... node #12, CPUs: #96 #97 #98 #99 #100 #101 #102 #103
[ 7.865005] .... node #13, CPUs: #104 #105 #106 #107 #108 #109 #110 #111
[ 8.466005] .... node #14, CPUs: #112 #113 #114 #115 #116 #117 #118 #119
[ 9.073006] .... node #15, CPUs: #120 #121 #122 #123 #124 #125 #126 #127
[ 9.679901] x86: Booted up 16 nodes, 128 CPUs
and drop useless elements.
Change num_digits() to hpa's division-avoiding, cell-phone-typed
version which he went at great lengths and pains to submit on a
Saturday evening.
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: huawei.libin@huawei.com
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20130930095624.GB16383@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-30 09:56:24 +00:00
|
|
|
pr_cont("\n");
|
2009-12-11 01:19:36 +00:00
|
|
|
current_node = node;
|
x86/boot: Further compress CPUs bootup message
Turn it into (for example):
[ 0.073380] x86: Booting SMP configuration:
[ 0.074005] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7
[ 0.603005] .... node #1, CPUs: #8 #9 #10 #11 #12 #13 #14 #15
[ 1.200005] .... node #2, CPUs: #16 #17 #18 #19 #20 #21 #22 #23
[ 1.796005] .... node #3, CPUs: #24 #25 #26 #27 #28 #29 #30 #31
[ 2.393005] .... node #4, CPUs: #32 #33 #34 #35 #36 #37 #38 #39
[ 2.996005] .... node #5, CPUs: #40 #41 #42 #43 #44 #45 #46 #47
[ 3.600005] .... node #6, CPUs: #48 #49 #50 #51 #52 #53 #54 #55
[ 4.202005] .... node #7, CPUs: #56 #57 #58 #59 #60 #61 #62 #63
[ 4.811005] .... node #8, CPUs: #64 #65 #66 #67 #68 #69 #70 #71
[ 5.421006] .... node #9, CPUs: #72 #73 #74 #75 #76 #77 #78 #79
[ 6.032005] .... node #10, CPUs: #80 #81 #82 #83 #84 #85 #86 #87
[ 6.648006] .... node #11, CPUs: #88 #89 #90 #91 #92 #93 #94 #95
[ 7.262005] .... node #12, CPUs: #96 #97 #98 #99 #100 #101 #102 #103
[ 7.865005] .... node #13, CPUs: #104 #105 #106 #107 #108 #109 #110 #111
[ 8.466005] .... node #14, CPUs: #112 #113 #114 #115 #116 #117 #118 #119
[ 9.073006] .... node #15, CPUs: #120 #121 #122 #123 #124 #125 #126 #127
[ 9.679901] x86: Booted up 16 nodes, 128 CPUs
and drop useless elements.
Change num_digits() to hpa's division-avoiding, cell-phone-typed
version which he went at great lengths and pains to submit on a
Saturday evening.
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: huawei.libin@huawei.com
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20130930095624.GB16383@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-30 09:56:24 +00:00
|
|
|
|
|
|
|
printk(KERN_INFO ".... node %*s#%d, CPUs: ",
|
|
|
|
node_width - num_digits(node), " ", node);
|
2009-12-11 01:19:36 +00:00
|
|
|
}
|
x86: Improve the printout of the SMP bootup CPU table
As the new x86 CPU bootup printout format code maintainer, I am
taking immediate action to improve and clean (and thus indulge
my OCD) the reporting of the cores when coming up online.
Fix padding to a right-hand alignment, cleanup code and bind
reporting width to the max number of supported CPUs on the
system, like this:
[ 0.074509] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.644008] smpboot: Booting Node 1, Processors: #8 #9 #10 #11 #12 #13 #14 #15 OK
[ 1.245006] smpboot: Booting Node 2, Processors: #16 #17 #18 #19 #20 #21 #22 #23 OK
[ 1.864005] smpboot: Booting Node 3, Processors: #24 #25 #26 #27 #28 #29 #30 #31 OK
[ 2.489005] smpboot: Booting Node 4, Processors: #32 #33 #34 #35 #36 #37 #38 #39 OK
[ 3.093005] smpboot: Booting Node 5, Processors: #40 #41 #42 #43 #44 #45 #46 #47 OK
[ 3.698005] smpboot: Booting Node 6, Processors: #48 #49 #50 #51 #52 #53 #54 #55 OK
[ 4.304005] smpboot: Booting Node 7, Processors: #56 #57 #58 #59 #60 #61 #62 #63 OK
[ 4.961413] Brought up 64 CPUs
and this:
[ 0.072367] smpboot: Booting Node 0, Processors: #1 #2 #3 #4 #5 #6 #7 OK
[ 0.686329] Brought up 8 CPUs
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Libin <huawei.libin@huawei.com>
Cc: wangyijing@huawei.com
Cc: fenghua.yu@intel.com
Cc: guohanjun@huawei.com
Cc: paul.gortmaker@windriver.com
Link: http://lkml.kernel.org/r/20130927143554.GF4422@pd.tnic
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-09-27 14:35:54 +00:00
|
|
|
|
|
|
|
/* Add padding for the BSP */
|
|
|
|
if (cpu == 1)
|
|
|
|
pr_cont("%*s", width + 1, " ");
|
|
|
|
|
|
|
|
pr_cont("%*s#%d", width - num_digits(cpu), " ", cpu);
|
|
|
|
|
2009-12-11 01:19:36 +00:00
|
|
|
} else
|
|
|
|
pr_info("Booting Node %d Processor %d APIC 0x%x\n",
|
|
|
|
node, cpu, apicid);
|
|
|
|
}
|
|
|
|
|
2012-11-14 12:36:53 +00:00
|
|
|
static int wakeup_cpu0_nmi(unsigned int cmd, struct pt_regs *regs)
|
|
|
|
{
|
|
|
|
int cpu;
|
|
|
|
|
|
|
|
cpu = smp_processor_id();
|
|
|
|
if (cpu == 0 && !cpu_online(cpu) && enable_start_cpu0)
|
|
|
|
return NMI_HANDLED;
|
|
|
|
|
|
|
|
return NMI_DONE;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Wake up AP by INIT, INIT, STARTUP sequence.
|
|
|
|
*
|
|
|
|
* Instead of waiting for STARTUP after INITs, BSP will execute the BIOS
|
|
|
|
* boot-strap code which is not a desired behavior for waking up BSP. To
|
|
|
|
* void the boot-strap code, wake up CPU0 by NMI instead.
|
|
|
|
*
|
|
|
|
* This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined
|
|
|
|
* (i.e. physically hot removed and then hot added), NMI won't wake it up.
|
|
|
|
* We'll change this code in the future to wake up hard offlined CPU0 if
|
|
|
|
* real platform and request are available.
|
|
|
|
*/
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
static int
|
2012-11-14 12:36:53 +00:00
|
|
|
wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid,
|
|
|
|
int *cpu0_nmi_registered)
|
|
|
|
{
|
|
|
|
int id;
|
|
|
|
int boot_error;
|
|
|
|
|
2014-01-27 19:14:06 +00:00
|
|
|
preempt_disable();
|
|
|
|
|
2012-11-14 12:36:53 +00:00
|
|
|
/*
|
|
|
|
* Wake up AP by INIT, INIT, STARTUP sequence.
|
|
|
|
*/
|
2014-01-27 19:14:06 +00:00
|
|
|
if (cpu) {
|
|
|
|
boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
|
|
|
|
goto out;
|
|
|
|
}
|
2012-11-14 12:36:53 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Wake up BSP by nmi.
|
|
|
|
*
|
|
|
|
* Register a NMI handler to help wake up CPU0.
|
|
|
|
*/
|
|
|
|
boot_error = register_nmi_handler(NMI_LOCAL,
|
|
|
|
wakeup_cpu0_nmi, 0, "wake_cpu0");
|
|
|
|
|
|
|
|
if (!boot_error) {
|
|
|
|
enable_start_cpu0 = 1;
|
|
|
|
*cpu0_nmi_registered = 1;
|
|
|
|
if (apic->dest_logical == APIC_DEST_LOGICAL)
|
|
|
|
id = cpu0_logical_apicid;
|
|
|
|
else
|
|
|
|
id = apicid;
|
|
|
|
boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip);
|
|
|
|
}
|
2014-01-27 19:14:06 +00:00
|
|
|
|
|
|
|
out:
|
|
|
|
preempt_enable();
|
2012-11-14 12:36:53 +00:00
|
|
|
|
|
|
|
return boot_error;
|
|
|
|
}
|
|
|
|
|
2015-04-01 14:12:14 +00:00
|
|
|
void common_cpu_up(unsigned int cpu, struct task_struct *idle)
|
|
|
|
{
|
|
|
|
/* Just in case we booted with a single CPU. */
|
|
|
|
alternatives_enable_smp();
|
|
|
|
|
|
|
|
per_cpu(current_task, cpu) = idle;
|
|
|
|
|
|
|
|
#ifdef CONFIG_X86_32
|
|
|
|
/* Stack for startup_32 can be just as for start_secondary onwards */
|
|
|
|
irq_ctx_init(cpu);
|
2017-11-02 07:59:15 +00:00
|
|
|
per_cpu(cpu_current_top_of_stack, cpu) = task_top_of_stack(idle);
|
2015-04-01 14:12:14 +00:00
|
|
|
#else
|
|
|
|
initial_gs = per_cpu_offset(cpu);
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
|
|
|
* NOTE - on most systems this is a PHYSICAL apic ID, but on multiquad
|
|
|
|
* (ie clustered apic addressing mode), this is a LOGICAL apic ID.
|
2009-02-26 12:51:40 +00:00
|
|
|
* Returns zero if CPU booted OK, else error code from
|
|
|
|
* ->wakeup_secondary_cpu.
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
2017-08-03 10:58:18 +00:00
|
|
|
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
|
|
|
|
int *cpu0_nmi_registered)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
2012-05-08 18:22:28 +00:00
|
|
|
volatile u32 *trampoline_status =
|
2012-05-08 18:22:41 +00:00
|
|
|
(volatile u32 *) __va(real_mode_header->trampoline_status);
|
2012-05-08 18:22:28 +00:00
|
|
|
/* start_ip had better be page-aligned! */
|
2012-05-08 18:22:43 +00:00
|
|
|
unsigned long start_ip = real_mode_header->trampoline_start;
|
2012-05-08 18:22:28 +00:00
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
unsigned long boot_error = 0;
|
2014-06-20 12:23:11 +00:00
|
|
|
unsigned long timeout;
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2016-09-21 21:04:03 +00:00
|
|
|
idle->thread.sp = (unsigned long)task_pt_regs(idle);
|
2017-03-14 17:05:07 +00:00
|
|
|
early_gdt_descr.address = (unsigned long)get_cpu_gdt_rw(cpu);
|
2008-05-28 16:01:54 +00:00
|
|
|
initial_code = (unsigned long)start_secondary;
|
2016-08-18 15:59:03 +00:00
|
|
|
initial_stack = idle->thread.sp;
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2017-12-17 09:56:29 +00:00
|
|
|
/* Enable the espfix hack for this CPU */
|
2015-07-03 09:37:19 +00:00
|
|
|
init_espfix_ap(cpu);
|
|
|
|
|
2009-12-11 01:19:36 +00:00
|
|
|
/* So we see what's up */
|
|
|
|
announce_cpu(cpu, apicid);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* This grunge runs the startup process for
|
|
|
|
* the targeted processor.
|
|
|
|
*/
|
|
|
|
|
2017-11-27 08:11:44 +00:00
|
|
|
if (x86_platform.legacy.warm_reset) {
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("Setting warm reset code and vector.\n");
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-04-16 16:45:15 +00:00
|
|
|
smpboot_setup_warm_reset_vector(start_ip);
|
|
|
|
/*
|
|
|
|
* Be paranoid about clearing APIC errors.
|
2008-10-22 14:00:09 +00:00
|
|
|
*/
|
2016-09-13 18:12:32 +00:00
|
|
|
if (APIC_INTEGRATED(boot_cpu_apic_version)) {
|
2008-10-22 14:00:09 +00:00
|
|
|
apic_write(APIC_ESR, 0);
|
|
|
|
apic_read(APIC_ESR);
|
|
|
|
}
|
2008-04-16 16:45:15 +00:00
|
|
|
}
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2014-06-20 12:23:11 +00:00
|
|
|
/*
|
|
|
|
* AP might wait on cpu_callout_mask in cpu_init() with
|
|
|
|
* cpu_initialized_mask set if previous attempt to online
|
|
|
|
* it timed-out. Clear cpu_initialized_mask so that after
|
|
|
|
* INIT/SIPI it could start with a clean state.
|
|
|
|
*/
|
|
|
|
cpumask_clear_cpu(cpu, cpu_initialized_mask);
|
|
|
|
smp_mb();
|
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
2012-11-14 12:36:53 +00:00
|
|
|
* Wake up a CPU in difference cases:
|
|
|
|
* - Use the method in the APIC driver if it's defined
|
|
|
|
* Otherwise,
|
|
|
|
* - Use an INIT boot APIC message for APs or NMI for BSP.
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
2009-02-26 12:51:40 +00:00
|
|
|
if (apic->wakeup_secondary_cpu)
|
|
|
|
boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
|
|
|
|
else
|
2012-11-14 12:36:53 +00:00
|
|
|
boot_error = wakeup_cpu_via_init_nmi(cpu, start_ip, apicid,
|
2017-08-03 10:58:18 +00:00
|
|
|
cpu0_nmi_registered);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
|
|
|
if (!boot_error) {
|
|
|
|
/*
|
x86/smpboot: Remove udelay(100) when polling cpu_initialized_map
After the BSP sends the APIC INIT/SIPI/SIPI to the AP,
it waits for the AP to come up and indicate that it is alive
by setting its own bit in the cpu_initialized_mask.
Linux polls for up to 10 seconds for this to happen.
Each polling loop has a udelay(100) and a call to schedule().
The udelay(100) adds no value.
For example, on my desktop, the BSP waits for the
other 3 CPUs to come on line at boot for 305, 404, 405 usec.
For resume from S3, it waits 317, 404, 405 usec.
But when the udelay(100) is removed, the BSP waits
305, 310, 306 for boot, and 305, 307, 306 for resume.
So for both boot and resume, removing the udelay(100)
speeds online by about 100us in 2 of 3 cases.
Signed-off-by: Len Brown <len.brown@intel.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jan H. Schönherr <jschoenh@amazon.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Zhu Guihua <zhugh.fnst@cn.fujitsu.com>
Link: http://lkml.kernel.org/r/33ef746c67d2489cad0a9b1958cf71167232ff2b.1439739165.git.len.brown@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-08-16 15:45:45 +00:00
|
|
|
* Wait 10s total for first sign of life from AP
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
2014-06-20 12:23:11 +00:00
|
|
|
boot_error = -1;
|
|
|
|
timeout = jiffies + 10*HZ;
|
|
|
|
while (time_before(jiffies, timeout)) {
|
|
|
|
if (cpumask_test_cpu(cpu, cpu_initialized_mask)) {
|
|
|
|
/*
|
|
|
|
* Tell AP to proceed with initialization
|
|
|
|
*/
|
|
|
|
cpumask_set_cpu(cpu, cpu_callout_mask);
|
|
|
|
boot_error = 0;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
schedule();
|
|
|
|
}
|
|
|
|
}
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2014-06-20 12:23:11 +00:00
|
|
|
if (!boot_error) {
|
2008-03-19 17:25:59 +00:00
|
|
|
/*
|
2014-06-20 12:23:11 +00:00
|
|
|
* Wait till AP completes initial initialization
|
2008-03-19 17:25:59 +00:00
|
|
|
*/
|
2014-06-20 12:23:11 +00:00
|
|
|
while (!cpumask_test_cpu(cpu, cpu_callin_mask)) {
|
2010-07-30 18:46:42 +00:00
|
|
|
/*
|
|
|
|
* Allow other tasks to run while we wait for the
|
|
|
|
* AP to come online. This also gives a chance
|
|
|
|
* for the MTRR work(triggered by the AP coming online)
|
|
|
|
* to be completed in the stop machine context.
|
|
|
|
*/
|
|
|
|
schedule();
|
2008-03-19 17:25:59 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* mark "stuck" area as not stuck */
|
2012-05-08 18:22:28 +00:00
|
|
|
*trampoline_status = 0;
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2017-11-27 08:11:44 +00:00
|
|
|
if (x86_platform.legacy.warm_reset) {
|
2009-04-04 00:15:53 +00:00
|
|
|
/*
|
|
|
|
* Cleanup possible dangling ends...
|
|
|
|
*/
|
|
|
|
smpboot_restore_warm_reset_vector();
|
|
|
|
}
|
2012-11-14 12:36:53 +00:00
|
|
|
|
2008-03-19 17:25:59 +00:00
|
|
|
return boot_error;
|
|
|
|
}
|
|
|
|
|
x86: delete __cpuinit usage from all x86 files
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/x86 uses of the __cpuinit macros from
all C files. x86 only had the one __CPUINIT used in assembly files,
and it wasn't paired off with a .previous or a __FINIT, so we can
delete it directly w/o any corresponding additional change there.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2013-06-18 22:23:59 +00:00
|
|
|
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
|
2008-03-19 17:25:59 +00:00
|
|
|
{
|
2009-01-28 05:50:47 +00:00
|
|
|
int apicid = apic->cpu_present_to_apicid(cpu);
|
2017-08-03 10:58:18 +00:00
|
|
|
int cpu0_nmi_registered = 0;
|
2008-03-19 17:25:59 +00:00
|
|
|
unsigned long flags;
|
2017-08-03 10:58:18 +00:00
|
|
|
int err, ret = 0;
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2017-11-06 15:01:23 +00:00
|
|
|
lockdep_assert_irqs_enabled();
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("++++++++++++++++++++=_---CPU UP %u\n", cpu);
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2012-11-13 19:32:41 +00:00
|
|
|
if (apicid == BAD_APICID ||
|
2011-12-22 01:45:19 +00:00
|
|
|
!physid_isset(apicid, phys_cpu_present_map) ||
|
2012-03-14 07:17:34 +00:00
|
|
|
!apic->apic_id_valid(apicid)) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_err("%s: bad cpu %d\n", __func__, cpu);
|
2008-03-19 17:25:59 +00:00
|
|
|
return -EINVAL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Already booted CPU?
|
|
|
|
*/
|
2009-01-04 13:18:03 +00:00
|
|
|
if (cpumask_test_cpu(cpu, cpu_callin_mask)) {
|
2008-07-21 19:35:38 +00:00
|
|
|
pr_debug("do_boot_cpu %d Already started\n", cpu);
|
2008-03-19 17:25:59 +00:00
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Save current MTRR state in case it was changed since early boot
|
|
|
|
* (e.g. by the ACPI SMI) to initialize new CPUs with MTRRs in sync:
|
|
|
|
*/
|
|
|
|
mtrr_save_state();
|
|
|
|
|
2015-02-25 19:42:15 +00:00
|
|
|
/* x86 CPUs take themselves offline, so delayed offline is OK. */
|
|
|
|
err = cpu_check_up_prepare(cpu);
|
|
|
|
if (err && err != -EBUSY)
|
|
|
|
return err;
|
2008-03-19 17:25:59 +00:00
|
|
|
|
2012-11-30 20:15:32 +00:00
|
|
|
/* the FPU context is blank, nobody can own it */
|
2016-10-14 12:15:30 +00:00
|
|
|
per_cpu(fpu_fpregs_owner_ctx, cpu) = NULL;
|
2012-11-30 20:15:32 +00:00
|
|
|
|
2015-04-01 14:12:14 +00:00
|
|
|
common_cpu_up(cpu, tidle);
|
|
|
|
|
2017-08-03 10:58:18 +00:00
|
|
|
err = do_boot_cpu(apicid, cpu, tidle, &cpu0_nmi_registered);
|
x86: fix app crashes after SMP resume
After resume on a 2cpu laptop, kernel builds collapse with a sed hang,
sh or make segfault (often on 20295564), real-time signal to cc1 etc.
Several hurdles to jump, but a manually-assisted bisect led to -rc1's
d2bcbad5f3ad38a1c09861bca7e252dde7bb8259 x86: do not zap_low_mappings
in __smp_prepare_cpus. Though the low mappings were removed at bootup,
they were left behind (with Global flags helping to keep them in TLB)
after resume or cpu online, causing the crashes seen.
Reinstate zap_low_mappings (with local __flush_tlb_all) for each cpu_up
on x86_32. This used to be serialized by smp_commenced_mask: that's now
gone, but a low_mappings flag will do. No need for native_smp_cpus_done
to repeat the zap: let mem_init zap BSP's low mappings just like on UP.
(In passing, fix error code from native_cpu_up: do_boot_cpu returns a
variety of diagnostic values, Dprintk what it says but convert to -EIO.
And save_pg_dir separately before zap_low_mappings: doesn't matter now,
but zapping twice in succession wiped out resume's swsusp_pg_dir.)
That worked well on the duo and one quad, but wouldn't boot 3rd or 4th
cpu on P4 Xeon, oopsing just after unlock_ipi_call_lock. The TLB flush
IPI now being sent reveals a long-standing bug: the booting cpu has its
APIC readied in smp_callin at the top of start_secondary, but isn't put
into the cpu_online_map until just before that unlock_ipi_call_lock.
So native_smp_call_function_mask to online cpus would send_IPI_allbutself,
including the cpu just coming up, though it has been excluded from the
count to wait for: by the time it handles the IPI, the call data on
native_smp_call_function_mask's stack may well have been overwritten.
So fall back to send_IPI_mask while cpu_online_map does not match
cpu_callout_map: perhaps there's a better APICological fix to be
made at the start_secondary end, but I wouldn't know that.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-05-13 13:26:57 +00:00
|
|
|
if (err) {
|
2014-06-05 13:42:44 +00:00
|
|
|
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
|
2017-08-03 10:58:18 +00:00
|
|
|
ret = -EIO;
|
|
|
|
goto unreg_nmi;
|
2008-03-19 17:25:59 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Check TSC synchronization with the AP (keep irqs disabled
|
|
|
|
* while doing so):
|
|
|
|
*/
|
|
|
|
local_irq_save(flags);
|
|
|
|
check_tsc_sync_source(cpu);
|
|
|
|
local_irq_restore(flags);
|
|
|
|
|
2008-04-19 14:55:17 +00:00
|
|
|
while (!cpu_online(cpu)) {
|
2008-03-19 17:25:59 +00:00
|
|
|
cpu_relax();
|
|
|
|
touch_nmi_watchdog();
|
|
|
|
}
|
|
|
|
|
2017-08-03 10:58:18 +00:00
|
|
|
unreg_nmi:
|
|
|
|
/*
|
|
|
|
* Clean up the nmi handler. Do this after the callin and callout sync
|
|
|
|
* to avoid impact of possible long unregister time.
|
|
|
|
*/
|
|
|
|
if (cpu0_nmi_registered)
|
|
|
|
unregister_nmi_handler(NMI_LOCAL, "wake_cpu0");
|
|
|
|
|
|
|
|
return ret;
|
2008-03-19 17:25:59 +00:00
|
|
|
}
|
|
|
|
|
2011-02-22 14:38:05 +00:00
|
|
|
/**
|
|
|
|
* arch_disable_smp_support() - disables SMP support for x86 at runtime
|
|
|
|
*/
|
|
|
|
void arch_disable_smp_support(void)
|
|
|
|
{
|
|
|
|
disable_ioapic_support();
|
|
|
|
}
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
/*
|
|
|
|
* Fall back to non SMP mode after errors.
|
|
|
|
*
|
|
|
|
* RED-PEN audit/test this more. I bet there is more state messed up here.
|
|
|
|
*/
|
|
|
|
static __init void disable_smp(void)
|
|
|
|
{
|
2015-01-15 21:22:42 +00:00
|
|
|
pr_info("SMP disabled\n");
|
|
|
|
|
2015-01-15 21:22:35 +00:00
|
|
|
disable_ioapic_support();
|
|
|
|
|
2009-03-13 04:19:54 +00:00
|
|
|
init_cpu_present(cpumask_of(0));
|
|
|
|
init_cpu_possible(cpumask_of(0));
|
2008-05-29 00:09:53 +00:00
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
if (smp_found_config)
|
2008-06-20 02:51:05 +00:00
|
|
|
physid_set_mask_of_physid(boot_cpu_physical_apicid, &phys_cpu_present_map);
|
2008-03-19 17:26:11 +00:00
|
|
|
else
|
2008-06-20 02:51:05 +00:00
|
|
|
physid_set_mask_of_physid(0, &phys_cpu_present_map);
|
2015-05-26 13:11:35 +00:00
|
|
|
cpumask_set_cpu(0, topology_sibling_cpumask(0));
|
|
|
|
cpumask_set_cpu(0, topology_core_cpumask(0));
|
2008-03-19 17:26:11 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Various sanity checks.
|
|
|
|
*/
|
2017-09-13 09:12:49 +00:00
|
|
|
static void __init smp_sanity_check(void)
|
2008-03-19 17:26:11 +00:00
|
|
|
{
|
2008-03-28 19:12:16 +00:00
|
|
|
preempt_disable();
|
2008-08-14 09:16:30 +00:00
|
|
|
|
2009-01-30 03:30:04 +00:00
|
|
|
#if !defined(CONFIG_X86_BIGSMP) && defined(CONFIG_X86_32)
|
2008-08-14 09:16:30 +00:00
|
|
|
if (def_to_bigsmp && nr_cpu_ids > 8) {
|
|
|
|
unsigned int cpu;
|
|
|
|
unsigned nr;
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_warn("More than 8 CPUs detected - skipping them\n"
|
|
|
|
"Use CONFIG_X86_BIGSMP\n");
|
2008-08-14 09:16:30 +00:00
|
|
|
|
|
|
|
nr = 0;
|
|
|
|
for_each_present_cpu(cpu) {
|
|
|
|
if (nr >= 8)
|
2009-01-04 13:18:03 +00:00
|
|
|
set_cpu_present(cpu, false);
|
2008-08-14 09:16:30 +00:00
|
|
|
nr++;
|
|
|
|
}
|
|
|
|
|
|
|
|
nr = 0;
|
|
|
|
for_each_possible_cpu(cpu) {
|
|
|
|
if (nr >= 8)
|
2009-01-04 13:18:03 +00:00
|
|
|
set_cpu_possible(cpu, false);
|
2008-08-14 09:16:30 +00:00
|
|
|
nr++;
|
|
|
|
}
|
|
|
|
|
|
|
|
nr_cpu_ids = 8;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
if (!physid_isset(hard_smp_processor_id(), phys_cpu_present_map)) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_warn("weird, boot CPU (#%d) not listed by the BIOS\n",
|
2008-12-05 11:42:20 +00:00
|
|
|
hard_smp_processor_id());
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
physid_set(hard_smp_processor_id(), phys_cpu_present_map);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Should not be necessary because the MP table should list the boot
|
|
|
|
* CPU too, but we do it for the sake of robustness anyway.
|
|
|
|
*/
|
2009-01-28 11:43:18 +00:00
|
|
|
if (!apic->check_phys_apicid_present(boot_cpu_physical_apicid)) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_notice("weird, boot CPU (#%d) not listed by the BIOS\n",
|
|
|
|
boot_cpu_physical_apicid);
|
2008-03-19 17:26:11 +00:00
|
|
|
physid_set(hard_smp_processor_id(), phys_cpu_present_map);
|
|
|
|
}
|
2008-03-28 19:12:16 +00:00
|
|
|
preempt_enable();
|
2008-03-19 17:26:11 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void __init smp_cpu_index_default(void)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
struct cpuinfo_x86 *c;
|
|
|
|
|
2008-04-19 14:55:17 +00:00
|
|
|
for_each_possible_cpu(i) {
|
2008-03-19 17:26:11 +00:00
|
|
|
c = &cpu_data(i);
|
|
|
|
/* mark all to hotplug */
|
2009-01-01 02:08:46 +00:00
|
|
|
c->cpu_index = nr_cpu_ids;
|
2008-03-19 17:26:11 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-09-13 09:12:47 +00:00
|
|
|
static void __init smp_get_logical_apicid(void)
|
|
|
|
{
|
|
|
|
if (x2apic_mode)
|
|
|
|
cpu0_logical_apicid = apic_read(APIC_LDR);
|
|
|
|
else
|
|
|
|
cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
|
|
|
|
}
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
/*
|
2017-09-13 09:12:54 +00:00
|
|
|
* Prepare for SMP bootup.
|
|
|
|
* @max_cpus: configured maximum number of CPUs, It is a legacy parameter
|
|
|
|
* for common interface support.
|
2008-03-19 17:26:11 +00:00
|
|
|
*/
|
|
|
|
void __init native_smp_prepare_cpus(unsigned int max_cpus)
|
|
|
|
{
|
2009-03-13 04:19:50 +00:00
|
|
|
unsigned int i;
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
smp_cpu_index_default();
|
2011-01-21 23:29:54 +00:00
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
/*
|
|
|
|
* Setup boot CPU information
|
|
|
|
*/
|
2012-11-13 19:32:41 +00:00
|
|
|
smp_store_boot_cpu_info(); /* Final full version of the data */
|
2011-01-21 23:29:54 +00:00
|
|
|
cpumask_copy(cpu_callin_mask, cpumask_of(0));
|
|
|
|
mb();
|
2011-01-23 13:37:27 +00:00
|
|
|
|
2009-03-13 04:19:50 +00:00
|
|
|
for_each_possible_cpu(i) {
|
2009-06-15 06:58:26 +00:00
|
|
|
zalloc_cpumask_var(&per_cpu(cpu_sibling_map, i), GFP_KERNEL);
|
|
|
|
zalloc_cpumask_var(&per_cpu(cpu_core_map, i), GFP_KERNEL);
|
2011-01-21 23:29:44 +00:00
|
|
|
zalloc_cpumask_var(&per_cpu(cpu_llc_shared_map, i), GFP_KERNEL);
|
2009-03-13 04:19:50 +00:00
|
|
|
}
|
2016-09-21 19:19:03 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Set 'default' x86 topology, this matches default_topology() in that
|
|
|
|
* it has NUMA nodes as a topology level. See also
|
|
|
|
* native_smp_cpus_done().
|
|
|
|
*
|
|
|
|
* Must be done before set_cpus_sibling_map() is ran.
|
|
|
|
*/
|
|
|
|
set_sched_topology(x86_topology);
|
|
|
|
|
2008-03-19 17:26:11 +00:00
|
|
|
set_cpu_sibling_map(0);
|
|
|
|
|
2017-09-13 09:12:49 +00:00
|
|
|
smp_sanity_check();
|
|
|
|
|
|
|
|
switch (apic_intr_mode) {
|
|
|
|
case APIC_PIC:
|
|
|
|
case APIC_VIRTUAL_WIRE_NO_CONFIG:
|
2015-01-15 21:22:42 +00:00
|
|
|
disable_smp();
|
|
|
|
return;
|
2017-09-13 09:12:49 +00:00
|
|
|
case APIC_SYMMETRIC_IO_NO_ROUTING:
|
2015-01-15 21:22:42 +00:00
|
|
|
disable_smp();
|
2017-09-13 09:12:46 +00:00
|
|
|
/* Setup local timer */
|
|
|
|
x86_init.timers.setup_percpu_clockev();
|
2014-12-05 08:48:29 +00:00
|
|
|
return;
|
2017-09-13 09:12:49 +00:00
|
|
|
case APIC_VIRTUAL_WIRE:
|
|
|
|
case APIC_SYMMETRIC_IO:
|
2015-01-15 21:22:42 +00:00
|
|
|
break;
|
2008-03-19 17:26:11 +00:00
|
|
|
}
|
|
|
|
|
2017-09-13 09:12:46 +00:00
|
|
|
/* Setup local timer */
|
|
|
|
x86_init.timers.setup_percpu_clockev();
|
2008-03-19 17:26:11 +00:00
|
|
|
|
2017-09-13 09:12:47 +00:00
|
|
|
smp_get_logical_apicid();
|
2015-01-15 21:22:35 +00:00
|
|
|
|
2016-10-24 17:38:42 +00:00
|
|
|
pr_info("CPU0: ");
|
2008-03-19 17:26:11 +00:00
|
|
|
print_cpu_info(&cpu_data(0));
|
2008-08-21 18:49:05 +00:00
|
|
|
|
2017-10-28 06:06:44 +00:00
|
|
|
native_pv_lock_init();
|
|
|
|
|
2017-01-25 16:35:23 +00:00
|
|
|
uv_system_init();
|
2009-08-20 01:05:36 +00:00
|
|
|
|
|
|
|
set_mtrr_aps_delayed_init();
|
2015-05-11 21:27:10 +00:00
|
|
|
|
|
|
|
smp_quirk_init_udelay();
|
2008-03-19 17:26:11 +00:00
|
|
|
}
|
2009-08-20 01:05:36 +00:00
|
|
|
|
|
|
|
void arch_enable_nonboot_cpus_begin(void)
|
|
|
|
{
|
|
|
|
set_mtrr_aps_delayed_init();
|
|
|
|
}
|
|
|
|
|
|
|
|
void arch_enable_nonboot_cpus_end(void)
|
|
|
|
{
|
|
|
|
mtrr_aps_init();
|
|
|
|
}
|
|
|
|
|
2008-03-19 17:26:01 +00:00
|
|
|
/*
|
|
|
|
* Early setup to make printk work.
|
|
|
|
*/
|
|
|
|
void __init native_smp_prepare_boot_cpu(void)
|
|
|
|
{
|
|
|
|
int me = smp_processor_id();
|
2009-01-30 08:47:53 +00:00
|
|
|
switch_to_new_gdt(me);
|
2009-01-04 13:18:03 +00:00
|
|
|
/* already set me in cpu_online_mask in boot_cpu_init() */
|
|
|
|
cpumask_set_cpu(me, cpu_callout_mask);
|
2015-02-25 19:42:15 +00:00
|
|
|
cpu_set_state_online(me);
|
2008-03-19 17:26:01 +00:00
|
|
|
}
|
|
|
|
|
2018-02-07 23:49:23 +00:00
|
|
|
void __init calculate_max_logical_packages(void)
|
2008-03-19 17:26:02 +00:00
|
|
|
{
|
2017-11-14 12:42:57 +00:00
|
|
|
int ncpus;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Today neither Intel nor AMD support heterogenous systems so
|
|
|
|
* extrapolate the boot cpu's data to all packages.
|
|
|
|
*/
|
2017-12-04 16:45:21 +00:00
|
|
|
ncpus = cpu_data(0).booted_cores * topology_max_smt_threads();
|
2017-11-14 12:42:57 +00:00
|
|
|
__max_logical_packages = DIV_ROUND_UP(nr_cpu_ids, ncpus);
|
|
|
|
pr_info("Max logical packages: %u\n", __max_logical_packages);
|
2018-02-07 23:49:23 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void __init native_smp_cpus_done(unsigned int max_cpus)
|
|
|
|
{
|
|
|
|
pr_debug("Boot done\n");
|
|
|
|
|
|
|
|
calculate_max_logical_packages();
|
2008-03-19 17:26:02 +00:00
|
|
|
|
2016-09-21 19:19:03 +00:00
|
|
|
if (x86_has_numa_in_package)
|
|
|
|
set_sched_topology(x86_numa_in_package_topology);
|
|
|
|
|
2011-10-13 19:14:26 +00:00
|
|
|
nmi_selftest();
|
2008-03-19 17:26:02 +00:00
|
|
|
impress_friends();
|
2009-08-20 01:05:36 +00:00
|
|
|
mtrr_aps_init();
|
2008-03-19 17:26:02 +00:00
|
|
|
}
|
|
|
|
|
2008-12-17 23:21:39 +00:00
|
|
|
static int __initdata setup_possible_cpus = -1;
|
|
|
|
static int __init _setup_possible_cpus(char *str)
|
|
|
|
{
|
|
|
|
get_option(&str, &setup_possible_cpus);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
early_param("possible_cpus", _setup_possible_cpus);
|
|
|
|
|
|
|
|
|
2008-03-03 17:12:42 +00:00
|
|
|
/*
|
2009-03-13 04:19:54 +00:00
|
|
|
* cpu_possible_mask should be static, it cannot change as cpu's
|
2008-03-03 17:12:42 +00:00
|
|
|
* are onlined, or offlined. The reason is per-cpu data-structures
|
|
|
|
* are allocated by some modules at init time, and dont expect to
|
|
|
|
* do this dynamically on cpu arrival/departure.
|
2009-03-13 04:19:54 +00:00
|
|
|
* cpu_present_mask on the other hand can change dynamically.
|
2008-03-03 17:12:42 +00:00
|
|
|
* In case when cpu_hotplug is not compiled, then we resort to current
|
|
|
|
* behaviour, which is cpu_possible == cpu_present.
|
|
|
|
* - Ashok Raj
|
|
|
|
*
|
|
|
|
* Three ways to find out the number of additional hotplug CPUs:
|
|
|
|
* - If the BIOS specified disabled CPUs in ACPI/mptables use that.
|
2008-12-17 23:21:39 +00:00
|
|
|
* - The user can overwrite it with possible_cpus=NUM
|
2008-03-03 17:12:42 +00:00
|
|
|
* - Otherwise don't reserve additional CPUs.
|
|
|
|
* We do this because additional CPUs waste a lot of memory.
|
|
|
|
* -AK
|
|
|
|
*/
|
|
|
|
__init void prefill_possible_map(void)
|
|
|
|
{
|
2008-10-05 15:51:52 +00:00
|
|
|
int i, possible;
|
2008-03-03 17:12:42 +00:00
|
|
|
|
2016-10-03 17:07:12 +00:00
|
|
|
/* No boot processor was found in mptable or ACPI MADT */
|
|
|
|
if (!num_processors) {
|
2016-10-22 02:18:04 +00:00
|
|
|
if (boot_cpu_has(X86_FEATURE_APIC)) {
|
|
|
|
int apicid = boot_cpu_physical_apicid;
|
|
|
|
int cpu = hard_smp_processor_id();
|
2016-10-03 17:07:12 +00:00
|
|
|
|
2016-10-22 02:18:04 +00:00
|
|
|
pr_warn("Boot CPU (id %d) not listed by BIOS\n", cpu);
|
2016-10-03 17:07:12 +00:00
|
|
|
|
2016-10-22 02:18:04 +00:00
|
|
|
/* Make sure boot cpu is enumerated */
|
|
|
|
if (apic->cpu_present_to_apicid(0) == BAD_APICID &&
|
|
|
|
apic->apic_id_valid(apicid))
|
|
|
|
generic_processor_info(apicid, boot_cpu_apic_version);
|
|
|
|
}
|
2016-10-03 17:07:12 +00:00
|
|
|
|
|
|
|
if (!num_processors)
|
|
|
|
num_processors = 1;
|
|
|
|
}
|
2008-07-03 01:54:40 +00:00
|
|
|
|
2010-05-24 19:13:17 +00:00
|
|
|
i = setup_max_cpus ?: 1;
|
|
|
|
if (setup_possible_cpus == -1) {
|
|
|
|
possible = num_processors;
|
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
if (setup_max_cpus)
|
|
|
|
possible += disabled_cpus;
|
|
|
|
#else
|
|
|
|
if (possible > i)
|
|
|
|
possible = i;
|
|
|
|
#endif
|
|
|
|
} else
|
2008-12-17 23:21:39 +00:00
|
|
|
possible = setup_possible_cpus;
|
|
|
|
|
2009-01-01 02:08:45 +00:00
|
|
|
total_cpus = max_t(int, possible, num_processors + disabled_cpus);
|
|
|
|
|
2010-02-10 09:20:37 +00:00
|
|
|
/* nr_cpu_ids could be reduced via nr_cpus= */
|
|
|
|
if (possible > nr_cpu_ids) {
|
2017-09-08 23:14:18 +00:00
|
|
|
pr_warn("%d Processors exceeds NR_CPUS limit of %u\n",
|
2010-02-10 09:20:37 +00:00
|
|
|
possible, nr_cpu_ids);
|
|
|
|
possible = nr_cpu_ids;
|
2008-12-17 23:21:39 +00:00
|
|
|
}
|
2008-03-03 17:12:42 +00:00
|
|
|
|
2010-05-24 19:13:17 +00:00
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
if (!setup_max_cpus)
|
|
|
|
#endif
|
|
|
|
if (possible > i) {
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_warn("%d Processors exceeds max_cpus limit of %u\n",
|
2010-05-24 19:13:17 +00:00
|
|
|
possible, setup_max_cpus);
|
|
|
|
possible = i;
|
|
|
|
}
|
|
|
|
|
2016-12-13 18:32:28 +00:00
|
|
|
nr_cpu_ids = possible;
|
|
|
|
|
2012-05-22 02:50:07 +00:00
|
|
|
pr_info("Allowing %d CPUs, %d hotplug CPUs\n",
|
2008-03-03 17:12:42 +00:00
|
|
|
possible, max_t(int, possible - num_processors, 0));
|
|
|
|
|
2016-12-13 18:32:28 +00:00
|
|
|
reset_cpu_possible_mask();
|
|
|
|
|
2008-03-03 17:12:42 +00:00
|
|
|
for (i = 0; i < possible; i++)
|
2009-01-04 13:18:03 +00:00
|
|
|
set_cpu_possible(i, true);
|
2008-03-03 17:12:42 +00:00
|
|
|
}
|
2008-03-03 17:13:07 +00:00
|
|
|
|
2008-09-29 22:29:42 +00:00
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
|
2016-05-20 00:09:55 +00:00
|
|
|
/* Recompute SMT state for all CPUs on offline */
|
|
|
|
static void recompute_smt_state(void)
|
|
|
|
{
|
|
|
|
int max_threads, cpu;
|
|
|
|
|
|
|
|
max_threads = 0;
|
|
|
|
for_each_online_cpu (cpu) {
|
|
|
|
int threads = cpumask_weight(topology_sibling_cpumask(cpu));
|
|
|
|
|
|
|
|
if (threads > max_threads)
|
|
|
|
max_threads = threads;
|
|
|
|
}
|
|
|
|
__max_smt_threads = max_threads;
|
|
|
|
}
|
|
|
|
|
2008-09-29 22:29:42 +00:00
|
|
|
static void remove_siblinginfo(int cpu)
|
|
|
|
{
|
|
|
|
int sibling;
|
|
|
|
struct cpuinfo_x86 *c = &cpu_data(cpu);
|
|
|
|
|
2015-05-26 13:11:35 +00:00
|
|
|
for_each_cpu(sibling, topology_core_cpumask(cpu)) {
|
|
|
|
cpumask_clear_cpu(cpu, topology_core_cpumask(sibling));
|
2008-09-29 22:29:42 +00:00
|
|
|
/*/
|
|
|
|
* last thread sibling in this cpu core going down
|
|
|
|
*/
|
2015-05-26 13:11:35 +00:00
|
|
|
if (cpumask_weight(topology_sibling_cpumask(cpu)) == 1)
|
2008-09-29 22:29:42 +00:00
|
|
|
cpu_data(sibling).booted_cores--;
|
|
|
|
}
|
|
|
|
|
2015-05-26 13:11:35 +00:00
|
|
|
for_each_cpu(sibling, topology_sibling_cpumask(cpu))
|
|
|
|
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
|
sched: Fix unreleased llc_shared_mask bit during CPU hotplug
The following bug can be triggered by hot adding and removing a large number of
xen domain0's vcpus repeatedly:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000004 IP: [..] find_busiest_group
PGD 5a9d5067 PUD 13067 PMD 0
Oops: 0000 [#3] SMP
[...]
Call Trace:
load_balance
? _raw_spin_unlock_irqrestore
idle_balance
__schedule
schedule
schedule_timeout
? lock_timer_base
schedule_timeout_uninterruptible
msleep
lock_device_hotplug_sysfs
online_store
dev_attr_store
sysfs_write_file
vfs_write
SyS_write
system_call_fastpath
Last level cache shared mask is built during CPU up and the
build_sched_domain() routine takes advantage of it to setup
the sched domain CPU topology.
However, llc_shared_mask is not released during CPU disable,
which leads to an invalid sched domainCPU topology.
This patch fix it by releasing the llc_shared_mask correctly
during CPU disable.
Yasuaki also reported that this can happen on real hardware:
https://lkml.org/lkml/2014/7/22/1018
His case is here:
==
Here is an example on my system.
My system has 4 sockets and each socket has 15 cores and HT is
enabled. In this case, each core of sockes is numbered as
follows:
| CPU#
Socket#0 | 0-14 , 60-74
Socket#1 | 15-29, 75-89
Socket#2 | 30-44, 90-104
Socket#3 | 45-59, 105-119
Then llc_shared_mask of CPU#30 has 0x3fff80000001fffc0000000.
It means that last level cache of Socket#2 is shared with
CPU#30-44 and 90-104.
When hot-removing socket#2 and #3, each core of sockets is
numbered as follows:
| CPU#
Socket#0 | 0-14 , 60-74
Socket#1 | 15-29, 75-89
But llc_shared_mask is not cleared. So llc_shared_mask of CPU#30
remains having 0x3fff80000001fffc0000000.
After that, when hot-adding socket#2 and #3, each core of
sockets is numbered as follows:
| CPU#
Socket#0 | 0-14 , 60-74
Socket#1 | 15-29, 75-89
Socket#2 | 30-59
Socket#3 | 90-119
Then llc_shared_mask of CPU#30 becomes
0x3fff8000fffffffc0000000. It means that last level cache of
Socket#2 is shared with CPU#30-59 and 90-104. So the mask has
the wrong value.
Signed-off-by: Wanpeng Li <wanpeng.li@linux.intel.com>
Tested-by: Linn Crosetto <linn@hp.com>
Reviewed-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Toshi Kani <toshi.kani@hp.com>
Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: <stable@vger.kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: Steven Rostedt <srostedt@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1411547885-48165-1-git-send-email-wanpeng.li@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-09-24 08:38:05 +00:00
|
|
|
for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
|
|
|
|
cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
|
|
|
|
cpumask_clear(cpu_llc_shared_mask(cpu));
|
2015-05-26 13:11:35 +00:00
|
|
|
cpumask_clear(topology_sibling_cpumask(cpu));
|
|
|
|
cpumask_clear(topology_core_cpumask(cpu));
|
2008-09-29 22:29:42 +00:00
|
|
|
c->cpu_core_id = 0;
|
2018-02-21 20:50:36 +00:00
|
|
|
c->booted_cores = 0;
|
2009-01-04 13:18:03 +00:00
|
|
|
cpumask_clear_cpu(cpu, cpu_sibling_setup_mask);
|
2016-05-20 00:09:55 +00:00
|
|
|
recompute_smt_state();
|
2008-09-29 22:29:42 +00:00
|
|
|
}
|
|
|
|
|
2015-07-20 16:32:53 +00:00
|
|
|
static void remove_cpu_from_maps(int cpu)
|
2008-03-03 17:13:07 +00:00
|
|
|
{
|
2009-01-04 13:18:03 +00:00
|
|
|
set_cpu_online(cpu, false);
|
|
|
|
cpumask_clear_cpu(cpu, cpu_callout_mask);
|
|
|
|
cpumask_clear_cpu(cpu, cpu_callin_mask);
|
2008-03-03 17:13:07 +00:00
|
|
|
/* was set by cpu_init() */
|
2009-01-04 13:18:03 +00:00
|
|
|
cpumask_clear_cpu(cpu, cpu_initialized_mask);
|
x86: cleanup early per cpu variables/accesses v4
* Introduce a new PER_CPU macro called "EARLY_PER_CPU". This is
used by some per_cpu variables that are initialized and accessed
before there are per_cpu areas allocated.
["Early" in respect to per_cpu variables is "earlier than the per_cpu
areas have been setup".]
This patchset adds these new macros:
DEFINE_EARLY_PER_CPU(_type, _name, _initvalue)
EXPORT_EARLY_PER_CPU_SYMBOL(_name)
DECLARE_EARLY_PER_CPU(_type, _name)
early_per_cpu_ptr(_name)
early_per_cpu_map(_name, _idx)
early_per_cpu(_name, _cpu)
The DEFINE macro defines the per_cpu variable as well as the early
map and pointer. It also initializes the per_cpu variable and map
elements to "_initvalue". The early_* macros provide access to
the initial map (usually setup during system init) and the early
pointer. This pointer is initialized to point to the early map
but is then NULL'ed when the actual per_cpu areas are setup. After
that the per_cpu variable is the correct access to the variable.
The early_per_cpu() macro is not very efficient but does show how to
access the variable if you have a function that can be called both
"early" and "late". It tests the early ptr to be NULL, and if not
then it's still valid. Otherwise, the per_cpu variable is used
instead:
#define early_per_cpu(_name, _cpu) \
(early_per_cpu_ptr(_name) ? \
early_per_cpu_ptr(_name)[_cpu] : \
per_cpu(_name, _cpu))
A better method is to actually check the pointer manually. In the
case below, numa_set_node can be called both "early" and "late":
void __cpuinit numa_set_node(int cpu, int node)
{
int *cpu_to_node_map = early_per_cpu_ptr(x86_cpu_to_node_map);
if (cpu_to_node_map)
cpu_to_node_map[cpu] = node;
else
per_cpu(x86_cpu_to_node_map, cpu) = node;
}
* Add a flag "arch_provides_topology_pointers" that indicates pointers
to topology cpumask_t maps are available. Otherwise, use the function
returning the cpumask_t value. This is useful if cpumask_t set size
is very large to avoid copying data on to/off of the stack.
* The coverage of CONFIG_DEBUG_PER_CPU_MAPS has been increased while
the non-debug case has been optimized a bit.
* Remove an unreferenced compiler warning in drivers/base/topology.c
* Clean up #ifdef in setup.c
For inclusion into sched-devel/latest tree.
Based on:
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
+ sched-devel/latest .../mingo/linux-2.6-sched-devel.git
Signed-off-by: Mike Travis <travis@sgi.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2008-05-12 19:21:12 +00:00
|
|
|
numa_remove_cpu(cpu);
|
2008-03-03 17:13:07 +00:00
|
|
|
}
|
|
|
|
|
2008-08-22 10:52:14 +00:00
|
|
|
void cpu_disable_common(void)
|
2008-03-03 17:13:07 +00:00
|
|
|
{
|
|
|
|
int cpu = smp_processor_id();
|
|
|
|
|
|
|
|
remove_siblinginfo(cpu);
|
|
|
|
|
|
|
|
/* It's now safe to remove this processor from the online map */
|
2008-08-09 22:09:02 +00:00
|
|
|
lock_vector_lock();
|
2008-03-03 17:13:07 +00:00
|
|
|
remove_cpu_from_maps(cpu);
|
2008-08-09 22:09:02 +00:00
|
|
|
unlock_vector_lock();
|
2008-12-17 01:33:58 +00:00
|
|
|
fixup_irqs();
|
2017-09-13 21:29:38 +00:00
|
|
|
lapic_offline();
|
2008-08-22 10:52:14 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
int native_cpu_disable(void)
|
|
|
|
{
|
x86: Add check for number of available vectors before CPU down
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=64791
When a cpu is downed on a system, the irqs on the cpu are assigned to
other cpus. It is possible, however, that when a cpu is downed there
aren't enough free vectors on the remaining cpus to account for the
vectors from the cpu that is being downed.
This results in an interesting "overflow" condition where irqs are
"assigned" to a CPU but are not handled.
For example, when downing cpus on a 1-64 logical processor system:
<snip>
[ 232.021745] smpboot: CPU 61 is now offline
[ 238.480275] smpboot: CPU 62 is now offline
[ 245.991080] ------------[ cut here ]------------
[ 245.996270] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:264 dev_watchdog+0x246/0x250()
[ 246.005688] NETDEV WATCHDOG: p786p1 (ixgbe): transmit queue 0 timed out
[ 246.013070] Modules linked in: lockd sunrpc iTCO_wdt iTCO_vendor_support sb_edac ixgbe microcode e1000e pcspkr joydev edac_core lpc_ich ioatdma ptp mdio mfd_core i2c_i801 dca pps_core i2c_core wmi acpi_cpufreq isci libsas scsi_transport_sas
[ 246.037633] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.12.0+ #14
[ 246.044451] Hardware name: Intel Corporation S4600LH ........../SVRBD-ROW_T, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 246.057371] 0000000000000009 ffff88081fa03d40 ffffffff8164fbf6 ffff88081fa0ee48
[ 246.065728] ffff88081fa03d90 ffff88081fa03d80 ffffffff81054ecc ffff88081fa13040
[ 246.074073] 0000000000000000 ffff88200cce0000 0000000000000040 0000000000000000
[ 246.082430] Call Trace:
[ 246.085174] <IRQ> [<ffffffff8164fbf6>] dump_stack+0x46/0x58
[ 246.091633] [<ffffffff81054ecc>] warn_slowpath_common+0x8c/0xc0
[ 246.098352] [<ffffffff81054fb6>] warn_slowpath_fmt+0x46/0x50
[ 246.104786] [<ffffffff815710d6>] dev_watchdog+0x246/0x250
[ 246.110923] [<ffffffff81570e90>] ? dev_deactivate_queue.constprop.31+0x80/0x80
[ 246.119097] [<ffffffff8106092a>] call_timer_fn+0x3a/0x110
[ 246.125224] [<ffffffff8106280f>] ? update_process_times+0x6f/0x80
[ 246.132137] [<ffffffff81570e90>] ? dev_deactivate_queue.constprop.31+0x80/0x80
[ 246.140308] [<ffffffff81061db0>] run_timer_softirq+0x1f0/0x2a0
[ 246.146933] [<ffffffff81059a80>] __do_softirq+0xe0/0x220
[ 246.152976] [<ffffffff8165fedc>] call_softirq+0x1c/0x30
[ 246.158920] [<ffffffff810045f5>] do_softirq+0x55/0x90
[ 246.164670] [<ffffffff81059d35>] irq_exit+0xa5/0xb0
[ 246.170227] [<ffffffff8166062a>] smp_apic_timer_interrupt+0x4a/0x60
[ 246.177324] [<ffffffff8165f40a>] apic_timer_interrupt+0x6a/0x70
[ 246.184041] <EOI> [<ffffffff81505a1b>] ? cpuidle_enter_state+0x5b/0xe0
[ 246.191559] [<ffffffff81505a17>] ? cpuidle_enter_state+0x57/0xe0
[ 246.198374] [<ffffffff81505b5d>] cpuidle_idle_call+0xbd/0x200
[ 246.204900] [<ffffffff8100b7ae>] arch_cpu_idle+0xe/0x30
[ 246.210846] [<ffffffff810a47b0>] cpu_startup_entry+0xd0/0x250
[ 246.217371] [<ffffffff81646b47>] rest_init+0x77/0x80
[ 246.223028] [<ffffffff81d09e8e>] start_kernel+0x3ee/0x3fb
[ 246.229165] [<ffffffff81d0989f>] ? repair_env_string+0x5e/0x5e
[ 246.235787] [<ffffffff81d095a5>] x86_64_start_reservations+0x2a/0x2c
[ 246.242990] [<ffffffff81d0969f>] x86_64_start_kernel+0xf8/0xfc
[ 246.249610] ---[ end trace fb74fdef54d79039 ]---
[ 246.254807] ixgbe 0000:c2:00.0 p786p1: initiating reset due to tx timeout
[ 246.262489] ixgbe 0000:c2:00.0 p786p1: Reset adapter
Last login: Mon Nov 11 08:35:14 from 10.18.17.119
[root@(none) ~]# [ 246.792676] ixgbe 0000:c2:00.0 p786p1: detected SFP+: 5
[ 249.231598] ixgbe 0000:c2:00.0 p786p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 246.792676] ixgbe 0000:c2:00.0 p786p1: detected SFP+: 5
[ 249.231598] ixgbe 0000:c2:00.0 p786p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
(last lines keep repeating. ixgbe driver is dead until module reload.)
If the downed cpu has more vectors than are free on the remaining cpus on the
system, it is possible that some vectors are "orphaned" even though they are
assigned to a cpu. In this case, since the ixgbe driver had a watchdog, the
watchdog fired and notified that something was wrong.
This patch adds a function, check_vectors(), to compare the number of vectors
on the CPU going down and compares it to the number of vectors available on
the system. If there aren't enough vectors for the CPU to go down, an
error is returned and propogated back to userspace.
v2: Do not need to look at percpu irqs
v3: Need to check affinity to prevent counting of MSIs in IOAPIC Lowest
Priority Mode
v4: Additional changes suggested by Gong Chen.
v5/v6/v7/v8: Updated comment text
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Link: http://lkml.kernel.org/r/1389613861-3853-1-git-send-email-prarit@redhat.com
Reviewed-by: Gong Chen <gong.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Seiji Aguchi <seiji.aguchi@hds.com>
Cc: Yang Zhang <yang.z.zhang@Intel.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Janet Morgan <janet.morgan@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Ruiv Wang <ruiv.wang@gmail.com>
Cc: Gong Chen <gong.chen@linux.intel.com>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Cc: <stable@vger.kernel.org>
2014-01-13 11:51:01 +00:00
|
|
|
int ret;
|
|
|
|
|
2017-09-13 21:29:53 +00:00
|
|
|
ret = lapic_can_unplug_cpu();
|
x86: Add check for number of available vectors before CPU down
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=64791
When a cpu is downed on a system, the irqs on the cpu are assigned to
other cpus. It is possible, however, that when a cpu is downed there
aren't enough free vectors on the remaining cpus to account for the
vectors from the cpu that is being downed.
This results in an interesting "overflow" condition where irqs are
"assigned" to a CPU but are not handled.
For example, when downing cpus on a 1-64 logical processor system:
<snip>
[ 232.021745] smpboot: CPU 61 is now offline
[ 238.480275] smpboot: CPU 62 is now offline
[ 245.991080] ------------[ cut here ]------------
[ 245.996270] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:264 dev_watchdog+0x246/0x250()
[ 246.005688] NETDEV WATCHDOG: p786p1 (ixgbe): transmit queue 0 timed out
[ 246.013070] Modules linked in: lockd sunrpc iTCO_wdt iTCO_vendor_support sb_edac ixgbe microcode e1000e pcspkr joydev edac_core lpc_ich ioatdma ptp mdio mfd_core i2c_i801 dca pps_core i2c_core wmi acpi_cpufreq isci libsas scsi_transport_sas
[ 246.037633] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.12.0+ #14
[ 246.044451] Hardware name: Intel Corporation S4600LH ........../SVRBD-ROW_T, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 246.057371] 0000000000000009 ffff88081fa03d40 ffffffff8164fbf6 ffff88081fa0ee48
[ 246.065728] ffff88081fa03d90 ffff88081fa03d80 ffffffff81054ecc ffff88081fa13040
[ 246.074073] 0000000000000000 ffff88200cce0000 0000000000000040 0000000000000000
[ 246.082430] Call Trace:
[ 246.085174] <IRQ> [<ffffffff8164fbf6>] dump_stack+0x46/0x58
[ 246.091633] [<ffffffff81054ecc>] warn_slowpath_common+0x8c/0xc0
[ 246.098352] [<ffffffff81054fb6>] warn_slowpath_fmt+0x46/0x50
[ 246.104786] [<ffffffff815710d6>] dev_watchdog+0x246/0x250
[ 246.110923] [<ffffffff81570e90>] ? dev_deactivate_queue.constprop.31+0x80/0x80
[ 246.119097] [<ffffffff8106092a>] call_timer_fn+0x3a/0x110
[ 246.125224] [<ffffffff8106280f>] ? update_process_times+0x6f/0x80
[ 246.132137] [<ffffffff81570e90>] ? dev_deactivate_queue.constprop.31+0x80/0x80
[ 246.140308] [<ffffffff81061db0>] run_timer_softirq+0x1f0/0x2a0
[ 246.146933] [<ffffffff81059a80>] __do_softirq+0xe0/0x220
[ 246.152976] [<ffffffff8165fedc>] call_softirq+0x1c/0x30
[ 246.158920] [<ffffffff810045f5>] do_softirq+0x55/0x90
[ 246.164670] [<ffffffff81059d35>] irq_exit+0xa5/0xb0
[ 246.170227] [<ffffffff8166062a>] smp_apic_timer_interrupt+0x4a/0x60
[ 246.177324] [<ffffffff8165f40a>] apic_timer_interrupt+0x6a/0x70
[ 246.184041] <EOI> [<ffffffff81505a1b>] ? cpuidle_enter_state+0x5b/0xe0
[ 246.191559] [<ffffffff81505a17>] ? cpuidle_enter_state+0x57/0xe0
[ 246.198374] [<ffffffff81505b5d>] cpuidle_idle_call+0xbd/0x200
[ 246.204900] [<ffffffff8100b7ae>] arch_cpu_idle+0xe/0x30
[ 246.210846] [<ffffffff810a47b0>] cpu_startup_entry+0xd0/0x250
[ 246.217371] [<ffffffff81646b47>] rest_init+0x77/0x80
[ 246.223028] [<ffffffff81d09e8e>] start_kernel+0x3ee/0x3fb
[ 246.229165] [<ffffffff81d0989f>] ? repair_env_string+0x5e/0x5e
[ 246.235787] [<ffffffff81d095a5>] x86_64_start_reservations+0x2a/0x2c
[ 246.242990] [<ffffffff81d0969f>] x86_64_start_kernel+0xf8/0xfc
[ 246.249610] ---[ end trace fb74fdef54d79039 ]---
[ 246.254807] ixgbe 0000:c2:00.0 p786p1: initiating reset due to tx timeout
[ 246.262489] ixgbe 0000:c2:00.0 p786p1: Reset adapter
Last login: Mon Nov 11 08:35:14 from 10.18.17.119
[root@(none) ~]# [ 246.792676] ixgbe 0000:c2:00.0 p786p1: detected SFP+: 5
[ 249.231598] ixgbe 0000:c2:00.0 p786p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 246.792676] ixgbe 0000:c2:00.0 p786p1: detected SFP+: 5
[ 249.231598] ixgbe 0000:c2:00.0 p786p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
(last lines keep repeating. ixgbe driver is dead until module reload.)
If the downed cpu has more vectors than are free on the remaining cpus on the
system, it is possible that some vectors are "orphaned" even though they are
assigned to a cpu. In this case, since the ixgbe driver had a watchdog, the
watchdog fired and notified that something was wrong.
This patch adds a function, check_vectors(), to compare the number of vectors
on the CPU going down and compares it to the number of vectors available on
the system. If there aren't enough vectors for the CPU to go down, an
error is returned and propogated back to userspace.
v2: Do not need to look at percpu irqs
v3: Need to check affinity to prevent counting of MSIs in IOAPIC Lowest
Priority Mode
v4: Additional changes suggested by Gong Chen.
v5/v6/v7/v8: Updated comment text
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Link: http://lkml.kernel.org/r/1389613861-3853-1-git-send-email-prarit@redhat.com
Reviewed-by: Gong Chen <gong.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Seiji Aguchi <seiji.aguchi@hds.com>
Cc: Yang Zhang <yang.z.zhang@Intel.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Janet Morgan <janet.morgan@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Ruiv Wang <ruiv.wang@gmail.com>
Cc: Gong Chen <gong.chen@linux.intel.com>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Cc: <stable@vger.kernel.org>
2014-01-13 11:51:01 +00:00
|
|
|
if (ret)
|
|
|
|
return ret;
|
|
|
|
|
2008-08-22 10:52:14 +00:00
|
|
|
clear_local_APIC();
|
|
|
|
cpu_disable_common();
|
2014-08-26 07:43:45 +00:00
|
|
|
|
2008-03-03 17:13:07 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2015-02-25 19:42:15 +00:00
|
|
|
int common_cpu_die(unsigned int cpu)
|
2014-10-31 15:49:32 +00:00
|
|
|
{
|
2015-02-25 19:42:15 +00:00
|
|
|
int ret = 0;
|
2014-10-31 15:49:32 +00:00
|
|
|
|
2008-03-03 17:13:07 +00:00
|
|
|
/* We don't do anything here: idle task is faking death itself. */
|
2014-10-31 15:49:32 +00:00
|
|
|
|
2014-08-26 07:43:45 +00:00
|
|
|
/* They ack this in play_dead() by setting CPU_DEAD */
|
2015-02-25 19:42:15 +00:00
|
|
|
if (cpu_wait_death(cpu, 5)) {
|
2014-08-26 07:43:45 +00:00
|
|
|
if (system_state == SYSTEM_RUNNING)
|
|
|
|
pr_info("CPU %u is now offline\n", cpu);
|
|
|
|
} else {
|
|
|
|
pr_err("CPU %u didn't die...\n", cpu);
|
2015-02-25 19:42:15 +00:00
|
|
|
ret = -1;
|
2008-03-03 17:13:07 +00:00
|
|
|
}
|
2015-02-25 19:42:15 +00:00
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
void native_cpu_die(unsigned int cpu)
|
|
|
|
{
|
|
|
|
common_cpu_die(cpu);
|
2008-03-03 17:13:07 +00:00
|
|
|
}
|
2008-08-22 10:52:13 +00:00
|
|
|
|
|
|
|
void play_dead_common(void)
|
|
|
|
{
|
|
|
|
idle_task_exit();
|
|
|
|
|
|
|
|
/* Ack it */
|
2015-02-25 19:42:15 +00:00
|
|
|
(void)cpu_report_death();
|
2008-08-22 10:52:13 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* With physical CPU hotplug, we should halt the cpu
|
|
|
|
*/
|
|
|
|
local_irq_disable();
|
|
|
|
}
|
|
|
|
|
2012-11-14 12:36:53 +00:00
|
|
|
static bool wakeup_cpu0(void)
|
|
|
|
{
|
|
|
|
if (smp_processor_id() == 0 && enable_start_cpu0)
|
|
|
|
return true;
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2010-09-17 22:39:11 +00:00
|
|
|
/*
|
|
|
|
* We need to flush the caches before going to sleep, lest we have
|
|
|
|
* dirty data in our caches when we come back up.
|
|
|
|
*/
|
|
|
|
static inline void mwait_play_dead(void)
|
|
|
|
{
|
|
|
|
unsigned int eax, ebx, ecx, edx;
|
|
|
|
unsigned int highest_cstate = 0;
|
|
|
|
unsigned int highest_subcstate = 0;
|
2010-09-20 20:04:45 +00:00
|
|
|
void *mwait_ptr;
|
2013-03-04 20:16:16 +00:00
|
|
|
int i;
|
2010-09-17 22:39:11 +00:00
|
|
|
|
2013-02-10 06:38:39 +00:00
|
|
|
if (!this_cpu_has(X86_FEATURE_MWAIT))
|
2010-09-17 22:39:11 +00:00
|
|
|
return;
|
2014-02-27 16:31:30 +00:00
|
|
|
if (!this_cpu_has(X86_FEATURE_CLFLUSH))
|
2010-09-20 20:04:45 +00:00
|
|
|
return;
|
2010-12-18 15:30:05 +00:00
|
|
|
if (__this_cpu_read(cpu_info.cpuid_level) < CPUID_MWAIT_LEAF)
|
2010-09-17 22:39:11 +00:00
|
|
|
return;
|
|
|
|
|
|
|
|
eax = CPUID_MWAIT_LEAF;
|
|
|
|
ecx = 0;
|
|
|
|
native_cpuid(&eax, &ebx, &ecx, &edx);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* eax will be 0 if EDX enumeration is not valid.
|
|
|
|
* Initialized below to cstate, sub_cstate value when EDX is valid.
|
|
|
|
*/
|
|
|
|
if (!(ecx & CPUID5_ECX_EXTENSIONS_SUPPORTED)) {
|
|
|
|
eax = 0;
|
|
|
|
} else {
|
|
|
|
edx >>= MWAIT_SUBSTATE_SIZE;
|
|
|
|
for (i = 0; i < 7 && edx; i++, edx >>= MWAIT_SUBSTATE_SIZE) {
|
|
|
|
if (edx & MWAIT_SUBSTATE_MASK) {
|
|
|
|
highest_cstate = i;
|
|
|
|
highest_subcstate = edx & MWAIT_SUBSTATE_MASK;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
eax = (highest_cstate << MWAIT_SUBSTATE_SIZE) |
|
|
|
|
(highest_subcstate - 1);
|
|
|
|
}
|
|
|
|
|
2010-09-20 20:04:45 +00:00
|
|
|
/*
|
|
|
|
* This should be a memory location in a cache line which is
|
|
|
|
* unlikely to be touched by other processors. The actual
|
|
|
|
* content is immaterial as it is not actually modified in any way.
|
|
|
|
*/
|
|
|
|
mwait_ptr = ¤t_thread_info()->flags;
|
|
|
|
|
2010-09-18 00:06:46 +00:00
|
|
|
wbinvd();
|
|
|
|
|
2010-09-17 22:39:11 +00:00
|
|
|
while (1) {
|
2010-09-20 20:04:45 +00:00
|
|
|
/*
|
|
|
|
* The CLFLUSH is a workaround for erratum AAI65 for
|
|
|
|
* the Xeon 7400 series. It's not clear it is actually
|
|
|
|
* needed, but it should be harmless in either case.
|
|
|
|
* The WBINVD is insufficient due to the spurious-wakeup
|
|
|
|
* case where we return around the loop.
|
|
|
|
*/
|
2013-12-19 20:30:03 +00:00
|
|
|
mb();
|
2010-09-20 20:04:45 +00:00
|
|
|
clflush(mwait_ptr);
|
2013-12-19 20:30:03 +00:00
|
|
|
mb();
|
2010-09-20 20:04:45 +00:00
|
|
|
__monitor(mwait_ptr, 0, 0);
|
2010-09-17 22:39:11 +00:00
|
|
|
mb();
|
|
|
|
__mwait(eax, 0);
|
2012-11-14 12:36:53 +00:00
|
|
|
/*
|
|
|
|
* If NMI wants to wake up CPU0, start CPU0.
|
|
|
|
*/
|
|
|
|
if (wakeup_cpu0())
|
|
|
|
start_cpu0();
|
2010-09-17 22:39:11 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
x86 / hibernate: Use hlt_play_dead() when resuming from hibernation
On Intel hardware, native_play_dead() uses mwait_play_dead() by
default and only falls back to the other methods if that fails.
That also happens during resume from hibernation, when the restore
(boot) kernel runs disable_nonboot_cpus() to take all of the CPUs
except for the boot one offline.
However, that is problematic, because the address passed to
__monitor() in mwait_play_dead() is likely to be written to in the
last phase of hibernate image restoration and that causes the "dead"
CPU to start executing instructions again. Unfortunately, the page
containing the address in that CPU's instruction pointer may not be
valid any more at that point.
First, that page may have been overwritten with image kernel memory
contents already, so the instructions the CPU attempts to execute may
simply be invalid. Second, the page tables previously used by that
CPU may have been overwritten by image kernel memory contents, so the
address in its instruction pointer is impossible to resolve then.
A report from Varun Koyyalagunta and investigation carried out by
Chen Yu show that the latter sometimes happens in practice.
To prevent it from happening, temporarily change the smp_ops.play_dead
pointer during resume from hibernation so that it points to a special
"play dead" routine which uses hlt_play_dead() and avoids the
inadvertent "revivals" of "dead" CPUs this way.
A slightly unpleasant consequence of this change is that if the
system is hibernated with one or more CPUs offline, it will generally
draw more power after resume than it did before hibernation, because
the physical state entered by CPUs via hlt_play_dead() is higher-power
than the mwait_play_dead() one in the majority of cases. It is
possible to work around this, but it is unclear how much of a problem
that's going to be in practice, so the workaround will be implemented
later if it turns out to be necessary.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=106371
Reported-by: Varun Koyyalagunta <cpudebug@centtech.com>
Original-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
2016-07-14 01:55:23 +00:00
|
|
|
void hlt_play_dead(void)
|
2010-09-17 22:39:11 +00:00
|
|
|
{
|
2010-12-18 15:30:05 +00:00
|
|
|
if (__this_cpu_read(cpu_info.x86) >= 4)
|
2010-09-18 00:06:46 +00:00
|
|
|
wbinvd();
|
|
|
|
|
2010-09-17 22:39:11 +00:00
|
|
|
while (1) {
|
|
|
|
native_halt();
|
2012-11-14 12:36:53 +00:00
|
|
|
/*
|
|
|
|
* If NMI wants to wake up CPU0, start CPU0.
|
|
|
|
*/
|
|
|
|
if (wakeup_cpu0())
|
|
|
|
start_cpu0();
|
2010-09-17 22:39:11 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2008-08-22 10:52:13 +00:00
|
|
|
void native_play_dead(void)
|
|
|
|
{
|
|
|
|
play_dead_common();
|
2009-07-01 02:31:07 +00:00
|
|
|
tboot_shutdown(TB_SHUTDOWN_WFS);
|
2010-09-17 22:39:11 +00:00
|
|
|
|
|
|
|
mwait_play_dead(); /* Only returns on failure */
|
2012-03-13 18:55:09 +00:00
|
|
|
if (cpuidle_play_dead())
|
|
|
|
hlt_play_dead();
|
2008-08-22 10:52:13 +00:00
|
|
|
}
|
|
|
|
|
2008-03-03 17:13:07 +00:00
|
|
|
#else /* ... !CONFIG_HOTPLUG_CPU */
|
2008-08-22 10:52:11 +00:00
|
|
|
int native_cpu_disable(void)
|
2008-03-03 17:13:07 +00:00
|
|
|
{
|
|
|
|
return -ENOSYS;
|
|
|
|
}
|
|
|
|
|
2008-08-22 10:52:11 +00:00
|
|
|
void native_cpu_die(unsigned int cpu)
|
2008-03-03 17:13:07 +00:00
|
|
|
{
|
|
|
|
/* We said "no" in __cpu_disable */
|
|
|
|
BUG();
|
|
|
|
}
|
2008-08-22 10:52:13 +00:00
|
|
|
|
|
|
|
void native_play_dead(void)
|
|
|
|
{
|
|
|
|
BUG();
|
|
|
|
}
|
|
|
|
|
2008-03-03 17:12:42 +00:00
|
|
|
#endif
|