Although warnings about close calls are printed by kvm-recheck.sh,
kvm-find-errors.sh currently ignores them. This could easily result
in someone failing to investigate close calls, so this commit makes
them visible to kvm-find-errors.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The torture scripting currently builds each kernel from a set of parallel
runs in its own build directory. This can waste quite a bit of space when
running large numbers of concurrent scenarios, and pointlessly given that
the builds are run sequentially (albeit with a largish -j number passed to
"make"). This commit therefore places all build-command output in the
results directory, then does all builds in a single "b1" build directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, kvm-find-errors.sh looks only for build errors ("error:"),
so this commit makes it also locate build warnings ("warning:").
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
With the addition of the end-of-test state, it is not uncommon for the
kvm.sh summary lines to overflow 80 characters. This commit therefore
applies abbreviations in order to make the line fit into 80 characters
with high probability.
And yes, I did make heavy use of punched cards back in the day, so 80
columns it is for my xterms! ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds the end-of-test test, if present in the console output,
to the kvm.sh test summary that is printed by kvm-recheck.sh. Note that
this only applies to rcutorture console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The rcutorture scripting scans the console output twice, once to look
for various sorts of hangs and again to find warnings and panics.
Unfortunately, only the output of the second scan gets written to the
console.log.diags file, which can cause hangs to be overlooked.
This commit therefore folds the parse-torture.sh script (which looks
for hangs) into the parse-console.sh script (which looks for warnings
and panics). This allows both types of failure information to be
added to console.log.diags, while still reliably removing this file
when it proves to be empty.
This also fixes a long-standing bug where rcuperf log files would
unconditionally complain about a hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds a script that allows viewing the build and/or
console output from failed rcutorture, locktorture, or rcuperf runs.
This replaces a time-honored but inefficient manual procedure that uses
cut and paste.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The rcuperf trace-event processing counted every "done" trace event
as a piggyback, which is incorrect because the task that started the
grace period didn't piggyback at all. This commit fixes this problem
by recording the task that started a given grace period and ignoring
that task's "done" record for that grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf event-trace processing assumes that expedited grace periods
start and end on the same task, an assumption that was violated by moving
expedited grace-period processing to workqueues. This commit removes
this now-fallacious assumption from rcuperf's event-trace processing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The purpose of jitter is to expose concurrency bugs due to invalid
assumptions about forward progress. There is usually little point
in jitter when measuring performance. This commit therefore defaults
jitter off when running rcuperf. You can override this by specifying
the kvm.sh "--jitter" argument -after- the "--torture rcuperf"
argument. No idea why you would want this, but if you do, that is
how you do it.
One example of a conccurrency bug that this jitter might expose is one
in which the developer assumed that a given short region of code would be
guaranteed to execute within some short time limit. Such assumptions are
invalid in virtualized environments because the hupervisor can preempt
the guest OS at any point, even when the guest OS thinks that it has
disabled interrupts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 512 megabyte memory size has served quite well, but more memory
is required when using large trace buffers on large systems. This
commit therefore adds a --memory argument to the kvm.sh script, which
allows the memory size to be specified on the command line, for example,
"--memory 768", --memory 800M", or "--memory 2G".
Reported-by: Sebastian Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds support of the qemu command qemu-system-aarch64
to rcutorture.
Signed-off-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm.sh header comment is a bit of a relic, so this commit brings
it up to date.
Reported-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Check for build-directory existence and write permissions are provided in
both 'kvm-test-1-run.sh' an 'kvm-build.sh'. Because the 'kvm-build.sh'
is dependent on 'kvm-test-1-run.sh' ('kvm-build.sh' uses variables that
defined from its caller.), these checks are unnecessarily duplicated.
This commit therefore removes the check in from the 'kvm-build.sh' script.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Inclusions of 'functions.sh' from 'kvm-test-1-run.sh' and
'kvm-recheck*.sh' use its absolute path. Because the directory containing
'functions.sh' is already in PATH, the full path is unnecessary. This
commit therefore simplifies the inclusions to use the short relative path.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Both the 'kvm.sh' and 'kvm-test-1-run.sh' scripts log messages by printing
the message to 'stdout' and then also printing it into the log file.
Generation of the message thus occurs twice, once for 'stdout' and once
for the log file. Moreover, many of the messages contain 'date' output,
which results in date being invoked twice (once for stdout print, once
for log file write). As a result, the date information in stdout and
log file can differ, which could cause confusion.
This commit therefore simplifies the logging procedure by using 'tee'.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm-recheck-(lock|rcu|rcuperf).sh scripts check whether the
user-specified results directory exists. If not, it prints out error
message that says the specified directory is unreadable. To make the
message more precise, this commit adds a readability check.
Fixes: 2193e1604e ("rcutorture: Abstract kvm-recheck.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'kvm.sh' rcutorture script requires that it be invoked from the top
of Linux-kernel source tree. It is just a subtle restriction, but users
using it for the first time could forget the restriction and be confused.
Moreover, it makes commands a little longer, which can be frustrating.
This commit therefore lets users invoke the script from any location.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The '--qemu-args' option's help text is wrongly copied from '--qemu-cmd'
option and its argument type description message format is inconsistent
with other arguments. This commit fixes the usage and type messages to
be consistent with others.
Fixes: e9ce640001 ("rcutorture: Add --qemu-args argument to kvm.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The variable `alldone` is defined but not used within an awk script.
This commit therefore removes it.
Fixes:53954671033d ("rcutorture: Do better bin packing")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'config2frag.sh' script is not used, so this commit removes it.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'configinit.sh' script checks the format of optional argument for the
build directory, printing an error message if the format is not valid.
However, the error message uses the wrong variable, indicating an empty
string even though the user entered a non-empty (but erroneous) string.
This commit fixes the script to use the correct variable.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because %p prints "(null)" and %pK prints "0000000000000000" or (on
32-bit systems) "00000000", this commit adjusts torture-test scripting
accordingly.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Both rcutorture and locktorture currently place temporary files in /tmp,
in keeping with decades-long tradition. However, sometimes it is useful
to specify an alternative temporary directory, for example, for space
or performance reasons. This commit therefore causes the torture-test
scripting to use the path specified in the TMPDIR environment variable,
or to fall back to traditional /tmp if this variable is not set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The maxcpus= kernel boot parameter limits the number of CPUs brought
online at boot time, but it does nothing to prevent additional CPUs
from being brought up later. Placing a hard cap on the total number
of CPUs is instead the job of the nr_cpus= boot parameter. This commit
therefore switches the configfrag_boot_cpus() shell function from maxcpus=
to nr_cpus=. This commit also adds a nr_cpus=43 kernel parameter to RCU's
TREE01 test scenario, but retains the maxcpus=8 kernel parameter in order
to test the ability of RCU expedited grace periods to handle new CPUs
coming online for the first time during grace-period initialization.
Finally, this commit makes the torture scheduling allow maxcpus= to
override other means of specifying the number of CPUs to allow for.
This last works because the torture kernel modules size their workloads
based on the number of CPUs present at the start of the test, not the
ultimate number of CPUs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, rcutorture groups runs in batches, building each scenario in
a given batch, then invoking qemu to run all the kernels in the batch.
Of course, if a given scenario's kernel fails to build, there is no qemu
run for that scenario. And if all of the kernels in a given batch fail
to build, there are no runs, and rcutorture immediately starts on the
next batch.
But not if --jitter has been specified, which it is by default. In this
case, the jitter scripts are started unconditionally, and rcutorture
waits for them to complete, even though there are no kernels to run.
This commit therefore checks for this situation, and refuses to start
jitter unless at least one of the kernels in the batch built successfully.
This saves substantial time when all scenarios' kernels fail to build,
particularly if a long --duration was specified.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, testing a variant of an existing scenario requires editing
that scenario's file or creating a new scenario file. This is messy
and error prone with respect to changes to scenarios.
This commit therefore adds a --kconfig argument to kvm.sh, so that
'--kconfig "CONFIG_RCU_TRACE=y CONFIG_RCU_EQS_DEBUG=n" will override those
two Kconfig options. In addition, there is now clear precedence:
the config fragment overrides CFcommon, and the --kconfig argument
overrides both.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, when running from a git archive, the testid.txt file contains
only the branch name, the output of "git status", and the SHA-1 of
the current HEAD. This is useful, but does not uniquely identify the
source code that was built. This commit therefore adds the output of
"git diff HEAD", which means that if two testid.txt files compare equal,
they correspond to exactly the same source code. Give or take the
possibility of SHA-1 collisions, that is. ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripting handles the CONFIG_*_TORTURE_TEST Kconfig
options specially, and therefore greps them out of the Kconfig-fragment
files. Unfortunately, a poor choice of grep pattern means that the
CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP, CONFIG_RCU_TORTURE_TEST_SLOW_INIT,
and CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT Kconfig options are also grepped
out, preventing rcutorture from using them. This commit therefore fixes
the offending grep pattern to focus only on the CONFIG_*_TORTURE_TEST
Kconfig options.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Kconfig "select" clauses can defeat Kconfig-fragment file attempts to
clear a given Kconfig variable, and dependencies can defeat attempts to
set a given Kconfig variable. Because "select" clauses and dependencies
can be added at any time, there needs to be a way to verify that the
Kconfig-fragment file's requests were honored. And there is, except
that it is buggy. This commit therefore provides the needed fix.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the rcutorture scripting will give an error message if
running a duplicate scenario that happens also to have a non-existent
build directory (b1, b2, ... in the rcutorture directory). Worse yet, if
the build directory has already been created and used for a real build,
the script will silently grab the wrong Kconfig fragment, which could
cause confusion to the poor sap (me) analyzing old test results. At
least the actual test runs correctly...
This commit therefore accesses the Kconfig fragment from the results
directory corresponding to the first of the duplicate scenarios, for
which a build was actually carried out. This prevents both the messages
and at least one form of later confusion.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, if the --jitter flag specifies jitter for a --build-only
run, the system will obediently build a kernel, refuse to launch it,
launch the requested number of jitter processes, and wait for the
specified kernel run time, which defaults to 30 minutes. This is
of course quite pointless.
This commit therefore disables jitter on build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit enables jitter by default. It may be manually disabled
by passing "--jitter 0" to kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because recent testing shows that "-soundhw pcspkr" is no longer required
in the kernel boot arguments, this commit drops this qemu argument.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The option "-soundhw pcspk" gives me a error on PPC as follow:
qemu-system-ppc64: ISA bus not available for pcspk
This means this option doesn't work on ppc by default. So simply make
this an x86-specific option via identify_qemu_args().
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The vmlinux image is available for all the architectures, and suitable
for running a KVM guest by QEMU, besides, we used to copy the vmlinux
to $resdir anyway. Therefore it makes sense to use it as the fallback
kernel image for rcutorture KVM tests.
This patch makes identify_boot_image() return vmlinux if
${TORTURE_BOOT_IMAGE} is not set on non-x86 architectures, also fixes
several places that hard-code "bzImage" as $KERNEL.
This also fixes a problem that PPC doesn't have a bzImage file as build
results.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a string of the form "Starves: 10" to the summary
line for error conditions found in the console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit avoids killing qemu if a trace dump is making progress
or if console log output is continuing and the console log timestamp
does not exceed the total plus grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, if the user specifies multiple runs of a given test
configuration, the scripting does multiple kernel builds. This wastes
both time and disk space, so this commit makes the scripting use the
first build for all runs of a given test configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a --jitter OS-jitter capability to expose bugs based
on no-delay assumptions.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit converts test duration from minutes to seconds early on
in order to prepare for upcoming OS-jitter-injection changes.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current hang-check machinery in the rcutorture scripts uses "$!" of
a parenthesized bash statement to capture the pid. Unfortunately, this
captures not qemu's pid, but rather that of its parent that implements
the parenthesized statement. This commit therefore adjusts things so as
to capture qemu's actual pid, which then allows the script to actually
kill qemu in event of a kernel hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf event-trace data is more accurate than are the rcuperf
printk()s because locking keeps things ordered. This commit therefore
parses and analyzes this event-trace data if present, and falls back on
the printk()s otherwise.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a line giving the number of grace periods, the number
of batches, and the ratio. The larger the ratio, the greater the
batching efficiency.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds the scripting changes to add support for the shiny
new rcuperf kernel module.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The current scripts parse console output only for cases where one CPU
detect a stall on some other CPU or task. This commit therefore adds
checks for self-detected stalls.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, an error-free run produces an empty console.log.diags file.
This can be annoying when using "vi */console.log.diags" to see a full
summary of the errors. This commit therefore removes any empty files
during the analysis process.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds checks for rcutorture writer starvation, so that
instances will be added to the test summary.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the console output files ("console.log") are placed in the
build directory initially, then copied to the results directory.
One problem with this is if a qemu refuses to die in a timely fashion
after a kernel hang, it will continue to write after the next qemu
starts up, resulting in confusing output from the old instance of
qemu. This commit prevents such confusion by placing the console.log
files into the results directory to begin with, so that a given instance
of qemu is always writing only to its own console.log file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
