These are being added to support builds that turn on extra debugging
or measurement options in Isabelle, which slow the builds down.
Might also be useful for running tests on very slow machines.
The run_tests script already has a -l/--list option to list all tests.
This adds a -L/--dry-run option to list the requested tests, which makes it
easier to figure out what tests -r/-x etc would select.
Also clarify in --help that -v adds more info to -l.
- timeout_output looks for isabelle relative to its file path
- run_tests looks for timeout_output relative to its file path
- output from timeout_output is utf8-decoded before attempting to
concatenate with other strings
Meta-regression should now pass.
The run_tests wrapper explicitly excludes any tests which are known to
fail on certain L4V_ARCH settings. The list of excluded tests includes
some tests which are only defined in private/internal repositories.
Recent changes to run_tests.py and testspec.py assumed that tests
excluded using `run_tests.py -r ...` would be defined in tests.xml
files. This caused regression to fail when performed with only
publicly-available repositories.
This change allows use of the `run_tests.py -r ...` option with test
names that are not defined in any tests.xml files.
tags: [NO_PROOF]
The watch_kill_switch loop was pretty busy, adding a simple
timeout reduces CPU consumption.
The CPU consumption of run_tests.py is still higher than I'd expect
to just update a terminal, but I don't know where to investigate
further.
The run_tests.py script already searches the directory for all
test.xml files. It will now also note any files named extra_tests,
and also search any directories which appear as lines of those
files.
(Following symlinks would been more obvious but create other issues.)
It is sometimes desirable to overwrite the default set of tests that are
being run in a per execution basis (ex: to allow wrapper scripts to have
a custom default) RUN_TESTS_DEFAULT is an space separated list of tests
that will be run if no specific tests are given.
tags: [NO_PROOF]
* commit '84e5f7d81403549f9d88fab3bf0da02b657f47bc':
regression: change CPU time non-mono from an error to a warning, so that it doesn't kill an otherwise working build.
style: regression: refactor use of print_test_line.
regression: report CPU usage; support timeouts based on CPU usage.
Japheth Lim
Thomas Sewell
Rafal Kolanski
Matthew Brecknell
Alejandro Gomez-Londono
Thomas Sewell
Alejandro Gomez-Londono
Japheth Lim
Corey Richardson
Corey Richardson