Commit Graph

62 Commits

Author SHA1 Message Date
Gerwin Klein 6da2d97c69 run_tests: echo L4V_FEATURES and L4V_PLAT
Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2023-06-06 15:37:01 +10:00
Rafal Kolanski 381ad05df9
run_tests: enable RefineOrphanage for AARCH64
Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2023-05-26 18:04:49 +10:00
Rafal Kolanski 496f70f7a6
run_tests: fix QUICK_AND_DIRTY handling
os.environ expects a string, not an integer

Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2023-05-26 18:04:49 +10:00
Rafal Kolanski 7b73a18757
run_tests: enable Refine (quick_and_dirty) for AARCH64
Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2023-05-26 18:04:49 +10:00
Rafal Kolanski 0794e0a427
run_tests: enable BaseRefine for AARCH64
Switch exclusion to Refine.

Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2023-05-25 19:34:15 +10:00
Gerwin Klein aadaabbfd3 run_tests: disable Access, Refine, Bism in AARCH64
These are depending on AInvs, so got enabled by enabling AInvs, but
are not yet in scope for development.

Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2022-06-03 09:36:43 +10:00
Rafal Kolanski e293dc4a38 run_tests: enable AInvs for AARCH64
Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2022-06-03 09:36:43 +10:00
Gerwin Klein cd32136604 enable AARCH64 ASpec test
Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2022-04-20 09:16:19 +10:00
Rafal Kolanski 7f83d32a77 run_tests: enable ExecSpec for AARCH64
Signed-off-by: Rafal Kolanski <rafal.kolanski@proofcraft.systems>
2022-04-20 09:16:19 +10:00
Gerwin Klein fcdbbf5bad aarch64 run_tests: add test setup for AARCH64
This adds the AARCH64 L4V_ARCH and adds a long initial test exclusion
list that will be reduced as verification proceeds.

Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2022-04-20 09:16:19 +10:00
Ryan Barry d619052b02 run_tests: enable RISCV64 InfoFlow tests
Signed-off-by: Ryan Barry <ryan.barry@unsw.edu.au>
2021-11-12 09:39:16 +11:00
Gerwin Klein 35aab82dbd runtests: remove obsolete exclusion
CamkesDarpaReport is not part of the test scope any more.

Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2021-09-30 16:53:17 +10:00
Ryan Barry c7cb94f8bf run_tests: enable RISCV64 access control test
Signed-off-by: Ryan Barry <ryan.barry@unsw.edu.au>
2021-07-22 10:44:43 +10:00
Gerwin Klein 115b12f1e2 run_tests: flush output
On GitHub, the output of external processes such as isabelle overtake
the stdout/stderr output of the test driver. Flushing stdout/stderr
in the right spots avoids that.

Signed-off-by: Gerwin Klein <gerwin.klein@proofcraft.systems>
2021-07-15 13:53:14 +10:00
Gerwin Klein a5545cc040 ROOT: make SepTacticsExamples part of DSpecProofs
Not really worth the trouble of an own session directory.

Signed-off-by: Gerwin Klein <gerwin.klein@data61.csiro.au>
2020-10-27 15:52:31 +10:00
Gerwin Klein e6f27344e9 run_tests: enable RISCV64 CRefine test
Signed-off-by: Gerwin Klein <gerwin.klein@data61.csiro.au>
2020-06-04 23:28:52 +10:00
Matthew Brecknell cf48906b26 regression: force use of python3
Python 2 has passed its sunset date, and many distributions are
withdrawing support for Python 2.

PEP 394 recommends distributions always install versioned interpreter
commands (e.g. `python3`), but does not make a recommendation about
whether or not an unversioned command (`python`) should exist, or what
version it should run.

It therefore seems advisable to explicitly run scripts using the
`python3` command, for scripts that are compatible with Python 3.

Here, we do this for Python scripts used by `run_tests`. For this to
work, some scripts have been updated in ways that will break Python 2
compatibility. But for some other scripts which were already compatible
with both Python 2 and 3, we have not yet removed Python 2
compatibility. There are also miscellaneous scripts that are not used by
`run_tests`, and these have not yet been updated to Python 3.

Signed-off-by: Matthew Brecknell <Matthew.Brecknell@data61.csiro.au>
2020-04-01 14:31:36 +11:00
Gerwin Klein a424d55e3e licenses: convert license tags to SPDX 2020-03-13 14:38:24 +08:00
Edward Pierzchalski 050c192e0d asmrefine riscv: enable SimplExportAndRefine 2019-12-19 17:05:10 +11:00
Gerwin Klein c826b33b88 bisim: Bisim session for ARM, X64, RISCV64
ARM_HYP would be possible, but require arch split for hyp-faults.
2019-11-15 12:04:50 +11:00
Gerwin Klein 0661581e44 runtests: enable RISCV64 Orphanage test 2019-11-12 18:28:40 +11:00
Gerwin Klein 46398221c9 runtests: enable RISCV64 Refine test 2019-11-12 18:28:39 +11:00
Rafal Kolanski 17ca50d695 run_tests: configure RISCV64 sessions for abstract invariant proofs
Enable AInvs, remove everything from Refine onwards, but include CSpec.
2019-07-31 16:55:31 +10:00
Japheth Lim 77a3a0746f trivial: run_tests: typos and formatting 2019-06-25 11:02:22 +10:00
Japheth Lim 1a3de82631 run_tests: don't set L4V_ARCH_IS_ARM for other archs
Leftover bug from 445b003cab.
2019-06-25 11:02:22 +10:00
Japheth Lim 5fef6f3443 run_tests: use standard license header 2019-06-13 11:33:43 +10:00
Japheth Lim 445b003cab run_tests: show --l4v-arches in --help output 2019-06-13 11:33:43 +10:00
Japheth Lim e4ad4e08a6 run_tests: support testing multiple L4V_ARCHs in one invocation
This adds the options `--l4v-arches` and `--l4v-arch-all` to the
`run_tests` wrapper script, which can be used to specify multiple
(or all) L4V_ARCHs to test.
2019-06-13 11:33:43 +10:00
Japheth Lim 1a504d1f6c run_tests: move selection of RefineOrphanage to run_tests
This is more consistent with how we handle other broken proof sessions
in the run_tests framework.
2018-10-03 19:47:04 +10:00
Gerwin Klein 39c11f161f riscv run_tests: enable ASpec and ASpecDoc test sessions 2018-09-07 08:13:13 +10:00
Gerwin Klein cc387f96dc Isabelle2018: AsmRefine currently only for 32bit platforms 2018-08-20 09:06:37 +10:00
Gerwin Klein db111292fe run_tests: switch off quick_and_dirty for X64 2018-07-09 09:23:40 +01:00
Rafal Kolanski 451e204a6e x64: test CRefine in quick_and_dirty mode 2018-07-05 16:23:14 +10:00
Gerwin Klein 15818f215a run_tests: add RISCV64 test architecture 2018-06-27 10:06:48 +02:00
Thomas Sewell 18cf880573 run_tests: Create and fetch timing/timeout info. 2018-01-22 12:04:08 +11:00
Miki Tanaka 9bdb47e114 reintroduce Orphanage test (for ARM only)
- Orphanage files in the ARM_HYP and X64 directories are not tested at the moment
- once we finish proving them, we will remove the restriction to ARM
2017-10-24 13:49:21 +11:00
Joel Beeren 7942cf915c run_tests: stop CRefine et al from being excluded from regression for x64 2017-10-04 15:57:46 +11:00
Alejandro Gomez-Londono 6aa0ef43e2 arm-hyp regression: minimise run_tests exclusions 2017-08-23 16:47:53 +10:00
Matthew Brecknell 4d81972127 x64 regression: minimise run_tests exclusions 2017-08-21 17:49:45 +10:00
Joel Beeren ac0af865d2 run_tests: add CRefine et al to X64 excludes 2017-08-09 22:21:02 +10:00
Joel Beeren bb4e703654 run_tests: fix X64 excludes 2017-08-09 17:17:11 +10:00
Joel Beeren 82863978bd Merge branch 'master' into x64 2017-08-09 17:10:06 +10:00
Joel Beeren d9afe3f45c run_tests: Adjust environment flags for build
*** ALERT: ANYONE USING SKIP_REFINE_PROOFS SHOULD CHANGE TO
SKIP_DUPLICATED_PROOFS IN ~/.isabelle/etc/settings!!! ***

Previously SKIP_REFINE_PROOFS was being used to skip duplicated Refine
and AInvs proofs when building CBaseRefine and InfoFlowC. This
conflicted with adding an option to actually skip building Refine proofs
(for example when trying to quickly build DBaseRefine).

After this change, we have the following SKIP_PROOFS flags:
    * SKIP_AINVS_PROOFS: used to skip AInvs proofs (for example when
      building Refine)
    * SKIP_REFINE_PROOFS: used to skip Refine proofs (for example when
      building DBaseRefine)
    * SKIP_DUPLICATED_PROOFS: used to skip the rebuild of AInvs and
      Refine when building forked images such as CBaseRefine and
      InfoFlowC

In addition, the QUICK_AND_DIRTY flag for AInvs has been changed:
        INVS_QUICK_AND_DIRTY -> AINVS_QUICK_AND_DIRTY
2017-08-08 16:11:20 +10:00
Alejandro Gomez-Londono 365643d751 run_tests: rewritten in python (used to be bash) 2017-07-26 14:39:05 +10:00
Matthew Brecknell 238e8b307e x64: merge master 2017-07-21 11:27:12 +10:00
Matthew Brecknell 63e1aa0c90 x64 cspec: temporarily sorry bitfield proofs 2017-07-04 11:59:15 +10:00
Matthew Brecknell ce748b7522 x64: create arch-specific CKernel 2017-06-22 17:24:53 +10:00
Alejandro Gomez-Londono 819f1492ee test: Reestablish ARM as the default platform for test 2017-06-19 14:32:45 +10:00
Alejandro Gomez-Londono 74500bd829 arm-hyp test: Adding exclusion test set for ARM_HYP + cleanup 2017-06-19 14:32:43 +10:00
Gerwin Klein 28c47bf706 arm-hyp: temporarily remove ASpec_ARM ExecSpec_ARM tests 2017-06-19 14:32:40 +10:00