dev:regtesting
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
dev:regtesting [2015/04/28 13:34] – oschuett | dev:regtesting [2019/01/13 11:30] – oschuett | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== CP2K Regression Testing ====== | ====== CP2K Regression Testing ====== | ||
- | CP2K comes with over 2500 test input files (located in [[src>cp2k/tests]]) which serve as both examples on how to use the many features in CP2K and also as a method for developers to test modifications and extensions to CP2K. In order to reduce the chance of bugs being introduced into the code, and ensure that all parts of the code are working, we recommend that all developers | + | CP2K comes with over 3000 test input files (located in [[src> |
==== Dashboard ==== | ==== Dashboard ==== | ||
- | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// | + | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// |
==== Code Coverage ==== | ==== Code Coverage ==== | ||
- | We aim that the regression test suite covers all the functionality of CP2K. For this purpose we regularly create [[http:// | + | We aim that the regression test suite covers all the functionality of CP2K. For this purpose we regularly create [[http:// |
===== How does it work? ===== | ===== How does it work? ===== | ||
- | The regression test suite is run using the [[src>cp2k/tools/ | + | The regression test suite is run using the [[src> |
It performs the following tasks: | It performs the following tasks: | ||
- | * update to the current SVN version (including new tests) | ||
* performs a realclean build of the source | * performs a realclean build of the source | ||
* executes a list of tests | * executes a list of tests | ||
Line 21: | Line 20: | ||
===== Running the regtests ===== | ===== Running the regtests ===== | ||
+ | |||
+ | ==== Step 0: make based testing ==== | ||
+ | |||
+ | * If you are able to build and run cp2k on the local machine the easiest way is to start the regtesting by running '' | ||
+ | * Be careful about the value of '' | ||
+ | * If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | ||
==== Step 1: Preparation ==== | ==== Step 1: Preparation ==== | ||
- | * You must be able to build and run cp2k on the given machine, the rest should be ' | ||
* Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | * Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | ||
- | * Checkout | + | * Clone a version of cp2k into '' |
* Set up the arch files so that you can cleanly build cp2k (test this) | * Set up the arch files so that you can cleanly build cp2k (test this) | ||
- | ==== Step 2: Configuration | + | ==== Step 2: Running |
+ | < | ||
+ | $ tools/ | ||
+ | Usage: do_regtest [OPTION] | ||
+ | Run the CP2K regression test suite | ||
+ | Example: do_regtest -c my_regtest.conf | ||
- | The // | + | General: |
+ | -h, -help, --help | ||
+ | -c, -config FILE read any of the following | ||
+ | -cp2kdir PATH | ||
+ | -dirout PATH root path of output folder (default: current working directory). | ||
+ | -mpiexec EXE name of the executable to run mpi-ranks. Default: mpiexec. | ||
- | ^ Config Variable ^ Default ^ Meaning | + | Build: |
- | | '' | + | -version VERSION |
- | | '' | + | -arch ARCH ARCH passed to make. Default: Linux-x86-64-gfortran. |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | Furthermore: | + | Runtime: |
- | < | + | |
- | # Script configuration. The value of the follow variables can be redefined, see below | + | |
- | # | + | -maxtasks NPROCS |
- | # make, awk, datum_full, datum_short, | + | |
- | # | + | |
- | </ | + | |
+ | Testing: | ||
+ | -noreset | ||
+ | -shard ISHARD NSHARDS | ||
+ | -skiptest | ||
+ | -skipunittest | ||
+ | -skipdir TESTDIR | ||
+ | -restrictdir TESTDIR | ||
+ | -retest | ||
- | ==== Step 3: Execution ==== | + | Exit codes: |
- | Normally, you just have to call the // | + | |
+ | 3 | ||
+ | 4 build errors | ||
+ | 5 | ||
+ | 6 | ||
+ | 7 | ||
+ | 8 | ||
- | <code> | + | For more information visit: |
- | $ ./cp2k/tools/regtesting/do_regtest -config < | + | |
</ | </ | ||
- | However, the // | + | ==== Step 3: Interpretation ==== |
- | ^ Command Line Switch | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | + | ||
- | The // | + | |
- | + | ||
- | ^ Exit Code | + | |
- | | 0 | clean exit with testing | + | |
- | | 1 | problem with svn update | + | |
- | | 3 | problem with realclean | + | |
- | | 4 | build errors | + | |
- | | 5 | problem with retest option - no TEST directory with latest test results found | | + | |
- | | 6 | problem with retest option - no error summary exists in the last TEST directory | | + | |
- | | 7 | reference directory is locked | + | |
- | | 8 | ctrl-C (SIGINT) and various other signals trapped | + | |
- | | 100 | no svn changes since last run - clean exit without testing | + | |
- | + | ||
- | + | ||
- | ==== Step 4: Interpretation ==== | + | |
A test results can be any of the following: | A test results can be any of the following: | ||
^ Test Result | ^ Test Result | ||
Line 105: | Line 94: | ||
===== Adding and Resetting Tests===== | ===== Adding and Resetting Tests===== | ||
- | The test-suite is fully controlled by the following files in the [[src>cp2k/tests]] directories | + | The test-suite is fully controlled by the following files in the [[src> |
^ File Name ^ Content | ^ File Name ^ Content | ||
| '' | | '' | ||
| '' | | '' | ||
- | | '' | + | | '' |
| '' | | '' | ||
dev/regtesting.txt · Last modified: 2023/10/19 14:03 by krack