dev:regtesting
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
dev:regtesting [2015/05/31 10:33] – [Code Coverage] 84.75.193.61 | dev:regtesting [2018/10/08 19:59] – oschuett | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== CP2K Regression Testing ====== | ====== CP2K Regression Testing ====== | ||
- | CP2K comes with over 2500 test input files (located in [[src>cp2k/tests]]) which serve as both examples on how to use the many features in CP2K and also as a method for developers to test modifications and extensions to CP2K. In order to reduce the chance of bugs being introduced into the code, and ensure that all parts of the code are working, we recommend that all developers complete a successful run of the full regression test suite before committing changes to SVN. | + | CP2K comes with over 2500 test input files (located in [[src> |
+ | * all developers complete a regression test before committing | ||
+ | * all users complete a test before using a self-compiled binary for their projects | ||
==== Dashboard ==== | ==== Dashboard ==== | ||
- | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// | + | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// |
==== Code Coverage ==== | ==== Code Coverage ==== | ||
Line 11: | Line 13: | ||
===== How does it work? ===== | ===== How does it work? ===== | ||
- | The regression test suite is run using the [[src>cp2k/tools/ | + | The regression test suite is run using the [[src> |
It performs the following tasks: | It performs the following tasks: | ||
* update to the current SVN version (including new tests) | * update to the current SVN version (including new tests) | ||
Line 21: | Line 23: | ||
===== Running the regtests ===== | ===== Running the regtests ===== | ||
+ | |||
+ | ==== Step 0: make based testing ==== | ||
+ | |||
+ | * If you are able to build and run cp2k on the local machine the easiest way is to start the regtesting by running '' | ||
+ | * Be careful about the value of '' | ||
+ | * If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | ||
==== Step 1: Preparation ==== | ==== Step 1: Preparation ==== | ||
- | * You must be able to build and run cp2k on the given machine, the rest should be ' | ||
* Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | * Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | ||
* Checkout a version of cp2k into '' | * Checkout a version of cp2k into '' | ||
* Set up the arch files so that you can cleanly build cp2k (test this) | * Set up the arch files so that you can cleanly build cp2k (test this) | ||
- | ==== Step 2: Configuration | + | ==== Step 2: Running |
+ | < | ||
+ | $ tools/ | ||
+ | Usage: do_regtest [OPTION] | ||
+ | Run the CP2K regression test suite | ||
+ | Example: do_regtest -c my_regtest.conf | ||
- | The // | + | General: |
+ | -h, -help, --help | ||
+ | -c, -config FILE read any of the following | ||
+ | -cp2kdir PATH | ||
- | ^ Config Variable ^ Default ^ Meaning | + | Svn: |
- | | '' | + | -svndate DATE |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
+ | Build: | ||
+ | -version VERSION | ||
+ | -arch ARCH ARCH passed to make. Default: Linux-x86-64-gfortran. | ||
+ | -quick | ||
+ | -nobuild | ||
- | ==== Step 3: Execution ==== | + | Runtime: |
- | Normally, you just have to call the // | + | |
+ | | ||
+ | -maxtasks NPROCS | ||
+ | -jobmaxtime SECONDS | ||
- | < | + | Testing: |
- | $ ./ | + | |
- | </ | + | |
+ | | ||
+ | -restrictdir TESTDIR | ||
+ | -retest | ||
- | However, the // | + | Exit codes: |
- | ^ Command Line Switch | + | 0 |
- | | '' | + | 1 |
- | | '' | + | 3 |
- | | '' | + | 4 |
- | | '' | + | 5 |
- | | '' | + | 6 |
- | | '' | + | 7 |
- | | '' | + | 8 |
- | | '' | + | 100 |
- | | '' | + | |
- | | '' | + | |
- | + | ||
- | The // | + | |
- | + | ||
- | ^ Exit Code | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
- | | | + | |
+ | For more information visit: < | ||
+ | </ | ||
- | ==== Step 4: Interpretation ==== | + | ==== Step 3: Interpretation ==== |
A test results can be any of the following: | A test results can be any of the following: | ||
^ Test Result | ^ Test Result | ||
Line 93: | Line 98: | ||
===== Adding and Resetting Tests===== | ===== Adding and Resetting Tests===== | ||
- | The test-suite is fully controlled by the following files in the [[src>cp2k/tests]] directories | + | The test-suite is fully controlled by the following files in the [[src> |
^ File Name ^ Content | ^ File Name ^ Content |
dev/regtesting.txt · Last modified: 2023/10/19 14:03 by krack