dev:regtesting
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
dev:regtesting [2015/11/18 09:37] – Add a note about -j in make test rivanvx | dev:regtesting [2023/10/19 14:03] (current) – Update flags krack | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== CP2K Regression Testing ====== | ====== CP2K Regression Testing ====== | ||
- | CP2K comes with over 2500 test input files (located in [[src>cp2k/tests]]) which serve as both examples on how to use the many features in CP2K and also as a method for developers to test modifications and extensions to CP2K. In order to reduce the chance of bugs being introduced into the code, and ensure that all parts of the code are working, we recommend that : | + | CP2K comes with over 3000 test input files (located in [[src> |
- | * all developers complete a regression test before committing any changes to SVN. | + | |
- | * all users complete a test before using a self-compiled binary for their projects | + | |
==== Dashboard ==== | ==== Dashboard ==== | ||
Line 13: | Line 11: | ||
===== How does it work? ===== | ===== How does it work? ===== | ||
- | The regression test suite is run using the [[src>cp2k/ | + | The regression test suite is run using the [[src>tests/do_regtest.py | do_regtest]] script. |
It performs the following tasks: | It performs the following tasks: | ||
- | * update to the current SVN version (including new tests) | ||
* performs a realclean build of the source | * performs a realclean build of the source | ||
* executes a list of tests | * executes a list of tests | ||
Line 26: | Line 23: | ||
==== Step 0: make based testing ==== | ==== Step 0: make based testing ==== | ||
- | * If you are able to build and run cp2k on the local machine the easiest way is to start the regtesting by running '' | + | * If you are able to build and run cp2k on the local machine the easiest way is to start the regtesting by running '' |
- | * Be careful about the value of '' | + | * Be careful about the value of '' |
* If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | * If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | ||
==== Step 1: Preparation ==== | ==== Step 1: Preparation ==== | ||
* Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | * Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | ||
- | * Checkout | + | * Clone a version of cp2k into '' |
* Set up the arch files so that you can cleanly build cp2k (test this) | * Set up the arch files so that you can cleanly build cp2k (test this) | ||
==== Step 2: Running ==== | ==== Step 2: Running ==== | ||
< | < | ||
- | $ cp2k/ | + | $ tests/do_regtest.py -h |
- | Usage: do_regtest [OPTION] | + | usage: do_regtest.py [-h] [--mpiranks MPIRANKS] [--ompthreads OMPTHREADS] |
- | Run the CP2K regression test suite | + | [--maxtasks MAXTASKS] [--num_gpus NUM_GPUS] |
- | Example: do_regtest | + | [--timeout TIMEOUT] [--maxerrors MAXERRORS] |
+ | | ||
+ | | ||
+ | | ||
+ | | ||
- | General: | + | Runs CP2K regression test suite. |
- | -h, -help, --help | + | |
- | -c, -config FILE read any of the following configuration switches from FILE. | + | |
- | -cp2kdir PATH | + | |
- | Svn: | + | positional arguments: |
- | | + | |
- | | + | |
- | -noemptycheck | + | |
- | Build: | + | options: |
- | -version VERSION | + | -h, --help |
- | -arch ARCH ARCH passed to make. Default: Linux-x86-64-gfortran. | + | --mpiranks MPIRANKS |
- | -quick rebuild if needed, but without realclean. Default: off. | + | |
- | -nobuild | + | --maxtasks MAXTASKS |
- | + | --num_gpus NUM_GPUS | |
- | Runtime: | + | --timeout TIMEOUT |
- | -mpiranks NRANKS | + | --maxerrors MAXERRORS |
- | -ompthreads NTHREADS | + | --mpiexec MPIEXEC |
- | -maxtasks NPROCS | + | --smoketest |
- | -jobmaxtime SECONDS | + | --valgrind |
- | + | --keepalive | |
- | Testing: | + | --flagslow |
- | -noreset | + | --debug |
- | -skiptest | + | --restrictdir RESTRICTDIR |
- | -skipdir TESTDIR | + | --skipdir SKIPDIR |
- | -restrictdir TESTDIR | + | --workbasedir WORKBASEDIR |
- | -retest | + | |
- | + | ||
- | Exit codes: | + | |
- | 0 clean exit with testing | + | |
- | 1 | + | |
- | 3 | + | |
- | 4 build errors | + | |
- | 5 | + | |
- | | + | |
- | 7 | + | |
- | 8 ctrl-C (SIGINT) and various other signals trapped | + | |
- | | + | |
- | + | ||
- | For more information visit: < | + | |
</ | </ | ||
Line 98: | Line 81: | ||
===== Adding and Resetting Tests===== | ===== Adding and Resetting Tests===== | ||
- | The test-suite is fully controlled by the following files in the [[src>cp2k/tests]] directories | + | The test-suite is fully controlled by the following files in the [[src> |
^ File Name ^ Content | ^ File Name ^ Content | ||
| '' | | '' | ||
| '' | | '' | ||
- | | '' | + | | '' |
| '' | | '' | ||
+ | ===== Run with sbatch ===== | ||
+ | |||
+ | The '' | ||
+ | |||
+ | What you need: | ||
+ | |||
+ | * '' | ||
+ | * a CP2K source tree with a built CP2K | ||
+ | |||
+ | ==== Instructions ==== | ||
+ | |||
+ | The way the regtest script works is that it goes through all the directories (for example '' | ||
+ | |||
+ | Also, '' | ||
+ | |||
+ | Append the following to your '' | ||
+ | |||
+ | <code bash> | ||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | # CP2K_REGTEST_SCRIPT_DIR="" | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | |||
+ | # the following is the default, adjust if you want to run single tests with more than 2 ranks/tasks | ||
+ | NTASKS_SINGLE_TEST=2 | ||
+ | NNODES_SINGLE_TEST=1 | ||
+ | SRUN_CMD=" | ||
+ | |||
+ | # the following should be sufficiently generic: | ||
+ | |||
+ | mkdir -p " | ||
+ | cd " | ||
+ | |||
+ | cp2k_rel_dir=$(realpath --relative-to=" | ||
+ | # srun does not like `-np`, override the complete command instead: | ||
+ | export cp2k_run_prefix=" | ||
+ | |||
+ | " | ||
+ | -arch " | ||
+ | -version " | ||
+ | -nobuild \ | ||
+ | -mpiranks ${NTASKS_SINGLE_TEST} \ | ||
+ | -ompthreads ${OMP_NUM_THREADS} \ | ||
+ | -maxtasks ${SLURM_NTASKS} \ | ||
+ | -cp2kdir " | ||
+ | |& tee " | ||
+ | |||
+ | # the above will output both to the slurm-*.out as well as a log file, | ||
+ | # if you want only the log file replace the `|& tee` with a `>& | ||
+ | |||
+ | # More options: | ||
+ | # -farming | ||
+ | # -retest | ||
+ | </ | ||
+ | |||
+ | A complete '' | ||
+ | |||
+ | <code bash> | ||
+ | #!/bin/bash -l | ||
+ | #SBATCH --time=01: | ||
+ | #SBATCH --nodes=4 | ||
+ | #SBATCH --ntasks-per-node=128 | ||
+ | #SBATCH --cpus-per-task=2 | ||
+ | #SBATCH --ntasks-per-core=1 | ||
+ | |||
+ | # More SBATCH options: | ||
+ | # If you need 512GB memory nodes (otherwise only 256GB guaranteed): | ||
+ | # #SBATCH --mem=497G | ||
+ | # To run on the debug queue (max 10 nodes, 30 min): | ||
+ | # # | ||
+ | |||
+ | set -o errexit | ||
+ | set -o nounset | ||
+ | set -o pipefail | ||
+ | |||
+ | export MPICH_OFI_STARTUP_CONNECT=1 | ||
+ | export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} | ||
+ | export OMP_PROC_BIND=close | ||
+ | export OMP_PLACES=cores | ||
+ | |||
+ | source " | ||
+ | |||
+ | module load cpeGNU | ||
+ | module load \ | ||
+ | cray-fftw \ | ||
+ | ELPA/ | ||
+ | libxsmm/ | ||
+ | libxc/5.1.3 \ | ||
+ | Libint-CP2K/ | ||
+ | gcc/10.2.0 | ||
+ | |||
+ | # Let the user see the currently loaded modules in the slurm log for completeness: | ||
+ | module list | ||
+ | |||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | |||
+ | NTASKS_SINGLE_TEST=2 | ||
+ | NNODES_SINGLE_TEST=1 | ||
+ | SRUN_CMD=" | ||
+ | |||
+ | # to run tests across nodes (to check for communication effects), use: | ||
+ | # NNODES_SINGLE_TEST=4 | ||
+ | # SRUN_CMD=" | ||
+ | |||
+ | # the following should be sufficiently generic: | ||
+ | |||
+ | mkdir -p " | ||
+ | cd " | ||
+ | |||
+ | cp2k_rel_dir=$(realpath --relative-to=" | ||
+ | # srun does not like `-np`, override the complete command instead: | ||
+ | export cp2k_run_prefix=" | ||
+ | |||
+ | " | ||
+ | -arch " | ||
+ | -version " | ||
+ | -nobuild \ | ||
+ | -mpiranks ${NTASKS_SINGLE_TEST} \ | ||
+ | -ompthreads ${OMP_NUM_THREADS} \ | ||
+ | -maxtasks ${SLURM_NTASKS} \ | ||
+ | -cp2kdir " | ||
+ | | ||
+ | </ | ||
+ | |||
+ | ==== Test parallelization via Farming ==== | ||
+ | |||
+ | Since the startup time of '' | ||
+ | |||
+ | Simply add the option '' | ||
+ | |||
+ | ==== Minimal directory setup ==== | ||
+ | |||
+ | If you want to test a precompiled executable there is a minimal directory layout you have to reproduce to run the regtest: | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | An example if your HPC center uses EasyBuild to provide the CP2K package: | ||
+ | |||
+ | < | ||
+ | cp2k-prebuilt | ||
+ | ├── data -> / | ||
+ | ├── exe | ||
+ | │ └── prebuilt -> / | ||
+ | └── tests | ||
+ | </ | ||
+ | |||
+ | and then update the variables as follows: | ||
+ | |||
+ | <code bash> | ||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | CP2K_REGTEST_SCRIPT_DIR="/ | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | </ | ||
+ | **Note**: if the '' |
dev/regtesting.1447839435.txt.gz · Last modified: 2020/08/21 10:14 (external edit)