howto:pao-ml
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
howto:pao-ml [2018/01/20 17:43] – [Step 4: Optimize machine learning hyper-parameters] oschuett | howto:pao-ml [2020/08/21 10:15] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 2: | Line 2: | ||
PAO-ML stands for Polarized Atomic Orbitals from Machine Learning. It uses machine learning to generate geometry adopted small basis sets. It also provides exact ionic forces. The scheme can serve as an almost drop-in replacement for conventional | PAO-ML stands for Polarized Atomic Orbitals from Machine Learning. It uses machine learning to generate geometry adopted small basis sets. It also provides exact ionic forces. The scheme can serve as an almost drop-in replacement for conventional | ||
- | basis sets to speedup otherwise standard DFT calculations. The method is similar to semi-empirical models based on minimal basis sets, but offers improved accuracy and quasi-automatic parameterization. However, the method is still in an early stage - so use with caution. For more information see: [[doi> | + | basis sets to speedup otherwise standard DFT calculations. The method is similar to semi-empirical models based on minimal basis sets, but offers improved accuracy and quasi-automatic parameterization. However, the method is still in an early stage - so use with caution. For more information see: [[doi> |
===== Step 1: Obtain training structures ===== | ===== Step 1: Obtain training structures ===== | ||
Line 10: | Line 10: | ||
===== Step 2: Calculate reference data in primary basis ===== | ===== Step 2: Calculate reference data in primary basis ===== | ||
- | Choose a primary basis set, e.g. '' | + | Choose a primary basis set, e.g. '' |
===== Step 3: Optimize PAO basis for training structures ===== | ===== Step 3: Optimize PAO basis for training structures ===== | ||
- | Choose a [[inp> | + | Choose a [[inp> |
Most of the PAO settings are in the [[inp> | Most of the PAO settings are in the [[inp> | ||
Line 54: | Line 54: | ||
</ | </ | ||
+ | ==== Tuning the PAO Optimization ===== | ||
+ | Finding the optimal PAO basis poses an intricate minimization problem, because the rotation matrix U and the Kohn-Sham | ||
+ | matrix H have to be optimized in a self-consistent manner. In order to speedup the optimization, | ||
+ | * The frequency with which H is recalculated is determined by [[inp> | ||
+ | * Overshooting during the U optimization is damped via [[inp> | ||
+ | |||
+ | The progress of the PAO optimization can be tracked from lines that start with '' | ||
+ | < | ||
+ | | ||
+ | PAO| step | ||
+ | </ | ||
+ | |||
+ | * The step number counts the number of energy evaluation, ie. the number of U matrices probed. It can increase with different intervals, when the [[inp> | ||
+ | * The energy is the quantity that is optimized. It contains **only the first order term** of the total energy, ie. $Tr[HP]$, but shares the same variational minima. It furthermore contains the contributions from the various regularization terms. | ||
+ | * The convergence criterion is the norm of the gradient normalized by system size. It is compared against [[inp> | ||
+ | * The step length is the outcome of the line search. It should be of order 1. If it starts to behave erratically towards the end of the optimization, | ||
+ | * The time is the time spend on this optimization step in seconds. This number can varry accordingly to the number of performed lines search steps. | ||
===== Step 4: Optimize machine learning hyper-parameters ===== | ===== Step 4: Optimize machine learning hyper-parameters ===== | ||
Line 59: | Line 76: | ||
For the simulation of larger systems the PAO-ML scheme infers new PAO basis sets from the training data. For this two heuristics are employed: A [[https:// | For the simulation of larger systems the PAO-ML scheme infers new PAO basis sets from the training data. For this two heuristics are employed: A [[https:// | ||
- | In order to obtain good results from the learning machinery a small number of so-called [[https:// | + | In order to obtain good results from the learning machinery a small number of so-called [[https:// |
- | For the optimization of the hyper-parameter exists no gradient, hence one has to use a derivative-free method like the one by [[https:// | + | For the optimization of the hyper-parameter exists no gradient, hence one has to use a derivative-free method like the one by [[https:// |
===== Step 5: Run simulation with PAO-ML ==== | ===== Step 5: Run simulation with PAO-ML ==== | ||
Line 116: | Line 133: | ||
Unfortunately, | Unfortunately, | ||
- | The most critical parameters for learnability are [[inp> | + | The most critical parameters for learnability are [[inp> |
howto/pao-ml.txt · Last modified: 2024/01/03 13:19 by oschuett