Represent restarts from checkpoints in HDF5 data file
Currently, the data.h5
file has the following structure:
data.h5
├── lattice ................... Attached attributes describe Bravais lattice and unit cell
├── X_scal........................................ Attached attribute: analysis_mode
│ ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│ └── sign................................................Dataset of shape (NBins)
├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│ ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│ ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│ ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│ └── sign................................................Dataset of shape (NBins)
└── parameters
├── namelist_1................Attached attributes are the parameters in namelist_1
├── .
└── namelist_n................Attached attributes are the parameters in namelist_n
Which does not resolve restarts from confout/confin. This has, for example, the following disadvantages:
- Parameter changes between restarts, for example for annealing, are not recorded in the HDF5 file
- It's harder to recover from a failed rerun
- Adding an observable between reruns is very intransparent.
Therefore it might make sense to add an additional hierarchy to the file, like this:
data.h5
├── step1
│ ├── lattice ................... Attached attributes describe Bravais lattice and unit cell
│ ├── X_scal........................................ Attached attribute: analysis_mode
│ │ ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│ │ └── sign................................................Dataset of shape (NBins)
│ ├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│ │ ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│ │ ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│ │ ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│ │ └── sign................................................Dataset of shape (NBins)
│ └── parameters
│ ├── namelist_1................Attached attributes are the parameters in namelist_1
│ ├── .
│ └── namelist_n................Attached attributes are the parameters in namelist_n
├── step2
│ ├── lattice ................... Attached attributes describe Bravais lattice and unit cell
│ ├── X_scal........................................ Attached attribute: analysis_mode
│ │ ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│ │ └── sign................................................Dataset of shape (NBins)
│ ├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│ │ ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│ │ ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│ │ ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│ │ └── sign................................................Dataset of shape (NBins)
│ └── parameters
│ ├── namelist_1................Attached attributes are the parameters in namelist_1
│ ├── .
│ └── namelist_n................Attached attributes are the parameters in namelist_n
...