Represent restarts from checkpoints in HDF5 data file

Currently, the data.h5 file has the following structure:

data.h5
├── lattice ................... Attached attributes describe Bravais lattice and unit cell
├── X_scal........................................ Attached attribute: analysis_mode
│   ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│   └── sign................................................Dataset of shape (NBins)
├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│   ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│   ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│   ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│   └── sign................................................Dataset of shape (NBins)
└── parameters
    ├── namelist_1................Attached attributes are the parameters in namelist_1
    ├── .
    └── namelist_n................Attached attributes are the parameters in namelist_n

Which does not resolve restarts from confout/confin. This has, for example, the following disadvantages:

  • Parameter changes between restarts, for example for annealing, are not recorded in the HDF5 file
  • It's harder to recover from a failed rerun
  • Adding an observable between reruns is very intransparent.

Therefore it might make sense to add an additional hierarchy to the file, like this:

data.h5
├── step1
│   ├── lattice ................... Attached attributes describe Bravais lattice and unit cell
│   ├── X_scal........................................ Attached attribute: analysis_mode
│   │   ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│   │   └── sign................................................Dataset of shape (NBins)
│   ├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│   │   ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│   │   ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│   │   ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│   │   └── sign................................................Dataset of shape (NBins)
│   └── parameters
│       ├── namelist_1................Attached attributes are the parameters in namelist_1
│       ├── .
│       └── namelist_n................Attached attributes are the parameters in namelist_n
├── step2
│   ├── lattice ................... Attached attributes describe Bravais lattice and unit cell
│   ├── X_scal........................................ Attached attribute: analysis_mode
│   │   ├── obser.......................................Dataset of shape (NBins, Nobs, 2)
│   │   └── sign................................................Dataset of shape (NBins)
│   ├── Y_eq Y_tau................................... Attached attributes: Channel, dtau
│   │   ├── lattice ................ Attached attributes describe Bravais lattice and unit cell
│   │   ├── obser.................... Dataset of shape (NBins, Norbs, Norbs, Ntau, Nlatt, 2)
│   │   ├── back....................................... Dataset of shape (NBins, Norbs, 2)
│   │   └── sign................................................Dataset of shape (NBins)
│   └── parameters
│       ├── namelist_1................Attached attributes are the parameters in namelist_1
│       ├── .
│       └── namelist_n................Attached attributes are the parameters in namelist_n
...