When we want to train neural networks, we have to run training loops that update the parameters many times. A typical training loop consists of the following procedures:
- Iterations over training datasets
- Preprocessing of extracted mini-batches
- Forward/backward computations of the neural networks
- Parameter updates
- Evaluations of the current parameters on validation datasets
- Logging and printing of the intermediate results
Chainer provides a simple yet powerful way to make it easy to write such training processes. The training loop abstraction mainly consists of two components:
- Dataset abstraction.
It implements 1 and 2 in the above list.
The core components are defined in the
datasetmodule. There are also many implementations of datasets and iterators in
It implements 3, 4, 5, and 6 in the above list.
The whole procedure is implemented by
Trainer. The way to update parameters (3 and 4) is defined by
Updater, which can be freely customized. 5 and 6 are implemented by instances of
Extension, which appends an extra procedure to the training loop. Users can freely customize the training procedure by adding extensions. Users can also implement their own extensions.