chainer.iterators.MultithreadIterator¶
-
class
chainer.iterators.
MultithreadIterator
(dataset, batch_size, repeat=True, shuffle=True, n_threads=1)[source]¶ Dataset iterator that loads examples in parallel.
This is an implementation of
Iterator
that loads examples with worker threads. It uses the standardthreading
module to parallelize the loading.Note that this iterator effectively prefetches the examples for the next batch asynchronously after the current batch is returned.
This iterator saves
-1
instead ofNone
in snapshots since some serializers do not supportNone
.Parameters: - dataset (Dataset) – Dataset to iterate.
- batch_size (int) – Number of examples within each batch.
- repeat (bool) – If
True
, it infinitely loops over the dataset. Otherwise, it stops iteration at the end of the first epoch. - shuffle (bool) – If
True
, the order of examples is shuffled at the beginning of each epoch. Otherwise, examples are extracted in the order of indexes. - n_threads (int) – Number of worker threads.
Methods
-
__enter__
()[source]¶ With statement context manager method
This method does nothing by default. Implementation may override it to better handle the internal resources by with statement.
-
__exit__
(exc_type, exc_value, traceback)[source]¶ With statement context manager method
This method does nothing by default. Implementation may override it to better handle the internal resources by with statement.
-
__next__
()[source]¶ Returns the next batch.
This is a part of the iterator protocol of Python. It may raise the
StopIteration
exception when it stops the iteration.
-
finalize
()[source]¶ Finalizes the iterator and possibly releases the resources.
This method does nothing by default. Implementation may override it to better handle the internal resources.
-
next
()[source]¶ Returns the next batch.
This is a part of the iterator protocol of Python. It may raise the
StopIteration
exception when it stops the iteration.
-
serialize
(serializer)[source]¶ Serializes the internal state of the iterator.
This is a method to support serializer protocol of Chainer.
Note
It should only serialize the internal state that changes over the iteration. It should not serializes what is set manually by users such as the batch size.
Attributes
-
epoch_detail
¶
-
previous_epoch_detail
¶
-
repeat
¶