chainer.UpdateRule¶
-
class
chainer.
UpdateRule
(parent_hyperparam=None)[source]¶ Base class of all update rules.
Update rule is an object that implements how to update one parameter variable using the gradient of a loss function. This class provides the interface and the common features of any update rules.
An update rule can be set to a
Variable
object that represents a parameter array of a model. AnOptimizer
instance defines which parameters to update, and the update rule instance of each parameter defines how to update it.Hook functions can be set to any update rule instance. The hook function is called just before or after any updates (configurable) in the order of registrations.
An implementation of update rule should override
update_core()
or its device-dependent variants (i.e.,update_core_cpu()
andupdate_core_gpu()
).The state (e.g. a moving average of the gradient) of the update rule is stored into the state dictionary. An implementation of update rule using state should also override
init_state()
to initialize the state at the first update. The values of the state dictionary are automatically copied to the appropriate device before the update based on the data and grad arrays.- Parameters
parent_hyperparam (Hyperparameter) – Hyperparameter that provides the default values.
- Variables
enabled (bool) – Flag to configure if this update rule is active. If the update rule is not active (i.e.,
enabled = False
), theupdate()
method does not update the parameter.hyperparam (Hyperparameter) – Hyperparameter of the update rule.
t (int) – Number of updates made by this update rule.
Methods
-
add_hook
(hook, name=None, timing='auto')[source]¶ Adds a hook function.
The hook function is called before or after any updates (see the timing attribute).
- Parameters
hook (callable) – Hook function to be added. It takes two arguments: the update rule object and the parameter variable.
name (str) – Name of the hook function. The name attribute of the hook function is used by default.
timing (str) – Specifies when the hook is called. If ‘auto’, the timimg property of the hook will decide the timing. If ‘pre’, the hook will be called before any updates. If ‘post’, the hook will be called after any updates. If ‘auto’ and the timing property of the hook is not available, timing will default to ‘pre’.
-
init_state
(param)[source]¶ Initializes the state.
Any implementations that use the state should override this mehtod. This method is called at the first update.
- Parameters
param (Variable) – Parameter variable. It can be used to extract the shape and the data type of the parameter.
-
remove_hook
(name)[source]¶ Removes the specified hook function.
- Parameters
name (str) – Name of the hook function to be removed. The hook function registered with this name will be removed.
-
serialize
(serializer)[source]¶ Serializes the update rule state.
Be careful that this method only saves/loads the state of the update rule. The parameters of the target link is not saved/loaded by this method, and so you need to serialize the target link separately if you want to fully recover the training state including parameters.
- Parameters
serializer (AbstractSerializer) – Serializer object.
-
update
(param)[source]¶ Invokes hook functions and updates the parameter.
- Parameters
param (Variable) – Variable to be updated.
-
update_core
(param)[source]¶ Updates the parameter.
Implementation of UpdateRule should override this method or both of
update_core_cpu()
andupdate_core_gpu()
.- Parameters
param (Variable) – Variable to be updated.
-
update_core_chainerx
(param)[source]¶ Updates the ChainerX parameter.
This method can be overridden to implement custom update logic. The default implementation is to convert the parameter to a memory-shared NumPy/CuPy parameter and call the corresponding update method.
See
update_core()
for details.- Parameters
param (Variable) – Variable to be updated.
-
update_core_cpu
(param)[source]¶ Updates the parameter on CPU.
See
update_core()
for details.- Parameters
param (Variable) – Variable to be updated.
-
update_core_gpu
(param)[source]¶ Updates the parameter on GPU.
See
update_core()
for details.- Parameters
param (Variable) – Variable to be updated.
-
use_fp32_update
(flag=True)[source]¶ Enables use of parameter update in fp32.
This method enables use of parameter update in fp32. When it is enabled and data type of original parameter variable is fp16, fp32 copy of parameter variable is automatically created and retained at self.fp32_param. And the parameter is update in fp32 in the following way.
copies the grad of original parameter variable to the grad of fp32 parameter variable, converting its data type from fp16 to fp32.
updates the parameter in fp32.
copies the data of fp32 parameter variable to the data of original parameter variable, converting its data type from fp32 to fp16.
See
update()
for details.
-
__eq__
()¶ Return self==value.
-
__ne__
()¶ Return self!=value.
-
__lt__
()¶ Return self<value.
-
__le__
()¶ Return self<=value.
-
__gt__
()¶ Return self>value.
-
__ge__
()¶ Return self>=value.
Attributes
-
is_elementwise
= False¶
-
state
¶ State dictionary.