chainer.UpdateRule

class chainer.UpdateRule(parent_hyperparam=None)[source]

Base class of all update rules.

Update rule is an object that implements how to update one parameter variable using the gradient of a loss function. This class provides the interface and the common features of any update rules.

An update rule can be set to a Variable object that represents a parameter array of a model. An Optimizer instance defines which parameters to update, and the update rule instance of each parameter defines how to update it.

Hook functions can be set to any update rule instance. The hook function is called just before or after any updates (configurable) in the order of registrations.

An implementation of update rule should override update_core() or its device-dependent variants (i.e., update_core_cpu() and update_core_gpu()).

The state (e.g. a moving average of the gradient) of the update rule is stored into the state dictionary. An implementation of update rule using state should also override init_state() to initialize the state at the first update. The values of the state dictionary are automatically copied to the appropriate device before the update based on the data and grad arrays.

Parameters

parent_hyperparam (Hyperparameter) – Hyperparameter that provides the default values.

Variables
  • ~UpdateRule.enabled (bool) – Flag to configure if this update rule is active. If the update rule is not active (i.e., enabled = False), the update() method does not update the parameter.

  • ~UpdateRule.hyperparam (Hyperparameter) – Hyperparameter of the update rule.

  • ~UpdateRule.t (int) – Number of updates made by this update rule.

Methods

add_hook(hook, name=None, timing='auto')[source]

Adds a hook function.

The hook function is called before or after any updates (see the timing attribute).

Parameters
  • hook (callable) – Hook function to be added. It takes two arguments: the update rule object and the parameter variable.

  • name (str) – Name of the hook function. The name attribute of the hook function is used by default.

  • timing (str) – Specifies when the hook is called. If ‘auto’, the timimg property of the hook will decide the timing. If ‘pre’, the hook will be called before any updates. If ‘post’, the hook will be called after any updates. If ‘auto’ and the timing property of the hook is not available, timing will default to ‘pre’.

init_state(param)[source]

Initializes the state.

Any implementations that use the state should override this mehtod. This method is called at the first update.

Parameters

param (Variable) – Parameter variable. It can be used to extract the shape and the data type of the parameter.

remove_hook(name)[source]

Removes the specified hook function.

Parameters

name (str) – Name of the hook function to be removed. The hook function registered with this name will be removed.

serialize(serializer)[source]

Serializes the update rule state.

Be careful that this method only saves/loads the state of the update rule. The parameters of the target link is not saved/loaded by this method, and so you need to serialize the target link separately if you want to fully recover the training state including parameters.

Parameters

serializer (AbstractSerializer) – Serializer object.

update(param)[source]

Invokes hook functions and updates the parameter.

Parameters

param (Variable) – Variable to be updated.

update_core(param)[source]

Updates the parameter.

Implementation of UpdateRule should override this method or both of update_core_cpu() and update_core_gpu().

Parameters

param (Variable) – Variable to be updated.

update_core_chainerx(param)[source]

Updates the ChainerX parameter.

This method can be overridden to implement custom update logic. The default implementation is to convert the parameter to a memory-shared NumPy/CuPy parameter and call the corresponding update method.

See update_core() for details.

Parameters

param (Variable) – Variable to be updated.

update_core_cpu(param)[source]

Updates the parameter on CPU.

See update_core() for details.

Parameters

param (Variable) – Variable to be updated.

update_core_gpu(param)[source]

Updates the parameter on GPU.

See update_core() for details.

Parameters

param (Variable) – Variable to be updated.

use_fp32_update(flag=True)[source]

Enables use of parameter update in fp32.

This method enables use of parameter update in fp32. When it is enabled and data type of original parameter variable is fp16, fp32 copy of parameter variable is automatically created and retained at self.fp32_param. And the parameter is update in fp32 in the following way.

  1. copies the grad of original parameter variable to the grad of fp32 parameter variable, converting its data type from fp16 to fp32.

  2. updates the parameter in fp32.

  3. copies the data of fp32 parameter variable to the data of original parameter variable, converting its data type from fp32 to fp16.

See update() for details.

__eq__(value, /)

Return self==value.

__ne__(value, /)

Return self!=value.

__lt__(value, /)

Return self<value.

__le__(value, /)

Return self<=value.

__gt__(value, /)

Return self>value.

__ge__(value, /)

Return self>=value.

Attributes

is_elementwise = False
state

State dictionary.