chainer.FunctionHook

class chainer.FunctionHook

Base class of hooks for Functions.

FunctionHook is a callback object that is registered to FunctionNode. Registered function hooks are invoked before and after forward and backward operations of each function.

Function hooks that derive FunctionHook are required to implement four methods: forward_preprocess(), forward_postprocess(), backward_preprocess(), and backward_postprocess(). By default, these methods do nothing.

Specifically, when __call__() method of some function is invoked, forward_preprocess() (resp. forward_postprocess()) of all function hooks registered to this function are called before (resp. after) forward propagation.

Likewise, when backward() of some Variable is invoked, backward_preprocess() (resp. backward_postprocess()) of all function hooks registered to the function which holds this variable as a gradient are called before (resp. after) backward propagation.

There are two ways to register FunctionHook objects to FunctionNode objects.

First one is to use with statement. Function hooks hooked in this way are registered to all functions within with statement and are unregistered at the end of with statement.

Example

The following code is a simple example in which we measure the elapsed time of a part of forward propagation procedure with TimerHook, which is a subclass of FunctionHook.

>>> from chainer import function_hooks
>>> class Model(chainer.Chain):
...   def __init__(self):
...     super(Model, self).__init__()
...     with self.init_scope():
...       self.l = L.Linear(10, 10)
...   def __call__(self, x1):
...     return F.exp(self.l(x1))
>>> model1 = Model()
>>> model2 = Model()
>>> x = chainer.Variable(np.zeros((1, 10), 'f'))
>>> with chainer.function_hooks.TimerHook() as m:
...   _ = model1(x)
...   y = model2(x)
...   print("Total time : " + str(m.total_time()))
...   model3 = Model()
...   z = model3(y) 
Total time : ...

In this example, we measure the elapsed times for each forward propagation of all functions in model1 and model2 (specifically, LinearFunction and Exp of model1 and model2). Note that model3 is not a target of measurement as TimerHook is unregistered before forward propagation of model3.

Note

Chainer stores the dictionary of registered function hooks as a thread local object. So, function hooks registered are different depending on threads.

The other one is to register directly to FunctionNode object with add_hook() method. Function hooks registered in this way can be removed by delete_hook() method. Contrary to former registration method, function hooks are registered only to the function which add_hook() is called.

Parameters:name (str) – Name of this function hook.

Methods

__enter__()

__exit__(*_)

added(function=None)

Callback function invoked when a function hook is added

Parameters:function (FunctionNode) – Function object to which the function hook is added.

backward_postprocess(function, in_data, out_grad)

Callback function invoked after backward propagation.

Parameters:

• function (FunctionNode) – Function object to which the function hook is registered.
• in_data (tuple of numpy.ndarray or tuple of cupy.ndarray) – Input of forward propagation.
• out_grad (tuple of numpy.ndarray or tuple of cupy.ndarray) – Gradient data of backward propagation.

backward_preprocess(function, in_data, out_grad)

Callback function invoked before backward propagation.

Parameters:

• function (FunctionNode) – Function object to which the function hook is registered.
• in_data (tuple of numpy.ndarray or tuple of cupy.ndarray) – Input data of forward propagation.
• out_grad (tuple of numpy.ndarray or tuple of cupy.ndarray) – Gradient data of backward propagation.

deleted(function=None)

Callback function invoked when a function hook is deleted

Parameters:function (FunctionNode) – Function object to which the function hook is deleted.

forward_postprocess(function, in_data)

Callback function invoked after forward propagation.

Parameters:

• function (FunctionNode) – Function object to which the function hook is registered.
• in_data (tuple of numpy.ndarray or tuple of cupy.ndarray) – Input data of forward propagation.

forward_preprocess(function, in_data)

Callback function invoked before forward propagation.

Parameters:

• function (FunctionNode) – Function object to which the function hook is registered.
• in_data (tuple of numpy.ndarray or tuple of cupy.ndarray) – Input data of forward propagation.