Function¶
-
class
chainer.
Function
[source]¶ Function on variables with backpropagation ability.
All function implementations defined in
chainer.functions
inherit this class.The main feature of this class is keeping track of function applications as a backward graph. When a function is applied to
Variable
objects, itsforward()
method is called ondata
fields of input variables, and at the same time it chains references from output variables to the function and from the function to its inputs.Note
As of v1.5, a function instance cannot be used twice in any computational graphs. In order to reuse a function object multiple times, use
copy.copy()
before the function applications to make a copy of the instance.This restriction also means that we cannot make a stateful function anymore. For example, it is now not allowed to let a function hold parameters. Define a function as a pure (stateless) procedure, and use
Link
to combine it with parameter variables.Example
Let
x
an instance ofVariable
andf
an instance ofFunction
taking only one argument. Then a line>>> import numpy, chainer, chainer.functions as F >>> x = chainer.Variable(numpy.zeros(10)) >>> f = F.Identity() >>> y = f(x)
computes a new variable
y
and creates backward references. Actually, backward references are set as per the following diagram:x <--- f <--- y
If an application of another function
g
occurs as>>> g = F.Identity() >>> z = g(x)
then the graph grows with a branch:
|--- f <--- y x <-+ |--- g <--- z
Note that the branching is correctly managed on backward computation, i.e. the gradients from
f
andg
are accumulated to the gradient ofx
.Every function implementation should provide
forward_cpu()
,forward_gpu()
,backward_cpu()
andbackward_gpu()
. Alternatively, one can provideforward()
andbackward()
instead of separate methods. Backward methods have default implementations that just returnNone
, which indicates that the function is non- differentiable.Variables: - inputs – A tuple or list of input variables.
- outputs – A tuple or list of output variables.
- type_check_enable – When it is
True
, the function checks types of input arguments. SetCHAINER_TYPE_CHECK
environment variable0
to disable type check, or set the variable directly in your own program.
-
__call__
(*inputs)[source]¶ Applies forward propagation with chaining backward references.
Basic behavior is expressed in documentation of
Function
class.Note
If the
data
attribute of input variables exist on GPU device, then, before it callsforward()
method, the appropriate device is selected, so in most cases implementers do not need to take care of device selection.Parameters: inputs – Tuple of input Variable
,numpy.ndarray
orcupy.ndarray
objects. The volatile flags of all input variables must agree. If the input is annumpy.ndarray
or acupy.ndarray
, it is automatically wrapped withVariable
.Returns: One Variable
object or a tuple of multipleVariable
objects.
-
add_hook
(hook, name=None)[source]¶ Registers the function hook.
Parameters: - hook (FunctionHook) – Function hook to be registered.
- name (str) – Name of the function hook.
name must be unique among function hooks
registered to the function. If
None
, default name of the function hook is used.
-
backward
(inputs, grad_outputs)[source]¶ Applies backprop to output gradient arrays.
It delegates the procedure to
backward_cpu()
orbackward_gpu()
by default. Which it selects is determined by the type of input arrays and output gradient arrays. Implementations ofFunction
must implement either CPU/GPU methods or this method, if the function is intended to be backprop-ed.Parameters: - inputs – Tuple of input arrays.
- grad_outputs – Tuple of output gradient arrays.
Returns: Tuple of input gradient arrays. Some or all of them can be
None
, if the function is not differentiable on inputs.Return type: Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.
-
backward_cpu
(inputs, grad_outputs)[source]¶ Applies backprop to output gradient arrays on CPU.
Parameters: - inputs – Tuple of input
numpy.ndarray
object(s). - grad_outputs – Tuple of output gradient
numpy.ndarray
object(s).
Returns: Tuple of input gradient
numpy.ndarray
object(s). Some or all of them can beNone
, if the function is not differentiable on corresponding inputs.Return type: Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.- inputs – Tuple of input
-
backward_gpu
(inputs, grad_outputs)[source]¶ Applies backprop to output gradient arrays on GPU.
Parameters: - inputs – Tuple of input
cupy.ndarray
object(s). - grad_outputs – Tuple of output gradient
cupy.ndarray
object(s).
Returns: Tuple of input gradient
cupy.ndarray
object(s). Some or all of them can beNone
, if the function is not differentiable on corresponding inputs.Return type: Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.- inputs – Tuple of input
-
check_type_forward
(in_types)[source]¶ Checks types of input data before forward propagation.
Before
forward()
is called, this function is called. You need to validate types of input data in this function using the type checking utilities.Parameters: in_types (TypeInfoTuple) – The type information of input data for forward()
.
-
delete_hook
(name)[source]¶ Unregisters the function hook.
Parameters: name (str) – the name of the function hook to be unregistered.
-
forward
(inputs)[source]¶ Applies forward propagation to input arrays.
It delegates the procedure to
forward_cpu()
orforward_gpu()
by default. Which it selects is determined by the type of input arrays. Implementations ofFunction
must implement either CPU/GPU methods or this method.Parameters: inputs – Tuple of input array(s). Returns: Tuple of output array(s). Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.
-
forward_cpu
(inputs)[source]¶ Applies forward propagation to input arrays on CPU.
Parameters: inputs – Tuple of numpy.ndarray
object(s).Returns: Tuple of numpy.ndarray
object(s).Return type: tuple Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.
-
forward_gpu
(inputs)[source]¶ Applies forward propagation to input arrays on GPU.
Parameters: inputs – Tuple of cupy.ndarray
object(s).Returns: Tuple of cupy.ndarray
object(s).Return type: tuple Warning
Implementations of
Function
must take care that the return value must be a tuple even if it returns only one array.
-
label
¶ Short text that represents the function.
The default implementation returns its type name. Each function should override it to give more information.
-
local_function_hooks
¶ Ordered Dictionary of registered function hooks.
Contrary to
chainer.thread_local.function_hooks
, which registers its elements to all functions, Function hooks in this property is specific to this function.
-
unchain
()[source]¶ Purges in/out variables and this function itself from the graph.
This method is called from
Variable.unchain_backward()
method.
-
chainer.
force_backprop_mode
(*args, **kwds)[source]¶ Enable back-propagation for Variable whose volatile is auto.
When you want to enable back-propagation in
no_backprop_mode()
, call this method. In this context,Variable
object whosevolatile
attribute is'auto'
behaves like a volatile variable. That means you can disableno_backprop_mode()
in this context.If you call this method outside of
no_backprop_mode()
context, it changes nothing.Variable
object withvolatile='auto'
behaves like a volatile variable by default.In this example, the volatility of
x
andy
is'auto'
. Inno_backprop_mode()
context,y
does not have a computational graph but inforce_backprop_mode()
it has a graph.>>> with chainer.no_backprop_mode(): ... # Variable with volatile='auto' behaves like volatile='on' ... with chainer.force_backprop_mode(): ... # Variable with volatile='auto' behaves like volatile='off' ... y = x + 1
See also
See
no_backprop_mode()
for details of back-prop mode.
-
chainer.
no_backprop_mode
(*args, **kwds)[source]¶ Disable back-propagation for Variable whose volatile is auto.
In the default setting a
Variable
object whosevolatile
attribute is'auto'
behaves like a non-volatile variable. That means such aVariable
object builds a computational graph, consumes memory to store the graph, and you can execute back-propagation for it. With this context such aVariable
object behaves like a volatile variable. So, you can easily switch training and evaluation.In this example, the volatility of
x
andy
is'auto'
. So,y
does not have a computational graph.>>> x = chainer.Variable(numpy.array([1,], 'f'), volatile='auto') >>> with chainer.no_backprop_mode(): ... y = x + 1