Array with a structure to keep track of computation
A Variable object may be constructed in two ways: by the user or by some function. When a variable is created by some function as one of its outputs, the variable holds a reference to that function. This reference is used in error backpropagation (a.k.a. backprop). It is also used in backward unchaining. A variable that does not hold a reference to its creator is called a root variable. A variable is root if it is created by the user, or if the reference is deleted by
Users can disable this chaining behavior by setting the volatile flag for the initial variables. When a function gets volatile variables as its inputs, the output variables do not hold references to the function. This acts like unchaining on every function application.
Gradient array. It is
Noneuntil backprop reaches this variable.
The function who creates this variable. It is
Noneif the variable is not created by any function.
Boolean flag. If True, the variable does not keep track of any function applications.
Returns the number of elements of the data array.
Returns: the number of elements of the data array. Return type: int
Runs error backpropagation (a.k.a. backprop) from this variable.
Function.backward()is called on each
Functionobject appearing in the backward graph starting from this variable. The backward graph is represented by backward references from variables to their creators, and from functions to their inputs. The backprop stops at all root variables. Some functions set
Noneas gradients of some inputs, where further backprop does not take place at such input variables.
This method uses
gradas the initial error array. User can manually set a gradient array before calling this method. If
datacontains only one element (i.e., it is scalar) and
gradis None, then this method automatically complement 1.0 as the initial error. This is useful on starting backprop from some scalar loss value.
Parameters: retain_grad (bool) –
If True, the gradient arrays of all intermediate variables are kept. Otherwise,
gradof the intermediate variables are set to
Noneon appropriate timing, which may reduce the maximum memory consumption.
In most cases of training some model, the purpose of backprop is to compute gradients of parameters, not of variables, so it is recommended to set this flag False.
Notifies the variable that the given function is its creator.
Parameters: gen_func (Function) – Function object that creates this variable as one of its outputs.
Deletes backward references of variables and functions backward, a.k.a. backward unchaining.
After this method completes, intermediate variables and functions that are not referenced from anywhere are deallocated by reference count GC. Also this variable itself deletes the reference to its creator function, i.e. this variable becomes root in the computation graph. It indicates that backprop after unchaining stops at this variable. This behavior is useful to implement truncated BPTT.