Functions¶
Chainer provides variety of built-in function implementations in chainer.functions
package.
These functions usually return a Variable
object or a tuple of multiple Variable
objects.
For a Variable
argument of a function, an N-dimensional array can be passed if you do not need its gradient.
Some functions additionally supports scalar arguments.
Note
Functions implemented in Chainer consists of the following two parts:
A class that inherits
FunctionNode
, which defines forward/backward computation.A “wrapper” function around the class.
APIs listed in this page are “wrapper” of FunctionNode
implementations.
In most cases, you don’t have to use FunctionNode
classes directly.
For example, chainer.functions.sum()
is a wrapper function defined as def sum(...):
in chainer/functions/math/sum.py, and it calls its corresponding FunctionNode
implementation, Sum
.
Some functions may not have the corresponding FunctionNode
implementation; one example is chainer.functions.average()
, which is defined in chainer/functions/math/average.py, which calls other wrapper functions to calculate average.
If you are implementing your own functions, please see Define your own function.
Arithmetic functions¶
Basic arithmetic operations for Variable
s are implemented as operators.
Refer to the Notes section of Variable
for details.
chainer.functions.add()
provides better performance when accumulating three or more Variable
s at once.
Element-wise addition. |
Activation functions¶
Clipped Rectifier Unit function. |
|
Concatenated Rectified Linear Unit function. |
|
Exponential Linear Unit function. |
|
Element-wise hard-sigmoid function. |
|
Leaky Rectified Linear Unit function. |
|
Channel-wise log-softmax function. |
|
Long Short-Term Memory units as an activation function. |
|
Maxout activation function. |
|
Parametric ReLU function. |
|
Randomized Leaky Rectified Liner Unit function. |
|
Rectified Linear Unit function. |
|
Rectifier Unit function clipped at 6. |
|
Scaled Exponential Linear Unit function. |
|
Element-wise sigmoid logistic function. |
|
S-LSTM units as an activation function. |
|
Softmax function. |
|
Element-wise softplus function. |
|
Swish activation function. |
|
Elementwise hyperbolic tangent function. |
|
TreeLSTM unit as an activation function. |
Array manipulations¶
Create a new view of array with the given shape, strides, and offset. |
|
Broadcast given variables. |
|
Broadcast a given variable to a given shape. |
|
Cast an input variable to a given type. |
|
Concatenates given variables along an axis. |
|
Copies the input variable onto the specified device. |
|
Computes the depth2space transformation for subpixel calculations. |
|
Take diagonal |
|
Concatenate variables along third axis (depth wise). |
|
Expands dimensions of an input variable without copy. |
|
Flatten a given array into one dimension. |
|
Flips an input variable in reverse order along the given axis. |
|
Flip array in the left/right direction. |
|
Flip array in the up/down direction. |
|
Extract elements from array with specified shape, axes and offsets. |
|
Concatenate variables horizontally (column wise). |
|
Extract patches from an image based on the filter. |
|
Move the source axes to the destination. |
|
Pad an input variable. |
|
Pad given arrays to make a matrix. |
|
Permutates a given variable along an axis. |
|
Construct an array by repeating a given array. |
|
Reshapes an input variable without copy. |
|
Resize images to the given shape. |
|
Roll the axis backwards to the given position. |
|
Adds given values to specified elements of an array. |
|
Select elements stored in given indices. |
|
Separates an array along a given axis. |
|
Computes the space2depth transformation for subpixel calculations. |
|
2D Spatial Transformer grid. |
|
2D Spatial Transformer sampler. |
|
Splits given variables along an axis. |
|
Remove dimensions of size one from the shape of a ndarray. |
|
Concatenate variables along a new axis. |
|
Swap two axes of a variable. |
|
Construct an array by tiling a given array. |
|
Permute the dimensions of an input variable without copy. |
|
Transpose a list of Variables. |
|
Concatenate variables vertically (row wise). |
|
Choose elements depending on condition. |
Neural network connections¶
Applies a bilinear function based on given parameters. |
|
1-dimensional convolution function. |
|
Two-dimensional convolution function. |
|
3-dimensional convolution function. |
|
N-dimensional convolution function. |
|
1-dimensional deconvolution function. |
|
Two dimensional deconvolution function. |
|
3-dimensional deconvolution function. |
|
N-dimensional deconvolution function. |
|
Two-dimensional depthwise convolution function. |
|
Two-dimensional deformable convolution function using computed offset. |
|
Two-dimensional dilated convolution function. |
|
Efficient linear function for one-hot input. |
|
Linear function, or affine transformation. |
|
Two-dimensional local convolution function. |
|
Stacked Bi-directional Gated Recurrent Unit function. |
|
Stacked Bi-directional Long Short-Term Memory function. |
|
Stacked Bi-directional RNN function for sequence inputs. |
|
Stacked Uni-directional Gated Recurrent Unit function. |
|
Stacked Uni-directional Long Short-Term Memory function. |
|
Stacked Uni-directional RNN function for sequence inputs. |
|
Shift function. |
Evaluation functions¶
Computes multiclass classification accuracy of the minibatch. |
|
Computes binary classification accuracy of the minibatch. |
|
Calculates Precision, Recall, F beta Score, and support. |
|
Computes R^2(coefficient of determination) regression score function. |
|
Loss functions¶
Element-wise absolute error function. |
|
Computes the negative log-likelihood of a Bernoulli distribution. |
|
BlackOut loss function. |
|
Connectionist Temporal Classification loss function. |
|
Computes contrastive loss. |
|
Calculates negative log-likelihood of linear-chain CRF. |
|
Computes a state that maximizes a joint probability of the given CRF. |
|
Computes the sum-squared cross-covariance penalty between |
|
Computes the DeCov loss of |
|
|
Discriminative margin-based clustering loss function |
Computes the KL-divergence of Gaussian variables from the standard one. |
|
Computes the negative log-likelihood of a Gaussian distribution. |
|
Computes the hinge loss for a one-of-many classification task. |
|
Computes the Huber loss. |
|
Mean absolute error function. |
|
Mean squared error function. |
|
Negative sampling loss function. |
|
Computes cross entropy loss for pre-sigmoid activations. |
|
Computes cross entropy loss for pre-softmax activations. |
|
Squared error function. |
|
Computes triplet loss. |
Mathematical functions¶
Element-wise absolute. |
|
Elementwise arccosine function. |
|
Elementwise arcsine function. |
|
Elementwise arctangent function. |
|
Elementwise arctangent function with two arguments. |
|
Elementwise inverse hyperbolic tangent function. |
|
Returns index which holds maximum of array elements over a given axis. |
|
Returns index which holds minimum of array elements over a given axis. |
|
Calculate weighted average of array elements over a given axis. |
|
Computes the inverse of a batch of square matrices. |
|
L2 norm (a.k.a. Euclidean norm) squared. |
|
Computes the batch matrix multiplications of two sets of arrays. |
|
Elementwise summation with broadcasting. |
|
Elementwise ceil function. |
|
Cholesky Decomposition |
|
Clips (limits) elements of input variable. |
|
Elementwise cos function. |
|
Elementwise hyperbolic cosine function. |
|
Cumulative prod of array elements over a given axis. |
|
Cumulative sum of array elements over a given axis. |
|
Computes the determinant of a single square matrix. |
|
Computes the determinant of a batch of square matrices. |
|
Digamma function. |
|
Einstein summation |
|
Elementwise error function. |
|
Elementwise complementary error function. |
|
Elementwise inverse function of complementary error function. |
|
Elementwise scaled complementary error function. |
|
Elementwise inverse function of error function. |
|
Elementwise exponential function. |
|
Elementwise exponential minus one function. |
|
Fast Fourier transform. |
|
Elementwise fix function. |
|
Elementwise mod function. |
|
Elementwise floor function. |
|
Just returns input variables. |
|
Inverse fast Fourier transform. |
|
Computes the inverse of square matrix. |
|
logarithm of gamma function. |
|
Elementwise linear-interpolation function. |
|
Elementwise natural logarithm function. |
|
Elementwise logarithm function to the base 10. |
|
Elementwise natural logarithm plus one function. |
|
Elementwise logarithm function to the base 2. |
|
Logarithm of cumulative distribution function of normal distribution. |
|
Log-sum-exp of array elements over a given axis. |
|
Computes the matrix multiplication of two arrays. |
|
Maximum of array elements over a given axis. |
|
Element-wise maximum of input variables. |
|
Calculate weighted average of array elements over a given axis. |
|
Minimum of array elements over a given axis. |
|
Element-wise minimum of input variables. |
|
Elementwise cumulative distribution function of normal distribution. |
|
Elementwise inverse function of ndtr. |
|
Product of array elements over a given axis. |
|
Polygamma function. |
|
Computes elementwise reciprocal of square root of input \(x_i\). |
|
Elementwise product with broadcasting. |
|
Elementwise sin function. |
|
Elementwise hyperbolic sine function. |
|
Elementwise sign function. |
|
Computes the batched multiplication of sparse and dense matrix. |
|
Elementwise square root function. |
|
Elementwise square function. |
|
Squared difference function. |
|
Sum of array elements over a given axis. |
|
Sum elements along axes to output an array of a given shape. |
|
Elementwise hyperbolic tangent function. |
|
Elementwise tan function. |
|
Returns the tensor dot product of two arrays along specified axes. |
|
Zeta function. |
Noise injections¶
Drops elements of input variable randomly. |
|
Gaussian sampling function. |
|
Gumbel-Softmax sampling function. |
|
Linear unit regularized by simplified dropconnect. |
|
Drops elements of input variable and sets to previous variable randomly. |
Normalization functions¶
Batch normalization function. |
|
Batch renormalization function. |
|
Decorrelated batch normalization function. |
|
Batch normalization function with fixed statistics. |
|
Decorrelated batch normalization function with fixed statistics. |
|
Group normalization function. |
|
Layer normalization. |
|
Local response normalization across neighboring channels. |
|
Normalize input by L2 norm. |
Spatial pooling¶
1-dimensional spatial average pooling function. |
|
Spatial average pooling function. |
|
3-dimensional spatial average pooling function. |
|
N-dimensionally spatial average pooling function. |
|
1-dimensional spatial max pooling function. |
|
Spatial max pooling function. |
|
3-dimensional spatial max pooling function. |
|
N-dimensionally spatial max pooling function. |
|
Spatial Region of Interest (ROI) average align function. |
|
Spatial Region of Interest (ROI) average pooling function. |
|
Spatial Region of Interest (ROI) max align function. |
|
Spatial Region of Interest (ROI) max pooling function. |
|
Spatial Region of Interest (ROI) pooling function. |
|
Spatial pyramid pooling function. |
|
Inverse operation of 1-dimensional spatial pooling. |
|
Inverse operation of pooling for 2d array. |
|
Inverse operation of 3-dimensional spatial pooling. |
|
Inverse operation of N-dimensional spatial pooling. |
|
Upsampling using pooling indices. |
Utility functions¶
Calls a function without storing intermediate results. |
Function base¶
Old-style interface of a differentiable function. |
|
Adapter class to wrap Function with FunctionNode. |
|
Function node of the computational graph. |
|
Make a context manager which enables back-propagation. |
|
Make a context manager which disables back-propagation. |
|
Computes the gradient of output variables w.r.t. the input variables. |
Function hooks¶
Chainer provides a function-hook mechanism that enriches the behavior of forward and backward propagation of FunctionNode
and Function
.
Function hook for measuring memory usage of functions in cupy memory pool. |
|
Function hook that prints debug information. |
|
Function hook for measuring elapsed time of functions. |
You can also implement your own function-hook to inject arbitrary code before/after the forward/backward propagation.
Base class of hooks for Functions. |