Functions

Chainer provides variety of built-in function implementations in chainer.functions package. These functions usually return a Variable object or a tuple of multiple Variable objects. For a Variable argument of a function, an N-dimensional array can be passed if you do not need its gradient. Some functions additionally supports scalar arguments.

Note

Functions implemented in Chainer consists of the following two parts:

  • A class that inherits FunctionNode, which defines forward/backward computation.

  • A “wrapper” function around the class.

APIs listed in this page are “wrapper” of FunctionNode implementations. In most cases, you don’t have to use FunctionNode classes directly.

For example, chainer.functions.sum() is a wrapper function defined as def sum(...): in chainer/functions/math/sum.py, and it calls its corresponding FunctionNode implementation, Sum. Some functions may not have the corresponding FunctionNode implementation; one example is chainer.functions.average(), which is defined in chainer/functions/math/average.py, which calls other wrapper functions to calculate average.

If you are implementing your own functions, please see Define your own function.

Arithmetic functions

Basic arithmetic operations for Variables are implemented as operators. Refer to the Notes section of Variable for details.

chainer.functions.add() provides better performance when accumulating three or more Variables at once.

chainer.functions.add

Element-wise addition.

Activation functions

chainer.functions.clipped_relu

Clipped Rectifier Unit function.

chainer.functions.crelu

Concatenated Rectified Linear Unit function.

chainer.functions.elu

Exponential Linear Unit function.

chainer.functions.hard_sigmoid

Element-wise hard-sigmoid function.

chainer.functions.leaky_relu

Leaky Rectified Linear Unit function.

chainer.functions.log_softmax

Channel-wise log-softmax function.

chainer.functions.lstm

Long Short-Term Memory units as an activation function.

chainer.functions.maxout

Maxout activation function.

chainer.functions.prelu

Parametric ReLU function.

chainer.functions.rrelu

Randomized Leaky Rectified Liner Unit function.

chainer.functions.relu

Rectified Linear Unit function.

chainer.functions.relu6

Rectifier Unit function clipped at 6.

chainer.functions.selu

Scaled Exponential Linear Unit function.

chainer.functions.sigmoid

Element-wise sigmoid logistic function.

chainer.functions.slstm

S-LSTM units as an activation function.

chainer.functions.softmax

Softmax function.

chainer.functions.softplus

Element-wise softplus function.

chainer.functions.swish

Swish activation function.

chainer.functions.tanh

Elementwise hyperbolic tangent function.

chainer.functions.tree_lstm

TreeLSTM unit as an activation function.

Array manipulations

chainer.functions.as_strided

Create a new view of array with the given shape, strides, and offset.

chainer.functions.broadcast

Broadcast given variables.

chainer.functions.broadcast_to

Broadcast a given variable to a given shape.

chainer.functions.cast

Cast an input variable to a given type.

chainer.functions.concat

Concatenates given variables along an axis.

chainer.functions.copy

Copies the input variable onto the specified device.

chainer.functions.depth2space

Computes the depth2space transformation for subpixel calculations.

chainer.functions.diagonal

Take diagonal

chainer.functions.dstack

Concatenate variables along third axis (depth wise).

chainer.functions.expand_dims

Expands dimensions of an input variable without copy.

chainer.functions.flatten

Flatten a given array into one dimension.

chainer.functions.flip

Flips an input variable in reverse order along the given axis.

chainer.functions.fliplr

Flip array in the left/right direction.

chainer.functions.flipud

Flip array in the up/down direction.

chainer.functions.get_item

Extract elements from array with specified shape, axes and offsets.

chainer.functions.hstack

Concatenate variables horizontally (column wise).

chainer.functions.im2col

Extract patches from an image based on the filter.

chainer.functions.moveaxis

Move the source axes to the destination.

chainer.functions.pad

Pad an input variable.

chainer.functions.pad_sequence

Pad given arrays to make a matrix.

chainer.functions.permutate

Permutates a given variable along an axis.

chainer.functions.repeat

Construct an array by repeating a given array.

chainer.functions.reshape

Reshapes an input variable without copy.

chainer.functions.resize_images

Resize images to the given shape.

chainer.functions.rollaxis

Roll the axis backwards to the given position.

chainer.functions.scatter_add

Adds given values to specified elements of an array.

chainer.functions.select_item

Select elements stored in given indices.

chainer.functions.separate

Separates an array along a given axis.

chainer.functions.space2depth

Computes the space2depth transformation for subpixel calculations.

chainer.functions.spatial_transformer_grid

2D Spatial Transformer grid.

chainer.functions.spatial_transformer_sampler

2D Spatial Transformer sampler.

chainer.functions.split_axis

Splits given variables along an axis.

chainer.functions.squeeze

Remove dimensions of size one from the shape of a ndarray.

chainer.functions.stack

Concatenate variables along a new axis.

chainer.functions.swapaxes

Swap two axes of a variable.

chainer.functions.tile

Construct an array by tiling a given array.

chainer.functions.transpose

Permute the dimensions of an input variable without copy.

chainer.functions.transpose_sequence

Transpose a list of Variables.

chainer.functions.vstack

Concatenate variables vertically (row wise).

chainer.functions.where

Choose elements depending on condition.

Neural network connections

chainer.functions.bilinear

Applies a bilinear function based on given parameters.

chainer.functions.convolution_1d

1-dimensional convolution function.

chainer.functions.convolution_2d

Two-dimensional convolution function.

chainer.functions.convolution_3d

3-dimensional convolution function.

chainer.functions.convolution_nd

N-dimensional convolution function.

chainer.functions.deconvolution_1d

1-dimensional deconvolution function.

chainer.functions.deconvolution_2d

Two dimensional deconvolution function.

chainer.functions.deconvolution_3d

3-dimensional deconvolution function.

chainer.functions.deconvolution_nd

N-dimensional deconvolution function.

chainer.functions.depthwise_convolution_2d

Two-dimensional depthwise convolution function.

chainer.functions.deformable_convolution_2d_sampler

Two-dimensional deformable convolution function using computed offset.

chainer.functions.dilated_convolution_2d

Two-dimensional dilated convolution function.

chainer.functions.embed_id

Efficient linear function for one-hot input.

chainer.functions.linear

Linear function, or affine transformation.

chainer.functions.local_convolution_2d

Two-dimensional local convolution function.

chainer.functions.n_step_bigru

Stacked Bi-directional Gated Recurrent Unit function.

chainer.functions.n_step_bilstm

Stacked Bi-directional Long Short-Term Memory function.

chainer.functions.n_step_birnn

Stacked Bi-directional RNN function for sequence inputs.

chainer.functions.n_step_gru

Stacked Uni-directional Gated Recurrent Unit function.

chainer.functions.n_step_lstm

Stacked Uni-directional Long Short-Term Memory function.

chainer.functions.n_step_rnn

Stacked Uni-directional RNN function for sequence inputs.

chainer.functions.shift

Shift function.

Evaluation functions

chainer.functions.accuracy

Computes multiclass classification accuracy of the minibatch.

chainer.functions.binary_accuracy

Computes binary classification accuracy of the minibatch.

chainer.functions.classification_summary

Calculates Precision, Recall, F beta Score, and support.

chainer.functions.f1_score

chainer.functions.precision

chainer.functions.r2_score

Computes R^2(coefficient of determination) regression score function.

chainer.functions.recall

Loss functions

chainer.functions.absolute_error

Element-wise absolute error function.

chainer.functions.bernoulli_nll

Computes the negative log-likelihood of a Bernoulli distribution.

chainer.functions.black_out

BlackOut loss function.

chainer.functions.connectionist_temporal_classification

Connectionist Temporal Classification loss function.

chainer.functions.contrastive

Computes contrastive loss.

chainer.functions.crf1d

Calculates negative log-likelihood of linear-chain CRF.

chainer.functions.argmax_crf1d

Computes a state that maximizes a joint probability of the given CRF.

chainer.functions.cross_covariance

Computes the sum-squared cross-covariance penalty between y and z

chainer.functions.decov

Computes the DeCov loss of h

chainer.functions.discriminative_margin_based_clustering_loss

Discriminative margin-based clustering loss function

chainer.functions.gaussian_kl_divergence

Computes the KL-divergence of Gaussian variables from the standard one.

chainer.functions.gaussian_nll

Computes the negative log-likelihood of a Gaussian distribution.

chainer.functions.hinge

Computes the hinge loss for a one-of-many classification task.

chainer.functions.huber_loss

Computes the Huber loss.

chainer.functions.mean_absolute_error

Mean absolute error function.

chainer.functions.mean_squared_error

Mean squared error function.

chainer.functions.negative_sampling

Negative sampling loss function.

chainer.functions.sigmoid_cross_entropy

Computes cross entropy loss for pre-sigmoid activations.

chainer.functions.softmax_cross_entropy

Computes cross entropy loss for pre-softmax activations.

chainer.functions.squared_error

Squared error function.

chainer.functions.triplet

Computes triplet loss.

Mathematical functions

chainer.functions.absolute

Element-wise absolute.

chainer.functions.arccos

Elementwise arccosine function.

chainer.functions.arcsin

Elementwise arcsine function.

chainer.functions.arctan

Elementwise arctangent function.

chainer.functions.arctan2

Elementwise arctangent function with two arguments.

chainer.functions.arctanh

Elementwise inverse hyperbolic tangent function.

chainer.functions.argmax

Returns index which holds maximum of array elements over a given axis.

chainer.functions.argmin

Returns index which holds minimum of array elements over a given axis.

chainer.functions.average

Calculate weighted average of array elements over a given axis.

chainer.functions.batch_inv

Computes the inverse of a batch of square matrices.

chainer.functions.batch_l2_norm_squared

L2 norm (a.k.a. Euclidean norm) squared.

chainer.functions.batch_matmul

Computes the batch matrix multiplications of two sets of arrays.

chainer.functions.bias

Elementwise summation with broadcasting.

chainer.functions.ceil

Elementwise ceil function.

chainer.functions.cholesky

Cholesky Decomposition

chainer.functions.clip

Clips (limits) elements of input variable.

chainer.functions.cos

Elementwise cos function.

chainer.functions.cosh

Elementwise hyperbolic cosine function.

chainer.functions.cumprod

Cumulative prod of array elements over a given axis.

chainer.functions.cumsum

Cumulative sum of array elements over a given axis.

chainer.functions.det

Computes the determinant of a single square matrix.

chainer.functions.batch_det

Computes the determinant of a batch of square matrices.

chainer.functions.digamma

Digamma function.

chainer.functions.einsum

Einstein summation

chainer.functions.erf

Elementwise error function.

chainer.functions.erfc

Elementwise complementary error function.

chainer.functions.erfcinv

Elementwise inverse function of complementary error function.

chainer.functions.erfcx

Elementwise scaled complementary error function.

chainer.functions.erfinv

Elementwise inverse function of error function.

chainer.functions.exp

Elementwise exponential function.

chainer.functions.expm1

Elementwise exponential minus one function.

chainer.functions.fft

Fast Fourier transform.

chainer.functions.fix

Elementwise fix function.

chainer.functions.fmod

Elementwise mod function.

chainer.functions.floor

Elementwise floor function.

chainer.functions.identity

Just returns input variables.

chainer.functions.ifft

Inverse fast Fourier transform.

chainer.functions.inv

Computes the inverse of square matrix.

chainer.functions.lgamma

logarithm of gamma function.

chainer.functions.linear_interpolate

Elementwise linear-interpolation function.

chainer.functions.log

Elementwise natural logarithm function.

chainer.functions.log10

Elementwise logarithm function to the base 10.

chainer.functions.log1p

Elementwise natural logarithm plus one function.

chainer.functions.log2

Elementwise logarithm function to the base 2.

chainer.functions.log_ndtr

Logarithm of cumulative distribution function of normal distribution.

chainer.functions.logsumexp

Log-sum-exp of array elements over a given axis.

chainer.functions.matmul

Computes the matrix multiplication of two arrays.

chainer.functions.max

Maximum of array elements over a given axis.

chainer.functions.maximum

Element-wise maximum of input variables.

chainer.functions.mean

Calculate weighted average of array elements over a given axis.

chainer.functions.min

Minimum of array elements over a given axis.

chainer.functions.minimum

Element-wise minimum of input variables.

chainer.functions.ndtr

Elementwise cumulative distribution function of normal distribution.

chainer.functions.ndtri

Elementwise inverse function of ndtr.

chainer.functions.prod

Product of array elements over a given axis.

chainer.functions.polygamma

Polygamma function.

chainer.functions.rsqrt

Computes elementwise reciprocal of square root of input \(x_i\).

chainer.functions.scale

Elementwise product with broadcasting.

chainer.functions.sin

Elementwise sin function.

chainer.functions.sinh

Elementwise hyperbolic sine function.

chainer.functions.sign

Elementwise sign function.

chainer.functions.sparse_matmul

Computes the batched multiplication of sparse and dense matrix.

chainer.functions.sqrt

Elementwise square root function.

chainer.functions.square

Elementwise square function.

chainer.functions.squared_difference

Squared difference function.

chainer.functions.sum

Sum of array elements over a given axis.

chainer.functions.sum_to

Sum elements along axes to output an array of a given shape.

chainer.functions.tanh

Elementwise hyperbolic tangent function.

chainer.functions.tan

Elementwise tan function.

chainer.functions.tensordot

Returns the tensor dot product of two arrays along specified axes.

chainer.functions.zeta

Zeta function.

Noise injections

chainer.functions.dropout

Drops elements of input variable randomly.

chainer.functions.gaussian

Gaussian sampling function.

chainer.functions.gumbel_softmax

Gumbel-Softmax sampling function.

chainer.functions.simplified_dropconnect

Linear unit regularized by simplified dropconnect.

chainer.functions.zoneout

Drops elements of input variable and sets to previous variable randomly.

Normalization functions

chainer.functions.batch_normalization

Batch normalization function.

chainer.functions.batch_renormalization

Batch renormalization function.

chainer.functions.decorrelated_batch_normalization

Decorrelated batch normalization function.

chainer.functions.fixed_batch_normalization

Batch normalization function with fixed statistics.

chainer.functions.fixed_batch_renormalization

chainer.functions.fixed_decorrelated_batch_normalization

Decorrelated batch normalization function with fixed statistics.

chainer.functions.group_normalization

Group normalization function.

chainer.functions.layer_normalization

Layer normalization.

chainer.functions.local_response_normalization

Local response normalization across neighboring channels.

chainer.functions.normalize

Normalize input by L2 norm.

Spatial pooling

chainer.functions.average_pooling_1d

1-dimensional spatial average pooling function.

chainer.functions.average_pooling_2d

Spatial average pooling function.

chainer.functions.average_pooling_3d

3-dimensional spatial average pooling function.

chainer.functions.average_pooling_nd

N-dimensionally spatial average pooling function.

chainer.functions.max_pooling_1d

1-dimensional spatial max pooling function.

chainer.functions.max_pooling_2d

Spatial max pooling function.

chainer.functions.max_pooling_3d

3-dimensional spatial max pooling function.

chainer.functions.max_pooling_nd

N-dimensionally spatial max pooling function.

chainer.functions.roi_average_align_2d

Spatial Region of Interest (ROI) average align function.

chainer.functions.roi_average_pooling_2d

Spatial Region of Interest (ROI) average pooling function.

chainer.functions.roi_max_align_2d

Spatial Region of Interest (ROI) max align function.

chainer.functions.roi_max_pooling_2d

Spatial Region of Interest (ROI) max pooling function.

chainer.functions.roi_pooling_2d

Spatial Region of Interest (ROI) pooling function.

chainer.functions.spatial_pyramid_pooling_2d

Spatial pyramid pooling function.

chainer.functions.unpooling_1d

Inverse operation of 1-dimensional spatial pooling.

chainer.functions.unpooling_2d

Inverse operation of pooling for 2d array.

chainer.functions.unpooling_3d

Inverse operation of 3-dimensional spatial pooling.

chainer.functions.unpooling_nd

Inverse operation of N-dimensional spatial pooling.

chainer.functions.upsampling_2d

Upsampling using pooling indices.

Utility functions

chainer.functions.forget

Calls a function without storing intermediate results.

Function base

chainer.Function

Old-style interface of a differentiable function.

chainer.FunctionAdapter

Adapter class to wrap Function with FunctionNode.

chainer.FunctionNode

Function node of the computational graph.

chainer.force_backprop_mode

Make a context manager which enables back-propagation.

chainer.no_backprop_mode

Make a context manager which disables back-propagation.

chainer.grad

Computes the gradient of output variables w.r.t. the input variables.

Function hooks

Chainer provides a function-hook mechanism that enriches the behavior of forward and backward propagation of FunctionNode and Function.

chainer.function_hooks.CUDAProfileHook

chainer.function_hooks.CupyMemoryProfileHook

Function hook for measuring memory usage of functions in cupy memory pool.

chainer.function_hooks.PrintHook

Function hook that prints debug information.

chainer.function_hooks.TimerHook

Function hook for measuring elapsed time of functions.

You can also implement your own function-hook to inject arbitrary code before/after the forward/backward propagation.

chainer.FunctionHook

Base class of hooks for Functions.