Chainer
stable

Tutorials

  • Chainer at a Glance
  • Concepts Walkthrough
    • Define-by-Run
    • Variables and Derivatives
    • Links
    • Define your own function
    • Creating Models
    • Optimizer
    • Trainer
    • Trainer Extensions
    • Using GPU(s) in Chainer
    • Type Checks
    • Serializers – saving and loading
    • Customize your own logging

Examples

  • Neural Net Examples
  • Colab Notebook Examples
  • Awesome Chainer

References

  • API Reference
  • Installation
  • ChainerX Documentation
  • Distributed Deep Learning with ChainerMN
  • Export Chainer to ONNX

Other

  • API Compatibility Policy
  • Contribution Guide
  • Tips and FAQs
  • Performance Best Practices
  • Upgrade Guide
  • License

Community

  • Slack Chat
  • Forums
Chainer
  • Docs »
  • Concepts Walkthrough »
  • Links
  • Edit on GitHub

Links¶

In order to write neural networks, we have to combine functions with parameters and optimize the parameters. You can use the class Link to do this. A Link is an object that holds parameters (i.e. optimization targets).

The most fundamental ones are links that behave like regular functions while replacing some arguments by their parameters. We will introduce higher level links, but here think of links as simply functions with parameters.

One of the most frequently used links is the Linear link (a.k.a. fully-connected layer or affine transformation). It represents a mathematical function \(f(x) = Wx + b\), where the matrix \(W\) and the vector \(b\) are parameters. This link corresponds to its pure counterpart linear(), which accepts \(x, W, b\) as arguments. A linear link from three-dimensional space to two-dimensional space is defined by the following line:

>>> f = L.Linear(3, 2)

Note

Most functions and links only accept mini-batch input, where the first dimension of the input array is considered as the batch dimension. In the above Linear link case, input must have shape of \((N, 3)\), where \(N\) is the mini-batch size.

The parameters of a link are stored as attributes. Each parameter is an instance of Variable. In the case of the Linear link, two parameters, W and b, are stored. By default, the matrix W is initialized randomly, while the vector b is initialized with zeros. This is the preferred way to initialize these parameters.

>>> f.W.array
array([[ 1.0184761 ,  0.23103087,  0.5650746 ],
       [ 1.2937803 ,  1.0782351 , -0.56423163]], dtype=float32)
>>> f.b.array
array([0., 0.], dtype=float32)

An instance of the Linear link acts like a usual function:

>>> x = Variable(np.array([[1, 2, 3], [4, 5, 6]], dtype=np.float32))
>>> y = f(x)
>>> y.array
array([[3.1757617, 1.7575557],
       [8.619507 , 7.1809077]], dtype=float32)

Note

Sometimes it is cumbersome to compute the dimension of the input space. The linear link and some of (de)convolution links can omit the input dimension in their instantiation and infer it from the first mini-batch.

For example, the following line creates a linear link whose output dimension is two:

>>> f = L.Linear(2)

If we feed a mini-batch of shape \((2, M)\), the input dimension will be inferred as M, which means l.W will be a 2 x M matrix. Note that its parameters are initialized in a lazy manner at the first mini-batch. Therefore, l does not have W attribute if no data is put to the link.

Gradients of parameters are computed by the backward() method. Note that gradients are accumulated by the method rather than overwritten. So first you must clear the gradients to renew the computation. It can be done by calling the cleargrads() method.

>>> f.cleargrads()

Now we can compute the gradients of parameters by simply calling the backward method and access them via the grad property.

>>> y.grad = np.ones((2, 2), dtype=np.float32)
>>> y.backward()
>>> f.W.grad
array([[5., 7., 9.],
       [5., 7., 9.]], dtype=float32)
>>> f.b.grad
array([2., 2.], dtype=float32)
Next Previous

© Copyright 2015, Preferred Networks, inc. and Preferred Infrastructure, inc. Revision e9da1423.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: stable
Versions
latest
stable
v7.8.1
v7.8.0
v7.7.0
v7.4.0
v7.2.0
v7.1.0
v7.0.0
v6.7.0
v6.6.0
v6.5.0
v6.4.0
v6.3.0
v6.2.0
v6.1.0
v6.0.0
v6
v5.4.0
v5.3.0
v5.2.0
v5.1.0
v5.0.0
v4.5.0
v4.4.0
v4.3.1
v4.3.0
v4.2.0
v4.1.0
v4.0.0
v3.5.0
v3.4.0
v3.3.0
v3.2.0
v3.1.0
v3.0.0
v2.1.0
v2.0.2
v1.24.0
v1.23.0
v1.22.0
v1.21.0
v1.20.0.1
v1.19.0
v1.18.0
v1.17.0
v1.16.0
v1.15.0.1
v1.14.0
v1.13.0
v1.12.0
v1.11.0
v1.10.0
v1.9.1
v1.8.2
v1.7.2
v1.6.2.1
v1.5.1
v1.4.1
v1.3.2
v1.2.0
v1.1.2
v1.0.1
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.