chainer.functions.swish¶
-
chainer.functions.
swish
(x, beta)[source]¶ Swish activation function.
\[f(x, \beta) = x \cdot \sigma(\beta x),\]where \(\sigma(\cdot)\) is the sigmoid function. It has the following properties:
\[\begin{split}f(x, 0) &= \frac{x}{2}, \\ \lim_{\beta \to \infty} f(x, \beta) &= \max(0, x).\end{split}\]Parameters: - x (Variable) – Input variable of shape \((s_B, s_1, s_2, ..., s_N)\), where \(s_B\) is assumed to be the minibatch dimension.
- beta (Variable) – Parameter variable \(\beta\) of shape
\((s_1, s_2, ..., s_M)\), where \(M\) is an arbitrary
integer between \(0 \leq M \leq N\). The number of
dimensions of
beta
will be matched withx
by reshaping it as \((1, s_1, ..., s_M, 1, ... 1)\), thenbeta
andx
are multiplied together in an element-wise manner.
Returns: Output variable of the same shape as
x
.Return type: See also