Optimizers

chainer.optimizers.AdaDelta Zeiler’s ADADELTA.
chainer.optimizers.AdaGrad AdaGrad optimizer.
chainer.optimizers.Adam Adam optimizer.
chainer.optimizers.MomentumSGD Momentum SGD optimizer.
chainer.optimizers.NesterovAG Nesterov’s Accelerated Gradient.
chainer.optimizers.RMSprop RMSprop optimizer.
chainer.optimizers.RMSpropGraves Alex Graves’s RMSprop.
chainer.optimizers.SGD Vanilla Stochastic Gradient Descent.
chainer.optimizers.SMORMS3 Simon Funk’s SMORMS3.

Optimizer base classes

chainer.Optimizer Base class of all numerical optimizers.
chainer.UpdateRule Base class of all update rules.
chainer.optimizer.Hyperparameter Set of hyperparameter entries of an optimizer.
chainer.GradientMethod Base class of all single gradient-based optimizers.

Hook functions

chainer.optimizer_hooks.WeightDecay Optimizer/UpdateRule hook function for weight decay regularization.
chainer.optimizer_hooks.Lasso Optimizer/UpdateRule hook function for Lasso regularization.
chainer.optimizer_hooks.GradientClipping Optimizer hook function for gradient clipping.
chainer.optimizer_hooks.GradientHardClipping Optimizer/UpdateRule hook function for gradient clipping.
chainer.optimizer_hooks.GradientNoise Optimizer/UpdateRule hook function for adding gradient noise.
chainer.optimizer_hooks.GradientLARS Optimizer/UpdateRule hook function for layer wise adaptive rate scaling.