Assertion and Testing¶

Chainer provides some facilities to make debugging easy.

Type checking utilities¶

Function uses a systematic type checking of the chainer.utils.type_check module. It enables users to easily find bugs of forward and backward implementations. You can find examples of type checking in some function implementations.

class chainer.utils.type_check.Expr(priority)[source]¶

Abstract syntax tree of an expression.

It represents an abstract syntax tree, and isn’t a value. You can get its actual value with eval() function, and get syntax representation with the __str__() method. Each comparison operator (e.g. ==) generates a new Expr object which represents the result of comparison between two expressions.

Example

Let x and y be instances of Expr, then

>>> x = Variable(1, 'x')
>>> y = Variable(1, 'y')
>>> c = (x == y)

is also an instance of Expr. To evaluate and get its value, call eval() method:

>>> c.eval()
True

Call str function to get a representation of the original equation:

>>> str(c)
'x == y'

You can actually compare an expression with a value:

>>> (x == 1).eval()
True

Note that you can’t use boolean operators such as and, as they try to cast expressions to boolean values:

>>> z = Variable(1, 'z')
>>> x == y and y == z  # raises an error
Traceback (most recent call last):
RuntimeError: Don't convert Expr to bool. Please call Expr.eval method to evaluate expression.

eval()[source]¶

Evaluates the tree to get actual value.

Behavior of this function depends on an implementation class. For example, a binary operator + calls the __add__ function with the two results of eval() function.

chainer.utils.type_check.expect(*bool_exprs)[source]¶

Evaluates and tests all given expressions.

This function evaluates given boolean expressions in order. When at least one expression is evaluated as False, that means the given condition is not satisfied. You can check conditions with this function.

Parameters:	bool_exprs (tuple of Bool expressions) – Bool expressions you want to evaluate.

class chainer.utils.type_check.TypeInfo(shape, dtype)[source]¶

Type information of an input/gradient array.

It contains type information of an array, such as the shape of array and the number of dimensions. This information is independent of CPU or GPU array.

class chainer.utils.type_check.TypeInfoTuple[source]¶

Type information of input/gradient tuples.

It is a sub-class of tuple containing TypeInfo. The i-th element of this object contains type information of the i-th input/gradient data. As each element is Expr, you can easily check its validity.

size()[source]¶

Returns an expression representing its length.

Returns:	An expression object representing length of the tuple.
Return type:	Expr

Gradient checking utilities¶

Most function implementations are numerically tested by gradient checking. This method computes numerical gradients of forward routines and compares their results with the corresponding backward routines. It enables us to make the source of issues clear when we hit an error of gradient computations. The chainer.gradient_check module makes it easy to implement the gradient checking.

chainer.gradient_check.check_backward(func, x_data, y_grad, params=(), eps=0.001, atol=1e-05, rtol=0.0001, no_grads=None, dtype=None)[source]¶

Test backward procedure of a given function.

This function automatically check backward-process of given function. For example, when you have a Function class MyFunc, that gets two arguments and returns one value, you can make its test like this:

>> def test_my_func(self):
>>   func = MyFunc()
>>   x1_data = xp.array(...)
>>   x2_data = xp.array(...)
>>   gy_data = xp.array(...)
>>   check_backward(func, (x1_data, x2_data), gy_data)

This method creates Variable objects with x_data and calls func with the Variable s to get its result as Variable. Then, it sets y_grad array to grad attribute of the result and calls backward method to get gradients of the inputs. To check correctness of the gradients, the function calls numerical_grad() to calculate numerically the gradients and compares the types of gradients with chainer.testing.assert_allclose(). If input objects (x1_data or/and x2_data in this example) represent integer variables, their gradients are ignored.

You can simplify a test when MyFunc gets only one argument:

>>   check_backward(func, x1_data, gy_data)

If MyFunc is a loss function which returns a zero-dimensional array, pass None to gy_data. In this case, it sets 1 to grad attribute of the result:

>>   check_backward(my_loss_func, (x1_data, x2_data), None)

If MyFunc returns multiple outputs, pass all gradients for outputs as a tuple:

>>   gy1_data = xp.array(...)
>>   gy2_data = xp.array(...)
>>   check_backward(func, x1_data, (gy1_data, gy2_data))

You can also test a Link. To check gradients of parameters of the link, set a tuple of the parameters to params arguments:

>>   check_backward(my_link, (x1_data, x2_data), gy_data,
>>                  (my_link.W, my_link.b))

Note that params are not ndarray s, but Variables s.

Function objects are acceptable as func argument:

>>   check_backward(lambda x1, x2: f(x1, x2),
>>                  (x1_data, x2_data), gy_data)

Note

func is called many times to get numerical gradients for all inputs. This function doesn’t work correctly when func behaves randomly as it gets different gradients.

Parameters:

func (callable) – A function which gets Variable s and returns Variable s. func must returns a tuple of Variable s or one Variable. You can use Function object, Link object or a function satisfying the condition.
x_data (ndarray or tuple of ndarrays) – A set of ndarray s to be passed to func. If x_data is one ndarray object, it is treated as (x_data,).
y_grad (ndarray or tuple of ndarrays or None) – A set of ndarray s representing gradients of return-values of func. If y_grad is one ndarray object, it is treated as (y_grad,). If func is a loss-function, y_grad should be set to None.
params (Variable or tuple of ~chainder.Variable) – A set of Variable s whose gradients are checked. When func is a Link object, set its parameters as params. If params is one Variable object, it is treated as (params,).
eps (float) – Epsilon value to be passed to numerical_grad().
atol (float) – Absolute tolerance to be passed to chainer.testing.assert_allclose().
rtol (float) – Relative tolerance to be passed to chainer.testing.assert_allclose().
no_grads (list of bool) – Flag to skip variable for gradient assertion. It should be same length as x_data.
dtype (dtype) – x_data and y_grad are casted to this dtype when calculating numerical gradients. Only float types and None are allowed.

See:: numerical_grad()

chainer.gradient_check.numerical_grad(f, inputs, grad_outputs, eps=0.001)[source]¶

Computes numerical gradient by finite differences.

This function is used to implement gradient check. For usage example, see unit tests of chainer.functions.

Parameters:	f (function) – Python function with no arguments that runs forward computation and returns the result. inputs (tuple of arrays) – Tuple of arrays that should be treated as inputs. Each element of them is slightly modified to realize numerical gradient by finite differences. grad_outputs (tuple of arrays) – Tuple of arrays that are treated as output gradients. eps (float) – Epsilon value of finite differences.
Returns:	Numerical gradient arrays corresponding to `inputs`.
Return type:	tuple

Standard Assertions¶

The assertions have same names as NumPy’s ones. The difference from NumPy is that they can accept both numpy.ndarray and cupy.ndarray.

chainer.testing.assert_allclose(x, y, atol=1e-05, rtol=0.0001, verbose=True)[source]¶

Asserts if some corresponding element of x and y differs too much.

This function can handle both CPU and GPU arrays simultaneously.

Parameters:	x – Left-hand-side array. y – Right-hand-side array. atol (float) – Absolute tolerance. rtol (float) – Relative tolerance. verbose (bool) – If `True`, it outputs verbose messages on error.

Function testing utilities¶

Chainer provides some utilities for testing its functions.

chainer.testing.unary_math_function_unittest(func, func_expected=None, label_expected=None, make_data=None)[source]¶

Decorator for testing unary mathematical Chainer functions.

This decorator makes test classes test unary mathematical Chainer functions. Tested are forward and backward computations on CPU and GPU across parameterized shape and dtype.

Parameters:

func (Function) – Chainer function to be tested by the decorated test class.
func_expected – Function used to provide expected values for testing forward computation. If not given, a corresponsing numpy function for func is implicitly picked up by its class name.
label_expected (string) – String used to test labels of Chainer functions. If not given, the class name of func lowered is implicitly used.
make_data – Function to customize input and gradient data used in the tests. It takes shape and dtype as its arguments, and returns a tuple of input and gradient data. By default, uniform destribution ranged [-1, 1] is used for both.

The decorated test class tests forward and backward computations on CPU and GPU across the following parameterize() ed parameters:

shape: rank of zero, and rank of more than zero
dtype: numpy.float16, numpy.float32 and numpy.float64

Additionally, it tests the label of the Chainer function.

Chainer functions tested by the test class decorated with the decorator should have the following properties:

Unary, taking one parameter and returning one value
dtype of input and output are the same
Elementwise operation for the supplied ndarray

Example

The following code defines a test class that tests sin() Chainer function, which takes a parameter with dtype of float and returns a value with the same dtype.

>>> import unittest
>>> from chainer import testing
>>> from chainer import functions as F
>>>
>>> @testing.unary_math_function_unittest(F.Sin())
... class TestSin(unittest.TestCase):
...     pass

Because the test methods are implicitly injected to TestSin class by the decorator, it is enough to place pass in the class definition.

Now the test is run with nose module.

>>> import nose
>>> nose.run(
...     defaultTest=__name__, argv=['', '-a', '!gpu'], exit=False)
True

To customize test data, make_data optional parameter can be used. The following is an example of testing sqrt Chainer function, which is tested in positive value domain here instead of the default input.

>>> import numpy
>>>
>>> def make_data(shape, dtype):
...     x = numpy.random.uniform(0.1, 1, shape).astype(dtype)
...     gy = numpy.random.uniform(-1, 1, shape).astype(dtype)
...     return x, gy
...
>>> @testing.unary_math_function_unittest(F.Sqrt(),
...                                       make_data=make_data)
... class TestSqrt(unittest.TestCase):
...     pass
...
>>> nose.run(
...     defaultTest=__name__, argv=['', '-a', '!gpu'], exit=False)
True

make_data function which returns input and gradient data generated in proper value domains with given shape and dtype parameters is defined, then passed to the decorator’s make_data parameter.