Testing Modules

CuPy offers testing utilities to support unit testing. They are under namespace cupy.testing.

Standard Assertions

The assertions have same names as NumPy’s ones. The difference from NumPy is that they can accept both numpy.ndarray and cupy.ndarray.

cupy.testing.assert_allclose(actual, desired, rtol=1e-07, atol=0, err_msg='', verbose=True)[source]

Raises an AssertionError if objects are not equal up to desired tolerance.

Parameters:
  • actual (numpy.ndarray or cupy.ndarray) – The actual object to check.
  • desired (numpy.ndarray or cupy.ndarray) – The desired, expected object.
  • rtol (float) – Relative tolerance.
  • atol (float) – Absolute tolerance.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.

See also

numpy.testing.assert_allclose()

cupy.testing.assert_array_almost_equal(x, y, decimal=6, err_msg='', verbose=True)[source]

Raises an AssertionError if objects are not equal up to desired precision.

Parameters:
  • x (numpy.ndarray or cupy.ndarray) – The actual object to check.
  • y (numpy.ndarray or cupy.ndarray) – The desired, expected object.
  • decimal (int) – Desired precision.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.

See also

numpy.testing.assert_array_almost_equal()

cupy.testing.assert_array_almost_equal_nulp(x, y, nulp=1)[source]

Compare two arrays relatively to their spacing.

Parameters:

See also

numpy.testing.assert_array_almost_equal_nulp()

cupy.testing.assert_array_max_ulp(a, b, maxulp=1, dtype=None)[source]

Check that all items of arrays differ in at most N Units in the Last Place.

Parameters:

See also

numpy.testing.assert_array_max_ulp()

cupy.testing.assert_array_equal(x, y, err_msg='', verbose=True)[source]

Raises an AssertionError if two array_like objects are not equal.

Parameters:
  • x (numpy.ndarray or cupy.ndarray) – The actual object to check.
  • y (numpy.ndarray or cupy.ndarray) – The desired, expected object.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.

See also

numpy.testing.assert_array_equal()

cupy.testing.assert_array_list_equal(xlist, ylist, err_msg='', verbose=True)[source]

Compares lists of arrays pairwise with assert_array_equal.

Parameters:
  • x (array_like) – Array of the actual objects.
  • y (array_like) – Array of the desired, expected objects.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.

Each element of x and y must be either numpy.ndarray or cupy.ndarray. x and y must have same length. Otherwise, this function raises AssertionError. It compares elements of x and y pairwise with assert_array_equal() and raises error if at least one pair is not equal.

See also

numpy.testing.assert_array_equal()

cupy.testing.assert_array_less(x, y, err_msg='', verbose=True)[source]

Raises an AssertionError if array_like objects are not ordered by less than.

Parameters:
  • x (numpy.ndarray or cupy.ndarray) – The smaller object to check.
  • y (numpy.ndarray or cupy.ndarray) – The larger object to compare.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.

See also

numpy.testing.assert_array_less()

NumPy-CuPy Consistency Check

The following decorators are for testing consistency between CuPy’s functions and corresponding NumPy’s ones.

cupy.testing.numpy_cupy_allclose(rtol=1e-07, atol=0, err_msg='', verbose=True, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks NumPy results and CuPy ones are close.

Parameters:
  • rtol (float) – Relative tolerance.
  • atol (float) – Absolute tolerance
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the arrays whose values are close between numpy case and cupy case. For example, this test case checks numpy.zeros and cupy.zeros should return same value.

>>> import unittest
>>> from cupy import testing
>>> @testing.gpu
... class TestFoo(unittest.TestCase):
...
...     @testing.numpy_cupy_allclose()
...     def test_foo(self, xp):
...         # ...
...         # Prepare data with xp
...         # ...
...
...         xp_result = xp.zeros(10)
...         return xp_result

See also

cupy.testing.assert_allclose()

cupy.testing.numpy_cupy_array_almost_equal(decimal=6, err_msg='', verbose=True, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks NumPy results and CuPy ones are almost equal.

Parameters:
  • decimal (int) – Desired precision.
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the same arrays in the sense of cupy.testing.assert_array_almost_equal() (except the type of array module) even if xp is numpy or cupy.

See also

cupy.testing.assert_array_almost_equal()

cupy.testing.numpy_cupy_array_almost_equal_nulp(nulp=1, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks results of NumPy and CuPy are equal w.r.t. spacing.

Parameters:
  • nulp (int) – The maximum number of unit in the last place for tolerance.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the same arrays in the sense of cupy.testing.assert_array_almost_equal_nulp() (except the type of array module) even if xp is numpy or cupy.

See also

cupy.testing.assert_array_almost_equal_nulp()

cupy.testing.numpy_cupy_array_max_ulp(maxulp=1, dtype=None, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks results of NumPy and CuPy ones are equal w.r.t. ulp.

Parameters:
  • maxulp (int) – The maximum number of units in the last place that elements of resulting two arrays can differ.
  • dtype (numpy.dtype) – Data-type to convert the resulting two array to if given.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the same arrays in the sense of assert_array_max_ulp() (except the type of array module) even if xp is numpy or cupy.

See also

cupy.testing.assert_array_max_ulp()

cupy.testing.numpy_cupy_array_equal(err_msg='', verbose=True, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks NumPy results and CuPy ones are equal.

Parameters:
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the same arrays in the sense of numpy_cupy_array_equal() (except the type of array module) even if xp is numpy or cupy.

See also

cupy.testing.assert_array_equal()

cupy.testing.numpy_cupy_array_list_equal(err_msg='', verbose=True, name='xp')[source]

Decorator that checks the resulting lists of NumPy and CuPy’s one are equal.

Parameters:
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.
  • name (str) – Argument name whose value is either numpy or cupy module.

Decorated test fixture is required to return the same list of arrays (except the type of array module) even if xp is numpy or cupy.

See also

cupy.testing.assert_array_list_equal()

cupy.testing.numpy_cupy_array_less(err_msg='', verbose=True, name='xp', type_check=True, accept_error=False)[source]

Decorator that checks the CuPy result is less than NumPy result.

Parameters:
  • err_msg (str) – The error message to be printed in case of failure.
  • verbose (bool) – If True, the conflicting values are appended to the error message.
  • name (str) – Argument name whose value is either numpy or cupy module.
  • type_check (bool) – If True, consistency of dtype is also checked.
  • accept_error (bool, Exception or tuple of Exception) – Sepcify acceptable errors. When both NumPy test and CuPy test raises the same type of errors, and the type of the errors is specified with this argument, the errors are ignored and not raised. If it is True all error types are acceptable. If it is False no error is acceptable.

Decorated test fixture is required to return the smaller array when xp is cupy than the one when xp is numpy.

See also

cupy.testing.assert_array_less()

cupy.testing.numpy_cupy_raises(name='xp')[source]

Decorator that checks the NumPy and CuPy throw same errors.

Parameters:
  • name (str) – Argument name whose value is either
  • or cupy module. (numpy) –

Decorated test fixture is required throw same errors even if xp is numpy or cupy.

See also

cupy.testing.assert_array_less()

Parameterized dtype Test

The following decorators offer the standard way for parameterized test with respect to single or the combination of dtype(s).

cupy.testing.for_dtypes(dtypes, name='dtype')[source]

Decorator for parameterized dtype test.

Parameters:
  • dtypes (list of dtypes) – dtypes to be tested.
  • name (str) – Argument name to which specified dtypes are passed.

This decorator adds a keyword argument specified by name to the test fixture. Then, it runs the fixtures in parallel by passing the each element of dtypes to the named argument.

cupy.testing.for_all_dtypes(name='dtype', no_float16=False, no_bool=False)[source]

Decorator that checks the fixture with all dtypes.

Parameters:
  • name (str) – Argument name to which specified dtypes are passed.
  • no_float16 (bool) – If, True, numpy.float16 is omitted from candidate dtypes.
  • no_bool (bool) – If, True, numpy.bool_ is omitted from candidate dtypes.

dtypes to be tested: numpy.float16 (optional), numpy.float32, numpy.float64, numpy.dtype('b'), numpy.dtype('h'), numpy.dtype('i'), numpy.dtype('l'), numpy.dtype('q'), numpy.dtype('B'), numpy.dtype('H'), numpy.dtype('I'), numpy.dtype('L'), numpy.dtype('Q'), and numpy.bool_ (optional).

The usage is as follows. This test fixture checks if cPickle successfully reconstructs cupy.ndarray for various dtypes. dtype is an argument inserted by the decorator.

>>> import unittest
>>> from cupy import testing
>>> @testing.gpu
... class TestNpz(unittest.TestCase):
...
...     @testing.for_all_dtypes()
...     def test_pickle(self, dtype):
...         a = testing.shaped_arange((2, 3, 4), dtype=dtype)
...         s = six.moves.cPickle.dumps(a)
...         b = six.moves.cPickle.loads(s)
...         testing.assert_array_equal(a, b)

Typically, we use this decorator in combination with decorators that check consistency between NumPy and CuPy like cupy.testing.numpy_cupy_allclose(). The following is such an example.

>>> import unittest
>>> from cupy import testing
>>> @testing.gpu
... class TestMean(unittest.TestCase):
...
...     @testing.for_all_dtypes()
...     @testing.numpy_cupy_allclose()
...     def test_mean_all(self, xp, dtype):
...         a = testing.shaped_arange((2, 3), xp, dtype)
...         return a.mean()

See also

cupy.testing.for_dtypes()

cupy.testing.for_float_dtypes(name='dtype', no_float16=False)[source]

Decorator that checks the fixture with all float dtypes.

Parameters:
  • name (str) – Argument name to which specified dtypes are passed.
  • no_float16 (bool) – If, True, numpy.float16 is omitted from candidate dtypes.

dtypes to be tested are numpy.float16 (optional), numpy.float32, and numpy.float64.

See also

cupy.testing.for_dtypes(), cupy.testing.for_all_dtypes()

cupy.testing.for_signed_dtypes(name='dtype')[source]

Decorator that checks the fixture with signed dtypes.

Parameters:name (str) – Argument name to which specified dtypes are passed.

dtypes to be tested are numpy.dtype('b'), numpy.dtype('h'), numpy.dtype('i'), numpy.dtype('l'), and numpy.dtype('q').

See also

cupy.testing.for_dtypes(), cupy.testing.for_all_dtypes()

cupy.testing.for_unsigned_dtypes(name='dtype')[source]

Decorator that checks the fixture with all dtypes.

Parameters:name (str) – Argument name to which specified dtypes are passed.

dtypes to be tested are numpy.dtype('B'), numpy.dtype('H'),

numpy.dtype('I'), numpy.dtype('L'), and numpy.dtype('Q').

See also

cupy.testing.for_dtypes(), cupy.testing.for_all_dtypes()

cupy.testing.for_int_dtypes(name='dtype', no_bool=False)[source]

Decorator that checks the fixture with integer and optionally bool dtypes.

Parameters:
  • name (str) – Argument name to which specified dtypes are passed.
  • no_bool (bool) – If True, numpy.bool_ is omitted from candidate dtypes.

dtypes to be tested are numpy.dtype('b'), numpy.dtype('h'), numpy.dtype('i'), numpy.dtype('l'), numpy.dtype('q'), numpy.dtype('B'), numpy.dtype('H'), numpy.dtype('I'), numpy.dtype('L'), numpy.dtype('Q'), and numpy.bool_ (optional).

See also

cupy.testing.for_dtypes(), cupy.testing.for_all_dtypes()

cupy.testing.for_dtypes_combination(types, names=('dtype', ), full=None)[source]

Decorator that checks the fixture with a product set of dtypes.

Parameters:
  • types (list of dtypes) – dtypes to be tested.
  • names (list of str) – Argument names to which dtypes are passed.
  • full (bool) – If True, then all combinations of dtypes will be tested. Otherwise, the subset of combinations will be tested (see the description below).

Decorator adds the keyword arguments specified by names to the test fixture. Then, it runs the fixtures in parallel with passing (possibly a subset of) the product set of dtypes. The range of dtypes is specified by types.

The combination of dtypes to be tested changes depending on the option full. If full is True, all combinations of types are tested. Sometimes, such an exhaustive test can be costly. So, if full is False, only the subset of possible combinations is tested. Specifically, at first, the shuffled lists of types are made for each argument name in names. Let the lists be D1, D2, ..., Dn where \(n\) is the number of arguments. Then, the combinations to be tested will be zip(D1, ..., Dn). If full is None, the behavior is switched by setting the environment variable CUPY_TEST_FULL_COMBINATION=1.

For example, let types be [float16, float32, float64] and names be ['a_type', 'b_type']. If full is True, then the decorated test fixture is executed with all \(2^3\) patterns. On the other hand, if full is False, shuffled lists are made for a_type and b_type. Suppose the lists are (16, 64, 32) for a_type and (32, 64, 16) for b_type (prefixes are removed for short). Then the combinations of (a_type, b_type) to be tested are (16, 32), (64, 64) and (32, 16).

cupy.testing.for_all_dtypes_combination(names=('dtyes', ), no_float16=False, no_bool=False, full=None)[source]

Decorator that checks the fixture with a product set of all dtypes.

Parameters:
  • names (list of str) – Argument names to which dtypes are passed.
  • no_float16 (bool) – If True, numpy.float16 is omitted from candidate dtypes.
  • no_bool (bool) – If True, numpy.bool_ is omitted from candidate dtypes.
  • full (bool) – If True, then all combinations of dtypes will be tested. Otherwise, the subset of combinations will be tested (see description in cupy.testing.for_dtypes_combination()).

See also

cupy.testing.for_dtypes_combination()

cupy.testing.for_signed_dtypes_combination(names=('dtype', ), full=None)[source]

Decorator for parameterized test w.r.t. the product set of signed dtypes.

Parameters:
  • names (list of str) – Argument names to which dtypes are passed.
  • full (bool) – If True, then all combinations of dtypes will be tested. Otherwise, the subset of combinations will be tested (see description in cupy.testing.for_dtypes_combination()).

See also

cupy.testing.for_dtypes_combination()

cupy.testing.for_unsigned_dtypes_combination(names=('dtype', ), full=None)[source]

Decorator for parameterized test w.r.t. the product set of unsigned dtypes.

Parameters:
  • names (list of str) – Argument names to which dtypes are passed.
  • full (bool) – If True, then all combinations of dtypes will be tested. Otherwise, the subset of combinations will be tested (see description in cupy.testing.for_dtypes_combination()).

See also

cupy.testing.for_dtypes_combination()

cupy.testing.for_int_dtypes_combination(names=('dtype', ), no_bool=False, full=None)[source]

Decorator for parameterized test w.r.t. the product set of int and boolean.

Parameters:
  • names (list of str) – Argument names to which dtypes are passed.
  • no_bool (bool) – If True, numpy.bool_ is omitted from candidate dtypes.
  • full (bool) – If True, then all combinations of dtypes will be tested. Otherwise, the subset of combinations will be tested (see description in cupy.testing.for_dtypes_combination()).

See also

cupy.testing.for_dtypes_combination()

Parameterized order Test

The following decorators offer the standard way to parameterize tests with orders.

cupy.testing.for_orders(orders, name='order')[source]

Decorator to parameterize tests with order.

Parameters:
  • orders (list of orders) – orders to be tested.
  • name (str) – Argument name to which the specified order is passed.

This decorator adds a keyword argument specified by name to the test fixtures. Then, the fixtures run by passing each element of orders to the named argument.

cupy.testing.for_CF_orders(name='order')[source]

Decorator that checks the fixture with orders ‘C’ and ‘F’.

Parameters:name (str) – Argument name to which the specified order is passed.

See also

cupy.testing.for_all_dtypes()