nanguardmode

Guide

The NanGuardMode aims to prevent the model from outputing NaNs or Infs. It has a number of self-checks, which can help to find out which apply node is generating those incorrect outputs. It provides automatic detection of 3 types of abnormal values: NaNs, Infs, and abnormally big values.

NanGuardMode can be used as follows:

import numpy
import theano
import theano.tensor as T
from theano.compile.nanguardmode import NanGuardMode

x = T.matrix()
w = theano.shared(numpy.random.randn(5, 7).astype(theano.config.floatX))
y = T.dot(x, w)
fun = theano.function(
    [x], y,
    mode=NanGuardMode(nan_is_error=True, inf_is_error=True, big_is_error=True)
)

While using the theano function fun, it will monitor the values of each input and output variable of each node. When abnormal values are detected, it raises an error to indicate which node yields the NaNs. For example, if we pass the following values to fun:

infa = numpy.tile(
    (numpy.asarray(100.) ** 1000000).astype(theano.config.floatX), (3, 5))
fun(infa)

It will raise an AssertionError indicating that Inf value is detected while executing the function.

You can also set the three parameters in NanGuardMode() to indicate which kind of abnormal values to monitor. nan_is_error and inf_is_error has no default values, so they need to be set explicitly, but big_is_error is set to be True by default.

Reference

class theano.compile.nanguardmode.NanGuardMode(nan_is_error=None, inf_is_error=None, big_is_error=None, optimizer=None, linker=None)

A Theano compilation Mode that makes the compiled function automatically detect NaNs and Infs and detect an error if they occur.

Parameters:
  • nan_is_error (bool) – If True, raise an error anytime a NaN is encountered.
  • inf_is_error (bool) – If True, raise an error anytime an Inf is encountered. Note that some pylearn2 modules currently use np.inf as a default value (e.g. mlp.max_pool) and these will cause an error if inf_is_error is True.
  • big_is_error (bool) – If True, raise an error when a value greater than 1e10 is encountered.

Note

We ignore the linker parameter