chainer.functions.gaussian_kl_divergence

chainer.functions.gaussian_kl_divergence(mean, ln_var, reduce='sum')[source]

Computes the KL-divergence of Gaussian variables from the standard one.

Given two variable mean representing \(\mu\) and ln_var representing \(\log(\sigma^2)\), this function calculates the KL-divergence in elementwise manner between the given multi-dimensional Gaussian \(N(\mu, S)\) and the standard Gaussian \(N(0, I)\)

\[D_{\mathbf{KL}}(N(\mu, S) \| N(0, I)),\]

where \(S\) is a diagonal matrix such that \(S_{ii} = \sigma_i^2\) and \(I\) is an identity matrix.

The output is a variable whose value depends on the value of the option reduce. If it is 'no', it holds the elementwise loss values. If it is 'sum' or 'mean', loss values are summed up or averaged respectively.

Parameters

  • mean (Variable or N-dimensional array) – A variable representing mean of given gaussian distribution, \(\mu\).

  • ln_var (Variable or N-dimensional array) – A variable representing logarithm of variance of given gaussian distribution, \(\log(\sigma^2)\).

  • reduce (str) – Reduction option. Its value must be either 'sum', 'mean' or 'no'. Otherwise, ValueError is raised.

Returns

A variable representing KL-divergence between given gaussian distribution and the standard gaussian. If reduce is 'no', the output variable holds array whose shape is same as one of (hence both of) input variables. If it is 'sum' or 'mean', the output variable holds a scalar value.

Return type

Variable