chainer.kl_divergence¶
-
chainer.
kl_divergence
(dist1, dist2)[source]¶ Computes Kullback-Leibler divergence.
For two continuous distributions \(p(x), q(x)\), it is expressed as
\[D_{KL}(p||q) = \int p(x) \log \frac{p(x)}{q(x)} dx\]For two discrete distributions \(p(x), q(x)\), it is expressed as
\[D_{KL}(p||q) = \sum_x p(x) \log \frac{p(x)}{q(x)}\]Parameters: - dist1 (
Distribution
) – Distribution to calculate KL divergence \(p\). This is the first (left) operand of the KL divergence. - dist2 (
Distribution
) – Distribution to calculate KL divergence \(q\). This is the second (right) operand of the KL divergence.
Returns: Output variable representing kl divergence \(D_{KL}(p||q)\).
Return type: Using
register_kl()
, we can define behavior ofkl_divergence()
for any two distributions.- dist1 (