The InfoGAN paper has the following lemma:

Lemma 5.1. For random variables $X, Y$ and function $f(x, y)$ under suitable regularity conditions: $\mathbb{E}_{x \sim X, y \sim Y|x}[f(x, y)] = \mathbb{E}_{x \sim X, y \sim Y|x, x' \sim X|y}[f(x', y)]$.

The statement is correct, but the proof in the paper is confused – here’s a step where $x$ mysteriously becomes $x'$:

After consulting with others, we weren’t able to understand that step of the proof. Instead, Nic Ford found this alternative proof. Hopefully this could help others reading the paper.

Proof.

(This note is also available as a PDF.)