Conditional VAE, Categorically

A conditional variational autoencoder is a stack of sampling steps threaded through an encoder and a decoder. Drawn as a string-diagram in a markov-category, the stack becomes one wiring diagram in which “sample from a distribution” is exactly “compose with a channel”, and the latent branching is a copy/compose pattern the diagram makes explicit.

Fix a Markov category (concretely ). A conditional VAE models through a latent : a prior channel , a decoder channel , and an amortised encoder channel used at training time. Each sampling step is post-composition with a Markov kernel; there is no separate “draw” operation to track, because a state fed into a channel already is a sample in the diagrammatic reading. The encoder/decoder split is a copy/compose pattern: the conditioning input is -branched to both the latent channel and the decoder, and is produced once and reused.

Definition. The generative diagram

Wire in, copy it, feed one branch to the prior to obtain , then compose (with the retained -branch) into the decoder to emit . Marginalising is a on its wire; the whole generative model is the single channel obtained by contracting this diagram. Read pointwise, that marginal is a mixture of experts, the prior gating decoder channels .

Remark. The reparametrisation trick as a factorisation

Writing with factors the encoder channel as a deterministic box precomposed with a fixed noise state. In the diagram this pushes all randomness onto one leftmost noise wire, leaving the parameter-dependent part deterministic. Gradients therefore flow through the sampling node along this split.

Remark. The objective reads off the same wiring

The Evidence Lower Bound (ELBO)‘s two terms mirror the diagram: the reconstruction term contracts the encoder-then-decoder path against the data, and the Kullback–Leibler (KL) term compares the prior branch with the encoder branch on the shared -wire. Conditioning, sampling, and the loss all live on one arrangement of copies and channels.

References

• D. P. Kingma, M. Welling, Auto-Encoding Variational Bayes (2013)
• K. Sohn, H. Lee, X. Yan, Learning Structured Output Representation using Deep Conditional Generative Models (2015)