Generalized Linear Models, Categorically

A generalized linear model composes a link function with a linear predictor and an unbiased noise channel, and three arrows parametrise, predict, and estimate it.

A generalized linear model relates a linear predictor to the mean of a response through a link . The systematic part is , and the model constrains the mean by , so that

The link and the noise together, not either alone, define the model. The observation carries an unbiased noise morphism with , and this is exactly what makes hold on the mean scale.

Remark on the three arrows that parametrise, predict, estimate

A generalized linear model is organised by three maps. The parameter-to-model map selects a hypothesis from the family ; prediction evaluates a chosen at an input; estimation is the arrow sending a dataset to a fitted model. Only moves within a fixed link-and-noise choice, so the family is a single slot to be filled rather than a new model per problem.

Example of the usual GLMs

Linear regression is with Gaussian noise; logistic regression is with Bernoulli noise; Poisson regression is with Poisson noise. The systematic part is shared throughout. Only the link and the response law change, and each response law is a member of an exponential-family.

References

• J. A. Nelder & R. W. M. Wedderburn, Generalized Linear Models, J. Royal Statistical Society Series A, 135 (1972)