Statistical Decision Theory

Decision theory is usually taught loss-first, fixing a loss and then minimising its risk. It reads more cleanly the other way. The primitive is the decision rule, a map from data to actions, from which risk, admissibility, and Bayes optimality are all derived through how rules compose. Cast in a channel idiom, a rule splits into a prediction layer and a learning layer.

A decision rule is a channel from observations to actions. It is deterministic when it is an ordinary function and stochastic when it is a genuine Markov kernel; samples live in , the input–label product. The prediction layer is the forward pass that emits an action, whereas the learning layer is the typically stochastic map from a sample to the rule itself. Taking the rule as primitive, a loss and its risk are derived data. They fix a scalar readout and are not part of the rule.

Definition of admissibility

A rule is admissible if no rule dominates it. Domination means some satisfies for all with strict inequality somewhere. Inadmissibility is the only unconditional verdict decision theory issues; among admissible rules the choice needs a further principle such as a prior or a minimax criterion.

Complete-class theorem

Under regularity, meaning a convex closed action space and a suitable loss, the admissible rules and the Bayes rules essentially coincide. Every admissible rule is Bayes against some prior, a limit of priors in the boundary cases, and conversely unique Bayes rules are admissible. Optimality is therefore always optimality relative to a prior, and there is no prior-free winner.

Remark on the ludic fallacy

The framework prices only risks inside the assumed model . Treating that within-model risk as the whole of uncertainty, as if the sample space were a fully specified game of chance, is the ludic fallacy. Model misspecification and unmodelled events are invisible to the risk integral, however carefully the Bayes rule is computed.

Example, the reject option

Augment the action space of a classifier with a distinguished action that rejects or abstains, carrying a fixed cost . The Bayes rule then predicts only where the posterior class probability clears a threshold set by and abstains otherwise. This derived rule trades coverage for accuracy, and it is a concrete case where enlarging the action space changes which rules are admissible.

References

• A. Wald, “Statistical Decision Functions” (1950)
• N. N. Taleb, “The Black Swan” (2007), on the ludic fallacy
• C. K. Chow, “On Optimum Recognition Error and Reject Tradeoff” (IEEE Trans. Inf. Theory, 1970)
• T. Fritz, “A Synthetic Approach to Markov Kernels, Conditional Independence and Theorems on Sufficient Statistics” (Advances in Mathematics, 2020)