Read a symmetry off the data domain and build it into the architecture, and the network respects that symmetry by construction rather than by penalty.
Geometric Deep Learning (GDL) imposes symmetry as a hard architectural constraint. From the domain on which the signal lives one reads a group of admissible transformations, such as translations of an image grid, rotations or rigid motions of a point cloud, or permutations of a graph’s nodes, and one demands that every layer commute with the action of . A layer is equivariant when for all , so symmetry of the input is carried to symmetry of the output. It is invariant when , the special case where the output action is trivial and typically the final read-out. Equivariance in the intermediate layers, collapsing to invariance only at the head, preserves geometric information through the depth of the network.
A linear equivariant layer is exactly a -intertwiner, and this is a genuine natural transformation rather than a trivial one. Present each representation as a functor out of the one-object group category. The input space is a functor and the output space a different functor , each sending the single object to a vector space and each to the linear map it acts by. A layer is then a natural transformation whose one component is the linear map, and its naturality square , demanded for every , is exactly the equivariance condition. Because and are distinct representations the transformation is not the identity, and its content is the whole family of squares. Weight sharing is that naturality. One learnable rule is replicated across the orbit, so the same kernel acts at every point. When the domain itself has structure the base category grows past and the components spread over its objects, as in GNN as functor, where permutation equivariance is naturality over a graph’s incidence category.
Remark on hard constraint versus soft penalty
GDL restricts the space of maps. The network is not merely encouraged towards symmetry by a loss term but is architecturally incapable of breaking it, since it is a morphism of a constrained kind, a -intertwiner. This is the defining move of one branch of a taxonomy of inductive biases, to be contrasted with soft, loss-penalty enforcement and with sibling constraints on other domains: simplicial and cellular structure in topological deep learning, the physical symmetries of energy in ML interatomic potentials, and differential-equation residuals in physics-informed neural networks.
The canonical instances are read off the group. Translation equivariance forces the linear layer to be a convolution, and the classical CNN is recovered as the unique translation-intertwiner. For and the intertwiners are described by representation theory, through irreducible representations and Clebsch–Gordan coupling, giving steerable CNNs, spherical CNNs, and tensor-field networks, whose features are typed by the irreps they transform under. The blueprint unifying grids, groups, graphs, and manifolds under this single principle, symmetry as the organising axiom in the spirit of Klein’s Erlangen programme, is due to Bronstein, Bruna, Cohen and Veličković, whereas the group-equivariant and steerable constructions are due to Cohen and Welling. This is one cell of the taxonomy laid out in constrained-neural-network-architectures.