Two ways to hold a network to a law. Bake it into the architecture and it holds exactly; add it to the loss and it holds approximately. One restricts the space of maps, the other only weights it.
Two registers:
- Hard. Restrict the maps. The model is forced to be a constrained morphism: an intertwiner, a functorial map, or a fixed head. Categorically a limit.
- Soft. Keep every map and add a penalty whose zero locus is the constraint surface. Categorically a relaxation.
Cross this with what the constraint is about, the structure of the state space or the law its values must obey:
| geometry / topology | dynamics / conservation | |
|---|---|---|
| hard | GDL, TDL | MLIP |
| soft | topological loss | PINN |
Hard, inside the forward map
- GDL. Every layer commutes with a symmetry group, so it is a
-intertwiner. Equivariance is naturality, and weight sharing follows from it. - TDL. A cellular sheaf fixes the restriction maps, so the network may only emit data that glues. The constraint is a functoriality to obey rather than a symmetry to commute with.
- MLIP. Energy is an extensive sum of local terms and forces are its exact gradient
, so conservation holds by the form. Its symmetry sits upstream in the encoder, the GDL cell.
Soft, on the loss
- PINN. Add the residual of the governing equation
. The solution manifold is the surface the penalty pulls toward. - topological loss. A differentiable persistence diagram scores the output topology against a target set of Betti numbers.
Notes
- Loss, not training step. A soft constraint here means a term in the objective. Imposing it through the training procedure would instead pull in the dataset and a layer of meta learning.
- One leg of a triple. A model is architecture, data, and objective. Binding all three is constrained machine learning. This piece binds the architecture, together with its soft shadow in the objective.
- Where the line blurs. The Para(Optics) construction folds the loss into the model, so whether a constraint lives in the architecture or in the loss is only a choice of where the morphism boundary is drawn.
In one line. Hard is a limit, an exact universal cut of the space of maps. Soft is a relaxation, a real valued push toward the same surface. Real architectures stack both, hard where the symmetry is known and soft where it is only desired.