What is the role of normalization in GNNs?

Example Answer

Normalization in GNNs can mean two different things. Degree normalization changes how messages are weighted during propagation; activation normalization changes the scale of learned node features. I would name the operation rather than treat them as interchangeable.

A common GCN uses the adjacency matrix with self-loops and symmetric degree weighting, so a high-degree neighborhood does not simply produce a much larger summed signal. GraphNorm instead normalizes feature values within each individual graph and includes a learned adjustment to how much of the graph mean is removed. It is not another name for dividing messages by node degree. LayerNorm generally normalizes a node's feature dimensions.

The right choice depends on the task. For graph classification, per-graph normalization may help optimization, but the graph mean can also carry useful information. I would compare normalizers on the same split and monitor training behavior and accuracy. None guarantees that a deep GNN avoids oversmoothing or learns the right features.

Concrete Example

For a GCN, the structural operator is D̃⁻¹ᐟ²(A + I)D̃⁻¹ᐟ², where D̃ is the degree matrix after adding self-loops. GraphNorm instead takes node-feature tensors and graph membership as inputs.

GCN propagation · PyG GraphNorm