A PyTorch Geometric GCN model on Cora failed to exceed 0.68 accuracy despite tuning efforts

The Cora dataset’s seven-class node classification task yielded accuracy near 0.68—just above the ~0.55 random baseline—using a two-layer GCN with add_self_loops=True. The issue stemmed from the symmetric normalization in GCNConv, where the adjacency matrix $\hat{A} = D^{-1/2} A D^{-1/2}$ divides each edge weight by $\sqrt{d_i d_j}$. This caused high-degree nodes to suppress their neighbors’ features, leading to severe over-smoothing after only two layers.

The default implementation:

import torch
import torch.nn.functional as F
from torch_geometric.nn import GCNConv

class GCN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, training=self.training)
        x = self.conv2(x, edge_index)
        return x

The normalization formula $\hat{A} = D^{-1/2} A D^{-1/2}$ ensures scale-invariance but inadvertently homogenizes node representations in deeper layers, collapsing discriminative information. This behavior persisted even after exhaustive hyperparameter searches.