Are GNNs capable of learning importance of initial feature regarding structure information target?

[walidgeuttala]

Hi guys, I have a question—any related papers would be helpful! Can a GNN learn a target for a node-level regression task if the initial features are pure noise or partially noisy (e.g., some features are noise, others are important, or features contain both noise and relevant information)? Specifically, the target in this case is structural, meaning it relies primarily on structural information, no need for any initial features, only for case example the idea of ID-GNN which shows the GNN could not learn the cycle length, extra, and some initial features are needed called the id feature.

I understand it may seem straightforward to just add ones as a feature and ignore all noise, which works for simple cases. However, in real-world problems, we often don’t know the importance of each feature to the target. I tested a basic scenario—predicting node degree—where I observed that models from DGL and PyG don’t perform well because the message-passing weights in most cases use a linear layer without bias. This makes the layers structure-dependent, I forget where I found this, but a paper shows that in scenario a simple MLP learns better than a GNN, I think it was that the target is dependet only in the initial features, no need for the structure information, how the GNN could pick both of the information and choose them, I thought maybe the bias is needed for such case.

When I added a bias term, the model performed much better. Do you know why DGL and PyG avoid using bias terms by default? And does this mean that with the bias, the model is able to focus on structural information, like counting ones, while disregarding noisy node features?