On Leveraging Classical Algorithms for Graph Neural Networks

This could be an interesting paper by Petar Veličković!

The key idea to use classical algorithms as inductive biases for pretraining seems convincing!

From the abstract:

"Neural networks excel at processing unstructured data but often fail to generalise out-of-distribution, whereas classical algorithms guarantee correctness but lack flexibility.

We explore whether pretraining Graph Neural Networks (GNNs) on classical algorithms can improve their performance on molecular property prediction tasks from the Open Graph Benchmark: ogbg-molhiv (HIV inhibition) and ogbg-molclintox (clinical toxicity). 

GNNs trained on 24 classical algorithms from the CLRS Algorithmic Reasoning Benchmark are used to initialise and freeze selected layers of a second GNN for molecular prediction. Compared to a randomly initialised baseline, the pretrained models achieve consistent wins or ties, with the Segments Intersect algorithm pretraining yielding a 6% absolute gain on ogbg-molhiv and Dijkstra pretraining achieving a 3% gain on ogbg-molclintox.

These results demonstrate embedding classical algorithmic priors into GNNs provides useful inductive biases, boosting performance on complex, real-world graph data."

[2510.21574] Leveraging Classical Algorithms for Graph Neural Networks

Research Highlights from The First Learning on Graphs (LoG) Conference

Recommendable!

Ignore that silly article about "Uber’s Facial Recognition Technology Locks Out Indian Drivers"

Update #39: Uber's Faulty Facial Recognition in India and The First Learning on Graphs Conference

Geometry-enhanced molecular representation learning for property prediction

This paper seems to be very interesting and promising! Previous graph neural networks focused more on the atoms and chemical bonds making up molecules, but not their geometric orientation and properties.

"... proposed geometry-enhanced molecular representation learning (GEM), an architecture and training method that classifies molecules and estimates their properties.

Key insight: Chemists have used graph neural networks (GNNs) to analyze molecules based on their atomic ingredients and the types of bonds between the atoms. However, these models weren’t trained on structural information, which plays a key role in determining a molecule’s behavior. They can be improved by training on structural features such as the distances between atoms and angles formed by their bonds. 

GNN basics: A GNN processes datasets in the form of graphs, which consist of nodes connected by edges. For example, a graph might depict customers and products as nodes and purchases as edges. This work used a vanilla neural network to update the representation of each node based on the representations of neighboring nodes and edges.

How it works: The authors trained a modified GNN on 18 million molecules whose properties were unlabeled to estimate structural attributes of molecules. ..."

From the abstract:

"Effective molecular representation learning is of great importance to facilitate molecular property prediction. Recent advances for molecular representation learning have shown great promise in applying graph neural networks to model molecules. Moreover, a few recent studies design self-supervised learning methods for molecular representation to address insufficient labelled molecules; however, these self-supervised frameworks treat the molecules as topological graphs without fully utilizing the molecular geometry information. The molecular geometry, also known as the three-dimensional spatial structure of a molecule, is critical for determining molecular properties. To this end, we propose a novel geometry-enhanced molecular representation learning method (GEM). The proposed GEM has a specially designed geometry-based graph neural network architecture as well as several dedicated geometry-level self-supervised learning strategies to learn the molecular geometry knowledge. We compare GEM with various state-of-the-art baselines on different benchmarks and show that it can considerably outperform them all, demonstrating the superiority of the proposed method."

Geometry-enhanced molecular representation learning for property prediction | Nature Machine Intelligence (open access)

Credits: The Batch by Andrew Ng

Fig. 1: Comparison between two stereoisomers with the same topology but different geometries

Fig. 2: Overall architecture of GEM

Diffusion Improves Graph Learning

Very recommendable! Well written. Convincing!

[1911.05485] Diffusion Improves Graph Learning

Beyond Message Passing: a Physics-Inspired Paradigm for Graph Neural Networks

Very recommendable! A critical overview written by one of the leading researchers in the field of Graph Neural Networks, i.e. Michael Bronstein (Head Graph ML Research at Twitter).

Beyond Message Passing: a Physics-Inspired Paradigm for Graph Neural Networks

Hierarchical Graph Representation Learning with Differentiable Pooling

Recommendable! Well written paper! 

Two things that surprised me:

1. The researchers used the COLLAB dataset to exemplify some of the results although it has only a single level cluster structure ("This is because many collaboration graphs in COLLAB show only single-layer community structures, which can be captured well with pre-computed graph clustering algorithm"). Given that the researchers used so several other chemical and social network datasets.
2. The researchers used only maximal two DiffPool layers for all experiments

Hierarchical Graph Representation Learning with Differentiable Pooling (NIPS not NeurIPS 2018)

Paper: Strategies for Pre-training Graph Neural Networks

Recommendable! Just finished reading it for the first time. Well done!

From the abstract:

"... The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction."

Strategies for Pre-training Graph Neural Networks | OpenReview

Google's DeepMind AI models transition of glass from a liquid to a solid

Amazing stuff! This kind of AI research opens up whole new worlds of discovery and understanding! This could be a breakthrough! What is next, protein folding, superconducting transitions, cell migration during embryonic development ...

For interested readers: You can gain access to the full Nature Physics article (behind a paywall) by using the link provided at the bottom of the corresponding Deepmind blog post.

"Despite decades of theoretical studies, the nature of the glass transition remains elusive and debated, while the existence of structural predictors of its dynamics is a major open question. Here we determine the long-time evolution of a glassy system solely from the initial particle positions and without any handcrafted features, using graph neural networks as a powerful model. We show that this method outperforms current state-of-the-art methods, generalizing over a wide range of temperatures, pressures and densities."

"These systems all operate under local constraints where the position of some elements inhibits the motion of others (termed frustration). Their dynamics are complex and cooperative, taking the form of large-scale, collective rearrangements which propagate through space in a heterogeneous manner."

DeepMind's AI models transition of glass from a liquid to a solid | VentureBeat: The DeepMind team trained a graph neural network that outperformed physics-inspired baselines and state-of-the-art AI models in predicting glassy dynamics.

AI system that predicts movement of glass molecules transitioning between liquid and solid states

DeepMind's blog post:
Towards understanding glasses with graph neural networks | DeepMind: Under a microscope, a pane of window glass doesn’t look like a collection of orderly molecules, as a crystal would, but rather a jumble with no discernable structure. Glass is made by starting with a glowing mixture of high-temperature melted sand and minerals. Once cooled, its viscosity (a measure of the friction in the fluid) increases a trillion-fold, and it becomes a solid, resisting tension from stretching or pulling. Yet the molecules in the glass remain in a seemingly disordered state, much like the original molten liquid – almost as though the disordered liquid state had been flash-frozen in place. The glass transition, then, first appears to be a dramatic arrest in the movement of the glass molecules. Whether this process corresponds to a structural phase transition (as in water freezing, or the superconducting transition) is a major open question in the field. Understanding the nature of the dynamics of glass is fundamental to understanding how the atomic-scale properties define the visible features of many solid materials.


The underlying research paper:

Unveiling the predictive power of static structure in glassy systems | Nature Physics: Despite decades of theoretical studies, the nature of the glass transition remains elusive and debated, while the existence of structural predictors of its dynamics is a major open question. Recent approaches propose inferring predictors from a variety of human-defined features using machine learning. Here we determine the long-time evolution of a glassy system solely from the initial particle positions and without any handcrafted features, using graph neural networks as a powerful model. We show that this method outperforms current state-of-the-art methods, generalizing over a wide range of temperatures, pressures and densities. In shear experiments, it predicts the locations of rearranging particles. The structural predictors learned by our network exhibit a correlation length that increases with larger timescales to reach the size of our system. Beyond glasses, our method could apply to many other physical systems that map to a graph of local interaction. The physics that underlies the glass transition is both subtle and non-trivial. A machine learning approach based on graph networks is now shown to accurately predict the dynamics of glasses over a wide range of temperatures, p