“Periodic table of machine learning” could fuel AI discovery

Amazing stuff! 

Caveat: I have not read this paper yet.

"... researchers have created a periodic table that shows how more than 20 classical machine-learning algorithms are connected. The new framework sheds light on how scientists could fuse strategies from different methods to improve existing AI models or come up with new ones. ...

The periodic table stems from one key idea: All these algorithms learn a specific kind of relationship between data points. While each algorithm may accomplish that in a slightly different way, the core mathematics behind each approach is the same. ...

Just like the periodic table of chemical elements, which initially contained blank squares that were later filled in by scientists, the periodic table of machine learning also has empty spaces. These spaces predict where algorithms should exist, but which haven’t been discovered yet. ...

The framework they created, information contrastive learning (I-Con), shows how a variety of algorithms can be viewed through the lens of this unifying equation. It includes everything from classification algorithms that can detect spam to the deep learning algorithms that power LLMs. ..."

From the abstract:

"As the field of representation learning grows, there has been a proliferation of different loss functions to solve different classes of problems.

We introduce a single information-theoretic equation that generalizes a large collection of modern loss functions in machine learning.

In particular, we introduce a framework that shows that several broad classes of machine learning methods are precisely minimizing an integrated KL divergence between two conditional distributions: the supervisory and learned representations.

This viewpoint exposes a hidden information geometry underlying clustering, spectral methods, dimensionality reduction, contrastive learning, and supervised learning.

This framework enables the development of new loss functions by combining successful techniques from across the literature.

We not only present a wide array of proofs, connecting over 23 different approaches, but we also leverage these theoretical results to create state-of-the-art unsupervised image classifiers that achieve a +8% improvement over the prior state-of-the-art on unsupervised classification on ImageNet-1K.

We also demonstrate that I-Con can be used to derive principled debiasing methods which improve contrastive representation learners."

“Periodic table of machine learning” could fuel AI discovery | MIT News | Massachusetts Institute of Technology "Researchers have created a unifying framework that can help scientists combine existing ideas to improve AI models or create new ones."

I-Con: A Unifying Framework for Representation Learning (preprint, open access, accepted for ICLR 2025)

MIT researchers created a periodic table of machine learning that shows how more than 20 classical algorithms are connected. The new framework sheds light on how scientists could fuse strategies from different methods to improve existing AI models or come up with new ones.