El Capitan, the First Exascale Computer is Here at the National Nuclear Security Administration’s (NNSA)

Good news! Not NSA, but NNSA! 😊

Supercomputer 'Aurora': Argonne National Laboratory is building the world's fastest computer

Good news!

Cerebras Systems sets record for largest AI models ever trained on one device

We are coming closer to a brain on chip! What a gigantic chip! Probably it does not yet fit in a typical smartphone! 😄

This could be a game changer!

"... for the first time ever, the ability to train models with up to 20 billion parameters on a single CS-2 system – a feat not possible on any other single device. By enabling a single CS-2 to train these models, Cerebras reduces the system engineering time necessary to run [train] large natural language processing (NLP) models from months to minutes. It also eliminates one of the most painful aspects of NLP — namely the partitioning of the model across hundreds or thousands of small graphics processing units (GPU)."

Cerebras Systems sets record for largest AI models ever trained on one device | VentureBeat

Cerebras Systems Sets Record for Largest AI Models Ever Trained on A Single Device Single CS-2 System trains multi-billion parameter NLP models including GPT-3L 1.3 Billion, GPT-J 6 Billion , GPT-3 13 Billion and GPT-NeoX 20 Billion; provides simple setup, faster training and enables switching between models with a few keystrokes

China supercomputer achieves global first with ‘brain-scale’ AI model

The arms race continues! However, whether this is more hype than reality can not easily be confirmed since communist regimes tend to be opaque!

However, that humanity is on the cusp of developing supercomputers that rivals the brain in computing power is undeniable!

"Research team says latest Sunway machine is on a par with the US Frontier, named as the world’s most powerful just weeks earlier

Its speed has made it possible to train an artificial intelligence program with 174 trillion parameters, rivalling the synapses in the human brain"

China supercomputer achieves global first with ‘brain-scale’ AI model | South China Morning Post

Nvidia's new AI-powered supercomputers to advance nuclear fusion and brain imaging research

Good news! Will humanity finally see a major breakthrough in sustainable nuclear fusion.

Nvidia's AI-powered supercomputers advance nuclear fusion research | VentureBeat

Introducing the Facebook AI Research SuperCluster

Very impressive!

"High-performance computing infrastructure is a critical component in training such large models, and Meta’s AI research team has been building these high-powered systems for many years. The first generation of this infrastructure, designed in 2017, has 22,000 NVIDIA V100 Tensor Core GPUs in a single cluster that performs 35,000 training jobs a day. Up until now, this infrastructure has set the bar for Meta’s researchers in terms of its performance, reliability, and productivity.

In early 2020, we decided the best way to accelerate progress was to design a new computing infrastructure from a clean slate to take advantage of new GPU and network fabric technology. We wanted this infrastructure to be able to train models with more than a trillion parameters on data sets as large as an exabyte — which, to provide a sense of scale, is the equivalent of 36,000 years of high-quality video.

While the high-performance computing community has been tackling scale for decades, we also had to make sure we have all the needed security and privacy controls in place to protect any training data we use. Unlike with our previous AI research infrastructure, which leveraged only open source and other publicly available data sets, RSC also helps us ensure that our research translates effectively into practice by allowing us to include real-world examples from Meta’s production systems in model training. By doing this, we can help advance research to perform downstream tasks such as identifying harmful content on our platforms as well as research into embodied AI and multimodal AI  to help improve user experiences on our family of apps. We believe this is the first time performance, reliability, security, and privacy have been tackled at such a scale. ...
RSC today comprises a total of 760 NVIDIA DGX A100 systems as its compute nodes, for a total of 6,080 GPUs — with each A100 GPU being more powerful than the V100 used in our previous system. Each DGX communicates via an NVIDIA Quantum 1600 Gb/s InfiniBand two-level Clos fabric that has no oversubscription. RSC’s storage tier has 175 petabytes of Pure Storage FlashArray, 46 petabytes of cache storage in Penguin Computing Altus systems, and 10 petabytes of Pure Storage FlashBlade. ..."

Introducing the AI Research SuperCluster — Meta’s cutting-edge AI supercomputer for AI research

Rensselaer focuses IBM's AiMOS supercomputer on machine learning

Recommendable! Covers more than just the latest IBM supercomputer!

"The race between China and the U.S. is fierce. In Top500 rankings, China two years ago surpassed the United States in total number of ranked supercomputers for the first time, with 202 to 143. That trend accelerated the following year; according to the Top500 fall 2018 report, the number of ranked U.S. supercomputers fell to 108 as China’s total climbed to 229."

Rensselaer focuses IBM's AiMOS supercomputer on machine learning | VentureBeat: IBM's AiMOS, which was recently delivered to the Rensselaer Polytechnic Institute, is the 24th-fastest supercomputer in the world.

Germany Ranked Only 6th On Top500

Posted: 6/18/2019

If you want to present just one number to show how backward Germany has become, take the Top 500 list of installed supercomputers in the world!

Germany is only ranked No 6th (14 supercomputers) on the Top500. It is in good company with the Netherlands (No. 8, 13 supercomputers) and Ireland (No. 7, 13).

Compare that to China (No. 1, 219), USA (No. 2, 116), and Japan (No. 3,  29). France (No 4; 19) and the UK (No. 5, 18) are also ahead of Germany.

Sources (S):

1. HIGHLIGHTS of the 53rd edition of the TOP500.