We’re Training AI Twice as Fast This Year as Last

Good news!

"According to the best measures we’ve got, a set of benchmarks called MLPerf, machine-learning systems can be trained nearly twice as quickly as they could last year. It’s a figure that outstrips Moore’s Law ... Most of the gain is thanks to software and systems innovations, but this year also gave the first peek at what some new processors, notably from Graphcore and Intel subsidiary Habana Labs, can do.

The once-crippling time it took to train a neural network to do its task is the problem that launched startups like Cerebras and SambaNova and drove companies like Google to develop machine-learning accelerator chips in house. But the new MLPerf data shows that training time for standard neural networks has gotten a lot less taxing in a short period of time. ...
As usual, systems built using Nvidia A100 GPUs dominated the results. Nvidia’s new GPU architecture, Hopper, was designed with architectural features aimed at speeding training. ...
Google’s TPU v4 offers a three-fold improvement in computations per watt over its predecessor, the company says. Google noted that two of its tests were done using what it calls a “full TPU v4 pod”—a system consisting of 4,096 chips, for a total of up to 1.1 billion billion operations per second."

We’re Training AI Twice as Fast This Year as Last - IEEE Spectrum New MLPerf rankings show training times plunging