Good news! Competition is good, more competition is better!
"Amazon Web Services, which has been building its own AI training chips for years now, just introduced a new version known as Trainium3 that comes with some impressive specs. ...
Trainium3 UltraServer, a system powered by the company’s state-of-the art, 3 nanometer Trainium3 chip, as well as its homegrown networking tech. As you might expect, the third-generation chip and system offer big bumps in performance for AI training and inference over the second-generation, according to AWS.
AWS says the systems are more than four times faster, with four times more memory, not just for training, but for delivering AI apps at peak demand. Additionally, thousands of UltraServers can be linked together to provide an app with up to 1 million Trainium3 chips — 10 times the previous generation. Each UltraServer can host 144 chips, according to the company. ..."
"Key takeaways
- Trainium3 UltraServers deliver high performance for AI workloads with up to 4.4x more compute performance, 4x greater energy efficiency, and almost 4x more memory bandwidth than Trainium2 UltraServers—enabling faster AI development with lower operational costs.
- Trn3 UltraServers scale up to 144 Trainium3 chips, delivering up to 362 FP8 PFLOPs with 4x lower latency to train larger models faster and serve inference at scale.
- Customers including Anthropic, Karakuri, Metagenomi, NetoAI, Ricoh, and Splash Music are reducing training and inference costs by up to 50% with Trainium, while Decart is achieving 4x faster inference for real-time generative video at half the cost of GPUs, and Amazon Bedrock is already serving production workloads on Trainium3.
..."
Trainium3 UltraServers now available: Enabling customers to train and deploy AI models faster at lower cost (original news release)
No comments:
Post a Comment