The pretraining of this very large language model should be finished any time soon! It is a model developed outside of the big tech companies through the collaboration of a large worldwide community of researchers. Stay tuned!
Beware of possibly significant ideological influences like diversity and inclusivity etc. See the ethical charter. Chinese harmony instead of Western tolerance. A requirement for collective ethical compliance (whatever that means).
"BLOOM (BigScience Language Open-science Open-access Multilingual) is unique not because it’s architecturally different than GPT-3 — it’s actually the most similar of all the above, being also a transformer-based model with 176B parameters (GPT-3 has 175B) —, but because it’s the starting point of a socio-political paradigm shift in AI ...
What they have in common is that they believe technology — and particularly AI — should be open, diverse, inclusive, responsible, and accessible for the benefit of humanity. ...
By having a system that is multilingual from its conception, with the immediate goal of covering the 20 most spoken languages in the world ..."
BLOOM is the culmination of these efforts. After more than a year of collective work that started in January 2021, and training for +3 months on the Jean Zay public French supercomputer, BLOOM is finally ready. It’s the result of the BigScience Research Workshop that comprises the work of +1000 researchers from all around the world and counts on the collaboration and support of +250 institutions, including Hugging Face, IDRIS, GENCI, and the Montreal AI Ethics Institute, among others.
By having a system that is multilingual from its conception, with the immediate goal of covering the 20 most spoken languages in the world ..."
BLOOM Is the Most Important AI Model of the Decade (Don't take this title too literally. This article has considerable ideological overtones)
Credits: Last Week in AI
No comments:
Post a Comment