{"id":12046,"date":"2022-07-25T07:00:00","date_gmt":"2022-07-25T15:00:00","guid":{"rendered":"https:\/\/formtek.com\/blog\/?p=12046"},"modified":"2022-07-20T13:24:53","modified_gmt":"2022-07-20T21:24:53","slug":"bloom-the-open-source-alternative-to-gpt-3-like-large-language-models","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/bloom-the-open-source-alternative-to-gpt-3-like-large-language-models\/","title":{"rendered":"BLOOM: The Open-Source Alternative to GPT-3-like Large Language Models"},"content":{"rendered":"\n

OpenAI’s GPT-3<\/a> may have been first, or at least first to generate a widespread ‘wow’ from people once they saw what AI Large learning Models can do. Since then there have been competitors, like Google’s LaMDA<\/a>, China’s Wu Dao 2.0<\/a>, MUM<\/a>, MetaAI’s OPT<\/a>, and other models, trained on massive data sets to have billions and even trillions of parameters.<\/p>\n\n\n\n

And now there is BLOOM<\/a> (BigScience Large Open-science Open-access Multilingual Language Model), an Open Source alternative to GPT-3. It was created by the BigScience research project<\/a>, a public-private collaboration with more than 1000 contributors, that was launched in 2021. The BLOOM model was trained on the new Jean Zay supercomputer in France and funded by a 3 million euro budget from a French research agency grant.<\/p>\n\n\n\n

Thomas Wolf<\/a>, the BigScience co-lead and Hugging Face co-founder, said that<\/a> “large ML models have changed the world of AI research over the last two years but the huge compute cost necessary to train them resulted in very few teams actually having the ability to train and research them.”<\/p>\n\n\n\n

Teven Le Scao<\/a>, AI engineer at Hugging Face, told VentureBeat<\/a> that while \u201cGPT-3 is monolingual, BLOOM was designed from the start to be multilingual so it was trained on several languages, and also to incorporate a significant amount of programming language data. BLOOM supports 46 human languages and 13 programming languages \u2014 so that\u2019s a very sizable difference.\u201d<\/p>\n\n\n\n

Alberto Romero<\/a>, CambrianAI analyst, wrote that<\/a> “BigScience and BLOOM are, without a doubt, the most notable attempt at bringing down all the barriers that big tech has erected \u2014 willingly or unwillingly \u2014 throughout the last decade in the AI field. And BLOOM is also the most sincere and honest undertaking to building AI (large language models in particular) that benefits everyone.” <\/p>\n

\"Digg<\/a><\/div>
\"Reddit<\/a><\/div>
\"Stumble<\/a><\/div>
\"Buzz<\/a><\/div>
\"Vote<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Kick<\/a><\/div>
\"Shout<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Post<\/a><\/div>
\"Google<\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"

OpenAI’s GPT-3 may have been first, or at least first to generate a widespread ‘wow’ from people once they saw what AI Large learning Models can do. Since then there have been competitors, like Google’s LaMDA, China’s Wu Dao 2.0,…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[69,24],"tags":[],"class_list":["post-12046","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-open-source"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/12046","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=12046"}],"version-history":[{"count":1,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/12046\/revisions"}],"predecessor-version":[{"id":12047,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/12046\/revisions\/12047"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=12046"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=12046"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=12046"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}