{"id":10995,"date":"2021-06-14T07:00:00","date_gmt":"2021-06-14T15:00:00","guid":{"rendered":"https:\/\/formtek.com\/blog\/?p=10995"},"modified":"2021-06-11T07:42:34","modified_gmt":"2021-06-11T15:42:34","slug":"gpt-3-competition-rapid-development-of-massive-natural-language-systems","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/gpt-3-competition-rapid-development-of-massive-natural-language-systems\/","title":{"rendered":"GPT-3 Competition: Rapid Development of Massive Natural Language Systems"},"content":{"rendered":"\n

GPT-3 has received reviews of astonishment since it was first announced in late May 2020 by OpenAI<\/a>. GPT-3 (Generative Pre-Trained Transformer 3) is the third generation of natural language processing models developed by OpenAI, an artificial intelligence research operation originally funded by Elon Musk. <\/p>\n\n\n\n

GPT-3 has been described<\/a> as “one of the most interesting and important AI systems ever produced.” After being trained on 45 terabytes of text data, it is able to compose essays and write computer code. It’s ability has stunned many and caused a rethink about the possibilities of machine intelligence.<\/p>\n\n\n\n

The burst of excitement around how people would use GPT-3 was tempered when in late September 2020 it was announced that Microsoft had secured privileged access to the GPT-3 source code. Others could license GPT-3 output, but not directly access the source code. The result has been keen interest by a number of groups to try to reproduce GPT-3 capabilities.<\/p>\n\n\n\n

Earlier this month, the Beijing Academy for AI (BAAI) announced a “Superscale Intelligence Model.” The Chinese system called Wu Dao 2.0<\/a> is 10 times bigger than OpenAI’s GPT-3 and not only composes natural language text in both Chinese and English, it can also generate images and voice.<\/p>\n\n\n\n

In the US, the group EleutherAI <\/a>is trying to replicate the success of GPT-3 with an open-source approach. EleutherAI has built a model they’re calling GPT-J-6B is nearly on par with what GPT-3 can do.<\/p>\n\n\n\n

Connor Leahy<\/a>, one of the founding members of EleutherAI, said that<\/a> \u201cwe think it\u2019s probably fair to say this is currently the best open-source autoregressive language model you can get by a pretty wide margin.”<\/p>\n\n\n\n

Chinese AI researcher Blake Yan said that<\/a> “these sophisticated models, trained on gigantic data sets, only require a small amount of new data when used for a specific feature because they can transfer knowledge already learned into new tasks, just like human beings. Large-scale pre-trained models are one of today\u2019s best shortcuts to artificial general intelligence.\u201d<\/p>\n

\"Digg<\/a><\/div>
\"Reddit<\/a><\/div>
\"Stumble<\/a><\/div>
\"Buzz<\/a><\/div>
\"Vote<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Kick<\/a><\/div>
\"Shout<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Post<\/a><\/div>
\"Google<\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"

GPT-3 has received reviews of astonishment since it was first announced in late May 2020 by OpenAI. GPT-3 (Generative Pre-Trained Transformer 3) is the third generation of natural language processing models developed by OpenAI, an artificial intelligence research operation originally…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[69],"tags":[],"class_list":["post-10995","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/10995","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=10995"}],"version-history":[{"count":1,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/10995\/revisions"}],"predecessor-version":[{"id":10996,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/10995\/revisions\/10996"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=10995"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=10995"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=10995"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}