What is GPT-3 and What does it Mean for AI Innovation?
The GPT-3 or Generative Pre-trained Transformer was released in June this year and is a language model created by Open AI, a non-profit founded by Elon Musk and Sam Altman, combined they invested over $1 billion with the goal of advancing AI to increase its applications and abilities. founded in 2015, the aims of the company are to share teams' work "with the world" Open AI stated that "AI should be an extension of individual human wills, and in the spirit of liberty, as broadly and evenly distributed as is possible safely".
The model has been generated over a number of years with this being the 3rd and most recent version, the idea behind pre-generative training was to move away from AI that is trained on labeled data, while there are huge arrays of non-labeled data. The difference is seen in that GPT-3 can create predictions of future text on unlabeled data because the words used to train and tune the AI are evaluated. Predictions start to converge and improve as the process is repeated.
The first version of the Generative Pre-Training used around 7,000 books in its collection, whereas the GPT-3 uses over 410 billion parameters from the internet, 67 billion from books, 3 billion from Wikipedia, and more, giving the GPT-3 an unimaginable knowledge depth from data.
What Can GPT-3 Do?
Put simply, the GPT-3 tool generates text by use of pre-trained algorithms that give the tool all it needs to complete commands, using deep learning to produce human-like text.
But what the tool does it much more extensive than generate text, the language model has a vast range of capabilities, using AI, Deep Learning, and Machine Learning to give specific answers to abstract questions, chat and communicate convincingly, and even translate text to any language, as well as simplify or even elaborate text. Many demos of the product have already been released showcasing just some of the huge range of capabilities of the tool.
The product can use an application called 'Figma', where the developer has created a plugin that allows the user to simply enter a text description of what they want as an application, the plug-in then produces an application tailored to the description commands.
Additionally, the computer can create code from text commands to create functions such as buttons and lists, the code is generated automatically by AI to suit what the description calls for. Additionally, the tool can work alongside excel to answer commands and search for information it predicts that you need, along with many other functionalities. The system can even generate lengths of texts such as poems and essays, as evidenced by the Guardian, who released an article that was entirely written by the language model from a command and some key information. These are just a few examples of the huge range of GPT-3s capabilities, Open AI often releases new information and demos about their projects and a huge range are available to view.
How Does GPT-3 Work?
The generative pre-trained transformer is essentially a language prediction model, as such its algorithmic structure is created to predict what is the most useful information to include when responding to a certain command, this is possible due to the "pre-training" that it was exposed to. This pre-training gave the tool immense amounts of information in order for it to understand language structure, of course, exposing the system to this amount of information took a huge amount of computing time, said to have cost the company over $4.5 million.
The understanding of language structure was made possible by the use of semantic analysis, which allows the system to comprehend and interpret sentences, paragraphs, or entire documents through the analysis of their grammatical structure, and the identification of relationships between individual words in a particular context.
It takes many repetitions to get the 'correct' output, and once it is produced, 'weight' will be assigned in order for the model to 'learn' probable methods and create a better basis for predictive text.
Who Can Use GPT-3?
Initially, the tool was released as an API as opposed to open-sourcing the model, meaning only larger organizations would be able to take advantage, due to the issue of the large cost of the product as well as the expertise needed to develop and deploy. Open AI also notes that because of its smaller availability the company will be better able to monitor, reduce, and solve the potentially harmful applications before broadening access over time with an open access model.
Although in September 2020 Open AI gave Microsoft exclusive license of GPT-3, who saw great potential and opportunity in the tool, CTO of Microsoft Kevin Scott stated that "We see this as an incredible opportunity to expand our Azure-powered AI platform in a way that democratizes AI technology, enables new products, services, and experiences, and increases the positive impact of AI at Scale".
Although this may seem as though the tool is being taken away from a 'broadly distributed' user base, Open AI stated that “the deal has no impact on continued access to the GPT-3 model through OpenAI’s API, and existing and future users of it will continue building applications with our API as usual”.
Why Does the Creation of GPT-3 Matter?
Because GPT-3's capability of creating anything with a language structure, which has such a large array of applications and uses, the system, will impact and change the future of technology, software, and the way we use the internet. While initially this will most likely be used by large companies, because of the huge cost of computing power needed to run its functions. Additionally, there are some corrections to be made, an example being its difficulty in producing extensive amounts of texts as opposed to smaller pieces.
Sam Altman, co-founder of Open AI stated that the 'hype' for GPT-3 is exaggerated, stating that the tool is 'just. an early glimpse' of the AI that will "change the world".
Keep up-to-date with the latest tech industry insights, trends as well as information technologies, app development, and small business content with the Proteams Blog