GPT-3 chatbots: What do they mean for conversational AI

There is a lot of buzz around a new technology with three letters and one number: GPT-3! What is the hype about, and is it worth it? Is this development as significant as some claim, or is it overrated? We take a closer look at GPT-3 chatbots, their pros and cons, and their potential for Conversational AI.

In 2020, the launch of a new technology called GPT-3 made quite a stir in the artificial intelligence (AI) and chatbot industry. This new machine-learning model was able to enerate any type of text based soley on internet data. And it was the best model of this nature ever created!

According to OpenAI, the organization that developed the technology, the GPT-3 chatbots are so evolved that they can pass the Turing test. The Turing test, named after mathematician Alan Turing, tests if a machine’s behavior is indistinguishable from a human’s.

The GPT-3 model seems to be quite a technological achievement, but could there be a commercial use? Companies have started to wonder about the usage of the GPT-3 model and how it could improve their businesses. And if so, what could be its main uses? Are there downsides? What does GPT-3 mean for the future of AI chatbots?

In the 2022 edition of the Conversational AI Fest, Fréderic Godin, Head of AI at Chatlayer by Sinch, took the virtual stage to answer these questions. And the insights could not come from a better source, as Chatlayer by Sinch offers some of the most advanced NLP and AI chatbots in the market.

What are GPT-3 chatbots, and why should you care?

GPT-3 is a general-purpose AI model. The abbreviation stands for “Generative Pre-Trained Transformer 3”. This is the third iteration of the tool that has been released.

The company behind it is OpenAI, a research business co-founded by Elon Musk. Experts have described the technology as one of the most important and useful advances in AI in years.

The GPT-3 model has a transformative nature, which means that it generates text (answers) based on existing knowledge. The information that GPT-3 chatbots use comes from the internet. Crawling tools feed it to the model by scanning billions of texts online.

The internet data that the transformer model uses is stored in computers and servers around the world. This has a (high) maintenance price tag: ten million US dollars per year, says Godin.

“In a nutshell, these were the three factors that made GPT-3 technology possible: its transformer architecture, the data that it was nourished with and the several computers and servers to process all this data.”

Fréderic Godin

Head of AI, Chatlayer by Sinch

Part of the hype around GPT-3 models is that, as a transformer chatbot, it can not only answer a question, but complete a myriad of tasks within a written structure: it can summarize long texts, take memos, write essays, translate languages, and even create computer code. Chatbots with transformer architecture learn how language works, how sentences are constructed, which kind of sentence follows the next sentence and how to leverage the current context.

Speaking of advanced tech: check out, how Chatlayer’s bot outperforms Microsoft and Google!

GPT-3 model exemplified: “chit-chat” and task-oriented chatbots

GPT-3s can work pretty well as “chit-chat” bots where they can have a casual conversation with a person. It is in this context that, at first sight, chatbots built with GPT-3 can also pass the Turing test. Their accurate and grammatically correct answers may prompt a user to think that the replies came from another human being.

GPT-3 also could come in handy as task-oriented chatbots. In this scenario, GPT-3 chatbots help customers to fulfill certain tasks, such as buying a product, or reporting a problem.

In the future, this could lead to sophisticated search chatbos, as Godin explained.

One such scenario could be a search bot on a real estate website. Instead of searching for properties themselves, users could just tell the bot what they are looking for. The bot would then be able to find a suitable house for them, and deliver a much more user-friendly search experience.

GPT-3 chatbots have many advantages, but also some drawbacks

The GPT-3 model shows how the technology has the potential to turn chatbots into more advanced search engines. However, not everything is perfect with this new technology.

For instance, it’s still difficult for a GPT-3 to work with external knowledge that it doesn’t have. Moreover, if a GPT-3 doesn’t have the necessary knowledge (information) to answer a question, it won’t say that it doesn’t know the answer, but instead come up with whatever answer it can fabricate. This makes it difficult to control its output.

This lack of direction doesn’t always have a good outcome. In some tests, GPT-3s have generated a biased, sexist, or even racist answers. These results prompted OpenAI to warn others that:

Despite making significant progress, our InstructGPT (=GPT-3) models are far from fully aligned or fully safe.

OpenAI

Even with OpenAI’s explanation, the fame of being unpredictable stayed with GPT-3 chatbots. “There are a lot of companies that initially think that all bots are like GPT-3 and are able to give uncontrolled answers”, says Joachim Jonkers, CPO of Chatlayer by Sinch. Having said that, can there be a future for GPT-3?

GPT-3 technology will improve with training

Part of the industry’s hesitation with the GPT-3 are the unpredictable answers it can give, but this can be counteracted with some improvements. “A way to increase the quality of the GPT-3’s output is by giving it more examples of expected answers”, says Godin.

Also, according to the AI specialist, another way to prevent GPT-3 from giving problematic answers could be to add detection mechanisms to it, and teach it the topics that it should not talk about.

To sum up, this broad and therefore powerful tool still needs teaching, training, and time to reach its full potential.

The potential of GPT-3 and conversational AI

Despite some issues, GPT-3 could, in the future, also be used to improve conversational AI chatbots. GPT-3s could use their conversational skills and the vast data they have access to to train NLP chatbots with expressions. This would save conversation designers time, and improve the quality of the NLP bots.

As mentioned, another potential use case would be to use GPT-3s for a new kind of search engine, that maybe, one day, could even be an alternative to Google or Bing.

As AI chatbots are becoming more advanced, it will be interesting to see how they evolve when combined with a data-rich tool like GPT-3.

At Chatlayer and around the world, specialists and businesses are excited to see what the future holds for these two technologies coming together.