What is Generative AI?

Generative AI is a subfield of artificial intelligence (AI) that involves creating algorithms that can generate new contents like:

• An email response
• A short story
• A simple image

Example:

Google search has auto completion feature. Think of it as a very basic implementation of Generative AI where query suggestions were generated by the model based on the terms you type on the search bar. This model was trained against the zillions of queries searched by users across the world and generated suggestions or queries.

Generative AI in details:

Generative AI is supported by GPT or Generative Pre-trained Transformers takes this “generate” capability further by several levels. GPT are a family of "large-language models" trained using "artificial neural networks" using "transformer architecture" on "massive un-labelled text data" to generate human-like text responses.

Now that we have a very basic understanding of the different jargons let us dig a bit deeper.

Large Language Model:

• What are Large Language Models and how are they defined?
• How large is a Large Language Model?

The term “Large” here refers to the number of parameters that were used to train the model. Here’s a summary of the versions of GPT models released so far along with the parameter count.

Pre-trained Transformers:

This means that the model’s wealth of knowledge is limited to the vast amount of data it has been trained with. If some data was not fed to the model, then it is not known to the model and therefore it cannot generate content around it or answer the question.

Transformers:

Transformers are a type of neural network capable of understanding the context of sequential data, such as sentences, by analyzing the relationships between the words.

The Transformer in NLP is a novel architecture (ANN) that aims to solve sequence-to-sequence tasks while handling long-range dependencies with ease. It relies entirely on "self-attention" to compute representations of its input and output WITHOUT using sequence-aligned RNNs or convolution.

Self-Attention:

Let us start with revisiting what attention is in the NLP universe?

Attention allowed us to focus on parts of our input sequence while we predicted our output sequence. In simpler terms, self-attention helps us create similar connections but within the same sentence. Look at the following example:

• “I poured water from the bottle into the cup until it was full.” it => cup
• “I poured water from the bottle into the cup until it was empty.” it=> bottle

By changing one word “full” to “empty” the reference object for “it” changed. If we are translating such a sentence, we will want to know what the word “it” refers to.

How does generative AI work?

The most common way to train a generative AI model is to use supervised learning - here the model is given a set of human-created content and corresponding labels. It then learns to generate content that is similar to the human-created content and labeled with the same labels.

Applications of Generative AI?

• Language: Marketing Content, Code Development, Essay Writing
• Visual: Image/Video Generation, Design, 3D Modelling
• Music: Music/Voice Generation

Traditional AI vs. Generative AI:

• Traditional AI is focused on detecting patterns, generating insights, automation, and prediction.
• Generative AI starts with a prompt that lets a user submit a question along with any relevant data to guide content generation.
• Traditional AI algorithms process data and return expected results, such as analyses or predictions
• Generative AI algorithms produce newly synthesized content, like text or images, based on training from existing data

Model Setup:

• A reason that Generative AI became so popular so quickly, is because it empowers the end user. Right now, anyone can log on to ChatGPT and start using it, which is a first for an AI application. Zero barrier to entry.
• whereas Traditional AI necessitates rigorous data preparation and processes to develop and test a model designed to produce a good outcome.

So, with generative AI, you can just start talking to it, and it will understand what you want to do and offer a response.

Example:

ChatGPT, DALL-E, and Bard are examples of generative AI applications that produce text or images based on user-given prompts or dialogue.

Applications by Industry:

• Automotive Industry: Synthetic data produced by AI can run simulations and train autonomous vehicles.
• Media and Entertainment: AI can be used to quickly, easily, and more cheaply generate content, or (as a tool) to enhance the work of creatives like writers and designers.

Image Reference:

• Traditional AI vs. Generative AI
• Transformers
• GPT Tarining data
• Prompt Engineering
• GPT