OpenAI recently released a beta feature named My GPTs, which allows anyone to easily customize a version of ChatGPT for a specific purpose. This opens the door to new uses of Large Language Models (LLMs) in creating specialized ChatBots. What I find innovative is the method of customizing GPT: by answering questions and giving instructions to ChatGPT, which then configures the bot. Now, anyone can create their own GPT without needing programming skills! This new no-code approach has been adopted by LlamaIndex, enabling anyone to create a custom bot that implements the Retrieval-Augmented Generation (RAG) pattern on a single document.

In this article, I'll explore how to create such a personalized GPT with OpenAI and then what the brand new rags and OpenGPTs projects from LlamaIndex and LangChain allow you to do similarly.

1. Customize a version of OpenAI ChatGPT

Access to the "My GPTs" feature requires a paid account on OpenAI.

Create a GPT

Once logged in, you can explore public GPTs or create your own.

Click on 'Create a GPT' and answer the questions. For our example, we will create a GPT specialized in mudras (hand gestures), with its specific knowledge based on a PDF file.

You can switch between the Create and Configure views to design your GPT.

The Configure view allows you to change or refine the GPT name, description, instructions, and prompt examples.

Implement RAG

This is where I uploaded the mudras.pdf file for my GPT to use in Retrieval-Augmented Generation (RAG).

You can test the GPT in the Preview panel and then publish it with private, unlisted, or public visibility.

Start a new chat to use the GPT.

After posing a question, you can see the GPT searching the document to provide an answer.

Once the search phase is completed, the GPT displays its response.

Add external API Calls

It's also possible to add actions to the GPT configuration. Let's add a call to an external API.

The schema I declared is as follows:

{
  "openapi": "3.1.0",
  "info": {
    "title": "YogArkana",
    "description": "Get detailed instructions on a Mudra",
    "version": "v0.0.1"
  },
  "servers": [
    {
      "url": "https://deck.yogarkana.com"
    }
  ],
  "paths": {
    "/json/mudra/{name}Fr": {
      "get": {
        "description": "From the Sanskrit name of a mudra get the following elements in JSON format: name, description, directions, details, concentration and benefits",
        "operationId": "GetMudra",
        "parameters": [
          {
            "name": "name",
            "in": "path",
            "description": "Name of the Mudra in Sanskrit, with the first letter in capital and the rest in Camel Case (remove spaces in the name). If not present at the end of the name append 'Mudra' with no space.",
            "required": true,
            "deprecated": false,
            "allowEmptyValue": false,
            "explode": false,
            "allowReserved": false,
            "schema": {
              "type": "string"
            }
          }
        ],
        "deprecated": false
      }
    }
  },
  "components": {
    "schemas": {}
  }
}

In summary, I instruct my GPT on how to call the YogArkana backend to get information on a specific mudra when provided with its Sanskrit name. Note that in the description fields, I give specific rules to format the API call properly.

We can test and debug the API call in the Preview panel.

Here are some examples of debug information provided.

[debug] Calling HTTP endpoint
{
  "domain": "deck.yogarkana.com",
  "method": "get",
  "path": "/json/mudra/{name}Fr",
  "operation": "GetMudra",
  "operation_hash": "44384b85e501956265a75d90b721b53a7bbe3efe",
  "is_consequential": false,
  "params": {
    "name": "ChinMudra"
  }
}

I can verify that the API call is well formatted and the name parameter is correctly formed.

[debug] Response received
{
  "response_data": {
    "_id": "ChinMudraFr",
    "_rev": "4-0f815dcb8505d3257afbcc7cb2d71911",
    "sanskrit": "Chin Mudra",
    "lang": "fr",
    "translation": "Geste de la conscience",
    "type": "mudra",
    "order": 20,
    "quality": "Abandon au divin",
    "arcane": true,
    "directions": [
      "Joignez les bouts des indexes et des pouces de chaque main",
      "Les pouces et les indexes forment un cercle",
      "Gardez les trois autres doigts allongés",
      "Tournez les paumes vers le bas et posez les sur les genoux ou les cuisses",
      "Détendez les épaules et gardez la colonne dans son alignement naturel"
    ],
...

Then, I can review the response returned by the external API. Note that the information is in French, but the final answer is translated into English and summarized by the LLM.

Let's publish the new version of the GPT, and there we have it: a fully operational GPT called "Mudra Guru" created without writing a single line of code.

2. Do (almost) the same with rags from LlamaIndex

LlamaIndex recently released the rags project to create a similar experience. It's still in its early stages, but I couldn't resist testing it.

The installation is straightforward

git clone https://github.com/run-llama/rags.git && cd rags
pip install -r requirements.txt

RAG with OpenAI GPT

After installing all dependencies, create the .streamlit/secrets.toml configuration file to store the OpenAI API key.

openai_key = "<openai_key>"

Then, run the application and navigate to the Web GUI.

streamlit run 1_🏠_Home.py

rags operates in three sequential steps.

Like with OpenAI's 'My GPTs', the first step is to explain in natural language what you want the ChatBot to accomplish. In my prompt, I mention the mudras.pdf file.

Here is the entire conversation I had to create the ChatBot:

rags: What RAG bot do you want to build?

me: Provide advice and detailed instructions on mudras based on 
the content on the mudras.pdf file

rags: Great, the content from "mudras.pdf" has been loaded 
successfully. Now, I can create a system prompt to generate advice 
and detailed instructions on mudras for you. Let's proceed with 
creating the system prompt.

What specific aspects of mudras would you like the advice and 
instructions to focus on? For example, are you interested in 
mudras for meditation, healing, or perhaps for specific health 
issues? Please provide me with a bit more detail so I can tailor 
the system prompt accordingly.

me: The use of mudras will be provided by the user

rags: The system prompt has been created successfully. Now, I can 
use this prompt to generate advice and detailed instructions on 
mudras as specified by the user.

Before we proceed to create the agent, would you like to set any 
specific parameters for the Retrieval-Augmented Generation (RAG) 
pipeline? For example, you can set the number of documents to 
retrieve or adjust the threshold for document relevance. Please 
let me know if you'd like to customize any settings.

me: no

rags: The agent has been created successfully and is now ready to 
provide advice and detailed instructions on mudras based on the 
content loaded from the "mudras.pdf" file.

If you have any specific questions or need information on 
particular mudras, feel free to ask, and the agent will assist you 
accordingly. Is there anything else you would like to do at this moment?

In step 2, "RAG Config", I can refine the proposed system prompt and change the LLM parameters. I just check the "Include Summarization" box and update the agent.

Here is the system prompt proposed by rags:

System Prompt:

"Utilize the provided tools to offer advice and detailed 
instructions on mudras. When responding to user queries, ensure 
that you always apply the information from the tools available. Do 
not provide responses without consulting these resources. Your 
guidance should be thorough and adhere to the details contained 
within the tools. Remember, the data source should not be 
explicitly mentioned in your responses; it is an implicit part of 
your analysis and the answers you provide."

Step 3 is the actual use of the ChatBot.

And there you have it! For now, rags is more limited than OpenAI's GPTs, supporting only a single PDF file for RAG and not offering actions, but it's still in its early stages. I anticipate many alternative open source solutions will soon adopt this no-code approach to setting up a ChatBot.

RAG with a local LLM

But rags has another exciting feature: the ability to support different LLMs like Claude 2 or Replicate, as well as local models from Hugging Face!

So, let's modify our ChatBot to use the Luna AI Llama2 model for inference and the BGE Small EN model for embeddings.

I downloaded the inference model myself and stored it in the rags directory due to its size, letting rags download the second, smaller model.

We need to install some new dependencies to enable rags to run the models locally.

pip install transformers torch llama-cpp-python

We can now reconnect to the rags GUI and, in step 2, change the LLM configuration with the two models, adding the local: prefix.

After moving to step 3, we can chat with our document using only local models!

Note that step 1 still uses OpenAI to configure the bot. It's also possible to modify this.

3. What about OpenGPTs from LangChain?

Let's nox delve into OpenGPTs by the LangChain project.

This new tool features a frontend developed in Typescript/React and a backend in Python. However, be mindful that its license is unique and prohibits providing hosted or managed services using the software. Additionally, contributing to the project requires signing a Contributor License Agreement and granting copyright to LangChain.

OpenGPTs, like rags, aims to facilitate the creation of custom GPTs without the need for programming. Users can simply select components and configurations. It utilizes the LangChain framework, offering options such as LLMS, vector databases, and various tools. One of its features includes content retrieval, alongside capabilities like external website searches.

A key difference from rags is OpenGPTs' ability to integrate tools into the custom GPT, enhancing it with specific functions. However, it does not support configuration through chat.

You can explore the solution online here, but let's proceed with a hands-on installation and test.

Installation and configuration

The default installation is straightforward. Begin by cloning and installing the backend requirements:

git clone https://github.com/langchain-ai/opengpts.git && cd opengpts

Next, configure your license keys in the .env file. This step allows customization of the default setup and the choice of different vectorstores or LLMs, such as Azure OpenAI, Anthropic, and Amazon Bedrock. It's also necessary to provide API keys for certain tools:

OPENAI_API_KEY=your_secret_here
ANTHROPIC_API_KEY=your_secret_here
AWS_ACCESS_KEY_ID=your_secret_here
AWS_SECRET_ACCESS_KEY=your_secret_here
AZURE_OPENAI_DEPLOYMENT_NAME=your_secret_here
AZURE_OPENAI_API_BASE=your_secret_here
AZURE_OPENAI_API_VERSION=your_secret_here
AZURE_OPENAI_API_KEY=your_secret_here

TAVILY_API_KEY=your_secret_here
FIREWORKS_API_KEY=your_secret_here
YDC_API_KEY=your_secret_here
KAY_API_KEY=your_secret_here

Although it's possible to customize further, we'll stick to the default setup for this demonstration and just rovide our OpenAI key.

Now, create the docker containers for the backend:

docker compose up -d

This command initiates a Redis container for both vectorstore and configuration persistence, as well as the backend container.

Then, start the frontend:

cd frontend
yarn
yarn dev

OpenGPTs also offers an API for building custom frontends or integrating with existing applications.

Custom Bots creation and use

Navigate to http://localhost:5173/ to create and use your custom ChatBots. Configuring a new Bot involves setting various parameters and options.

In my test, I selected the GPT-3.5 model, enabled the 'Retrieval' tool, uploaded two documents, and named my bot 'TS Bot'.

The logs confirm the Bot's creation and document ingestion into Redis:

opengpts-backend-1  | INFO:     172.22.0.1:56704 - "PUT /assistants/fbc5a3c7-ffc6-47b3-bb0d-0e6948269968 HTTP/1.1" 200 OK
redis               | 1:M 01 Dec 2023 21:51:46.611 # <search> creating vector index. Server memory limit: 13064253440B, required memory: 6308144B, available memory: 13063239320B
redis               | 1:M 01 Dec 2023 21:51:46.612 * <module> Scanning index opengpts in background
redis               | 1:M 01 Dec 2023 21:51:46.613 * <module> Scanning index opengpts in background: done (scanned=2)
opengpts-backend-1  | INFO:     172.22.0.1:56708 - "POST /ingest HTTP/1.1" 200 OK

Now, you can interact with your newly created bot.

OpenGPTs indicates the use of the Retriever tool, and you can view the response of the invoked function:

Here is a summary of the three solutions we covered in this article.

In conclusion, the advent of OpenAI's My GPTs, LlamaIndex's rags, and LangChain's OpenGPTs projects marks a significant milestone in the democratization of AI technology. These platforms not only simplify the process of creating specialized ChatBots but also open up a world of possibilities for users without extensive programming knowledge. Through the intuitive interfaces of My GPTs, rags and OpenGPTs, we can now tailor ChatBots to specific needs, whether it's for detailed knowledge on mudras or any other niche topic.

As we continue to explore and expand the capabilities of AI, tools like My GPTs, rags and OpenGPTs will play a crucial role in shaping the future of personalized AI interactions. Stay tuned for upcoming features, and in the meantime, begin creating your own assistants!

The journey into the realm of customized AI has just begun, and the possibilities are as limitless as our imagination.