Combine LangChain 🦜🔗 & Llama-Index 🦙
Adheeban Manoharan
Posted on May 22, 2023
In the last post of this blog series, we saw how we could use LangChain's memory classes for building a chatbot-like interface with OpenAI's GPT-3 API. In this blog, we are basically gonna use these memory classes to keep track of our previous conversations with the LLM all the while making it answer our queries regarding the contents of the document we feed to it.
We'll try to keep the dependencies on LlamaIndex to a bare minimum, because Langchain almost covers all the aspects of utilities that we would need to build this. We'll only be using the adapters available in the LlamaIndex library, because they perform better than the Langchain ones.
Now before we dive in, let me set the context for you as to what we are trying to achieve. We are trying to build an intelligent system that reads the contents of the document and should be able to answer any queries from that content. This intelligent should also be able to keep track of our past conversations and if at all any question pointing one of the previous queries, it should be able to answer that.
Remember when I told you in the last blog about completion tokens and context tokens? The maximum context window for GPT-3 is 4096
tokens. We should be able to squeeze all our context, memory, prompt and response all within the same window.
Something like this ->
With that out of the way, let's hustle !
Memory meets Custom Knowledge 🧠📔
Let's start right away with the code:
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferWindowMemory
from langchain.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from llama_index import download_loader
import os
os.environ["OPENAI_API_KEY"] = 'Your API Key Here'
file_path = input('Enter the youtube link: ')
YoutubeTranscriptReader = download_loader("YoutubeTranscriptReader")
documents = YoutubeTranscriptReader().load_data(ytlinks=[file_path])
documents = [doc.to_langchain_format() for doc in documents]
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(documents)
embeddings = OpenAIEmbeddings()
retriever = FAISS.from_documents(documents, embeddings).as_retriever(k=4)
llm=ChatOpenAI(temperature=1, model_name="gpt-3.5-turbo", max_tokens=2048)
memory = ConversationBufferWindowMemory(
memory_key='chat_history',
return_messages=True,
k=6
)
conversation = ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=retriever,
# verbose=True,
memory=memory,
max_tokens_limit=1536
)
def chatbot(pt):
res = conversation({'question': pt})['answer']
return res
if __name__=='__main__':
while True:
print('########################################\n')
pt = input('ASK: ')
if pt.lower()=='end':
break
response = chatbot(pt)
print('\n----------------------------------------\n')
print('ChatGPT says: \n')
print(response, '\n')
You know the drill, just copy the code and paste it in a file and while running the file, install the dependencies if anything is prompted.
These are some important points, that I thought I should highlight in the above code:
FAISS
is a library built my MetaAI for semantic search, that helps us with the similarity search within the context.
ConversationBufferWindowMemory
is the memory that we've used, this type of memory keeps track of the lastk
conversations between the user and the LLM. I have givenk=6
.
RecursiveCharacterTextSplitter
is an utility from LangChain that splits the context into chunks.
ConversationalRetrievalChain
is a type of chain that aids in a conversational chatbot-like interface while also keeping the document context and memory intact.
That should give you an idea. If you have any more questions about the code, feel free to comment below.
Now, for a change, I have used the YoutubeTranscriptReader
from the llama-index, this reads the transcript of the youtube link that you provide and builds the context out of it for the LLM. In the last blog we had used SimpleDirectoryReader
.
Here's what I tried with Matt Walker's talk on Sleep is your superpower:
Enter the youtube link: https://youtu.be/5MuIMqhT8DM
########################################
ASK: list down the key points in the speech
----------------------------------------
ChatGPT says:
1. Regularity in sleep timing is important to anchor sleep and improve its quantity and quality.
2. Keeping the bedroom cool aids in initiating and maintaining sleep.
3. Sleep is a non-negotiable biological necessity for human beings' health and well-being.
4. Sleep loss is a silent epidemic in industrialized nations and a significant public health challenge.
5. Sleep debt cannot be accumulated and repaid at a later point in time.
6. Getting out of bed when unable to sleep for too long establishes the association between bed and sleep.
7. The importance of prioritizing sleep for overall health and well-being.
########################################
ASK: now convert that into a song for a 5 year old
----------------------------------------
ChatGPT says:
Sure, here's a simple song about the importance of sleep:
Verse 1:
We all need to rest our heads
And go to sleep in our cozy beds
Regular sleep is the key
To feeling good and healthy
Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright
Verse 2:
Keep your room nice and cool
And your sleep will be extra smooth
And if you can't fall asleep
Get out of bed and don't make a peep
Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright
Verse 3:
Sleep is important, it's true
For our brains and our bodies too
We won't feel so good without it
So let's all just go to bed
Chorus:
Sleep, sleep, the magic elixir
It helps us grow and gives us vigor
Don't stay up too late at night
Or you won't feel so bright
Outro:
So let's all get a good night's sleep
And tomorrow, we'll feel so sweet
Sleep tight and have nice dreams
And wake up feeling like a new team!
########################################
Notice how the LLM is able to sense that I'm referring the previous conversation in the second question and converts that into a song? Exactly what we were trying to achieve. Do try it out with your own Youtube video.
Now, that's about LangChain and Llama-Index. In the next blog of this series, we'll have a look at all the awesome adapters that are available in the Llama-Hub. Cheers, see ya in the next one 😉
Posted on May 22, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
March 28, 2023