Introduction
In the rapidly advancing world of Artificial Intelligence, chatbots have emerged as indispensable tools that augment our day-to-day interactions. They provide a gamut of services, from answering customer queries to offering personalized assistance. However, traditional chatbots often lack the ability to recall the context of a conversation, leading to responses that can sometimes feel disjointed and impersonal. This is primarily because most chatbots are stateless, treating each user query as an isolated interaction without reference to previous exchanges.
To address this limitation and enhance the conversational experience, the concept of Langchain conversational memory has been introduced. This innovative solution enables chatbots to remember past interactions, use that context to generate more relevant responses, and create a more seamless, human-like dialogue.
In this blog post, we will delve into the world of Langchain memory, exploring its different types and their functionalities. We will also guide you on how to incorporate it with Streamlit and OpenAI's GPT API to build a smarter, more responsive chatbot. Whether you're an AI enthusiast, a seasoned developer, or a curious reader, this exploration of Langchain memory will offer valuable insights into the future of conversational AI. Stay tuned as we unravel the magic behind Langchain memory and its transformative role in the realm of chatbots.
Langchain
Langchain is a comprehensive framework designed for developing applications powered by language models. The creators of Langchain believe that the most impactful and differentiated applications will not only interact with a language model but will also be data-aware and agentic. This means that they will be capable of connecting a language model to other data sources and allowing the language model to interact with its environment.
What is Langchain Memory?
Langchain conversational memory is a module in the Langchain library that enables chatbots and other conversational agents to remember the context from past interaction history and use that information to generate relevant responses. It provides a unified interface for managing and accessing different types of memory including Conversation Buffer Memory, Conversation Summary Memory etc. It can store and retrieve information from previous conversations to learn and adapt to new information and use context to generate more accurate responses. It allows developers to incorporate memory into their conversational AI systems easily and can be used with different types of language models, including pre-trained models such as GPT-3, ChatGPT as well as custom models.
Types of Langchain Memory
This section delves into the various types of memory available in the Langchain library. We will provide an in-depth analysis of how each type works by using them with a ConversationChain and comparing their prompts. We will also evaluate their respective advantages and disadvantages.
Install and import the following libraries and modules which will be used moving forward into the blog
pip install langchain
pip install openai
from langchain.llms import OpenAI
from langchain.chains import ConversationChain
ConversationBufferMemory
The ConversationBufferMemory mechanism in the LangChain library is a simple and intuitive approach that involves storing every chat interaction directly in the buffer. This allows the LangChain Language Model (LLM) to easily recall the conversation history. However, although it provides the maximum amount of information to the LLM by storing everything, there are certain drawbacks to this approach. One of the downsides is that high usage of tokens leads to slower response times and higher computational costs. Additionally, due to the LLM token limit, it is not always possible to store everything as intended.
In Langchain, a chain is used to often break down tasks and is made up of links. Langchain provides ConversationChain which is specially created to have some concept of memory. When creating an instance of the ConversationChain class, three parameters must be provided: llm, which specifies the language model to be used for generating responses; memory, which determines the type of memory to be used to store conversation history; and verbose, which controls whether to print prompts and other information during the conversation.
After initializing the ConversationChain instance, we use the predict method to simulate a conversation based on the user input provided. Here, for example, I have taken three turns of conversation.
from langchain.memory import ConversationBufferMemory
conversation_with_memory = ConversationChain(
llm=OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY"),
memory=ConversationBufferMemory(),
verbose=True
)
conversation_with_memory.predict(input="Hi, I am Sara")
conversation_with_memory.predict(input="I am an AI enthusiast and love sharing my knowledge through blogs")
conversation_with_memory.predict(input="I want you to suggest a good and professional name for my AI blog page based on my name")
conversation_with_memory.predict(input="Can you give more options")
Given below is the sample output of the above-simulated conversation. We observe that the response generated from AI and user input for each turn is stored in memory and passed in the prompt as context for the upcoming query
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi, I am Sara
AI: Hi Sara, my name is AI. It's nice to meet you. What can I do for you today?
Human: I am an AI enthusiast and love sharing my knowledge through blogs
AI: That's great to hear! It sounds like you have a passion for AI. What topics do you like to write about?
Human: I want you to suggest a good and professional name for my AI blog page based on my name
AI: Hmm, that's a tough one. Let me think about it for a moment. How about "Sara's AI Insights"? That has a nice ring to it. What do you think?
Human: Can you give more options
AI:
> Finished chain.
Sure! How about "Sara\'s AI Adventures" or "Sara\'s AI Explorations"?
ConversationBufferWindowMemory
The ConversationBufferWindowMemory is similar to the ConversationBufferMemory except it stores only the recent specified number of interactions by maintaining a window memory and discards the rest.
from langchain.memory import ConversationBufferWindowMemory
conversation_with_memory = ConversationChain(
llm=OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY"),
memory=ConversationBufferMemory(k=2),
verbose=True
)
conversation_with_memory.predict(input="Hi, I am Sara")
conversation_with_memory.predict(input="I am an AI enthusiast and love sharing my knowledge through blogs")
conversation_with_memory.predict(input="I want you to suggest a good and professional name for my AI blog page based on my name")
conversation_with_memory.predict(input="Can you give more options")
Here, we specify the parameter k=2 indicating that we want the last two turns of the conversation to be stored in memory. We can see this effective change in memory compared to above in the sample prompt below
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: I am an AI enthusiast and love sharing my knowledge through blogs
AI: That's great to hear! It sounds like you have a passion for AI. What topics do you like to write about?
Human: I want you to suggest a good and professional name for my AI blog page based on my name
AI: Hmm, that's a tough one. Let me think about it for a moment. How about "Sara's AI Insights"? That has a nice ring to it. What do you think?
Human: Can you give more options
AI:
> Finished chain.
Sure! How about "Sara\'s AI Adventures" or "Sara\'s AI Explorations"?
This approach however is not suited for retaining long-term memory but helps us limit the number of tokens being used.
ConversationTokenBufferMemory
The key difference in ConversationTokenBufferMemory is that it uses a token limit, which is determined by the number of words in the stored messages. This is different from a ConversationBufferWindowMemory, which discards interactions based on the number of turns.
from langchain.llms import OpenAI
from langchain.memory import ConversationTokenBufferMemory
from langchain.chains import ConversationChain
llm=OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY")
conversation_with_memory = ConversationChain(
llm=llm,
memory=ConversationTokenBufferMemory(llm=llm,max_token_limit=60),
verbose=True
)
conversation_with_memory.predict(input="Hi, I am Sara")
conversation_with_memory.predict(input="I am an AI enthusiast and love sharing my knowledge through blogs")
conversation_with_memory.predict(input="I want you to suggest a good and professional name for my AI blog page based on my name")
conversation_with_memory.predict(input="Can you give more options")
The sample output is given below
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: I want you to suggest a good and professional name for my AI blog page based on my name
AI: That's a great idea! Let me think about it for a moment. How about "The AI Adventures of [Your Name]"? That has a nice ring to it.
Human: Can you give more options
AI:
> Finished chain.
Sure! How about "AI Insights with [Your Name]"? Or "AI Explorations with [Your Name]"? Or "AI Journeys with [Your Name]"?
ConversationSummaryMemory
Practically, it's not possible to store all interactions in a long conversation as in ConversationBufferMemory as this will quite often lead to token limit error of even the most advanced LLMs available today. Also, this leads to huge surges in computational costs. Alternatively, if we use ConversationBufferWindowMemory or ConversationTokenBufferMemory, then there is a possibility that we fail to remember the original aim of the problem. We can address this situation by using ConversationSummaryMemory which summarizes interactions between the user and the AI building up a "running summary" of all past interactions. However, it should be noted that it is highly reliant on the summarization capability of the llm being passed as a parameter. Also, ConversationSummaryMemory might take up more tokens for shorter conversations but enables longer conversations with fewer tokens.
from langchain.llms import OpenAI
from langchain.memory import ConversationSummaryMemory
from langchain.chains import ConversationChain
llm=OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY")
conversation_with_memory = ConversationChain(
llm=llm,
memory=ConversationSummaryMemory(llm=llm),
verbose=True
)
conversation_with_memory.predict(input="Hi, I am Sara")
conversation_with_memory.predict(input="I am an AI enthusiast and love sharing my knowledge through blogs")
conversation_with_memory.predict(input="I want you to suggest a good and professional name for my AI blog page based on my name")
conversation_with_memory.predict(input="Can you give more options")
The sample output is given below
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
The human introduces themselves as Sara and the AI introduces itself as AI. The AI then asks what it can do for Sara, to which Sara responds that she is an AI enthusiast and loves sharing her knowledge through blogs. The AI then expresses its appreciation for Sara's passion for AI and inquires about the topics she usually writes about in her blogs. Sara then requests the AI to suggest a good and professional name for her AI blog page based on her name, to which the AI agrees to help.
Human: Can you give more options
AI:
> Finished chain.
Sure, I can provide you with a few more options. How about "Sara\'s AI Adventures", "Sara\'s AI Insights", or "Sara\'s AI Explorations"?
ConversationSummaryBufferMemory
The ConversationSummaryBufferMemory combines both ideas of maintaining a buffer and summarizing the conversation. It stores the recent conversations in a buffer and instead of discarding the past turns, it summarizes these conversations and uses both. The token limit is used here to flush out conversations
from langchain.llms import OpenAI
from langchain.memory import ConversationSummaryBufferMemory
from langchain.chains import ConversationChain
llm=OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY")
conversation_with_memory = ConversationChain(
llm=llm,
memory=ConversationSummaryBufferMemory(llm=llm,max_token_limit=60),
verbose=True
)
conversation_with_memory.predict(input="Hi, I am Sara")
conversation_with_memory.predict(input="I am an AI enthusiast and love sharing my knowledge through blogs")
conversation_with_memory.predict(input="I want you to suggest a good and professional name for my AI blog page based on my name")
conversation_with_memory.predict(input="Can you give more options")
The sample output is as follows
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
System:
The human introduces themselves as Sara and the AI introduces itself as AI. The AI then asks what it can do for Sara, to which Sara responds that she is an AI enthusiast and loves sharing her knowledge through blogs. The AI then expresses its appreciation for Sara's passion for AI and inquires about the topics she likes to write about.
Human: I want you to suggest a good and professional name for my AI blog page based on my name
AI: Sure, I'd be happy to help! What kind of name are you looking for? Do you want something that is related to your name, or something more abstract?
Human: Can you give more options
AI:
> Finished chain.
Absolutely! How about "Sara\'s AI Adventures" or "Sara\'s AI Insights"? Alternatively, you could go for something more abstract like "AI Explorations" or "AI Innovations".
Entity Memory
It is a type of memory designed to store information about specific entities. Here we pass the llm as a parameter that helps in extracting the entities and relevant information about them. As the conversation continues, it gradually accumulates its knowledge about these entities. Note that here we also use the ENTITY_MEMORY_CONVERSATION_TEMPLATE HERE because otherwise, we override the default prompt template for ConversationChain.
from langchain.llms import OpenAI
from langchain.memory import ConversationEntityMemory
from langchain.chains import ConversationChain
from langchain.memory.prompt import ENTITY_MEMORY_CONVERSATION_TEMPLATE
llm = OpenAI(temperature=0,openai_api_key="YOUR_OPENAI_KEY")
conversation_with_memory = ConversationChain(
llm=llm,
verbose=True,
prompt=ENTITY_MEMORY_CONVERSATION_TEMPLATE,
memory=ConversationEntityMemory(llm=llm)
)
conversation_with_memory.predict(input="Sara and John work for the same company")
conversation_with_memory.predict(input="However their departments differ. Sara works for the IT Department while John works for the Finance Department")
conversation_with_memory.predict(input="Sara got a promotion and hence both of them are going out for celebration")
conversation_with_memory.predict(input="What do you know about Sara and John")
The sample output is as follows
> Entering new ConversationChain chain...
Prompt after formatting:
You are an assistant to a human, powered by a large language model trained by OpenAI.
You are designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, you are able to generate human-like text based on the input you receive, allowing you to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
You are constantly learning and improving, and your capabilities are constantly evolving. You are able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. You have access to some personalized information provided by the human in the Context section below. Additionally, you are able to generate your own text based on the input you receive, allowing you to engage in discussions and provide explanations and descriptions on a wide range of topics.
Overall, you are a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether the human needs help with a specific question or just wants to have a conversation about a particular topic, you are here to assist.
Context:
{'Sara': 'Sara works for the same company as John and recently got a promotion, which they are celebrating together.', 'John': 'John works for the same company as Sara and is celebrating her promotion.'}
Current conversation:
Human: Sara and John work for the same company
AI: That's great! It must be nice for them to have a colleague they can rely on.
Human: However their departments differ. Sara works for the IT Department while John works for the Financ Department
AI: That's true. It's great that they can still work together despite their different departments. It shows the strength of the company's culture.
Human: Sara got a promotion and hence both of them are going out for celebration
AI: That's wonderful! It's always nice to see colleagues celebrating each other's successes.
Last line:
Human: What do you know about Sara and John
You:
> Finished chain.
I know that Sara and John work for the same company and recently celebrated Sara's promotion. They work in different departments, with Sara in the IT Department and John in the Finance Department.
Building a smart bot
In an earlier tutorial, we learned how to build a chatbot using Streamlit and ChatGPT. If you haven't checked it out yet, you can find it by clicking here - https://blog.futuresmart.ai/building-a-gpt-4-chatbot-using-chatgpt-api-and-streamlit-chat
To enhance the capabilities of our bot and make it more comparable to human-like intelligence, we will incorporate a new feature by integrating memory storage using langchain. So, let's get coding.
Start by installing and importing the necessary libraries
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationChain
from langchain.chains.conversation.memory import ConversationBufferWindowMemory
import streamlit as st
from streamlit_chat import message
langchain.chat_models: This is the module that contains the language model that can be used for generating responses. We'll be using OpenAI's GPT-3.5 turbo model for our chatbot.
langchain.chains: This is the module that contains different types of chains that can be used for building conversational agents. In this case, we'll be using a ConversationChain to keep track of the conversation history and generate responses.
langchain.chains.conversation.memory: This module contains different types of memory objects that can be used for storing conversation history. In this case, we'll be using ConversationBufferWindowMemory, which stores a fixed-length window of the conversation history.
streamlit: Streamlit is a Python framework for building interactive data science web applications, while Streamlit Chat is a Python package that provides a chatbot interface for Streamlit applications
Let's now give a cool name for our Chatbot using st.title() function and also create a text area for the user input query using st.text_input() function. We initialise the session state variables 'past' and 'generated'. st.session_state is used to store the queries and generated responses from the ChatGPT API. This way, the chat history can be preserved and displayed to the user in a chat-like interface, even if the user refreshes the page or navigates away from the app and comes back later.
st.title("Smartbot")
query = st.text_input("Query: ", key="input")
if 'responses' not in st.session_state:
st.session_state['responses'] = []
if 'requests' not in st.session_state:
st.session_state['requests'] = []
Next, you need to set your Openai API key and initiate our LLM using the ChatOpenAI class with the GPT-3.5 Turbo model which is currently being used by ChatGPT
llm = ChatOpenAI(model_name="gpt-3.5-turbo", openai_api_key="YOUR_API_KEY")
Here we create a ConversationBufferWindowMemory object with k=3 and assigns it to the session state variable 'buffer_memory' if it is not already present. The ConversationBufferWindowMemory is a type of memory to store the history of a conversation in a limited-size buffer window. The "k" parameter specifies the maximum number of turns (i.e., back-and-forth exchanges between the user and the chatbot) that can be stored in the memory buffer window. Once the maximum number of turns is reached, the oldest turn will be removed from the memory to make room for the newest turn.
if 'buffer_memory' not in st.session_state:
st.session_state.buffer_memory= ConversationBufferWindowMemory(k=3)
In the next step, we initialize a conversation chain by passing in the necessary parameters. The "llm" parameter specifies the language model to use for generating responses, which is a ChatOpenAI object in this case. The "verbose" parameter is set to True, which allows us to print the running conversation details for easy debugging.
However, the most crucial parameter is "memory". We pass the ConversationBufferWindowMemory object stored in the st.session_state.buffer_memory to the memory parameter of the ConversationChain class. This is because we want our chatbot to be able to store and retrieve previous conversations in its memory buffer
conversation = ConversationChain(
llm=llm,
memory=st.session_state.buffer_memory,
verbose=True
)
If there is input provided by the user, we use the chain.run method to get a response from the llm. Then we add the user's query and the generated response to the chat history, which is stored in the session state variables past and generated.
if query:
response = conversation.run(query)
st.session_state.requests.append(query)
st.session_state.responses.append(response)
Now we move on to the final step of displaying the conversation. If there are any generated responses in the st.session_state object, a for loop is initiated. The for loop iterates through the generated list in order of the latest response to the earliest.
For each generated response, the message() function is called twice to display the query made by the user and the response generated by the llm. The key parameter is used to uniquely identify each message.
if st.session_state['responses']:
for i in range(len(st.session_state['responses'])-1, -1, -1):
message(st.session_state['requests'][i], is_user=True, key=str(i) + '_user')
message(st.session_state["responses"][i], key=str(i))
That's it. We have successfully built a custom chatbot with conversational memory and can run the application using the following command.
streamlit run file_name.py
If everything has been set up correctly, you should now have a chatbot that resembles the one shown in the example.
If you are more interested to learn about Langchains integration with SQL , then be sure to check out the below YouTube tutorial.
Building a Document-based Question Answering System with LangChain, Pinecone, and LLMs like GPT-4.
Follow FutureSmart AI to stay up-to-date with the latest and most fascinating AI-related blogs - FutureSmart AI
Looking to catch up on the latest AI tools and applications? Look no further than AI Demos This directory features a wide range of video demonstrations showcasing the latest and most innovative AI technologies. Whether you're an AI enthusiast, researcher, or simply curious about the possibilities of this exciting field, AI Demos is your go-to resource for education and inspiration. Explore the future of AI today with aidemos.com
