LangChain: Enhancing Performance with Memory Capacity | by Marcello Politi | Jun, 2023


Photo by Milad Fakurian on Unsplash

LangChain’s Boost through Memory Expansion Techniques

I have already published articles about LangChain before, introducing the library and all its capabilities. Now I would like to focus on a key aspect, how to manage memory in intelligent chatbots.

Chatbots or agents also need an information storage mechanism, which can take different forms and perform different functions. Implementing a memory system in chatbots not only helps make them more clever, but also more natural and useful for users.

Fortunately, LangChain provides APIs that make it easy for developers to implement memory in their applications. In this article, we will explore this aspect in more detail.

Use Memory in LangChain

A best practice when developing chatbots is to save all the interactions the chatbot has with the user. This is because the state of the LLM can change depending on the past conversation, in fact, the LLM to the same question from 2 users will also answer differently because they have a different past conversation with the chatbot and therefore it is in a different state.

So what the chatbot memory creates is nothing more than a list of old messages, which are fed back to it before a new question is asked. Of course, the LLM have limited context, so you have to be a little creative and choose how to feed this history back to the LLM. The most common methods are to return a summary of the old messages or return only the N latest messages that are probably the most informative.

Start with the basics with ChatMessageHistory

This is the main class that allows us to manage the messages that occur between the chatbot (AI) and the user (Human). This class provides two main methods which are The following.

  • add_user_message: allows us to add a message into the chatbot’s memory and tag the message as “user”
  • add_ai_message: allows us to add a message into the chatbot’s memory and tag the message as “AI”
!pip install langchain
from langchain.memory import ChatMessageHistory

history = ChatMessageHistory()

history.add_user_message("Hi!")
history.add_ai_message("Hey, how can I help you today?")

#print messages
history.messages

This class allows you to do various things, but in its simplest use, you can look at it as saving various messages to a list from time to time. Then you can also review all the messages you have added simply by iterating over the history in the following way.

for message in history.messages:
print(message.content)

Advanced Memory with ConversationBufferMemory

The ConversationBufferMemory class behaves somewhat like the ChatMessageHistory class with respect to the message store, though it provides clever methods to retrieve old messages.
For example, we can retrieve old messages, as a list of messages or as one big string depending on what we need. If we want to ask the LLM to make a summary of the past conversation it might be useful to have the past conversation as one big string. If we want to do a detailed analysis of the past instead, we can read one message at a time by extracting a list.

Also with the ConversationBufferMemory class, we can add messages to the history using the add_user_message and add_user_message methods.
The load_memory_variables method on the other hand is used to extract old messages in list or dictionary form depending on what is specified, let’s see an example.

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory()

memory.chat_memory.add_user_message("Hi, how are you?")
memory.chat_memory.add_ai_message("hey,I am fine! How are you?")

memory_variables = memory.load_memory_variables({})
print(memory_variables['history'])

Manage Memory in Multiple Conversations

We have seen toy examples of how to manage memory by saving and retrieving messages. But in a real-world application you will probably need to manage the memory of several conversations. LangChain allows you to manage this case as well with the use of what are called chains.
A chain is nothing more than a workflow of various simple or complex steps that allow you to achieve a certain goal.

For example, an LLM that looks up a piece of information on Wikipedia because it does not know how to answer a certain question is a chain.
To handle various conversations, it is enough to associate a ConversationBufferMemory with each chain that is created with an instantiation of the ConversationChain class.
This way when the predict method of the model is called, all the steps of the chain are run so the model will read the past messages of the conversation.
Let’s look at a simple example.

from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory()
conversation = ConversationChain(llm= OpenAI(), memory=memory)

conversation.predict(input="Hey! how are you?")

In conclusion, memory is a critical component of a chatbot, and LangChain provides several frameworks and tools to manage memory effectively. Through the use of classes such as ChatMessageHistory and ConversationBufferMemory, you can capture and store user interactions with the AI, and use this information to guide future AI responses. I hope this information helps you build smarter and more capable chatbots!

In the next article, I will show you how to use LangChain tools.

If you found this article useful follow me here on Medium! 😉

Marcello Politi

Linkedin, Twitter, Website





Source link

Leave a Comment