When using rag and memory, multiple identical copies of the same information is sent to the ai, when asking related questions.
I have
import java.util.ArrayList;
import java.util.List;
import dev.langchain4j.data.message.ChatMessage;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.model.openai.OpenAiChatModelName;
import dev.langchain4j.rag.content.Content;
import dev.langchain4j.rag.content.retriever.ContentRetriever;
import dev.langchain4j.rag.query.Query;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.MemoryId;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.memory.ChatMemoryAccess;
interface ChatInterface extends ChatMemoryAccess {
String chat(@memoryid int memoryId, @Usermessage String userMessage);
}
public class RagHistoryBug {
public static void main(String args[]) throws Exception {
String apiKey="Can't provide my key, but I am sure this bug also shows up with local run ai";
OpenAiChatModel chatModel=OpenAiChatModel.builder()
.apiKey(apiKey)
.modelName(OpenAiChatModelName.GPT_4_O_MINI)
.build();
AiServices<ChatInterface> builder=AiServices.builder(ChatInterface.class)
.chatModel(chatModel)
.chatMemoryProvider(memoryId -> MessageWindowChatMemory.withMaxMessages(11));
ContentRetriever contentRetriever=new ContentRetriever() {
@Override public List<Content> retrieve(Query q) {
List<Content> list = new ArrayList<>();
list.add(Content.from("I have a cat called Pulasy. It likes mice very much"));
return list;
}
};
builder=builder.contentRetriever(contentRetriever);
ChatInterface assistant=builder.build();
System.out.println(assistant.chat(1,"What is the name of my cat?"));
System.out.println(assistant.chat(1,"Do my cat like mice?"));
System.out.println(assistant.chat(1,"Is Pulasy a lion?"));
ChatMemory mem = assistant.getChatMemory(1);
List<ChatMessage> messages = mem.messages();
for(ChatMessage msg: messages) {
System.out.println("Msg:" + msg);
}
}
}
If I run this program, the output is:
Your cat's name is Pulasy.
Yes, your cat Pulasy likes mice very much.
No, Pulasy is not a lion; Pulasy is a cat.
Msg:UserMessage { name = null contents = [TextContent { text = "What is the name of my cat?
Answer using the following information:
I have a cat called Pulasy. It likes mice very much" }] }
Msg:AiMessage { text = "Your cat's name is Pulasy." toolExecutionRequests = [] }
Msg:UserMessage { name = null contents = [TextContent { text = "Do my cat like mice?
Answer using the following information:
I have a cat called Pulasy. It likes mice very much" }] }
Msg:AiMessage { text = "Yes, your cat Pulasy likes mice very much." toolExecutionRequests = [] }
Msg:UserMessage { name = null contents = [TextContent { text = "Is Pulasy a lion?
Answer using the following information:
I have a cat called Pulasy. It likes mice very much" }] }
Msg:AiMessage { text = "No, Pulasy is not a lion; Pulasy is a cat." toolExecutionRequests = [] }
Note that:
Answer using the following information:
I have a cat called Pulasy. It likes mice very much" }] }
Msg:AiMessage { text = "No, Pulasy is not a lion; Pulasy is a cat." toolExecutionRequests = [] }
is included 3 times. (And I checked the used input tokens. It really is included 3 times. How do I prevent that?
CodeRetrieverreturns the result. Use aHashSet<TheResultClass>. If the process is more abstract (like a WebServer), use aWeakHashMap<UserInfos, TheResultClass>. (UserInfosis a class with data regarding the user, the current process, result etc, to identify the exact case where you want to prevent duplicates. Make sureequals()andhashCode()exist). With this, register answers and identify duplicates.