如何向 RetrievalQA.from_chain_type 添加内存？或者，如何向 ConversationalRetrievalChain 添加自定义提示？

Question

如何向 RetrievalQA.from_chain_type 添加内存？或者，如何向 ConversationalRetrievalChain 添加自定义提示？

在过去的两周里，我一直在尝试制作一个可以通过文档聊天的聊天机器人（因此不仅仅是语义搜索/质量保证，因此需要记忆），而且还可以使用自定义提示。我已经尝试了所有链的每种组合，到目前为止，我得到的最接近的是 ConversationalRetrievalChain，但没有自定义提示，以及 RetrievalQA.from_chain_type 但没有内存

Answer 1

更新：这篇文章回答了OP问题的第一部分：

如何将内存添加到 RetrievalQA.from_chain_type？

对于第二部分，请参阅@andrew_reece的回答

或者，如何向 ConversationalRetrievalChain 添加自定义提示？

原来的：

您是否尝试过传入chain_type_kwargs（底部是源代码的屏幕截图以供快速参考）？

文档并没有让你很容易理解底层的内容，但这里有一些可以实现你的目标的东西。

您可以在此GitHub 链接 设置中找到该笔记本

from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.memory import ConversationBufferMemory
from langchain import PromptTemplate
from langchain.retrievers import TFIDFRetriever


retriever = TFIDFRetriever.from_texts(
    ["Our client, a gentleman named Jason, has a dog whose name is Dobby",
     "Jason has a good friend called Emma",
     "Emma has a cat whose name is Sullivan"])

然后定义您的自定义提示：

template = """
Use the following context (delimited by <ctx></ctx>) and the chat history (delimited by <hs></hs>) to answer the question:
------
<ctx>
{context}
</ctx>
------
<hs>
{history}
</hs>
------
{question}
Answer:
"""
prompt = PromptTemplate(
    input_variables=["history", "context", "question"],
    template=template,
)

记下您用于输入变量的内容，尤其是'history'和'question'，因为您在设置内存时需要匹配这些变量：

qa = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(),
    chain_type='stuff',
    retriever=retriever,
    verbose=True,
    chain_type_kwargs={
        "verbose": True,
        "prompt": prompt,
        "memory": ConversationBufferMemory(
            memory_key="history",
            input_key="question"),
    }
)

现在你可以打电话qa.run({"query": "who's the client's friend?"})

“客户的朋友是艾玛。”

进而qa.run("and her pet's name is?")

“艾玛的宠物叫沙利文。”

检查并验证内存/聊天记录：qa.combine_documents_chain.memory

ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content="客户的朋友是谁？",additional_kwargs={}), AIMessage(content="客户的朋友是艾玛。",additional_kwargs={}), HumanMessage(content= "她的宠物的名字是？",additional_kwargs={}), AIMessage(content="艾玛的宠物的名字是沙利文。",additional_kwargs={})]),output_key=None, input_key='question', return_messages=False, human_prefix='人类', ai_prefix='AI', memory_key='历史')

Answer 2

这是一个带有ConversationalRetrievalChain内存和自定义提示的解决方案，使用默认的'stuff'链类型。

这里有两个提示可以自定义。首先，是压缩对话历史记录加上当前用户输入的提示 ( condense_question_prompt)，其次是指示链如何向用户返回最终响应的提示（发生在combine_docs_chain）。

from langchain import PromptTemplate

# note that the input variables ('question', etc) are defaults, and can be changed

condense_prompt = PromptTemplate.from_template(
    ('Do X with user input ({question}), and do Y with chat history ({chat_history}).')
)

combine_docs_custom_prompt = PromptTemplate.from_template(
    ('Write a haiku about a dolphin.\n\n'
     'Completely ignore any context, such as {context}, or the question ({question}).')
)

ConversationalRetrievalChain现在我们可以使用自定义提示来初始化。

from langchain.llms import OpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

chain = ConversationalRetrievalChain.from_llm(
    OpenAI(temperature=0), 
    vectorstore.as_retriever(), # see below for vectorstore definition
    memory=memory,
    condense_question_prompt=condense_prompt,
    combine_docs_chain_kwargs=dict(prompt=combine_docs_custom_prompt)
)

请注意，这是_load_stuff_chain()在幕后调用的，它允许可选的promptkwarg（这是我们可以自定义的）。这用于设置LLMChain，然后初始化StuffDocumentsChain。

我们可以通过对矢量存储进行简单查询来测试设置（请参阅下面的矢量存储数据示例） - 您可以看到输出是如何完全由自定义提示确定的：

chain("What color is mentioned in the document about cats?")['answer']
#'\n\nDolphin leaps in sea\nGraceful and playful in blue\nJoyful in the waves'

并且内存工作正常：

chain.memory
#ConversationBufferMemory(chat_memory=ChatMessageHistory(messages=[HumanMessage(content='What color is mentioned in the document about cats?', additional_kwargs={}), AIMessage(content='\n\nDolphin leaps in sea\nGraceful and playful in blue\nJoyful in the waves', additional_kwargs={})]), output_key=None, input_key=None, return_messages=True, human_prefix='Human', ai_prefix='AI', memory_key='chat_history')

具有临时 ChromaDB 实例的向量存储数据集示例：

from langchain.vectorstores import Chroma
from langchain.document_loaders import DataFrameLoader
from langchain.embeddings.openai import OpenAIEmbeddings

data = {
    'index': ['001', '002', '003'], 
    'text': [
        'title: cat friend\ni like cats and the color blue.', 
        'title: dog friend\ni like dogs and the smell of rain.', 
        'title: bird friend\ni like birds and the feel of sunshine.'
    ]
}

df = pd.DataFrame(data)
loader = DataFrameLoader(df, page_content_column="text")
docs = loader.load()

embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(docs, embeddings)

Answer 3

#这就是函数

 def qasystem(query):

loader = TextLoader("details.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
documents = text_splitter.split_documents(documents)

vectordb = Chroma.from_documents(
    documents,
    embedding=OpenAIEmbeddings(),
    persist_directory='./data'
)
vectordb.persist()

_template = """Given the following conversation and a follow up question, rephrase the follow up question to be a 
standalone question without changing the content in given question.

Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:"""
condense_question_prompt_template = PromptTemplate.from_template(_template)

prompt_template = """You are helpful information giving QA System and make sure you don't answer anything 
not related to following context. You are always provide useful information & details available in the given context. Use the following pieces of context to answer the question at the end. 
If you don't know the answer, just say that you don't know, don't try to make up an answer. 

{context}

Question: {question}
Helpful Answer:"""

qa_prompt = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
llm = ChatOpenAI(temperature=0.1)
question_generator = LLMChain(llm=llm, prompt=condense_question_prompt_template, memory=memory)
doc_chain = load_qa_chain(llm, chain_type="stuff", prompt=qa_prompt)
qa_chain = ConversationalRetrievalChain(
    retriever=vectordb.as_retriever(search_kwargs={'k': 6}),
    question_generator=question_generator,
    combine_docs_chain=doc_chain,
    memory=memory,

)

chat_history = []
while True:
    result = qa_chain({'question': question, 'chat_history': chat_history})

    response = result['answer']
    chat_history.append((question, response))
    return result['answer'