为 RAG 的 Llama 2 添加自定义提示

Question

我已经在本地下载了 Llama 2 并且可以运行。现在我想调整我的提示/更改默认提示以强制 Llama 2 使用不同的语言（例如德语）进行回答。这是我的代码：

from langchain.llms import LlamaCpp
from langchain.chains import LLMChain
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.prompts import PromptTemplate
from langchain.document_loaders import PyPDFLoader
from langchain.vectorstores import Chroma
# embeddings are numerical representations of the question and answer text
from langchain.embeddings import HuggingFaceEmbeddings

# use a common text splitter to split text into chunks
from langchain.text_splitter import RecursiveCharacterTextSplitter

# for token-wise streaming so you'll see the answer gets generated token by token when Llama is answering your question
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

llm = LlamaCpp(
    model_path="MYPATH/llama.cpp/models/7B/ggml-model-q4_0.bin",
    temperature=0.0,
    top_p=1,
    n_ctx=6000,
    callback_manager=callback_manager, 
    verbose=True
)

# Load Pdf-File
loader = PyPDFLoader("myfile.pdf")
documents = loader.load()

# split the loaded documents into chunks 
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=20)
all_splits = text_splitter.split_documents(documents)

# create the vector db to store all the split chunks as embeddings
embeddings = HuggingFaceEmbeddings()
vectordb = Chroma.from_documents(
    documents=all_splits,
    embedding=embeddings,
)

# use another LangChain's chain, RetrievalQA, to associate Llama with the loaded documents stored in the vector db
from langchain.chains import RetrievalQA

qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever()
)

question = "What is the biggest city in Germany?"
result = qa_chain({"query": question})

我必须在哪一部分插入自己的提示？据我正确理解，现在正在使用默认的 Llama 2 提示符。我尝试在以下部分插入提示，但模型一直用英语回答：

qa_chain = RetrievalQA.from_chain_type(
    llm(prompt="Please answer only in the German language!"),
    retriever=vectordb.as_retriever()
)

我看到Llama 2的提示模板如下：

<s>[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.  Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
<</SYS>>

There's a llama in my garden 😱 What should I do? [/INST]

提前致谢！

Answer 1

我正在使用 llama2 的 70b 微调版本，该版本是使用英语数据进行微调的。我们发现，为了以所需的语言获得标准答案，最好以该语言进行提示。此代码还应该帮助您了解可以在哪里放置自定义提示模板：

from langchain.prompts import PromptTemplate

template = """Verwenden die folgenden Kontextinformationen, um die Frage am Ende zu beantworten. Wenn du die Antwort nicht kennst, sag einfach, dass du es nicht weisst. versuche nicht eine Antwort zu erfinden. Verwende maximal drei Sätze und halte die Antwort so kurz wie möglich.

Kontext:
{context}
Frage: {question}
Hilfreiche Antwort:"""

QA_CHAIN_PROMPT = PromptTemplate(input_variables=["context", "question"], template=template)

# use another LangChain's chain, RetrievalQA, to associate Llama with the loaded documents stored in the vector db
from langchain.chains import RetrievalQA

qa_chain = RetrievalQA.from_chain_type(llm,
                                chain_type='stuff',
                                retriever=vectordb.as_retriever(),
                                chain_type_kwargs={"prompt": QA_CHAIN_PROMPT})

question = "Welches ist die grösste Stadt in Deutschland?"
print(qa_chain.run(query))

我从这里获取了一些代码并将其改编为您的代码： PromptTemplate 如何与 RetrievalQA 交互？

希望有帮助！