Monday, 13 May, 2024 UTC
Summary
The createStuffDocumentsChain is one of the chains we can use in the Retrieval Augmented Generation (RAG) process.
RAG is the process of optimizing the output of a Large Language Model, by providing an external knowledge base outside of its training data sources.
For example, ChatGPT 3.5 has its knowledge cutoff date of January 2022. It does not know about new stuff after this date. How can it provide us with valid data about LangChain, which was released in October 2022?
This is where the RAG process steps in, and the
createStuffDocumentsChain() is one tool from this process.The
A Javascript example for createStuffDocumentsChain
createStuffDocumentsChain() takes a list of documents and formats them all into a prompt. It then passes that prompt to the LLM. We will build a simple Javascript example to illustrate how the createStuffDocuments chain works.
We will use the simple
Document class from langchain/chains/combine_documents to teach ChatGPT 3.5 a few basic stuff about LangChain.One of the easiest ways to expand the context of an LLM with LangChain is by using local documents.
We can create small pieces of information and feed them to the LLM:
//🟠manually create static documents
const documentA = new Document({
pageContent:
`LangChain is a framework that was designed to simplify
the making of applications that integrate Large Language Models.
It does this in a modular, flexible and scalable architecture,
easy to maintain. Langchain has Javascript and Python implmenations.`
})
const documentB = new Document({
pageContent: `LangChain was first launched in October 2022.`
})Below is the full code that uses these documents. We will use
createStuffDocumentsChain() to expand the context of our LLM:import { ChatOpenAI } from "@langchain/openai"
import { ChatPromptTemplate } from "@langchain/core/prompts"
//🟠we need the this one to create local documents
import { Document } from "@langchain/core/documents"
import { createStuffDocumentsChain } from "langchain/chains/combine_documents"
const model = new ChatOpenAI({
openAIApiKey: process.env.OPENAI_API_KEY,
//🟠zero temperature, no extra creativity
temperature: 0
})
//🟠adding the extra context to the prompt
const prompt = ChatPromptTemplate.fromTemplate(
`Answer the user's question from the following context:
{context}
Question: {input}`
)
//🟠manually create static documents
const documentA = new Document({
pageContent:
`LangChain is a framework that was designed to simplify
the making of applications that integrate Large Language Models.
It does this in a modular, flexible and scalable architecture,
easy to maintain. Langchain has Javascript and Python implmenations.`
})
const documentB = new Document({
pageContent: `LangChain was first launched in October 2022.`
})
const chain = await createStuffDocumentsChain({
llm: model,
prompt
})
const question = `When was LangChain released and
in what languages can I use it?`
//🟠pass the documents array as the context
const answer = await chain.invoke({
input: question,
context: [documentA, documentB]
})
console.log(answer)The above example will output this answer:
LangChain was released in October 2022. You can use LangChain
in both Javascript and Python languages.In this example, we have used just some simple text pieces, but we can also have locally stored PDF, CSV, and other types of files. Or even video and audio. These files can contain a larger quantity of information.
Some considerations about createStuffDocumentsChain
The
createStuffDocumentsChain() passes ALL documents to the LLM, so you should make sure it fits within the context length of the LLM you are using.Context length is like the maximum number of words a model can handle at once. Think of it as the model's memory or attention span. This is a set feature in transformer-based models such as ChatGPT and Llama.
Below is the default context length of some popular LLMs:
Also, note that expanding the context of the LLM with these documents does not retrain the model. The weights don't change in this process.