In this article, I will introduce LangChain and explore its capabilities by building a simple question-answering app querying a pdf that is part of Azure Functions Documentation.
Langchain
Harrison Chase's LangChain is a powerful Python library that simplifies the process of building NLP applications using large language models. Its primary goal is to create intelligent agents that can understand and execute human language instructions. With LangChain, you can connect to a variety of data and computation sources and build applications that perform NLP tasks on domain-specific data sources, private repositories, and more.
As of May 2023, the LangChain GitHub repository has garnered over 42,000 stars and has received contributions from more than 270 developers worldwide.
The langchain library is comprised of different modules:
- LLMs and Prompts
This includes prompt management, prompt optimization, a generic interface for all LLMs, and common utilities for working with LLMs like Azure OpenAI. It supports a variety of LLMs, including OpenAI, LLama, and GPT4All.
- Chains
Chains in LangChain involve sequences of calls that can be chained together to perform specific tasks. For instance, you may need to retrieve data from a particular URL, summarize the returned text, and answer questions using the generated summary. Chains can also be simple, such as reading user input, constructing a prompt, and generating a response.
- Data Augmented Generation
Data Augmented Generation involves specific types of chains that first interact with an external data source to fetch data for use in the generation step. Examples include summarization of long pieces of text and question/answering over specific data sources. LangChain’sDocument LoadersandUtilsmodules facilitate connecting to sources of data and computation. If you have a mix of text files, PDF documents, HTML web pages, etc, you can use the document loaders in Langchain.
- Agents
Agents involve an LLM making decisions about which actions to take, taking that action, seeing an observation, and repeating that until done.
As we explained before, chains can help chain together a sequence of LLM calls. In some tasks, however, the sequence of calls is often not deterministic and the next step will depend on the user input and the response in the previous steps.
“Agents” can take actions based on inputs along the way instead of a hardcoded deterministic sequence.
- Memory
Memory refers to persisting state using VectorStores. Vector databases are optimized for doing quick searches in high dimensional spaces. LangChain makes this effortless.
Embeddings
An embedding is a mapping of a discrete, categorical variable to a vector of continuous numbers.In the context of neural networks, embeddingsarelow-dimensional,learnedcontinuous vector representations of discrete variables. Neural network embeddings are useful because they canreduce the dimensionalityof categorical variables andmeaningfully representcategories in the transformed space.
Neural network embeddings have 3 primary purposes:
- Finding nearest neighbours in the embedding space. These can be used to make recommendations based on user interests or cluster categories.
- As input to a machine learning model for a supervised task.
- For visualization of concepts and relations between categories.
Example of clustering of vector values for sentences
Vector Stores or Vector Databases
A vector database is a specialized type of database that stores data as high-dimensional vectors. These vectors are mathematical representations of the features or attributes of the data being stored. The number of dimensions in each vector can vary widely, ranging from tens to thousands, depending on the complexity and granularity of the data. In this article, we will explore the concept of vector databases and their applications in various fields.
Let’s build the Application
Let’s build a tool that can read developers documentation – in this case Azure Functions Documentation as PDF.
Then answer arbitrary questions by referencing the documentation text.
We will follow these steps:
One time procedure:
- Index the pdf document (azure functions documentation), split the document into chunks, indexing all of the text creating embeddings.
- Store all of the embeddings in a vector store (Faiss in our case) which can be searched in the application.
The application:
- When a user asks a question, we will use the FAISS vector index to find the closest matching text.
- Feed that into GPT-3.5 as context in the prompt
- GPT-3.5 will generate an answer that accurately answers the question.
Steps
- Download the Documents to search. In our case we can download Azure functions documentation from here and save it in data/documentation folder.
- In Azure OpenAI deploy
- Ada
- Gpt35
Get Azure OpenAI endpoint and key and add it to a file called .env as follows:
OPENAI_DEPLOYMENT_ENDPOINT = "https://<your openai>.openai.azure.com/"OPENAI_API_KEY = "<your openai api key>"OPENAI_DEPLOYMENT_NAME = "<your gpt35 deployment name>"OPENAI_DEPLOYMENT_VERSION = "2023-03-15-preview"OPENAI_MODEL_NAME="gpt-35-turbo"OPENAI_EMBEDDING_DEPLOYMENT_NAME = "<your ada deployment name>"OPENAI_EMBEDDING_MODEL_NAME = "text-embedding-ada-002"Creating the embeddings
The flow of indexer.py is:
- Load the PDF
- Split up all of the text into chunks.
- Send those chunks to the OpenAI Embeddings API, which returns a 1536 dimensional vector for each chunk.
- Index all of the vectors into a FAISS index.
- Save the FAISS index to a.faiss and .pklfile.
Note: As you probably know, LLMs cannot accept long instructions since there is a token limitation, so we will be splitting the document into chunks, see below.
Running this code takes time since we need to read and split the whole document and send the chunks to Ada model to get the embeddings.
Here is the code for indexer.py
from langchain.document_loaders import PyPDFLoaderfrom langchain.embeddings.openai import OpenAIEmbeddingsfrom langchain.vectorstores import FAISSfrom dotenv import load_dotenvimport openaiimport os#load environment variablesload_dotenv()OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")OPENAI_MODEL_NAME = os.getenv("OPENAI_MODEL_NAME")OPENAI_EMBEDDING_DEPLOYMENT_NAME = os.getenv("OPENAI_EMBEDDING_DEPLOYMENT_NAME")OPENAI_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_EMBEDDING_MODEL_NAME")OPENAI_DEPLOYMENT_VERSION = os.getenv("OPENAI_DEPLOYMENT_VERSION")#init Azure OpenAIopenai.api_type = "azure"openai.api_version = OPENAI_DEPLOYMENT_VERSIONopenai.api_base = OPENAI_DEPLOYMENT_ENDPOINTopenai.api_key = OPENAI_API_KEYif __name__ == "__main__": embeddings = OpenAIEmbeddings(model=OPENAI_EMBEDDING_MODEL_NAME, chunk_size=1) dataPath = "./data/documentation/" fileName = dataPath + "azure-azure-functions.pdf" #use langchain PDF loader loader = PyPDFLoader(fileName) #split the document into chunks pages = loader.load_and_split() #Use Langchain to create the embeddings using text-embedding-ada-002 db = FAISS.from_documents(documents=pages, embedding=embeddings) #save the embeddings into FAISS vector store db.save_local("./dbs/documentation/faiss_index")Creating the Application
The flow of app.py works something like:
- FAISS index is loaded into RAM
- User asks a question
- User's question is sent to the OpenAI Embeddings API, which returns a 1536 dimensional vector.
- The FAISS index is queried for the closest matching vector.
- The closest matching vector is returned, along with the text that it was generated from.
- The returned text is fed into GPT-35 as context in a GPT-35 prompt
- GPT-35 generates a response, which is returned to the user.
Note: What is important to note here is that Langchain does most of the heavy lifting for us and this happens behind the scenes.
Here is the code for app.py
from langchain.vectorstores import FAISSfrom dotenv import load_dotenvimport openaiimport osfrom langchain.chains import RetrievalQAfrom langchain.vectorstores import FAISSfrom langchain.chains.question_answering import load_qa_chainfrom dotenv import load_dotenvfrom langchain.chat_models import AzureChatOpenAIfrom langchain.embeddings.openai import OpenAIEmbeddings#load environment variablesload_dotenv()OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")OPENAI_MODEL_NAME = os.getenv("OPENAI_MODEL_NAME")OPENAI_EMBEDDING_DEPLOYMENT_NAME = os.getenv("OPENAI_EMBEDDING_DEPLOYMENT_NAME")OPENAI_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_EMBEDDING_MODEL_NAME")OPENAI_DEPLOYMENT_VERSION = os.getenv("OPENAI_DEPLOYMENT_VERSION")#init Azure OpenAIopenai.api_type = "azure"openai.api_version = OPENAI_DEPLOYMENT_VERSIONopenai.api_base = OPENAI_DEPLOYMENT_ENDPOINTopenai.api_key = OPENAI_API_KEYdef ask_question(qa, question): result = qa({"query": question}) print("Question:", question) print("Answer:", result["result"])if __name__ == "__main__": #init openai llm = AzureChatOpenAI(deployment_name=OPENAI_DEPLOYMENT_NAME, model_name=OPENAI_MODEL_NAME, openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT, openai_api_version=OPENAI_DEPLOYMENT_VERSION, openai_api_key=OPENAI_API_KEY) embeddings = OpenAIEmbeddings(model=OPENAI_EMBEDDING_MODEL_NAME, chunk_size=1) #load the faiss vector store we saved into memory vectorStore = FAISS.load_local("./dbs/documentation/faiss_index", embeddings) #use the faiss vector store we saved to search the local document retriever = vectorStore.as_retriever(search_type="similarity", search_kwargs={"k":2}) #use the vector store as a retriever qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=False) while True: query = input('you: ') if query == 'q': break ask_question(qa, query)Now we can run app.py and start asking questions:
you: what are azure functions?Question: what are azure functions?Answer: Azure Functions is a cloud service available on-demand that provides all the continually updated infrastructure and resources needed to run your applications. You focus on the code that matters most to you, in the most productive language for you, and Functions handles the rest. Functions provides serverless compute for Azure. You can use Functions to build web APIs, respond to database changes, process IoT streams, manage message queues, and more.you: can I use events hub as a trigger for an azure function?Question: can I use events hub as a trigger for an azure function?Answer: Yes, you can use events hub as a trigger for an Azure Function.`Azure Functions supports trigger and output bindings for Event Hubs. Use the function trigger to respond to an event sent to an event hub event stream. You must have read access to the underlying event hub to set up the trigger. When the function is triggered, the message passed to the function is typed as a string.`<|im_end|>you: what languages I can use to build azure functions?Question: what languages I can use to build azure functions?you: can I deploy azure functions in multi-region?Question: can I deploy azure functions in multi-region?Answer: Yes, you can deploy Azure Functions in multi-region. There are two patterns to consider: Active/Active which is used for HTTP trigger functions and Active/Passive which is used for event-driven, non-HTTP triggered functions. Azure Front Door needs to be used to coordinate requests between both regions when using the active/active pattern for HTTP trigger functions. When using the active/passive pattern, the second region is activated when failover is required and takes over processing. To learn more about multi-region deployments, see the guidance in Highly available multi-region web application.<|im_end|>Thanks for reading, hope you enjoyed it.
Denise
