# Create a YouTube Video Summarizer Using Whisper and LangChain
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, chain_type_kwargs=chain_type_kwargs) print( qa.run("Summarize the mentions of google according to their AI program") ) ``` ``` • Google has developed an AI program to help people with their everyday tasks. • The AI program can be used to search for information, make recommendations, and provide personalized experiences. • Google is using AI to improve its products and services, such as Google Maps and Google Assistant. • Google is also using AI to help with medical research and to develop new technologies. ``` Of course, you can always tweak the prompt to get the desired result, experiment more with modified prompts using different types of chains and find the most suitable combination. Ultimately, the choice of strategy depends on the specific needs and constraints of your project. ### Conclusion When working with large documents and language models, it is essential to choose the right approach to effectively utilize the information available. We have discussed three main strategies: "stuff," "map-reduce," and "refine." The "stuff" approach is the simplest and most naive one, in which all the text from the documents is used in a single prompt. This method may raise exceptions if all text is longer than the available context size of the LLM and may not be the most efficient way to handle large amounts of text. On the other hand, the "map-reduce" and "refine" approaches offer more sophisticated ways to process and extract useful information from longer documents. While the "map-reduce" method can be parallelized, resulting in faster processing times, the "refine" approach is empirically known to produce better results. However, it is sequential in nature, making it slower compared to the "map-reduce" method. By considering the trade-offs between speed and quality, you can select the most suitable approach to leverage the power of LLMs for your tasks effectively. Throughout this lesson, we have demonstrated a powerful and efficient solution for summarizing YouTube videos using Whisper and LangChain. By downloading YouTube audio files, transcribing them with Whisper, and leveraging LangChain's advanced summarization techniques (stuff, refine, and map_reduce), you can extract the most valuable information from your chosen content with ease. Additionally, we showcased the customizability of LangChain, which allows you to create personalized prompts, generate summaries in different languages, and even store URLs in a Deep Lake vector store for quick retrieval. This powerful feature set enables you to access and process a wealth of information more efficiently. Using the summarizing chain, you can swiftly retrieve the information stored in the vector store, condensing it into easily digestible summaries. By implementing these cutting-edge tools, you can save time and effort while enhancing your knowledge retention and understanding of a wide range of topics. We hope you enjoy this advanced, tailored experience in data storage and summarization. Happy summarizing! In the next lesson, we’ll create a voice assistant for a knowledge base of documentation articles. **THE CODE:** [langchain/yt-whisper-sum.ipynb at main · idontcalculate/langchain](https://github.com/idontcalculate/langchain/blob/main/yt-whisper-sum.ipynb) **RESOURCES:** [Summarization | 🦜🔗 Langchain](https://python.langchain.com/docs/use_cases/summarization) [Introducing Whisper](https://openai.com/research/whisper) [Deep Lake Vector Store in LangChain](https://docs.activeloop.ai/tutorials/vector-store/deep-lake-vector-store-in-langchain) You can find the code of this lesson in this online [Notebook](https://colab.research.google.com/drive/1MvFJpyMStxDlq31wRshWB07nykxoME3-?usp=sharing).


# Exploring The Role of LangChain's Indexes and Retrievers
### **I**ntroduction **** In LangChain, indexes and retrievers play a crucial role in structuring documents and fetching relevant data for LLMs. We will explore some of the advantages and disadvantages of using document based LLMs (i.e., LLMs that leverage relevant pieces of documents inside their prompts), with a particular focus on the role of indexes and retrievers. An `index` is a powerful data structure that meticulously organizes and stores documents to enable efficient searching, while a `retriever` harnesses the index to locate and return pertinent documents in response to user queries. Within LangChain, the primary index types are centered on vector databases, with embeddings-based indexes being the most prevalent. Retrievers focus on extracting relevant documents to merge with prompts for language models. A retriever exposes a `get_relevant_documents` method, which accepts a query string as input and returns a list of related documents. Here we use the TextLoader class to load a text file. Remember to install the required packages with the following command: `pip install langchain==0.0.208 deeplake openai tiktoken`. ```python from langchain.document_loaders import TextLoader # text to write to a local file # taken from https://www.theverge.com/2023/3/14/23639313/google-ai-language-model-palm-api-challenge-openai text = """Google opens up its AI language model PaLM to challenge OpenAI and GPT-3 Google is offering developers access to one of its most advanced AI language models: PaLM. The search giant is launching an API for PaLM alongside a number of AI enterprise tools it says will help businesses “generate text, images, code, videos, audio, and more from simple natural language prompts.” PaLM is a large language model, or LLM, similar to the GPT series created by OpenAI or Meta’s LLaMA family of models. Google first announced PaLM in April 2022. Like other LLMs, PaLM is a flexible system that can potentially carry out all sorts of text generation and editing tasks. You could train PaLM to be a conversational chatbot like ChatGPT, for example, or you could use it for tasks like summarizing text or even writing code. (It’s similar to features Google also announced today for its Workspace apps like Google Docs and Gmail.) """ # write text to local file with open("my_file.txt", "w") as file: file.write(text) # use TextLoader to load text from local file loader = TextLoader("my_file.txt") docs_from_file = loader.load() print(len(docs_from_file)) # 1 ``` Then, we use CharacterTextSplitter to split the docs into texts. ```python from langchain.text_splitter import CharacterTextSplitter # create a text splitter text_splitter = CharacterTextSplitter(chunk_size=200, chunk_overlap=20) # split documents into chunks docs = text_splitter.split_documents(docs_from_file) print(len(docs)) # 2 ``` These embeddings allow us to effectively search for documents or portions of documents that relate to our query by examining their semantic similarities. ```python from langchain.embeddings import OpenAIEmbeddings # Before executing the following code, make sure to have # your OpenAI key saved in the “OPENAI_API_KEY” environment variable. embeddings = OpenAIEmbeddings(model="text-embedding-ada-002") ``` We'll employ the Deep Lake vector store with our embeddings in place. Deep Lake provides several advantages over the typical vector store: - It’s **multimodal**, which means that it can be used to store items of diverse modalities, such as texts, images, audio, and video, along with their vector representations. - It’s **serverless**, which means that we can create and manage cloud datasets without the need to create and managing a database instance. This aspect gives a great speedup to new projects. - It’s possible to easily create a streaming **data loader** out of the data loaded into a Deep Lake dataset, which is convenient for fine-tuning machine learning models using common frameworks like PyTorch and TensorFlow. - Data can be queried and visualized easily from the web. Thanks to its nature, Deep Lake is well


# Exploring The Role of LangChain's Indexes and Retrievers
suited for being the serverless memory that LLM chains and agents need for several tasks, like storing relevant documents for question-answering or storing images to control some guided image-generation tasks. Here’s a diagram that visually summarizes this aspect. ![Screenshot 2023-06-01 at 17.28.06.png](Exploring%20The%20Role%20of%20LangChain's%20Indexes%20and%20Retr%20c11a449aca9443c7b9b62deb6d2a05ef/Screenshot_2023-06-01_at_17.28.06.png) Let’s create an instance of a Deep Lake dataset. ```python from langchain.vectorstores import DeepLake # Before executing the following code, make sure to have your # Activeloop key saved in the “ACTIVELOOP_TOKEN” environment variable. # create Deep Lake dataset # TODO: use your organization id here. (by default, org id is your username) my_activeloop_org_id = "<YOUR-ACTIVELOOP-ORG-ID>" my_activeloop_dataset_name = "langchain_course_indexers_retrievers" dataset_path = f"hub://{my_activeloop_org_id}/{my_activeloop_dataset_name}" db = DeepLake(dataset_path=dataset_path, embedding_function=embeddings) # add documents to our Deep Lake dataset db.add_documents(docs) ``` In this example, we are adding text documents to the dataset. However, being Deep Lake multimodal, we could have also added images to it, specifying an image embedder model. This could be useful for searching images according to a text query or using an image as a query (and thus looking for similar images). As datasets become bigger, storing them in local memory becomes less manageable. In this example, we could have also used a local vector store, as we are uploading only two documents. However, in a typical production scenario, thousands or millions of documents could be used and accessed from different programs, thus having the need for a centralized cloud dataset. Back to the code example of this lesson. Next, we create a retriever. ```python # create retriever from db retriever = db.as_retriever() ``` Once we have the retriever, we can start with question-answering. ```python from langchain.chains import RetrievalQA from langchain.llms import OpenAI # create a retrieval chain qa_chain = RetrievalQA.from_chain_type( llm=OpenAI(model="text-davinci-003"), chain_type="stuff", retriever=retriever ) ``` We can query our document that is an about specific topic that can be found in the documents. ```python query = "How Google plans to challenge OpenAI?" response = qa_chain.run(query) print(response) ``` You should see something like the following. ``` Google plans to challenge OpenAI by offering access to its AI language model PaLM, which is similar to OpenAI's GPT series and Meta's LLaMA family of models. PaLM is a large language model that can be used for tasks like summarizing text or writing code. ``` ### What occurred behind the scenes? Initially, we employed a so-called "stuff chain" (refer to CombineDocuments Chains). Stuffing is one way to supply information to the LLM. Using this technique, we "stuff" all the information into the LLM's prompt. However, this method is only effective with shorter documents, as most LLMs have a context length limit. Additionally, a similarity search is conducted using the embeddings to identify matching documents to be used as context for the LLM. Although it might not seem particularly useful with just one document, we are effectively working with multiple documents since we "chunked" our text. Preselecting the most suitable documents based on semantic similarity enables us to provide the model with meaningful knowledge through the prompt while remaining within the allowed context size. So, in this exploration, we have discovered the significant role that indexes and retrievers play in improving the performance of Large Language Models when handling document-based data. The system becomes more efficient in finding and presenting relevant information by converting documents and user queries into numerical vectors (embeddings) and storing them in specialized databases like Deep Lake, which serves as our vector store database. The retriever's ability to identify documents that are closely related to a user's query in the embedding space demonstrates the effectiveness of this approach in enhancing the overall language understanding capabilities of LLMs. ### A Potential


# Exploring The Role of LangChain's Indexes and Retrievers
Problem This method has a downside: you might not know how to get the right documents later when storing data. In the Q&A example, we cut the text into equal parts, causing both useful and useless text to show up when a user asks a question. Including unrelated information in the LLM prompt is detrimental because: 1. It can divert the LLM's focus from pertinent details. 2. It occupies valuable space that could be utilized for more relevant information. ### Possible Solution A `DocumentCompressor` abstraction has been introduced to address this issue, allowing compress_documents on the retrieved documents. The `ContextualCompressionRetriever` is a wrapper around another retriever in LangChain. It takes a base retriever and a `DocumentCompressor` and automatically compresses the retrieved documents from the base retriever. This means that only the most relevant parts of the retrieved documents are returned, given a specific query. A popular compressor choice is the `LLMChainExtractor`, which uses an LLMChain to extract only the statements relevant to the query from the documents. To improve the retrieval process, a ContextualCompressionRetriever is used, wrapping the base retriever with an LLMChainExtractor. The LLMChainExtractor iterates over the initially returned documents and extracts only the content relevant to the query. Here's an example of how to use `ContextualCompressionRetriever` with `LLMChainExtractor`: ```python from langchain.retrievers import ContextualCompressionRetriever from langchain.retrievers.document_compressors import LLMChainExtractor # create GPT3 wrapper llm = OpenAI(model="text-davinci-003", temperature=0) # create compressor for the retriever compressor = LLMChainExtractor.from_llm(llm) compression_retriever = ContextualCompressionRetriever( base_compressor=compressor, base_retriever=retriever ) ``` Once we have created the `compression_retriever`, we can use it to retrieve the compressed relevant documents to a query. ```python # retrieving compressed documents retrieved_docs = compression_retriever.get_relevant_documents( "How Google plans to challenge OpenAI?" ) print(retrieved_docs[0].page_content) ``` You should see an output like the following. ```python Google is offering developers access to one of its most advanced AI language models: PaLM. The search giant is launching an API for PaLM alongside a number of AI enterprise tools it says will help businesses “generate text, images, code, videos, audio, and more from simple natural language prompts.” ``` Compressors aim to make it easy to pass **only** the relevant information to the LLM. Doing this also enables you to pass along **more** information to the LLM since in the initial retrieval step, you can focus on recall (e.g., by increasing the number of documents returned) and let the compressors handle precision: [Contextual compression | 🦜🔗 Langchain](https://python.langchain.com/docs/modules/data_connection/retrievers/contextual_compression/) ### Conclusion In summary, LangChain's indexes and retrievers offer modular, flexible, and customizable solutions for working with unstructured data and language models. However, they have limited support for structured data and are mainly focused on vector databases. In the next lesson, we’ll see some convenient LangChain classes for loading data from different sources, that is data loaders. **RESOURCES:** [Improving Document Retrieval with Contextual Compression](https://blog.langchain.dev/improving-document-retrieval-with-contextual-compression/)


# Building Autonomous Agents to Create Analysis Reports
### Introduction In this lesson, our aim is to create an autonomous agent using the LangChain framework. We will explore the concept of "Plan and Execute" LangChain agents and their ability to generate insightful analysis reports based on retrieved documents from Deep Lake. We will start by understanding the fundamentals of the "Plan and Execute" LangChain agent framework and its benefits for complex long-term planning. Then, we will delve into our project's implementation details and workflow. By the end of the lesson, you will have a solid understanding of building autonomous agents using the LangChain framework and be equipped with the skills to create analysis reports using them. ### Workflow This is the workflow we’ll follow in this project: 1. **Saving Documents on Deep Lake**: We will begin by learning how to save documents on Deep Lake, which serves as our knowledge repository. Deep Lake provides information that our agents can leverage for analysis and report generation. 2. **Creating a Document Retrieval Tool**: Next, we will develop a tool that enables our agent to retrieve the most relevant documents from Deep Lake based on a given query. 3. **Using the Plan and Execute Agent**: The core of our project involves employing a "Plan and Execute" agent to devise a plan for answering a specific query about creating an overview of a topic. Our specific objective is to generate a comprehensive outline of recent events related to Artificial Intelligence regulations by governments, but the final agent could also work for other similar objectives as well. To accomplish this, we will feed the query into the **planner component** of the agent, which will utilize a language model's reasoning ability to plan out the steps required. The planner will consider various factors, including the complexity of the query and instructions for the tool used to generate a step-by-step plan or lower-level queries. The plan will then be passed to the **executor component**, which will determine the appropriate tools or actions required to execute each step of the plan. The executor, initially implemented as an Action Agent, will make use of the tools we developed earlier, such as the document retrieval tool, to gather relevant information and execute the plan. By employing the "Plan and Execute" agent framework, we can achieve more accurate and reliable analysis reports while handling complex long-term planning scenarios. So let's dive in and explore the potential for generating insightful analysis reports! ### Plan and Execute **Plan and Execute** agents are a new type of agent executor offering a different approach than the traditional agents supported in LangChain. These agents are heavily inspired by the BabyAGI framework and the recent Plan-and-Solve paper. The primary goal of "Plan and Execute" agents is to enable more complex long-term planning, even at the cost of making more calls to the language model. - The **planner** in the "Plan-and-Execute" framework typically utilizes a language model's reasoning ability to plan out steps and handle ambiguity and edge cases. - The **executor**, initially an Action Agent, takes the planner's high-level objectives (steps) and determines the tools or actions required to accomplish each step. This separation of planning and execution allows for improved reliability and flexibility. It also facilitates the possibility of replacing these components with smaller, fine-tuned models in the future. We will explore the implementation of the "Plan and Execute" agent and how to integrate it with Deep Lake for document retrieval and see the agent in action as it generates an analysis report based on the given query. ### Implementation Let’s set up the OpenAI API and Activeloop keys in environment variables. ```python import


# Building Autonomous Agents to Create Analysis Reports
os os.environ["OPENAI_API_KEY"] = "<YOUR-OPENAI-API-KEY>" os.environ["ACTIVELOOP_TOKEN"] = "<YOUR-ACTIVELOOP-TOKEN>" ``` <aside> 💡 Langchain has recently (as of August 2023) moved some classes from "langchain.experimental" to another library called "library_experimental", in an attempt to make the "langchain" library smaller. If you try the following code with the suggested version “langchain==0.0.208” it should work fine, but if you want to run it with the latest langchain version then you have to (1) install the experimental library with “pip install langchain-experimental” and (2) replace all the occurrences of “langchain.experimental” with “langchain_experimental”. </aside> We then use the `requests` library to send HTTP requests and the `newspaper` library for article parsing. By iterating over a list of article URLs, the code downloads the HTML of each webpage, extracts the article text, and stores it along with the corresponding URL. We could also load our private files on Deep Lake, but for this project's scope, we’ll upload content downloaded from public web pages. ```python # We scrape several Artificial Intelligence news import requests from newspaper import Article # https://github.com/codelucas/newspaper import time headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36' } article_urls = [ "https://www.artificialintelligence-news.com/2023/05/23/meta-open-source-speech-ai-models-support-over-1100-languages/", "https://www.artificialintelligence-news.com/2023/05/18/beijing-launches-campaign-against-ai-generated-misinformation/" "https://www.artificialintelligence-news.com/2023/05/16/openai-ceo-ai-regulation-is-essential/", "https://www.artificialintelligence-news.com/2023/05/15/jay-migliaccio-ibm-watson-on-leveraging-ai-to-improve-productivity/", "https://www.artificialintelligence-news.com/2023/05/15/iurii-milovanov-softserve-how-ai-ml-is-helping-boost-innovation-and-personalisation/", "https://www.artificialintelligence-news.com/2023/05/11/ai-and-big-data-expo-north-america-begins-in-less-than-one-week/", "https://www.artificialintelligence-news.com/2023/05/11/eu-committees-green-light-ai-act/", "https://www.artificialintelligence-news.com/2023/05/09/wozniak-warns-ai-will-power-next-gen-scams/", "https://www.artificialintelligence-news.com/2023/05/09/infocepts-ceo-shashank-garg-on-the-da-market-shifts-and-impact-of-ai-on-data-analytics/", "https://www.artificialintelligence-news.com/2023/05/02/ai-godfather-warns-dangers-and-quits-google/", "https://www.artificialintelligence-news.com/2023/04/28/palantir-demos-how-ai-can-used-military/", "https://www.artificialintelligence-news.com/2023/04/26/ftc-chairwoman-no-ai-exemption-to-existing-laws/", "https://www.artificialintelligence-news.com/2023/04/24/bill-gates-ai-teaching-kids-literacy-within-18-months/", "https://www.artificialintelligence-news.com/2023/04/21/google-creates-new-ai-division-to-challenge-openai/" ] session = requests.Session() pages_content = [] # where we save the scraped articles for url in article_urls: try: time.sleep(2) # sleep two seconds for gentle scraping response = session.get(url, headers=headers, timeout=10) if response.status_code == 200: article = Article(url) article.download() # download HTML of webpage article.parse() # parse HTML to extract the article text pages_content.append({ "url": url, "text": article.text }) else: print(f"Failed to fetch article at {url}") except Exception as e: print(f"Error occurred while fetching article at {url}: {e}") #If an error occurs while fetching an article, we catch the exception and print #an error message. This ensures that even if one article fails to download, #the rest of the articles can still be processed. ``` Then, we import the `OpenAIEmbeddings` class, which will be used to compute embeddings for our documents. We also import the `Deep Lake` class from the `langchain.vectorstores` module will serve as the storage for our documents and their embeddings. By setting up the `Deep Lake` instance with a specified dataset path and the `embedding_function` parameter set to the `OpenAIEmbeddings` instance, we establish a connection to Deep Lake and configure it to use the specified embedding model for computing document embeddings. Remember to install the required packages with the following command: `pip install langchain==0.0.208 deeplake openai tiktoken`. ```python # We'll use an embedding model to compute our documents' embeddings from langchain.embeddings.openai import OpenAIEmbeddings # We'll store the documents and their embeddings in the deep lake vector db from langchain.vectorstores import DeepLake # Setup deep lake embeddings = OpenAIEmbeddings(model="text-embedding-ada-002") # create Deep Lake dataset # TODO: use your organization id here. (by default, org id is your username) my_activeloop_org_id = "<YOUR-ACTIVELOOP-ORG-ID>" my_activeloop_dataset_name = "langchain_course_analysis_outline" dataset_path = f"hub://{my_activeloop_org_id}/{my_activeloop_dataset_name}" db = DeepLake(dataset_path=dataset_path, embedding_function=embeddings) ``` Next, we create an instance of `RecursiveCharacterTextSplitter` with specified `chunk_size` and `chunk_overlap` parameters. Then, we iterated over the `pages_content` and use the `split_text` method of the `text_splitter` to split each article text into chunks. These chunks are then appended to the `all_texts` list, resulting in a collection of smaller text chunks derived from the original articles. ```python # We split the article texts into small chunks from langchain.text_splitter import RecursiveCharacterTextSplitter text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100) all_texts = [] for d in pages_content: chunks = text_splitter.split_text(d["text"]) for chunk in chunks: all_texts.append(chunk) ``` Now we can add those chunks to the Deep Lake database. ```python # we add all the chunks


# Building Autonomous Agents to Create Analysis Reports
to the Deep lake db.add_texts(all_texts) ``` We are done with setting up the Deep Lake dataset with our documents! Let’s now focus on building the “Plan and Execute” agent that will leverage our dataset. Now, we can set up our **Plan and Execute** agent. Let’s create a `retriever` from the Deep Lake dataset and a function for our custom tool that retrieves the most similar documents to a query from the dataset. ```python # Get the retriever object from the deep lake db object and set the number # of retrieved documents to 3 retriever = db.as_retriever() retriever.search_kwargs['k'] = 3 # We define some variables that will be used inside our custom tool CUSTOM_TOOL_DOCS_SEPARATOR ="\n---------------\n" # how to join together the retrieved docs to form a single string # This is the function that defines our custom tool that retrieves relevant # docs from Deep Lake def retrieve_n_docs_tool(query: str) -> str: """Searches for relevant documents that may contain the answer to the query.""" docs = retriever.get_relevant_documents(query) texts = [doc.page_content for doc in docs] texts_merged = "---------------\n" + CUSTOM_TOOL_DOCS_SEPARATOR.join(texts) + "\n---------------" return texts_merged ``` We retrieve the retriever object from the Deep Lake database and set the number of retrieved documents to 3. This is important for the plan and execution agent because it allows us to retrieve a specific number of relevant documents from Deep Lake based on a given query. Also, we defined a custom tool function called `retrieve_n_docs_tool` that takes a query as input and uses the retriever to search for relevant documents containing the answer to the query. The retrieved document texts are then merged using the `CUSTOM_TOOL_DOCS_SEPARATOR` variable, representing the separator string used to join the documents into a single string. The merged text is returned as the output of the custom tool function. This functionality enables the plan and execution agent to retrieve and process relevant documents for further analysis and decision-making. ```python from langchain.agents.tools import Tool # We create the tool that uses the "retrieve_n_docs_tool" function tools = [ Tool( name="Search Private Docs", func=retrieve_n_docs_tool, description="useful for when you need to answer questions about current events about Artificial Intelligence" ) ] ``` The tool is named "Search Private Docs," and its functionality is based on the `retrieve_n_docs_tool` function. The purpose of this tool is to provide a way to search for and retrieve relevant documents from Deep Lake in order to answer questions about current events related to Artificial Intelligence. The tool is described as being useful in situations where there is a need to gather information and insights from private documents. We are now ready to create the agent! ```python from langchain.chat_models import ChatOpenAI from langchain.experimental.plan_and_execute import PlanAndExecute, load_agent_executor, load_chat_planner # let's create the Plan and Execute agent model = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0) planner = load_chat_planner(model) executor = load_agent_executor(model, tools, verbose=True) agent = PlanAndExecute(planner=planner, executor=executor, verbose=True) ``` The agent consists of two components: a planner and an executor. The **planner** is responsible for generating a plan based on the given input, and the **executor** executes the plan by interacting with the tools and external systems. The agent is set to be verbose, which means it will provide detailed information and logs during its operation. ```python # we test the agent response = agent.run("Write an overview of Artificial Intelligence regulations by governments by country") ``` You should see something like the following output. Here we split it into multiple sections and comment on them individually, keeping only the most relevant ones. ```python **> Entering new PlanAndExecute chain...** steps=[Step(value='Research the current state of Artificial Intelligence (AI) regulations in various countries.'), Step(value='Identify the key countries with significant AI


# Building Autonomous Agents to Create Analysis Reports
regulations or ongoing discussions about AI regulations.'), Step(value='Summarize the AI regulations or discussions in each identified country.'), Step(value='Organize the information by country, providing an overview of the AI regulations in each.'), Step(value='Given the above steps taken, provide an overview of Artificial Intelligence regulations by governments by country.\n')] ``` At first, the planning agent creates a plan for our query with multiple steps. Each step is a query that the action agent will be asked to give an answer to. Here are the identified steps. - Research the current state of Artificial Intelligence (AI) regulations in various countries. - Identify the key countries with significant AI regulations or ongoing discussions about AI regulations. - Summarize the AI regulations or discussions in each identified country. - Organize the information by country, providing an overview of each AI regulation. - Given the above steps taken, provide an overview of Artificial Intelligence regulations by governments by country. Let’s see how the output continues. ```python **> Entering new AgentExecutor chain...*Action: ``` { "action": "Search Private Docs", "action_input": "current state of Artificial Intelligence regulations in various countries" } ```*** Observation: ***--------------- “US-based AI developers will likely steal a march on their European competitors given the news that the EU parliamentary committees have green-lit its groundbreaking AI Act, where AI systems will need to be categorized according to their potential for harm from the outset. The US tech approach is typically to experiment first and, once market and product fit is established, to retrofit to other markets and their regulatory framework. This approach fosters innovation whereas EU-based AI developers will need to take note of the new rules and develop systems and processes which may take the edge off their ability to innovate. The UK is adopting a similar approach to the US, although the proximity of the EU market means that UK-based developers are more likely to fall into step with the EU ruleset from the outset. However, the potential to experiment in a safe space – a regulatory sandbox – may prove very attractive.” --------------- To boost AI innovation, MEPs added exemptions to these rules for research activities and AI components provided under open-source licenses. The new law also promotes regulatory sandboxes – or controlled environments established by public authorities – to test AI before its deployment. MEPs want to boost citizens’ right to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their rights. MEPs also reformed the role of the EU AI Office, which would be tasked with monitoring how the AI rulebook is implemented. Tim Wright, Tech and AI Regulatory Partner at London-based law firm Fladgate commented: --------------- Ryan is a senior editor at TechForge Media with over a decade of experience covering the latest technology and interviewing leading industry figures. He can often be sighted at tech conferences with a strong coffee in one hand and a laptop in the other. If it's geeky, he’s probably into it. Find him on Twitter (@Gadget_Ry) or Mastodon (@gadgetry@techhub.social) The Internal Market Committee and the Civil Liberties Committee of the European Parliament have endorsed new transparency and risk-management rules for artificial intelligence systems known as the AI Act. This marks a major step in the development of AI regulation in Europe, as these are the first-ever rules for AI. The rules aim to ensure that AI systems are safe, transparent, traceable, and non-discriminatory. After the vote, co-rapporteur Brando Benifei (S&D, Italy) said: ---------------*** Thought: ***I have information about the current state of AI regulations in the EU, US, and UK. I can provide a summary


# Building Autonomous Agents to Create Analysis Reports
of the AI regulations in these regions. Action: ``` { "action": "Final Answer", "action_input": "The current state of AI regulations varies across countries. In the European Union, the AI Act has been endorsed by the Internal Market Committee and the Civil Liberties Committee, marking a major step in AI regulation. The AI Act aims to ensure that AI systems are safe, transparent, traceable, and non-discriminatory. In the United States, AI developers typically experiment first and then retrofit their products to other markets and regulatory frameworks, fostering innovation. The UK is adopting a similar approach to the US, but its proximity to the EU market means that UK-based developers are more likely to align with the EU ruleset from the outset. Regulatory sandboxes, or controlled environments established by public authorities, are being promoted to test AI before deployment." } ```*> Finished chain.** ***** ``` In this step, we see the executor agent searching for relevant documents with our Deep Lake tool and showing the top 3 retrieved chunks. After analyzing them, the agent decides to return the final answer to the question, which is: ``` ***The current state of AI regulations varies across countries. In the European Union, the AI Act has been endorsed by the Internal Market Committee and the Civil Liberties Committee, marking a major step in AI regulation. The AI Act aims to ensure that AI systems are safe, transparent, traceable, and non-discriminatory. In the United States, AI developers typically experiment first and then retrofit their products to other markets and regulatory frameworks, fostering innovation. The UK is adopting a similar approach to the US, but its proximity to the EU market means that UK-based developers are more likely to align with the EU ruleset from the outset. Regulatory sandboxes, or controlled environments established by public authorities, are being promoted to test AI before deployment.*** ``` Here we omit the outputs of all the other steps done by the executor agent. Let’s print the final response from the agent. ```python print(response) ``` ``` European Union: The AI Act has been endorsed by the Internal Market Committee and the Civil Liberties Committee, aiming to ensure AI systems are safe, transparent, traceable, and non-discriminatory. United States: AI developers typically experiment first and then retrofit their products to other markets and regulatory frameworks, fostering innovation. United Kingdom: The UK is adopting a similar approach to the US, but its proximity to the EU market means that UK-based developers are more likely to align with the EU ruleset from the outset. Regulatory sandboxes are being promoted to test AI before deployment. ``` We see that the agent has been able to iteratively create an overview of AI regulations by diverse documents, leveraging several documents. ### Conclusion The experiment involving the Plan and Execute agent has been successful in providing a comprehensive overview of Artificial Intelligence regulations by governments, specifically by finding information about the European Union, United States, and United Kingdom. The agent effectively performed various steps, including researching the current state of AI regulations, identifying key countries, summarizing regulations, and organizing the information by country. The output generated by the agent demonstrates its ability to understand and interpret complex information about AI regulations. It accurately summarizes the AI regulations in each country, highlighting the endorsement of the AI Act in the European Union to ensure the safety, transparency, traceability, and non-discrimination of AI systems. The agent successfully executes its plan by retrieving relevant information, summarizing it, and providing a concise and informative overview. It demonstrates its capability to gather insights from multiple sources and deliver a coherent response. The agent's performance


# Building Autonomous Agents to Create Analysis Reports
in this experiment highlights its potential to assist with research, generate informative summaries, and provide valuable insights. In the next lesson we’ll learn about recent developments and trends about LLM-based agents. **RESOURCES:** [Plan-and-Execute Agents](https://blog.langchain.dev/plan-and-execute-agents/) [Deep Lake | 🦜🔗 Langchain](https://python.langchain.com/docs/modules/data_connection/vectorstores/integrations/deeplake)


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
### Introduction These days, artificial intelligence is changing the copywriting field by serving as a writing assistant. These language models can find spelling or grammatical errors, change tones, summarize, or even extend the content. However, there are times when the model may not have the specialized knowledge in a particular field to provide expert-level suggestions for extending parts of an article. In this lesson, we will take you step by step through the process of building an application that can effortlessly expand text sections. The process begins by asking an LLM (ChatGPT) to generate a few search queries based on the text at hand. These queries are then will be used to search the Internet using Google Search API that, captures relevant information on the subject. Lastly, the most relevant results will be presented as context to the model to suggest better content. We've got three variables here that hold an article's title and content (`text_all`). (From [Artificial Intelligence News](https://www.artificialintelligence-news.com/2023/05/16/openai-ceo-ai-regulation-is-essential/)) Also, the `text_to_change` variable specifies which part of the text we want to expand upon. These constants are mentioned as a reference and will remain unchanged throughout the lesson. ```python title = "OpenAI CEO: AI regulation ‘is essential’" text_all = """ Altman highlighted the potential benefits of AI technologies like ChatGPT and Dall-E 2 to help address significant challenges such as climate change and cancer, but he also stressed the need to mitigate the risks associated with increasingly powerful AI models. Altman proposed that governments consider implementing licensing and testing requirements for AI models that surpass a certain threshold of capabilities. He highlighted OpenAI’s commitment to safety and extensive testing before releasing any new systems, emphasising the company’s belief that ensuring the safety of AI is crucial. Senators Josh Hawley and Richard Blumenthal expressed their recognition of the transformative nature of AI and the need to understand its implications for elections, jobs, and security. Blumenthal played an audio introduction using an AI voice cloning software trained on his speeches, demonstrating the potential of the technology. Blumenthal raised concerns about various risks associated with AI, including deepfakes, weaponised disinformation, discrimination, harassment, and impersonation fraud. He also emphasised the potential displacement of workers in the face of a new industrial revolution driven by AI.""" text_to_change = """ Senators Josh Hawley and Richard Blumenthal expressed their recognition of the transformative nature of AI and the need to understand its implications for elections, jobs, and security. Blumenthal played an audio introduction using an AI voice cloning software trained on his speeches, demonstrating the potential of the technology.""" ``` The following diagram explains the workflow of this project. ![Screenshot 2023-06-09 at 13.42.59.png](Supercharge%20Your%20Blog%20Posts%20Automatically%20with%20Lan%2058fffaaca00340419c417bfcc8e62542/Screenshot_2023-06-09_at_13.42.59.png) First we generate candidate search queries from the selected paragraph that we want to expand. The queries are then used to extract relevant documents using a search engine (e.g. Bing or Google Search), which are the split into small chunks. We then compute embeddings of these chunks and save chunks and embeddings in a Deep Lake dataset. Last, the most similar chunks to the paragraph that we want to expand are retrieved from Deep Lake, and used in a prompt to expand the paragraph with further knowledge. Remember to install the required packages with the following command: `pip install langchain==0.0.208 deeplake openai tiktoken`. Refer to the course introduction if you are looking for the specific versions we used to write the codes in this lesson. Additionally, install the *newspaper3k* package with version `0.2.8`. ```python !pip install -q newspaper3k==0.2.8 python-dotenv ```


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
## Generate Search Queries
The code below uses OpenAI’s ChatGPT model to process an article and suggest three relevant search phrases. We define a prompt that asks the model to suggest Google search queries that could be used to with finding more information about the subject. The `LLMChain` ties the `ChatOpenAI` model and `ChatPromptTemplate` together to create the chain to communicate with the model. Lastly, it splits the response by newline and removes the first characters to extract the data. The mentioned format works because we asked the API to generate each query in a new line that starts with `-`. (It is possible to achieve the same effect by using the `OutputParser` class) Prior to running the code provided below, make sure to store your OpenAI key in the `OPENAI_API_KEY` environment variable. ```python from langchain.chat_models import ChatOpenAI from langchain.chains import LLMChain from langchain.prompts import PromptTemplate from langchain.prompts.chat import ( ChatPromptTemplate, HumanMessagePromptTemplate, ) template = """ You are an exceptional copywriter and content creator. You're reading an article with the following title: ---------------- {title} ---------------- You've just read the following piece of text from that article. ---------------- {text_all} ---------------- Inside that text, there's the following TEXT TO CONSIDER that you want to enrich with new details. ---------------- {text_to_change} ---------------- What are some simple and high-level Google queries that you'd do to search for more info to add to that paragraph? Write 3 queries as a bullet point list, prepending each line with -. """ human_message_prompt = HumanMessagePromptTemplate( prompt=PromptTemplate( template=template, input_variables=["text_to_change", "text_all", "title"], ) ) chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt]) # Before executing the following code, make sure to have # your OpenAI key saved in the “OPENAI_API_KEY” environment variable. chat = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.9) chain = LLMChain(llm=chat, prompt=chat_prompt_template) response = chain.run({ "text_to_change": text_to_change, "text_all": text_all, "title": title }) queries = [line[2:] for line in response.split("\n")] print(queries) ``` ```python ['AI technology implications for elections', 'AI technology implications for jobs', 'AI technology implications for security] ``` <aside> 💡 The queries you receive from the model might differ from the results above. It is because we set the model’s `temperature` argument to `0.9` which makes it highly creative, so it generates more diverse results. </aside>


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
## Get Search Results
We must set up the API Key and a custom search engine to be able to use Google search API. To get the key, head to the [Google Cloud console](https://console.cloud.google.com/apis/credentials) and generate the key by pressing the CREATE CREDENTIALS buttons from the top and choosing API KEY. Then, head to the [Programmable Search Engine](https://programmablesearchengine.google.com/controlpanel/create) dashboard and remember to select the “Search the entire web” option. The Search engine ID will be visible in the details. You might also need to enable the “Custom Search API” service under the Enable APIs and services. (You will receive the instruction from API if required) We can now configure the environment variables `GOOGLE_CSE_ID` and `GOOGLE_API_KEY`, allowing the Google wrapper to connect with the API. The next step is to use the generated queries from the previous section to get a number of sources from Google searches. The LangChain library provides the `GoogleSearchAPIWrapper` utility that takes care of receiving search results and makes a function to run it `top_n_results`. Then, the `Tool` class will create a wrapper around the said function to make it compatible with agents and help them to interact with the outside world. We only ask for the top 5 results and concatenate the results for each query in the `all_results` variable. ```python from langchain.tools import Tool from langchain.utilities import GoogleSearchAPIWrapper # Remember to set the "GOOGLE_CSE_ID" and "GOOGLE_API_KEY" environment variable. search = GoogleSearchAPIWrapper() TOP_N_RESULTS = 5 def top_n_results(query): return search.results(query, TOP_N_RESULTS) tool = Tool( name = "Google Search", description="Search Google for recent results.", func=top_n_results ) all_results = [] for query in queries: results = tool.run(query) all_results += results ``` The `all_results` variable holds 15 web addresses. (3 queries from ChatGPT x 5 top Google search results) However, it is not optimal flow to use all the contents as a context in our application. There are technical, financial, and contextual considerations to keep in mind. Firstly, the input length of the LLMs is restricted to a range of 2K to 4K tokens, which varies based on the model we choose. Although we can overcome this limitation by opting for a different chain type, it is more efficient and tends to yield superior outcomes when we adhere to the model's window size. Secondly, it's important to note that increasing the number of words we provide to the API results in a higher cost. While dividing a prompt into multiple chains is possible, we should be cautious as the cost of these models is determined by the token count. And lastly, the content that the stored search results will provide is going to be close in context. So, it is a good idea to use the most relevant results.


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
## Find the Most Relevant Results
As mentioned before, Google Search will return the URL for each source. However, we need the content of these pages. The `newspaper` package can extract the contents of a web link using the `.parse()` method. The following code will loop through the results and attempt to extract the content. ```python import newspaper pages_content = [] for result in all_results: try: article = newspaper.Article(result["link"]) article.download() article.parse() if len(article.text) > 0: pages_content.append({ "url": result["link"], "text": article.text }) except: continue print("Number of pages: ", len(pages_content)) ``` ``` Number of pages: 14 ``` The output above shows that 14 pages were processed while we expected 15. There are specific scenarios in which the `newspaper` library may encounter difficulties extracting information. These include search results that lead to a PDF file or websites that restrict access to web scraping. Now, it is crucial to split the saved contents into smaller chunks to ensure the articles do not exceed the model’s input length. The code below splits the text by either newline or spaces, depending on the situation. It makes sure that each chunk has 3000 characters with 100 overlaps between the chunks. ```python from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.docstore.document import Document text_splitter = RecursiveCharacterTextSplitter(chunk_size=3000, chunk_overlap=100) docs = [] for d in pages_content: chunks = text_splitter.split_text(d["text"]) for chunk in chunks: new_doc = Document(page_content=chunk, metadata={ "source": d["url"] }) docs.append(new_doc) print("Number of chunks: ", len(docs)) ``` ``` Number of chunks: 46 ``` As you can see, 46 chunks of data are in the `docs` variable. It is time to find the most relevant chunks to pass them as context to the large language model. The `OpenAIEmbeddings` class will use OpenAI to convert the texts into vector space that holds semantics. We proceeded to embed both document chunks and the desired sentence from the main article that was chosen for expansion. The selected sentence was chosen at the beginning of this lesson and represented by the `text_to_change` variable. ```python from langchain.embeddings import OpenAIEmbeddings embeddings = OpenAIEmbeddings(model="text-embedding-ada-002") docs_embeddings = embeddings.embed_documents([doc.page_content for doc in docs]) query_embedding = embeddings.embed_query(text_to_change) ``` Finding the distance between the high-dimensionality embedding vectors is possible using the cosine similarity metric. It determines how close two points are within the vector space. Since the embeddings contain contextual information, their proximity indicates a shared meaning. So, the document with a higher similarity score can be used as the source. We used the `cosine_similarity` function from the `sklearn` library. It calculates the distance between each chunk and the chosen sentence to return the index of the best three results. ```python import numpy as np from sklearn.metrics.pairwise import cosine_similarity def get_top_k_indices(list_of_doc_vectors, query_vector, top_k): # convert the lists of vectors to numpy arrays list_of_doc_vectors = np.array(list_of_doc_vectors) query_vector = np.array(query_vector) # compute cosine similarities similarities = cosine_similarity(query_vector.reshape(1, -1), list_of_doc_vectors).flatten() # sort the vectors based on cosine similarity sorted_indices = np.argsort(similarities)[::-1] # retrieve the top K indices from the sorted list top_k_indices = sorted_indices[:top_k] return top_k_indices top_k = 3 best_indexes = get_top_k_indices(docs_embeddings, query_embedding, top_k) best_k_documents = [doc for i, doc in enumerate(docs) if i in best_indexes] ```


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
## Extend the Sentence
We can now define the prompt using the additional information from Google search. There are six input variables in the template: - `title` that holds the main article’s title; - `text_all` to present the whole article we are working on; - `text_to_change` is the selected part of the article that requires expansion; - `doc_1`, `doc_2`, `doc_3` to include the close Google search results as context. The remaining part of the code should be familiar, as it follows the same structure used for generating Google queries. It defines a `HumanMessage` template to be compatible with the ChatGPT API, which is defined with a high-temperature value to encourage creativity. The `LLMChain` class will create a chain that combines the model and prompt to finish up the process by using `.run()` method ```python template = """You are an exceptional copywriter and content creator. You're reading an article with the following title: ---------------- {title} ---------------- You've just read the following piece of text from that article. ---------------- {text_all} ---------------- Inside that text, there's the following TEXT TO CONSIDER that you want to enrich with new details. ---------------- {text_to_change} ---------------- Searching around the web, you've found this ADDITIONAL INFORMATION from distinct articles. ---------------- {doc_1} ---------------- {doc_2} ---------------- {doc_3} ---------------- Modify the previous TEXT TO CONSIDER by enriching it with information from the previous ADDITIONAL INFORMATION. """ human_message_prompt = HumanMessagePromptTemplate( prompt=PromptTemplate( template=template, input_variables=["text_to_change", "text_all", "title", "doc_1", "doc_2", "doc_3"], ) ) chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt]) chat = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.9) chain = LLMChain(llm=chat, prompt=chat_prompt_template) response = chain.run({ "text_to_change": text_to_change, "text_all": text_all, "title": title, "doc_1": best_k_documents[0].page_content, "doc_2": best_k_documents[1].page_content, "doc_3": best_k_documents[2].page_content }) print("Text to Change: ", text_to_change) print("Expanded Variation:", response) ``` ``` **Text to Change:** Senators Josh Hawley and Richard Blumenthal expressed their recognition of the transformative nature of AI and the need to understand its implications for elections, jobs, and security. Blumenthal played an audio introduction using an AI voice cloning software trained on his speeches, demonstrating the potential of the technology. **Expanded Variation:** During a Senate Judiciary Subcommittee on Privacy, Technology, and the Law hearing titled “Oversight of AI: Rules for Artificial Intelligence,” Senators Josh Hawley and Richard Blumenthal expressed their recognition of the transformative nature of AI and its implications for elections, jobs, and security. Blumenthal even demonstrated the potential of AI voice cloning software trained on his speeches, highlighting the need for AI regulations. Recent advances in generative AI tools can create hyper-realistic images, videos, and audio in seconds, making it easy to spread fake and digitally created content that could potentially mislead voters and undermine elections. Legislation has been introduced that would require candidates to label campaign advertisements created with AI and add a watermark indicating the fact for synthetic images. Blumenthal raised concerns about various risks associated with AI, including deepfakes, weaponized disinformation, discrimination, harassment, and impersonation fraud. The Senate Judiciary Subcommittee on Privacy, Technology, and the Law has jurisdiction over legal issues pertaining to technology and social media platforms, including online privacy and civil rights, as well as the impact of new or emerging technologies. ```


# Supercharge Your Blog Posts Automatically with LangChain and Google Search
## Conclusion
In this lesson, we gained insights into leveraging Google search results to enrich the prompt to the model by incorporating additional information. The demonstration showcased the utilization of embedding vectors to identify content that shares a similar meaning or context—also the process of adding relevant information to a prompt to achieve better output. Incorporating external information, such as Google search, is a potent tool for enhancing models by offering supplementary context in situations lacking sufficient data. In the upcoming lesson, we will employ the same concept to create a chatbot capable of providing accurate answers by utilizing Google search results. --- You can find the code of this lesson in this online [Notebook](https://drive.google.com/file/d/1-r-yr4n9rfwQ3h_31C96FdUBEaCXQDmz/view?usp=sharing).


Chat with a GitHub Repository

# Chat with a GitHub Repository
### Introduction Large language models (LLMs) accomplish a remarkable level of language comprehension during their training process. It enables them to generate human-like text and creates powerful representations from textual data. We already covered leveraging LangChain to use LLMs for writing content with hands-on projects. This lesson will focus on using the language models for generating embeddings from corpora. The mentioned representation will power a chat application that can answer questions from any text by finding the closest data point to an inquiry. This project focuses on finding answers from a GitHub repository’s text files like `.md` and `.txt`. So, we will start by capturing data from a GitHub repository and converting it to embeddings. These embeddings will be saved on the Activeloop’s Deep Lake vector database for fast and easy access. The Deep Lake’s retriever object will find the related files based on the user’s query and provide them as the context to the model. Lastly, the model leverages the provided information to the best of its ability to answer the question.


Chat with a GitHub Repository

# Chat with a GitHub Repository
## What is Deep Lake?
It is a vector database that offers multi-modality storage for all kinds of data (including but not limited to PDFs, Audio, and Videos) alongside their vectorized representations. This service eliminates the need to create data infrastructure while dealing with high-dimensionality tensors. Furthermore, it provides a wide range of functionalities like visualizing, parallel computation, data versioning, integration with major AI frameworks, and, most importantly, embedding search. The supported vector operations like `cosine_similarity` allow us to find relevant points in an embedding space. --- The rest of the lesson is based on the code from the “[Chat with Github Repo](https://github.com/peterw/Chat-with-Github-Repo/)” repository and is organized as follows: **1) Processing the Files** **2) Saving the Embedding** **3) Retrieving from Database** **4) Creating an Interface**.


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Processing the Repository Files
In order to access the files in the target repository, the script will clone the desired repository onto your computer, placing the files in a folder named "repos". Once we download the files, it is a matter of looping through the directory to create a list of files. It is possible to filter out specific extensions or environmental items. ```python root_dir = "./path/to/cloned/repository" docs = [] file_extensions = [] for dirpath, dirnames, filenames in os.walk(root_dir): for file in filenames: file_path = os.path.join(dirpath, file) if file_extensions and os.path.splitext(file)[1] not in file_extensions: continue loader = TextLoader(file_path, encoding="utf-8") docs.extend(loader.load_and_split()) ``` The sample code above creates a list of all the files in a repository. It is possible to filter each item by extension types like `file_extensions=['.md', '.txt']` which only focus on markdown and text files. The original implementation has more filters and a fail-safe approach; Please refer to the [complete code](https://github.com/peterw/Chat-with-Github-Repo/blob/main/src/utils/process.py#L20). Now that the list of files are created, the `split_documents` method from the `CharacterTextSplitter` class in the LangChain library will read the files and split their contents into chunks of 1000 characters. ```python from langchain.text_splitter import CharacterTextSplitter text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) splitted_text = text_splitter.split_documents(docs) ``` The `splitted_text` variable holds the textual content which is ready to be converted to embedding representations.


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Saving the Embeddings
Let’s create the database before going through the process of converting texts to embeddings. It is where the integration between LangChain and Deep Lake comes in handy! We initialize the database in cloud using the `hub://...` format and the `OpenAIEmbeddings()` from LangChain as the embedding function. The Deep Lake library will iterate through the content and generate the embedding automatically. ```python from langchain.embeddings.openai import OpenAIEmbeddings from langchain.vectorstores import DeepLake # Before executing the following code, make sure to have # your OpenAI key saved in the “OPENAI_API_KEY” environment variable. embeddings = OpenAIEmbeddings(model="text-embedding-ada-002") # TODO: use your organization id here. (by default, org id is your username) my_activeloop_org_id = "<YOUR-ACTIVELOOP-ORG-ID>" my_activeloop_dataset_name = "langchain_course_chat_with_gh" dataset_path = f"hub://{my_activeloop_org_id}/{my_activeloop_dataset_name}" db = DeepLake(dataset_path=dataset_path, embedding_function=embeddings) db.add_documents(splitted_text) ```


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Retrieving from Database
The last step is to code the process to answer the user’s question based on the database’s information. Once again, the integration of LangChain and Deep Lake simplifies the process significantly, making it exceptionally easy. We need 1) a retriever object from the Deep Lake database using the `.as_retriever()` method, and 2) a conversational model like ChatGPT using the `ChatOpenAI()` class. Finally, LangChain’s `RetrievalQA` class ties everything together! It uses the user’s input as the prompt while including the results from the database as the context. So, the ChatGPT model can find the correct one from the provided context. It is worth noting that the database retriever is configured to gather instances closely related to the user’s query by utilizing cosine similarities. ```python # Create a retriever from the DeepLake instance retriever = db.as_retriever() # Set the search parameters for the retriever retriever.search_kwargs["distance_metric"] = "cos" retriever.search_kwargs["fetch_k"] = 100 retriever.search_kwargs["maximal_marginal_relevance"] = True retriever.search_kwargs["k"] = 10 # Create a ChatOpenAI model instance model = ChatOpenAI() # Create a RetrievalQA instance from the model and retriever qa = RetrievalQA.from_llm(model, retriever=retriever) # Return the result of the query qa.run("What is the repository's name?") ```


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Create an Interface
Creating a user interface (UI) for the bot to be accessed through a web browser is an optional yet crucial step. This addition will elevate your ideas to new heights, allowing users to engage with the application effortlessly, even without any programming expertise. This repository uses the [Streamlit](https://streamlit.io/) platform, a fast and easy way to build and deploy an application instantly for free. It provides a wide range of widgets to eliminate the need for using backend or frontend frameworks to build a web application. We must install the library and its chat component using the pip command. We strongly recommend installing the latest version of each library. Furthermore, the provided codes have been tested using streamlit and streamlit-chat versions `2023.6.21` and `20230314`, respectively. ```bash pip install streamlit streamlit_chat ``` The [API documentation](https://docs.streamlit.io/) page provides a comprehensive list of available widgets that can use in your application. We need a simple UI that accepts the input from the user and shows the conversation in a chat-like interface. Luckily, Streamlit provides both. ```python import streamlit as st from streamlit_chat import message # Set the title for the Streamlit app st.title(f"Chat with GitHub Repository") # Initialize the session state for placeholder messages. if "generated" not in st.session_state: st.session_state["generated"] = ["i am ready to help you ser"] if "past" not in st.session_state: st.session_state["past"] = ["hello"] # A field input to receive user queries input_text = st.text_input("", key="input") # Search the databse and add the responses to state if user_input: output = qa.run(user_input) st.session_state.past.append(user_input) st.session_state.generated.append(output) # Create the conversational UI using the previous states if st.session_state["generated"]: for i in range(len(st.session_state["generated"])): message(st.session_state["past"][i], is_user=True, key=str(i) + "_user") message(st.session_state["generated"][i], key=str(i)) ``` The code above is straightforward. We call `st.text_input()` to create text input for users queries. The query will be passed to the previously declared `RetrievalQA` object, and the results will be shown using the `message` component. You should store the mentioned code in a Python file (for example, `chat.py`) and run the following command to see the interface locally. ```bash streamlit run ./chat.py ``` Please read the [documentation](https://docs.streamlit.io/library/get-started) on how to [deploy](https://docs.streamlit.io/library/get-started/create-an-app#share-your-app) the application on the web so anyone can access it.


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Putting Everything Together
As we mentioned previously, the codes in this lesson are available in the “[Chat with GitHub Repo](https://github.com/peterw/Chat-with-Github-Repo),” you can easily fork and run it in 3 simple steps. First, fork the repository and install the required libraries using pip. ```bash git clone https://github.com/peterw/Chat-with-Git-Repo.git cd Chat-with-Git-Repo pip install -r requirements.txt ``` Second, rename the environment file from .env.example to .env and fill in the API keys. You must have accounts in both [OpenAI](https://openai.com/product) and [Activeloop](https://www.activeloop.ai/). ```bash cp .env.example .env # OPENAI_API_KEY=your_openai_api_key # ACTIVELOOP_TOKEN=your_activeloop_api_token # ACTIVELOOP_USERNAME=your_activeloop_username ``` Lastly, use the `process` command to read and store the contents of any repository on the Deep Lake by passing the repository URL to the `--repo-url` argument. ```bash python src/main.py process --repo-url https://github.com/username/repo_name ``` <aside> 💡 Be aware of the costs associated with generating embeddings using the OpenAI API. Using a smaller repository that needs fewer resources and faster processing is better. </aside> And run the chat interface by using the `chat` command followed by the database name. It is the same as `repo_name` from the above sample. You can also see the database name by logging in to the Deep Lake dashboard. ```bash python src/main.py chat --activeloop-dataset-name <dataset_name> ``` The application will be accessible using a browser on the [http://localhost:8501](http://localhost:8501/) URL or the next available port. (as demonstrated in the image below) Please read the complete [instruction](https://github.com/peterw/Chat-with-Github-Repo/tree/main#setup) for more information, like filtering a repository content by file extension. ![Sample usage of the chatbot using the “CompressedBART” repository.](Chat%20with%20a%20GitHub%20Repository%208f94e7f951d541389a3d599b31387114/the_output.png) Sample usage of the chatbot using the “CompressedBART” repository.


Chat with a GitHub Repository

# Chat with a GitHub Repository
## Conclusion
We broke down the crucial sections of the “Chat with GitHub Repo” repository to teach creating a chatbot with a user interface. You have learned how to use the Deep Lake database to store the large dimensional embeddings and query them using similarity functions like cosine. Their integration with the LangChain library provided easy-to-use APIs for storing and retrieving data. Lastly, we created a user interface using the Streamlit library to make the bot available for everyone. In the next lesson, we’ll build a question-answering chatbot that leverages external documents as knowledge base, while also providing references along to its answers.


# What other models can we use?  Popular LLM models compared
### Introduction This lesson will delve into integrating several LLM models in LangChain. We will examine the platforms supporting these LLM models and compare their features. LangChain has built-in support for some of the most popular publicly available pre-trained models. In previous lessons, we already discussed several options like ChatGPT, GPT-4, GPT-3, and GPT4ALL. This framework provides close to 30 integrations with well-known AI platforms like OpenAI, Cohere, Writer, and Replicate, to name a few. Most notably, they provide access to Huggingface Hub API with more than 120K available models that can be easily incorporated into your applications. These organizations offer different ways to access their services. It is a common practice to pay for the API interfaces. The prices are usually determined by factors such as the number of processed tokens, as seen in OpenAI, or the process’s duration measured in hours of GPU usage, as is the case with Huggingface Interface or Amazon Sagemaker. These options are generally easy and fast to set up. However, it is worth noting that you do not own the models, even if it was fine-tuned on your valuable datasets. They just provide access to the API with a pay-as-you-go plan. On the other side of the spectrum, hosting the models locally on your servers is possible. It will enable you to have full and exclusive control over the network and your dataset. It is important to be aware of the hardware (high-end GPU for low latency) and maintenance (the expertise to deploy and fine-tune models) costs that are associated with this approach. Additionally, a number of publicly available models are not accessible for commercial use, like LLaMA. The right approach is different for each use case and depends on details like budget, model capability, expertise, and trade secrets. It is straightforward to create a custom fine-tuned model by feeding your data to OpenAI’s API. On the other hand, you might consider doing fine-tuning in-house if the dataset is part of your intellectual property and cannot be shared. The different models’ characteristics are another consideration. The network sizes and the dataset quality directly impact its language understanding capability. In contrast, a larger model is not always the best answer. The GPT-3’s Ada variation is the smallest model in the collection, making it the fastest and most cost-effective option with low latency. However, it suits more straightforward tasks like parsing text or classification. Conversely, the latest GPT-4 version is the largest model to generate high-quality results for every task. But, the large number of parameters makes it a slow and the most expensive option. Therefore, selecting the model based on their ability is also necessary. It might be cheaper to use Ada to implement an application to hold a conversation, but it is not the model’s objective and will result in disappointing responses. (You can read [this article](https://levelup.gitconnected.com/how-to-benchmark-language-models-by-openai-deepmind-google-microsoft-783d4307ec50) for a comparison between a number of well-known LLMs) We will introduce a number of LangChain integrations in the rest of this lesson to help choose the right approach.


# What other models can we use?  Popular LLM models compared
## Popular LLM models accessible to LangChain via API
### Cohere Command The Cohere service provides a variety of models such as *Command* (`command`) for dialogue-like interactions, *Generation* (`base`) for generative tasks, *Summarize* (`summarize-xlarge`) for generating summaries, [and more](https://docs.cohere.com/docs/models). You can get free, rate-limited usage for learning and prototyping. This means that usage is free until you go into production, however some of the models may be a bit more expensive than OpenAI APIs when you do—for example, $2.5 for generating 1K tokens. However, since Cohere offers more customized models for each task, this could lead to a more use case-specific model having improved outcomes in downstream tasks. The LangChain’s Cohere class makes it easy to access these models. `Cohere(model="<MODEL_NAME>", cohere_api_key="<API_KEY>")` <aside> 💡 You might see deprecated model names in the LangChain documentation. (like `command-xlarge-20221108`) Please refer to the Cohere [documentation](https://docs.cohere.com/docs/models) for the latest naming convention. </aside> ### GPT-3.5 GPT-3.5 is a language model developed by OpenAI. Its turbo version (recommended by OpenAI over [other variants](https://platform.openai.com/docs/models/gpt-3-5)) offers a more affordable option for generating human-like text through an API accessible via OpenAI endpoints. The model is optimized for chat applications while remaining powerful on other generative tasks and can process 96 languages. GPT-3.5-turbo has a up to 16K tokens context length and is the most cost-effective option from the OpenAI collection with only $0.002 per 1000 tokens. It is possible to access this model’s API by using the `gpt-3.5-turbo` key while initializing either `ChatOpenAI` or `OpenAI` classes. ### GPT-4 OpenAI's GPT-4 is a competent multimodal model with an undisclosed number of parameters or training procedures. It is the latest and most powerful model published by OpenAI, and the multi-modality enables the model to process both text and image as input. Unfortunately, It is not publicly available; however, it can be accessed by submitting your early access request through the OpenAI platform. The two variants of the model are `gpt-4` and `gpt-4-32k` with different context lengths, 8192 and 32768 tokens, respectively. ### Jurassic-2 The AI21’s Jurassic-2 is a language model with three sizes and different price points: Jumbo, Grande, and Large. The model sizes are not publicly available, but their documentation marks the Jumbo version as the most powerful model. They describe the models as general-purpose with excellent capability on every generative task. Their J2 model understands seven languages and can be fine-tuned on custom datasets. Getting your API key from the AI21 platform and using the `AI21()`class to access these models is possible. ### StableLM StableLM Alpha is a language model developed by Stable Diffusion, which can be accessed via HuggingFace Hub (with the following id `stabilityai/stablelm-tuned-alpha-3b`) to host locally or Replicate API with a rate from $0.0002 to $0.0023 per second. So far, it comes in two sizes, 3 billion and 7 billion parameters. The weights for StableLM Alpha are available under CC BY-SA 4.0 license with commercial use access. The context length of StableLM is 4096 tokens. ### Dolly-v2-12B Dolly-v2-12B is a language model created by Databricks, which can be accessed via HuggingFace Hub (with the following id `databricks/dolly-v2-3b`) to host locally or Replicate API with the same price range as mentioned in the previous subsection. It has 12 billion parameters and is available under an open source license for commercial use. The base model used for Dolly-v2-12B is Pythia-12B. ### GPT4ALL GPT4ALL is based on meta’s LLaMA model with 7B parameters. It is a language model developed by Nomic-AI that can be accessed through GPT4ALL and Hugging Face Local Pipelines. The model is published with a GPL 3.0 open-source license. However, it is not free to use for commercial applications. It is available for researchers to use


# What other models can we use?  Popular LLM models compared
## Popular LLM models accessible to LangChain via API
for their projects and experiments. We went through this model’s capability and usage process in the previous lesson.


# What other models can we use?  Popular LLM models compared
## LLM Platforms that can integrate into LangChain
[Cohere](https://python.langchain.com/en/latest/modules/models/llms/integrations/cohere.html) Cohere is a Canadian-based startup specializing in natural language processing models that help companies enhance human-machine interactions. Cohere provides access to their Cohere `xlarge` model through API, which has 52 billion parameters. Their API pricing is based on embeddings and is $1 for every 1000 embeddings. Cohere provides an easy-to-follow installation process for their package, which is required to access their API. Using LangChain, developers can easily interact with Cohere models by creating prompts incorporating input variables, which can then be passed to the Cohere API to generate responses. [OpenAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/openai.html) OpenAI platform is one of the biggest companies focusing on large language models. By introducing their conversational model, ChatGPT, they were the first service to catch mainstream media attention on the potency of LLMs. They also provide a large variety of API endpoints for different NLP tasks with different price points. The LangChain library provides multiple classes for convenient access, examples of which we saw in previous lessons, like `ChatGPT` and `GPT4` classes. [Hugging Face Hub](https://python.langchain.com/en/latest/modules/models/llms/integrations/huggingface_hub.html) Hugging Face is a company that develops natural language processing (NLP) technologies, including pre-trained language models, and offers a platform for developing and deploying NLP models. The platform hosts over 120k models and 20k datasets. They offer the Spaces service for researchers and developers to create a demo and showcase their model’s capabilities quickly. The platform hosts large-scale models such as StableLM by Stability AI, Dolly by DataBricks, or Camel by Writer. The `HuggingFaceHub` class takes care of downloading and initializing the models. This integration provides access to many models that are optimized for Intel CPUs using [Intel® Extension for PyTorch](https://github.com/intel/intel-extension-for-pytorch) library. The mentioned package can be applied to models with minimal code change. It enables the networks to take advantage of Intel**®**’s advanced architectural designs to significantly enhance CPU and GPU lines' performance. For example, the reports reveal a 3.8 speed up while running the BLOOMZ model (text-to-image) on the Intel® Xeon® 4s CPU compared to the previous generation with no changes in architecture/weights. When the mentioned optimization library was used alongside the 4th generation of Intel® Xeon® CPU, the inference speed rate increased nearly twofold to 6.5 times its original value. ([online demo](https://huggingface.co/spaces/Intel/Stable-Diffusion-Side-by-Side)) [Whisper](https://huggingface.co/Intel/whisper-large-int8-dynamic) and [GPT-J](https://huggingface.co/Intel/gpt-j-6B-int8-dynamic) are two other examples of widely recognized models that leverage these efficiency gains. [Amazon SageMakerEndpoint](https://python.langchain.com/en/latest/modules/models/llms/integrations/sagemaker.html) The Amazon SageMaker infrastructure enables users to train and host their machine-learning models easily. It is a high-performance and low-cost environment for experimenting and using large-scale models. The LangChain library provides a simple-to-use interface that simplifies the process of querying the deployed models. So, There is no need to write API codes for accessing the model. It is possible to load a model by using the `endpoint_name` which is the model’s unique name from SageMaker, followed by `credentials_profile_name` which is the name of the profile you want to use for authentication. [Hugging Face Local Pipelines](https://python.langchain.com/en/latest/modules/models/llms/integrations/huggingface_pipelines.html) Hugging Face Local Pipelines is a powerful tool that allows users to run Hugging Face models locally using the `HuggingFacePipeline` class. The Hugging Face Model Hub is home to an impressive collection of more than 120,000 models, 20,000 datasets, and 50,000 demo apps (Spaces) that are all publicly available and open source, making it easy for individuals to collaborate and build machine learning models together. To access these models, users can either utilize the local pipeline wrapper or call the hosted inference endpoints via the `HuggingFaceHub` class. Before getting started, the Transformers Python package must be installed. Once installed, users can load their desired model using the `model_id` and `task` and any additional model arguments. Finally, the model can be integrated into an LLMChain by


# What other models can we use?  Popular LLM models compared
## LLM Platforms that can integrate into LangChain
creating a PromptTemplate and LLMChain object and running the input through it. --- [Azure OpenAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/azure_openai_example.html) OpenAI’s models can also be accessed via Microsoft’s Azure platform. --- [AI21](https://python.langchain.com/en/latest/modules/models/llms/integrations/ai21.html) AI21 is a company that offers access to their powerful Jurassic-2 large language models through their API. The API provides access to their `Jurassic-2` model, which has an impressive 178 billion parameters. The API comes at quite a reasonable cost of only $0.01 for every 1k tokens. Developers can easily interact with the AI21 models by creating prompts with LangChain that incorporate input variables. With this simple process, developers can take advantage of their powerful language processing capabilities. --- --- [Aleph Alpha](https://python.langchain.com/en/latest/modules/models/llms/integrations/aleph_alpha.html) Aleph Alpha is a company that offers a family of large language models known as the Luminous series. The Luminous family includes three models, namely Luminous-base, Luminous-extended, and Luminous-supreme, which vary in terms of complexity and capabilities. Aleph Alpha's pricing model is token-based, and the table provides the base prices per model for every 1000 input tokens. The Luminous-base model costs 0.03€ per 1000 input tokens, Luminous-extended costs 0.045€ per 1000 input tokens, Luminous-supreme costs 0.175€ per 1000 input tokens, and Luminous-supreme-control costs 0.21875€ per 1000 input tokens. --- [Banana](https://python.langchain.com/en/latest/modules/models/llms/integrations/banana.html) Banana is a machine learning infrastructure-focused company that provides developers with the tools to build machine learning models. Using LangChain, one can interact with Banana models by installing the Banana package, including an SDK for Python. Next, two following tokens are required: the `BANANA_API_KEY` and the `YOUR_MODEL_KEY`, which can be obtained from their platform. After setting the keys, we can create an object by providing the `YOUR_MODEL_KEY`. It is then possible to integrate the `Banana` model into an LLMChain by creating a `PromptTemplate` and `LLMChain` object and running the desired input through it. --- [CerebriumAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/cerebriumai_example.html) CerebriumAI is an excellent alternative to AWS Sagemaker, providing access to several LLM models through its API. The available pre-trained LLM models include Whisper, MT0, FlanT5, GPT-Neo, Roberta, Pygmalion, Tortoise, and GPT4All. Developers create an instance of CerebriumAI by providing the `endpoint URL` and other relevant parameters such as `max length`, `temperature`, etc. --- [DeepInfra](https://python.langchain.com/en/latest/modules/models/llms/integrations/deepinfra_example.html) DeepInfra is a unique API that offers a range of LLMs, such as distilbert-base-multilingual-cased, bert-base, whisper-large, gpt2, dolly-v2-12b, and more. It is connected to LangChain via API and runs on A100 GPUs that are optimized for inference performance and low latency. Compared to Replicate, DeepInfra's pricing is much more affordable, at $0.0005/second and $0.03/minute. With DeepInfra, we are given a 1-hour free trial of serverless GPU computing to experiment with different models. --- [ForefrontAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/forefrontai_example.html) ForefrontAI is a platform that allows users to fine-tune and utilize various open-source large language models like GPT-J, GPT-NeoX, T5, and more. The platform offers different pricing plans, including the Starter plan for $29/month, which comes with 5 million serverless tokens, 5 fine-tuned models, 1 user, and Discord support. With ForefrontAI, developers have access to various models that can be fine-tuned to suit our specific needs. --- [GooseAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/gooseai_example.html) GooseAI is a fully managed NLP-as-a-Service platform that offers access to various models, including GPT-Neo, Fairseq, and GPT-J. The pricing for GooseAI is based on different model sizes and usage. For the 125M model, the base price for up to 25 tokens is $0.000035 per request, with an additional fee of $0.000001. To use GooseAI with LangChain, you need to install the openai package and set the Environment API Key, which can be obtained from GooseAI. Once you have the API key, you can create a GooseAI instance and define a Prompt Template for Question and Answer. The LLMChain can then be initiated, and you can provide a question


# What other models can we use?  Popular LLM models compared
## LLM Platforms that can integrate into LangChain
to run the LLMChain. --- [Llama-cpp](https://python.langchain.com/en/latest/modules/models/llms/integrations/llamacpp.html) Llama-cpp, a Python binding for llama.cpp, has been seamlessly integrated into the LangChain framework. This integration allows users to access a variety of LLM (Large Language Model) models offered by Llama-cpp, including LLaMA 🦙, Alpaca, GPT4All, Chinese LLaMA / Alpaca, Vigogne (French), Vicuna, Koala, OpenBuddy 🐶 (Multilingual), Pygmalion 7B, and Metharme 7B. With this integration, users have a wide range of options to choose from based on their specific language processing needs. By integrating Llama-cpp into LangChain, users can benefit from the powerful language models and generate humanistic and step-by-step responses to their input questions. --- [Manifest](https://python.langchain.com/en/latest/modules/models/llms/integrations/manifest.html) Manifest is an integration tool that enhances the capabilities of LangChain, making it more powerful and user-friendly for language processing tasks. It acts as a bridge between LangChain and local Hugging Face models, allowing users to access and utilize these models within LangChain easily. Manifest has been seamlessly integrated into LangChain, providing users with enhanced capabilities for language processing tasks. To utilize Manifest within LangChain, users can follow the provided instructions, which involve installing the manifest-ml package and configuring the connection settings. Once integrated, users can leverage Manifest's functionalities alongside LangChain for a comprehensive language processing experience. --- [Modal](https://python.langchain.com/en/latest/modules/models/llms/integrations/modal.html) Modal is seamlessly integrated into LangChain, adding powerful cloud computing capabilities to the language processing workflow. While Modal does not provide any specific language models (LLMs), it serves as the infrastructure enabling LangChain to leverage serverless cloud computing. By integrating Modal into LangChain, users can directly harness the benefits of on-demand access to cloud resources from their Python scripts on their local computers. By installing the Modal client library and generating a new token, users can authenticate and establish a connection to the Modal server. In the LangChain example, a Modal LLM is instantiated using the `endpoint URL`, and a `PromptTemplate` is defined to structure the input. LangChain then executes the LLMChain with the specified prompt and runs a language processing task, such as answering a question. --- [NLP Cloud](https://python.langchain.com/en/latest/modules/models/llms/integrations/nlpcloud.html) NLP Cloud seamlessly integrates with LangChain, providing a comprehensive suite of high-performance pre-trained and custom models for a wide range of natural language processing (NLP) tasks. These models are designed for production use and can be accessed through a REST API. By executing the LLMChain with the specified prompt, users can seamlessly perform NLP tasks like answering questions. --- [Petals](https://python.langchain.com/en/latest/modules/models/llms/integrations/petals_example.html) Petals are seamlessly integrated into LangChain, enabling the utilization of over 100 billion language models within a decentralized architecture similar to BitTorrent. This notebook provides guidance on incorporating Petals into the LangChain workflow. Petals offer a diverse range of language models, and its integration with LangChain enhances natural language understanding and generation capabilities. Petals operate under a decentralized model, providing users with powerful language processing capabilities in a distributed environment. --- [PipelineAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/pipelineai_example.html) PipelineAI is seamlessly integrated into LangChain, allowing users to scale their machine-learning models in the cloud. Additionally, PipelineAI offers API access to a range of LLM (Large Language Model) models. It includes GPT-J, Stable Diffusion, ESRGAN, DALL·E, GPT-2, and GPT-Neo, each with its own specific model parameters and capabilities. PipelineAI empowers users to leverage the scalability and power of the cloud for their machine-learning workflows within the LangChain ecosystem. --- [PredictionGuard](https://python.langchain.com/en/latest/modules/models/llms/integrations/predictionguard.html) PredictionGuard is seamlessly integrated into LangChain, providing users with a powerful wrapper for their language model usage. To begin using PredictionGuard within the LangChain framework, the `predictionguard` and LangChain libraries need to be installed. PredictionGuard can also be seamlessly integrated into LangChain's LLMChain for more advanced tasks. PredictionGuard enhances the LangChain experience by providing an additional layer of control and safety to language model outputs.


# What other models can we use?  Popular LLM models compared
## LLM Platforms that can integrate into LangChain
--- [PromptLayer OpenAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/promptlayer_openai.html) PredictionGuard is seamlessly integrated into LangChain, offering users enhanced control and management of their GPT prompt engineering. PromptLayer acts as a middleware between users' code and OpenAI's Python library, enabling the recording, tracking, and exploration of OpenAI API requests through the PromptLayer dashboard. To utilize PromptLayer with OpenAI, the 'promptlayer' package needs to be installed. Users can attach templates to requests, enabling the evaluation of different templates and models within the PromptLayer dashboard. --- [Replicate](https://python.langchain.com/en/latest/modules/models/llms/integrations/replicate.html) Replicate is seamlessly integrated into LangChain, providing a wide range of LLM models for various applications. Some of the LLM models offered by Replicate include vicuna-13b, bark, speaker-transcription, stablelm-tuned-alpha-7b, Kandinsky-2, and stable-diffusion. These models cover diverse areas such as language generation, generative audio, speaker transcription, language modeling, and text-to-image generation. Each model has specific parameters and capabilities, enabling users to choose the most suitable model for their needs. Replicate provides flexible pricing options based on the computational resources required for running the models. Replication simplifies the deployment of custom machine-learning models at scale. Users can integrate Replicate into LangChain to interact with these models effectively. --- [Runhouse](https://python.langchain.com/en/latest/modules/models/llms/integrations/runhouse.html) Runhouse is seamlessly integrated into LangChain, providing powerful remote compute and data management capabilities across different environments and users. Runhouse offers the flexibility to host models on your own GPU infrastructure or leverage on-demand GPUs from cloud providers such as AWS, GCP, and Azure. Runhouse provides several LLM models that can be utilized within LangChain, such as gpt2 and google/flan-t5-small. Users can specify the desired hardware configuration. By combining Runhouse and LangChain, users can easily create advanced language model workflows, enabling efficient model execution and collaboration across different environments and users. --- [StochasticAI](https://python.langchain.com/en/latest/modules/models/llms/integrations/stochasticai.html) StochasticAI aims to simplify the workflow of deep learning models within LangChain, providing users with an efficient and user-friendly environment for model interaction and deployment. It provides a streamlined process for the lifecycle management of Deep Learning models. StochasticAI's Acceleration Platform simplifies tasks such as model uploading, versioning, training, compression, and acceleration, ultimately facilitating the deployment of models into production. Within LangChain, users can interact with StochasticAI models effortlessly. The available LLM models from StochasticAI include FLAN-T5, GPT-J, Stable Diffusion 1, and Stable Diffusion 2. These models offer diverse capabilities for various language-related tasks. --- [Writer](https://python.langchain.com/en/latest/modules/models/llms/integrations/writer.html) The writer is seamlessly integrated into LangChain, providing users with a powerful platform for generating diverse language content. With Writer integration, LangChain users can effortlessly interact with a range of LLM models to meet their language generation needs. The available LLM models provided by Writer include Palmyra Small (128m), Palmyra 3B (3B), Palmyra Base (5B), Camel 🐪 (5B), Palmyra Large (20B), InstructPalmyra (30B), Palmyra-R (30B), Palmyra-E (30B), and Silk Road. These models offer different capacities for improving language understanding, generative pre-training, following instructions, and retrieval-augmented generation. --- ### Conclusion It's understandable to feel overwhelmed by the number of choices when integrating the mentioned foundational models. That's why, in this lesson, we have explained the different paths one can take. This information can be a valuable reference tool to help make an informed decision. Depending on your requirements, you may host the model locally or opt for a pay-as-you-go service. The former will enable you to have complete control over the model's implementation, while the latter can be more cost-effective for those with limited resources. Whatever your preference, choosing the option that best suits your needs and budget is essential. Good job on completing all the lessons of the first module of the course! Head up to the module quiz to test your new knowledge. After that, the following module focuses on prompting, i.e. the best way


# What other models can we use?  Popular LLM models compared
## LLM Platforms that can integrate into LangChain
of giving instructions to LLMs. *Intel, the Intel logo and Xeon are trademarks of Intel Corporation or its subsidiaries.* ---


# Exploring the World of Language Models: LLMs vs Chat Models
### Introduction Large Language Models have made significant advancements in the field of Natural Language Processing (NLP), enabling AI systems to understand and generate human-like text. ChatGPT is a popular language model based on Transformers architecture, enabling it to understand long texts and figure out how words or ideas are connected. It's great at making predictions about language and relationships between words. LLMs and Chat Models are two types of models in LangChain, serving different purposes in natural language processing tasks. This lesson will examine the differences between LLMs and Chat Models, their unique use cases, and how they are implemented within LangChain.