Docugami | 🦜🔗 Langchain

orebucks LLC_AZ.pdf', 'structure': 'h1 h1 p', 'tag': 'chunk Lease', 'Lease Date': 'March  29th , 2019', 'Landlord': 'Menlo Group', 'Tenant': 'Shorebucks LLC', 'Premises Address': '1564  E Broadway Rd ,  Tempe ,  Arizona  85282', 'Term of Lease': '96  full calendar months', 'Square Feet': '16,159'}  We can use a self-querying retriever to improve our query accuracy, using this additional metadata:  poetry run pip install --upgrade lark --quiet  from langchain.chains.query_constructor.schema import AttributeInfo from langchain.retrievers.self_query.base import SelfQueryRetriever from langchain.vectorstores.chroma import Chroma  EXCLUDE_KEYS = ["id", "xpath", "structure"] metadata_field_info = [     AttributeInfo(         name=key,         description=f"The {key} for this chunk",         type="string",     )     for key in chunks[0].metadata     if key.lower() not in EXCLUDE_KEYS ]  document_content_description = "Contents of this chunk" llm = OpenAI(temperature=0)  vectordb = Chroma.from_documents(documents=chunks, embedding=embedding) retriever = SelfQueryRetriever.from_llm(     llm, vectordb, document_content_description, metadata_field_info, verbose=True ) qa_chain = RetrievalQA.from_chain_type(     llm=OpenAI(),     chain_type="stuff",     retriever=retriever,     return_source_documents=True,     verbose=True, )  Let's run the same question again. It returns the correct result since all the chunks have metadata key/value pairs on them carrying key information about the document even if this information is physically very far away from the source chunk used to generate the answer.  qa_chain(     "What is rentable area for the property owned by DHA Group?" )  # correct answer should be 13,500 sq ft                > Entering new RetrievalQA chain...          > Finished chain.          {'query': 'What is rentable area for the property owned by DHA Group?',      'result': ' The rentable area of the property owned by DHA Group is 13,500 square feet.',      'source_documents': [Document(page_content='1.6 Rentable Area of the Premises.', metadata={'Landlord': 'DHA Group', 'Lease Date': 'March  29th , 2019', 'Premises Address': '111  Bauer Dr ,  Oakland ,  New Jersey ,  07436', 'Square Feet': '13,500', 'Tenant': 'Shorebucks LLC', 'Term of Lease': '84  full calendar  months', 'id': '5b39a1ae84d51682328dca1467be211f', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'lim h1', 'tag': 'chunk', 'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/dg:chunk/dg:chunk/docset:BasicLeaseInformation/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS-section/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS/docset:DhaGroup/dg:chunk[6]/dg:chunk'}),       Document(page_content='<RentableAreaofthePremises><SquareFeet>13,500 </SquareFeet>square feet. This square footage figure includes an add-on factor for Common Areas in the Building and has been agreed upon by the parties as final and correct and is not subject to challenge or dispute by either party. </RentableAreaofthePremises>', metadata={'Landlord': 'DHA Group', 'Lease Date': 'March  29th , 2019', 'Premises Address': '111  Bauer Dr ,  Oakland ,  New Jersey ,  07436', 'Square Feet': '13,500', 'Tenant': 'Shorebucks LLC', 'Term of Lease': '84  full calendar  months', 'id': '4c06903d087f5a83e486ee42cd702d31', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'div', 'tag': 'RentableAreaofthePremises', 'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/dg:chunk/dg:chunk/docset:BasicLeaseInformation/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS-section/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS/docset:DhaGroup/dg:chunk[6]/docset:RentableAreaofthePremises-section/docset:RentableAreaofthePremises'}),       Document(page_content='<TheTermAnnualMarketRent>shall mean (i) for the initial Lease Year (“Year 1”) <Money>$2,239,748.00 </Money>per year (i.e., the product of the Rentable Area of the Premises multiplied by <Money>$82.00</Money>) (the “Year 1 Market Rent Hurdle”); (ii) for the Lease Year thereafter, <Percent>one hundred three percent (103%) </Percent>of the Year 1 Market Rent Hurdle, and (iii) for each Lease Year thereafter until the termination or expiration of this Lease, the Annual Market Rent Threshold shall be <AnnualMarketRentThreshold>one hundred three percent (103%) </AnnualMarketRentThreshold>of the Annual Market Rent Threshold for the immediately prior Lease Year. </TheTermAnnualMarketRent>', metadata={'Landlord': 'DHA Group', 'Lease Date': 'March  29th , 2019', 'Premises Address': '111  Bauer Dr ,  Oakland ,  New Jersey ,  07436', 'Square Feet': '13,500', 'Tenant': 'Shorebucks LLC', 'Term of Lease': '84  full calendar  months', 'id': '6b90beeadace5d4d12b25706fb48e631', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'div', 'tag': 'TheTermAnnualMarketRent', 'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/docset:GrossRentCredit-section/docset:GrossRentCredit/dg:chunk/dg:chunk/dg:chunk/dg:chunk[2]/docset:PercentageRent/dg:chunk[2]/dg:chunk[2]/docset:TenantSRevenue/dg:chunk[2]/docset:TenantSRevenue/dg:chunk[3]/docset:TheTermAnnualMarketRent-section/docset:TheTermAnnualMarketRent'}),       Document(page_content='1.11 Percentage Rent.\n (a) <GrossRevenue><Percent>55% </Percent>of Gross Revenue to Landlord until Landlord receives Percentage Rent in an amount equal to the Annual Market Rent Hurdle (as escalated); and </GrossRevenue>', metadata={'Landlord': 'DHA Group', 'Lease Date': 'March  29th , 2019', 'Premises Address': '111  Bauer Dr ,  Oakland ,  New Jersey ,  07436', 'Square Feet': '13,500', 'T


Docugami | 🦜🔗 Langchain

enant': 'Shorebucks LLC', 'Term of Lease': '84  full calendar  months', 'id': 'c8bb9cbedf65a578d9db3f25f519dd3d', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'lim h1 lim p', 'tag': 'chunk GrossRevenue', 'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/docset:GrossRentCredit-section/docset:GrossRentCredit/dg:chunk/dg:chunk/dg:chunk/docset:PercentageRent/dg:chunk[1]/dg:chunk[1]'})]}  This time the answer is correct, since the self-querying retriever created a filter on the landlord attribute of the metadata, correctly filtering to document that specifically is about the DHA Group landlord. The resulting source chunks are all relevant to this landlord, and this improves answer accuracy even though the landlord is not directly mentioned in the specific chunk that contains the correct answer.  Advanced Topic: Small-to-Big Retrieval with Document Knowledge Graph Hierarchy Documents are inherently semi-structured and the DocugamiLoader is able to navigate the semantic and structural contours of the document to provide parent chunk references on the chunks it returns. This is useful e.g. with the MultiVector Retriever for small-to-big retrieval.  To get parent chunk references, you can set loader.parent_hierarchy_levels to a non-zero value.  from typing import Dict, List  from langchain.document_loaders import DocugamiLoader from langchain.schema.document import Document  loader = DocugamiLoader(docset_id="zo954yqy53wp") loader.include_xml_tags = (     True  # for additional semantics from the Docugami knowledge graph ) loader.parent_hierarchy_levels = 3  # for expanded context loader.max_text_length = (     1024 * 8 )  # 8K chars are roughly 2K tokens (ref: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them) loader.include_project_metadata_in_doc_metadata = (     False  # Not filtering on vector metadata, so remove to lighten the vectors ) chunks: List[Document] = loader.load()  # build separate maps of parent and child chunks parents_by_id: Dict[str, Document] = {} children_by_id: Dict[str, Document] = {} for chunk in chunks:     chunk_id = chunk.metadata.get("id")     parent_chunk_id = chunk.metadata.get(loader.parent_id_key)     if not parent_chunk_id:         # parent chunk         parents_by_id[chunk_id] = chunk     else:         # child chunk         children_by_id[chunk_id] = chunk  # Explore some of the parent chunk relationships for id, chunk in list(children_by_id.items())[:5]:     parent_chunk_id = chunk.metadata.get(loader.parent_id_key)     if parent_chunk_id:         # child chunks have the parent chunk id set         print(f"PARENT CHUNK {parent_chunk_id}: {parents_by_id[parent_chunk_id]}")         print(f"CHUNK {id}: {chunk}")      PARENT CHUNK 7df09fbfc65bb8377054808aac2d16fd: page_content='OFFICE LEASE\n THIS OFFICE LEASE\n <Lease>(the "Lease") is made and entered into as of <LeaseDate>March 29th, 2019</LeaseDate>, by and between Landlord and Tenant. "Date of this Lease" shall mean the date on which the last one of the Landlord and Tenant has signed this Lease. </Lease>\nW I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>\n1. BASIC LEASE INFORMATION AND DEFINED TERMS.\nThe key business terms of this Lease and the defined terms used in this Lease are as follows:' metadata={'xpath': '/docset:OFFICELEASE-section/dg:chunk', 'id': '7df09fbfc65bb8377054808aac2d16fd', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'h1 h1 p h1 p lim h1 p', 'tag': 'chunk Lease chunk TheTerms'}     CHUNK 47297e277e556f3ce8b570047304560b: page_content='OFFICE LEASE\n THIS OFFICE LEASE\n <Lease>(the "Lease") is made and entered into as of <LeaseDate>March 29th, 2019</LeaseDate>, by and between Landlord and Tenant. "Date of this Lease" shall mean the date on which the last one of the Landlord and Tenant has signed this Lease. </Lease>' metadata={'xpath': '/docset:OFFICELEASE-section/dg:chunk', 'id': '47297e277e556f3ce8b570047304560b', 'name': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_NJ.pdf', 'structure': 'h1 h1 p', 'tag': 'chunk Lease', 'doc_id': '7df09fbfc65bb8377054808aac2d16fd'}     PARENT CHUNK bb84925da3bed22c30ea1bdc173ff54f: page_content='OFFICE LEASE\n THIS OFFICE LEASE\n <Lease>(the "Lease") is made and entered into as of <LeaseDate>January 8th, 2018</LeaseDate>, by and between Landlord and Tenant. "Date of this Lease" shall mean the date on which the last one of the Landlord and Tenant has signed this Lease. </Lease>\nW I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>\n1. BASIC LEASE INFORMATION AND DEFINED TERMS.\nThe key business terms of this Lease and the defined terms used in this Lease are as follows:\n1.1 Landlord.\n <Landlord>Catalyst Group LLC </Landlord>' metadata={'xpath': '/docset:OFFICELEASE-section/dg:chunk', 'id': 'bb84925da3bed22c30ea1bdc173ff54f', 'name': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'h1 h1 p h1 p lim h1 p lim h1 div', 'tag': 'chunk Lease chunk TheTerms chunk Landlord'}     CHUNK 2f1746cbd546d1d61a9250c50de7a7fa: page_content='W I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>' metadata={'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/dg:chunk', 'id': '2f1746cbd546d1d61a9250c50de7a7fa', 'name': 'Sample Commercial Leases/Shorebucks


Docugami | 🦜🔗 Langchain

LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'h1 p', 'tag': 'chunk TheTerms', 'doc_id': 'bb84925da3bed22c30ea1bdc173ff54f'}     PARENT CHUNK 0b0d765b6e504a6ba54fa76b203e62ec: page_content='OFFICE LEASE\n THIS OFFICE LEASE\n <Lease>(the "Lease") is made and entered into as of <LeaseDate>January 8th, 2018</LeaseDate>, by and between Landlord and Tenant. "Date of this Lease" shall mean the date on which the last one of the Landlord and Tenant has signed this Lease. </Lease>\nW I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>\n1. BASIC LEASE INFORMATION AND DEFINED TERMS.\nThe key business terms of this Lease and the defined terms used in this Lease are as follows:\n1.1 Landlord.\n <Landlord>Catalyst Group LLC </Landlord>\n1.2 Tenant.\n <Tenant>Shorebucks LLC </Tenant>' metadata={'xpath': '/docset:OFFICELEASE-section/dg:chunk', 'id': '0b0d765b6e504a6ba54fa76b203e62ec', 'name': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'h1 h1 p h1 p lim h1 p lim h1 div lim h1 div', 'tag': 'chunk Lease chunk TheTerms chunk Landlord chunk Tenant'}     CHUNK b362dfe776ec5a7a66451a8c7c220b59: page_content='1. BASIC LEASE INFORMATION AND DEFINED TERMS.' metadata={'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/dg:chunk/dg:chunk/docset:BasicLeaseInformation/dg:chunk', 'id': 'b362dfe776ec5a7a66451a8c7c220b59', 'name': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'lim h1', 'tag': 'chunk', 'doc_id': '0b0d765b6e504a6ba54fa76b203e62ec'}     PARENT CHUNK c942010baaf76aa4d4657769492f6edb: page_content='OFFICE LEASE\n THIS OFFICE LEASE\n <Lease>(the "Lease") is made and entered into as of <LeaseDate>January 8th, 2018</LeaseDate>, by and between Landlord and Tenant. "Date of this Lease" shall mean the date on which the last one of the Landlord and Tenant has signed this Lease. </Lease>\nW I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>\n1. BASIC LEASE INFORMATION AND DEFINED TERMS.\nThe key business terms of this Lease and the defined terms used in this Lease are as follows:\n1.1 Landlord.\n <Landlord>Catalyst Group LLC </Landlord>\n1.2 Tenant.\n <Tenant>Shorebucks LLC </Tenant>\n1.3 Building.\n <Building>The building containing the Premises located at <PremisesAddress><PremisesStreetAddress><MainStreet>600 </MainStreet><StreetName>Main Street</StreetName></PremisesStreetAddress>, <City>Bellevue</City>, <State>WA</State>, <Premises>98004</Premises></PremisesAddress>. The Building is located within the Project. </Building>' metadata={'xpath': '/docset:OFFICELEASE-section/dg:chunk', 'id': 'c942010baaf76aa4d4657769492f6edb', 'name': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'h1 h1 p h1 p lim h1 p lim h1 div lim h1 div lim h1 div', 'tag': 'chunk Lease chunk TheTerms chunk Landlord chunk Tenant chunk Building'}     CHUNK a95971d693b7aa0f6640df1fbd18c2ba: page_content='The key business terms of this Lease and the defined terms used in this Lease are as follows:' metadata={'xpath': '/docset:OFFICELEASE-section/docset:OFFICELEASE-section/docset:OFFICELEASE/docset:WITNESSETH-section/docset:WITNESSETH/dg:chunk/dg:chunk/docset:BasicLeaseInformation/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS-section/docset:BASICLEASEINFORMATIONANDDEFINEDTERMS/dg:chunk', 'id': 'a95971d693b7aa0f6640df1fbd18c2ba', 'name': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'source': 'Sample Commercial Leases/Shorebucks LLC_WA.pdf', 'structure': 'p', 'tag': 'chunk', 'doc_id': 'c942010baaf76aa4d4657769492f6edb'}     PARENT CHUNK f34b649cde7fc4ae156849a56d690495: page_content='W I T N E S S E T H\n <TheTerms> Subject to and on the terms and conditions of this Lease, Landlord leases to Tenant and Tenant hires from Landlord the Premises. </TheTerms>\n1. BASIC LEASE INFORMATION AND DEFINED TERMS.\n<BASICLEASEINFORMATIONANDDEFINEDTERMS>The key business terms of this Lease and the defined terms used in this Lease are as follows: </BASICLEASEINFORMATIONANDDEFINEDTERMS>\n1.1 Landlord.\n <Landlord><Landlord>Menlo Group</Landlord>, a <USState>Delaware </USState>limited liability company authorized to transact business in <USState>Arizona</USState>. </Landlord>\n1.2 Tenant.\n <Tenant>Shorebucks LLC </Tenant>\n1.3 Building.\n <Building>The building containing the Premises located at <PremisesAddress><PremisesStreetAddress><Premises>1564 </Premises><Premises>E Broadway Rd</Premises></PremisesStreetAddress>, <City>Tempe</City>, <USState>Arizona </USState><Premises>85282</Premises></PremisesAddress>. The Building is located within the Project. </Building>\n1.4 Project.\n <Project>The parcel of land and the buildings and improvements located on such land known as Shorebucks Office <ShorebucksOfficeAddress><ShorebucksOfficeStreetAddress><ShorebucksOffice>6 </ShorebucksOffice><ShorebucksOffice6>located at <Number>1564 </Number>E Broadway Rd</ShorebucksOffice6></ShorebucksOfficeStreetAddress>, <City>Tempe</City>, <USState>Arizona </USState><Number>85282</Number></ShorebucksOfficeAddress>. The Project is legally described in EXHIBIT "A" to this Lease. </Project>' metadata={'xpath': '/dg:chunk/docset:WITNESSETH-section/dg:chunk', 'id': 'f34b649cde7fc4ae156849a56d690495', 'name': 'Sample Commercial Leases/Shorebucks LLC_AZ.docx', 'source': 'Sample Commercial Leases/Shorebucks LLC_AZ.docx', 'structure': 'h1 p lim h1 div lim h1 div lim h1 div lim h1 div lim h1 div', 'tag': 'chunk TheTerms BASICLEASEINFORMATIONANDDEFINEDTERMS chunk Landlord chunk Tenant chunk Building chunk Project'}


Docugami | 🦜🔗 Langchain

CHUNK 21b4d9517f7ccdc0e3a028ce5043a2a0: page_content='1.1 Landlord.\n <Landlord><Landlord>Menlo Group</Landlord>, a <USState>Delaware </USState>limited liability company authorized to transact business in <USState>Arizona</USState>. </Landlord>' metadata={'xpath': '/dg:chunk/docset:WITNESSETH-section/docset:WITNESSETH/dg:chunk[1]/dg:chunk[1]/dg:chunk/dg:chunk[2]/dg:chunk', 'id': '21b4d9517f7ccdc0e3a028ce5043a2a0', 'name': 'Sample Commercial Leases/Shorebucks LLC_AZ.docx', 'source': 'Sample Commercial Leases/Shorebucks LLC_AZ.docx', 'structure': 'lim h1 div', 'tag': 'chunk Landlord', 'doc_id': 'f34b649cde7fc4ae156849a56d690495'}  from langchain.embeddings import OpenAIEmbeddings from langchain.retrievers.multi_vector import MultiVectorRetriever, SearchType from langchain.storage import InMemoryStore from langchain.vectorstores.chroma import Chroma  # The vectorstore to use to index the child chunks vectorstore = Chroma(collection_name="big2small", embedding_function=OpenAIEmbeddings())  # The storage layer for the parent documents store = InMemoryStore()  # The retriever (empty to start) retriever = MultiVectorRetriever(     vectorstore=vectorstore,     docstore=store,     search_type=SearchType.mmr,  # use max marginal relevance search     search_kwargs={"k": 2}, )  # Add child chunks to vector store retriever.vectorstore.add_documents(list(children_by_id.values()))  # Add parent chunks to docstore retriever.docstore.mset(parents_by_id.items())  # Query vector store directly, should return chunks found_chunks = vectorstore.similarity_search(     "what signs does Birch Street allow on their property?", k=2 )  for chunk in found_chunks:     print(chunk.page_content)     print(chunk.metadata[loader.parent_id_key])      24. SIGNS.      <SIGNS>No signage shall be placed by Tenant on any portion of the Project. However, Tenant shall be permitted to place a sign bearing its name in a location approved by Landlord near the entrance to the Premises (at Tenant's cost) and will be furnished a single listing of its name in the Building's directory (at Landlord's cost), all in accordance with the criteria adopted <Frequency>from time to time </Frequency>by Landlord for the Project. Any changes or additional listings in the directory shall be furnished (subject to availability of space) for the then Building Standard charge. </SIGNS>     43090337ed2409e0da24ee07e2adbe94     <TheExterior> Tenant agrees that all signs, awnings, protective gates, security devices and other installations visible from the exterior of the Premises shall be subject to Landlord's prior written approval, shall be subject to the prior approval of the <Org>Landmarks </Org><Landmarks>Preservation Commission </Landmarks>of the City of <USState>New <Org>York</Org></USState>, if required, and shall not interfere with or block either of the adjacent stores, provided, however, that Landlord shall not unreasonably withhold consent for signs that Tenant desires to install. Tenant agrees that any permitted signs, awnings, protective gates, security devices, and other installations shall be installed at Tenant’s sole cost and expense professionally prepared and dignified and subject to Landlord's prior written approval, which shall not be unreasonably withheld, delayed or conditioned, and subject to such reasonable rules and restrictions as Landlord <Frequency>from time to time </Frequency>may impose. Tenant shall submit to Landlord drawings of the proposed signs and other installations, showing the size, color, illumination and general appearance thereof, together with a statement of the manner in which the same are to be affixed to the Premises. Tenant shall not commence the installation of the proposed signs and other installations unless and until Landlord shall have approved the same in writing. . Tenant shall not install any neon sign. The aforesaid signs shall be used solely for the purpose of identifying Tenant's business. No changes shall be made in the signs and other installations without first obtaining Landlord's prior written consent thereto, which consent shall not be unreasonably withheld, delayed or conditioned. Tenant shall, at its own cost and expense, obtain and exhibit to Landlord such permits or certificates of approval as Tenant may be required to obtain from any and all City, State and other authorities having jurisdiction covering the erection, installation, maintenance or use of said signs or other installations, and Tenant shall maintain the said signs and other installations together with any appurtenances thereto in good order and condition and to the satisfaction of the Landlord and in accordance with any and all orders, regulations, requirements and rules of any public authorities having jurisdiction thereover. Landlord consents to Tenant’s Initial Signage described in annexed Exhibit D. </TheExterior>     54ddfc3e47f41af7e747b2bc439ea96b  # Query retriever, should return parents (using MMR since that was set as search_type above) retrieved_parent_docs = retriever.get_relevant_documents(     "what signs does Birch Street allow on their property?" ) for chunk in retrieved_parent_docs:     print(chunk.page_content)     print(chunk.metadata["id"])      21. SERVICES AND UTILITIES.      <SERVICESANDUTILITIES>Landlord shall have no obligation to provide any utilities or services to the Premises other than passenger elevator service to the Premises. Tenant shall be solely responsible for and shall promptly pay all charges for water, electricity, or any other utility used or consumed in the Premises, including all costs associated with separately metering for the Premises. Tenant shall be responsible for repairs and maintenance to exit lighting, emergency lighting, and fire extinguishers for the Premises. Tenant is responsible for interior janitorial, pest control, and waste removal services. Landlord may at any time change the electrical utility provider for the Building. Tenant’s use of electrical, HVAC, or othe


Docugami | 🦜🔗 Langchain

r services furnished by Landlord shall not exceed, either in voltage, rated capacity, use, or overall load, that which Landlord deems to be standard for the Building. In no event shall Landlord be liable for damages resulting from the failure to furnish any service, and any interruption or failure shall in no manner entitle Tenant to any remedies including abatement of Rent. If at any time during the Lease Term the Project has any type of card access system for the Parking Areas or the Building, Tenant shall purchase access cards for all occupants of the Premises from Landlord at a Building Standard charge and shall comply with Building Standard terms relating to access to the Parking Areas and the Building. </SERVICESANDUTILITIES>     22. SECURITY DEPOSIT.      <SECURITYDEPOSIT>The Security Deposit shall be held by Landlord as security for Tenant's full and faithful performance of this Lease including the payment of Rent. Tenant grants Landlord a security interest in the Security Deposit. The Security Deposit may be commingled with other funds of Landlord and Landlord shall have no liability for payment of any interest on the Security Deposit. Landlord may apply the Security Deposit to the extent required to cure any default by Tenant. If Landlord so applies the Security Deposit, Tenant shall deliver to Landlord the amount necessary to replenish the Security Deposit to its original sum within <Deliver>five days </Deliver>after notice from Landlord. The Security Deposit shall not be deemed an advance payment of Rent or a measure of damages for any default by Tenant, nor shall it be a defense to any action that Landlord may bring against Tenant. </SECURITYDEPOSIT>     23. GOVERNMENTAL REGULATIONS.      <GOVERNMENTALREGULATIONS>Tenant, at Tenant's sole cost and expense, shall promptly comply (and shall cause all subtenants and licensees to comply) with all laws, codes, and ordinances of governmental authorities, including the Americans with Disabilities Act of <AmericanswithDisabilitiesActDate>1990 </AmericanswithDisabilitiesActDate>as amended (the "ADA"), and all recorded covenants and restrictions affecting the Project, pertaining to Tenant, its conduct of business, and its use and occupancy of the Premises, including the performance of any work to the Common Areas required because of Tenant's specific use (as opposed to general office use) of the Premises or Alterations to the Premises made by Tenant. </GOVERNMENTALREGULATIONS>     24. SIGNS.      <SIGNS>No signage shall be placed by Tenant on any portion of the Project. However, Tenant shall be permitted to place a sign bearing its name in a location approved by Landlord near the entrance to the Premises (at Tenant's cost) and will be furnished a single listing of its name in the Building's directory (at Landlord's cost), all in accordance with the criteria adopted <Frequency>from time to time </Frequency>by Landlord for the Project. Any changes or additional listings in the directory shall be furnished (subject to availability of space) for the then Building Standard charge. </SIGNS>     25. BROKER.      <BROKER>Landlord and Tenant each represent and warrant that they have neither consulted nor negotiated with any broker or finder regarding the Premises, except the Landlord's Broker and Tenant's Broker. Tenant shall indemnify, defend, and hold Landlord harmless from and against any claims for commissions from any real estate broker other than Landlord's Broker and Tenant's Broker with whom Tenant has dealt in connection with this Lease. Landlord shall indemnify, defend, and hold Tenant harmless from and against payment of any leasing commission due Landlord's Broker and Tenant's Broker in connection with this Lease and any claims for commissions from any real estate broker other than Landlord's Broker and Tenant's Broker with whom Landlord has dealt in connection with this Lease. The terms of this article shall survive the expiration or earlier termination of this Lease. </BROKER>     26. END OF TERM.      <ENDOFTERM>Tenant shall surrender the Premises to Landlord at the expiration or sooner termination of this Lease or Tenant's right of possession in good order and condition, broom-clean, except for reasonable wear and tear. All Alterations made by Landlord or Tenant to the Premises shall become Landlord's property on the expiration or sooner termination of the Lease Term. On the expiration or sooner termination of the Lease Term, Tenant, at its expense, shall remove from the Premises all of Tenant's personal property, all computer and telecommunications wiring, and all Alterations that Landlord designates by notice to Tenant. Tenant shall also repair any damage to the Premises caused by the removal. Any items of Tenant's property that shall remain in the Premises after the expiration or sooner termination of the Lease Term, may, at the option of Landlord and without notice, be deemed to have been abandoned, and in that case, those items may be retained by Landlord as its property to be disposed of by Landlord, without accountability or notice to Tenant or any other party, in the manner Landlord shall determine, at Tenant's expense. </ENDOFTERM>     27. ATTORNEYS' FEES.      <ATTORNEYSFEES>Except as otherwise provided in this Lease, the prevailing party in any litigation or other dispute resolution proceeding, including arbitration, arising out of or in any manner based on or relating to this Lease, including tort actions and actions for injunctive, declaratory, and provisional relief, shall be entitled to recover from the losing party actual attorneys' fees and costs, including fees for litigating the entitlement to or amount of fees or costs owed under this provision, and fees in connection with bankruptcy, appellate, or collection proceedings. No person or entity other than Landlord or Tenant has any right to recover fees under this paragraph. In addition, if Landlord becomes a party to any suit or proceeding affecting the Premise


Docugami | 🦜🔗 Langchain

s or involving this Lease or Tenant's interest under this Lease, other than a suit between Landlord and Tenant, or if Landlord engages counsel to collect any of the amounts owed under this Lease, or to enforce performance of any of the agreements, conditions, covenants, provisions, or stipulations of this Lease, without commencing litigation, then the costs, expenses, and reasonable attorneys' fees and disbursements incurred by Landlord shall be paid to Landlord by Tenant. </ATTORNEYSFEES>     43090337ed2409e0da24ee07e2adbe94     <TenantsSoleCost> Tenant, at Tenant's sole cost and expense, shall be responsible for the removal and disposal of all of garbage, waste, and refuse from the Premises on a <Frequency>daily </Frequency>basis. Tenant shall cause all garbage, waste and refuse to be stored within the Premises until <Stored>thirty (30) minutes </Stored>before closing, except that Tenant shall be permitted, to the extent permitted by law, to place garbage outside the Premises after the time specified in the immediately preceding sentence for pick up prior to <PickUp>6:00 A.M. </PickUp>next following. Garbage shall be placed at the edge of the sidewalk in front of the Premises at the location furthest from he main entrance to the Building or such other location in front of the Building as may be specified by Landlord. </TenantsSoleCost>     <ItsSoleCost> Tenant, at its sole cost and expense, agrees to use all reasonable diligence in accordance with the best prevailing methods for the prevention and extermination of vermin, rats, and mice, mold, fungus, allergens, <Bacterium>bacteria </Bacterium>and all other similar conditions in the Premises. Tenant, at Tenant's expense, shall cause the Premises to be exterminated <Exterminated>from time to time </Exterminated>to the reasonable satisfaction of Landlord and shall employ licensed exterminating companies. Landlord shall not be responsible for any cleaning, waste removal, janitorial, or similar services for the Premises, and Tenant sha ll not be entitled to seek any abatement, setoff or credit from the Landlord in the event any conditions described in this Article are found to exist in the Premises. </ItsSoleCost>     42B. Sidewalk Use and Maintenance     <TheSidewalk> Tenant shall, at its sole cost and expense, keep the sidewalk in front of the Premises 18 inches into the street from the curb clean free of garbage, waste, refuse, excess water, snow, and ice and Tenant shall pay, as additional rent, any fine, cost, or expense caused by Tenant's failure to do so. In the event Tenant operates a sidewalk café, Tenant shall, at its sole cost and expense, maintain, repair, and replace as necessary, the sidewalk in front of the Premises and the metal trapdoor leading to the basement of the Premises, if any. Tenant shall post warning signs and cones on all sides of any side door when in use and attach a safety bar across any such door at all times when open. </TheSidewalk>     <Display> In no event shall Tenant use, or permit to be used, the space adjacent to or any other space outside of the Premises, for display, sale or any other similar undertaking; except [1] in the event of a legal and licensed “street fair” type program or [<Number>2</Number>] if the local zoning, Community Board [if applicable] and other municipal laws, rules and regulations, allow for sidewalk café use and, if such I s the case, said operation shall be in strict accordance with all of the aforesaid requirements and conditions. . In no event shall Tenant use, or permit to be used, any advertising medium and/or loud speaker and/or sound amplifier and/or radio or television broadcast which may be heard outside of the Premises or which does not comply with the reasonable rules and regulations of Landlord which then will be in effect. </Display>     42C. Store Front Maintenance      <TheBulkheadAndSecurityGate> Tenant agrees to wash the storefront, including the bulkhead and security gate, from the top to the ground, monthly or more often as Landlord reasonably requests and make all repairs and replacements as and when deemed necessary by Landlord, to all windows and plate and ot her glass in or about the Premises and the security gate, if any. In case of any default by Tenant in maintaining the storefront as herein provided, Landlord may do so at its own expense and bill the cost thereof to Tenant as additional rent. </TheBulkheadAndSecurityGate>     42D. Music, Noise, and Vibration     4474c92ae7ccec9184ed2fef9f072734  Previous Discord Next Docusaurus Community Discord Twitter GitHub Python JS/TS More Homepage Blog Copyright © 2023 LangChain, Inc.


Docusaurus | 🦜🔗 Langchain

Main:  On this page #Docusaurus [Docusaurus](https://docusaurus.io/) is a static-site generator which provides out-of-the-box documentation features.  By utilizing the existing SitemapLoader, this loader scans and loads all pages from a given Docusaurus application and returns the main documentation content of each page as a Document.  from langchain.document_loaders import DocusaurusLoader  Install necessary dependencies  pip install -U beautifulsoup4 lxml  # fixes a bug with asyncio and jupyter import nest_asyncio  nest_asyncio.apply()  loader = DocusaurusLoader("https://python.langchain.com")  docs = loader.load()      Fetching pages: 100%|##########| 939/939 [01:19<00:00, 11.85it/s]  SitemapLoader also provides the ability to utilize and tweak concurrency which can help optimize the time it takes to load the source documentation. Refer to the [sitemap docs](/docs/integrations/document_loaders/sitemap) for more info.  docs[0]      Document(page_content="\n\n\n\n\nCookbook | 🦜🔗 Langchain\n\n\n\n\n\n\nSkip to main content🦜🔗 LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKCookbookThe page you're looking for has been moved to the cookbook section of the repo as a notebook.CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright © 2023 LangChain, Inc.\n\n\n\n", metadata={'source': 'https://python.langchain.com/cookbook', 'loc': 'https://python.langchain.com/cookbook', 'changefreq': 'weekly', 'priority': '0.5'})  ##Filtering sitemap URLs[](#filtering-sitemap-urls) Sitemaps can contain thousands of URLs and ften you don't need every single one of them. You can filter the URLs by passing a list of strings or regex patterns to the url_filter parameter. Only URLs that match one of the patterns will be loaded.  loader = DocusaurusLoader(     "https://python.langchain.com",     filter_urls=[         "https://python.langchain.com/docs/integrations/document_loaders/sitemap"     ], ) documents = loader.load()      Fetching pages: 100%|##########| 1/1 [00:00<00:00,  5.21it/s]  documents[0]      Document(page_content='\n\n\n\n\nSitemap | 🦜🔗 Langchain\n\n\n\n\n\n\nSkip to main content🦜🔗 LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsChat modelsDocument loadersacreomAirbyte CDKAirbyte GongAirbyte HubspotAirbyte JSONAirbyte SalesforceAirbyte ShopifyAirbyte StripeAirbyte TypeformAirbyte Zendesk SupportAirtableAlibaba Cloud MaxComputeApify DatasetArcGISArxivAssemblyAI Audio TranscriptsAsync ChromiumAsyncHtmlAWS S3 DirectoryAWS S3 FileAZLyricsAzure Blob Storage ContainerAzure Blob Storage FileAzure Document IntelligenceBibTeXBiliBiliBlackboardBlockchainBrave SearchBrowserlessChatGPT DataCollege ConfidentialConcurrent LoaderConfluenceCoNLL-UCopy PasteCSVCube Semantic LayerDatadog LogsDiffbotDiscordDocugamiDropboxDuckDBEmailEmbaasEPubEtherscanEverNoteexample_dataMicrosoft ExcelFacebook ChatFaunaFigmaGeopandasGitGitBookGitHubGoogle BigQueryGoogle Cloud Storage DirectoryGoogle Cloud Storage FileGoogle DriveGrobidGutenbergHacker NewsHuawei OBS DirectoryHuawei OBS FileHuggingFace datasetiFixitImagesImage captionsIMSDbIuguJoplinJupyter NotebookLarkSuite (FeiShu)MastodonMediaWiki DumpMerge Documents LoadermhtmlMicrosoft OneDriveMicrosoft PowerPointMicrosoft SharePointMicrosoft WordModern TreasuryMongoDBNews URLNotion DB 1/2Notion DB 2/2NucliaObsidianOpen Document Format (ODT)Open City DataOrg-modePandas DataFrameAmazon TextractPolars DataFramePsychicPubMedPySparkReadTheDocs DocumentationRecursive URLRedditRoamRocksetrspaceRSS FeedsRSTSitemapSlackSnowflakeSource CodeSpreedlyStripeSubtitleTelegramTencent COS DirectoryTencent COS FileTensorFlow Datasets2MarkdownTOMLTrelloTSVTwitterUnstructured FileURLWeatherWebBaseLoaderWhatsApp ChatWikipediaXMLXorbits Pandas DataFrameYouTube audioYouTube transcriptsDocument transformersText embedding modelsVector storesRetrieversToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsDocument loadersSitemapOn this pageSitemapExtends from the WebBaseLoader, SitemapLoader loads a sitemap from a given URL, and then scrape and load all pages in the sitemap, returning each page as a Document.The scraping is done concurrently.  There are reasonable limits to concurrent requests, defaulting to 2 per second.  If you aren\'t concerned about being a good citizen, or you control the scrapped server, or don\'t care about load. Note, while this will speed up the scraping process, but it may cause the server to block you.  Be careful!pip install nest_asyncio    Requirement already satisfied: nest_asyncio in /Users/tasp/Code/projects/langchain/.venv/lib/python3.10/site-packages (1.5.6)        [notice] A new release of pip available: 22.3.1 -> 23.0.1    [notice] To update, run: pip install --upgrade pip# fixes a bug with asyncio and jupyterimport nest_asyncionest_asyncio.apply()from langchain.document_loaders.sitemap import SitemapLoadersitemap_loader = SitemapLoader(web_path="https://langchain.readthedocs.io/sitemap.xml")docs = sitemap_loader.load()You can change the requests_per_second parameter to increase the max concurrent requests. and use requests_kwargs to pass kwargs when send requests.sitemap_loader.requests_per_second = 2# Optional: avoid `[SSL: CERTIFICATE_VERIFY_FAILED]` issuesitemap_loader.requests_kwargs = {"verify": False}docs[0]    Document(page_content=\'\\n\\n\\n\\n\\n\\nWelcome to LangChain — 🦜🔗 LangChain 0.0.123\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nSkip to main content\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nCtrl+K\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n🦜🔗 LangChain 0.0.123\\n\\n\\n\\nGetting Started\\n\\nQuickstart Guide\\n\\nModules\\n\\nPrompt Templates\\nGetting Started\\nKey Concepts\\nHow-To Guides\\nCreate a custom prompt template\\nCreate a custom example selector\\nProvide few shot examples to a prompt\\nPrompt Seriali


Docusaurus | 🦜🔗 Langchain

zation\\nExample Selectors\\nOutput Parsers\\n\\n\\nReference\\nPromptTemplates\\nExample Selector\\n\\n\\n\\n\\nLLMs\\nGetting Started\\nKey Concepts\\nHow-To Guides\\nGeneric Functionality\\nCustom LLM\\nFake LLM\\nLLM Caching\\nLLM Serialization\\nToken Usage Tracking\\n\\n\\nIntegrations\\nAI21\\nAleph Alpha\\nAnthropic\\nAzure OpenAI LLM Example\\nBanana\\nCerebriumAI LLM Example\\nCohere\\nDeepInfra LLM Example\\nForefrontAI LLM Example\\nGooseAI LLM Example\\nHugging Face Hub\\nManifest\\nModal\\nOpenAI\\nPetals LLM Example\\nPromptLayer OpenAI\\nSageMakerEndpoint\\nSelf-Hosted Models via Runhouse\\nStochasticAI\\nWriter\\n\\n\\nAsync API for LLM\\nStreaming with LLMs\\n\\n\\nReference\\n\\n\\nDocument Loaders\\nKey Concepts\\nHow To Guides\\nCoNLL-U\\nAirbyte JSON\\nAZLyrics\\nBlackboard\\nCollege Confidential\\nCopy Paste\\nCSV Loader\\nDirectory Loader\\nEmail\\nEverNote\\nFacebook Chat\\nFigma\\nGCS Directory\\nGCS File Storage\\nGitBook\\nGoogle Drive\\nGutenberg\\nHacker News\\nHTML\\niFixit\\nImages\\nIMSDb\\nMarkdown\\nNotebook\\nNotion\\nObsidian\\nPDF\\nPowerPoint\\nReadTheDocs Documentation\\nRoam\\ns3 Directory\\ns3 File\\nSubtitle Files\\nTelegram\\nUnstructured File Loader\\nURL\\nWeb Base\\nWord Documents\\nYouTube\\n\\n\\n\\n\\nUtils\\nKey Concepts\\nGeneric Utilities\\nBash\\nBing Search\\nGoogle Search\\nGoogle Serper API\\nIFTTT WebHooks\\nPython REPL\\nRequests\\nSearxNG Search API\\nSerpAPI\\nWolfram Alpha\\nZapier Natural Language Actions API\\n\\n\\nReference\\nPython REPL\\nSerpAPI\\nSearxNG Search\\nDocstore\\nText Splitter\\nEmbeddings\\nVectorStores\\n\\n\\n\\n\\nIndexes\\nGetting Started\\nKey Concepts\\nHow To Guides\\nEmbeddings\\nHypothetical Document Embeddings\\nText Splitter\\nVectorStores\\nAtlasDB\\nChroma\\nDeep Lake\\nElasticSearch\\nFAISS\\nMilvus\\nOpenSearch\\nPGVector\\nPinecone\\nQdrant\\nRedis\\nWeaviate\\nChatGPT Plugin Retriever\\nVectorStore Retriever\\nAnalyze Document\\nChat Index\\nGraph QA\\nQuestion Answering with Sources\\nQuestion Answering\\nSummarization\\nRetrieval Question/Answering\\nRetrieval Question Answering with Sources\\nVector DB Text Generation\\n\\n\\n\\n\\nChains\\nGetting Started\\nHow-To Guides\\nGeneric Chains\\nLoading from LangChainHub\\nLLM Chain\\nSequential Chains\\nSerialization\\nTransformation Chain\\n\\n\\nUtility Chains\\nAPI Chains\\nSelf-Critique Chain with Constitutional AI\\nBashChain\\nLLMCheckerChain\\nLLM Math\\nLLMRequestsChain\\nLLMSummarizationCheckerChain\\nModeration\\nPAL\\nSQLite example\\n\\n\\nAsync API for Chain\\n\\n\\nKey Concepts\\nReference\\n\\n\\nAgents\\nGetting Started\\nKey Concepts\\nHow-To Guides\\nAgents and Vectorstores\\nAsync API for Agent\\nConversation Agent (for Chat Models)\\nChatGPT Plugins\\nCustom Agent\\nDefining Custom Tools\\nHuman as a tool\\nIntermediate Steps\\nLoading from LangChainHub\\nMax Iterations\\nMulti Input Tools\\nSearch Tools\\nSerialization\\nAdding SharedMemory to an Agent and its Tools\\nCSV Agent\\nJSON Agent\\nOpenAPI Agent\\nPandas Dataframe Agent\\nPython Agent\\nSQL Database Agent\\nVectorstore Agent\\nMRKL\\nMRKL Chat\\nReAct\\nSelf Ask With Search\\n\\n\\nReference\\n\\n\\nMemory\\nGetting Started\\nKey Concepts\\nHow-To Guides\\nConversationBufferMemory\\nConversationBufferWindowMemory\\nEntity Memory\\nConversation Knowledge Graph Memory\\nConversationSummaryMemory\\nConversationSummaryBufferMemory\\nConversationTokenBufferMemory\\nAdding Memory To an LLMChain\\nAdding Memory to a Multi-Input Chain\\nAdding Memory to an Agent\\nChatGPT Clone\\nConversation Agent\\nConversational Memory Customization\\nCustom Memory\\nMultiple Memory\\n\\n\\n\\n\\nChat\\nGetting Started\\nKey Concepts\\nHow-To Guides\\nAgent\\nChat Vector DB\\nFew Shot Examples\\nMemory\\nPromptLayer ChatOpenAI\\nStreaming\\nRetrieval Question/Answering\\nRetrieval Question Answering with Sources\\n\\n\\n\\n\\n\\nUse Cases\\n\\nAgents\\nChatbots\\nGenerate Examples\\nData Augmented Generation\\nQuestion Answering\\nSummarization\\nQuerying Tabular Data\\nExtraction\\nEvaluation\\nAgent Benchmarking: Search + Calculator\\nAgent VectorDB Question Answering Benchmarking\\nBenchmarking Template\\nData Augmented Question Answering\\nUsing Hugging Face Datasets\\nLLM Math\\nQuestion Answering Benchmarking: Paul Graham Essay\\nQuestion Answering Benchmarking: State of the Union Address\\nQA Generation\\nQuestion Answering\\nSQL Question Answering Benchmarking: Chinook\\n\\n\\nModel Comparison\\n\\nReference\\n\\nInstallation\\nIntegrations\\nAPI References\\nPrompts\\nPromptTemplates\\nExample Selector\\n\\n\\nUtilities\\nPython REPL\\nSerpAPI\\nSearxNG Search\\nDocstore\\nText Splitter\\nEmbeddings\\nVectorStores\\n\\n\\nChains\\nAgents\\n\\n\\n\\nEcosystem\\n\\nLangChain Ecosystem\\nAI21 Labs\\nAtlasDB\\nBanana\\nCerebriumAI\\nChroma\\nCohere\\nDeepInfra\\nDeep Lake\\nForefrontAI\\nGoogle Search Wrapper\\nGoogle Serper Wrapper\\nGooseAI\\nGraphsignal\\nHazy Research\\nHelicone\\nHugging Face\\nMilvus\\nModal\\nNLPCloud\\nOpenAI\\nOpenSearch\\nPetals\\nPGVector\\nPinecone\\nPromptLayer\\nQdrant\\nRunhouse\\nSearxNG Search API\\nSerpAPI\\nStochasticAI\\nUnstructured\\nWeights & Biases\\nWeaviate\\nWolfram Alpha Wrapper\\nWriter\\n\\n\\n\\nAdditional Resources\\n\\nLangChainHub\\nGlossary\\nLangChain Gallery\\nDeployments\\nTracing\\nDiscord\\nProduction Support\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n.rst\\n\\n\\n\\n\\n\\n\\n\\n.pdf\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nWelcome to LangChain\\n\\n\\n\\n\\n Contents \\n\\n\\n\\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\n\\n\\n\\n\\n\\n\\n\\n\\nWelcome to LangChain#\\nLarge language models (LLMs) are emerging as a transformative technology, enabling\\ndevelopers to build applications that they previously could not.\\nBut using these LLMs in isolation is often not enou


Docusaurus | 🦜🔗 Langchain

gh to\\ncreate a truly powerful app - the real power comes when you are able to\\ncombine them with other sources of computation or knowledge.\\nThis library is aimed at assisting in the development of those types of applications. Common examples of these types of applications include:\\n❓ Question Answering over specific documents\\n\\nDocumentation\\nEnd-to-end Example: Question Answering over Notion Database\\n\\n💬 Chatbots\\n\\nDocumentation\\nEnd-to-end Example: Chat-LangChain\\n\\n🤖 Agents\\n\\nDocumentation\\nEnd-to-end Example: GPT+WolframAlpha\\n\\n\\nGetting Started#\\nCheckout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\\n\\nGetting Started Documentation\\n\\n\\n\\n\\n\\nModules#\\nThere are several main modules that LangChain provides support for.\\nFor each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\\nThese modules are, in increasing order of complexity:\\n\\nPrompts: This includes prompt management, prompt optimization, and prompt serialization.\\nLLMs: This includes a generic interface for all LLMs, and common utilities for working with LLMs.\\nDocument Loaders: This includes a standard interface for loading documents, as well as specific integrations to all types of text data sources.\\nUtils: Language models are often more powerful when interacting with other sources of knowledge or computation. This can include Python REPLs, embeddings, search engines, and more. LangChain provides a large collection of common utils to use in your application.\\nChains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\\nIndexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.\\nAgents: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.\\nMemory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.\\nChat: Chat models are a variation on Language Models that expose a different API - rather than working with raw text, they work with messages. LangChain provides a standard interface for working with them and doing all the same things as above.\\n\\n\\n\\n\\n\\nUse Cases#\\nThe above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.\\n\\nAgents: Agents are systems that use a language model to interact with other tools. These can be used to do more grounded question/answering, interact with APIs, or even take actions.\\nChatbots: Since language models are good at producing text, that makes them ideal for creating chatbots.\\nData Augmented Generation: Data Augmented Generation involves specific types of chains that first interact with an external datasource to fetch data to use in the generation step. Examples of this include summarization of long pieces of text and question/answering over specific data sources.\\nQuestion Answering: Answering questions over specific documents, only utilizing the information in those documents to construct an answer. A type of Data Augmented Generation.\\nSummarization: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.\\nQuerying Tabular Data: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.\\nEvaluation: Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.\\nGenerate similar examples: Generating similar examples to a given input. This is a common use case for many applications, and LangChain provides some prompts/chains for assisting in this.\\nCompare models: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.\\n\\n\\n\\n\\n\\nReference Docs#\\nAll of LangChain’s reference documentation, in one place. Full documentation on all methods, classes, installation methods, and integration setups for LangChain.\\n\\nReference Documentation\\n\\n\\n\\n\\n\\nLangChain Ecosystem#\\nGuides for how other companies/products can be used with LangChain\\n\\nLangChain Ecosystem\\n\\n\\n\\n\\n\\nAdditional Resources#\\nAdditional collection of resources we think may be useful as you develop your application!\\n\\nLangChainHub: The LangChainHub is a place to share and explore other prompts, chains, and agents.\\nGlossary: A glossary of all related terms, papers, methods, etc. Whether implemented in LangChain or not!\\nGallery: A collection of our favorite projects that use LangChain. Useful for finding inspiration or seeing how things were done in other applications.\\nDeployments: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.\\nDiscord: Join us on our Discord to discuss all things LangChain!\\nTracing: A guide on using tracing in LangChain to visualize the execution of chains and agents.\\nProduction Support: As you move your LangChains into production, we’d love to offer more comprehensive support. Please fill out this form and we’ll set up a dedicated support Slack channel.\\n\\n\\n\\n\\n\\n\\n


Docusaurus | 🦜🔗 Langchain

\\n\\n\\n\\n\\nnext\\nQuickstart Guide\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n Contents\\n  \\n\\n\\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nBy Harrison Chase\\n\\n\\n\\n\\n    \\n      © Copyright 2023, Harrison Chase.\\n      \\n\\n\\n\\n\\n  Last updated on Mar 24, 2023.\\n  \\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\', lookup_str=\'\', metadata={\'source\': \'https://python.langchain.com/en/stable/\', \'loc\': \'https://python.langchain.com/en/stable/\', \'lastmod\': \'2023-03-24T19:30:54.647430+00:00\', \'changefreq\': \'weekly\', \'priority\': \'1\'}, lookup_index=0)Filtering sitemap URLs\u200bSitemaps can be massive files, with thousands of URLs.  Often you don\'t need every single one of them.  You can filter the URLs by passing a list of strings or regex patterns to the url_filter parameter.  Only URLs that match one of the patterns will be loaded.loader = SitemapLoader(    "https://langchain.readthedocs.io/sitemap.xml",    filter_urls=["https://python.langchain.com/en/latest/"],)documents = loader.load()documents[0]    Document(page_content=\'\\n\\n\\n\\n\\n\\nWelcome to LangChain — 🦜🔗 LangChain 0.0.123\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nSkip to main content\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nCtrl+K\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n🦜🔗 LangChain 0.0.123\\n\\n\\n\\nGetting Started\\n\\nQuickstart Guide\\n\\nModules\\n\\nModels\\nLLMs\\nGetting Started\\nGeneric Functionality\\nHow to use the async API for LLMs\\nHow to write a custom LLM wrapper\\nHow (and why) to use the fake LLM\\nHow to cache LLM calls\\nHow to serialize LLM classes\\nHow to stream LLM responses\\nHow to track token usage\\n\\n\\nIntegrations\\nAI21\\nAleph Alpha\\nAnthropic\\nAzure OpenAI LLM Example\\nBanana\\nCerebriumAI LLM Example\\nCohere\\nDeepInfra LLM Example\\nForefrontAI LLM Example\\nGooseAI LLM Example\\nHugging Face Hub\\nManifest\\nModal\\nOpenAI\\nPetals LLM Example\\nPromptLayer OpenAI\\nSageMakerEndpoint\\nSelf-Hosted Models via Runhouse\\nStochasticAI\\nWriter\\n\\n\\nReference\\n\\n\\nChat Models\\nGetting Started\\nHow-To Guides\\nHow to use few shot examples\\nHow to stream responses\\n\\n\\nIntegrations\\nAzure\\nOpenAI\\nPromptLayer ChatOpenAI\\n\\n\\n\\n\\nText Embedding Models\\nAzureOpenAI\\nCohere\\nFake Embeddings\\nHugging Face Hub\\nInstructEmbeddings\\nOpenAI\\nSageMaker Endpoint Embeddings\\nSelf Hosted Embeddings\\nTensorflowHub\\n\\n\\n\\n\\nPrompts\\nPrompt Templates\\nGetting Started\\nHow-To Guides\\nHow to create a custom prompt template\\nHow to create a prompt template that uses few shot examples\\nHow to work with partial Prompt Templates\\nHow to serialize prompts\\n\\n\\nReference\\nPromptTemplates\\nExample Selector\\n\\n\\n\\n\\nChat Prompt Template\\nExample Selectors\\nHow to create a custom example selector\\nLengthBased ExampleSelector\\nMaximal Marginal Relevance ExampleSelector\\nNGram Overlap ExampleSelector\\nSimilarity ExampleSelector\\n\\n\\nOutput Parsers\\nOutput Parsers\\nCommaSeparatedListOutputParser\\nOutputFixingParser\\nPydanticOutputParser\\nRetryOutputParser\\nStructured Output Parser\\n\\n\\n\\n\\nIndexes\\nGetting Started\\nDocument Loaders\\nCoNLL-U\\nAirbyte JSON\\nAZLyrics\\nBlackboard\\nCollege Confidential\\nCopy Paste\\nCSV Loader\\nDirectory Loader\\nEmail\\nEverNote\\nFacebook Chat\\nFigma\\nGCS Directory\\nGCS File Storage\\nGitBook\\nGoogle Drive\\nGutenberg\\nHacker News\\nHTML\\niFixit\\nImages\\nIMSDb\\nMarkdown\\nNotebook\\nNotion\\nObsidian\\nPDF\\nPowerPoint\\nReadTheDocs Documentation\\nRoam\\ns3 Directory\\ns3 File\\nSubtitle Files\\nTelegram\\nUnstructured File Loader\\nURL\\nWeb Base\\nWord Documents\\nYouTube\\n\\n\\nText Splitters\\nGetting Started\\nCharacter Text Splitter\\nHuggingFace Length Function\\nLatex Text Splitter\\nMarkdown Text Splitter\\nNLTK Text Splitter\\nPython Code Text Splitter\\nRecursiveCharacterTextSplitter\\nSpacy Text Splitter\\ntiktoken (OpenAI) Length Function\\nTiktokenText Splitter\\n\\n\\nVectorstores\\nGetting Started\\nAtlasDB\\nChroma\\nDeep Lake\\nElasticSearch\\nFAISS\\nMilvus\\nOpenSearch\\nPGVector\\nPinecone\\nQdrant\\nRedis\\nWeaviate\\n\\n\\nRetrievers\\nChatGPT Plugin Retriever\\nVectorStore Retriever\\n\\n\\n\\n\\nMemory\\nGetting Started\\nHow-To Guides\\nConversationBufferMemory\\nConversationBufferWindowMemory\\nEntity Memory\\nConversation Knowledge Graph Memory\\nConversationSummaryMemory\\nConversationSummaryBufferMemory\\nConversationTokenBufferMemory\\nHow to add Memory to an LLMChain\\nHow to add memory to a Multi-Input Chain\\nHow to add Memory to an Agent\\nHow to customize conversational memory\\nHow to create a custom Memory class\\nHow to use multiple memroy classes in the same chain\\n\\n\\n\\n\\nChains\\nGetting Started\\nHow-To Guides\\nAsync API for Chain\\nLoading from LangChainHub\\nLLM Chain\\nSequential Chains\\nSerialization\\nTransformation Chain\\nAnalyze Document\\nChat Index\\nGraph QA\\nHypothetical Document Embeddings\\nQuestion Answering with Sources\\nQuestion Answering\\nSummarization\\nRetrieval Question/Answering\\nRetrieval Question Answering with Sources\\nVector DB Text Generation\\nAPI Chains\\nSelf-Critique Chain with Constitutional AI\\nBashChain\\nLLMCheckerChain\\nLLM Math\\nLLMRequestsChain\\nLLMSummarizationCheckerChain\\nModeration\\nPAL\\nSQLite example\\n\\n\\nReference\\n\\n\\nAgents\\nGetting Started\\nTools\\nGetting Started\\nDefining Custom Tools\\nMulti Input Tools\\nBash\\nBing Search\\nChatGPT Plugins\\nGoogle Search\\nGoogle Serper API\\nHuman as a tool\\nIFTTT WebHooks\\nPython REPL\\nRequests\\nSearch Tools\\nSearxNG Search API\\nSerpAPI\\nWolfram Alpha\\nZapier Natural Language Actions API\\n\\n\\nAgents\\nAgent Types\\nCustom Agent\\nConversation Agent (for Chat Models)\\nConversation Agent\\nMRKL\\nMRKL Chat\\nReAct\\nSelf Ask With


Docusaurus | 🦜🔗 Langchain

Search\\n\\n\\nToolkits\\nCSV Agent\\nJSON Agent\\nOpenAPI Agent\\nPandas Dataframe Agent\\nPython Agent\\nSQL Database Agent\\nVectorstore Agent\\n\\n\\nAgent Executors\\nHow to combine agents and vectorstores\\nHow to use the async API for Agents\\nHow to create ChatGPT Clone\\nHow to access intermediate steps\\nHow to cap the max number of iterations\\nHow to add SharedMemory to an Agent and its Tools\\n\\n\\n\\n\\n\\nUse Cases\\n\\nPersonal Assistants\\nQuestion Answering over Docs\\nChatbots\\nQuerying Tabular Data\\nInteracting with APIs\\nSummarization\\nExtraction\\nEvaluation\\nAgent Benchmarking: Search + Calculator\\nAgent VectorDB Question Answering Benchmarking\\nBenchmarking Template\\nData Augmented Question Answering\\nUsing Hugging Face Datasets\\nLLM Math\\nQuestion Answering Benchmarking: Paul Graham Essay\\nQuestion Answering Benchmarking: State of the Union Address\\nQA Generation\\nQuestion Answering\\nSQL Question Answering Benchmarking: Chinook\\n\\n\\n\\nReference\\n\\nInstallation\\nIntegrations\\nAPI References\\nPrompts\\nPromptTemplates\\nExample Selector\\n\\n\\nUtilities\\nPython REPL\\nSerpAPI\\nSearxNG Search\\nDocstore\\nText Splitter\\nEmbeddings\\nVectorStores\\n\\n\\nChains\\nAgents\\n\\n\\n\\nEcosystem\\n\\nLangChain Ecosystem\\nAI21 Labs\\nAtlasDB\\nBanana\\nCerebriumAI\\nChroma\\nCohere\\nDeepInfra\\nDeep Lake\\nForefrontAI\\nGoogle Search Wrapper\\nGoogle Serper Wrapper\\nGooseAI\\nGraphsignal\\nHazy Research\\nHelicone\\nHugging Face\\nMilvus\\nModal\\nNLPCloud\\nOpenAI\\nOpenSearch\\nPetals\\nPGVector\\nPinecone\\nPromptLayer\\nQdrant\\nRunhouse\\nSearxNG Search API\\nSerpAPI\\nStochasticAI\\nUnstructured\\nWeights & Biases\\nWeaviate\\nWolfram Alpha Wrapper\\nWriter\\n\\n\\n\\nAdditional Resources\\n\\nLangChainHub\\nGlossary\\nLangChain Gallery\\nDeployments\\nTracing\\nDiscord\\nProduction Support\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n.rst\\n\\n\\n\\n\\n\\n\\n\\n.pdf\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nWelcome to LangChain\\n\\n\\n\\n\\n Contents \\n\\n\\n\\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\n\\n\\n\\n\\n\\n\\n\\n\\nWelcome to LangChain#\\nLangChain is a framework for developing applications powered by language models. We believe that the most powerful and differentiated applications will not only call out to a language model via an API, but will also:\\n\\nBe data-aware: connect a language model to other sources of data\\nBe agentic: allow a language model to interact with its environment\\n\\nThe LangChain framework is designed with the above principles in mind.\\nThis is the Python specific portion of the documentation. For a purely conceptual guide to LangChain, see here. For the JavaScript documentation, see here.\\n\\nGetting Started#\\nCheckout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\\n\\nGetting Started Documentation\\n\\n\\n\\n\\n\\nModules#\\nThere are several main modules that LangChain provides support for.\\nFor each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\\nThese modules are, in increasing order of complexity:\\n\\nModels: The various model types and model integrations LangChain supports.\\nPrompts: This includes prompt management, prompt optimization, and prompt serialization.\\nMemory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.\\nIndexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.\\nChains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\\nAgents: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.\\n\\n\\n\\n\\n\\nUse Cases#\\nThe above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.\\n\\nPersonal Assistants: The main LangChain use case. Personal assistants need to take actions, remember interactions, and have knowledge about your data.\\nQuestion Answering: The second big LangChain use case. Answering questions over specific documents, only utilizing the information in those documents to construct an answer.\\nChatbots: Since language models are good at producing text, that makes them ideal for creating chatbots.\\nQuerying Tabular Data: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.\\nInteracting with APIs: Enabling LLMs to interact with APIs is extremely powerful in order to give them more up-to-date information and allow them to take actions.\\nExtraction: Extract structured information from text.\\nSummarization: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.\\nEvaluation: Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.\\n\\n\\n\\n\\n\\nReference Docs#\\nAll of LangChain’s reference documentation, in one place. Full documentation on all methods, classes, installation methods, and integration setup


Docusaurus | 🦜🔗 Langchain

s for LangChain.\\n\\nReference Documentation\\n\\n\\n\\n\\n\\nLangChain Ecosystem#\\nGuides for how other companies/products can be used with LangChain\\n\\nLangChain Ecosystem\\n\\n\\n\\n\\n\\nAdditional Resources#\\nAdditional collection of resources we think may be useful as you develop your application!\\n\\nLangChainHub: The LangChainHub is a place to share and explore other prompts, chains, and agents.\\nGlossary: A glossary of all related terms, papers, methods, etc. Whether implemented in LangChain or not!\\nGallery: A collection of our favorite projects that use LangChain. Useful for finding inspiration or seeing how things were done in other applications.\\nDeployments: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.\\nTracing: A guide on using tracing in LangChain to visualize the execution of chains and agents.\\nModel Laboratory: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.\\nDiscord: Join us on our Discord to discuss all things LangChain!\\nProduction Support: As you move your LangChains into production, we’d love to offer more comprehensive support. Please fill out this form and we’ll set up a dedicated support Slack channel.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nnext\\nQuickstart Guide\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n Contents\\n  \\n\\n\\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nBy Harrison Chase\\n\\n\\n\\n\\n    \\n      © Copyright 2023, Harrison Chase.\\n      \\n\\n\\n\\n\\n  Last updated on Mar 27, 2023.\\n  \\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\', lookup_str=\'\', metadata={\'source\': \'https://python.langchain.com/en/latest/\', \'loc\': \'https://python.langchain.com/en/latest/\', \'lastmod\': \'2023-03-27T22:50:49.790324+00:00\', \'changefreq\': \'daily\', \'priority\': \'0.9\'}, lookup_index=0)Add custom scraping rules\u200bThe SitemapLoader uses beautifulsoup4 for the scraping process, and it scrapes every element on the page by default. The SitemapLoader constructor accepts a custom scraping function. This feature can be helpful to tailor the scraping process to your specific needs; for example, you might want to avoid scraping headers or navigation elements. The following example shows how to develop and use a custom function to avoid navigation and header elements.Import the beautifulsoup4 library and define the custom function.pip install beautifulsoup4from bs4 import BeautifulSoupdef remove_nav_and_header_elements(content: BeautifulSoup) -> str:    # Find all \'nav\' and \'header\' elements in the BeautifulSoup object    nav_elements = content.find_all("nav")    header_elements = content.find_all("header")    # Remove each \'nav\' and \'header\' element from the BeautifulSoup object    for element in nav_elements + header_elements:        element.decompose()    return str(content.get_text())Add your custom function to the SitemapLoader object.loader = SitemapLoader(    "https://langchain.readthedocs.io/sitemap.xml",    filter_urls=["https://python.langchain.com/en/latest/"],    parsing_function=remove_nav_and_header_elements,)Local Sitemap\u200bThe sitemap loader can also be used to load local files.sitemap_loader = SitemapLoader(web_path="example_data/sitemap.xml", is_local=True)docs = sitemap_loader.load()    Fetching pages: 100%|####################################################################################################################################| 3/3 [00:00<00:00,  3.91it/s]PreviousRSTNextSlackFiltering sitemap URLsAdd custom scraping rulesLocal SitemapCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright © 2023 LangChain, Inc.\n\n\n\n', metadata={'source': 'https://python.langchain.com/docs/integrations/document_loaders/sitemap', 'loc': 'https://python.langchain.com/docs/integrations/document_loaders/sitemap', 'changefreq': 'weekly', 'priority': '0.5'})  ##Add custom scraping rules[](#add-custom-scraping-rules) By default, the parser removes all but the main content of the docusaurus page, which is normally the <article> tag. You also have the option to define an inclusive list HTML tags by providing them as a list utilizing the custom_html_tags parameter. For example:  loader = DocusaurusLoader(     "https://python.langchain.com",     filter_urls=[         "https://python.langchain.com/docs/integrations/document_loaders/sitemap"     ],     # This will only include the content that matches these tags, otherwise they will be removed     custom_html_tags=["#content", ".main"], )  You can also define an entirely custom parsing function if you need finer-grained control over the returned content for each page.  The following example shows how to develop and use a custom function to avoid navigation and header elements.  from bs4 import BeautifulSoup   def remove_nav_and_header_elements(content: BeautifulSoup) -> str:     # Find all 'nav' and 'header' elements in the BeautifulSoup object     nav_elements = content.find_all("nav")     header_elements = content.find_all("header")      # Remove each 'nav' and 'header' element from the BeautifulSoup object     for element in nav_elements + header_elements:         element.decompose()      return str(content.get_text())  Add your custom function to the DocusaurusLoader object.  loader = DocusaurusLoader(     "https://python.langchain.com",     filter_urls=[         "https://python.langchain.com/docs/integrations/document_loaders/sitemap"     ],     parsing_function=remove_nav_and_header_elements, )


Dropbox | 🦜🔗 Langchain

Main:  On this page #Dropbox [Dropbox](https://en.wikipedia.org/wiki/Dropbox) is a file hosting service that brings everything-traditional files, cloud content, and web shortcuts together in one place.  This notebook covers how to load documents from Dropbox. In addition to common files such as text and PDF files, it also supports Dropbox Paper files.  ##Prerequisites[](#prerequisites) - Create a Dropbox app. - Give the app these scope permissions: files.metadata.read and files.content.read. - Generate access token: [https://www.dropbox.com/developers/apps/create](https://www.dropbox.com/developers/apps/create). - pip install dropbox (requires pip install "unstructured[pdf]" for PDF filetype). ##Instructions[](#instructions) `DropboxLoader`` requires you to create a Dropbox App and generate an access token. This can be done from [https://www.dropbox.com/developers/apps/create](https://www.dropbox.com/developers/apps/create). You also need to have the Dropbox Python SDK installed (pip install dropbox).  DropboxLoader can load data from a list of Dropbox file paths or a single Dropbox folder path. Both paths should be relative to the root directory of the Dropbox account linked to the access token.  pip install dropbox      Requirement already satisfied: dropbox in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (11.36.2)     Requirement already satisfied: requests>=2.16.2 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from dropbox) (2.31.0)     Requirement already satisfied: six>=1.12.0 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from dropbox) (1.16.0)     Requirement already satisfied: stone>=2 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from dropbox) (3.3.1)     Requirement already satisfied: charset-normalizer<4,>=2 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from requests>=2.16.2->dropbox) (3.2.0)     Requirement already satisfied: idna<4,>=2.5 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from requests>=2.16.2->dropbox) (3.4)     Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from requests>=2.16.2->dropbox) (2.0.4)     Requirement already satisfied: certifi>=2017.4.17 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from requests>=2.16.2->dropbox) (2023.7.22)     Requirement already satisfied: ply>=3.4 in /Users/rbarragan/.local/share/virtualenvs/langchain-kv0dsrF5/lib/python3.11/site-packages (from stone>=2->dropbox) (3.11)     Note: you may need to restart the kernel to use updated packages.  from langchain.document_loaders import DropboxLoader  # Generate access token: https://www.dropbox.com/developers/apps/create. dropbox_access_token = "<DROPBOX_ACCESS_TOKEN>" # Dropbox root folder dropbox_folder_path = ""  loader = DropboxLoader(     dropbox_access_token=dropbox_access_token,     dropbox_folder_path=dropbox_folder_path,     recursive=False, )  documents = loader.load()      File /JHSfLKn0.jpeg could not be decoded as text. Skipping.     File /A REPORT ON WILES’ CAMBRIDGE LECTURES.pdf could not be decoded as text. Skipping.  for document in documents:     print(document)      page_content='# 🎉 Getting Started with Dropbox Paper\nDropbox Paper is great for capturing ideas and gathering quick feedback from your team. You can use words, images, code, or media from other apps, or go ahead and connect your calendar and add to-dos for projects.\n\n*Explore and edit this doc to play with some of these features. This doc is all yours. No one will see your edits unless you share this doc.*\n\n\n# The basics\n\n**Selecting text** activates the formatting toolbar, where you can apply basic formatting, create lists, and add comments.\n\n[ ] Create to-do lists\n- Bulleted lists\n1. Numbered lists\n\n**Starting a new line** activates the insert toolbar, where you can add media from other apps, links to Dropbox files, photos, and more.\n\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523574441249_paper-insert.png)\n\n\n\n**Add emojis** to your doc or comment by typing `**:**` ****and choosing a character. \n\n# 👍 👎 👏 ✅ ❌ ❤ ⭐ 💡 📌\n\n\n# Images\n\n**Selecting images** activates the image toolbar, where you can align images left, center, right or expand them to full width.\n\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523473869783_Hot_Sauce.jpg)\n\n\nPaste images or gifs right next to each other and they\'ll organize automatically. Click on an image twice to start full-screen gallery view.\n\n\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523564536543_Clock_Melt.png)\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523564528339_Boom_Box_Melt.png)\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523564549819_Soccerball_Melt.png)\n\n![You can add captions too](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523564518899_Cacti_Melt.png)\n![What a strange, melting toaster!](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523564508553_Toaster_Melt.png)\n\n\n \n\n\n# Form meets function\n\nYou and your team can create the way you want, with what you want. Dropbox Paper adapts to the way your team captures ideas.\n\n**Add media from apps** like YouTube and Vimeo, or add audio from Spotify and SoundCloud. Files from Google Drive and Dropbox update auto


Dropbox | 🦜🔗 Langchain

matically. Start a new line and choose add media, or drop in a link to try it out.\n\n\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523575138939_paper-embed.png)\n\n\n\n## YouTube\nhttps://www.youtube.com/watch?v=fmsq1uKOa08&\n\n\n[https://youtu.be/fmsq1uKOa08](https://youtu.be/fmsq1uKOa08)\n\n\n\n## SoundCloud\nhttps://w.soundcloud.com/player/?url=https%3A%2F%2Fsoundcloud.com%2Ftycho%2Fspoon-inside-out-tycho-version&autoplay=false\n\n\n[https://soundcloud.com/tycho/spoon-inside-out-tycho-version](https://soundcloud.com/tycho/spoon-inside-out-tycho-version) \n\n\n\n## Dropbox files\nhttps://www.dropbox.com/s/bgi58tkovntch5e/Wireframe%20render.pdf?dl=0\n\n\n\n\n## Code\n\n**Write code** in Dropbox Paper with automatic language detection and syntax highlighting. Start a new line and type three backticks (```).\n\n\n    public class HelloWorld { \n       public static void main(String[] args) { \n          System.out.println("Hello, World");\n       }\n    }\n\n\n\n## Tables\n\n**Create a table** with the menu that shows up on the right when you start a new line.\n\n| To insert a row or column, hover over a dividing line and click the +                                   | ⭐     |\n| ------------------------------------------------------------------------------------------------------- | ----- |\n| To delete, select rows/columns and click the trash can                                                  | ⭐ ⭐   |\n| To delete the entire table, click inside a cell, then click the dot in the top left corner of the table | ⭐ ⭐ ⭐ |\n\n\n\n\n\n# Collaborate with people\n\n**Invite people to your doc** so they can view, comment, and edit. Invite anyone you’d like—team members, contractors, stakeholders—to give them access to your doc.\n\n![](https://paper-attachments.dropbox.com/s_72143DBFDAF4C9DE702BB246920BC47FE7E1FA76AC23CC699374430D94E96DD2_1523574876795_paper-invite.png)\n\n\n**Make your docs discoverable to your team** by adding them to shared folders. Invite-only folders create more privacy.\n\n\n## Comments\n\n**Add comments** on a single character, an entire document, or any asset by highlighting it. **Add stickers** by clicking the 😄 in the message box.\n\n\n## To-dos\n\n**Bring someone’s attention to a comment or to-do** by typing **@** and their name or email address. Reference a doc or folder by typing **+** and its name.\n\n[ ] Mentioning someone on a to-do assigns it to them and sends an email [@Patricia J](http://#)\n[ ] Add a due date by clicking the calendar icon [@Jonathan C](http://#) [@Patricia J](http://#)\n[ ] You can also mention docs [+🎉  Getting Started with Dropbox Paper](http://#)\n\n\n\n# Go mobile\n\nEdit, create, and share Paper docs on Android or iOS phones and tablets. Download the apps in the [App Store](https://itunes.apple.com/us/app/paper-by-dropbox/id1126623662) and [Google Play Store](https://play.google.com/store/apps/details?id=com.dropbox.paper).\n\n\n\n# Help\n\n**Visit the** [**help center**](https://www.dropbox.com/help/topics/paper) for more about Dropbox Paper.\n\n**For more tips,** click the **?** in the bottom right of the screen and choose **Paper guide**.\n\n**Give us feedback** by selecting “Feedback” from the **?** in the bottom right of the screen. We’d love to hear what you think. \n\n' metadata={'source': 'dropbox:///_ Getting Started with Dropbox Paper.paper', 'title': '_ Getting Started with Dropbox Paper.paper'}     page_content='# 🥂 Toast to Droplets\n❓ **Rationale:** Reflection, especially writing, is the key to deep learning! Let’s take a few minutes to reflect on your first day at Dropbox individually, and then one lucky person will have the chance to share their toast.\n\n✍ **How to fill out this template:**\n\n- Option 1: You can sign in and then click “Create doc” to make a copy of this template. Fill in the blanks!\n- Option 2: If you don’t know your personal Dropbox login quickly, you can copy and paste this text into another word processing tool and start typing! \n\n\n\n## To my Droplet class:\n\nI feel so happy and excited to be making a toast to our newest Droplet class at Dropbox Basecamp.\n\nAt the beginning of our first day, I felt a bit underwhelmed with all information, and now, at the end of our first day at Dropbox, I feel I know enough for me to ramp up, but still a lot to learn**.**\n\nI can’t wait to explore every drl, but especially drl/(App Center)/benefits/allowance. I heard it’s so informative!\n\nDesigning an enlightened way of working is important, and to me, it means **a lot since I love what I do and I can help people around the globe**.\n\nI am excited to work with my team and flex my **technical and social** skills in my role as a **Software Engineer**.\n\nAs a Droplet, I pledge to:\n\n\n1. Be worthy of trust by **working always with values and integrity**.\n\n\n1. Keep my customers first by  **caring about their happiness and the value that we provide as a company**.\n\n\n1. Own it, keep it simple, and especially make work human by **providing value to people****.**\n\nCongrats, Droplets!\n\n' metadata={'source': 'dropbox:///_ Toast to Droplets.paper', 'title': '_ Toast to Droplets.paper'}     page_content='APPEARED IN BULLETIN OF THE AMERICAN MATHEMATICAL SOCIETY Volume 31, Number 1, July 1994, Pages 15-38\n\nA REPORT ON WILES’ CAMBRIDGE LECTURES\n\n4 9 9 1\n\nK. RUBIN AND A. SILVERBERG\n\nl u J\n\nAbstract. In lectures at the Newton Institute in June of 1993, Andrew Wiles announced a proof of a large part of the Taniyama-Shimura Conjecture and, as a consequence, Fermat’s Last Theorem. This report for nonexperts dis- cusses the mathematics involved in Wiles’ lectures, including the necessary background and the mathematical history.\n\n1\n\n] T N . h t a m\n\nIntroduction\n\nOn June 23, 1993, Andrew Wiles wrote on a blackboard, before an audience at the Newton Institute in Cambridge, England, that if p is a prime number, u, v, and w are ration


Dropbox | 🦜🔗 Langchain

al numbers, and up + vp + wp = 0, then uvw = 0. In other words, he announced that he could prove Fermat’s Last Theorem. His announce- ment came at the end of his series of three talks entitled “Modular forms, elliptic curves, and Galois representations” at the week-long workshop on “p-adic Galois representations, Iwasawa theory, and the Tamagawa numbers of motives”.\n\n[\n\n1 v 0 2 2 7 0 4 9 / h t a m : v i X r a\n\nIn the margin of his copy of the works of Diophantus, next to a problem on\n\nPythagorean triples, Pierre de Fermat (1601–1665) wrote:\n\nCubum autem in duos cubos, aut quadratoquadratum in duos quadrato- quadratos, et generaliter nullam in inﬁnitum ultra quadratum potestatem in duos ejusdem nominis fas est dividere : cujus rei demonstrationem mirabilem sane detexi. Hanc marginis exiguitas non caperet.\n\n(It is impossible to separate a cube into two cubes, or a fourth power into two fourth powers, or in general, any power higher than the second into two like powers. I have discovered a truly marvelous proof of this, which this margin is too narrow to contain.)\n\nWe restate Fermat’s conjecture as follows.\n\nFermat’s Last Theorem. If n > 2, then an +bn = cn has no solutions in nonzero integers a, b, and c.\n\nA proof by Fermat has never been found, and the problem has remained open, inspiring many generations of mathematicians. Much of modern number theory has been built on attempts to prove Fermat’s Last Theorem. For details on the\n\nReceived by the editors November 29, 1993. 1991 Mathematics Subject Classiﬁcation. Primary 11G05; Secondary 11D41, 11G18. The authors thank the National Science Foundation for ﬁnancial support.\n\nc(cid:13)1994 American Mathematical Society 0273-0979/94 $1.00 + $.25 per page\n\n1\n\n2\n\nK. RUBIN AND A. SILVERBERG\n\nhistory of Fermat’s Last Theorem (last because it is the last of Fermat’s questions to be answered) see [5], [6], and [26].\n\nWhat Andrew Wiles announced in Cambridge was that he could prove “many” elliptic curves are modular, suﬃciently many to imply Fermat’s Last Theorem. In this paper we will explain Wiles’ work on elliptic curves and its connection with 1 we introduce elliptic curves and modularity, and Fermat’s Last Theorem. give the connection between Fermat’s Last Theorem and the Taniyama-Shimura Conjecture on the modularity of elliptic curves. In 2 we describe how Wiles re- duces the proof of the Taniyama-Shimura Conjecture to what we call the Modular Lifting Conjecture (which can be viewed as a weak form of the Taniyama-Shimura Conjecture), by using a theorem of Langlands and Tunnell. In 4 we show § how the Semistable Modular Lifting Conjecture is related to a conjecture of Mazur on deformations of Galois representations (Conjecture 4.2), and in 5 we describe Wiles’ method of attack on this conjecture. In order to make this survey as acces- sible as possible to nonspecialists, the more technical details are postponed as long as possible, some of them to the appendices.\n\nIn\n\n§\n\n§\n\n3 and §\n\n§\n\nMuch of this report is based on Wiles’ lectures in Cambridge. The authors apol- ogize for any errors we may have introduced. We also apologize to those whose mathematical contributions we, due to our incomplete understanding, do not prop- erly acknowledge.\n\nThe ideas Wiles introduced in his Cambridge lectures will have an important inﬂuence on research in number theory. Because of the great interest in this subject and the lack of a publicly available manuscript, we hope this report will be useful to the mathematics community. In early December 1993, shortly before this paper went to press, Wiles announced that “the ﬁnal calculation of a precise upper bound for the Selmer group in the semistable case” (see 5.4 below) “is not yet § complete as it stands,” but that he believes he will be able to ﬁnish it in the near future using the ideas explained in his Cambridge lectures. While Wiles’ proof of Theorem 5.3 below and Fermat’s Last Theorem depends on the calculation he referred to in his December announcement, Theorem 5.4 and Corollary 5.5 do not. Wiles’ work provides for the ﬁrst time inﬁnitely many modular elliptic curves over the rational numbers which are not isomorphic over the complex numbers (see 5.5 for an explicit inﬁnite family).\n\n5.3 and\n\n§\n\n§\n\nNotation. The integers, rational numbers, complex numbers, and p-adic integers will be denoted Z, Q, C, and Zp, respectively. If F is a ﬁeld, then ¯F denotes an algebraic closure of F .\n\n1. Connection between Fermat’s Last Theorem and elliptic curves\n\n1.1. Fermat’s Last Theorem follows from modularity of elliptic curves. Suppose Fermat’s Last Theorem were false. Then there would exist nonzero integers a, b, c, and n > 2 such that an + bn = cn. It is easy to see that no generality is lost by assuming that n is a prime greater than three (or greater than four million, by [2]; see [14] for n = 3 and 4) and that a and b are relatively prime. Write down the cubic curve:\n\ny2 = x(x + an)(x\n\nbn).\n\n(1)\n\n−\n\nA REPORT ON WILES’ CAMBRIDGE LECTURES\n\n3\n\n1.4 we will explain what it means for an elliptic curve to be modular. Kenneth Ribet [27] proved that if n is a prime greater than three, a, b, and c are nonzero integers, and an + bn = cn, then the elliptic curve (1) is not modular. But the results announced by Wiles imply the following.\n\nIn\n\n1.3 we will see that such curves are elliptic curves, and in\n\n§\n\n§\n\nTheorem 1.1 (Wiles). If A and B are distinct, nonzero, relatively prime integers, and AB(A\n\nB) is divisible by 16, then the elliptic curve\n\n−\n\ny2 = x(x + A)(x + B)\n\nis modular.\n\nbn with a, b, c, and n coming from our hypothetical solution to a Fermat equation as above, we see that the conditions of Theorem 1.1 are satisﬁed since n 5 and one of a, b, and c is even. Thus Theorem 1.1 and Ribet’s result together imply Fermat’s Last Theorem!\n\nTaking A = an and B =\n\n−\n\n≥\n\n1.2. History. The story of the connection between Fermat’s Las


Dropbox | 🦜🔗 Langchain

t Theorem and elliptic curves begins in 1955, when Yutaka Taniyama (1927–1958) posed problems which may be viewed as a weaker version of the following conjecture (see [38]).\n\nTaniyama-Shimura Conjecture. Every elliptic curve over Q is modular.\n\nThe conjecture in the present form was made by Goro Shimura around 1962–64 and has become better understood due to work of Shimura [33–37] and of Andr´e Weil [42] (see also [7]). The Taniyama-Shimura Conjecture is one of the major conjectures in number theory.\n\nBeginning in the late 1960s [15–18], Yves Hellegouarch connected Fermat equa- tions an + bn = cn with elliptic curves of the form (1) and used results about Fer- mat’s Last Theorem to prove results about elliptic curves. The landscape changed abruptly in 1985 when Gerhard Frey stated in a lecture at Oberwolfach that elliptic curves arising from counterexamples to Fermat’s Last Theorem could not be mod- ular [11]. Shortly thereafter Ribet [27] proved this, following ideas of Jean-Pierre Serre [32] (see [24] for a survey). In other words, “Taniyama-Shimura Conjecture\n\nFermat’s Last Theorem”. Thus, the stage was set. A proof of the Taniyama-Shimura Conjecture (or enough of it to know that elliptic curves coming from Fermat equations are modular) would be a proof of Fermat’s Last Theorem.\n\n⇒\n\n1.3. Elliptic curves.\n\nDeﬁnition. An elliptic curve over Q is a nonsingular curve deﬁned by an equation of the form\n\ny2 + a1xy + a3y = x3 + a2x2 + a4x + a6\n\n(2)\n\nwhere the coeﬃcients ai are integers. The solution ( on the elliptic curve.\n\n, ∞\n\n) will be viewed as a point\n\n∞\n\n4\n\nK. RUBIN AND A. SILVERBERG\n\nRemarks. (i) A singular point on a curve f (x, y) = 0 is a point where both partial derivatives vanish. A curve is nonsingular if it has no singular points.\n\n(ii) Two elliptic curves over Q are isomorphic if one can be obtained from the other by changing coordinates x = A2x′ + B, y = A3y′ + Cx′ + D, with A, B, C, D\n\nQ and dividing through by A6.\n\n∈ (iii) Every elliptic curve over Q is isomorphic to one of the form\n\ny2 = x3 + a2x2 + a4x + a6\n\nwith integers ai. A curve of this form is nonsingular if and only if the cubic on the right side has no repeated roots.\n\nExample. The equation y2 = x(x + 32)(x\n\n42) deﬁnes an elliptic curve over Q.\n\n−\n\n1.4. Modularity. Let H denote the complex upper half plane C : Im(z) > 0 } where Im(z) is the imaginary part of z. If N is a positive integer, deﬁne a group of matrices\n\nz\n\n{\n\n∈\n\na b c d\n\nSL2(Z) : c is divisible by N\n\n.\n\nΓ0(N ) =\n\n∈\n\n(z) = az+b The group Γ0(N ) acts on H by linear fractional transformations cz+d . (cid:9) (cid:1) The quotient space H/Γ0(N ) is a (noncompact) Riemann surface. It can be com- pleted to a compact Riemann surface, denoted X0(N ), by adjoining a ﬁnite set of points called cusps. The cusps are the ﬁnitely many equivalence classes of Q ∞} under the action of Γ0(N ) (see Chapter 1 of [35]). The complex points of an elliptic curve can also be viewed as a compact Riemann surface.\n\na b c d\n\n(cid:8)(cid:0)\n\n(cid:1)\n\n(cid:0)\n\ni\n\n∪{\n\nDeﬁnition. An elliptic curve E is modular if, for some integer N , there is a holo- morphic map from X0(N ) onto E.\n\nExample. It can be shown that there is a (holomorphic) isomorphism from X0(15) onto the elliptic curve y2 = x(x + 32)(x\n\n42).\n\n−\n\nRemark . There are many equivalent deﬁnitions of modularity (see II.4.D of [24] and appendix of [22]). In some cases the equivalence is a deep result. For Wiles’ 1.7 proof of Fermat’s Last Theorem it suﬃces to use only the deﬁnition given in below.\n\n§\n\n§\n\n1.5. Semistability.\n\nDeﬁnition. An elliptic curve over Q is semistable at the prime q if it is isomorphic to an elliptic curve over Q which modulo q either is nonsingular or has a singu- lar point with two distinct tangent directions. An elliptic curve over Q is called semistable if it is semistable at every prime.\n\nExample. The elliptic curve y2 = x(x + 32)(x isomorphic to y2 + xy + y = x3 + x2 x(x + 42)(x\n\n42) is semistable because it is − 10, but the elliptic curve y2 =\n\n10x\n\n−\n\n−\n\n32) is not semistable (it is not semistable at 2).\n\n−\n\n2 we explain how Wiles shows that his main result on Galois representations (Theorem 5.3) implies the following part of the Taniyama-Shimura Conjecture.\n\nBeginning in\n\n§\n\nSemistable Taniyama-Shimura Conjecture. Every semistable elliptic curve over Q is modular.\n\nA REPORT ON WILES’ CAMBRIDGE LECTURES\n\n5\n\nProposition 1.2. The Semistable Taniyama-Shimura Conjecture implies Theorem 1.1.\n\nProof. If A and B are distinct, nonzero, relatively prime integers, write EA,B for the elliptic curve deﬁned by y2 = x(x + A)(x + B). Since EA,B and E−A,−B are isomorphic over the complex numbers (i.e., as Riemann surfaces), EA,B is modular if and only if E−A,−B is modular. If further AB(A B) is divisible by 16, then either EA,B or E−A,−B is semistable (this is easy to check directly; see for example I.1 of [24]). The Semistable Taniyama-Shimura Conjecture now implies that both § EA,B and E−A,−B are modular, and thus implies Theorem 1.1.\n\n−\n\nRemark . In 1.1 we saw that Theorem 1.1 and Ribet’s Theorem together imply Fermat’s Last Theorem. Therefore, the Semistable Taniyama-Shimura Conjecture implies Fermat’s Last Theorem.\n\n§\n\n1.6. Modular forms. In this paper we will work with a deﬁnition of modularity which uses modular forms.\n\nDeﬁnition. If N is a positive integer, a modular form f of weight k for Γ0(N ) is C which satisﬁes a holomorphic function f : H\n\n→\n\nf (γ(z)) = (cz + d)kf (z)\n\na b c d\n\nH,\n\n(3)\n\nΓ0(N ) and z\n\nfor every γ =\n\n∈\n\n∈\n\n(cid:1)\n\n(cid:0)\n\nand is holomorphic at the cusps (see Chapter 2 of [35]).\n\n1 1 0 1\n\nΓ0(N )), so ∞ n=0 ane2πinz, with complex numbers an and it has a Fourier expansion f (z) = (cid:1) . We say f is a cusp form if it with n vanishes at all the cusps; in particular for a cusp form the coeﬃcient a0 (the value at i\n\nA modu