GitHub | 🦜🔗 Langchain

Main:  On this page #GitHub This notebooks shows how you can load issues and pull requests (PRs) for a given repository on [GitHub](https://github.com/). We will use the LangChain Python repository as an example.  ##Setup access token[](#setup-access-token) To access the GitHub API, you need a personal access token - you can set up yours here: [https://github.com/settings/tokens?type=beta](https://github.com/settings/tokens?type=beta). You can either set this token as the environment variable GITHUB_PERSONAL_ACCESS_TOKEN and it will be automatically pulled in, or you can pass it in directly at initialization as the access_token named parameter.  # If you haven't set your access token as an environment variable, pass it in here. from getpass import getpass  ACCESS_TOKEN = getpass()  ##Load Issues and PRs[](#load-issues-and-prs) from langchain.document_loaders import GitHubIssuesLoader  loader = GitHubIssuesLoader(     repo="langchain-ai/langchain",     access_token=ACCESS_TOKEN,  # delete/comment out this argument if you've set the access token as an env var.     creator="UmerHA", )  Let's load all issues and PRs created by "UmerHA".  Here's a list of all filters you can use:  - include_prs - milestone - state - assignee - creator - mentioned - labels - sort - direction - since For more info, see [https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#list-repository-issues](https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#list-repository-issues).  docs = loader.load()  print(docs[0].page_content) print(docs[0].metadata)      # Creates GitHubLoader (#5257)          GitHubLoader is a DocumentLoader that loads issues and PRs from GitHub.          Fixes #5257          Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:     DataLoaders     - @eyurtsev          {'url': 'https://github.com/langchain-ai/langchain/pull/5408', 'title': 'DocumentLoader for GitHub', 'creator': 'UmerHA', 'created_at': '2023-05-29T14:50:53Z', 'comments': 0, 'state': 'open', 'labels': ['enhancement', 'lgtm', 'doc loader'], 'assignee': None, 'milestone': None, 'locked': False, 'number': 5408, 'is_pull_request': True}  ##Only load issues[](#only-load-issues) By default, the GitHub API returns considers pull requests to also be issues. To only get 'pure' issues (i.e., no pull requests), use include_prs=False  loader = GitHubIssuesLoader(     repo="langchain-ai/langchain",     access_token=ACCESS_TOKEN,  # delete/comment out this argument if you've set the access token as an env var.     creator="UmerHA",     include_prs=False, ) docs = loader.load()  print(docs[0].page_content) print(docs[0].metadata)      ### System Info          LangChain version = 0.0.167     Python version = 3.11.0     System = Windows 11 (using Jupyter)          ### Who can help?          - @hwchase17     - @agola11     - @UmerHA (I have a fix ready, will submit a PR)          ### Information          - [ ] The official example notebooks/scripts     - [X] My own modified scripts          ### Related Components          - [X] LLMs/Chat Models     - [ ] Embedding Models     - [X] Prompts / Prompt Templates / Prompt Selectors     - [ ] Output Parsers     - [ ] Document Loaders     - [ ] Vector Stores / Retrievers     - [ ] Memory     - [ ] Agents / Agent Executors     - [ ] Tools / Toolkits     - [ ] Chains     - [ ] Callbacks/Tracing     - [ ] Async          ### Reproduction          ```     import os     os.environ["OPENAI_API_KEY"] = "..."          from langchain.chains import LLMChain     from langchain.chat_models import ChatOpenAI     from langchain.prompts import PromptTemplate     from langchain.prompts.chat import ChatPromptTemplate     from langchain.schema import messages_from_dict          role_strings = [         ("system", "you are a bird expert"),          ("human", "which bird has a point beak?")     ]     prompt = ChatPromptTemplate.from_role_strings(role_strings)     chain = LLMChain(llm=ChatOpenAI(), prompt=prompt)     chain.run({})     ```          ### Expected behavior          Chain should run     {'url': 'https://github.com/langchain-ai/langchain/issues/5027', 'title': "ChatOpenAI models don't work with prompts created via ChatPromptTemplate.from_role_strings", 'creator': 'UmerHA', 'created_at': '2023-05-20T10:39:18Z', 'comments': 1, 'state': 'open', 'labels': [], 'assignee': None, 'milestone': None, 'locked': False, 'number': 5027, 'is_pull_request': False}


Google BigQuery | 🦜🔗 Langchain

Main:  On this page #Google BigQuery [Google BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data. BigQuery is a part of the Google Cloud Platform.  Load a BigQuery query with one document per row.  #!pip install google-cloud-bigquery  from langchain.document_loaders import BigQueryLoader  BASE_QUERY = """ SELECT   id,   dna_sequence,   organism FROM (   SELECT     ARRAY (     SELECT       AS STRUCT 1 AS id, "ATTCGA" AS dna_sequence, "Lokiarchaeum sp. (strain GC14_75)." AS organism     UNION ALL     SELECT       AS STRUCT 2 AS id, "AGGCGA" AS dna_sequence, "Heimdallarchaeota archaeon (strain LC_2)." AS organism     UNION ALL     SELECT       AS STRUCT 3 AS id, "TCCGGA" AS dna_sequence, "Acidianus hospitalis (strain W1)." AS organism) AS new_array),   UNNEST(new_array) """  ##Basic Usage[](#basic-usage) loader = BigQueryLoader(BASE_QUERY)  data = loader.load()  print(data)      [Document(page_content='id: 1\ndna_sequence: ATTCGA\norganism: Lokiarchaeum sp. (strain GC14_75).', lookup_str='', metadata={}, lookup_index=0), Document(page_content='id: 2\ndna_sequence: AGGCGA\norganism: Heimdallarchaeota archaeon (strain LC_2).', lookup_str='', metadata={}, lookup_index=0), Document(page_content='id: 3\ndna_sequence: TCCGGA\norganism: Acidianus hospitalis (strain W1).', lookup_str='', metadata={}, lookup_index=0)]  ##Specifying Which Columns are Content vs Metadata[](#specifying-which-columns-are-content-vs-metadata) loader = BigQueryLoader(     BASE_QUERY,     page_content_columns=["dna_sequence", "organism"],     metadata_columns=["id"], )  data = loader.load()  print(data)      [Document(page_content='dna_sequence: ATTCGA\norganism: Lokiarchaeum sp. (strain GC14_75).', lookup_str='', metadata={'id': 1}, lookup_index=0), Document(page_content='dna_sequence: AGGCGA\norganism: Heimdallarchaeota archaeon (strain LC_2).', lookup_str='', metadata={'id': 2}, lookup_index=0), Document(page_content='dna_sequence: TCCGGA\norganism: Acidianus hospitalis (strain W1).', lookup_str='', metadata={'id': 3}, lookup_index=0)]  ##Adding Source to Metadata[](#adding-source-to-metadata) # Note that the `id` column is being returned twice, with one instance aliased as `source` ALIASED_QUERY = """ SELECT   id,   dna_sequence,   organism,   id as source FROM (   SELECT     ARRAY (     SELECT       AS STRUCT 1 AS id, "ATTCGA" AS dna_sequence, "Lokiarchaeum sp. (strain GC14_75)." AS organism     UNION ALL     SELECT       AS STRUCT 2 AS id, "AGGCGA" AS dna_sequence, "Heimdallarchaeota archaeon (strain LC_2)." AS organism     UNION ALL     SELECT       AS STRUCT 3 AS id, "TCCGGA" AS dna_sequence, "Acidianus hospitalis (strain W1)." AS organism) AS new_array),   UNNEST(new_array) """  loader = BigQueryLoader(ALIASED_QUERY, metadata_columns=["source"])  data = loader.load()  print(data)      [Document(page_content='id: 1\ndna_sequence: ATTCGA\norganism: Lokiarchaeum sp. (strain GC14_75).\nsource: 1', lookup_str='', metadata={'source': 1}, lookup_index=0), Document(page_content='id: 2\ndna_sequence: AGGCGA\norganism: Heimdallarchaeota archaeon (strain LC_2).\nsource: 2', lookup_str='', metadata={'source': 2}, lookup_index=0), Document(page_content='id: 3\ndna_sequence: TCCGGA\norganism: Acidianus hospitalis (strain W1).\nsource: 3', lookup_str='', metadata={'source': 3}, lookup_index=0)]


Google Cloud Storage Directory | 🦜🔗 Langchain

Main:  On this page #Google Cloud Storage Directory [Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.  This covers how to load document objects from an Google Cloud Storage (GCS) directory (bucket).  # !pip install google-cloud-storage  from langchain.document_loaders import GCSDirectoryLoader  loader = GCSDirectoryLoader(project_name="aist", bucket="testing-hwc")  loader.load()      /Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/       warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)     /Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/       warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)          [Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpz37njh7u/fake.docx'}, lookup_index=0)]  ##Specifying a prefix[](#specifying-a-prefix) You can also specify a prefix for more finegrained control over what files to load.  loader = GCSDirectoryLoader(project_name="aist", bucket="testing-hwc", prefix="fake")  loader.load()      /Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/       warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)     /Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/       warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)          [Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpylg6291i/fake.docx'}, lookup_index=0)]


Google Cloud Storage File | 🦜🔗 Langchain

Main:  #Google Cloud Storage File [Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.  This covers how to load document objects from an Google Cloud Storage (GCS) file object (blob).  # !pip install google-cloud-storage  from langchain.document_loaders import GCSFileLoader  loader = GCSFileLoader(project_name="aist", bucket="testing-hwc", blob="fake.docx")  loader.load()      /Users/harrisonchase/workplace/langchain/.venv/lib/python3.10/site-packages/google/auth/_default.py:83: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK without a quota project. You might receive a "quota exceeded" or "API not enabled" error. We recommend you rerun `gcloud auth application-default login` and make sure a quota project is added. Or you can use service accounts instead. For more information about service accounts, see https://cloud.google.com/docs/authentication/       warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)          [Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmp3srlf8n8/fake.docx'}, lookup_index=0)]  If you want to use an alternative loader, you can provide a custom function, for example:  from langchain.document_loaders import PyPDFLoader   def load_pdf(file_path):     return PyPDFLoader(file_path)   loader = GCSFileLoader(     project_name="aist", bucket="testing-hwc", blob="fake.pdf", loader_func=load_pdf )


Google Drive | 🦜🔗 Langchain

Main:  On this page #Google Drive [Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.  This notebook covers how to load documents from Google Drive. Currently, only Google Docs are supported.  ##Prerequisites[](#prerequisites) - Create a Google Cloud project or use an existing project - Enable the [Google Drive API](https://console.cloud.google.com/flows/enableapi?apiid=drive.googleapis.com) - [Authorize credentials for desktop app](https://developers.google.com/drive/api/quickstart/python#authorize_credentials_for_a_desktop_application) - pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib ##🧑 Instructions for ingesting your Google Docs data[](#-instructions-for-ingesting-your-google-docs-data) By default, the GoogleDriveLoader expects the credentials.json file to be ~/.credentials/credentials.json, but this is configurable using the credentials_path keyword argument. Same thing with token.json - token_path. Note that token.json will be created automatically the first time you use the loader.  The first time you use GoogleDriveLoader, you will be displayed with the consent screen in your browser. If this doesn't happen and you get a RefreshError, do not use credentials_path in your GoogleDriveLoader constructor call. Instead, put that path in a GOOGLE_APPLICATION_CREDENTIALS environmental variable.  GoogleDriveLoader can load from a list of Google Docs document ids or a folder id. You can obtain your folder and document id from the URL:  - Folder: [https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5](https://drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5) -> folder id is "1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5" - Document: [https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit](https://docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit) -> document id is "1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw" pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib  from langchain.document_loaders import GoogleDriveLoader  loader = GoogleDriveLoader(     folder_id="1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5",     token_path="/path/where/you/want/token/to/be/created/google_token.json",     # Optional: configure whether to recursively fetch files from subfolders. Defaults to False.     recursive=False, )  docs = loader.load()  When you pass a folder_id by default all files of type document, sheet and pdf are loaded. You can modify this behaviour by passing a file_types argument  loader = GoogleDriveLoader(     folder_id="1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5",     file_types=["document", "sheet"],     recursive=False, )  ##Passing in Optional File Loaders[](#passing-in-optional-file-loaders) When processing files other than Google Docs and Google Sheets, it can be helpful to pass an optional file loader to GoogleDriveLoader. If you pass in a file loader, that file loader will be used on documents that do not have a Google Docs or Google Sheets MIME type. Here is an example of how to load an Excel document from Google Drive using a file loader.  from langchain.document_loaders import GoogleDriveLoader, UnstructuredFileIOLoader  file_id = "1x9WBtFPWMEAdjcJzPScRsjpjQvpSo_kz" loader = GoogleDriveLoader(     file_ids=[file_id],     file_loader_cls=UnstructuredFileIOLoader,     file_loader_kwargs={"mode": "elements"}, )  docs = loader.load()  docs[0]  You can also process a folder with a mix of files and Google Docs/Sheets using the following pattern:  folder_id = "1asMOHY1BqBS84JcRbOag5LOJac74gpmD" loader = GoogleDriveLoader(     folder_id=folder_id,     file_loader_cls=UnstructuredFileIOLoader,     file_loader_kwargs={"mode": "elements"}, )  docs = loader.load()  docs[0]  ##Extended usage[](#extended-usage) An external component can manage the complexity of Google Drive : langchain-googledrive It's compatible with the ̀langchain.document_loaders.GoogleDriveLoader and can be used in its place.  To be compatible with containers, the authentication uses an environment variable ̀GOOGLE_ACCOUNT_FILE` to credential file (for user or service).  pip install langchain-googledrive  folder_id = "root" # folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'  # Use the advanced version. from langchain_googledrive.document_loaders import GoogleDriveLoader  loader = GoogleDriveLoader(     folder_id=folder_id,     recursive=False,     num_results=2,  # Maximum number of file to load )  By default, all files with these mime-type can be converted to Document.  - text/text - text/plain - text/html - text/csv - text/markdown - image/png - image/jpeg - application/epub+zip - application/pdf - application/rtf - application/vnd.google-apps.document (GDoc) - application/vnd.google-apps.presentation (GSlide) - application/vnd.google-apps.spreadsheet (GSheet) - application/vnd.google.colaboratory (Notebook colab) - application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX) - application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX) It's possible to update or customize this. See the documentation of GDriveLoader.  But, the corresponding packages must be installed.  pip install unstructured  for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")  ###Customize the search pattern[](#customize-the-search-pattern) All parameter compatible with Google [list()](https://developers.google.com/drive/api/v3/reference/files/list) API can be set.  To specify the new pattern of the Google request, you can use a PromptTemplate(). The variables for the prompt can be set with kwargs in the constructor. Some pre-formated request are proposed (use {query}, {folder_id} and/or {mime_type}):  You can customize the criteria to select the files. A set of predefined filter are proposed: | template | description | | -------------------------------------


Google Drive | 🦜🔗 Langchain

- | --------------------------------------------------------------------- | | gdrive-all-in-folder | Return all compatible files from a folder_id | | gdrive-query | Search query in all drives | | gdrive-by-name | Search file with name query | | gdrive-query-in-folder | Search query in folder_id (and sub-folders if recursive=true) | | gdrive-mime-type | Search a specific mime_type | | gdrive-mime-type-in-folder | Search a specific mime_type in folder_id | | gdrive-query-with-mime-type | Search query with a specific mime_type | | gdrive-query-with-mime-type-and-folder | Search query with a specific mime_type and in folder_id |  loader = GoogleDriveLoader(     folder_id=folder_id,     recursive=False,     template="gdrive-query",  # Default template to use     query="machine learning",     num_results=2,  # Maximum number of file to load     supportsAllDrives=False,  # GDrive `list()` parameter ) for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")  You can customize your pattern.  from langchain.prompts.prompt import PromptTemplate  loader = GoogleDriveLoader(     folder_id=folder_id,     recursive=False,     template=PromptTemplate(         input_variables=["query", "query_name"],         template="fullText contains '{query}' and name contains '{query_name}' and trashed=false",     ),  # Default template to use     query="machine learning",     query_name="ML",     num_results=2,  # Maximum number of file to load ) for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")  ####Modes for GSlide and GSheet[](#modes-for-gslide-and-gsheet) The parameter mode accepts different values:  - "document": return the body of each document - "snippets": return the description of each file (set in metadata of Google Drive files). The conversion can manage in Markdown format:  - bullet - link - table - titles The parameter gslide_mode accepts different values:  - "single" : one document with <PAGE BREAK> - "slide" : one document by slide - "elements" : one document for each elements. loader = GoogleDriveLoader(     template="gdrive-mime-type",     mime_type="application/vnd.google-apps.presentation",  # Only GSlide files     gslide_mode="slide",     num_results=2,  # Maximum number of file to load ) for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")  The parameter gsheet_mode accepts different values:  - "single": Generate one document by line - "elements" : one document with markdown array and <PAGE BREAK> tags. loader = GoogleDriveLoader(     template="gdrive-mime-type",     mime_type="application/vnd.google-apps.spreadsheet",  # Only GSheet files     gsheet_mode="elements",     num_results=2,  # Maximum number of file to load ) for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")  ###Advanced usage[](#advanced-usage) All Google File have a 'description' in the metadata. This field can be used to memorize a summary of the document or others indexed tags (See method lazy_update_description_with_summary()).  If you use the mode="snippet", only the description will be used for the body. Else, the metadata['summary'] has the field.  Sometime, a specific filter can be used to extract some information from the filename, to select some files with specific criteria. You can use a filter.  Sometimes, many documents are returned. It's not necessary to have all documents in memory at the same time. You can use the lazy versions of methods, to get one document at a time. It's better to use a complex query in place of a recursive search. For each folder, a query must be applied if you activate recursive=True.  import os  loader = GoogleDriveLoader(     gdrive_api_file=os.environ["GOOGLE_ACCOUNT_FILE"],     num_results=2,     template="gdrive-query",     filter=lambda search, file: "#test" not in file.get("description", ""),     query="machine learning",     supportsAllDrives=False, ) for doc in loader.load():     print("---")     print(doc.page_content.strip()[:60] + "...")


Google Speech-to-Text Audio Transcripts | 🦜🔗 Langchain

Main:  On this page #Google Speech-to-Text Audio Transcripts The GoogleSpeechToTextLoader allows to transcribe audio files with the [Google Cloud Speech-to-Text API](https://cloud.google.com/speech-to-text) and loads the transcribed text into documents.  To use it, you should have the google-cloud-speech python package installed, and a Google Cloud project with the [Speech-to-Text API enabled](https://cloud.google.com/speech-to-text/v2/docs/transcribe-client-libraries#before_you_begin).  - [Bringing the power of large models to Google Cloud’s Speech API](https://cloud.google.com/blog/products/ai-machine-learning/bringing-power-large-models-google-clouds-speech-api) ##Installation & setup[](#installation--setup) First, you need to install the google-cloud-speech python package.  You can find more info about it on the [Speech-to-Text client libraries](https://cloud.google.com/speech-to-text/v2/docs/libraries) page.  Follow the [quickstart guide](https://cloud.google.com/speech-to-text/v2/docs/sync-recognize) in the Google Cloud documentation to create a project and enable the API.  %pip install google-cloud-speech  ##Example[](#example) The GoogleSpeechToTextLoader must include the project_id and file_path arguments. Audio files can be specified as a Google Cloud Storage URI (gs://...) or a local file path.  Only synchronous requests are supported by the loader, which has a [limit of 60 seconds or 10MB](https://cloud.google.com/speech-to-text/v2/docs/sync-recognize#:~:text=60%20seconds%20and/or%2010%20MB) per audio file.  from langchain.document_loaders import GoogleSpeechToTextLoader  project_id = "<PROJECT_ID>" file_path = "gs://cloud-samples-data/speech/audio.flac" # or a local file path: file_path = "./audio.wav"  loader = GoogleSpeechToTextLoader(project_id=project_id, file_path=file_path)  docs = loader.load()  Note: Calling loader.load() blocks until the transcription is finished.  The transcribed text is available in the page_content:  docs[0].page_content  "How old is the Brooklyn Bridge?"  The metadata contains the full JSON response with more meta information:  docs[0].metadata  {   'language_code': 'en-US',   'result_end_offset': datetime.timedelta(seconds=1) }  ##Recognition Config[](#recognition-config) You can specify the config argument to use different speech recognition models and enable specific features.  Refer to the [Speech-to-Text recognizers documentation](https://cloud.google.com/speech-to-text/v2/docs/recognizers) and the [RecognizeRequest](https://cloud.google.com/python/docs/reference/speech/latest/google.cloud.speech_v2.types.RecognizeRequest) API reference for information on how to set a custom configuation.  If you don't specify a config, the following options will be selected automatically:  - Model: [Chirp Universal Speech Model](https://cloud.google.com/speech-to-text/v2/docs/chirp-model) - Language: en-US - Audio Encoding: Automatically Detected - Automatic Punctuation: Enabled from google.cloud.speech_v2 import (     AutoDetectDecodingConfig,     RecognitionConfig,     RecognitionFeatures, ) from langchain.document_loaders import GoogleSpeechToTextLoader  project_id = "<PROJECT_ID>" location = "global" recognizer_id = "<RECOGNIZER_ID>" file_path = "./audio.wav"  config = RecognitionConfig(     auto_decoding_config=AutoDetectDecodingConfig(),     language_codes=["en-US"],     model="long",     features=RecognitionFeatures(         enable_automatic_punctuation=False,         profanity_filter=True,         enable_spoken_punctuation=True,         enable_spoken_emojis=True,     ), )  loader = GoogleSpeechToTextLoader(     project_id=project_id,     location=location,     recognizer_id=recognizer_id,     file_path=file_path,     config=config, )


Grobid | 🦜🔗 Langchain

Main:  #Grobid GROBID is a machine learning library for extracting, parsing, and re-structuring raw documents.  It is designed and expected to be used to parse academic papers, where it works particularly well. Note: if the articles supplied to Grobid are large documents (e.g. dissertations) exceeding a certain number of elements, they might not be processed.  This loader uses Grobid to parse PDFs into Documents that retain metadata associated with the section of text.  The best approach is to install Grobid via docker, see [https://grobid.readthedocs.io/en/latest/Grobid-docker/](https://grobid.readthedocs.io/en/latest/Grobid-docker/).  (Note: additional instructions can be found [here](https://python.langchain.com/docs/docs/integrations/providers/grobid.mdx).)  Once grobid is up-and-running you can interact as described below.  Now, we can use the data loader.  from langchain.document_loaders.generic import GenericLoader from langchain.document_loaders.parsers import GrobidParser  loader = GenericLoader.from_filesystem(     "../Papers/",     glob="*",     suffixes=[".pdf"],     parser=GrobidParser(segment_sentences=False), ) docs = loader.load()  docs[3].page_content      'Unlike Chinchilla, PaLM, or GPT-3, we only use publicly available data, making our work compatible with open-sourcing, while most existing models rely on data which is either not publicly available or undocumented (e.g."Books -2TB" or "Social media conversations").There exist some exceptions, notably OPT (Zhang et al., 2022), GPT-NeoX (Black et al., 2022), BLOOM (Scao et al., 2022) and GLM (Zeng et al., 2022), but none that are competitive with PaLM-62B or Chinchilla.'  docs[3].metadata      {'text': 'Unlike Chinchilla, PaLM, or GPT-3, we only use publicly available data, making our work compatible with open-sourcing, while most existing models rely on data which is either not publicly available or undocumented (e.g."Books -2TB" or "Social media conversations").There exist some exceptions, notably OPT (Zhang et al., 2022), GPT-NeoX (Black et al., 2022), BLOOM (Scao et al., 2022) and GLM (Zeng et al., 2022), but none that are competitive with PaLM-62B or Chinchilla.',      'para': '2',      'bboxes': "[[{'page': '1', 'x': '317.05', 'y': '509.17', 'h': '207.73', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '522.72', 'h': '220.08', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '536.27', 'h': '218.27', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '549.82', 'h': '218.65', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '563.37', 'h': '136.98', 'w': '9.46'}], [{'page': '1', 'x': '446.49', 'y': '563.37', 'h': '78.11', 'w': '9.46'}, {'page': '1', 'x': '304.69', 'y': '576.92', 'h': '138.32', 'w': '9.46'}], [{'page': '1', 'x': '447.75', 'y': '576.92', 'h': '76.66', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '590.47', 'h': '219.63', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '604.02', 'h': '218.27', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '617.56', 'h': '218.27', 'w': '9.46'}, {'page': '1', 'x': '306.14', 'y': '631.11', 'h': '220.18', 'w': '9.46'}]]",      'pages': "('1', '1')",      'section_title': 'Introduction',      'section_number': '1',      'paper_title': 'LLaMA: Open and Efficient Foundation Language Models',      'file_path': '/Users/31treehaus/Desktop/Papers/2302.13971.pdf'}


Gutenberg | 🦜🔗 Langchain

Main:  #Gutenberg [Project Gutenberg](https://www.gutenberg.org/about/) is an online library of free eBooks.  This notebook covers how to load links to Gutenberg e-books into a document format that we can use downstream.  from langchain.document_loaders import GutenbergLoader  loader = GutenbergLoader("https://www.gutenberg.org/cache/epub/69972/pg69972.txt")  data = loader.load()  data[0].page_content[:300]      'The Project Gutenberg eBook of The changed brides, by Emma Dorothy\r\n\n\nEliza Nevitte Southworth\r\n\n\n\r\n\n\nThis eBook is for the use of anyone anywhere in the United States and\r\n\n\nmost other parts of the world at no cost and with almost no restrictions\r\n\n\nwhatsoever. You may copy it, give it away or re-u'  data[0].metadata      {'source': 'https://www.gutenberg.org/cache/epub/69972/pg69972.txt'}


Hacker News | 🦜🔗 Langchain

Main:  #Hacker News [Hacker News](https://en.wikipedia.org/wiki/Hacker_News) (sometimes abbreviated as HN) is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. In general, content that can be submitted is defined as "anything that gratifies one's intellectual curiosity."  This notebook covers how to pull page data and comments from [Hacker News](https://news.ycombinator.com/)  from langchain.document_loaders import HNLoader  loader = HNLoader("https://news.ycombinator.com/item?id=34817881")  data = loader.load()  data[0].page_content[:300]      "delta_p_delta_x 73 days ago  \n             | next [–] \n\nAstrophysical and cosmological simulations are often insightful. They're also very cross-disciplinary; besides the obvious astrophysics, there's networking and sysadmin, parallel computing and algorithm theory (so that the simulation programs a"  data[0].metadata      {'source': 'https://news.ycombinator.com/item?id=34817881',      'title': 'What Lights the Universe’s Standard Candles?'}


Huawei OBS Directory | 🦜🔗 Langchain

Main:  On this page #Huawei OBS Directory The following code demonstrates how to load objects from the Huawei OBS (Object Storage Service) as documents.  # Install the required package # pip install esdk-obs-python  from langchain.document_loaders import OBSDirectoryLoader  endpoint = "your-endpoint"  # Configure your access credentials\n config = {"ak": "your-access-key", "sk": "your-secret-key"} loader = OBSDirectoryLoader("your-bucket-name", endpoint=endpoint, config=config)  loader.load()  ##Specify a Prefix for Loading[](#specify-a-prefix-for-loading) If you want to load objects with a specific prefix from the bucket, you can use the following code:  loader = OBSDirectoryLoader(     "your-bucket-name", endpoint=endpoint, config=config, prefix="test_prefix" )  loader.load()  ##Get Authentication Information from ECS[](#get-authentication-information-from-ecs) If your langchain is deployed on Huawei Cloud ECS and [Agency is set up](https://support.huaweicloud.com/intl/en-us/usermanual-ecs/ecs_03_0166.html#section7), the loader can directly get the security token from ECS without needing access key and secret key.  config = {"get_token_from_ecs": True} loader = OBSDirectoryLoader("your-bucket-name", endpoint=endpoint, config=config)  loader.load()  ##Use a Public Bucket[](#use-a-public-bucket) If your bucket's bucket policy allows anonymous access (anonymous users have listBucket and GetObject permissions), you can directly load the objects without configuring the config parameter.  loader = OBSDirectoryLoader("your-bucket-name", endpoint=endpoint)  loader.load()


Huawei OBS File | 🦜🔗 Langchain

Main:  On this page #Huawei OBS File The following code demonstrates how to load an object from the Huawei OBS (Object Storage Service) as document.  # Install the required package # pip install esdk-obs-python  from langchain.document_loaders.obs_file import OBSFileLoader  endpoint = "your-endpoint"  from obs import ObsClient  obs_client = ObsClient(     access_key_id="your-access-key",     secret_access_key="your-secret-key",     server=endpoint, ) loader = OBSFileLoader("your-bucket-name", "your-object-key", client=obs_client)  loader.load()  ##Each Loader with Separate Authentication Information[](#each-loader-with-separate-authentication-information) If you don't need to reuse OBS connections between different loaders, you can directly configure the config. The loader will use the config information to initialize its own OBS client.  # Configure your access credentials\n config = {"ak": "your-access-key", "sk": "your-secret-key"} loader = OBSFileLoader(     "your-bucket-name", "your-object-key", endpoint=endpoint, config=config )  loader.load()  ##Get Authentication Information from ECS[](#get-authentication-information-from-ecs) If your langchain is deployed on Huawei Cloud ECS and [Agency is set up](https://support.huaweicloud.com/intl/en-us/usermanual-ecs/ecs_03_0166.html#section7), the loader can directly get the security token from ECS without needing access key and secret key.  config = {"get_token_from_ecs": True} loader = OBSFileLoader(     "your-bucket-name", "your-object-key", endpoint=endpoint, config=config )  loader.load()  ##Access a Publicly Accessible Object[](#access-a-publicly-accessible-object) If the object you want to access allows anonymous user access (anonymous users have GetObject permission), you can directly load the object without configuring the config parameter.  loader = OBSFileLoader("your-bucket-name", "your-object-key", endpoint=endpoint)  loader.load()


HuggingFace dataset | 🦜🔗 Langchain

Main:  On this page #HuggingFace dataset The [Hugging Face Hub](https://huggingface.co/docs/hub/index) is home to over 5,000 [datasets](https://huggingface.co/docs/hub/index#datasets) in more than 100 languages that can be used for a broad range of tasks across NLP, Computer Vision, and Audio. They used for a diverse range of tasks such as translation, automatic speech recognition, and image classification.  This notebook shows how to load Hugging Face Hub datasets to LangChain.  from langchain.document_loaders import HuggingFaceDatasetLoader  dataset_name = "imdb" page_content_column = "text"   loader = HuggingFaceDatasetLoader(dataset_name, page_content_column)  data = loader.load()  data[:15]      [Document(page_content='I rented I AM CURIOUS-YELLOW from my video store because of all the controversy that surrounded it when it was first released in 1967. I also heard that at first it was seized by U.S. customs if it ever tried to enter this country, therefore being a fan of films considered "controversial" I really had to see this for myself.

The plot is centered around a young Swedish drama student named Lena who wants to learn everything she can about life. In particular she wants to focus her attentions to making some sort of documentary on what the average Swede thought about certain political issues such as the Vietnam War and race issues in the United States. In between asking politicians and ordinary denizens of Stockholm about their opinions on politics, she has sex with her drama teacher, classmates, and married men.

What kills me about I AM CURIOUS-YELLOW is that 40 years ago, this was considered pornographic. Really, the sex and nudity scenes are few and far between, even then it\'s not shot like some cheaply made porno. While my countrymen mind find it shocking, in reality sex and nudity are a major staple in Swedish cinema. Even Ingmar Bergman, arguably their answer to good old boy John Ford, had sex scenes in his films.

I do commend the filmmakers for the fact that any sex shown in the film is shown for artistic purposes rather than just to shock people and make money to be shown in pornographic theaters in America. I AM CURIOUS-YELLOW is a good film for anyone wanting to study the meat and potatoes (no pun intended) of Swedish cinema. But really, this film doesn\'t have much of a plot.', metadata={'label': 0}),      Document(page_content='"I Am Curious: Yellow" is a risible and pretentious steaming pile. It doesn\'t matter what one\'s political views are because this film can hardly be taken seriously on any level. As for the claim that frontal male nudity is an automatic NC-17, that isn\'t true. I\'ve seen R-rated films with male nudity. Granted, they only offer some fleeting views, but where are the R-rated films with gaping vulvas and flapping labia? Nowhere, because they don\'t exist. The same goes for those crappy cable shows: schlongs swinging in the breeze but not a clitoris in sight. And those pretentious indie movies like The Brown Bunny, in which we\'re treated to the site of Vincent Gallo\'s throbbing johnson, but not a trace of pink visible on Chloe Sevigny. Before crying (or implying) "double-standard" in matters of nudity, the mentally obtuse should take into account one unavoidably obvious anatomical difference between men and women: there are no genitals on display when actresses appears nude, and the same cannot be said for a man. In fact, you generally won\'t see female genitals in an American film in anything short of porn or explicit erotica. This alleged double-standard is less a double standard than an admittedly depressing ability to come to terms culturally with the insides of women\'s bodies.', metadata={'label': 0}),      Document(page_content="If only to avoid making this type of film in the future. This film is interesting as an experiment but tells no cogent story.

One might feel virtuous for sitting thru it because it touches on so many IMPORTANT issues but it does so without any discernable motive. The viewer comes away with no new perspectives (unless one comes up with one while one's mind wanders, as it will invariably do during this pointless film).

One might better spend one's time staring out a window at a tree growing.

", metadata={'label': 0}),      Document(page_content="This film was probably inspired by Godard's Masculin, féminin and I urge you to see that film instead.

The film has two strong elements and those are, (1) the realistic acting (2) the impressive, undeservedly good, photo. Apart from that, what strikes me most is the endless stream of silliness. Lena Nyman has to be most annoying actress in the world. She acts so stupid and with all the nudity in this film,...it's unattractive. Comparing to Godard's film, intellectuality has been replaced with stupidity. Without going too far on this subject, I would say that follows from the difference in ideals between the French and the Swedish society.

A movie of its time, and place. 2/10.", metadata={'label': 0}),      Document(page_content='Oh, brother...after hearing about this ridiculous film for umpteen years all I can think of is that old Peggy Lee song..

"Is that all there is??" ...I was just an early teen when this smoked fish hit the U.S. I was too young to get in the theater (although I did manage to sneak into "Goodbye Columbus"). Then a screening at a local film museum beckoned - Finally I could see this film, except now I was as old as my parents were when they schlepped to see it!!

The ONLY reason this film was not condemned to the anonymous sands of time was because of the obscenity case sparked by its U.S. release. MILLIONS of people flocked to this stinker, thinking they were going to see a sex film...Instead, they got lots of closeups of gnarly, repulsive Swedes, on-street interviews in bland shopping malls, asinie political pretension...and feeble who-c


HuggingFace dataset | 🦜🔗 Langchain

ares simulated sex scenes with saggy, pale actors.

Cultural icon, holy grail, historic artifact..whatever this thing was, shred it, burn it, then stuff the ashes in a lead box!

Elite esthetes still scrape to find value in its boring pseudo revolutionary political spewings..But if it weren\'t for the censorship scandal, it would have been ignored, then forgotten.

Instead, the "I Am Blank, Blank" rhythymed title was repeated endlessly for years as a titilation for porno films (I am Curious, Lavender - for gay films, I Am Curious, Black - for blaxploitation films, etc..) and every ten years or so the thing rises from the dead, to be viewed by a new generation of suckers who want to see that "naughty sex film" that "revolutionized the film industry"...

Yeesh, avoid like the plague..Or if you MUST see it - rent the video and fast forward to the "dirty" parts, just to get it over with.

', metadata={'label': 0}),      Document(page_content="I would put this at the top of my list of films in the category of unwatchable trash! There are films that are bad, but the worst kind are the ones that are unwatchable but you are suppose to like them because they are supposed to be good for you! The sex sequences, so shocking in its day, couldn't even arouse a rabbit. The so called controversial politics is strictly high school sophomore amateur night Marxism. The film is self-consciously arty in the worst sense of the term. The photography is in a harsh grainy black and white. Some scenes are out of focus or taken from the wrong angle. Even the sound is bad! And some people call this art?

", metadata={'label': 0}),      Document(page_content="Whoever wrote the screenplay for this movie obviously never consulted any books about Lucille Ball, especially her autobiography. I've never seen so many mistakes in a biopic, ranging from her early years in Celoron and Jamestown to her later years with Desi. I could write a whole list of factual errors, but it would go on for pages. In all, I believe that Lucille Ball is one of those inimitable people who simply cannot be portrayed by anyone other than themselves. If I were Lucie Arnaz and Desi, Jr., I would be irate at how many mistakes were made in this film. The filmmakers tried hard, but the movie seems awfully sloppy to me.", metadata={'label': 0}),      Document(page_content='When I first saw a glimpse of this movie, I quickly noticed the actress who was playing the role of Lucille Ball. Rachel York\'s portrayal of Lucy is absolutely awful. Lucille Ball was an astounding comedian with incredible talent. To think about a legend like Lucille Ball being portrayed the way she was in the movie is horrendous. I cannot believe out of all the actresses in the world who could play a much better Lucy, the producers decided to get Rachel York. She might be a good actress in other roles but to play the role of Lucille Ball is tough. It is pretty hard to find someone who could resemble Lucille Ball, but they could at least find someone a bit similar in looks and talent. If you noticed York\'s portrayal of Lucy in episodes of I Love Lucy like the chocolate factory or vitavetavegamin, nothing is similar in any way-her expression, voice, or movement.

To top it all off, Danny Pino playing Desi Arnaz is horrible. Pino does not qualify to play as Ricky. He\'s small and skinny, his accent is unreal, and once again, his acting is unbelievable. Although Fred and Ethel were not similar either, they were not as bad as the characters of Lucy and Ricky.

Overall, extremely horrible casting and the story is badly told. If people want to understand the real life situation of Lucille Ball, I suggest watching A&E Biography of Lucy and Desi, read the book from Lucille Ball herself, or PBS\' American Masters: Finding Lucy. If you want to see a docudrama, "Before the Laughter" would be a better choice. The casting of Lucille Ball and Desi Arnaz in "Before the Laughter" is much better compared to this. At least, a similar aspect is shown rather than nothing.', metadata={'label': 0}),      Document(page_content='Who are these "They"- the actors? the filmmakers? Certainly couldn\'t be the audience- this is among the most air-puffed productions in existence. It\'s the kind of movie that looks like it was a lot of fun to shoot\x97 TOO much fun, nobody is getting any actual work done, and that almost always makes for a movie that\'s no fun to watch.

Ritter dons glasses so as to hammer home his character\'s status as a sort of doppleganger of the bespectacled Bogdanovich; the scenes with the breezy Ms. Stratten are sweet, but have an embarrassing, look-guys-I\'m-dating-the-prom-queen feel to them. Ben Gazzara sports his usual cat\'s-got-canary grin in a futile attempt to elevate the meager plot, which requires him to pursue Audrey Hepburn with all the interest of a narcoleptic at an insomnia clinic. In the meantime, the budding couple\'s respective children (nepotism alert: Bogdanovich\'s daughters) spew cute and pick up some fairly disturbing pointers on \'love\' while observing their parents. (Ms. Hepburn, drawing on her dignity, manages to rise above the proceedings- but she has the monumental challenge of playing herself, ostensibly.) Everybody looks great, but so what? It\'s a movie and we can expect that much, if that\'s what you\'re looking for you\'d be better off picking up a copy of Vogue.

Oh- and it has to be mentioned that Colleen Camp thoroughly annoys, even apart from her singing, which, while competent, is wholly unconvincing... the country and western numbers are woefully mismatched with the standards on the soundtrack. Surely this is NOT what Gershwin (who wrote the song from which the movie\'s title is derived) had in mind; his stage musicals of the 20\'s may have been slight, but at least they were long on charm. "They All Laughed" tries to coast on its good intentions, but


HuggingFace dataset | 🦜🔗 Langchain

nobody- least of all Peter Bogdanovich - has the good sense to put on the brakes.

Due in no small part to the tragic death of Dorothy Stratten, this movie has a special place in the heart of Mr. Bogdanovich- he even bought it back from its producers, then distributed it on his own and went bankrupt when it didn\'t prove popular. His rise and fall is among the more sympathetic and tragic of Hollywood stories, so there\'s no joy in criticizing the film... there _is_ real emotional investment in Ms. Stratten\'s scenes. But "Laughed" is a faint echo of "The Last Picture Show", "Paper Moon" or "What\'s Up, Doc"- following "Daisy Miller" and "At Long Last Love", it was a thundering confirmation of the phase from which P.B. has never emerged.

All in all, though, the movie is harmless, only a waste of rental. I want to watch people having a good time, I\'ll go to the park on a sunny day. For filmic expressions of joy and love, I\'ll stick to Ernest Lubitsch and Jaques Demy...', metadata={'label': 0}),      Document(page_content="This is said to be a personal film for Peter Bogdonavitch. He based it on his life but changed things around to fit the characters, who are detectives. These detectives date beautiful models and have no problem getting them. Sounds more like a millionaire playboy filmmaker than a detective, doesn't it? This entire movie was written by Peter, and it shows how out of touch with real people he was. You're supposed to write what you know, and he did that, indeed. And leaves the audience bored and confused, and jealous, for that matter. This is a curio for people who want to see Dorothy Stratten, who was murdered right after filming. But Patti Hanson, who would, in real life, marry Keith Richards, was also a model, like Stratten, but is a lot better and has a more ample part. In fact, Stratten's part seemed forced; added. She doesn't have a lot to do with the story, which is pretty convoluted to begin with. All in all, every character in this film is somebody that very few people can relate with, unless you're millionaire from Manhattan with beautiful supermodels at your beckon call. For the rest of us, it's an irritating snore fest. That's what happens when you're out of touch. You entertain your few friends with inside jokes, and bore all the rest.", metadata={'label': 0}),      Document(page_content='It was great to see some of my favorite stars of 30 years ago including John Ritter, Ben Gazarra and Audrey Hepburn. They looked quite wonderful. But that was it. They were not given any characters or good lines to work with. I neither understood or cared what the characters were doing.

Some of the smaller female roles were fine, Patty Henson and Colleen Camp were quite competent and confident in their small sidekick parts. They showed some talent and it is sad they didn\'t go on to star in more and better films. Sadly, I didn\'t think Dorothy Stratten got a chance to act in this her only important film role.

The film appears to have some fans, and I was very open-minded when I started watching it. I am a big Peter Bogdanovich fan and I enjoyed his last movie, "Cat\'s Meow" and all his early ones from "Targets" to "Nickleodeon". So, it really surprised me that I was barely able to keep awake watching this one.

It is ironic that this movie is about a detective agency where the detectives and clients get romantically involved with each other. Five years later, Bogdanovich\'s ex-girlfriend, Cybil Shepherd had a hit television series called "Moonlighting" stealing the story idea from Bogdanovich. Of course, there was a great difference in that the series relied on tons of witty dialogue, while this tries to make do with slapstick and a few screwball lines.

Bottom line: It ain\'t no "Paper Moon" and only a very pale version of "What\'s Up, Doc".', metadata={'label': 0}),      Document(page_content="I can't believe that those praising this movie herein aren't thinking of some other film. I was prepared for the possibility that this would be awful, but the script (or lack thereof) makes for a film that's also pointless. On the plus side, the general level of craft on the part of the actors and technical crew is quite competent, but when you've got a sow's ear to work with you can't make a silk purse. Ben G fans should stick with just about any other movie he's been in. Dorothy S fans should stick to Galaxina. Peter B fans should stick to Last Picture Show and Target. Fans of cheap laughs at the expense of those who seem to be asking for it should stick to Peter B's amazingly awful book, Killing of the Unicorn.", metadata={'label': 0}),      Document(page_content='Never cast models and Playboy bunnies in your films! Bob Fosse\'s "Star 80" about Dorothy Stratten, of whom Bogdanovich was obsessed enough to have married her SISTER after her murder at the hands of her low-life husband, is a zillion times more interesting than Dorothy herself on the silver screen. Patty Hansen is no actress either..I expected to see some sort of lost masterpiece a la Orson Welles but instead got Audrey Hepburn cavorting in jeans and a god-awful "poodlesque" hair-do....Very disappointing...."Paper Moon" and "The Last Picture Show" I could watch again and again. This clunker I could barely sit through once. This movie was reputedly not released because of the brouhaha surrounding Ms. Stratten\'s tawdry death; I think the real reason was because it was so bad!', metadata={'label': 0}),      Document(page_content="Its not the cast. A finer group of actors, you could not find. Its not the setting. The director is in love with New York City, and by the end of the film, so are we all! Woody Allen could not improve upon what Bogdonovich has done here. If you are going to fall in love, or find love, Manhattan is the place to go. No, the problem with the movie is the script. There is none. The actors fall in love at first sight, words are unnecessary


HuggingFace dataset | 🦜🔗 Langchain

. In the director's own experience in Hollywood that is what happens when they go to work on the set. It is reality to him, and his peers, but it is a fantasy to most of us in the real world. So, in the end, the movie is hollow, and shallow, and message-less.", metadata={'label': 0}),      Document(page_content='Today I found "They All Laughed" on VHS on sale in a rental. It was a really old and very used VHS, I had no information about this movie, but I liked the references listed on its cover: the names of Peter Bogdanovich, Audrey Hepburn, John Ritter and specially Dorothy Stratten attracted me, the price was very low and I decided to risk and buy it. I searched IMDb, and the User Rating of 6.0 was an excellent reference. I looked in "Mick Martin & Marsha Porter Video & DVD Guide 2003" and \x96 wow \x96 four stars! So, I decided that I could not waste more time and immediately see it. Indeed, I have just finished watching "They All Laughed" and I found it a very boring overrated movie. The characters are badly developed, and I spent lots of minutes to understand their roles in the story. The plot is supposed to be funny (private eyes who fall in love for the women they are chasing), but I have not laughed along the whole story. The coincidences, in a huge city like New York, are ridiculous. Ben Gazarra as an attractive and very seductive man, with the women falling for him as if her were a Brad Pitt, Antonio Banderas or George Clooney, is quite ridiculous. In the end, the greater attractions certainly are the presence of the Playboy centerfold and playmate of the year Dorothy Stratten, murdered by her husband pretty after the release of this movie, and whose life was showed in "Star 80" and "Death of a Centerfold: The Dorothy Stratten Story"; the amazing beauty of the sexy Patti Hansen, the future Mrs. Keith Richards; the always wonderful, even being fifty-two years old, Audrey Hepburn; and the song "Amigo", from Roberto Carlos. Although I do not like him, Roberto Carlos has been the most popular Brazilian singer since the end of the 60\'s and is called by his fans as "The King". I will keep this movie in my collection only because of these attractions (manly Dorothy Stratten). My vote is four.

Title (Brazil): "Muito Riso e Muita Alegria" ("Many Laughs and Lots of Happiness")', metadata={'label': 0})]  ###Example[](#example) In this example, we use data from a dataset to answer a question  from langchain.document_loaders.hugging_face_dataset import HuggingFaceDatasetLoader from langchain.indexes import VectorstoreIndexCreator  dataset_name = "tweet_eval" page_content_column = "text" name = "stance_climate"   loader = HuggingFaceDatasetLoader(dataset_name, page_content_column, name)  index = VectorstoreIndexCreator().from_loaders([loader])      Found cached dataset tweet_eval          0%|          | 0/3 [00:00<?, ?it/s]       Using embedded DuckDB without persistence: data will be transient  query = "What are the most used hashtag?" result = index.query(query)  result      ' The most used hashtags in this context are #UKClimate2015, #Sustainability, #TakeDownTheFlag, #LoveWins, #CSOTA, #ClimateSummitoftheAmericas, #SM, and #SocialMedia.'


iFixit | 🦜🔗 Langchain

Main:  On this page #iFixit [iFixit](https://www.ifixit.com) is the largest, open repair community on the web. The site contains nearly 100k repair manuals, 200k Questions & Answers on 42k devices, and all the data is licensed under CC-BY-NC-SA 3.0.  This loader will allow you to download the text of a repair guide, text of Q&A's and wikis from devices on iFixit using their open APIs. It's incredibly useful for context related to technical documents and answers to questions about devices in the corpus of data on iFixit.  from langchain.document_loaders import IFixitLoader  loader = IFixitLoader("https://www.ifixit.com/Teardown/Banana+Teardown/811") data = loader.load()  data      [Document(page_content="# Banana Teardown\nIn this teardown, we open a banana to see what's inside.  Yellow and delicious, but most importantly, yellow.\n\n\n###Tools Required:\n\n - Fingers\n\n - Teeth\n\n - Thumbs\n\n\n###Parts Required:\n\n - None\n\n\n## Step 1\nTake one banana from the bunch.\nDon't squeeze too hard!\n\n\n## Step 2\nHold the banana in your left hand and grip the stem between your right thumb and forefinger.\n\n\n## Step 3\nPull the stem downward until the peel splits.\n\n\n## Step 4\nInsert your thumbs into the split of the peel and pull the two sides apart.\nExpose the top of the banana.  It may be slightly squished from pulling on the stem, but this will not affect the flavor.\n\n\n## Step 5\nPull open the peel, starting from your original split, and opening it along the length of the banana.\n\n\n## Step 6\nRemove fruit from peel.\n\n\n## Step 7\nEat and enjoy!\nThis is where you'll need your teeth.\nDo not choke on banana!\n", lookup_str='', metadata={'source': 'https://www.ifixit.com/Teardown/Banana+Teardown/811', 'title': 'Banana Teardown'}, lookup_index=0)]  loader = IFixitLoader(     "https://www.ifixit.com/Answers/View/318583/My+iPhone+6+is+typing+and+opening+apps+by+itself" ) data = loader.load()  data      [Document(page_content='# My iPhone 6 is typing and opening apps by itself\nmy iphone 6 is typing and opening apps by itself. How do i fix this. I just bought it last week.\nI restored as manufactures cleaned up the screen\nthe problem continues\n\n## 27 Answers\n\nFilter by: \n\nMost Helpful\nNewest\nOldest\n\n### Accepted Answer\nHi,\nWhere did you buy it? If you bought it from Apple or from an official retailer like Carphone warehouse etc. Then you\'ll have a year warranty and can get it replaced free.\nIf you bought it second hand, from a third part repair shop or online, then it may still have warranty, unless it is refurbished and has been repaired elsewhere.\nIf this is the case, it may be the screen that needs replacing to solve your issue.\nEither way, wherever you got it, it\'s best to return it and get a refund or a replacement device. :-)\n\n\n\n### Most Helpful Answer\nI had the same issues, screen freezing, opening apps by itself, selecting the screens and typing on it\'s own. I first suspected aliens and then ghosts and then hackers.\niPhone 6 is weak physically and tend to bend on pressure. And my phone had no case or cover.\nI took the phone to apple stores and they said sensors need to be replaced and possibly screen replacement as well. My phone is just 17 months old.\nHere is what I did two days ago and since then it is working like a charm..\nHold the phone in portrait (as if watching a movie). Twist it very very gently. do it few times.Rest the phone for 10 mins (put it on a flat surface). You can now notice those self typing things gone and screen getting stabilized.\nThen, reset the hardware (hold the power and home button till the screen goes off and comes back with apple logo). release the buttons when you see this.\nThen, connect to your laptop and log in to iTunes and reset your phone completely. (please take a back-up first).\nAnd your phone should be good to use again.\nWhat really happened here for me is that the sensors might have stuck to the screen and with mild twisting, they got disengaged/released.\nI posted this in Apple Community and the moderators deleted it, for the best reasons known to them.\nInstead of throwing away your phone (or selling cheaply), try this and you could be saving your phone.\nLet me know how it goes.\n\n\n\n### Other Answer\nIt was the charging cord! I bought a gas station braided cord and it was the culprit. Once I plugged my OEM cord into the phone the GHOSTS went away.\n\n\n\n### Other Answer\nI\'ve same issue that I just get resolved.  I first tried to restore it from iCloud back, however it was not a software issue or any virus issue, so after restore same problem continues. Then I get my phone to local area iphone repairing lab, and they detected that it is an LCD issue. LCD get out of order without any reason (It was neither hit or nor slipped, but LCD get out of order all and sudden, while using it) it started opening things at random. I get LCD replaced with new one, that cost me $80.00 in total  ($70.00 LCD charges + $10.00 as labor charges to fix it). iPhone is back to perfect mode now.  It was iphone 6s. Thanks.\n\n\n\n### Other Answer\nI was having the same issue with my 6 plus, I took it to a repair shop, they opened the phone, disconnected the three ribbons the screen has, blew up and cleaned the connectors and connected the screen again and it solved the issue… it’s hardware, not software.\n\n\n\n### Other Answer\nHey.\nJust had this problem now. As it turns out, you just need to plug in your phone. I use a case and when I took it off I noticed that there was a lot of dust and dirt around the areas that the case didn\'t cover. I shined a light in my ports and noticed they were filled with dust. Tomorrow I plan on using pressurized air to clean it out and the problem should be solved.  If you plug in your phone and unplug it and it stops the issue, I recommend cleaning your phone thoroughly.\n\n\n\n### Other Answer\nI simply changed the power supply and problem was gone. The block that plugs in the wal


iFixit | 🦜🔗 Langchain

l not the sub cord. The cord was fine but not the block.\n\n\n\n### Other Answer\nSomeone ask!  I purchased my iPhone 6s Plus for 1000 from at&t.  Before I touched it, I purchased a otter defender case.  I read where at&t said touch desease was due to dropping!  Bullshit!!  I am 56 I have never dropped it!! Looks brand new!  Never dropped or abused any way!  I have my original charger.  I am going to clean it and try everyone’s advice.  It really sucks!  I had 40,000,000 on my heart of Vegas slots!  I play every day.  I would be spinning and my fingers were no where max buttons and it would light up and switch to max.  It did it 3 times before I caught it light up by its self.  It sucks. Hope I can fix it!!!!\n\n\n\n### Other Answer\nNo answer, but same problem with iPhone 6 plus--random, self-generated jumping amongst apps and typing on its own--plus freezing regularly (aha--maybe that\'s what the "plus" in "6 plus" refers to?).  An Apple Genius recommended upgrading to iOS 11.3.1 from 11.2.2, to see if that fixed the trouble.  If it didn\'t, Apple will sell me a new phone for $168!  Of couese the OS upgrade didn\'t fix the problem.  Thanks for helping me figure out that it\'s most likely a hardware problem--which the "genius" probably knows too.\nI\'m getting ready to go Android.\n\n\n\n### Other Answer\nI experienced similar ghost touches.  Two weeks ago, I changed my iPhone 6 Plus shell (I had forced the phone into it because it’s pretty tight), and also put a new glass screen protector (the edges of the protector don’t stick to the screen, weird, so I brushed pressure on the edges at times to see if they may smooth out one day miraculously).  I’m not sure if I accidentally bend the phone when I installed the shell,  or, if I got a defective glass protector that messes up the touch sensor. Well, yesterday was the worse day, keeps dropping calls and ghost pressing keys for me when I was on a call.  I got fed up, so I removed the screen protector, and so far problems have not reoccurred yet. I’m crossing my fingers that problems indeed solved.\n\n\n\n### Other Answer\nthank you so much for this post! i was struggling doing the reset because i cannot type userids and passwords correctly because the iphone 6 plus i have kept on typing letters incorrectly. I have been doing it for a day until i come across this article. Very helpful! God bless you!!\n\n\n\n### Other Answer\nI just turned it off, and turned it back on.\n\n\n\n### Other Answer\nMy problem has not gone away completely but its better now i changed my charger and turned off prediction ....,,,now it rarely happens\n\n\n\n### Other Answer\nI tried all of the above. I then turned off my home cleaned it with isopropyl alcohol 90%. Then I baked it in my oven on warm for an hour and a half over foil. Took it out and set it cool completely on the glass top stove. Then I turned on and it worked.\n\n\n\n### Other Answer\nI think at& t should man up and fix your phone for free!  You pay a lot for a Apple they should back it.  I did the next 30 month payments and finally have it paid off in June.  My iPad sept.  Looking forward to a almost 100 drop in my phone bill!  Now this crap!!! Really\n\n\n\n### Other Answer\nIf your phone is JailBroken, suggest downloading a virus.  While all my symptoms were similar, there was indeed a virus/malware on the phone which allowed for remote control of my iphone (even while in lock mode).  My mistake for buying a third party iphone i suppose.  Anyway i have since had the phone restored to factory and everything is working as expected for now.  I will of course keep you posted if this changes.  Thanks to all for the helpful posts, really helped me narrow a few things down.\n\n\n\n### Other Answer\nWhen my phone was doing this, it ended up being the screen protector that i got from 5 below. I took it off and it stopped. I ordered more protectors from amazon and replaced it\n\n\n\n### Other Answer\niPhone 6 Plus first generation….I had the same issues as all above, apps opening by themselves, self typing, ultra sensitive screen, items jumping around all over….it even called someone on FaceTime twice by itself when I was not in the room…..I thought the phone was toast and i’d have to buy a new one took me a while to figure out but it was the extra cheap block plug I bought at a dollar store for convenience of an extra charging station when I move around the house from den to living room…..cord was fine but bought a new Apple brand block plug…no more problems works just fine now. This issue was a recent event so had to narrow things down to what had changed recently to my phone so I could figure it out.\nI even had the same problem on a laptop with documents opening up by themselves…..a laptop that was plugged in to the same wall plug as my phone charger with the dollar store block plug….until I changed the block plug.\n\n\n\n### Other Answer\nHad the problem: Inherited a 6s Plus from my wife. She had no problem with it.\nLooks like it was merely the cheap phone case I purchased on Amazon. It was either pinching the edges or torquing the screen/body of the phone. Problem solved.\n\n\n\n### Other Answer\nI bought my phone on march 6 and it was a brand new, but It sucks me uo because it freezing, shaking and control by itself. I went to the store where I bought this and I told them to replacr it, but they told me I have to pay it because Its about lcd issue. Please help me what other ways to fix it. Or should I try to remove the screen or should I follow your step above.\n\n\n\n### Other Answer\nI tried everything and it seems to come back to needing the original iPhone cable…or at least another 1 that would have come with another iPhone…not the $5 Store fast charging cables.  My original cable is pretty beat up - like most that I see - but I’ve been beaten up much MUCH less by sticking with its use!  I didn’t find that the casing/shell around it or not made any diff.\n\n\n\n### Other Answer\ngreat now I hav


iFixit | 🦜🔗 Langchain

e to wait one more hour to reset my phone and while I was tryin to connect my phone to my computer the computer also restarted smh does anyone else knows how I can get my phone to work… my problem is I have a black dot on the bottom left of my screen an it wont allow me to touch a certain part of my screen unless I rotate my phone and I know the password but the first number is a 2 and it won\'t let me touch 1,2, or 3 so now I have to find a way to get rid of my password and all of a sudden my phone wants to touch stuff on its own which got my phone disabled many times to the point where I have to wait a whole hour and I really need to finish something on my phone today PLEASE HELPPPP\n\n\n\n### Other Answer\nIn my case , iphone 6 screen was faulty. I got it replaced at local repair shop, so far phone is working fine.\n\n\n\n### Other Answer\nthis problem in iphone 6 has many different scenarios and solutions, first try to reconnect the lcd screen to the motherboard again, if didnt solve, try to replace the lcd connector on the motherboard, if not solved, then remains two issues, lcd screen it self or touch IC. in my country some repair shops just change them all for almost 40$ since they dont want to troubleshoot one by one. readers of this comment also should know that partial screen not responding in other iphone models might also have an issue in LCD connector on the motherboard, specially if you lock/unlock screen and screen works again for sometime. lcd connectors gets disconnected lightly from the motherboard due to multiple falls and hits after sometime. best of luck for all\n\n\n\n### Other Answer\nI am facing the same issue whereby these ghost touches type and open apps , I am using an original Iphone cable , how to I fix this issue.\n\n\n\n### Other Answer\nThere were two issues with the phone I had troubles with. It was my dads and turns out he carried it in his pocket. The phone itself had a little bend in it as a result. A little pressure in the opposite direction helped the issue. But it also had a tiny crack in the screen which wasnt obvious, once we added a screen protector this fixed the issues entirely.\n\n\n\n### Other Answer\nI had the same problem with my 64Gb iPhone 6+. Tried a lot of things and eventually downloaded all my images and videos to my PC and restarted the phone - problem solved. Been working now for two days.', lookup_str='', metadata={'source': 'https://www.ifixit.com/Answers/View/318583/My+iPhone+6+is+typing+and+opening+apps+by+itself', 'title': 'My iPhone 6 is typing and opening apps by itself'}, lookup_index=0)]  loader = IFixitLoader("https://www.ifixit.com/Device/Standard_iPad") data = loader.load()  data      [Document(page_content="Standard iPad\nThe standard edition of the tablet computer made by Apple.\n== Background Information ==\n\nOriginally introduced in January 2010, the iPad is Apple's standard edition of their tablet computer. In total, there have been ten generations of the standard edition of the iPad.\n\n== Additional Information ==\n\n* [link|https://www.apple.com/ipad-select/|Official Apple Product Page]\n* [link|https://en.wikipedia.org/wiki/IPad#iPad|Official iPad Wikipedia]", lookup_str='', metadata={'source': 'https://www.ifixit.com/Device/Standard_iPad', 'title': 'Standard iPad'}, lookup_index=0)]  ##Searching iFixit using /suggest[](#searching-ifixit-using-suggest) If you're looking for a more general way to search iFixit based on a keyword or phrase, the /suggest endpoint will return content related to the search term, then the loader will load the content from each of the suggested items and prep and return the documents.  data = IFixitLoader.load_suggestions("Banana")  data      [Document(page_content='Banana\nTasty fruit. Good source of potassium. Yellow.\n== Background Information ==\n\nCommonly misspelled, this wildly popular, phone shaped fruit serves as nutrition and an obstacle to slow down vehicles racing close behind you. Also used commonly as a synonym for “crazy” or “insane”.\n\nBotanically, the banana is considered a berry, although it isn’t included in the culinary berry category containing strawberries and raspberries. Belonging to the genus Musa, the banana originated in Southeast Asia and Australia. Now largely cultivated throughout South and Central America, bananas are largely available throughout the world. They are especially valued as a staple food group in developing countries due to the banana tree’s ability to produce fruit year round.\n\nThe banana can be easily opened. Simply remove the outer yellow shell by cracking the top of the stem. Then, with the broken piece, peel downward on each side until the fruity components on the inside are exposed. Once the shell has been removed it cannot be put back together.\n\n== Technical Specifications ==\n\n* Dimensions: Variable depending on genetics of the parent tree\n* Color: Variable depending on ripeness, region, and season\n\n== Additional Information ==\n\n[link|https://en.wikipedia.org/wiki/Banana|Wiki: Banana]', lookup_str='', metadata={'source': 'https://www.ifixit.com/Device/Banana', 'title': 'Banana'}, lookup_index=0),      Document(page_content="# Banana Teardown\nIn this teardown, we open a banana to see what's inside.  Yellow and delicious, but most importantly, yellow.\n\n\n###Tools Required:\n\n - Fingers\n\n - Teeth\n\n - Thumbs\n\n\n###Parts Required:\n\n - None\n\n\n## Step 1\nTake one banana from the bunch.\nDon't squeeze too hard!\n\n\n## Step 2\nHold the banana in your left hand and grip the stem between your right thumb and forefinger.\n\n\n## Step 3\nPull the stem downward until the peel splits.\n\n\n## Step 4\nInsert your thumbs into the split of the peel and pull the two sides apart.\nExpose the top of the banana.  It may be slightly squished from pulling on the stem, but this will not affect the flavor.\n\n\n## Step 5\nPull open the peel, starting from your original split, and opening it along the length of the banana.\n\n\n## S


iFixit | 🦜🔗 Langchain

tep 6\nRemove fruit from peel.\n\n\n## Step 7\nEat and enjoy!\nThis is where you'll need your teeth.\nDo not choke on banana!\n", lookup_str='', metadata={'source': 'https://www.ifixit.com/Teardown/Banana+Teardown/811', 'title': 'Banana Teardown'}, lookup_index=0)]


Images | 🦜🔗 Langchain

Main:  On this page #Images This covers how to load images such as JPG or PNG into a document format that we can use downstream.  ##Using Unstructured[](#using-unstructured) #!pip install pdfminer  from langchain.document_loaders.image import UnstructuredImageLoader  loader = UnstructuredImageLoader("layout-parser-paper-fast.jpg")  data = loader.load()  data[0]      Document(page_content="LayoutParser: A Unified Toolkit for Deep\nLearning Based Document Image Analysis\n\n\n‘Zxjiang Shen' (F3}, Ruochen Zhang”, Melissa Dell*, Benjamin Charles Germain\nLeet, Jacob Carlson, and Weining LiF\n\n\nsugehen\n\nshangthrows, et\n\n“Abstract. Recent advanocs in document image analysis (DIA) have been\n‘pimarliy driven bythe application of neural networks dell roar\n{uteomer could be aly deployed in production and extended fo farther\n[nvetigtion. However, various factory ke lcely organize codebanee\nsnd sophisticated modal cnigurations compat the ey ree of\n‘erin! innovation by wide sence, Though there have been sng\n‘Hors to improve reuablty and simplify deep lees (DL) mode\n‘aon, sone of them ae optimized for challenge inthe demain of DIA,\nThis roprscte a major gap in the extng fol, sw DIA i eal to\nscademic research acon wie range of dpi in the social ssencee\n[rary for streamlining the sage of DL in DIA research and appicn\n‘tons The core LayoutFaraer brary comes with a sch of simple and\nIntative interfaee or applying and eutomiing DI. odel fr Inyo de\npltfom for sharing both protrined modes an fal document dist\n{ation pipeline We demonutate that LayootPareer shea fr both\nlightweight and lrgeseledgtieation pipelines in eal-word uae ces\nThe leary pblely smal at Btspe://layost-pareergsthab So\n\n\n\n‘Keywords: Document Image Analysis» Deep Learning Layout Analysis\n‘Character Renguition - Open Serres dary « Tol\n\n\nIntroduction\n\n\n‘Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\ndoctiment image analysis (DIA) tea including document image clasiffeation [I]\n", lookup_str='', metadata={'source': 'layout-parser-paper-fast.jpg'}, lookup_index=0)  ###Retain Elements[](#retain-elements) Under the hood, Unstructured creates different "elements" for different chunks of text. By default we combine those together, but you can easily keep that separation by specifying mode="elements".  loader = UnstructuredImageLoader("layout-parser-paper-fast.jpg", mode="elements")  data = loader.load()  data[0]      Document(page_content='LayoutParser: A Unified Toolkit for Deep\nLearning Based Document Image Analysis\n', lookup_str='', metadata={'source': 'layout-parser-paper-fast.jpg', 'filename': 'layout-parser-paper-fast.jpg', 'page_number': 1, 'category': 'Title'}, lookup_index=0)


Image captions | 🦜🔗 Langchain

Main:  On this page #Image captions By default, the loader utilizes the pre-trained [Salesforce BLIP image captioning model](https://huggingface.co/Salesforce/blip-image-captioning-base).  This notebook shows how to use the ImageCaptionLoader to generate a query-able index of image captions  #!pip install transformers  from langchain.document_loaders import ImageCaptionLoader  ###Prepare a list of image urls from Wikimedia[](#prepare-a-list-of-image-urls-from-wikimedia) list_image_urls = [     "https://upload.wikimedia.org/wikipedia/commons/thumb/5/5a/Hyla_japonica_sep01.jpg/260px-Hyla_japonica_sep01.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/7/71/Tibur%C3%B3n_azul_%28Prionace_glauca%29%2C_canal_Fayal-Pico%2C_islas_Azores%2C_Portugal%2C_2020-07-27%2C_DD_14.jpg/270px-Tibur%C3%B3n_azul_%28Prionace_glauca%29%2C_canal_Fayal-Pico%2C_islas_Azores%2C_Portugal%2C_2020-07-27%2C_DD_14.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Thure_de_Thulstrup_-_Battle_of_Shiloh.jpg/251px-Thure_de_Thulstrup_-_Battle_of_Shiloh.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Passion_fruits_-_whole_and_halved.jpg/270px-Passion_fruits_-_whole_and_halved.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/5/5e/Messier83_-_Heic1403a.jpg/277px-Messier83_-_Heic1403a.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/b/b6/2022-01-22_Men%27s_World_Cup_at_2021-22_St._Moritz%E2%80%93Celerina_Luge_World_Cup_and_European_Championships_by_Sandro_Halank%E2%80%93257.jpg/288px-2022-01-22_Men%27s_World_Cup_at_2021-22_St._Moritz%E2%80%93Celerina_Luge_World_Cup_and_European_Championships_by_Sandro_Halank%E2%80%93257.jpg",     "https://upload.wikimedia.org/wikipedia/commons/thumb/9/99/Wiesen_Pippau_%28Crepis_biennis%29-20220624-RM-123950.jpg/224px-Wiesen_Pippau_%28Crepis_biennis%29-20220624-RM-123950.jpg", ]  ###Create the loader[](#create-the-loader) loader = ImageCaptionLoader(path_images=list_image_urls) list_docs = loader.load() list_docs  import requests from PIL import Image  Image.open(requests.get(list_image_urls[0], stream=True).raw).convert("RGB")  ###Create the index[](#create-the-index) from langchain.indexes import VectorstoreIndexCreator  index = VectorstoreIndexCreator().from_loaders([loader])  ###Query[](#query) query = "What's the painting about?" index.query(query)  query = "What kind of images are there?" index.query(query)


IMSDb | 🦜🔗 Langchain

Main:  #IMSDb [IMSDb](https://imsdb.com/) is the Internet Movie Script Database.  This covers how to load IMSDb webpages into a document format that we can use downstream.  from langchain.document_loaders import IMSDbLoader  loader = IMSDbLoader("https://imsdb.com/scripts/BlacKkKlansman.html")  data = loader.load()  data[0].page_content[:500]      '\n\r\n\r\n\r\n\r\n                                    BLACKKKLANSMAN\r\n                         \r\n                         \r\n                         \r\n                         \r\n                                      Written by\r\n\r\n                          Charlie Wachtel & David Rabinowitz\r\n\r\n                                         and\r\n\r\n                              Kevin Willmott & Spike Lee\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n                         FADE IN:\r\n                         \r\n          SCENE FROM "GONE WITH'  data[0].metadata      {'source': 'https://imsdb.com/scripts/BlacKkKlansman.html'}


Iugu | 🦜🔗 Langchain

Main:  #Iugu [Iugu](https://www.iugu.com/) is a Brazilian services and software as a service (SaaS) company. It offers payment-processing software and application programming interfaces for e-commerce websites and mobile applications.  This notebook covers how to load data from the Iugu REST API into a format that can be ingested into LangChain, along with example usage for vectorization.  from langchain.document_loaders import IuguLoader from langchain.indexes import VectorstoreIndexCreator  The Iugu API requires an access token, which can be found inside of the Iugu dashboard.  This document loader also requires a resource option which defines what data you want to load.  Following resources are available:  Documentation [Documentation](https://dev.iugu.com/reference/metadados)  iugu_loader = IuguLoader("charges")  # Create a vectorstore retriever from the loader # see https://python.langchain.com/en/latest/modules/data_connection/getting_started.html for more details  index = VectorstoreIndexCreator().from_loaders([iugu_loader]) iugu_doc_retriever = index.vectorstore.as_retriever()


Joplin | 🦜🔗 Langchain

Main:  #Joplin [Joplin](https://joplinapp.org/) is an open-source note-taking app. Capture your thoughts and securely access them from any device.  This notebook covers how to load documents from a Joplin database.  Joplin has a [REST API](https://joplinapp.org/api/references/rest_api/) for accessing its local database. This loader uses the API to retrieve all notes in the database and their metadata. This requires an access token that can be obtained from the app by following these steps:  - Open the Joplin app. The app must stay open while the documents are being loaded. - Go to settings / options and select "Web Clipper". - Make sure that the Web Clipper service is enabled. - Under "Advanced Options", copy the authorization token. You may either initialize the loader directly with the access token, or store it in the environment variable JOPLIN_ACCESS_TOKEN.  An alternative to this approach is to export the Joplin's note database to Markdown files (optionally, with Front Matter metadata) and use a Markdown loader, such as ObsidianLoader, to load them.  from langchain.document_loaders import JoplinLoader  loader = JoplinLoader(access_token="<access-token>")  docs = loader.load()


Jupyter Notebook | 🦜🔗 Langchain

Main:  #Jupyter Notebook [Jupyter Notebook](https://en.wikipedia.org/wiki/Project_Jupyter#Applications) (formerly IPython Notebook) is a web-based interactive computational environment for creating notebook documents.  This notebook covers how to load data from a Jupyter notebook (.html) into a format suitable by LangChain.  from langchain.document_loaders import NotebookLoader  loader = NotebookLoader(     "example_data/notebook.html",     include_outputs=True,     max_output_length=20,     remove_newline=True, )  NotebookLoader.load() loads the .html notebook file into a Document object.  Parameters:  - include_outputs (bool): whether to include cell outputs in the resulting document (default is False). - max_output_length (int): the maximum number of characters to include from each cell output (default is 10). - remove_newline (bool): whether to remove newline characters from the cell sources and outputs (default is False). - traceback (bool): whether to include full traceback (default is False). loader.load()      [Document(page_content='\'markdown\' cell: \'[\'# Notebook\', \'\', \'This notebook covers how to load data from an .html notebook into a format suitable by LangChain.\']\'\n\n \'code\' cell: \'[\'from langchain.document_loaders import NotebookLoader\']\'\n\n \'code\' cell: \'[\'loader = NotebookLoader("example_data/notebook.html")\']\'\n\n \'markdown\' cell: \'[\'`NotebookLoader.load()` loads the `.html` notebook file into a `Document` object.\', \'\', \'**Parameters**:\', \'\', \'* `include_outputs` (bool): whether to include cell outputs in the resulting document (default is False).\', \'* `max_output_length` (int): the maximum number of characters to include from each cell output (default is 10).\', \'* `remove_newline` (bool): whether to remove newline characters from the cell sources and outputs (default is False).\', \'* `traceback` (bool): whether to include full traceback (default is False).\']\'\n\n \'code\' cell: \'[\'loader.load(include_outputs=True, max_output_length=20, remove_newline=True)\']\'\n\n', metadata={'source': 'example_data/notebook.html'})]


lakeFS | 🦜🔗 Langchain

Main:  On this page #lakeFS [lakeFS](https://docs.lakefs.io/) provides scalable version control over the data lake, and uses Git-like semantics to create and access those versions.  This notebooks covers how to load document objects from a lakeFS path (whether it's an object or a prefix).  ##Initializing the lakeFS loader[](#initializing-the-lakefs-loader) Replace ENDPOINT, LAKEFS_ACCESS_KEY, and LAKEFS_SECRET_KEY values with your own.  from langchain.document_loaders import LakeFSLoader  ENDPOINT = "" LAKEFS_ACCESS_KEY = "" LAKEFS_SECRET_KEY = ""  lakefs_loader = LakeFSLoader(     lakefs_access_key=LAKEFS_ACCESS_KEY,     lakefs_secret_key=LAKEFS_SECRET_KEY,     lakefs_endpoint=ENDPOINT, )  ##Specifying a path[](#specifying-a-path) You can specify a prefix or a complete object path to control which files to load.  Specify the repository, reference (branch, commit id, or tag), and path in the corresponding REPO, REF, and PATH to load the documents from:  REPO = "" REF = "" PATH = ""  lakefs_loader.set_repo(REPO) lakefs_loader.set_ref(REF) lakefs_loader.set_path(PATH)  docs = lakefs_loader.load() docs


LarkSuite (FeiShu) | 🦜🔗 Langchain

Main:  #LarkSuite (FeiShu) [LarkSuite](https://www.larksuite.com/) is an enterprise collaboration platform developed by ByteDance.  This notebook covers how to load data from the LarkSuite REST API into a format that can be ingested into LangChain, along with example usage for text summarization.  The LarkSuite API requires an access token (tenant_access_token or user_access_token), checkout [LarkSuite open platform document](https://open.larksuite.com/document) for API details.  from getpass import getpass  from langchain.document_loaders.larksuite import LarkSuiteDocLoader  DOMAIN = input("larksuite domain") ACCESS_TOKEN = getpass("larksuite tenant_access_token or user_access_token") DOCUMENT_ID = input("larksuite document id")  from pprint import pprint  larksuite_loader = LarkSuiteDocLoader(DOMAIN, ACCESS_TOKEN, DOCUMENT_ID) docs = larksuite_loader.load()  pprint(docs)      [Document(page_content='Test Doc\nThis is a Test Doc\n\n1\n2\n3\n\n', metadata={'document_id': 'V76kdbd2HoBbYJxdiNNccajunPf', 'revision_id': 11, 'title': 'Test Doc'})]  # see https://python.langchain.com/docs/use_cases/summarization for more details from langchain.chains.summarize import load_summarize_chain  chain = load_summarize_chain(llm, chain_type="map_reduce") chain.run(docs)


Mastodon | 🦜🔗 Langchain

Main:  #Mastodon [Mastodon](https://joinmastodon.org/) is a federated social media and social networking service.  This loader fetches the text from the "toots" of a list of Mastodon accounts, using the Mastodon.py Python package.  Public accounts can the queried by default without any authentication. If non-public accounts or instances are queried, you have to register an application for your account which gets you an access token, and set that token and your account's API base URL.  Then you need to pass in the Mastodon account names you want to extract, in the @account@instance format.  from langchain.document_loaders import MastodonTootsLoader  #!pip install Mastodon.py  loader = MastodonTootsLoader(     mastodon_accounts=["@Gargron@mastodon.social"],     number_toots=50,  # Default value is 100 )  # Or set up access information to use a Mastodon app. # Note that the access token can either be passed into # constructor or you can set the environment "MASTODON_ACCESS_TOKEN". # loader = MastodonTootsLoader( #     access_token="<ACCESS TOKEN OF MASTODON APP>", #     api_base_url="<API BASE URL OF MASTODON APP INSTANCE>", #     mastodon_accounts=["@Gargron@mastodon.social"], #     number_toots=50,  # Default value is 100 # )  documents = loader.load() for doc in documents[:3]:     print(doc.page_content)     print("=" * 80)
It is tough to leave this behind and go back to reality. And some people live here! I’m sure there are downsides but it sounds pretty good to me right now.
     ================================================================================
I wish we could stay here a little longer, but it is time to go home 🥲
     ================================================================================
Last day of the honeymoon. And it’s #caturday! This cute tabby came to the restaurant to beg for food and got some chicken.
     ================================================================================  The toot texts (the documents' page_content) is by default HTML as returned by the Mastodon API.


MediaWiki Dump | 🦜🔗 Langchain

Main:  #MediaWiki Dump [MediaWiki XML Dumps](https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps) contain the content of a wiki (wiki pages with all their revisions), without the site-related data. A XML dump does not create a full backup of the wiki database, the dump does not contain user accounts, images, edit logs, etc.  This covers how to load a MediaWiki XML dump file into a document format that we can use downstream.  It uses mwxml from mediawiki-utilities to dump and mwparserfromhell from earwig to parse MediaWiki wikicode.  Dump files can be obtained with dumpBackup.php or on the Special:Statistics page of the Wiki.  # mediawiki-utilities supports XML schema 0.11 in unmerged branches pip install -qU git+https://github.com/mediawiki-utilities/python-mwtypes@updates_schema_0.11 # mediawiki-utilities mwxml has a bug, fix PR pending pip install -qU git+https://github.com/gdedrouas/python-mwxml@xml_format_0.11 pip install -qU mwparserfromhell  from langchain.document_loaders import MWDumpLoader  loader = MWDumpLoader(     file_path="example_data/testmw_pages_current.xml",     encoding="utf8",     # namespaces = [0,2,3] Optional list to load only specific namespaces. Loads all namespaces by default.     skip_redirects=True,  # will skip over pages that just redirect to other pages (or not if False)     stop_on_error=False,  # will skip over pages that cause parsing errors (or not if False) ) documents = loader.load() print(f"You have {len(documents)} document(s) in your data ")      You have 177 document(s) in your data   documents[:5]      [Document(page_content='\t\n\t\n\tArtist\n\tReleased\n\tRecorded\n\tLength\n\tLabel\n\tProducer', metadata={'source': 'Album'}),      Document(page_content='{| class="article-table plainlinks" style="width:100%;"\n|- style="font-size:18px;"\n! style="padding:0px;" | Template documentation\n|-\n| Note: portions of the template sample may not be visible without values provided.\n|-\n| View or edit this documentation. (About template documentation)\n|-\n| Editors can experiment in this template\'s [ sandbox] and [ test case] pages.\n|}Category:Documentation templates', metadata={'source': 'Documentation'}),      Document(page_content='Description\nThis template is used to insert descriptions on template pages.\n\nSyntax\nAdd <noinclude></noinclude> at the end of the template page.\n\nAdd <noinclude></noinclude> to transclude an alternative page from the /doc subpage.\n\nUsage\n\nOn the Template page\nThis is the normal format when used:\n\nTEMPLATE CODE\n<includeonly>Any categories to be inserted into articles by the template</includeonly>\n<noinclude>{{Documentation}}</noinclude>\n\nIf your template is not a completed div or table, you may need to close the tags just before {{Documentation}} is inserted (within the noinclude tags).\n\nA line break right before {{Documentation}} can also be useful as it helps prevent the documentation template "running into" previous code.\n\nOn the documentation page\nThe documentation page is usually located on the /doc subpage for a template, but a different page can be specified with the first parameter of the template (see Syntax).\n\nNormally, you will want to write something like the following on the documentation page:\n\n==Description==\nThis template is used to do something.\n\n==Syntax==\nType {{t|templatename}} somewhere.\n\n==Samples==\n<nowiki>{{templatename|input}}</nowiki> \n\nresults in...\n\n{{templatename|input}}\n\n<includeonly>Any categories for the template itself</includeonly>\n<noinclude>[[Category:Template documentation]]</noinclude>\n\nUse any or all of the above description/syntax/sample output sections. You may also want to add "see also" or other sections.\n\nNote that the above example also uses the Template:T template.\n\nCategory:Documentation templatesCategory:Template documentation', metadata={'source': 'Documentation/doc'}),      Document(page_content='Description\nA template link with a variable number of parameters (0-20).\n\nSyntax\n \n\nSource\nImproved version not needing t/piece subtemplate developed on Templates wiki see the list of authors. Copied here via CC-By-SA 3.0 license.\n\nExample\n\nCategory:General wiki templates\nCategory:Template documentation', metadata={'source': 'T/doc'}),      Document(page_content='\t\n\t\t    \n\t\n\t\t    Aliases\n\t    Relatives\n\t    Affiliation\n        Occupation\n    \n            Biographical information\n        Marital status\n    \tDate of birth\n        Place of birth\n        Date of death\n        Place of death\n    \n            Physical description\n        Species\n        Gender\n        Height\n        Weight\n        Eye color\n\t\n           Appearances\n       Portrayed by\n       Appears in\n       Debut\n    ', metadata={'source': 'Character'})]


Merge Documents Loader | 🦜🔗 Langchain

Main:  #Merge Documents Loader Merge the documents returned from a set of specified data loaders.  from langchain.document_loaders import WebBaseLoader  loader_web = WebBaseLoader(     "https://github.com/basecamp/handbook/blob/master/37signals-is-you.md" )  from langchain.document_loaders import PyPDFLoader  loader_pdf = PyPDFLoader("../MachineLearning-Lecture01.pdf")  from langchain.document_loaders.merge import MergedDataLoader  loader_all = MergedDataLoader(loaders=[loader_web, loader_pdf])  docs_all = loader_all.load()  len(docs_all)      23


mhtml | 🦜🔗 Langchain

Main:  #mhtml MHTML is a is used both for emails but also for archived webpages. MHTML, sometimes referred as MHT, stands for MIME HTML is a single file in which entire webpage is archived. When one saves a webpage as MHTML format, this file extension will contain HTML code, images, audio files, flash animation etc.  from langchain.document_loaders import MHTMLLoader  # Create a new loader object for the MHTML file loader = MHTMLLoader(     file_path="../../../../../../tests/integration_tests/examples/example.mht" )  # Load the document from the file documents = loader.load()  # Print the documents to see the results for doc in documents:     print(doc)      page_content='LangChain\nLANG CHAIN 🦜🔗Official Home Page\xa0\n\n\n\n\n\n\n\nIntegrations\n\n\n\nFeatures\n\n\n\n\nBlog\n\n\n\nConceptual Guide\n\n\n\n\nPython Repo\n\n\nJavaScript Repo\n\n\n\nPython Documentation \n\n\nJavaScript Documentation\n\n\n\n\nPython ChatLangChain \n\n\nJavaScript ChatLangChain\n\n\n\n\nDiscord \n\n\nTwitter\n\n\n\n\nIf you have any comments about our WEB page, you can \nwrite us at the address shown above.  However, due to \nthe limited number of personnel in our corporate office, we are unable to \nprovide a direct response.\n\nCopyright © 2023-2023 LangChain Inc.\n\n\n' metadata={'source': '../../../../../../tests/integration_tests/examples/example.mht', 'title': 'LangChain'}