.. image:: https://readthedocs.org/projects/plumbum/badge/
   :target: https://plumbum.readthedocs.io/en/latest/
   :alt: Documentation Status
.. image:: https://github.com/tomerfiliba/plumbum/workflows/CI/badge.svg
   :target: https://github.com/tomerfiliba/plumbum/actions
   :alt: Build Status
.. image:: https://coveralls.io/repos/tomerfiliba/plumbum/badge.svg?branch=master&service=github
   :target: https://coveralls.io/github/tomerfiliba/plumbum?branch=master
   :alt: Coverage Status
.. image:: https://img.shields.io/pypi/v/plumbum.svg
   :target: https://pypi.python.org/pypi/plumbum/
   :alt: PyPI Status
.. image:: https://img.shields.io/pypi/pyversions/plumbum.svg
   :target: https://pypi.python.org/pypi/plumbum/
   :alt: PyPI Versions
.. image:: https://img.shields.io/conda/vn/conda-forge/plumbum.svg
   :target: https://github.com/conda-forge/plumbum-feedstock
   :alt: Conda-Forge Badge
.. image:: https://img.shields.io/pypi/l/plumbum.svg
   :target: https://pypi.python.org/pypi/plumbum/
   :alt: PyPI License
.. image:: https://badges.gitter.im/plumbumpy/Lobby.svg
   :alt: Join the chat at https://gitter.im/plumbumpy/Lobby
   :target: https://gitter.im/plumbumpy/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :alt: Code styled with Black
   :target: https://github.com/psf/black

Plumbum: Shell Combinators
==========================

Ever wished the compactness of shell scripts be put into a **real** programming language?
Say hello to *Plumbum Shell Combinators*. Plumbum (Latin for *lead*, which was used to create
pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python.
The motto of the library is **"Never write shell scripts again"**, and thus it attempts to mimic
the **shell syntax** ("shell combinators") where it makes sense, while keeping it all **Pythonic
and cross-platform**.

Apart from shell-like syntax and handy shortcuts, the library provides local and remote command
execution (over SSH), local and remote file-system paths, easy working-directory and environment
manipulation, and a programmatic Command-Line Interface (CLI) application toolkit.
Now let's see some code!

*This is only a teaser; the full documentation can be found at*
`Read the Docs <https://plumbum.readthedocs.io>`_

Cheat Sheet
-----------

Basics
******

.. code-block:: python

    >>> from plumbum import local
    >>> local.cmd.ls
    LocalCommand(/bin/ls)
    >>> local.cmd.ls()
    'build.py\nCHANGELOG.rst\nconda.recipe\nCONTRIBUTING.rst\ndocs\nexamples\nexperiments\nLICENSE\nMANIFEST.in\nPipfile\nplumbum\nplumbum.egg-info\npytest.ini\nREADME.rst\nsetup.cfg\nsetup.py\ntests\ntranslations.py\n'
    >>> notepad = local["c:\\windows\\notepad.exe"]
    >>> notepad()                                   # Notepad window pops up
    ''                                              # Notepad window is closed by user, command returns

In the example above, you can use ``local["ls"]`` if you have an unusually named executable or a full path to an executable. The ``local`` object represents your local machine. As you'll see, Plumbum also provides remote machines that use the same API!
You can also use ``from plumbum.cmd import ls`` as well for accessing programs in the ``PATH``.

Piping
******

.. code-block:: python

    >>> from plumbum.cmd import ls, grep, wc
    >>> chain = ls["-a"] | grep["-v", r"\.py"] | wc["-l"]
    >>> print(chain)
    /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l
    >>> chain()
    '27\n'

Redirection
***********

.. code-block:: python

    >>> from plumbum.cmd import cat, head
    >>> ((cat < "setup.py") | head["-n", 4])()
    '#!/usr/bin/env python3\nimport os\n\ntry:\n'
    >>> (ls["-a"] > "file.list")()
    ''
    >>> (cat["file.list"] | wc["-l"])()
    '31\n'

Working-directory manipulation
******************************

.. code-block:: python

    >>> local.cwd
    <LocalWorkdir /home/tomer/workspace/plumbum>
    >>> with local.cwd(local.cwd / "docs"):
    ...     chain()
    ...
    '22\n'

Foreground and background execution
***********************************

.. code-block:: python

    >>> from plumbum import FG, BG
    >>> (ls["-a"] | grep[r"\.py"]) & FG         # The output is printed to stdout directly
    build.py
    setup.py
    translations.py
    >>> (ls["-a"] | grep[r"\.py"]) & BG         # The process runs "in the background"
    <Future ['/bin/grep', '\\.py'] (running)>

Command nesting
***************

.. code-block:: python

    >>> from plumbum.cmd import sudo, ifconfig
    >>> print(sudo[ifconfig["-a"]])
    /usr/bin/sudo /sbin/ifconfig -a
    >>> (sudo[ifconfig["-a"]] | grep["-i", "loop"]) & FG
    lo        Link encap:Local Loopback
              UP LOOPBACK RUNNING  MTU:16436  Metric:1

Remote commands (over SSH)
**************************

Supports `openSSH <http://www.openssh.org/>`_-compatible clients,
`PuTTY <http://www.chiark.greenend.org.uk/~sgtatham/putty/>`_ (on Windows)
and `Paramiko <https://github.com/paramiko/paramiko/>`_ (a pure-Python implementation of SSH2)

.. code-block:: python

    >>> from plumbum import SshMachine
    >>> remote = SshMachine("somehost", user = "john", keyfile = "/path/to/idrsa")
    >>> r_ls = remote["ls"]
    >>> with remote.cwd("/lib"):
    ...     (r_ls | grep["0.so.0"])()
    ...
    'libusb-1.0.so.0\nlibusb-1.0.so.0.0.0\n'

CLI applications
****************

.. code-block:: python

    import logging
    from plumbum import cli

    class MyCompiler(cli.Application):
        verbose = cli.Flag(["-v", "--verbose"], help = "Enable verbose mode")
        include_dirs = cli.SwitchAttr("-I", list = True, help = "Specify include directories")

        @cli.switch("--loglevel", int)
        def set_log_level(self, level):
            """Sets the log-level of the logger"""
            logging.root.setLevel(level)

        def main(self, *srcfiles):
            print("Verbose:", self.verbose)
            print("Include dirs:", self.include_dirs)
            print("Compiling:", srcfiles)

    if __name__ == "__main__":
        MyCompiler.run()

Sample output
+++++++++++++

::

    $ python3 simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp
    Verbose: True
    Include dirs: ['foo/bar', 'spam/eggs']
    Compiling: ('x.cpp', 'y.cpp', 'z.cpp')

Colors and Styles
-----------------

.. code-block:: python

    from plumbum import colors
    with colors.red:
        print("This library provides safe, flexible color access.")
        print(colors.bold | "(and styles in general)", "are easy!")
    print("The simple 16 colors or",
          colors.orchid & colors.underline | '256 named colors,',
          colors.rgb(18, 146, 64) | "or full rgb colors",
          'can be used.')
    print("Unsafe " + colors.bg.dark_khaki + "color access" + colors.bg.reset + " is available too.")


====================
django-import-export
====================

.. |build| image:: https://github.com/django-import-export/django-import-export/actions/workflows/release.yml/badge.svg
    :target: https://github.com/django-import-export/django-import-export/actions/workflows/release.yml
    :alt: Build status on Github

.. |coveralls| image:: https://coveralls.io/repos/github/django-import-export/django-import-export/badge.svg?branch=main
    :target: https://coveralls.io/github/django-import-export/django-import-export?branch=main

.. |pypi| image:: https://img.shields.io/pypi/v/django-import-export.svg
    :target: https://pypi.org/project/django-import-export/
    :alt: Current version on PyPi

.. |docs| image:: http://readthedocs.org/projects/django-import-export/badge/?version=stable
    :target: https://django-import-export.readthedocs.io/en/stable/
    :alt: Documentation

.. |pyver| image:: https://img.shields.io/pypi/pyversions/django-import-export
    :alt: PyPI - Python Version

.. |djangover| image:: https://img.shields.io/pypi/djversions/django-import-export
    :alt: PyPI - Django Version

.. |downloads| image:: https://static.pepy.tech/personalized-badge/django-import-export?period=month&units=international_system&left_color=black&right_color=blue&left_text=Downloads/month
    :target: https://pepy.tech/project/django-import-export

.. |xfollow| image:: https://img.shields.io/twitter/url/https/twitter.com/django_import.svg?style=social&label=Follow%20%40django_import
   :alt: Follow us on X
   :target: https://twitter.com/django_import

.. |discord|  image:: https://img.shields.io/discord/1240294048653119508?style=flat
   :alt: Discord

|build| |coveralls| |pypi| |docs| |pyver| |djangover| |downloads| |xfollow| |discord|

Introduction
============

Straightforward, reliable and comprehensive file import / export for your Django application.

*django-import-export* is an application and library which lets you manage import / export from / to a variety of sources (csv, xlsx, json etc).

Can be run programmatically, or with optional integration with the Django Admin site:

..
  source of this video uploaded to this issue comment:
  https://github.com/django-import-export/django-import-export/pull/1833#issuecomment-2118777440

https://github.com/django-import-export/django-import-export/assets/6249838/ab56d8ba-c307-4bdf-8fa9-225669c72b37

`Screenshots <https://django-import-export.readthedocs.io/en/latest/screenshots.html>`_

Features
========

* Import / export via `Admin UI Integration <https://django-import-export.readthedocs.io/en/latest/admin_integration.html>`_ or `programmatically <https://django-import-export.readthedocs.io/en/latest/getting_started.html#importing-data>`_
* Import to and from a variety of file formats (csv, json, xlsx, pandas, HTML, YAML... and anything else that `tablib <https://github.com/jazzband/tablib>`_ supports)
* `Preview <https://django-import-export.readthedocs.io/en/latest/screenshots.html/>`_ data before importing in Admin UI
* Support for `bulk import <https://django-import-export.readthedocs.io/en/latest/bulk_import.html>`_
* Handles `CRUD (and 'skip') operations during import <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#create-or-update-model-instances>`_
* Flexible handling of `foreign key <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#importing-model-relations>`_ relationships
* `Many-to-many relationship <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#many-to-many-relations>`_ support
* `Validation <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#validation-during-import>`_ of imported data
* Define custom `transformations <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#advanced-data-manipulation-on-export>`_ for exported data
* Import / export the same model instance as `different views <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#customize-resource-options>`_
* Export using `natural keys <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#django-natural-keys>`__ for portability between environments
* `Select items for export <https://django-import-export.readthedocs.io/en/latest/screenshots.html/>`_ via the Admin UI object list
* `Select fields for export <https://django-import-export.readthedocs.io/en/latest/screenshots.html/>`_ via the export form
* `Export single object instances <https://django-import-export.readthedocs.io/en/latest/admin_integration.html#export-from-model-instance-change-form>`_
* Use `django permissions <https://django-import-export.readthedocs.io/en/latest/installation.html#import-export-import-permission-code>`_ to control import / export authorization
* Internationalization support
* Based on `tablib <https://github.com/jazzband/tablib>`__
* Support for MySQL / PostgreSQL / SQLite
* Extensible - `add custom logic to control import / export <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html>`_
* Handle import from various character encodings
* `Celery <https://django-import-export.readthedocs.io/en/latest/celery.html>`_ integration
* Test locally with `Docker <https://django-import-export.readthedocs.io/en/latest/testing.html>`_
* Comprehensive `documentation <https://django-import-export.readthedocs.io/en/latest/index.html>`__
* `Extensible API <https://django-import-export.readthedocs.io/en/latest/api_admin.html>`_
* test coverage :100:
* Supports dark mode :rocket:

Example use-cases
=================

*django-import-export* is designed to be extensible and can be used to support a variety of operations.
Here are some examples of how it has been used in the wild:

* Configure external cron jobs to run an import or export at set times
* Use `permissions <https://django-import-export.readthedocs.io/en/latest/installation.html#import-export-import-permission-code>`_ to define a subset of users able to import and export project data
* Safely update project reference data by importing from version controlled csv
* Create portable data to transfer between environments using `natural keys <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#django-natural-keys>`_
* Manage user access to an application by importing externally version controlled auth user lists
* Add `hooks <https://django-import-export.readthedocs.io/en/latest/advanced_usage.html#advanced-data-manipulation-on-export>`_ to anonymize data on export
* `Modify import / export UI forms <https://django-import-export.readthedocs.io/en/latest/admin_integration.html#customize-admin-import-forms>`_ to add dynamic filtering on import / export.

Getting started
===============

* `Installation <https://django-import-export.readthedocs.io/en/latest/installation.html>`_
* `Getting started <https://django-import-export.readthedocs.io/en/latest/getting_started.html>`__
* `Example application <https://django-import-export.readthedocs.io/en/latest/installation.html#exampleapp>`_

Help and support
================

* `Documentation <https://django-import-export.readthedocs.io/en/latest/>`_
* `FAQ <https://django-import-export.readthedocs.io/en/latest/faq.html>`_
* `Getting help <https://django-import-export.readthedocs.io/en/latest/faq.html#what-s-the-best-way-to-communicate-a-problem-question-or-suggestion>`_
* `Contributing <https://django-import-export.readthedocs.io/en/latest/faq.html#how-can-i-help>`_
* Become a `sponsor <https://github.com/sponsors/django-import-export>`_
* `Raise a security issue <https://github.com/django-import-export/django-import-export/blob/main/SECURITY.md>`_
* Join our `discord <https://discord.gg/aCcec52kY4>`_

Commercial support
==================

Commercial support is provided by `Bellaport Systems Ltd <https://www.bellaport.co.uk>`_

Releases
========

* `Release notes <https://django-import-export.readthedocs.io/en/latest/release_notes.html>`_
* `Changelog <https://django-import-export.readthedocs.io/en/latest/changelog.html>`_


[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) [![Downloads](https://pepy.tech/badge/simpletransformers)](https://pepy.tech/project/simpletransformers)

[![All Contributors](https://img.shields.io/badge/all_contributors-93-orange.svg?style=flat-square)](#contributors-)

# Simple Transformers

This library is based on the [Transformers](https://github.com/huggingface/transformers) library by HuggingFace. `Simple Transformers` lets you quickly train and evaluate Transformer models. Only 3 lines of code are needed to **initialize**, **train**, and **evaluate** a model.

**Supported Tasks:**

- Information Retrieval (Dense Retrieval)
- (Large) Language Models (Training, Fine-tuning, and Generation)
- Encoder Model Training and Fine-tuning
- Sequence Classification
- Token Classification (NER)
- Question Answering
- Language Generation
- T5 Model
- Seq2Seq Tasks
- Multi-Modal Classification
- Conversational AI.

# Table of contents

- [Simple Transformers](#simple-transformers)
- [Table of contents](#table-of-contents)
  - [Setup](#setup)
    - [With Conda](#with-conda)
      - [Optional](#optional)
  - [Usage](#usage)
    - [A quick example](#a-quick-example)
    - [Experiment Tracking with Weights and Biases](#experiment-tracking-with-weights-and-biases)
  - [Current Pretrained Models](#current-pretrained-models)
  - [Contributors ✨](#contributors-)
  - [How to Contribute](#how-to-contribute)
    - [How to Update Docs](#how-to-update-docs)
  - [Acknowledgements](#acknowledgements)

## Setup

### With Conda

1. Install `Anaconda` or `Miniconda` Package Manager from [here](https://www.anaconda.com/distribution/)
2. Create a new virtual environment and install packages.

```bash
$ conda create -n st python pandas tqdm
$ conda activate st
```

Using Cuda:

```bash
$ conda install pytorch>=1.6 cudatoolkit=11.0 -c pytorch
```

Without using Cuda

```bash
$ conda install pytorch cpuonly -c pytorch
```

3. Install `simpletransformers`.

```bash
$ pip install simpletransformers
```

#### Optional

1. Install `Weights` and `Biases` (wandb) for tracking and visualizing training in a web browser.

```bash
$ pip install wandb
```

## Usage

**All documentation is now live at [simpletransformers.ai](https://simpletransformers.ai/)**

`Simple Transformer` models are built with a particular Natural Language Processing (NLP) task in mind. Each such model comes equipped with features and functionality designed to best fit the task that they are intended to perform. The high-level process of using Simple Transformers models follows the same pattern.

1. Initialize a task-specific model
2. Train the model with `train_model()`
3. Evaluate the model with `eval_model()`
4. Make predictions on (unlabelled) data with `predict()`

However, there are necessary differences between the different models to ensure that they are well suited for their intended task. The key differences will typically be the differences in input/output data formats and any task specific features/configuration options. These can all be found in the documentation section for each task.

The currently implemented task-specific `Simple Transformer` models, along with their task, are given below.

| Task                                                      | Model                           |
| --------------------------------------------------------- | ------------------------------- |
| Binary and multi-class text classification                | `ClassificationModel`           |
| Conversational AI (chatbot training)                      | `ConvAIModel`                   |
| Language generation                                       | `LanguageGenerationModel`       |
| Language model training/fine-tuning                       | `LanguageModelingModel`         |
| Multi-label text classification                           | `MultiLabelClassificationModel` |
| Multi-modal classification (text and image data combined) | `MultiModalClassificationModel` |
| Named entity recognition                                  | `NERModel`                      |
| Question answering                                        | `QuestionAnsweringModel`        |
| Regression                                                | `ClassificationModel`           |
| Sentence-pair classification                              | `ClassificationModel`           |
| Text Representation Generation                            | `RepresentationModel`           |
| Document Retrieval                                        | `RetrievalModel`                |

- **Please refer to the relevant section in the [docs](https://simpletransformers.ai/) for more information on how to use these models.**
- Example scripts can be found in the [examples](https://github.com/ThilinaRajapakse/simpletransformers/tree/master/examples) directory.
- See the [Changelog](https://github.com/ThilinaRajapakse/simpletransformers/blob/master/CHANGELOG.md) for up-to-date changes to the project.

### A quick example

```python
from simpletransformers.classification import ClassificationModel, ClassificationArgs
import pandas as pd
import logging

logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)

# Preparing train data
train_data = [
    ["Aragorn was the heir of Isildur", 1],
    ["Frodo was the heir of Isildur", 0],
]
train_df = pd.DataFrame(train_data)
train_df.columns = ["text", "labels"]

# Preparing eval data
eval_data = [
    ["Theoden was the king of Rohan", 1],
    ["Merry was the king of Rohan", 0],
]
eval_df = pd.DataFrame(eval_data)
eval_df.columns = ["text", "labels"]

# Optional model configuration
model_args = ClassificationArgs(num_train_epochs=1)

# Create a ClassificationModel
model = ClassificationModel(
    "roberta", "roberta-base", args=model_args
)

# Train the model
model.train_model(train_df)

# Evaluate the model
result, model_outputs, wrong_predictions = model.eval_model(eval_df)

# Make predictions with the model
predictions, raw_outputs = model.predict(["Sam was a Wizard"])

```

### Experiment Tracking with Weights and Biases

- Weights and Biases makes it incredibly easy to keep track of all your experiments. Check it out on Colab here: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/simpletformers-QA-colab)

---

## Current Pretrained Models

For a list of pretrained models, see [Hugging Face docs](https://huggingface.co/pytorch-transformers/pretrained_models.html).

The `model_types` available for each task can be found under their respective section. Any pretrained model of that type
found in the Hugging Face docs should work. To use any of them set the correct `model_type` and `model_name` in the `args`
dictionary.

---

## Contributors ✨

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

hawktang
💻

Mabu Manaileng
💻

Ali Hamdi Ali Fadel
💻

Tovly Deutsch
💻

hlo-world
💻

huntertl
💻

Yann Defretin
💻 📖 💬 🤔

Manuel
📖 💻

Gilles Jacobs
📖

shasha79
💻

Mercedes Garcia
💻

Hammad Hassan Tarar
💻 📖

Todd Cook
💻

Knut O. Hellan
💻 📖

nagenshukla
💻

flaviussn
💻 📖

Marc Torrellas
🚧

Adrien Renaud
💻

jacky18008
💻

Matteo Senese
💻

sarthakTUM
📖 💻

djstrong
💻

Hyeongchan Kim
📖

Pradhy729
💻 🚧

Iknoor Singh
📖

Gabriel Altay
💻

flozi00
📖 💻 🚧

alexysdussier
💻

Jean-Louis Queguiner
📖

aced125
💻

Laksh1997
💻

Changlin_NLP
💻

jpotoniec
💻

fcggamou
💻 📖

guy-mor
🐛 💻

Cahya Wirawan
💻

BjarkePedersen
💻

tekkkon
💻

Amit Garg
💻

caprone
🐛

Ather Fawaz
💻

Santiago Castro
📖

taranais
💻

Pablo N. Marino
💻 📖

Anton Kiselev
💻 📖

Alex
💻

Karthik Ganesan
💻

Zhylko Dima
💻

Jonatan Kłosko
💻

sarapapi
💻 💬

Abdul
💻

James Milliman
📖

Suraj Parmar
📖

KwanHong Lee
💬

Erik Fäßler
💻

Thomas Søvik
💬

Gagandeep Singh
💻 📖

Andrea Esuli
💻

DM2493
💻

Nick Doiron
💻

Abhinav Gupta
💻

Martin H. Normark
📖

Mossad Helali
💻

calebchiam
💻

Daniele Sartiano
💻

tuner007
📖

xia jiang
💻

Hendrik Buschmeier
📖

Mana Borwornpadungkitti
📖

rayline
💻

Mehdi Heidari
💻

William Roe
💻

Álvaro Abella Bascarán
💻

Brett Fazio
📖

Viet-Tien
💻

Bisola Olasehinde
💻 📖

William Chen
📖

Reza Ebrahimi
📖

gabriben
📖

Prashanth Kurella
💻

dopc
💻

Tanish Tyagi
📖 💻

kongyurui
💻

Andrew Lensen
💻

jinschoi
💻

Le Nguyen Khang
💻

Jordi Mas
📖

mxa
💻

MichelBartels
💻

Luke Tudge
📖

Saint
💻

deltaxrg
💻 📖

Fortune Adekogbe
💻

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!

*If you should be on this list but you aren't, or you are on the list but don't want to be, please don't hesitate to contact me!*

---

## How to Contribute

### How to Update Docs

The latest version of the docs is hosted on [Github Pages](https://simpletransformers.ai/), if you want to help document Simple Transformers
below are the steps to edit the docs.
Docs are built using [Jekyll](https://jekyllrb.com/) library, refer to their webpage for a detailed explanation of how it works.

1) **Install [Jekyll](https://jekyllrb.com/)**: Run the command `gem install bundler jekyll`
2) **Visualizing the docs on your local computer**:
   In your terminal cd into the docs directory of this repo, eg: `cd simpletransformers/docs`
   From the docs directory run this command to serve the Jekyll docs locally: `bundle exec jekyll serve`
   Browse to http://localhost:4000 or whatever url you see in the console to visualize the docs.
3) **Edit and visualize changes**:
   All the section pages of our docs can be found under `docs/_docs` directory, you can edit any file you want by following the markdown format and visualize the changes after refreshing the browser tab.

---

## Acknowledgements

None of this would have been possible without the hard work by the HuggingFace team in developing the [Transformers](https://github.com/huggingface/transformers) library.

_`<div>`Icon for the Social Media Preview made by `<a href="https://www.flaticon.com/authors/freepik" title="Freepik">`Freepik`</a>` from `<a href="https://www.flaticon.com/" title="Flaticon">`www.flaticon.com`</a></div>`_


tslearn

   The machine learning toolkit for time series analysis in Python

| Section | Description |
|-|-|
| [Installation](#installation) | Installing the dependencies and tslearn |
| [Getting started](#getting-started) | A quick introduction on how to use tslearn |
| [Available features](#available-features) | An extensive overview of tslearn's functionalities |
| [Documentation](#documentation) | A link to our API reference and a gallery of examples |
| [Contributing](#contributing) | A guide for heroes willing to contribute |
| [Citation](#referencing-tslearn) | A citation for tslearn for scholarly articles |

## Installation
There are different alternatives to install tslearn:
* PyPi: `python -m pip install tslearn`
* Conda: `conda install -c conda-forge tslearn`
* Git: `python -m pip install https://github.com/tslearn-team/tslearn/archive/main.zip`

In order for the installation to be successful, the required dependencies must be installed. For a more detailed guide on how to install tslearn, please see the [Documentation](https://tslearn.readthedocs.io/en/stable/?badge=stable#installation).

## Getting started

### 1. Getting the data in the right format
tslearn expects a time series dataset to be formatted as a 3D `numpy` array. The three dimensions correspond to the number of time series, the number of measurements per time series and the number of dimensions respectively (`n_ts, max_sz, d`). In order to get the data in the right format, different solutions exist:
* [You can use the utility functions such as `to_time_series_dataset`.](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.utils.html#module-tslearn.utils)
* [You can convert from other popular time series toolkits in Python.](https://tslearn.readthedocs.io/en/stable/integration_other_software.html)
* [You can load any of the UCR datasets in the required format.](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.datasets.html#module-tslearn.datasets)
* [You can generate synthetic data using the `generators` module.](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.generators.html#module-tslearn.generators)

It should further be noted that tslearn [supports variable-length timeseries](https://tslearn.readthedocs.io/en/stable/variablelength.html).

```python3
>>> from tslearn.utils import to_time_series_dataset
>>> my_first_time_series = [1, 3, 4, 2]
>>> my_second_time_series = [1, 2, 4, 2]
>>> my_third_time_series = [1, 2, 4, 2, 2]
>>> X = to_time_series_dataset([my_first_time_series,
                                my_second_time_series,
                                my_third_time_series])
>>> y = [0, 1, 1]
```

### 2. Data preprocessing and transformations
Optionally, tslearn has several utilities to preprocess the data. In order to facilitate the convergence of different algorithms, you can [scale time series](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.preprocessing.html#module-tslearn.preprocessing). Alternatively, in order to speed up training times, one can [resample](https://tslearn.readthedocs.io/en/stable/gen_modules/preprocessing/tslearn.preprocessing.TimeSeriesResampler.html#tslearn.preprocessing.TimeSeriesResampler) the data or apply a [piece-wise transformation](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.piecewise.html#module-tslearn.piecewise).

```python3
>>> from tslearn.preprocessing import TimeSeriesScalerMinMax
>>> X_scaled = TimeSeriesScalerMinMax().fit_transform(X)
>>> print(X_scaled)
[[[0.] [0.667] [1.] [0.333] [nan]]
 [[0.] [0.333] [1.] [0.333] [nan]]
 [[0.] [0.333] [1.] [0.333] [0.333]]]
```

### 3. Training a model

After getting the data in the right format, a model can be trained. Depending on the use case, tslearn supports different tasks: classification, clustering and regression. For an extensive overview of possibilities, check out our [gallery of examples](https://tslearn.readthedocs.io/en/stable/auto_examples/index.html).

```python3
>>> from tslearn.neighbors import KNeighborsTimeSeriesClassifier
>>> knn = KNeighborsTimeSeriesClassifier(n_neighbors=1)
>>> knn.fit(X_scaled, y)
>>> print(knn.predict(X_scaled))
[0 1 1]
```

As can be seen, the models in tslearn follow the same API as those of the well-known scikit-learn. Moreover, they are fully compatible with it, allowing to use different scikit-learn utilities such as [hyper-parameter tuning and pipelines](https://tslearn.readthedocs.io/en/stable/auto_examples/neighbors/plot_knnts_sklearn.html).

### 4. More analyses

tslearn further allows to perform all different types of analysis. Examples include [calculating barycenters](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.barycenters.html#module-tslearn.barycenters) of a group of time series or calculate the distances between time series using a [variety of distance metrics](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.metrics.html#module-tslearn.metrics).

## Available features

| data                                                                                | processing                                                                                | clustering                                                                                | classification                                                                                | regression                                                                                | metrics                                                                                |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|
| [UCR Datasets](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.datasets.html#module-tslearn.datasets)                                                                           | [Scaling](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.preprocessing.html#module-tslearn.preprocessing) | [TimeSeriesKMeans](https://tslearn.readthedocs.io/en/stable/gen_modules/clustering/tslearn.clustering.TimeSeriesKMeans.html#tslearn.clustering.TimeSeriesKMeans) | [KNN Classifier](https://tslearn.readthedocs.io/en/stable/gen_modules/neighbors/tslearn.neighbors.KNeighborsTimeSeriesClassifier.html#tslearn.neighbors.KNeighborsTimeSeriesClassifier) | [KNN Regressor](https://tslearn.readthedocs.io/en/stable/gen_modules/neighbors/tslearn.neighbors.KNeighborsTimeSeriesRegressor.html#tslearn.neighbors.KNeighborsTimeSeriesRegressor) | [Dynamic Time Warping](https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.dtw.html#tslearn.metrics.dtw)    |
| [Generators](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.generators.html#module-tslearn.generators)                                                                         | [Piecewise](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.piecewise.html#module-tslearn.piecewise)       | [KShape](https://tslearn.readthedocs.io/en/stable/gen_modules/clustering/tslearn.clustering.KShape.html#tslearn.clustering.KShape)                               | [TimeSeriesSVC](https://tslearn.readthedocs.io/en/stable/gen_modules/svm/tslearn.svm.TimeSeriesSVC.html#tslearn.svm.TimeSeriesSVC)                                                      | [TimeSeriesSVR](https://tslearn.readthedocs.io/en/stable/gen_modules/svm/tslearn.svm.TimeSeriesSVR.html#tslearn.svm.TimeSeriesSVR)                                                   | [Global Alignment Kernel](https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.gak.html#tslearn.metrics.gak) |
| Conversion([1](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.utils.html#module-tslearn.utils), [2](https://tslearn.readthedocs.io/en/stable/integration_other_software.html)) |                                                                                | [KernelKmeans](https://tslearn.readthedocs.io/en/stable/gen_modules/clustering/tslearn.clustering.KernelKMeans.html#tslearn.clustering.KernelKMeans)             | [LearningShapelets](https://tslearn.readthedocs.io/en/stable/gen_modules/shapelets/tslearn.shapelets.LearningShapelets.html)                                    | [MLP](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.neural_network.html#module-tslearn.neural_network)                                                                | [Barycenters](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.barycenters.html#module-tslearn.barycenters)              |
|                                                                                |                                                                                |                                                                                | [Early Classification](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.early_classification.html#module-tslearn.early_classification)                                      |                                                                                | [Matrix Profile](https://tslearn.readthedocs.io/en/stable/gen_modules/tslearn.matrix_profile.html#module-tslearn.matrix_profile)     |

## Documentation

The documentation is hosted at [readthedocs](http://tslearn.readthedocs.io/en/stable/index.html). It includes an [API](https://tslearn.readthedocs.io/en/stable/reference.html), [gallery of examples](https://tslearn.readthedocs.io/en/stable/auto_examples/index.html) and a [user guide](https://tslearn.readthedocs.io/en/stable/user_guide/userguide.html).

## Contributing

If you would like to contribute to `tslearn`, please have a look at [our contribution guidelines](CONTRIBUTING.md). A list of interesting TODO's can be found [here](https://github.com/tslearn-team/tslearn/issues?utf8=✓&q=is%3Aissue%20is%3Aopen%20label%3A%22new%20feature%22%20). **If you want other ML methods for time series to be added to this TODO list, do not hesitate to [open an issue](https://github.com/tslearn-team/tslearn/issues/new/choose)!**

## Referencing tslearn

If you use `tslearn` in a scientific publication, we would appreciate citations:

```bibtex
@article{JMLR:v21:20-091,
  author  = {Romain Tavenard and Johann Faouzi and Gilles Vandewiele and
             Felix Divo and Guillaume Androz and Chester Holtz and
             Marie Payne and Roman Yurchak and Marc Ru{\ss}wurm and
             Kushal Kolar and Eli Woods},
  title   = {Tslearn, A Machine Learning Toolkit for Time Series Data},
  journal = {Journal of Machine Learning Research},
  year    = {2020},
  volume  = {21},
  number  = {118},
  pages   = {1-6},
  url     = {http://jmlr.org/papers/v21/20-091.html}
}
```

#### Acknowledgments
Authors would like to thank Mathieu Blondel for providing code for [Kernel k-means](https://gist.github.com/mblondel/6230787) and [Soft-DTW](https://github.com/mblondel/soft-dtw), and to Mehran Maghoumi for his [`torch`-compatible implementation of SoftDTW](https://github.com/Maghoumi/pytorch-softdtw-cuda).


.. image::
    https://raw.github.com/sigmavirus24/github3.py/master/docs/img/gh3-logo.png

github3.py is a comprehensive, actively developed, and extraordinarily stable
wrapper around the GitHub API (v3).

Note: This library currently works with Python 3.7+ or pypy3. For older versions, please use version 1.3.0.

Installation
------------

::

    $ pip install github3.py

Dependencies
------------

- requests_
- uritemplate_
- python-dateutil_
- PyJWT_

.. _requests: https://github.com/kennethreitz/requests
.. _uritemplate: https://github.com/sigmavirus24/uritemplate
.. _python-dateutil: https://github.com/dateutil/dateutil
.. _PyJWT: https://github.com/jpadilla/pyjwt

Contributing
------------

Please read the `CONTRIBUTING`_ document.

.. _CONTRIBUTING: https://github.com/sigmavirus24/github3.py/blob/master/CONTRIBUTING.rst

Testing
~~~~~~~

You can run ``pip install -e .[dev]`` to install the following before testing or
simply ``make test-deps``. It is suggested you do this in a virtual environment.
These need to be installed for the tests to run.

- betamax_
- coverage_ by Ned Batchelder

.. _betamax: https://github.com/sigmavirus24/betamax
.. _coverage: http://nedbatchelder.com/code/coverage/

Build status
~~~~~~~~~~~~

You can find build statuses for different environments.

- Github_

.. _Github: https://github.com/sigmavirus24/github3.py/actions

License
-------

Modified BSD license_

.. _license: https://github.com/sigmavirus24/github3.py/blob/master/LICENSE

Examples
--------

See the docs_ for more examples.

.. _docs: https://github3.readthedocs.io/en/latest/index.html#more-examples

Testing
~~~~~~~

Install the dependencies from requirements.txt e.g.:

::

    make tests

Author
------

Ian Stapleton Cordasco (sigmavirus24_)

.. _sigmavirus24: https://github.com/sigmavirus24

Contact Options
---------------

- Feel free to use the `github3.py`_ tag on Stack Overflow for any questions
  you may have.
- If you dislike Stack Overflow, it is preferred that you use Github
  discussions (https://github.com/sigmavirus24/github3.py/discussions).
- You may also contact (via email_) the author directly with
  questions/suggestions/comments or if you wish to include sensitive data.

.. _github3.py: http://stackoverflow.com/questions/tagged/github3.py
.. _email: mailto:graffatcolmingov@gmail.com


# Stripe Python Library

[![pypi](https://img.shields.io/pypi/v/stripe.svg)](https://pypi.python.org/pypi/stripe)
[![Build Status](https://github.com/stripe/stripe-python/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/stripe/stripe-python/actions?query=branch%3Amaster)
[![Coverage Status](https://coveralls.io/repos/github/stripe/stripe-python/badge.svg?branch=master)](https://coveralls.io/github/stripe/stripe-python?branch=master)

The Stripe Python library provides convenient access to the Stripe API from
applications written in the Python language. It includes a pre-defined set of
classes for API resources that initialize themselves dynamically from API
responses which makes it compatible with a wide range of versions of the Stripe
API.

## Documentation

See the [Python API docs](https://stripe.com/docs/api?lang=python).

See [video demonstrations][youtube-playlist] covering how to use the library.

## Installation

You don't need this source code unless you want to modify the package. If you just
want to use the package, just run:

```sh
pip install --upgrade stripe
```

Install from source with:

```sh
python setup.py install
```

### Requirements

-   Python 3.6+ (PyPy supported)

#### Python 2.7 deprecation

[The Python Software Foundation (PSF)](https://www.python.org/psf-landing/) community [announced the end of support of Python 2](https://www.python.org/doc/sunset-python-2/) on 01 January 2020.
Starting with version 6.0.0 Stripe SDK Python packages will no longer support Python 2.7. To continue to get new features and security updates, please make sure to update your Python runtime to Python 3.6+.

The last version of the Stripe SDK that supports Python 2.7 is 5.5.0.

## Usage

The library needs to be configured with your account's secret key which is
available in your [Stripe Dashboard][api-keys]. Set `stripe.api_key` to its
value:

```python
from stripe import StripeClient

client = StripeClient("sk_test_...")

# list customers
customers = client.customers.list()

# print the first customer's email
print(customers.data[0].email)

# retrieve specific Customer
customer = client.customers.retrieve("cus_123456789")

# print that customer's email
print(customer.email)
```

### Handling exceptions

Unsuccessful requests raise exceptions. The class of the exception will reflect
the sort of error that occurred. Please see the [Api
Reference](https://stripe.com/docs/api/errors/handling) for a description of
the error classes you should handle, and for information on how to inspect
these errors.

### Per-request Configuration

Configure individual requests with the `options` argument. For example, you can make
requests with a specific [Stripe Version](https://stripe.com/docs/api#versioning)
or as a [connected account](https://stripe.com/docs/connect/authentication#authentication-via-the-stripe-account-header):

```python
from stripe import StripeClient

client = StripeClient("sk_test_...")

# list customers
client.customers.list(
    options={
        "api_key": "sk_test_...",
        "stripe_account": "acct_...",
        "stripe_version": "2019-02-19",
    }
)

# retrieve single customer
client.customers.retrieve(
    "cus_123456789",
    options={
        "api_key": "sk_test_...",
        "stripe_account": "acct_...",
        "stripe_version": "2019-02-19",
    }
)
```

### Configuring an HTTP Client

You can configure your `StripeClient` to use `urlfetch`, `requests`, `pycurl`, or
`urllib2` with the `http_client` option:

```python
client = StripeClient("sk_test_...", http_client=stripe.UrlFetchClient())
client = StripeClient("sk_test_...", http_client=stripe.RequestsClient())
client = StripeClient("sk_test_...", http_client=stripe.PycurlClient())
client = StripeClient("sk_test_...", http_client=stripe.Urllib2Client())
```

Without a configured client, by default the library will attempt to load
libraries in the order above (i.e. `urlfetch` is preferred with `urllib2` used
as a last resort). We usually recommend that people use `requests`.

### Configuring a Proxy

A proxy can be configured with the `proxy` client option:

```python
client = StripeClient("sk_test_...", proxy="https://user:pass@example.com:1234")
```

### Configuring Automatic Retries

You can enable automatic retries on requests that fail due to a transient
problem by configuring the maximum number of retries:

```python
client = StripeClient("sk_test_...", max_network_retries=2)
```

Various errors can trigger a retry, like a connection error or a timeout, and
also certain API responses like HTTP status `409 Conflict`.

[Idempotency keys][idempotency-keys] are automatically generated and added to
requests, when not given, to guarantee that retries are safe.

### Logging

The library can be configured to emit logging that will give you better insight
into what it's doing. The `info` logging level is usually most appropriate for
production use, but `debug` is also available for more verbosity.

There are a few options for enabling it:

1. Set the environment variable `STRIPE_LOG` to the value `debug` or `info`

    ```sh
    $ export STRIPE_LOG=debug
    ```

2. Set `stripe.log`:

    ```python
    import stripe
    stripe.log = 'debug'
    ```

3. Enable it through Python's logging module:

    ```python
    import logging
    logging.basicConfig()
    logging.getLogger('stripe').setLevel(logging.DEBUG)
    ```

### Accessing response code and headers

You can access the HTTP response code and headers using the `last_response` property of the returned resource.

```python
customer = client.customers.retrieve(
    "cus_123456789"
)

print(customer.last_response.code)
print(customer.last_response.headers)
```

### Writing a Plugin

If you're writing a plugin that uses the library, we'd appreciate it if you
identified using `stripe.set_app_info()`:

```py
stripe.set_app_info("MyAwesomePlugin", version="1.2.34", url="https://myawesomeplugin.info")
```

This information is passed along when the library makes calls to the Stripe
API.

### Telemetry

By default, the library sends telemetry to Stripe regarding request latency and feature usage. These
numbers help Stripe improve the overall latency of its API for all users, and
improve popular features.

You can disable this behavior if you prefer:

```python
stripe.enable_telemetry = False
```

## Types

In [v7.1.0](https://github.com/stripe/stripe-python/releases/tag/v7.1.0) and
newer, the
library includes type annotations. See [the wiki](https://github.com/stripe/stripe-python/wiki/Inline-type-annotations)
for a detailed guide.

Please note that some annotations use features that were only fairly recently accepted,
such as [`Unpack[TypedDict]`](https://peps.python.org/pep-0692/#specification) that was
[accepted](https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwargs-typing/17314/81)
in January 2023. We have tested that these types are recognized properly by [Pyright](https://github.com/microsoft/pyright).
Support for `Unpack` in MyPy is still experimental, but appears to degrade gracefully.
Please [report an issue](https://github.com/stripe/stripe-python/issues/new/choose) if there
is anything we can do to improve the types for your type checker of choice.

### Types and the Versioning Policy

We release type changes in minor releases. While stripe-python follows semantic
versioning, our semantic versions describe the _runtime behavior_ of the
library alone. Our _type annotations are not reflected in the semantic
version_. That is, upgrading to a new minor version of stripe-python might
result in your type checker producing a type error that it didn't before. You
can use a `~=x.x` or `x.x.*` [version specifier](https://peps.python.org/pep-0440/#examples)
in your `requirements.txt` to constrain `pip` to a certain minor range of `stripe-python`.

### Types and API Versions

The types describe the [Stripe API version](https://stripe.com/docs/api/versioning)
that was the latest at the time of release. This is the version that your library
sends by default. If you are overriding `stripe.api_version` / `stripe_version` on the `StripeClient`, or using a
[webhook endpoint](https://stripe.com/docs/webhooks#api-versions) tied to an older version,
be aware that the data you see at runtime may not match the types.

## Beta SDKs

Stripe has features in the beta phase that can be accessed via the beta version of this package.
We would love for you to try these and share feedback with us before these features reach the stable phase.
To install a beta version use `pip install` with the exact version you'd like to use:

```
pip install --pre stripe
```

> **Note**
> There can be breaking changes between beta versions. Therefore we recommend pinning the package version to a specific beta version in your [requirements file](https://pip.pypa.io/en/stable/user_guide/#requirements-files) or `setup.py`. This way you can install the same version each time without breaking changes unless you are intentionally looking for the latest beta version.

We highly recommend keeping an eye on when the beta feature you are interested in goes from beta to stable so that you can move from using a beta version of the SDK to the stable version.

If your beta feature requires a `Stripe-Version` header to be sent, set the `stripe.api_version` field using the `stripe.add_beta_version` function:

```python
stripe.add_beta_version("feature_beta", "v3")
```

### Async

Asynchronous versions of request-making methods are available by suffixing the method name
with `_async`.

```python
# With StripeClient
client = StripeClient("sk_test_...")
customer = await client.customers.retrieve_async("cus_xyz")

# With global client
stripe.api_key = "sk_test_..."
customer = await stripe.Customer.retrieve_async("cus_xyz")

# .auto_paging_iter() implements both AsyncIterable and Iterable
async for c in await stripe.Customer.list_async().auto_paging_iter():
  ...
```

There is no `.save_async` as `.save` is [deprecated since stripe-python v5](https://github.com/stripe/stripe-python/wiki/Migration-guide-for-v5#deprecated). Please migrate to `.modify_async`.

The default HTTP client uses `requests` for making synchronous requests but
`httpx` for making async requests. If you're migrating to async, we recommend
you to explicitly initialize your own http client and pass it to StripeClient
or set it as the global default.

```python
# By default, an explicitly initialized HTTPXClient will raise an exception if you
# attempt to call a sync method. If you intend to only use async, this is useful to
# make sure you don't unintentionally make a synchronous request.
my_http_client = stripe.HTTPXClient()

# If you want to use httpx to make sync requests, you can disable this
# behavior.
my_http_client = stripe.HTTPXClient(allow_sync_methods=True)

# aiohttp is also available (does not support sync requests)
my_http_client = stripe.AIOHTTPClient()

# With StripeClient
client = StripeClient("sk_test_...", http_client=my_http_client)

# With the global client
stripe.default_http_client = my_http_client
```

You can also subclass `stripe.HTTPClient` and provide your own instance.

## Support

New features and bug fixes are released on the latest major version of the Stripe Python library. If you are on an older major version, we recommend that you upgrade to the latest in order to use the new features and bug fixes including those for security vulnerabilities. Older major versions of the package will continue to be available for use, but will not be receiving any updates.

## Development

The test suite depends on [stripe-mock], so make sure to fetch and run it from a
background terminal ([stripe-mock's README][stripe-mock] also contains
instructions for installing via Homebrew and other methods):

```sh
go install github.com/stripe/stripe-mock@latest
stripe-mock
```

Run the following command to set up the development virtualenv:

```sh
make
```

Run all tests on all supported Python versions:

```sh
make test
```

Run all tests for a specific Python version (modify `-e` according to your Python target):

```sh
TOX_ARGS="-e py37" make test
```

Run all tests in a single file:

```sh
TOX_ARGS="-e py37 -- tests/api_resources/abstract/test_updateable_api_resource.py" make test
```

Run a single test suite:

```sh
TOX_ARGS="-e py37 -- tests/api_resources/abstract/test_updateable_api_resource.py::TestUpdateableAPIResource" make test
```

Run a single test:

```sh
TOX_ARGS="-e py37 -- tests/api_resources/abstract/test_updateable_api_resource.py::TestUpdateableAPIResource::test_save" make test
```

Run the linter with:

```sh
make lint
```

The library uses [Ruff][ruff] for code formatting. Code must be formatted
with Black before PRs are submitted, otherwise CI will fail. Run the formatter
with:

```sh
make fmt
```

[api-keys]: https://dashboard.stripe.com/account/apikeys
[ruff]: https://github.com/astral-sh/ruff
[connect]: https://stripe.com/connect
[poetry]: https://github.com/sdispater/poetry
[stripe-mock]: https://github.com/stripe/stripe-mock
[idempotency-keys]: https://stripe.com/docs/api/idempotent_requests?lang=python
[youtube-playlist]: https://www.youtube.com/playlist?list=PLy1nL-pvL2M55YVn0mGoQ5r-39A1-ZypO

<!--
# vim: set tw=79:
-->


html5lib
========

.. image:: https://github.com/html5lib/html5lib-python/actions/workflows/python-tox.yml/badge.svg
    :target: https://github.com/html5lib/html5lib-python/actions/workflows/python-tox.yml

html5lib is a pure-python library for parsing HTML. It is designed to
conform to the WHATWG HTML specification, as is implemented by all major
web browsers.

Usage
-----

Simple usage follows this pattern:

.. code-block:: python

  import html5lib
  with open("mydocument.html", "rb") as f:
      document = html5lib.parse(f)

or:

.. code-block:: python

  import html5lib
  document = html5lib.parse("
Hello World!")

By default, the ``document`` will be an ``xml.etree`` element instance.
Whenever possible, html5lib chooses the accelerated ``ElementTree``
implementation (i.e. ``xml.etree.cElementTree`` on Python 2.x).

Two other tree types are supported: ``xml.dom.minidom`` and
``lxml.etree``. To use an alternative format, specify the name of
a treebuilder:

.. code-block:: python

  import html5lib
  with open("mydocument.html", "rb") as f:
      lxml_etree_document = html5lib.parse(f, treebuilder="lxml")

When using with ``urllib2`` (Python 2), the charset from HTTP should be
pass into html5lib as follows:

.. code-block:: python

  from contextlib import closing
  from urllib2 import urlopen
  import html5lib

  with closing(urlopen("http://example.com/")) as f:
      document = html5lib.parse(f, transport_encoding=f.info().getparam("charset"))

When using with ``urllib.request`` (Python 3), the charset from HTTP
should be pass into html5lib as follows:

.. code-block:: python

  from urllib.request import urlopen
  import html5lib

  with urlopen("http://example.com/") as f:
      document = html5lib.parse(f, transport_encoding=f.info().get_content_charset())

To have more control over the parser, create a parser object explicitly.
For instance, to make the parser raise exceptions on parse errors, use:

.. code-block:: python

  import html5lib
  with open("mydocument.html", "rb") as f:
      parser = html5lib.HTMLParser(strict=True)
      document = parser.parse(f)

When you're instantiating parser objects explicitly, pass a treebuilder
class as the ``tree`` keyword argument to use an alternative document
format:

.. code-block:: python

  import html5lib
  parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
  minidom_document = parser.parse("
Hello World!")

More documentation is available at https://html5lib.readthedocs.io/.

Installation
------------

html5lib works on CPython 2.7+, CPython 3.5+ and PyPy. To install:

.. code-block:: bash

    $ pip install html5lib

The goal is to support a (non-strict) superset of the versions that `pip
supports
<https://pip.pypa.io/en/stable/installing/#python-and-os-compatibility>`_.

Optional Dependencies
---------------------

The following third-party libraries may be used for additional
functionality:

- ``lxml`` is supported as a tree format (for both building and
  walking) under CPython (but *not* PyPy where it is known to cause
  segfaults);

- ``genshi`` has a treewalker (but not builder); and

- ``chardet`` can be used as a fallback when character encoding cannot
  be determined.

Bugs
----

Please report any bugs on the `issue tracker
<https://github.com/html5lib/html5lib-python/issues>`_.

Tests
-----

Unit tests require the ``pytest`` and ``mock`` libraries and can be
run using the ``pytest`` command in the root directory.

Test data are contained in a separate `html5lib-tests
<https://github.com/html5lib/html5lib-tests>`_ repository and included
as a submodule, thus for git checkouts they must be initialized::

  $ git submodule init
  $ git submodule update

If you have all compatible Python implementations available on your
system, you can run tests on all of them using the ``tox`` utility,
which can be found on PyPI.

Questions?
----------

Check out `the docs <https://html5lib.readthedocs.io/en/latest/>`_. Still
need help? Go to our `GitHub Discussions
<https://github.com/html5lib/html5lib-python/discussions>`_.

You can also browse the archives of the `html5lib-discuss mailing list
<https://www.mail-archive.com/html5lib-discuss@googlegroups.com/>`_.


# NodeGraphQt

[![API Documentation](https://github.com/jchanvfx/NodeGraphQt/actions/workflows/sphinx_doc_build.yml/badge.svg)](https://github.com/jchanvfx/NodeGraphQt/actions/workflows/sphinx_doc_build.yml)
[![PyPI Package](https://github.com/jchanvfx/NodeGraphQt/actions/workflows/pypi_publish.yml/badge.svg)](https://github.com/jchanvfx/NodeGraphQt/actions/workflows/pypi_publish.yml)
![GitHub Repo stars](https://img.shields.io/github/stars/jchanvfx/NodeGraphQt?style=social)

NodeGraphQt is a node graph UI framework for `PySide2` that can be implemented and re-purposed into
applications.

## Install

NodeGraphQt is available on The Python Package Index (PyPI) [here](https://pypi.org/project/NodeGraphQt) so it can be installed with:
```
pip install NodeGraphQt
```
or you can download previous versions from the [releases](https://github.com/jchanvfx/NodeGraphQt/releases) page.

## Documentation

https://jchanvfx.github.io/NodeGraphQt

See the [basic_example.py](/examples/basic_example.py) script to get started or check out the API example overview
here.

## Vertical Layout

https://jchanvfx.github.io/NodeGraphQt/api/examples/ex_pipe.html#layout-direction

## Pipe Layout

https://jchanvfx.github.io/NodeGraphQt/api/examples/ex_pipe.html#layout-styles

## Custom Widgets

https://jchanvfx.github.io/NodeGraphQt/api/custom_widgets.html


==============
MPI for Python
==============

.. image::  https://github.com/mpi4py/mpi4py/workflows/ci/badge.svg?branch=master
   :target: https://github.com/mpi4py/mpi4py/actions/
.. image::  https://readthedocs.org/projects/mpi4py/badge/?version=latest
   :target: https://mpi4py.readthedocs.io/en/latest/
.. image::  https://dev.azure.com/mpi4py/mpi4py/_apis/build/status/ci?branchName=master
   :target: https://dev.azure.com/mpi4py/mpi4py/_build
.. image::  https://ci.appveyor.com/api/projects/status/whh5xovp217h0f7n?svg=true
   :target: https://ci.appveyor.com/project/mpi4py/mpi4py
.. image::  https://circleci.com/gh/mpi4py/mpi4py.svg?style=shield
   :target: https://circleci.com/gh/mpi4py/mpi4py
.. image::  https://codecov.io/gh/mpi4py/mpi4py/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/mpi4py/mpi4py
.. image::  https://scan.coverity.com/projects/mpi4py-mpi4py/badge.svg
   :target: https://scan.coverity.com/projects/mpi4py-mpi4py

Overview
--------

This package provides Python bindings for the *Message Passing
Interface* (`MPI <https://www.mpi-forum.org/>`_) standard. It is
implemented on top of the MPI specification and exposes an API which
grounds on the standard MPI-2 C++ bindings.

Prerequisites
-------------

* `Python <https://www.python.org/>`_ 3.6 or above,
  or `PyPy <https://www.pypy.org/>`_ 7.2 or above.

* An MPI implementation like `MPICH <https://www.mpich.org/>`_ or
  `Open MPI <https://www.open-mpi.org/>`_ built with shared/dynamic
  libraries.

Documentation
-------------

* Read the Docs: https://mpi4py.readthedocs.io/
* GitHub Pages:  https://mpi4py.github.io/

Support
-------

* Mailing List:       mpi4py@googlegroups.com
* Google Groups:      https://groups.google.com/g/mpi4py
* GitHub Discussions: https://github.com/mpi4py/mpi4py/discussions

Testsuite
---------

The testsuite is run periodically on

* `GitHub Actions <https://github.com/mpi4py/mpi4py/actions/>`_

* `Read the Docs <https://readthedocs.org/projects/mpi4py/builds/>`_

* `Azure Pipelines <https://dev.azure.com/mpi4py/mpi4py>`_

* `AppVeyor <https://ci.appveyor.com/project/mpi4py/mpi4py>`_

* `Circle CI <https://circleci.com/gh/mpi4py/mpi4py>`_

* `Codecov <https://app.codecov.io/gh/mpi4py/mpi4py>`_

Citation
--------

+ L. Dalcin and Y.-L. L. Fang,
  *mpi4py: Status Update After 12 Years of Development*,
  Computing in Science & Engineering, 23(4):47-54, 2021.
  https://doi.org/10.1109/MCSE.2021.3083216

* M. Rogowski, S. Aseeri, D. Keyes, and L. Dalcin,
  *mpi4py.futures: MPI-Based Asynchronous Task Execution for Python*,
  IEEE Transactions on Parallel and Distributed Systems, 34(2):611-622, 2023.
  https://doi.org/10.1109/TPDS.2022.3225481


futurecoder

This is a 100% free and interactive course for people to teach themselves programming in Python, especially complete beginners at programming.
It is carefully designed to reduce frustration and guide the user while still ensuring that they learn how to solve problems.
The goal is for as many people as possible to learn programming.

You can try it out here: https://futurecoder.io/

**Please consider [contributing](how_to_contribute.md) or [donating](https://opencollective.com/futurecoder)!**

Alternatively, [come have a chat on slack](https://join.slack.com/t/futurecoder/shared_invite/zt-tp8cmwra-CbdEeX9u3k1VyoMLDupAeQ).

## Features

For a fuller demonstration and explanation, [watch this video](https://www.youtube.com/watch?v=un1BrrV57PA&feature=youtu.be).

The course is a fully interactive 'book' which requires the user to run code in the provided editor or shell to
advance:

![full](images/full.png)

This requires a mixture of solving problems or simply typing and running provided code. In the latter case, the
user is often kept engaged by being asked to predict the output in a simple multiple choice question:

The code at each step is checked automatically. If
needed, the student can get small hints to gradually guide them to the solution:

![predict_output](images/predict_output.png)

![hints](images/hints.png)

If they're still really stuck, they can reveal the solution bit by bit:

Or in some cases solve a *Parsons problem* instead, where they have to put a shuffled solution in the correct
order:

![solution](images/solution.png)

![parsons](images/parsons.png)

Tracebacks are more helpful than usual, with several enhancements:

- Highlighting the exact operation that failed, not just the line, using [executing](https://github.com/alexmojaki/executing)
- Tables of local variables and simple expressions evaluated by [pure_eval](https://github.com/alexmojaki/pure_eval)
- Suggestions for fixes provided by [DidYouMean](https://github.com/SylvainDe/DidYouMean-Python)
- Beginner friendly explanations provided by [friendly-traceback](https://github.com/aroberge/friendly-traceback) (shown when hovering over the little `i` icon)
- Showing multiline statements in full thanks to [stack_data](https://github.com/alexmojaki/stack_data) without showing unnecessary extra lines

Common mistakes can be caught and pointed out to the student. This includes specific checks in some steps as well as linting tailored for beginners.

![traceback](images/traceback.png)

![executing](images/messages.png)

Several debuggers are provided, including [snoop](https://github.com/alexmojaki/snoop)...

[...birdseye...](https://github.com/alexmojaki/birdseye)

![snoop](images/snoop.png)

![birdseye](images/birdseye.png)

...and [Python Tutor](http://pythontutor.com/)

![pythontutor](images/pythontutor.png)

## Running locally

1. Fork this repo, `git clone` it, and enter the top-level `futurecoder` folder.
2. Install Python 3.11.2 and [poetry](https://python-poetry.org/docs/#installation).
3. Run `poetry install` to install Python dependencies.
4. Run `./scripts/generate.sh`. This will generate various static files from Python used by the frontend and run some tests. Repeat this step whenever you change Python files.
5. In the `frontend` folder:
    1. Install `node`. We recommend using version 16.17.1 (lts/gallium) but there are no known issues with other versions.

    2. Run `npm ci` to download dependencies.

    3. Run `npm run build` and then copy `service-worker.js` from the `course` folder to the `public` folder. The service worker is used by [sync-message](https://github.com/alexmojaki/sync-message) to handle `time.sleep` and reading from stdin (e.g. `input()`) in Python. If you want to enable caching (not a good idea when developing), set the environment variable `REACT_APP_PRECACHE=1` when building.

    4. Run `npm start` to start the frontend development server.
6. Go to http://localhost:3000/course/

To learn more about the system, see the [contributing guide](how_to_contribute.md).

## Controls

To explore the course more freely:

1. Click the hamburger menu icon in the top left.
2. Click Settings.
3. Turn Developer mode on.
4. This should give you two red buttons floating at the bottom of the screen. They change the currently active step, so you can move forward without having to complete exercises or backwards to test a step again.


# Jupyter QtConsole

![Windows tests](https://github.com/jupyter/qtconsole/workflows/Windows%20tests/badge.svg)
![Macos tests](https://github.com/jupyter/qtconsole/workflows/Macos%20tests/badge.svg)
![Linux tests](https://github.com/jupyter/qtconsole/workflows/Linux%20tests/badge.svg)
[![Coverage Status](https://coveralls.io/repos/github/jupyter/qtconsole/badge.svg?branch=master)](https://coveralls.io/github/jupyter/qtconsole?branch=master)
[![Documentation Status](https://readthedocs.org/projects/qtconsole/badge/?version=stable)](https://qtconsole.readthedocs.io/en/stable/)
[![Google Group](https://img.shields.io/badge/-Google%20Group-lightgrey.svg)](https://groups.google.com/forum/#!forum/jupyter)

A rich Qt-based console for working with Jupyter kernels,
supporting rich media output, session export, and more.

The Qtconsole is a very lightweight application that largely feels like a terminal, but
provides a number of enhancements only possible in a GUI, such as inline
figures, proper multiline editing with syntax highlighting, graphical calltips,
and more.

![qtconsole](https://raw.githubusercontent.com/jupyter/qtconsole/master/docs/source/_images/qtconsole.png)

## Install Qtconsole
The Qtconsole requires Python bindings for Qt, such as [PyQt6](https://pypi.org/project/PyQt6/), [PySide6](https://pypi.org/project/PySide6/), [PyQt5](https://pypi.org/project/PyQt5/) or [PySide2](https://pypi.org/project/PySide2/).

Although [pip](https://pypi.python.org/pypi/pip) and
[conda](http://conda.pydata.org/docs) may be used to install the Qtconsole, conda
is simpler to use since it automatically installs PyQt5. Alternatively,
the Qtconsole installation with pip needs additional steps since pip doesn't install
the Qt requirement.

### Install using conda
To install:

    conda install qtconsole

**Note:** If the Qtconsole is installed using conda, it will **automatically**
install the Qt requirement as well.

### Install using pip
To install:

    pip install qtconsole

**Note:** Make sure that Qt is installed. Unfortunately, Qt is not
installed when using pip. The next section gives instructions on doing it.

### Installing Qt (if needed)
You can install PyQt5 with pip using the following command:

    pip install pyqt5

or with a system package manager on Linux. For Windows, PyQt binary packages may be
used.

**Note:** Additional information about using a system package manager may be
found in the [qtconsole documentation](https://qtconsole.readthedocs.io).

More installation instructions for PyQt can be found in the [PyQt5 documentation](http://pyqt.sourceforge.net/Docs/PyQt5/installation.html) and [PyQt4 documentation](http://pyqt.sourceforge.net/Docs/PyQt4/installation.html)

Source packages for Windows/Linux/MacOS can be found here: [PyQt5](https://www.riverbankcomputing.com/software/pyqt/download5) and [PyQt4](https://riverbankcomputing.com/software/pyqt/download).

## Usage
To run the Qtconsole:

    jupyter qtconsole

## Resources
- [Project Jupyter website](https://jupyter.org)
- Documentation for the Qtconsole
  * [latest version](https://qtconsole.readthedocs.io/en/latest/) [[PDF](https://media.readthedocs.org/pdf/qtconsole/latest/qtconsole.pdf)]
  * [stable version](https://qtconsole.readthedocs.io/en/stable/) [[PDF](https://media.readthedocs.org/pdf/qtconsole/stable/qtconsole.pdf)]
- [Documentation for Project Jupyter](https://jupyter.readthedocs.io/en/latest/index.html) [[PDF](https://media.readthedocs.org/pdf/jupyter/latest/jupyter.pdf)]
- [Issues](https://github.com/jupyter/qtconsole/issues)
- [Technical support - Jupyter Google Group](https://groups.google.com/forum/#!forum/jupyter)


.. image:: https://raw.githubusercontent.com/pymeasure/pymeasure/master/docs/images/PyMeasure.png
    :alt: PyMeasure Scientific package

PyMeasure scientific package
############################

PyMeasure makes scientific measurements easy to set up and run.
The package contains a repository of instrument classes and a system for running experiment procedures, which provides graphical interfaces for graphing live data and managing queues of experiments.
Both parts of the package are independent, and when combined provide all the necessary requirements for advanced measurements with only limited coding.

PyMeasure is currently under active development, so please report any issues you experience to our `Issues page`_.

.. _Issues page: https://github.com/pymeasure/pymeasure/issues

PyMeasure runs on Python 3.8-3.12, and is tested with continuous-integration on Linux, macOS, and Windows.

.. image:: https://github.com/pymeasure/pymeasure/actions/workflows/pymeasure_CI.yml/badge.svg
    :target: https://github.com/pymeasure/pymeasure/actions/workflows/pymeasure_CI.yml

.. image:: http://readthedocs.org/projects/pymeasure/badge/?version=latest
    :target: http://pymeasure.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.595633.svg
   :target: https://doi.org/10.5281/zenodo.595633

.. image:: https://anaconda.org/conda-forge/pymeasure/badges/version.svg
   :target: https://anaconda.org/conda-forge/pymeasure

.. image:: https://anaconda.org/conda-forge/pymeasure/badges/downloads.svg
   :target: https://anaconda.org/conda-forge/pymeasure

.. image:: https://codecov.io/gh/pymeasure/pymeasure/graph/badge.svg
   :target: https://codecov.io/gh/pymeasure/pymeasure

Quick start
===========

Check out `the documentation`_ for the `quick start guide`_, that covers the installation of Python and PyMeasure.

There are a number of examples in the `examples`_ directory that can help you get up and running.

.. _the documentation: http://pymeasure.readthedocs.org/en/latest/
.. _quick start guide: http://pymeasure.readthedocs.io/en/latest/quick_start.html
.. _examples: https://github.com/pymeasure/pymeasure/tree/master/examples


# Bitcoin Hardware Wallet Interface

[![Build Status](https://api.cirrus-ci.com/github/bitcoin-core/HWI.svg)](https://cirrus-ci.com/github/bitcoin-core/HWI)
[![Documentation Status](https://readthedocs.org/projects/hwi/badge/?version=latest)](https://hwi.readthedocs.io/en/latest/?badge=latest)

The Bitcoin Hardware Wallet Interface is a Python library and command line tool for interacting with hardware wallets.
It provides a standard way for software to work with hardware wallets without needing to implement device specific drivers.
Python software can use the provided library (`hwilib`). Software in other languages can execute the `hwi` tool.

Caveat emptor: Inclusion of a specific hardware wallet vendor does not imply any endorsement of quality or security.

## Prerequisites

Python 3 is required. The libraries and [udev rules](hwilib/udev/README.md) for each device must also be installed. Some libraries will need to be installed

For Ubuntu/Debian:
```
sudo apt install libusb-1.0-0-dev libudev-dev python3-dev
```

For Centos:
```
sudo yum -y install python3-devel libusbx-devel systemd-devel
```

For macOS:
```
brew install libusb
```

## Install

```
git clone https://github.com/bitcoin-core/HWI.git
cd HWI
poetry install # or 'pip3 install .' or 'python3 setup.py install'
```

This project uses the [Poetry](https://github.com/sdispater/poetry) dependency manager. HWI and its dependencies can be installed via poetry by executing the following in the root source directory:

```
poetry install
```

Pip can also be used to automatically install HWI and its dependencies using the `setup.py` file (which is usually in sync with `pyproject.toml`):

```
pip3 install .
```

The `setup.py` file can be used to install HWI and its dependencies so long as `setuptools` is also installed:

```
pip3 install -U setuptools
python3 setup.py install
```

## Dependencies

See `pyproject.toml` for all dependencies. Dependencies under `[tool.poetry.dependencies]` are user dependencies, and `[tool.poetry.dev-dependencies]` for development based dependencies. These dependencies will be installed with any of the three above installation methods.

## Usage

To use, first enumerate all devices and find the one that you want to use with

```
./hwi.py enumerate
```

Once the device type and device path are known, issue commands to it like so:

```
./hwi.py -t <type> -d <path> <command> <command args>
```

All output will be in JSON form and sent to `stdout`.
Additional information or prompts will be sent to `stderr` and will not necessarily be in JSON.
This additional information is for debugging purposes.

To see a complete list of available commands and global parameters, run
`./hwi.py --help`.  To see options specific to a particular command,
pass the `--help` parameter after the command name; for example:

```
./hwi.py getdescriptors --help
```

## Documentation

Documentation for HWI can be found on [readthedocs.io](https://hwi.readthedocs.io/).

### Device Support

For documentation on devices supported and how they are supported, please check the [device support page](https://hwi.readthedocs.io/en/latest/devices/index.html#support-matrix)

### Using with Bitcoin Core

See [Using Bitcoin Core with Hardware Wallets](https://hwi.readthedocs.io/en/latest/examples/bitcoin-core-usage.html).

## License

This project is available under the MIT License, Copyright Andrew Chow.