# ![](https://github.com/CTFd/CTFd/blob/master/CTFd/themes/core/static/img/logo.png?raw=true)

![CTFd MySQL CI](https://github.com/CTFd/CTFd/workflows/CTFd%20MySQL%20CI/badge.svg?branch=master)
![Linting](https://github.com/CTFd/CTFd/workflows/Linting/badge.svg?branch=master)
[![MajorLeagueCyber Discourse](https://img.shields.io/discourse/status?server=https%3A%2F%2Fcommunity.majorleaguecyber.org%2F)](https://community.majorleaguecyber.org/)
[![Documentation Status](https://api.netlify.com/api/v1/badges/6d10883a-77bb-45c1-a003-22ce1284190e/deploy-status)](https://docs.ctfd.io)

## What is CTFd?

CTFd is a Capture The Flag framework focusing on ease of use and customizability. It comes with everything you need to run a CTF and it's easy to customize with plugins and themes.

![CTFd is a CTF in a can.](https://github.com/CTFd/CTFd/blob/master/CTFd/themes/core/static/img/scoreboard.png?raw=true)

## Features

- Create your own challenges, categories, hints, and flags from the Admin Interface
  - Dynamic Scoring Challenges
  - Unlockable challenge support
  - Challenge plugin architecture to create your own custom challenges
  - Static & Regex based flags
    - Custom flag plugins
  - Unlockable hints
  - File uploads to the server or an Amazon S3-compatible backend
  - Limit challenge attempts & hide challenges
  - Automatic bruteforce protection
- Individual and Team based competitions
  - Have users play on their own or form teams to play together
- Scoreboard with automatic tie resolution
  - Hide Scores from the public
  - Freeze Scores at a specific time
- Scoregraphs comparing the top 10 teams and team progress graphs
- Markdown content management system
- SMTP + Mailgun email support
  - Email confirmation support
  - Forgot password support
- Automatic competition starting and ending
- Team management, hiding, and banning
- Customize everything using the [plugin](https://docs.ctfd.io/docs/plugins/overview) and [theme](https://docs.ctfd.io/docs/themes/overview) interfaces
- Importing and Exporting of CTF data for archival
- And a lot more...

## Install

1. Install dependencies: `pip install -r requirements.txt`
   1. You can also use the `prepare.sh` script to install system dependencies using apt.
2. Modify [CTFd/config.ini](https://github.com/CTFd/CTFd/blob/master/CTFd/config.ini) to your liking.
3. Use `python serve.py` or `flask run` in a terminal to drop into debug mode.

You can use the auto-generated Docker images with the following command:

`docker run -p 8000:8000 -it ctfd/ctfd`

Or you can use Docker Compose with the following command from the source repository:

`docker compose up`

Check out the [CTFd docs](https://docs.ctfd.io/) for [deployment options](https://docs.ctfd.io/docs/deployment/installation) and the [Getting Started](https://docs.ctfd.io/tutorials/getting-started/) guide

## Live Demo

https://demo.ctfd.io/

## Support

To get basic support, you can join the [MajorLeagueCyber Community](https://community.majorleaguecyber.org/): [![MajorLeagueCyber Discourse](https://img.shields.io/discourse/status?server=https%3A%2F%2Fcommunity.majorleaguecyber.org%2F)](https://community.majorleaguecyber.org/)

If you prefer commercial support or have a special project, feel free to [contact us](https://ctfd.io/contact/).

## Managed Hosting

Looking to use CTFd but don't want to deal with managing infrastructure? Check out [the CTFd website](https://ctfd.io/) for managed CTFd deployments.

## MajorLeagueCyber

CTFd is heavily integrated with [MajorLeagueCyber](https://majorleaguecyber.org/). MajorLeagueCyber (MLC) is a CTF stats tracker that provides event scheduling, team tracking, and single sign on for events.

By registering your CTF event with MajorLeagueCyber users can automatically login, track their individual and team scores, submit writeups, and get notifications of important events.

To integrate with MajorLeagueCyber, simply register an account, create an event, and install the client ID and client secret in the relevant portion in `CTFd/config.py` or in the admin panel:

```python
OAUTH_CLIENT_ID = None
OAUTH_CLIENT_SECRET = None
```

## Credits

- Logo by [Laura Barbera](http://www.laurabb.com/)
- Theme by [Christopher Thompson](https://github.com/breadchris)
- Notification Sound by [Terrence Martin](https://soundcloud.com/tj-martin-composer)


# Quick reference
- **Where to get help**:
[the Rhino Security Labs Discord](https://discord.gg/8vFE7ucx9W), or [Stack Overflow](https://stackoverflow.com/search?tab=newest&q=pacu)

- **Where to file issues**:
[https://github.com/RhinoSecurityLabs/pacu/issues](https://github.com/RhinoSecurityLabs/pacu/issues)

- **Maintained by**:
[Rhino Security Labs](https://rhinosecuritylabs.com/)

# What is Pacu?

Pacu is an open-source AWS exploitation framework, designed for offensive security testing against cloud environments. Created and maintained by Rhino Security Labs, Pacu allows penetration testers to exploit configuration flaws within an AWS account, using modules to easily expand its functionality. Current modules enable a range of attacks, including user privilege escalation, backdooring of IAM users, attacking vulnerable Lambda functions, and much more.

## Installation

Pacu is a fairly lightweight program, as it requires only [Python3.7+](https://www.python.org/downloads/) and pip3 to install a handful of Python libraries.

## Quick Installation

```
  > pip3 install -U pip
  > pip3 install -U pacu
  > pacu
```

For a more detailed and user-friendly set of user instructions, please check out the Wiki's [installation guide](https://github.com/RhinoSecurityLabs/pacu/wiki/Installation).

## How to use Pacu's Docker image

[![Try in PWD](https://github.com/play-with-docker/stacks/raw/cff22438cb4195ace27f9b15784bbb497047afa7/assets/images/button.png)](http://play-with-docker.com?stack=https://raw.githubusercontent.com/RhinoSecurityLabs/pacu/master/docker_stack.yml)

### Option 1: Run with default entrypoint which directly runs  Pacu
```console
$ docker run -it rhinosecuritylabs/pacu:latest
```

### Option 2: Run without default entrypoint
```console
$ docker run -it --entrypoint /bin/sh rhinosecuritylabs/pacu:latest
```

### Option 3: Run with AWS config and credentials

> Warning: Running this command will mount your local AWS configuration files into the Docker container when it is launched. This means that any user with access to the container will have access to your host computer's AWS credentials.

```console
$ docker run -it -v ~/.aws:/root/.aws rhinosecuritylabs/pacu:latest
```
## Getting Started

The first time Pacu is launched, you will be prompted to start and name a new session. This session will be used to store AWS key pairs, as well as any data obtained from running various modules. You can have any number of different sessions in Pacu, each with their own sets of AWS keys and data, and resume a session at any time (though a restart is currently required to switch between sessions).

Modules require an AWS key, which grants you minimal access to an AWS environment and is comprised of an access key ID and a secret access key. To set your session's keys, use the `set_keys` command, and then follow the prompts to supply a key alias (nickname for reference), an AWS access key ID, an AWS secret access key, and an AWS session token (if you are using one).

If you are ever stuck, `help` will bring up a list of available commands.

## Basic Commands in Pacu

* `list` will list the available modules for the regions that were set in the current session.
* `help module_name` will return the applicable help information for the specified module.
* `run module_name` will run the specified module with its default parameters.
* `run module_name --regions eu-west-1,us-west-1` will run the specified module against the eu-west-1 and us-west-1 regions (for modules that support the --regions argument)

## Running Pacu From the CLI

* `pacu --help` will display the help menu
* `pacu --session <session name>` sets the session to use for commands that require one
* `pacu --list-modules` will list all modules available (does not require session)
* `pacu --pacu-help` will list the pacu help window (does not require session)
* `pacu --module-name <module name>` the name of a module to perform an action on, you can execute or get information on the module
* `pacu --exec` execute the module provided in `--module-name`
* `pacu --module-info` get information on the module provided in `--module-name`
* `pacu --data <service name || all>` query the local SQLAlchemy database to retrieve enumerated information
* `pacu --module-args="<arg1> <value> <arg2> <value>"` supply optional module arguments to the module being executed
* `pacu --set-regions <region1 region2 || all>` set the regions to use in the session, separate regions by a space or enter `all` for all regions
* `pacu --whoami` get information about the current user

## Pacu's Modular Power

Pacu uses a range of [plug-in modules](https://github.com/RhinoSecurityLabs/pacu/wiki/Module-Details) to assist an attacker in enumeration, privilege escalation, data exfiltration, service exploitation, and log manipulation within AWS environments. Contributions or ideas for new modules are welcome.

To keep pace with ongoing AWS product developments, we've designed Pacu from the ground up with extensibility in mind. A common syntax and data structure keep modules easy to build and expand on - no need to specify AWS regions or make redundant permission checks between modules.  A local SQLite database is used to manage and manipulate retrieved data, minimizing API calls (and associated logs). Reporting and attack auditing is also built into the framework; Pacu assists the documentation process through command logging and exporting, helping build a timeline for the testing process.

## Community

We're always happy to get bug reports in the Pacu framework itself, as well as testing and feedback on different modules, and generally, critical feedback to help refine the framework. Any support for Pacu through use, testing, improvement, or just by spreading the word, would be very much appreciated.

If you're interested in contributing directly to the Pacu Framework itself, please read our [contribution guidelines](https://github.com/RhinoSecurityLabs/pacu/wiki/Contribution-Guidelines) for code conventions and git-flow notes.

## Developing Pacu Modules

If you're interested in writing your own modules for Pacu, check out our [Module Development](https://github.com/RhinoSecurityLabs/pacu/wiki/Module-Development-Guide) wiki page. As you develop new capabilities please reach out to us -- we'd love to add your new modules into the core collection that comes with Pacu.

## Pacu Framework Development Goals
  * Improve interface formatting
  * Database forward-migrations and version tracking
  * "Attack Playbooks" to allow for easier use of complex module execution chains
  * Colored console output
  * Module Dry-Run functionality
  * Allow use of standalone config files
  * Plugin architecture improvements

## Notes

* Pacu is officially supported in OSX and Linux.
* Pacu is Open-Source Software and is distributed with a BSD-3-Clause License.

## Submitting Requests / Bug Reports

- Report vulnerabilities in Pacu directly to us via email: [pacu@rhinosecuritylabs.com](mailto:pacu@rhinosecuritylabs.com) .
- Pacu creates error logs within each session's folder, as well as a global error log for out-of-session errors which is created in the main directory. If you can, please include these logs with your bug reports, as it will dramatically simplify the debugging process.
- If you have a feature request, an idea, or a bug to report, please [submit them here](https://github.com/RhinoSecurityLabs/pacu/issues/new/choose).
  - Please include a description sufficient to reproduce the bug you found, including tracebacks and reproduction steps, and check for other reports of your bug before filing a new bug report. Don't submit duplicates.

## Wiki

For walkthroughs and full documentation, please visit the [Pacu wiki](https://github.com/RhinoSecurityLabs/pacu/wiki).

## Contact Us

* We'd love to hear from you, whatever the reason. Reach out on [the Rhino Security Labs Discord](https://discord.gg/8vFE7ucx9W).

## Disclaimers, and the AWS Acceptable Use Policy

* To the best of our knowledge Pacu's capabilities are compliant with the AWS Acceptable Use Policy, but as a flexible and modular tool, we cannot guarantee this will be true in every situation. It is entirely your responsibility to ensure that how you use Pacu is compliant with the AWS Acceptable Use Policy.
* Depending on what AWS services you use and what your planned testing entails, you may need to review [AWS Customer Support Policy for Penetration Testing](https://aws.amazon.com/security/penetration-testing/) before actually running Pacu against your infrastructure.
* As with any penetration testing tool, it is your responsibility to get proper authorization before using Pacu outside of your environment.
* Pacu is software that comes with absolutely no warranties whatsoever. By using Pacu, you take full responsibility for any and all outcomes that result.


***
PEX
***
.. image:: https://github.com/pantsbuild/pex/workflows/CI/badge.svg?branch=main
    :target: https://github.com/pantsbuild/pex/actions?query=branch%3Amain+workflow%3ACI
.. image:: https://img.shields.io/pypi/l/pex.svg
    :target: https://pypi.org/project/pex/
.. image:: https://img.shields.io/pypi/v/pex.svg
    :target: https://pypi.org/project/pex/
.. image:: https://img.shields.io/pypi/pyversions/pex.svg
    :target: https://pypi.org/project/pex/
.. image:: https://img.shields.io/pypi/wheel/pex.svg
    :target: https://pypi.org/project/pex/#files

.. contents:: **Contents**

Overview
========
pex is a library for generating .pex (Python EXecutable) files which are
executable Python environments in the spirit of `virtualenvs <http://virtualenv.org>`_.
pex is an expansion upon the ideas outlined in
`PEP 441 <https://peps.python.org/pep-0441/>`_
and makes the deployment of Python applications as simple as ``cp``.  pex files may even
include multiple platform-specific Python distributions, meaning that a single pex file
can be portable across Linux and OS X.

pex files can be built using the ``pex`` tool.  Build systems such as `Pants
<http://pantsbuild.org/>`_, `Buck <http://facebook.github.io/buck/>`_, and  `{py}gradle <https://github.com/linkedin/pygradle>`_  also
support building .pex files directly.

Still unsure about what pex does or how it works?  Watch this quick lightning
talk: `WTF is PEX? <http://www.youtube.com/watch?v=NmpnGhRwsu0>`_.

pex is licensed under the Apache2 license.

Installation
============

To install pex, simply

.. code-block:: bash

    $ pip install pex

You can also build pex in a git clone using tox:

.. code-block:: bash

    $ tox -e package
    $ cp dist/pex ~/bin

This builds a pex binary in ``dist/pex`` that can be copied onto your ``$PATH``.
The advantage to this approach is that it keeps your Python environment as empty as
possible and is more in-line with what pex does philosophically.

Simple Examples
===============

Launch an interpreter with ``requests``, ``flask`` and ``psutil`` in the environment:

.. code-block:: bash

    $ pex requests flask 'psutil>2,<3'

Save Dependencies From Pip
~~~~~~~~~~~~~~~~~~~~~~~~~~

Or instead freeze your current virtualenv via requirements.txt and execute it anywhere:

.. code-block:: bash

    $ pex $(pip freeze) -o my_virtualenv.pex
    $ deactivate
    $ ./my_virtualenv.pex

Ephemeral Environments
~~~~~~~~~~~~~~~~~~~~~~

Run webserver.py in an environment containing ``flask`` as a quick way to experiment:

.. code-block:: bash

    $ pex flask -- webserver.py

Launch Sphinx in an ephemeral pex environment using the Sphinx entry point ``sphinx:main``:

.. code-block:: bash

    $ pex sphinx -e sphinx:main -- --help

Using Entry Points
~~~~~~~~~~~~~~~~~~

Projects specifying a ``console_scripts`` entry point in their configuration
can build standalone executables for those entry points.

To build a standalone ``pex-tools-executable.pex`` binary that runs the
``pex-tools`` console script found in all pex version ``2.1.35`` and newer distributions:

.. code-block:: bash

    $ pex "pex>=2.1.35" --console-script pex-tools --output-file pex-tools-executable.pex

Specifying A Specific Interpreter
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can also build pex files that use a specific interpreter type:

.. code-block:: bash

    $ pex "pex>=2.1.35" -c pex-tools --python=pypy -o pex-tools-pypy-executable.pex

Most pex options compose well with one another, so the above commands can be
mixed and matched, and equivalent short options are available.

For a full list of options, just type ``pex --help``.

Integrating pex into your workflow
==================================

If you use tox (and you should!), a simple way to integrate pex into your
workflow is to add a packaging test environment to your ``tox.ini``:

.. code-block:: ini

    [testenv:package]
    deps = pex
    commands = pex . -o dist/app.pex

Then ``tox -e package`` will produce a relocatable copy of your application
that you can copy to staging or production environments.

Documentation
=============

More documentation about Pex, building .pex files, and how .pex files work
is available at https://pex.readthedocs.io.

Development
===========

Pex uses `tox <https://tox.wiki/en/latest/>`_ for test and development automation. To run
the test suite, just invoke tox:

.. code-block:: bash

    $ tox

If you don't have tox, you can generate a pex of tox:

.. code-block::

    $ pex tox -c tox -o ~/bin/tox

Tox provides many useful commands and options, explained at https://tox.wiki/en/latest/ .
Below, we provide some of the most commonly used commands used when working on Pex, but the
docs are worth acquainting yourself with to better understand how Tox works and how to do more
advanced commands.

To run a specific environment, identify the name of the environment you'd like to invoke by
running ``tox --listenvs-all``, then invoke like this:

.. code-block::

    $ tox -e format-run

To run MyPy:

.. code-block::

    $ tox -e typecheck

All of our tox test environments allow passthrough arguments, which can be helpful to run
specific tests:

.. code-block::

    $ tox -e py37-integration -- -k test_reproducible_build

To run Pex from source, rather than through what is on your PATH, invoke via Python:

.. code-block::

    $ python -m pex

Contributing
============

To contribute, follow these instructions: https://www.pantsbuild.org/docs/contributor-overview


#  HoloViews

**Stop plotting your data - annotate your data and let it visualize
itself.**

|    |    |
| --- | --- |
| Downloads | ![https://pypistats.org/packages/holoviews](https://img.shields.io/pypi/dm/holoviews?label=pypi) ![https://anaconda.org/pyviz/holoviews](https://pyviz.org/_static/cache/holoviews_conda_downloads_badge.svg)
| Build Status | [![Build Status](https://github.com/holoviz/holoviews/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/holoviz/holoviews/actions/workflows/test.yaml?query=branch%3Amain) |
| Coverage | [![codecov](https://codecov.io/gh/holoviz/holoviews/branch/main/graph/badge.svg)](https://codecov.io/gh/holoviz/holoviews) |
| Latest dev release | [![Github tag](https://img.shields.io/github/tag/holoviz/holoviews.svg?label=tag&colorB=11ccbb)](https://github.com/holoviz/holoviews/tags) [![dev-site](https://img.shields.io/website-up-down-green-red/http/dev.holoviews.org.svg?label=dev%20website)](https://dev.holoviews.org) |
| Latest release | [![Github release](https://img.shields.io/github/release/holoviz/holoviews.svg?label=tag&colorB=11ccbb)](https://github.com/holoviz/holoviews/releases) [![PyPI version](https://img.shields.io/pypi/v/holoviews.svg?colorB=cc77dd)](https://pypi.python.org/pypi/holoviews) [![holoviews version](https://img.shields.io/conda/v/pyviz/holoviews.svg?colorB=4488ff&style=flat)](https://anaconda.org/pyviz/holoviews) [![conda-forge version](https://img.shields.io/conda/v/conda-forge/holoviews.svg?label=conda%7Cconda-forge&colorB=4488ff)](https://anaconda.org/conda-forge/holoviews) [![defaults version](https://img.shields.io/conda/v/anaconda/holoviews.svg?label=conda%7Cdefaults&style=flat&colorB=4488ff)](https://anaconda.org/anaconda/holoviews) |
| Python | [![Python support](https://img.shields.io/pypi/pyversions/holoviews.svg)](https://pypi.org/project/holoviews/) |
| Docs | [![DocBuildStatus](https://github.com/holoviz/holoviews/workflows/docs/badge.svg?query=branch%3Amain)](https://github.com/holoviz/holoviews/actions?query=workflow%3Adocs+branch%3Amain) [![site](https://img.shields.io/website-up-down-green-red/https/holoviews.org.svg)](https://holoviews.org) |
| Binder | [![Binder](https://img.shields.io/badge/Launch%20JupyterLab-v1.15.4-579ACA.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFkAAABZCAMAAABi1XidAAAB8lBMVEX///9XmsrmZYH1olJXmsr1olJXmsrmZYH1olJXmsr1olJXmsrmZYH1olL1olJXmsr1olJXmsrmZYH1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olJXmsrmZYH1olL1olL0nFf1olJXmsrmZYH1olJXmsq8dZb1olJXmsrmZYH1olJXmspXmspXmsr1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olLeaIVXmsrmZYH1olL1olL1olJXmsrmZYH1olLna31Xmsr1olJXmsr1olJXmsrmZYH1olLqoVr1olJXmsr1olJXmsrmZYH1olL1olKkfaPobXvviGabgadXmsqThKuofKHmZ4Dobnr1olJXmsr1olJXmspXmsr1olJXmsrfZ4TuhWn1olL1olJXmsqBi7X1olJXmspZmslbmMhbmsdemsVfl8ZgmsNim8Jpk8F0m7R4m7F5nLB6jbh7jbiDirOEibOGnKaMhq+PnaCVg6qWg6qegKaff6WhnpKofKGtnomxeZy3noG6dZi+n3vCcpPDcpPGn3bLb4/Mb47UbIrVa4rYoGjdaIbeaIXhoWHmZYHobXvpcHjqdHXreHLroVrsfG/uhGnuh2bwj2Hxk17yl1vzmljzm1j0nlX1olL3AJXWAAAAbXRSTlMAEBAQHx8gICAuLjAwMDw9PUBAQEpQUFBXV1hgYGBkcHBwcXl8gICAgoiIkJCQlJicnJ2goKCmqK+wsLC4usDAwMjP0NDQ1NbW3Nzg4ODi5+3v8PDw8/T09PX29vb39/f5+fr7+/z8/Pz9/v7+zczCxgAABC5JREFUeAHN1ul3k0UUBvCb1CTVpmpaitAGSLSpSuKCLWpbTKNJFGlcSMAFF63iUmRccNG6gLbuxkXU66JAUef/9LSpmXnyLr3T5AO/rzl5zj137p136BISy44fKJXuGN/d19PUfYeO67Znqtf2KH33Id1psXoFdW30sPZ1sMvs2D060AHqws4FHeJojLZqnw53cmfvg+XR8mC0OEjuxrXEkX5ydeVJLVIlV0e10PXk5k7dYeHu7Cj1j+49uKg7uLU61tGLw1lq27ugQYlclHC4bgv7VQ+TAyj5Zc/UjsPvs1sd5cWryWObtvWT2EPa4rtnWW3JkpjggEpbOsPr7F7EyNewtpBIslA7p43HCsnwooXTEc3UmPmCNn5lrqTJxy6nRmcavGZVt/3Da2pD5NHvsOHJCrdc1G2r3DITpU7yic7w/7Rxnjc0kt5GC4djiv2Sz3Fb2iEZg41/ddsFDoyuYrIkmFehz0HR2thPgQqMyQYb2OtB0WxsZ3BeG3+wpRb1vzl2UYBog8FfGhttFKjtAclnZYrRo9ryG9uG/FZQU4AEg8ZE9LjGMzTmqKXPLnlWVnIlQQTvxJf8ip7VgjZjyVPrjw1te5otM7RmP7xm+sK2Gv9I8Gi++BRbEkR9EBw8zRUcKxwp73xkaLiqQb+kGduJTNHG72zcW9LoJgqQxpP3/Tj//c3yB0tqzaml05/+orHLksVO+95kX7/7qgJvnjlrfr2Ggsyx0eoy9uPzN5SPd86aXggOsEKW2Prz7du3VID3/tzs/sSRs2w7ovVHKtjrX2pd7ZMlTxAYfBAL9jiDwfLkq55Tm7ifhMlTGPyCAs7RFRhn47JnlcB9RM5T97ASuZXIcVNuUDIndpDbdsfrqsOppeXl5Y+XVKdjFCTh+zGaVuj0d9zy05PPK3QzBamxdwtTCrzyg/2Rvf2EstUjordGwa/kx9mSJLr8mLLtCW8HHGJc2R5hS219IiF6PnTusOqcMl57gm0Z8kanKMAQg0qSyuZfn7zItsbGyO9QlnxY0eCuD1XL2ys/MsrQhltE7Ug0uFOzufJFE2PxBo/YAx8XPPdDwWN0MrDRYIZF0mSMKCNHgaIVFoBbNoLJ7tEQDKxGF0kcLQimojCZopv0OkNOyWCCg9XMVAi7ARJzQdM2QUh0gmBozjc3Skg6dSBRqDGYSUOu66Zg+I2fNZs/M3/f/Grl/XnyF1Gw3VKCez0PN5IUfFLqvgUN4C0qNqYs5YhPL+aVZYDE4IpUk57oSFnJm4FyCqqOE0jhY2SMyLFoo56zyo6becOS5UVDdj7Vih0zp+tcMhwRpBeLyqtIjlJKAIZSbI8SGSF3k0pA3mR5tHuwPFoa7N7reoq2bqCsAk1HqCu5uvI1n6JuRXI+S1Mco54YmYTwcn6Aeic+kssXi8XpXC4V3t7/ADuTNKaQJdScAAAAAElFTkSuQmCC)](https://mybinder.org/v2/gh/holoviz/holoviews/v1.15.4?urlpath=lab/tree/examples) |
| Support | [![Discourse](https://img.shields.io/discourse/status?server=https%3A%2F%2Fdiscourse.holoviz.org)](https://discourse.holoviz.org/) |

HoloViews is an [open-source](https://github.com/holoviz/holoviews/blob/main/LICENSE.txt)
Python library designed to make data analysis and visualization seamless
and simple. With HoloViews, you can usually express what you want to do
in very few lines of code, letting you focus on what you are trying to
explore and convey, not on the process of plotting.

Check out the [HoloViews web site](https://holoviews.org) for extensive examples and documentation.

# Installation

HoloViews works with [Python](https://github.com/holoviz/holoviews/actions/workflows/test.yaml)
on Linux, Windows, or Mac, and works seamlessly with [Jupyter Notebook and JupyterLab](https://jupyter.org).

You can install HoloViews either with `conda` or `pip`, for more information see the [install guide](https://holoviews.org/install.html).

    conda install holoviews

    pip install holoviews

# Developer Guide

If you want to help develop HoloViews, you can checkout the [developer guide](https://dev.holoviews.org/developer_guide/index.html),
this guide will help you get set-up. Making it easy to contribute.

# Support & Feedback

If you find any bugs or have any feature suggestions please file a GitHub
[issue](https://github.com/holoviz/holoviews/issues).

If you have any usage questions, please ask them on [HoloViz Discourse](https://discourse.holoviz.org/),

For general discussion, we have a [Discord channel](https://discord.gg/AXRHnJU6sP).


# docker-python

[Kaggle Notebooks](https://www.kaggle.com/notebooks) allow users to run a Python Notebook in the cloud against our competitions and datasets without having to download data or set up their environment.

This repository includes the [Dockerfile](Dockerfile.tmpl) for building the CPU-only and GPU image that runs Python Notebooks on Kaggle.

Our Python Docker images are stored on the Google Container Registry at:

* CPU-only: [gcr.io/kaggle-images/python](https://gcr.io/kaggle-images/python)
* GPU: [gcr.io/kaggle-gpu-images/python](https://gcr.io/kaggle-gpu-images/python)

## Requesting new packages

First, evaluate whether installing the package yourself in your own notebooks suits your needs. See [guide](https://github.com/Kaggle/docker-python/wiki/Missing-Packages).

If you the first step above doesn't work for your use case, [open an issue](https://github.com/Kaggle/docker-python/issues/new) or a [pull request](https://github.com/Kaggle/docker-python/pulls).

## Opening a pull request

1. Edit the [Dockerfile](Dockerfile.tmpl).
1. Follow the instructions below to build a new image.
1. Add tests for your new package. See this [example](https://github.com/Kaggle/docker-python/blob/main/tests/test_fastai.py).
1. Follow the instructions below to test the new image.
1. Open a PR on this repo and you are all set!

## Building a new image

```sh
./build
```

Flags:

* `--gpu` to build an image for GPU.
* `--use-cache` for faster iterative builds.

## Testing a new image

A suite of tests can be found under the `/tests` folder. You can run the test using this command:

```sh
./test
```

Flags:

* `--gpu` to test the GPU image.
* `--pattern test_keras.py` or `-p test_keras.py` to run a single test
* `--image gcr.io/kaggle-images/python:ci-pretest` or `-i gcr.io/kaggle-images/python:ci-pretest` to test against a specific image

## Running the image

For the CPU-only image:

```sh
# Run the image built locally:
docker run --rm -it kaggle/python-build /bin/bash
# Run the pre-built image from gcr.io
docker run --rm -it gcr.io/kaggle-images/python /bin/bash
```

For the GPU image:

```sh
# Run the image built locally:
docker run --runtime nvidia --rm -it kaggle/python-gpu-build /bin/bash
# Run the image pre-built image from gcr.io
docker run --runtime nvidia --rm -it gcr.io/kaggle-gpu-images/python /bin/bash
```

To ensure your container can access the GPU, follow the instructions posted [here](https://github.com/Kaggle/docker-python/issues/361#issuecomment-448093930).


# Ibis

[![Documentation status](https://img.shields.io/badge/docs-docs.ibis--project.org-blue.svg)](http://ibis-project.org)
[![Project chat](https://img.shields.io/badge/zulip-join_chat-purple.svg?logo=zulip)](https://ibis-project.zulipchat.com)
[![Anaconda badge](https://anaconda.org/conda-forge/ibis-framework/badges/version.svg)](https://anaconda.org/conda-forge/ibis-framework)
[![PyPI](https://img.shields.io/pypi/v/ibis-framework.svg)](https://pypi.org/project/ibis-framework)
[![Build status](https://github.com/ibis-project/ibis/actions/workflows/ibis-main.yml/badge.svg)](https://github.com/ibis-project/ibis/actions/workflows/ibis-main.yml?query=branch%3Amain)
[![Build status](https://github.com/ibis-project/ibis/actions/workflows/ibis-backends.yml/badge.svg)](https://github.com/ibis-project/ibis/actions/workflows/ibis-backends.yml?query=branch%3Amain)
[![Codecov branch](https://img.shields.io/codecov/c/github/ibis-project/ibis/main.svg)](https://codecov.io/gh/ibis-project/ibis)

## What is Ibis?

Ibis is the portable Python dataframe library:

- Fast local dataframes (via DuckDB by default)
- Lazy dataframe expressions
- Interactive mode for iterative data exploration
- [Compose Python dataframe and SQL code](#python--sql-better-together)
- Use the same dataframe API for [20+ backends](#backends)
- Iterate locally and deploy remotely by [changing a single line of code](#portability)

See the documentation on ["Why Ibis?"](https://ibis-project.org/why) to learn more.

## Getting started

You can `pip install` Ibis with a backend and example data:

```bash
pip install 'ibis-framework[duckdb,examples]'
```

> 💡 **Tip**
>
> See the [installation guide](https://ibis-project.org/install) for more installation options.

Then use Ibis:

```python
>>> import ibis
>>> ibis.options.interactive = True
>>> t = ibis.examples.penguins.fetch()
>>> t
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓
┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃
┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩
│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │
├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤
│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │
│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │
│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │
│ Adelie  │ Torgersen │           NULL │          NULL │              NULL │        NULL │ NULL   │  2007 │
│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │
│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │
│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │
│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │
│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │
│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │
│ …       │ …         │              … │             … │                 … │           … │ …      │     … │
└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘
>>> g = t.group_by("species", "island").agg(count=t.count()).order_by("count")
>>> g
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ species   ┃ island    ┃ count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ string    │ string    │ int64 │
├───────────┼───────────┼───────┤
│ Adelie    │ Biscoe    │    44 │
│ Adelie    │ Torgersen │    52 │
│ Adelie    │ Dream     │    56 │
│ Chinstrap │ Dream     │    68 │
│ Gentoo    │ Biscoe    │   124 │
└───────────┴───────────┴───────┘
```

> 💡 **Tip**
>
> See the [getting started tutorial](https://ibis-project.org/tutorials/getting_started) for a full introduction to Ibis.

## Python + SQL: better together

For most backends, Ibis works by compiling its dataframe expressions into SQL:

```python
>>> ibis.to_sql(g)
SELECT
  "t1"."species",
  "t1"."island",
  "t1"."count"
FROM (
  SELECT
    "t0"."species",
    "t0"."island",
    COUNT(*) AS "count"
  FROM "penguins" AS "t0"
  GROUP BY
    1,
    2
) AS "t1"
ORDER BY
  "t1"."count" ASC
```

You can mix SQL and Python code:

```python
>>> a = t.sql("SELECT species, island, count(*) AS count FROM penguins GROUP BY 1, 2")
>>> a
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ species   ┃ island    ┃ count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ string    │ string    │ int64 │
├───────────┼───────────┼───────┤
│ Adelie    │ Torgersen │    52 │
│ Adelie    │ Biscoe    │    44 │
│ Adelie    │ Dream     │    56 │
│ Gentoo    │ Biscoe    │   124 │
│ Chinstrap │ Dream     │    68 │
└───────────┴───────────┴───────┘
>>> b = a.order_by("count")
>>> b
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓
┃ species   ┃ island    ┃ count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩
│ string    │ string    │ int64 │
├───────────┼───────────┼───────┤
│ Adelie    │ Biscoe    │    44 │
│ Adelie    │ Torgersen │    52 │
│ Adelie    │ Dream     │    56 │
│ Chinstrap │ Dream     │    68 │
│ Gentoo    │ Biscoe    │   124 │
└───────────┴───────────┴───────┘
```

This allows you to combine the flexibility of Python with the scale and performance of modern SQL.

## Backends

Ibis supports 20+ backends:

- [Apache DataFusion](https://ibis-project.org/backends/datafusion/)
- [Apache Druid](https://ibis-project.org/backends/druid/)
- [Apache Flink](https://ibis-project.org/backends/flink)
- [Apache Impala](https://ibis-project.org/backends/impala/)
- [Apache PySpark](https://ibis-project.org/backends/pyspark/)
- [BigQuery](https://ibis-project.org/backends/bigquery/)
- [ClickHouse](https://ibis-project.org/backends/clickhouse/)
- [Dask](https://ibis-project.org/backends/dask/)
- [DuckDB](https://ibis-project.org/backends/duckdb/)
- [Exasol](https://ibis-project.org/backends/exasol)
- [MySQL](https://ibis-project.org/backends/mysql/)
- [Oracle](https://ibis-project.org/backends/oracle/)
- [pandas](https://ibis-project.org/backends/pandas/)
- [Polars](https://ibis-project.org/backends/polars/)
- [PostgreSQL](https://ibis-project.org/backends/postgresql/)
- [RisingWave](https://ibis-project.org/backends/risingwave/)
- [SQL Server](https://ibis-project.org/backends/mssql/)
- [SQLite](https://ibis-project.org/backends/sqlite/)
- [Snowflake](https://ibis-project.org/backends/snowflake)
- [Trino](https://ibis-project.org/backends/trino/)

## How it works

Most Python dataframes are tightly coupled to their execution engine. And many databases only support SQL, with no Python API. Ibis solves this problem by providing a common API for data manipulation in Python, and compiling that API into the backend’s native language. This means you can learn a single API and use it across any supported backend (execution engine).

Ibis broadly supports two types of backend:

1. SQL-generating backends
2. DataFrame-generating backends

![Ibis backend types](https://raw.githubusercontent.com/ibis-project/ibis/main/docs/images/backends.png)

## Portability

To use different backends, you can set the backend Ibis uses:

```python
>>> ibis.set_backend("duckdb")
>>> ibis.set_backend("polars")
>>> ibis.set_backend("datafusion")
```

Typically, you'll create a connection object:

```python
>>> con = ibis.duckdb.connect()
>>> con = ibis.polars.connect()
>>> con = ibis.datafusion.connect()
```

And work with tables in that backend:

```python
>>> con.list_tables()
['penguins']
>>> t = con.table("penguins")
```

You can also read from common file formats like CSV or Apache Parquet:

```python
>>> t = con.read_csv("penguins.csv")
>>> t = con.read_parquet("penguins.parquet")
```

This allows you to iterate locally and deploy remotely by changing a single line of code.

> 💡 **Tip**
>
> Check out [the blog on backend agnostic arrays](https://ibis-project.org/posts/backend-agnostic-arrays/) for one example using the same code across DuckDB and BigQuery.

## Community and contributing

Ibis is an open source project and welcomes contributions from anyone in the community.

- Read [the contributing guide](https://github.com/ibis-project/ibis/blob/main/docs/CONTRIBUTING.md).
- We care about keeping the community welcoming for all. Check out [the code of conduct](https://github.com/ibis-project/ibis/blob/main/CODE_OF_CONDUCT.md).
- The Ibis project is open sourced under the [Apache License](https://github.com/ibis-project/ibis/blob/main/LICENSE.txt).

Join our community by interacting on GitHub or chatting with us on [Zulip](https://ibis-project.zulipchat.com/).

For more information visit https://ibis-project.org/.


# Timesketch
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/timesketch/blob/master/notebooks/colab-timesketch-demo.ipynb)
[![Open In Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/google/timesketch/master?urlpath=%2Flab)

[![Version](https://img.shields.io/pypi/v/timesketch?label=timesketch&style=plastic)](https://pypi.python.org/pypi/timesketch)
[![Version API](https://img.shields.io/pypi/v/timesketch_api_client?label=api_client&style=plastic)](https://pypi.python.org/pypi/timesketch_api_client)
[![Version Import](https://img.shields.io/pypi/v/timesketch_import_client?label=import_client&style=plastic)](https://pypi.python.org/pypi/timesketch_import_client)

![](https://github.com/google/timesketch/workflows/timesketch-end-to-end/badge.svg)
![](https://github.com/google/timesketch/workflows/pipenv%20unittests/badge.svg)
![](https://github.com/google/timesketch/workflows/ppa%20unittests/badge.svg)

## Table of Contents
1. [About Timesketch](#about-timesketch)
2. [Getting started](#getting-started)
3. [Community](#community)
4. [Contributing](#contributing)

## About Timesketch
Timesketch is an open-source tool for collaborative forensic timeline analysis. Using sketches you and your collaborators can easily organize your timelines and analyze them all at the same time.  Add meaning to your raw data with rich annotations, comments, tags and stars.

## Getting started

#### Installation
* [Install Timesketch](docs/guides/admin/install.md)

#### Adding timelines
* [Upload data](docs/guides/user/import-from-json-csv.md)

#### Using Timesketch
* [Users guide](docs/guides/user/basic-concepts.md)

#### Adding a Notebook Container
* [Installation](docs/guides/user/notebook.md)

## Community
* [Community guide](docs/community/resources.md)

## Contributing
* [Prerequisites](CONTRIBUTING.md)
* [Developers guide](docs/developers/getting-started.md)

---

##### Obligatory Fine Print
This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.


Rally

Rally is the macrobenchmarking framework for Elasticsearch

## What is Rally?

You want to benchmark Elasticsearch? Then Rally is for you. It can help you with the following tasks:

* Setup and teardown of an Elasticsearch cluster for benchmarking
* Management of benchmark data and specifications even across Elasticsearch versions
* Running benchmarks and recording results
* Finding performance problems by attaching so-called telemetry devices
* Comparing performance results

We have also put considerable effort in Rally to ensure that benchmarking data are reproducible.

## Quick Start

Rally is developed for Unix and is actively tested on Linux and macOS. Rally supports [benchmarking Elasticsearch clusters running on Windows](http://esrally.readthedocs.io/en/stable/recipes.html#benchmarking-an-existing-cluster) but Rally itself needs to be installed on machines running Unix.

### Installing Rally

**Note**: If you actively develop on Elasticsearch, we recommend that you [install Rally in development mode](https://esrally.readthedocs.io/en/latest/developing.html#installation-instructions-for-development) instead as Elasticsearch is fast moving and Rally always adapts accordingly to the latest main version.

Install Python 3.8+ including `pip3`, git 1.9+ and an [appropriate JDK to run Elasticsearch](https://www.elastic.co/support/matrix#matrix_jvm). Be sure that `JAVA_HOME` points to that JDK. Then run the following command, optionally prefixed by `sudo` if necessary:

    pip3 install esrally

If you have any trouble or need more detailed instructions, please look in the [detailed installation guide](https://esrally.readthedocs.io/en/latest/install.html).

## Run your first race

Now we're ready to run our first race:

    esrally race --distribution-version=6.0.0 --track=geonames

This will download Elasticsearch 6.0.0 and run Rally's default track - the [geonames track](https://github.com/elastic/rally-tracks/tree/master/geonames) - against it. After the race, a summary report is written to the command line:

    ------------------------------------------------------
        _______             __   _____
       / ____(_)___  ____ _/ /  / ___/_________  ________
      / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
     / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
    /_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
    ------------------------------------------------------

    |                         Metric |                 Task |     Value |   Unit |
    |-------------------------------:|---------------------:|----------:|-------:|
    |            Total indexing time |                      |   28.0997 |    min |
    |               Total merge time |                      |   6.84378 |    min |
    |             Total refresh time |                      |   3.06045 |    min |
    |               Total flush time |                      |  0.106517 |    min |
    |      Total merge throttle time |                      |   1.28193 |    min |
    |               Median CPU usage |                      |     471.6 |      % |
    |             Total Young Gen GC |                      |    16.237 |      s |
    |               Total Old Gen GC |                      |     1.796 |      s |
    |                     Index size |                      |   2.60124 |     GB |
    |                  Total written |                      |   11.8144 |     GB |
    |         Heap used for segments |                      |   14.7326 |     MB |
    |       Heap used for doc values |                      |  0.115917 |     MB |
    |            Heap used for terms |                      |   13.3203 |     MB |
    |            Heap used for norms |                      | 0.0734253 |     MB |
    |           Heap used for points |                      |    0.5793 |     MB |
    |    Heap used for stored fields |                      |  0.643608 |     MB |
    |                  Segment count |                      |        97 |        |
    |                 Min Throughput |         index-append |   31925.2 | docs/s |
    |              Median Throughput |         index-append |   39137.5 | docs/s |
    |                 Max Throughput |         index-append |   39633.6 | docs/s |
    |      50.0th percentile latency |         index-append |   872.513 |     ms |
    |      90.0th percentile latency |         index-append |   1457.13 |     ms |
    |      99.0th percentile latency |         index-append |   1874.89 |     ms |
    |       100th percentile latency |         index-append |   2711.71 |     ms |
    | 50.0th percentile service time |         index-append |   872.513 |     ms |
    | 90.0th percentile service time |         index-append |   1457.13 |     ms |
    | 99.0th percentile service time |         index-append |   1874.89 |     ms |
    |  100th percentile service time |         index-append |   2711.71 |     ms |
    |                           ...  |                  ... |       ... |    ... |
    |                           ...  |                  ... |       ... |    ... |
    |                 Min Throughput |     painless_dynamic |   2.53292 |  ops/s |
    |              Median Throughput |     painless_dynamic |   2.53813 |  ops/s |
    |                 Max Throughput |     painless_dynamic |   2.54401 |  ops/s |
    |      50.0th percentile latency |     painless_dynamic |    172208 |     ms |
    |      90.0th percentile latency |     painless_dynamic |    310401 |     ms |
    |      99.0th percentile latency |     painless_dynamic |    341341 |     ms |
    |      99.9th percentile latency |     painless_dynamic |    344404 |     ms |
    |       100th percentile latency |     painless_dynamic |    344754 |     ms |
    | 50.0th percentile service time |     painless_dynamic |    393.02 |     ms |
    | 90.0th percentile service time |     painless_dynamic |   407.579 |     ms |
    | 99.0th percentile service time |     painless_dynamic |   430.806 |     ms |
    | 99.9th percentile service time |     painless_dynamic |   457.352 |     ms |
    |  100th percentile service time |     painless_dynamic |   459.474 |     ms |

    ----------------------------------
    [INFO] SUCCESS (took 2634 seconds)
    ----------------------------------

## Getting help

* Quick help: `esrally --help`
* Look in [Rally's user guide](https://esrally.readthedocs.io/) for more information
* Ask questions about Rally in the [Rally Discuss forum](https://discuss.elastic.co/tags/c/elastic-stack/elasticsearch/rally).
* File improvements or bug reports in our [Github repo](https://github.com/elastic/rally/issues).

## How to Contribute

See all details in the [contributor guidelines](https://github.com/elastic/rally/blob/master/CONTRIBUTING.md).

## License

This software is licensed under the Apache License, version 2 ("ALv2"), quoted below.

Copyright 2015-2021 Elasticsearch <https://www.elastic.co>

Licensed under the Apache License, Version 2.0 (the "License"); you may not
use this file except in compliance with the License. You may obtain a copy of
the License at

> http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.


django-simple-history |pypi-version|
====================================

.. Start of PyPI readme

|jazzband| |build-status| |docs| |coverage| |maintainability| |code-style| |downloads|

.. |pypi-version| image:: https://img.shields.io/pypi/v/django-simple-history.svg
   :target: https://pypi.org/project/django-simple-history/
   :alt: PyPI Version

.. |jazzband| image:: https://jazzband.co/static/img/badge.svg
   :target: https://jazzband.co/
   :alt: Jazzband

.. |build-status| image:: https://github.com/jazzband/django-simple-history/actions/workflows/test.yml/badge.svg
   :target: https://github.com/jazzband/django-simple-history/actions/workflows/test.yml
   :alt: Build Status

.. |docs| image:: https://readthedocs.org/projects/django-simple-history/badge/?version=latest
   :target: https://django-simple-history.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status

.. |coverage| image:: https://img.shields.io/codecov/c/github/jazzband/django-simple-history/master.svg
   :target: https://app.codecov.io/github/jazzband/django-simple-history?branch=master
   :alt: Test Coverage

.. |maintainability| image:: https://api.codeclimate.com/v1/badges/66cfd94e2db991f2d28a/maintainability
   :target: https://codeclimate.com/github/jazzband/django-simple-history/maintainability
   :alt: Maintainability

.. |code-style| image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/psf/black
   :alt: Code Style

.. |downloads| image:: https://static.pepy.tech/badge/django-simple-history
   :target: https://pepy.tech/project/django-simple-history
   :alt: Downloads

``django-simple-history`` stores Django model state on every create/update/delete.

This app supports the following combinations of Django and Python:

==========  ========================
  Django      Python
==========  ========================
4.2         3.8, 3.9, 3.10, 3.11, 3.12, 3.13-dev
5.0         3.10, 3.11, 3.12, 3.13-dev
main        3.10, 3.11, 3.12, 3.13-dev
==========  ========================

Getting Help
------------

Documentation is available at https://django-simple-history.readthedocs.io/en/stable/

Pull requests are welcome. Read the `CONTRIBUTING`_ file for tips on
submitting a pull request.

.. _CONTRIBUTING: https://github.com/jazzband/django-simple-history/blob/master/CONTRIBUTING.rst

License
-------

This project is licensed under the
`BSD 3-Clause license <https://choosealicense.com/licenses/bsd-3-clause/>`_.


# TensorFlow Model Analysis

[![Python](https://img.shields.io/badge/python%20-3.9%7C3.10%7C3.11-blue)](https://github.com/tensorflow/model-analysis)
[![PyPI](https://badge.fury.io/py/tensorflow-model-analysis.svg)](https://badge.fury.io/py/tensorflow-model-analysis)
[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://www.tensorflow.org/tfx/model_analysis/api_docs/python/tfma)

*TensorFlow Model Analysis* (TFMA) is a library for evaluating TensorFlow
models.  It allows users to evaluate their models on large amounts of data in a
distributed manner, using the same metrics defined in their trainer. These
metrics can be computed over different slices of data and visualized in Jupyter
notebooks.

![TFMA Slicing Metrics Browser](https://raw.githubusercontent.com/tensorflow/model-analysis/master/g3doc/images/tfma-slicing-metrics-browser.gif)

Caution: TFMA may introduce backwards incompatible changes before version 1.0.

## Installation

The recommended way to install TFMA is using the
[PyPI package](https://pypi.org/project/tensorflow-model-analysis/):

pip install tensorflow-model-analysis

pip install from https://pypi-nightly.tensorflow.org

pip install -i https://pypi-nightly.tensorflow.org/simple tensorflow-model-analysis

pip install from the HEAD of the git:

pip install git+https://github.com/tensorflow/model-analysis.git#egg=tensorflow_model_analysis

pip install from a released version directly from git:

pip install git+https://github.com/tensorflow/model-analysis.git@v0.21.3#egg=tensorflow_model_analysis

If you have cloned the repository locally, and want to test your local change,
pip install from a local folder.

pip install -e $FOLDER_OF_THE_LOCAL_LOCATION

Note that protobuf must be installed correctly for the above option since it is
building TFMA from source and it requires protoc and all of its includes
reference-able. Please see [protobuf install instruction](https://github.com/protocolbuffers/protobuf#protocol-compiler-installation)
for see the latest install instructions.

Currently, TFMA requires that TensorFlow is installed but does not have an
explicit dependency on the TensorFlow PyPI package. See the
[TensorFlow install guides](https://www.tensorflow.org/install/) for
instructions.

### Build TFMA from source

To build from source follow the following steps:

Install the protoc as per the link mentioned:
[protoc](https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os)

Create a virtual environment by running the commands

```
python3 -m venv <virtualenv_name>
source <virtualenv_name>/bin/activate
pip3 install setuptools wheel
git clone https://github.com/tensorflow/model-analysis.git
cd model-analysis
python3 setup.py bdist_wheel
```
This will build the TFMA wheel in the dist directory. To install the wheel from
dist directory run the commands

```
cd dist
pip3 install tensorflow_model_analysis-<version>-py3-none-any.whl
```

### Jupyter Lab

As of writing, because of https://github.com/pypa/pip/issues/9187, `pip install`
might never finish. In that case, you should revert pip to version 19 instead of
20: `pip install "pip<20"`.

Using a JupyterLab extension requires installing dependencies on the command
line. You can do this within the console in the JupyterLab UI or on the command
line. This includes separately installing any pip package dependencies and
JupyterLab labextension plugin dependencies, and the version numbers must be
compatible.  JupyterLab labextension packages refer to npm packages
(eg, [tensorflow_model_analysis](https://www.npmjs.com/package/tensorflow_model_analysis).

The examples below use 0.32.0. Check available [versions](#compatible-versions)
below to use the latest.

#### Jupyter Lab 3.0.x

```Shell
pip install tensorflow_model_analysis==0.32.0
jupyter labextension install tensorflow_model_analysis@0.32.0
pip install jupyterlab_widgets==1.0.0
```

#### Jupyter Lab 2.2.x

```Shell
pip install tensorflow_model_analysis==0.32.0
jupyter labextension install tensorflow_model_analysis@0.32.0
jupyter labextension install @jupyter-widgets/jupyterlab-manager@2
```

#### Jupyter Lab 1.2.x

```Shell
pip install tensorflow_model_analysis==0.32.0
jupyter labextension install tensorflow_model_analysis@0.32.0
jupyter labextension install @jupyter-widgets/jupyterlab-manager@1.1
```

#### Classic Jupyter Notebook

To enable TFMA visualization in the classic Jupyter Notebook (either through
`jupyter notebook` or
[through the JupyterLab UI](https://jupyterlab.readthedocs.io/en/stable/getting_started/starting.html)),
you'll also need to run:

```shell
jupyter nbextension enable --py widgetsnbextension
jupyter nbextension enable --py tensorflow_model_analysis
```

Note: If Jupyter notebook is already installed in your home directory, add
`--user` to these commands. If Jupyter is installed as root, or using a virtual
environment, the parameter `--sys-prefix` might be required.

#### Building TFMA from source

If you want to build TFMA from source and use the UI in JupyterLab, you'll need
to make sure that the source contains valid version numbers.  Check that the
Python package version number and npm package version number are exactly the
same, and that both valid version numbers (eg, remove the `-dev` suffix).

#### Troubleshooting

Check pip packages:

```Shell
pip list
```

Check JupyterLab extensions:

```Shell
jupyter labextension list  # for JupyterLab
jupyter nbextension list  # for classic Jupyter Notebook
```

### Standalone HTML page with `embed_minimal_html`

TFMA notebook extension can be built into a standalone HTML file that also
bundles data into the HTML file.  See the Jupyter Widgets docs on
[embed_minimal_html](https://ipywidgets.readthedocs.io/en/latest/embedding.html#python-interface).

### Kubeflow Pipelines

[Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/sdk/output-viewer/)
includes integrations that embed the TFMA notebook extension ([code](https://github.com/kubeflow/pipelines/blob/1.5.0-rc.2/backend/src/apiserver/visualization/types/tfma.py#L17)).
This integration relies on network access at runtime to load a variant of the
JavaScript build published on unpkg.com (see [config](https://github.com/tensorflow/model-analysis/blob/v0.29.0/tensorflow_model_analysis/notebook/jupyter/js/webpack.config.js#L78)
and [loader code](https://github.com/tensorflow/model-analysis/blob/v0.29.0/tensorflow_model_analysis/notebook/jupyter/js/lib/widget.js#L23)).

### Notable Dependencies

TensorFlow is required.

[Apache Beam](https://beam.apache.org/) is required; it's the way that efficient
distributed computation is supported. By default, Apache Beam runs in local
mode but can also run in distributed mode using
[Google Cloud Dataflow](https://cloud.google.com/dataflow/) and other Apache
Beam
[runners](https://beam.apache.org/documentation/runners/capability-matrix/).

[Apache Arrow](https://arrow.apache.org/) is also required. TFMA uses Arrow to
represent data internally in order to make use of vectorized numpy functions.

## Getting Started

For instructions on using TFMA, see the [get started
guide](https://github.com/tensorflow/model-analysis/blob/master/g3doc/get_started.md).

## Compatible Versions

The following table is the TFMA package versions that are compatible with each
other. This is determined by our testing framework, but other *untested*
combinations may also work.

|tensorflow-model-analysis                                                            |apache-beam[gcp]|pyarrow   |tensorflow         |tensorflow-metadata |tfx-bsl   |
|------------------------------------------------------------------------------------ |----------------|----------|-------------------|--------------------|----------|
|[GitHub master](https://github.com/tensorflow/model-analysis/blob/master/RELEASE.md) | 2.47.0         | 10.0.0   | nightly (2.x)     | 1.15.0             | 1.15.1   |
|[0.46.0](https://github.com/tensorflow/model-analysis/blob/v0.46.0/RELEASE.md)       | 2.47.0         | 10.0.0   | 2.15              | 1.15.0             | 1.15.1   |
|[0.45.0](https://github.com/tensorflow/model-analysis/blob/v0.45.0/RELEASE.md)       | 2.47.0         | 10.0.0   | 2.13              | 1.14.0             | 1.14.0   |
|[0.44.0](https://github.com/tensorflow/model-analysis/blob/v0.44.0/RELEASE.md)       | 2.40.0         | 6.0.0    | 2.12              | 1.13.1             | 1.13.0   |
|[0.43.0](https://github.com/tensorflow/model-analysis/blob/v0.43.0/RELEASE.md)       | 2.40.0         | 6.0.0    | 2.11              | 1.12.0             | 1.12.0   |
|[0.42.0](https://github.com/tensorflow/model-analysis/blob/v0.42.0/RELEASE.md)       | 2.40.0         | 6.0.0    | 1.15.5 / 2.10     | 1.11.0             | 1.11.1   |
|[0.41.0](https://github.com/tensorflow/model-analysis/blob/v0.41.0/RELEASE.md)       | 2.40.0         | 6.0.0    | 1.15.5 / 2.9      | 1.10.0             | 1.10.1   |
|[0.40.0](https://github.com/tensorflow/model-analysis/blob/v0.40.0/RELEASE.md)       | 2.38.0         | 5.0.0    | 1.15.5 / 2.9      | 1.9.0              | 1.9.0    |
|[0.39.0](https://github.com/tensorflow/model-analysis/blob/v0.39.0/RELEASE.md)       | 2.38.0         | 5.0.0    | 1.15.5 / 2.8      | 1.8.0              | 1.8.0    |
|[0.38.0](https://github.com/tensorflow/model-analysis/blob/v0.38.0/RELEASE.md)       | 2.36.0         | 5.0.0    | 1.15.5 / 2.8      | 1.7.0              | 1.7.0    |
|[0.37.0](https://github.com/tensorflow/model-analysis/blob/v0.37.0/RELEASE.md)       | 2.35.0         | 5.0.0    | 1.15.5 / 2.7      | 1.6.0              | 1.6.0    |
|[0.36.0](https://github.com/tensorflow/model-analysis/blob/v0.36.0/RELEASE.md)       | 2.34.0         | 5.0.0    | 1.15.5 / 2.7      | 1.5.0              | 1.5.0    |
|[0.35.0](https://github.com/tensorflow/model-analysis/blob/v0.35.0/RELEASE.md)       | 2.33.0         | 5.0.0    | 1.15 / 2.6        | 1.4.0              | 1.4.0    |
|[0.34.1](https://github.com/tensorflow/model-analysis/blob/v0.34.1/RELEASE.md)       | 2.32.0         | 2.0.0    | 1.15 / 2.6        | 1.2.0              | 1.3.0    |
|[0.34.0](https://github.com/tensorflow/model-analysis/blob/v0.34.0/RELEASE.md)       | 2.31.0         | 2.0.0    | 1.15 / 2.6        | 1.2.0              | 1.3.1    |
|[0.33.0](https://github.com/tensorflow/model-analysis/blob/v0.33.0/RELEASE.md)       | 2.31.0         | 2.0.0    | 1.15 / 2.5        | 1.2.0              | 1.2.0    |
|[0.32.1](https://github.com/tensorflow/model-analysis/blob/v0.32.1/RELEASE.md)       | 2.29.0         | 2.0.0    | 1.15 / 2.5        | 1.1.0              | 1.1.1    |
|[0.32.0](https://github.com/tensorflow/model-analysis/blob/v0.32.0/RELEASE.md)       | 2.29.0         | 2.0.0    | 1.15 / 2.5        | 1.1.0              | 1.1.0    |
|[0.31.0](https://github.com/tensorflow/model-analysis/blob/v0.31.0/RELEASE.md)       | 2.29.0         | 2.0.0    | 1.15 / 2.5        | 1.0.0              | 1.0.0    |
|[0.30.0](https://github.com/tensorflow/model-analysis/blob/v0.30.0/RELEASE.md)       | 2.28.0         | 2.0.0    | 1.15 / 2.4        | 0.30.0             | 0.30.0   |
|[0.29.0](https://github.com/tensorflow/model-analysis/blob/v0.29.0/RELEASE.md)       | 2.28.0         | 2.0.0    | 1.15 / 2.4        | 0.29.0             | 0.29.0   |
|[0.28.0](https://github.com/tensorflow/model-analysis/blob/v0.28.0/RELEASE.md)       | 2.28.0         | 2.0.0    | 1.15 / 2.4        | 0.28.0             | 0.28.0   |
|[0.27.0](https://github.com/tensorflow/model-analysis/blob/v0.27.0/RELEASE.md)       | 2.27.0         | 2.0.0    | 1.15 / 2.4        | 0.27.0             | 0.27.0   |
|[0.26.1](https://github.com/tensorflow/model-analysis/blob/v0.26.1/RELEASE.md)       | 2.28.0         | 0.17.0   | 1.15 / 2.3        | 0.26.0             | 0.26.0   |
|[0.26.0](https://github.com/tensorflow/model-analysis/blob/v0.26.0/RELEASE.md)       | 2.25.0         | 0.17.0   | 1.15 / 2.3        | 0.26.0             | 0.26.0   |
|[0.25.0](https://github.com/tensorflow/model-analysis/blob/v0.25.0/RELEASE.md)       | 2.25.0         | 0.17.0   | 1.15 / 2.3        | 0.25.0             | 0.25.0   |
|[0.24.3](https://github.com/tensorflow/model-analysis/blob/v0.24.3/RELEASE.md)       | 2.24.0         | 0.17.0   | 1.15 / 2.3        | 0.24.0             | 0.24.1   |
|[0.24.2](https://github.com/tensorflow/model-analysis/blob/v0.24.2/RELEASE.md)       | 2.23.0         | 0.17.0   | 1.15 / 2.3        | 0.24.0             | 0.24.0   |
|[0.24.1](https://github.com/tensorflow/model-analysis/blob/v0.24.1/RELEASE.md)       | 2.23.0         | 0.17.0   | 1.15 / 2.3        | 0.24.0             | 0.24.0   |
|[0.24.0](https://github.com/tensorflow/model-analysis/blob/v0.24.0/RELEASE.md)       | 2.23.0         | 0.17.0   | 1.15 / 2.3        | 0.24.0             | 0.24.0   |
|[0.23.0](https://github.com/tensorflow/model-analysis/blob/v0.23.0/RELEASE.md)       | 2.23.0         | 0.17.0   | 1.15 / 2.3        | 0.23.0             | 0.23.0   |
|[0.22.2](https://github.com/tensorflow/model-analysis/blob/v0.22.2/RELEASE.md)       | 2.20.0         | 0.16.0   | 1.15 / 2.2        | 0.22.2             | 0.22.0   |
|[0.22.1](https://github.com/tensorflow/model-analysis/blob/v0.22.1/RELEASE.md)       | 2.20.0         | 0.16.0   | 1.15 / 2.2        | 0.22.2             | 0.22.0   |
|[0.22.0](https://github.com/tensorflow/model-analysis/blob/v0.22.0/RELEASE.md)       | 2.20.0         | 0.16.0   | 1.15 / 2.2        | 0.22.0             | 0.22.0   |
|[0.21.6](https://github.com/tensorflow/model-analysis/blob/v0.21.6/RELEASE.md)       | 2.19.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.3   |
|[0.21.5](https://github.com/tensorflow/model-analysis/blob/v0.21.5/RELEASE.md)       | 2.19.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.3   |
|[0.21.4](https://github.com/tensorflow/model-analysis/blob/v0.21.4/RELEASE.md)       | 2.19.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.3   |
|[0.21.3](https://github.com/tensorflow/model-analysis/blob/v0.21.3/RELEASE.md)       | 2.17.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.0   |
|[0.21.2](https://github.com/tensorflow/model-analysis/blob/v0.21.2/RELEASE.md)       | 2.17.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.0   |
|[0.21.1](https://github.com/tensorflow/model-analysis/blob/v0.21.1/RELEASE.md)       | 2.17.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.0   |
|[0.21.0](https://github.com/tensorflow/model-analysis/blob/v0.21.0/RELEASE.md)       | 2.17.0         | 0.15.0   | 1.15 / 2.1        | 0.21.0             | 0.21.0   |
|[0.15.4](https://github.com/tensorflow/model-analysis/blob/v0.15.4/RELEASE.md)       | 2.16.0         | 0.15.0   | 1.15 / 2.0        | n/a                | 0.15.1   |
|[0.15.3](https://github.com/tensorflow/model-analysis/blob/v0.15.3/RELEASE.md)       | 2.16.0         | 0.15.0   | 1.15 / 2.0        | n/a                | 0.15.1   |
|[0.15.2](https://github.com/tensorflow/model-analysis/blob/v0.15.2/RELEASE.md)       | 2.16.0         | 0.15.0   | 1.15 / 2.0        | n/a                | 0.15.1   |
|[0.15.1](https://github.com/tensorflow/model-analysis/blob/v0.15.1/RELEASE.md)       | 2.16.0         | 0.15.0   | 1.15 / 2.0        | n/a                | 0.15.0   |
|[0.15.0](https://github.com/tensorflow/model-analysis/blob/v0.15.0/RELEASE.md)       | 2.16.0         | 0.15.0   | 1.15              | n/a                | n/a      |
|[0.14.0](https://github.com/tensorflow/model-analysis/blob/v0.14.0/RELEASE.md)       | 2.14.0         | n/a      | 1.14              | n/a                | n/a      |
|[0.13.1](https://github.com/tensorflow/model-analysis/blob/v0.13.1/RELEASE.md)       | 2.11.0         | n/a      | 1.13              | n/a                | n/a      |
|[0.13.0](https://github.com/tensorflow/model-analysis/blob/v0.13.0/RELEASE.md)       | 2.11.0         | n/a      | 1.13              | n/a                | n/a      |
|[0.12.1](https://github.com/tensorflow/model-analysis/blob/v0.12.1/RELEASE.md)       | 2.10.0         | n/a      | 1.12              | n/a                | n/a      |
|[0.12.0](https://github.com/tensorflow/model-analysis/blob/v0.12.0/RELEASE.md)       | 2.10.0         | n/a      | 1.12              | n/a                | n/a      |
|[0.11.0](https://github.com/tensorflow/model-analysis/blob/v0.11.0/RELEASE.md)       | 2.8.0          | n/a      | 1.11              | n/a                | n/a      |
|[0.9.2](https://github.com/tensorflow/model-analysis/blob/v0.9.2/RELEASE.md)         | 2.6.0          | n/a      | 1.9               | n/a                | n/a      |
|[0.9.1](https://github.com/tensorflow/model-analysis/blob/v0.9.1/RELEASE.md)         | 2.6.0          | n/a      | 1.10              | n/a                | n/a      |
|[0.9.0](https://github.com/tensorflow/model-analysis/blob/v0.9.0/RELEASE.md)         | 2.5.0          | n/a      | 1.9               | n/a                | n/a      |
|[0.6.0](https://github.com/tensorflow/model-analysis/blob/v0.6.0/RELEASE.md)         | 2.4.0          | n/a      | 1.6               | n/a                | n/a      |

## Questions

Please direct any questions about working with TFMA to
[Stack Overflow](https://stackoverflow.com) using the
[tensorflow-model-analysis](https://stackoverflow.com/questions/tagged/tensorflow-model-analysis)
tag.


==================================
JSON:API and Django REST framework
==================================

.. image:: https://github.com/django-json-api/django-rest-framework-json-api/workflows/Tests/badge.svg
   :alt: Tests
   :target: https://github.com/django-json-api/django-rest-framework-json-api/actions

.. image:: https://readthedocs.org/projects/django-rest-framework-json-api/badge/?version=latest
   :alt: Read the docs
   :target: https://django-rest-framework-json-api.readthedocs.org/

.. image:: https://img.shields.io/pypi/v/djangorestframework-jsonapi.svg
   :alt: PyPi Version
   :target: https://pypi.org/project/djangorestframework-jsonapi/

--------
Overview
--------

**JSON:API support for Django REST framework**

* Documentation: https://django-rest-framework-json-api.readthedocs.org/
* Format specification: https://jsonapi.org/format/

By default, Django REST framework will produce a response like:

.. code:: JSON

    {
        "count": 20,
        "next": "https://example.com/api/1.0/identities/?page=3",
        "previous": "https://example.com/api/1.0/identities/?page=1",
        "results": [{
            "id": 3,
            "username": "john",
            "full_name": "John Coltrane"
        }]
    }

However, for an ``identity`` model in JSON:API format the response should look
like the following:

.. code:: JSON

    {
        "links": {
            "prev": "https://example.com/api/1.0/identities",
            "self": "https://example.com/api/1.0/identities?page=2",
            "next": "https://example.com/api/1.0/identities?page=3",
        },
        "data": [{
            "type": "identities",
            "id": "3",
            "attributes": {
                "username": "john",
                "full-name": "John Coltrane"
            }
        }],
        "meta": {
            "pagination": {
              "count": 20
            }
        }
    }

-----
Goals
-----

As a Django REST framework JSON:API (short DJA) we are trying to address following goals:

1. Support the `JSON:API`_ spec to compliance

2. Be as compatible with `Django REST framework`_ as possible

   e.g. issues in Django REST framework should be fixed upstream and not worked around in DJA

3. Have sane defaults to be as easy to pick up as possible

4. Be solid and tested with good coverage

5. Be performant

.. _JSON:API: https://jsonapi.org
.. _Django REST framework: https://www.django-rest-framework.org/

------------
Requirements
------------

1. Python (3.8, 3.9, 3.10, 3.11, 3.12)
2. Django (4.2, 5.0)
3. Django REST framework (3.14, 3.15)

We **highly** recommend and only officially support the latest patch release of each Python, Django and REST framework series.

Generally Python and Django series are supported till the official end of life. For Django REST framework the last two series are supported.

For optional dependencies such as Django Filter only the latest release is officially supported even though lower versions should work as well.

------------
Installation
------------

Install using ``pip``...

.. code:: sh

    $ pip install djangorestframework-jsonapi
    $ # for optional package integrations
    $ pip install djangorestframework-jsonapi['django-filter']
    $ pip install djangorestframework-jsonapi['django-polymorphic']
    $ pip install djangorestframework-jsonapi['openapi']

or from source...

.. code:: sh

    $ git clone https://github.com/django-json-api/django-rest-framework-json-api.git
    $ cd django-rest-framework-json-api
    $ pip install -e .

and add ``rest_framework_json_api`` to your ``INSTALLED_APPS`` setting below ``rest_framework``.

.. code:: python

    INSTALLED_APPS = [
        ...
        'rest_framework',
        'rest_framework_json_api',
        ...
    ]

Running the example app
^^^^^^^^^^^^^^^^^^^^^^^

It is recommended to create a virtualenv for testing. Assuming it is already
installed and activated:

.. code:: sh

    $ git clone https://github.com/django-json-api/django-rest-framework-json-api.git
    $ cd django-rest-framework-json-api
    $ pip install -Ur requirements.txt
    $ django-admin migrate --settings=example.settings
    $ django-admin loaddata drf_example --settings=example.settings
    $ django-admin runserver --settings=example.settings

Browse to

* http://localhost:8000 for the list of available collections (in a non-JSON:API format!),
* http://localhost:8000/swagger-ui/ for a Swagger user interface to the dynamic schema view, or
* http://localhost:8000/openapi for the schema view's OpenAPI specification document.

-----
Usage
-----

``rest_framework_json_api`` assumes you are using class-based views in Django
REST framework.

Settings
^^^^^^^^

One can either add ``rest_framework_json_api.parsers.JSONParser`` and
``rest_framework_json_api.renderers.JSONRenderer`` to each ``ViewSet`` class, or
override ``settings.REST_FRAMEWORK``

.. code:: python

    REST_FRAMEWORK = {
        'PAGE_SIZE': 10,
        'EXCEPTION_HANDLER': 'rest_framework_json_api.exceptions.exception_handler',
        'DEFAULT_PAGINATION_CLASS':
            'rest_framework_json_api.pagination.JsonApiPageNumberPagination',
        'DEFAULT_PARSER_CLASSES': (
            'rest_framework_json_api.parsers.JSONParser',
            'rest_framework.parsers.FormParser',
            'rest_framework.parsers.MultiPartParser'
        ),
        'DEFAULT_RENDERER_CLASSES': (
            'rest_framework_json_api.renderers.JSONRenderer',
            'rest_framework_json_api.renderers.BrowsableAPIRenderer',
        ),
        'DEFAULT_METADATA_CLASS': 'rest_framework_json_api.metadata.JSONAPIMetadata',
        'DEFAULT_FILTER_BACKENDS': (
            'rest_framework_json_api.filters.QueryParameterValidationFilter',
            'rest_framework_json_api.filters.OrderingFilter',
            'rest_framework_json_api.django_filters.DjangoFilterBackend',
            'rest_framework.filters.SearchFilter',
        ),
        'SEARCH_PARAM': 'filter[search]',
        'TEST_REQUEST_RENDERER_CLASSES': (
            'rest_framework_json_api.renderers.JSONRenderer',
        ),
        'TEST_REQUEST_DEFAULT_FORMAT': 'vnd.api+json'
    }

This package provides much more including automatic inflection of JSON keys, extra top level data (using nested
serializers), relationships, links, paginators, filters, and handy shortcuts.
Read more at https://django-rest-framework-json-api.readthedocs.org/


Exchange Web Services client library
====================================

This module is an ORM for your Exchange mailbox, providing Django-style access to all your data. It is a
platform-independent, well-performing, well-behaving, well-documented, well-tested and simple interface for
communicating with an on-premise Microsoft Exchange 2007-2016 server or Office365 using Exchange Web Services
(EWS). Among other things, it implements autodiscover, and functions for searching, creating, updating, deleting,
exporting and uploading calendar, mailbox, task, contact and distribution list items.

[![image](https://img.shields.io/pypi/v/exchangelib.svg)](https://pypi.org/project/exchangelib/)
[![image](https://img.shields.io/pypi/pyversions/exchangelib.svg)](https://pypi.org/project/exchangelib/)
[![image](https://api.codacy.com/project/badge/Grade/5f805ad901054a889f4b99a82d6c1cb7)](https://app.codacy.com/gh/ecederstrand/exchangelib)
[![image](https://coveralls.io/repos/github/ecederstrand/exchangelib/badge.svg?branch=master)](https://coveralls.io/github/ecederstrand/exchangelib?branch=master)
[![xscode](https://img.shields.io/badge/Available%20on-xs%3Acode-blue)](https://xscode.com/ecederstrand/exchangelib)

## Teaser

Here's a short example of how `exchangelib` works. Let's print the first
100 inbox messages in reverse order:

```python
from exchangelib import Credentials, Account

credentials = Credentials("john@example.com", "topsecret")
account = Account("john@example.com", credentials=credentials, autodiscover=True)

for item in account.inbox.all().order_by("-datetime_received")[:100]:
    print(item.subject, item.sender, item.datetime_received)
```

## Documentation
Documentation is available at [https://ecederstrand.github.io/exchangelib/](https://ecederstrand.github.io/exchangelib/).
Source code documentation is available at [https://ecederstrand.github.io/exchangelib/exchangelib/](https://ecederstrand.github.io/exchangelib/exchangelib/).


.. image:: https://badges.gitter.im/bd2k-genomics-toil/Lobby.svg
   :alt: Join the chat at https://gitter.im/bd2k-genomics-toil/Lobby
   :target: https://gitter.im/bd2k-genomics-toil/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge

Toil is a scalable, efficient, cross-platform (Linux & macOS) pipeline management system,
written entirely in Python, and designed around the principles of functional
programming. It supports running workflows written in either Common Workflow Language (`CWL`_) 1.0-1.2 or
Workflow Description Language (`WDL`_) 1.0-1.1, as well as having its own rich Python API for writing workflows against.
It supports running workflows locally on your system (e.g. a laptop), on an HPC cluster, or in the cloud.

* Check the `website`_ for a description of Toil and its features.
* Full documentation for the latest stable release can be found at
  `Read the Docs`_.
* Please subscribe to low-volume `announce`_ mailing list so we keep you informed
* Google Groups discussion `forum`_
* See our occasional `blog`_ for tutorials.
* Use `biostars`_ channel for discussion.

.. _website: http://toil.ucsc-cgl.org/
.. _Read the Docs: https://toil.readthedocs.io/en/latest
.. _announce: https://groups.google.com/forum/#!forum/toil-announce
.. _forum: https://groups.google.com/forum/#!forum/toil-community
.. _blog: https://toilpipelines.wordpress.com/
.. _biostars: https://www.biostars.org/t/toil/
.. _CWL: https://www.commonwl.org/
.. _WDL: https://openwdl.org/

Notes:

* Toil moved from https://github.com/BD2KGenomics/toil to https://github.com/DataBiosphere/toil on July 5th, 2018.
* Toil dropped Python 2.7 support on February 13, 2020 (the last working py2.7 version is 3.24.0).


    All-in-one embeddings database

txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows.

![architecture](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/architecture.png#gh-light-mode-only)
![architecture](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/architecture-dark.png#gh-dark-mode-only)

Embeddings databases are a union of vector indexes (sparse and dense), graph networks and relational databases. This enables vector search with SQL, topic modeling, retrieval augmented generation and more.

Embeddings databases can stand on their own and/or serve as a powerful knowledge source for large language model (LLM) prompts.

Summary of txtai features:

- 🔎 Vector search with SQL, object storage, topic modeling, graph analysis and multimodal indexing
- 📄 Create embeddings for text, documents, audio, images and video
- 💡 Pipelines powered by language models that run LLM prompts, question-answering, labeling, transcription, translation, summarization and more
- ↪ Workflows to join pipelines together and aggregate business logic. txtai processes can be simple microservices or multi-model workflows.
- ⚙ Build with Python or YAML. API bindings available for [JavaScript](https://github.com/neuml/txtai.js), [Java](https://github.com/neuml/txtai.java), [Rust](https://github.com/neuml/txtai.rs) and [Go](https://github.com/neuml/txtai.go).
- ☁ Run local or scale out with container orchestration

txtai is built with Python 3.8+, [Hugging Face Transformers](https://github.com/huggingface/transformers), [Sentence Transformers](https://github.com/UKPLab/sentence-transformers) and [FastAPI](https://github.com/tiangolo/fastapi). txtai is open-source under an Apache 2.0 license.

*Interested in an easy and secure way to run hosted txtai applications? Then join the [txtai.cloud](https://txtai.cloud) preview to learn more.*

## Why txtai?

![why](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/why.png#gh-light-mode-only)
![why](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/why-dark.png#gh-dark-mode-only)

New vector databases, LLM frameworks and everything in between are sprouting up daily. Why build with txtai?

- Up and running in minutes with [pip](https://neuml.github.io/txtai/install/) or [Docker](https://neuml.github.io/txtai/cloud/)
```python
# Get started in a couple lines
import txtai

embeddings = txtai.Embeddings()
embeddings.index(["Correct", "Not what we hoped"])
embeddings.search("positive", 1)
#[(0, 0.29862046241760254)]
```
- Built-in API makes it easy to develop applications using your programming language of choice
```yaml
# app.yml
embeddings:
    path: sentence-transformers/all-MiniLM-L6-v2
```
```bash
CONFIG=app.yml uvicorn "txtai.api:app"
curl -X GET "http://localhost:8000/search?query=positive"
```
- Run local - no need to ship data off to disparate remote services
- Work with micromodels all the way up to large language models (LLMs)
- Low footprint - install additional dependencies and scale up when needed
- [Learn by example](https://neuml.github.io/txtai/examples) - notebooks cover all available functionality

## Use Cases

The following sections introduce common txtai use cases. A comprehensive set of over 50 [example notebooks and applications](https://neuml.github.io/txtai/examples) are also available.

### Semantic Search

Build semantic/similarity/vector/neural search applications.

![demo](https://raw.githubusercontent.com/neuml/txtai/master/demo.gif)

Traditional search systems use keywords to find data. Semantic search has an understanding of natural language and identifies results that have the same meaning, not necessarily the same keywords.

![search](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/search.png#gh-light-mode-only)
![search](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/search-dark.png#gh-dark-mode-only)

Get started with the following examples.

| Notebook  | Description  |       |
|:----------|:-------------|------:|
| [Introducing txtai](https://github.com/neuml/txtai/blob/master/examples/01_Introducing_txtai.ipynb) [▶](https://www.youtube.com/watch?v=SIezMnVdmMs) | Overview of the functionality provided by txtai | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/01_Introducing_txtai.ipynb) |
| [Similarity search with images](https://github.com/neuml/txtai/blob/master/examples/13_Similarity_search_with_images.ipynb) | Embed images and text into the same space for search | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/13_Similarity_search_with_images.ipynb) |
| [Build a QA database](https://github.com/neuml/txtai/blob/master/examples/34_Build_a_QA_database.ipynb) | Question matching with semantic search | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/34_Build_a_QA_database.ipynb) |
| [Semantic Graphs](https://github.com/neuml/txtai/blob/master/examples/38_Introducing_the_Semantic_Graph.ipynb) | Explore topics, data connectivity and run network analysis| [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/38_Introducing_the_Semantic_Graph.ipynb) |

### LLM Orchestration

LLM chains, retrieval augmented generation (RAG), chat with your data, pipelines and workflows that interface with large language models (LLMs).

#### Chains

Integrate LLM chains (known as workflows in txtai), multiple LLM agents and self-critique.

![llm](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/llm.png)

See below to learn more.

| Notebook  | Description  |       |
|:----------|:-------------|------:|
| [Prompt templates and task chains](https://github.com/neuml/txtai/blob/master/examples/44_Prompt_templates_and_task_chains.ipynb) | Build model prompts and connect tasks together with workflows | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/44_Prompt_templates_and_task_chains.ipynb) |
| [Integrate LLM frameworks](https://github.com/neuml/txtai/blob/master/examples/53_Integrate_LLM_Frameworks.ipynb) | Integrate llama.cpp, LiteLLM and custom generation frameworks | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/53_Integrate_LLM_Frameworks.ipynb) |
| [Build knowledge graphs with LLMs](https://github.com/neuml/txtai/blob/master/examples/57_Build_knowledge_graphs_with_LLM_driven_entity_extraction.ipynb) | Build knowledge graphs with LLM-driven entity extraction | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/57_Build_knowledge_graphs_with_LLM_driven_entity_extraction.ipynb) |

#### Retrieval augmented generation

Retrieval augmented generation (RAG) reduces the risk of LLM hallucinations by constraining the output with a knowledge base as context. RAG is commonly used to "chat with your data".

![rag](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/rag.png#gh-light-mode-only)
![rag](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/rag-dark.png#gh-dark-mode-only)

A novel feature of txtai is that it can provide both an answer and source citation.

| Notebook  | Description  |       |
|:----------|:-------------|------:|
| [Build RAG pipelines with txtai](https://github.com/neuml/txtai/blob/master/examples/52_Build_RAG_pipelines_with_txtai.ipynb) | Guide on retrieval augmented generation including how to create citations | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/52_Build_RAG_pipelines_with_txtai.ipynb) |
| [How RAG with txtai works](https://github.com/neuml/txtai/blob/master/examples/63_How_RAG_with_txtai_works.ipynb) | Create RAG processes, API services and Docker instances | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/63_How_RAG_with_txtai_works.ipynb) |
| [Advanced RAG with graph path traversal](https://github.com/neuml/txtai/blob/master/examples/58_Advanced_RAG_with_graph_path_traversal.ipynb) | Graph path traversal to collect complex sets of data for advanced RAG | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/58_Advanced_RAG_with_graph_path_traversal.ipynb) |
| [Advanced RAG with guided generation](https://github.com/neuml/txtai/blob/master/examples/60_Advanced_RAG_with_guided_generation.ipynb) | Retrieval Augmented and Guided Generation | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/60_Advanced_RAG_with_guided_generation.ipynb) |

### Language Model Workflows

Language model workflows, also known as semantic workflows, connect language models together to build intelligent applications.

![flows](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/flows.png#gh-light-mode-only)
![flows](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/flows-dark.png#gh-dark-mode-only)

While LLMs are powerful, there are plenty of smaller, more specialized models that work better and faster for specific tasks. This includes models for extractive question-answering, automatic summarization, text-to-speech, transcription and translation.

| Notebook  | Description  |       |
|:----------|:-------------|------:|
| [Run pipeline workflows](https://github.com/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) [▶](https://www.youtube.com/watch?v=UBMPDCn1gEU) | Simple yet powerful constructs to efficiently process data | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/14_Run_pipeline_workflows.ipynb) |
| [Building abstractive text summaries](https://github.com/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) | Run abstractive text summarization | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) |
| [Transcribe audio to text](https://github.com/neuml/txtai/blob/master/examples/11_Transcribe_audio_to_text.ipynb) | Convert audio files to text | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/11_Transcribe_audio_to_text.ipynb) |
| [Translate text between languages](https://github.com/neuml/txtai/blob/master/examples/12_Translate_text_between_languages.ipynb) | Streamline machine translation and language detection | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/12_Translate_text_between_languages.ipynb) |

## Installation

![install](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/install.png#gh-light-mode-only)
![install](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/install-dark.png#gh-dark-mode-only)

The easiest way to install is via pip and PyPI

```
pip install txtai
```

Python 3.8+ is supported. Using a Python [virtual environment](https://docs.python.org/3/library/venv.html) is recommended.

See the detailed [install instructions](https://neuml.github.io/txtai/install) for more information covering [optional dependencies](https://neuml.github.io/txtai/install/#optional-dependencies), [environment specific prerequisites](https://neuml.github.io/txtai/install/#environment-specific-prerequisites), [installing from source](https://neuml.github.io/txtai/install/#install-from-source), [conda support](https://neuml.github.io/txtai/install/#conda) and how to [run with containers](https://neuml.github.io/txtai/cloud).

## Model guide

![models](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/models.png)

See the table below for the current recommended models. These models all allow commercial use and offer a blend of speed and performance.

| Component                                                                     | Model(s)                                                                 |
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| [Embeddings](https://neuml.github.io/txtai/embeddings)                        | [all-MiniLM-L6-v2](https://hf.co/sentence-transformers/all-MiniLM-L6-v2) |
| [Image Captions](https://neuml.github.io/txtai/pipeline/image/caption)        | [BLIP](https://hf.co/Salesforce/blip-image-captioning-base)              |
| [Labels - Zero Shot](https://neuml.github.io/txtai/pipeline/text/labels)      | [BART-Large-MNLI](https://hf.co/facebook/bart-large)                     |
| [Labels - Fixed](https://neuml.github.io/txtai/pipeline/text/labels)          | Fine-tune with [training pipeline](https://neuml.github.io/txtai/pipeline/train/trainer)          |
| [Large Language Model (LLM)](https://neuml.github.io/txtai/pipeline/text/llm) | [Mistral 7B OpenOrca](https://hf.co/Open-Orca/Mistral-7B-OpenOrca)       |
| [Summarization](https://neuml.github.io/txtai/pipeline/text/summary)          | [DistilBART](https://hf.co/sshleifer/distilbart-cnn-12-6)                |
| [Text-to-Speech](https://neuml.github.io/txtai/pipeline/audio/texttospeech)   | [ESPnet JETS](https://hf.co/NeuML/ljspeech-jets-onnx)                    |
| [Transcription](https://neuml.github.io/txtai/pipeline/audio/transcription)   | [Whisper](https://hf.co/openai/whisper-base)                             |
| [Translation](https://neuml.github.io/txtai/pipeline/text/translation)        | [OPUS Model Series](https://hf.co/Helsinki-NLP)                          |

Models can be loaded as either a path from the Hugging Face Hub or a local directory. Model paths are optional, defaults are loaded when not specified. For tasks with no recommended model, txtai uses the default models as shown in the Hugging Face Tasks guide.

See the following links to learn more.

- [Hugging Face Tasks](https://hf.co/tasks)
- [Hugging Face Model Hub](https://hf.co/models)
- [MTEB Leaderboard](https://hf.co/spaces/mteb/leaderboard)
- [LMSYS LLM Leaderboard](https://chat.lmsys.org/?leaderboard)
- [Open LLM Leaderboard](https://hf.co/spaces/HuggingFaceH4/open_llm_leaderboard)

## Powered by txtai

The following applications are powered by txtai.

![apps](https://raw.githubusercontent.com/neuml/txtai/master/apps.jpg)

| Application  | Description  |
|:------------ |:-------------|
| [txtchat](https://github.com/neuml/txtchat) | Retrieval Augmented Generation (RAG) powered search |
| [paperai](https://github.com/neuml/paperai) | Semantic search and workflows for medical/scientific papers |
| [codequestion](https://github.com/neuml/codequestion) | Semantic search for developers |
| [tldrstory](https://github.com/neuml/tldrstory) | Semantic search for headlines and story text |

In addition to this list, there are also many other [open-source projects](https://github.com/neuml/txtai/network/dependents), [published research](https://scholar.google.com/scholar?q=txtai&hl=en&as_ylo=2022) and closed proprietary/commercial projects that have built on txtai in production.

## Further Reading

![further](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/further.png#gh-light-mode-only)
![further](https://raw.githubusercontent.com/neuml/txtai/master/docs/images/further-ghdark.png#gh-dark-mode-only)

- [Introducing txtai, the all-in-one embeddings database](https://medium.com/neuml/introducing-txtai-the-all-in-one-embeddings-database-c721f4ff91ad)
- [Tutorial series on Hashnode](https://neuml.hashnode.dev/series/txtai-tutorial) | [dev.to](https://dev.to/neuml/tutorial-series-on-txtai-ibg)
- [What's new in txtai 7.0](https://medium.com/neuml/whats-new-in-txtai-7-0-855ad6a55440) | [6.0](https://medium.com/neuml/whats-new-in-txtai-6-0-7d93eeedf804) | [5.0](https://medium.com/neuml/whats-new-in-txtai-5-0-e5c75a13b101) | [4.0](https://medium.com/neuml/whats-new-in-txtai-4-0-bbc3a65c3d1c)
- [Getting started with semantic search](https://medium.com/neuml/getting-started-with-semantic-search-a9fd9d8a48cf) | [rag](https://medium.com/neuml/getting-started-with-rag-9a0cca75f748) | [workflows](https://medium.com/neuml/getting-started-with-semantic-workflows-2fefda6165d9)
- [Running txtai at scale](https://medium.com/neuml/running-at-scale-with-txtai-71196cdd99f9)

## Documentation

[Full documentation on txtai](https://neuml.github.io/txtai) including configuration settings for embeddings, pipelines, workflows, API and a FAQ with common questions/issues is available.

## Contributing

For those who would like to contribute to txtai, please see [this guide](https://github.com/neuml/.github/blob/master/CONTRIBUTING.md).