[![PyPI Status](https://badge.fury.io/py/lhotse.svg)](https://badge.fury.io/py/lhotse)
[![Python Versions](https://img.shields.io/pypi/pyversions/lhotse.svg)](https://pypi.org/project/lhotse/)
[![PyPI Status](https://pepy.tech/badge/lhotse)](https://pepy.tech/project/lhotse)
[![Build Status](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fpzelasko%2Flhotse%2Fbadge%3Fref%3Dmaster&style=flat)](https://actions-badge.atrox.dev/pzelasko/lhotse/goto?ref=master)
[![Documentation Status](https://readthedocs.org/projects/lhotse/badge/?version=latest)](https://lhotse.readthedocs.io/en/latest/?badge=latest)
[![codecov](https://codecov.io/gh/lhotse-speech/lhotse/branch/master/graph/badge.svg)](https://codecov.io/gh/lhotse-speech/lhotse)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse-speech.github.io/blob/master/notebooks/lhotse-introduction.ipynb)
[![arXiv](https://img.shields.io/badge/arXiv-2110.12561-b31b1b.svg)](https://arxiv.org/abs/2110.12561)

# Lhotse

Lhotse is a Python library aiming to make speech and audio data preparation flexible and accessible to a wider community. Alongside [k2](https://github.com/k2-fsa/k2), it is a part of the next generation [Kaldi](https://github.com/kaldi-asr/kaldi) speech processing library.

## Tutorial presentations and materials

- (_Interspeech 2023_) Tutorial notebook [![Interspeech 2023 Tutorial](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1obZjUuVwks3A4oFX3gXFtPOM2LtrPQfL?usp=sharing)
- (_Interspeech 2023_) [Tutorial slides](https://livejohnshopkins-my.sharepoint.com/:p:/g/personal/mwiesne2_jh_edu/EYqRDl8cIr5BsVDxi1MOW5EBUpdqh10WFkzqixPIFM63hg?e=u3lrmL)
- (_Interspeech 2021_) [Recorded lecture (3h)](https://www.youtube.com/watch?v=y6CJLFQlmhc&pp=ygUgaW50ZXJzcGVlY2ggMjAyMSBsaG90c2UgdHV0b3JpYWw%3D)

## About

### Main goals

- Attract a wider community to speech processing tasks with a **Python-centric design**.
- Accommodate experienced Kaldi users with an **expressive command-line interface**.
- Provide **standard data preparation recipes** for commonly used corpora.
- Provide **PyTorch Dataset classes** for speech and audio related tasks.
- Flexible data preparation for model training with the notion of **audio cuts**.
- **Efficiency**, especially in terms of I/O bandwidth and storage capacity.

### Tutorials

We currently have the following tutorials available in `examples` directory:
- Basic complete Lhotse workflow [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/00-basic-workflow.ipynb)
- Transforming data with Cuts [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/01-cut-python-api.ipynb)
- WebDataset integration [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/02-webdataset-integration.ipynb)
- How to combine multiple datasets [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/03-combining-datasets.ipynb)
- Lhotse Shar: storage format optimized for sequential I/O and modularity [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/04-lhotse-shar.ipynb)

### Examples of use

Check out the following links to see how Lhotse is being put to use:
- [Icefall recipes](https://github.com/k2-fsa/icefall): where k2 and Lhotse meet.
- Minimal ESPnet+Lhotse example: [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1HKSYPsWx_HoCdrnLpaPdYj5zwlPsM3NH)

### Main ideas

Like Kaldi, Lhotse provides standard data preparation recipes, but extends that with a seamless PyTorch integration
through task-specific Dataset classes. The data and meta-data are represented in human-readable text manifests and
exposed to the user through convenient Python classes.

![image](https://raw.githubusercontent.com/lhotse-speech/lhotse/master/docs/lhotse-concept-graph.png)

Lhotse introduces the notion of audio cuts, designed to ease the training data construction with operations such as
mixing, truncation and padding that are performed on-the-fly to minimize the amount of storage required. Data
augmentation and feature extraction are supported both in pre-computed mode, with highly-compressed feature matrices
stored on disk, and on-the-fly mode that computes the transformations upon request. Additionally, Lhotse introduces
feature-space cut mixing to make the best of both worlds.

![image](https://raw.githubusercontent.com/lhotse-speech/lhotse/master/docs/lhotse-cut-illustration.png)

## Installation

Lhotse supports Python version 3.7 and later.

### Pip

Lhotse is available on PyPI:

    pip install lhotse

To install the latest, unreleased version, do:

    pip install git+https://github.com/lhotse-speech/lhotse

### Development installation

For development installation, you can fork/clone the GitHub repo and install with pip:

    git clone https://github.com/lhotse-speech/lhotse
    cd lhotse
    pip install -e '.[dev]'
    pre-commit install  # installs pre-commit hooks with style checks

    # Running unit tests
    pytest test

    # Running linter checks
    pre-commit run

This is an editable installation (`-e` option), meaning that your changes to the source code are automatically
reflected when importing lhotse (no re-install needed). The `[dev]` part means you're installing extra dependencies
that are used to run tests, build documentation or launch jupyter notebooks.

### Environment variables

Lhotse uses several environment variables to customize it's behavior. They are as follows:
- `LHOTSE_REQUIRE_TORCHAUDIO` - when it's set and not any of `1|True|true|yes`, we'll not check for torchaudio being installed and remove it from the requirements. It will disable many functionalities of Lhotse but the basic capabilities will remain (including reading audio with `soundfile`).
- `LHOTSE_AUDIO_DURATION_MISMATCH_TOLERANCE` - used when we load audio from a file and receive a different number of samples than declared in `Recording.num_samples`. This is sometimes necessary because different codecs (or even different versions of the same codec) may use different padding when decoding compressed audio. Typically values up to 0.1, or even 0.3 (second) are still reasonable, and anything beyond that indicates a serious issue.
- `LHOTSE_AUDIO_BACKEND` - may be set to any of the values returned from CLI `lhotse list-audio-backends` to override the default behavior of trial-and-error and always use a specific audio backend.
- `LHOTSE_AUDIO_LOADING_EXCEPTION_VERBOSE` - when set to `1` we'll emit full exception stack traces when every available audio backend fails to load a given file (they might be very large).
- `LHOTSE_DILL_ENABLED` - when it's set to `1|True|true|yes`, we will enable `dill`-based serialization of `CutSet` and `Sampler` across processes (it's disabled by default even when `dill` is installed).
- `LHOTSE_LEGACY_OPUS_LOADING` - (`=1`) reverts to a legacy OPUS loading mechanism that triggered a new ffmpeg subprocess for each OPUS file.
- `LHOTSE_PREPARING_RELEASE` - used internally by developers when releasing a new version of Lhotse.
- `TORCHAUDIO_USE_BACKEND_DISPATCHER` - when set to `1` and torchaudio version is below 2.1, we'll enable the experimental ffmpeg backend of torchaudio.
- `AIS_ENDPOINT` is read by AIStore client to determine AIStore endpoint URL. Required for AIStore dataloading.
- `RANK`, `WORLD_SIZE`, `WORKER`, and `NUM_WORKERS` are internally used to inform Lhotse Shar dataloading subprocesses.
- `READTHEDOCS` is internally used for documentation builds.

### Optional dependencies

**Other pip packages.** You can leverage optional features of Lhotse by installing the relevant supporting package like this: `pip install lhotse[package_name]`. The supported optional packages include:
- `pip install lhotse[kaldi]` for a maximal feature set related to Kaldi compatibility. It includes libraries such as `kaldi_native_io` (a more efficient variant of `kaldi_io`) and `kaldifeat` that port some of Kaldi functionality into Python.
- `pip install lhotse[orjson]` for up to 50% faster reading of JSONL manifests.
- `pip install lhotse[webdataset]`. We support "compiling" your data into WebDataset tarball format for more effective IO. You can still interact with the data as if it was a regular lazy CutSet. To learn more, check out the following tutorial: [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lhotse-speech/lhotse/blob/master/examples/02-webdataset-integration.ipynb)
- `pip install h5py` if you want to extract speech features and store them as HDF5 arrays.
- `pip install dill`. When `dill` is installed, we'll use it to pickle CutSet that uses a lambda function in calls such as `.map` or `.filter`. This is helpful in PyTorch DataLoader with `num_jobs>0`. Without `dill`, depending on your environment, you'll see an exception or a hanging script.
- `pip install aistore` to read manifests, tar fles, and other data from AIStore using AIStore-supported URLs (set `AIS_ENDPOINT` environment variable to activate it). See [AIStore documentation](https://aiatscale.org) for more details.
- `pip install smart_open` to read and write manifests and data in any location supported by `smart_open` (e.g. cloud, http).
- `pip install opensmile` for feature extraction using the OpenSmile toolkit's Python wrapper.

**sph2pipe.** For reading older LDC SPHERE (.sph) audio files that are compressed with codecs unsupported by ffmpeg and sox, please run:

    # CLI
    lhotse install-sph2pipe

    # Python
    from lhotse.tools import install_sph2pipe
    install_sph2pipe()

It will download it to `~/.lhotse/tools`, compile it, and auto-register in `PATH`. The program should be automatically detected and used by Lhotse.

## Examples

We have example recipes showing how to prepare data and load it in Python as a PyTorch `Dataset`.
They are located in the `examples` directory.

A short snippet to show how Lhotse can make audio data preparation quick and easy:

```python
from torch.utils.data import DataLoader
from lhotse import CutSet, Fbank
from lhotse.dataset import VadDataset, SimpleCutSampler
from lhotse.recipes import prepare_switchboard

# Prepare data manifests from a raw corpus distribution.
# The RecordingSet describes the metadata about audio recordings;
# the sampling rate, number of channels, duration, etc.
# The SupervisionSet describes metadata about supervision segments:
# the transcript, speaker, language, and so on.
swbd = prepare_switchboard('/export/corpora3/LDC/LDC97S62')

# CutSet is the workhorse of Lhotse, allowing for flexible data manipulation.
# We create 5-second cuts by traversing SWBD recordings in windows.
# No audio data is actually loaded into memory or stored to disk at this point.
cuts = CutSet.from_manifests(
    recordings=swbd['recordings'],
    supervisions=swbd['supervisions']
).cut_into_windows(duration=5)

# We compute the log-Mel filter energies and store them on disk;
# Then, we pad the cuts to 5 seconds to ensure all cuts are of equal length,
# as the last window in each recording might have a shorter duration.
# The padding will be performed once the features are loaded into memory.
cuts = cuts.compute_and_store_features(
    extractor=Fbank(),
    storage_path='feats',
    num_jobs=8
).pad(duration=5.0)

# Construct a Pytorch Dataset class for Voice Activity Detection task:
dataset = VadDataset()
sampler = SimpleCutSampler(cuts, max_duration=300)
dataloader = DataLoader(dataset, sampler=sampler, batch_size=None)
batch = next(iter(dataloader))
```

The `VadDataset` will yield a batch with pairs of feature and supervision tensors such as the following - the speech
starts roughly at the first second (100 frames):

![image](https://raw.githubusercontent.com/lhotse-speech/lhotse/master/docs/vad_sample.png)


Introduction
============

.. image:: https://readthedocs.org/projects/adafruit-micropython-blinka/badge/?version=latest
    :target: https://circuitpython.readthedocs.io/projects/blinka/en/latest/
    :alt: Documentation Status

.. image:: https://img.shields.io/discord/327254708534116352.svg
    :target: https://adafru.it/discord
    :alt: Discord

.. image:: https://travis-ci.com/adafruit/Adafruit_Blinka.svg?branch=master
    :target: https://travis-ci.com/adafruit/Adafruit_Blinka
    :alt: Build Status

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
    :target: https://github.com/psf/black
    :alt: Code Style: Black

This repository contains a selection of packages emulating the CircuitPython API
for devices or hosts running CPython or MicroPython. Working code exists to emulate these CircuitPython packages:

* **analogio** - analog input/output pins, using pin identities from board+microcontroller packages
* **bitbangio** - software-driven interfaces for I2C, SPI
* **board** - breakout-specific pin identities
* **busio** - hardware-driven interfaces for I2C, SPI, UART
* **digitalio** - digital input/output pins, using pin identities from board+microcontroller packages
* **keypad** - support for scanning keys and key matrices
* **microcontroller** - chip-specific pin identities
* **micropython** - MicroPython-specific module
* **neopixel_write** - low-level interface to NeoPixels
* **pulseio** - contains classes that provide access to basic pulse IO (PWM)
* **pwmio** - contains classes that provide access to basic pulse IO (PWM)
* **rainbowio** - provides the colorwheel() function
* **usb_hid** - act as a hid-device using usb_gadget kernel driver

For details, see the `Blinka API reference
<https://circuitpython.readthedocs.io/projects/blinka/en/latest/index.html>`_.

Dependencies
=============

The emulation described above is intended to provide a
CircuitPython-like API for devices which are running CPython or
Micropython. Since corresponding packages should be built-in to any
standard CircuitPython image, they have no value on a device already
running CircuitPython and would likely conflict in unhappy ways.

The test suites in the test/src folder under **testing.universal** are by design
intended to run on *either* CircuitPython *or* CPython/Micropython+compatibility layer to prove conformance.

Installing from PyPI
=====================

On supported GNU/Linux systems like the Raspberry Pi, you can install the driver locally `from
PyPI <https://pypi.org/project/Adafruit-Blinka/>`_. To install for current user:

.. code-block:: shell

    pip3 install Adafruit-Blinka

To install system-wide (this may be required in some cases):

.. code-block:: shell

    sudo pip3 install Adafruit-Blinka

To install in a virtual environment in your current project:

.. code-block:: shell

    mkdir project-name && cd project-name
    python3 -m venv .env
    source .env/bin/activate
    pip3 install Adafruit-Blinka

Usage Example
=============

The pin names may vary by board, so you may need to change the pin names in the code. This
example runs on the Raspberry Pi boards to blink an LED connected to GPIO 18 (Pin 12):

.. code-block:: python

    import time
    import board
    import digitalio

    PIN = board.D18

    print("hello blinky!")

    led = digitalio.DigitalInOut(PIN)
    led.direction = digitalio.Direction.OUTPUT

    while True:
        led.value = True
        time.sleep(0.5)
        led.value = False
        time.sleep(0.5)

Contributing
============

Contributions are welcome! Please read our `Code of Conduct
<https://github.com/adafruit/Adafruit_Blinka/blob/master/CODE_OF_CONDUCT.md>`_
before contributing to help this project stay welcoming.

Building locally
================

Sphinx documentation
-----------------------

Sphinx is used to build the documentation based on rST files and comments in the code. First,
install dependencies (feel free to reuse the virtual environment from above):

.. code-block:: shell

    python3 -m venv .env
    source .env/bin/activate
    pip install Sphinx sphinx-rtd-theme Adafruit-PlatformDetect

Now, once you have the virtual environment activated:

.. code-block:: shell

    cd docs
    sphinx-build -E -W -b html . _build/html

This will output the documentation to ``docs/_build/html``. Open the index.html in your browser to
view them. It will also (due to -W) error out on any warning like Travis will. This is a good way to
locally verify it will pass.


pysteps - Python framework for short-term ensemble prediction systems
=====================================================================

.. start-badges

.. list-table::
    :stub-columns: 1
    :widths: 10 90

    * - docs
      - |stable| |colab| |gallery|
    * - status
      - |test| |docs| |codecov| |codacy| |black|
    * - package
      - |github| |conda| |pypi| |zenodo|
    * - community
      - |contributors| |downloads| |license|

.. |docs| image:: https://readthedocs.org/projects/pysteps/badge/?version=latest
    :alt: Documentation Status
    :target: https://pysteps.readthedocs.io/

.. |test| image:: https://github.com/pySTEPS/pysteps/workflows/Test%20pysteps/badge.svg
    :alt: Test pysteps
    :target: https://github.com/pySTEPS/pysteps/actions?query=workflow%3A"Test+Pysteps"

.. |black| image:: https://github.com/pySTEPS/pysteps/workflows/Check%20Black/badge.svg
    :alt: Check Black
    :target: https://github.com/pySTEPS/pysteps/actions?query=workflow%3A"Check+Black"

.. |codecov| image:: https://codecov.io/gh/pySTEPS/pysteps/branch/master/graph/badge.svg
    :alt: Coverage
    :target: https://codecov.io/gh/pySTEPS/pysteps

.. |github| image:: https://img.shields.io/github/release/pySTEPS/pysteps.svg
    :target: https://github.com/pySTEPS/pysteps/releases/latest
    :alt: Latest github release

.. |conda| image:: https://anaconda.org/conda-forge/pysteps/badges/version.svg
    :target: https://anaconda.org/conda-forge/pysteps
    :alt: Anaconda Cloud

.. |pypi| image:: https://badge.fury.io/py/pysteps.svg
    :target: https://pypi.org/project/pysteps/
    :alt: Latest PyPI version

.. |license| image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg
    :alt: License
    :target: https://opensource.org/licenses/BSD-3-Clause

.. |contributors| image:: https://img.shields.io/github/contributors/pySTEPS/pysteps
    :alt: GitHub contributors
    :target: https://github.com/pySTEPS/pysteps/graphs/contributors

.. |downloads| image:: https://img.shields.io/conda/dn/conda-forge/pysteps
    :alt: Conda downloads
    :target: https://anaconda.org/conda-forge/pysteps

.. |colab| image:: https://colab.research.google.com/assets/colab-badge.svg
    :alt: My first nowcast
    :target: https://colab.research.google.com/github/pySTEPS/pysteps/blob/master/examples/my_first_nowcast.ipynb

.. |gallery| image:: https://img.shields.io/badge/example-gallery-blue.svg
    :alt: pysteps example gallery
    :target: https://pysteps.readthedocs.io/en/stable/auto_examples/index.html

.. |stable| image:: https://img.shields.io/badge/docs-stable-blue.svg
    :alt: pysteps documentation
    :target: https://pysteps.readthedocs.io/en/stable/

.. |codacy| image:: https://api.codacy.com/project/badge/Grade/6cff9e046c5341a4afebc0347362f8de
   :alt: Codacy Badge
   :target: https://app.codacy.com/gh/pySTEPS/pysteps?utm_source=github.com&utm_medium=referral&utm_content=pySTEPS/pysteps&utm_campaign=Badge_Grade

.. |zenodo| image:: https://zenodo.org/badge/140263418.svg
   :alt: DOI
   :target: https://zenodo.org/badge/latestdoi/140263418

.. end-badges

What is pysteps?
================

Pysteps is an open-source and community-driven Python library for probabilistic precipitation nowcasting, i.e. short-term ensemble prediction systems.

The aim of pysteps is to serve two different needs. The first is to provide a modular and well-documented framework for researchers interested in developing new methods for nowcasting and stochastic space-time simulation of precipitation. The second aim is to offer a highly configurable and easily accessible platform for practitioners ranging from weather forecasters to hydrologists.

The pysteps library supports standard input/output file formats and implements several optical flow methods as well as advanced stochastic generators to produce ensemble nowcasts. In addition, it includes tools for visualizing and post-processing the nowcasts and methods for deterministic, probabilistic, and neighbourhood forecast verification.

Quick start
-----------

Use pysteps to compute and plot a radar extrapolation nowcast in Google Colab with `this interactive notebook <https://colab.research.google.com/github/pySTEPS/pysteps/blob/master/examples/my_first_nowcast.ipynb>`_.

Installation
============

The recommended way to install pysteps is with `conda <https://docs.conda.io/>`_ from the conda-forge channel::

    $ conda install -c conda-forge pysteps

More details can be found in the `installation guide <https://pysteps.readthedocs.io/en/stable/user_guide/install_pysteps.html>`_.

Usage
=====

Have a look at the `gallery of examples <https://pysteps.readthedocs.io/en/stable/auto_examples/index.html>`__ to get a good overview of what pysteps can do.

For a more detailed description of all the available methods, check the  `API reference <https://pysteps.readthedocs.io/en/stable/pysteps_reference/index.html>`_ page.

Example data
============

A set of example radar data is available in a separate repository: `pysteps-data <https://github.com/pySTEPS/pysteps-data>`_.
More information on how to download and install them is available `here <https://pysteps.readthedocs.io/en/stable/user_guide/example_data.html>`_.

Contributions
=============

*We welcome contributions!*

For feedback, suggestions for developments, and bug reports please use the dedicated `issues page <https://github.com/pySTEPS/pysteps/issues>`_.

For more information, please read our `contributors guidelines <https://pysteps.readthedocs.io/en/stable/developer_guide/contributors_guidelines.html>`_.

Reference publications
======================

The overall library is described in

Pulkkinen, S., D. Nerini, A. Perez Hortal, C. Velasco-Forero, U. Germann,
A. Seed, and L. Foresti, 2019:  Pysteps:  an open-source Python library for
probabilistic precipitation nowcasting (v1.0). *Geosci. Model Dev.*, **12 (10)**,
4185–4219, doi:`10.5194/gmd-12-4185-2019 <https://doi.org/10.5194/gmd-12-4185-2019>`_.

While the more recent blending module is described in

Imhoff, R.O., L. De Cruz, W. Dewettinck, C.C. Brauer, R. Uijlenhoet, K-J. van Heeringen,
C. Velasco-Forero, D. Nerini, M. Van Ginderachter, and A.H. Weerts, 2023:
Scale-dependent blending of ensemble rainfall nowcasts and NWP in the open-source
pysteps library. *Q J R Meteorol Soc.*, 1-30,
doi: `10.1002/qj.4461 <https://doi.org/10.1002/qj.4461>`_.

Contributors
============

.. image:: https://contrib.rocks/image?repo=pySTEPS/pysteps
   :target: https://github.com/pySTEPS/pysteps/graphs/contributors


# DepthAI API Demo Program

[![Discord](https://img.shields.io/discord/790680891252932659?label=Discord)](https://discord.gg/luxonis)
[![Forum](https://img.shields.io/badge/Forum-discuss-orange)](https://discuss.luxonis.com/)
[![Docs](https://img.shields.io/badge/Docs-DepthAI-yellow)](https://docs.luxonis.com)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)

This repo contains demo application, which can load different networks, create pipelines, record video, etc.

_Click on the GIF below to see a full example run_

[![depthai demo](https://user-images.githubusercontent.com/5244214/142426845-82f5f8fd-ad1a-4873-97a5-2b3fcdb0ca2e.gif)](https://www.youtube.com/watch?v=sCZpsFQBffk)

__Documentation is available at [https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/](https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#first-steps-with-depthai).__

## Installation
First you need to clone this repository with
```
git clone --recursive https://github.com/luxonis/depthai.git
```
In case you have repository already cloned, you can update your submodules with:
```
git pull --recurse-submodules
```
There are two installation steps that need to be performed to make sure the demo works:
- **One-time installation** that will add all necessary packages to your OS.
  ```
  $ sudo curl -fL https://docs.luxonis.com/install_dependencies.sh | bash
  ```
  Please follow [this installation page](https://docs.luxonis.com/projects/api/en/latest/install/) to see instructions for other platforms

- **Python dependencies installation** that makes sure your Python interpreter has all required packages installed.
  This script is safe to be run multiple times and should be ran after every demo update
  ```
  $ python3 install_requirements.py
  ```

## Docker

One may start any DepthAI programs also through Docker:
(Allowing X11 access from container might be required: `xhost local:root`)

DepthAI Demo
```
docker run --privileged -v /dev/bus/usb:/dev/bus/usb --device-cgroup-rule='c 189:* rmw' -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --network host --rm -i -t luxonis/depthai:latest python3 /depthai/depthai_demo.py
```

Calibration
```
docker run --privileged -v /dev/bus/usb:/dev/bus/usb --device-cgroup-rule='c 189:* rmw' -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --network host --rm -i -t luxonis/depthai:latest python3 /depthai/calibrate.py [parameters]
```

## Usage

This repository and the demo script itself can be used in various independent cases:
- As a tool to try out different DepthAI features, explorable either with command line arguments (with `--guiType cv`) or in clickable QT interface (with `--guiType qt`)
- As a quick prototyping backbone, either utilising [callbacks mechanism](https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#callbacks-file) or by [extending the `Demo` class itself](https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#importing-demo-as-a-class)

### QT GUI usage

See instuctions [here](https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#default-run)

![qt demo](https://user-images.githubusercontent.com/5244214/142722740-47e545b7-c7fe-4132-9704-ae3b47d60957.png)

### Command line usage

**Examples**

`python3 depthai_demo.py -gt cv` - RGB & CNN inference example

`python3 depthai_demo.py -gt cv -vid <path_to_video_or_yt_link>` - CNN inference on video example

`python3 depthai_demo.py -gt cv -cnn person-detection-retail-0013` - Runs `person-detection-retail-0013` model from `resources/nn` directory

`python3 depthai_demo.py -gt cv -cnn tiny-yolo-v3 -sh 8` - Run `tiny-yolo-v3` model from `resources/nn` directory and compile for 8 shaves

**Demo**

![cv demo](https://user-images.githubusercontent.com/18037362/177605180-8f327513-131d-4c31-9829-3579bd717c6a.jpg)

For the full reference, run ``$ depthai_demo.py --help``.

## DepthAI Apps

We currently have 2 apps, ``uvc`` and ``record``.

### UVC app

This app will upload an UVC pipeline to the connected OAK camera, so you will be able to use an [OAK as a webcam](https://docs.luxonis.com/en/latest/pages/oak_webcam/).

### Record App

Record app lets you record encoded and synced video streams (MJPEG/H265) across all available devices into .mp4, [Foxglove's .MCAP](https://mcap.dev/), or [ROS' .bag ](http://wiki.ros.org/rosbag). Since mono streams are synced, you will be able to reconstruct the whole stereo depth perception.

Run using ``$ depthai_demo.py -app record [-p SAVE_PATH] [-q QUALITY] [--save STREAMS] [-fc] [-tl]``. More information about the arguments and replaying can be [found here](https://github.com/luxonis/depthai-experiments/tree/master/gen2-record-replay).

## Supported models

We have added support for a number of different AI models that work (decoding and visualization) out-of-the-box with the demo. You can specify which model to run with `-cnn` argument, as shown above. Currently supported models:

```
- deeplabv3p_person
- face-detection-adas-0001
- face-detection-retail-0004
- human-pose-estimation-0001
- mobilenet-ssd
- openpose2
- pedestrian-detection-adas-0002
- person-detection-retail-0013
- person-vehicle-bike-detection-crossroad-1016
- road-segmentation-adas-0001
- tiny-yolo-v3
- vehicle-detection-adas-0002
- vehicle-license-plate-detection-barrier-0106
- yolo-v3
```

If you would like to use a custom AI model, see [documentation here](https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#using-custom-models).

## Usage statistics

By default, the demo script will collect anonymous usage statistics during runtime. These include:
- Device-specific information (like mxid, connected cameras, device state and connection type)
- Environment-specific information (like OS type, python version, package versions)

We gather this data to better understand what environemnts are our users using, as well as assist better in support questions.

**All of the data we collect is anonymous and you can disable it at any time**. To do so, click on the "Misc" tab and disable sending the statistics or create a `.consent` file in repository root with the following content

```
{"statistics": false}
```

## Reporting issues

We are actively developing the DepthAI framework, and it's crucial for us to know what kind of problems you are facing.
If you run into a problem, please follow the steps below and email support@luxonis.com:

1. Run `log_system_information.sh` and share the output from (`log_system_information.txt`).
2. Take a photo of a device you are using (or provide us a device model)
3. Describe the expected results;
4. Describe the actual running results (what you see after started your script with DepthAI)
5. How you are using the DepthAI python API (code snippet, for example)
6. Console output


# Jupyter Server Proxy

[![ReadTheDocs badge](https://img.shields.io/readthedocs/jupyter-server-proxy?logo=read-the-docs)](https://jupyter-server-proxy.readthedocs.io/)
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/jupyterhub/jupyter-server-proxy/test.yaml?logo=github&branch=main)](https://github.com/jupyterhub/jupyter-server-proxy/actions)
[![PyPI badge](https://img.shields.io/pypi/v/jupyter-server-proxy.svg?logo=pypi)](https://pypi.python.org/pypi/jupyter-server-proxy)
[![Conda badge](https://img.shields.io/conda/vn/conda-forge/jupyter-server-proxy?logo=conda-forge)](https://anaconda.org/conda-forge/jupyter-server-proxy)
[![NPM badge](https://img.shields.io/npm/v/@jupyterhub/jupyter-server-proxy.svg?logo=npm)](https://www.npmjs.com/package/@jupyterhub/jupyter-server-proxy)

Jupyter Server Proxy lets you run arbitrary external processes (such as
RStudio, Shiny Server, Syncthing, PostgreSQL, Code Server, etc)
alongside your notebook server and provide authenticated web access to
them using a path like `/rstudio` next to others like `/lab`. Alongside
the python package that provides the main functionality, the JupyterLab
extension (`@jupyterhub/jupyter-server-proxy`) provides buttons in the
JupyterLab launcher window to get to RStudio for example.

**Note:** This project used to be called **nbserverproxy**. As
nbserverproxy is an older version of jupyter-server-proxy, uninstall
nbserverproxy before installing jupyter-server-proxy to avoid conflicts.

The primary use cases are:

1.  Use with JupyterHub / Binder to allow launching users into web
    interfaces that have nothing to do with Jupyter - such as RStudio,
    Shiny, or OpenRefine.
2.  Allow access from frontend javascript (in classic notebook or
    JupyterLab extensions) to access web APIs of other processes running
    locally in a safe manner. This is used by the [JupyterLab
    extension](https://github.com/dask/dask-labextension) for
    [dask](https://www.dask.org/).

[The documentation](https://jupyter-server-proxy.readthedocs.io/)
contains information on installation & usage.

## Security warning

Jupyter Server Proxy is often used to start a user defined process listening to
some network port (e.g. `http://localhost:4567`) for a user starting a Jupyter Server
that only that user has permission to access. The user can then access the
started process proxied through the Jupyter Server.

For safe use of Jupyter Server Proxy, you should ensure that the process started
by Jupyter Server proxy can't be accessed directly by another user and bypass
the Jupyter Server's authorization!

A common strategy to enforce access proxied via Jupyter Server is to start
Jupyter Server within a container and only allow network access to the Jupyter
Server via the container.

> For more insights, see [Ryan Lovett's comment about
> it](https://github.com/jupyterhub/jupyter-server-proxy/pull/359#issuecomment-1350118197).

## Install

### Python package

#### `pip`

```bash
pip install jupyter-server-proxy
```

#### `conda`

```bash
conda install -c conda-forge jupyter-server-proxy
```

### Jupyter Client Extensions

A JupyterLab and Notebook extension is bundled with the Python package to
provide:

- servers in the _New_ dropwdown of the Notebook Tree view
- launch buttons in JupyterLab's Launcher panel for registered server processes.
  - ![a screenshot of the JupyterLab Launcher](docs/source/_static/images/labextension-launcher.png "launch proxied servers as JupyterLab panels or new browser tabs")

#### Client compatibility

For historical compatibility ranges, see the table below:

| `jupyter-server-proxy` | `notebook` | `jupyterlab` |
| :--------------------: | :--------: | :----------: |
|        `4.1.x`         |  `>=6,<8`  |   `>=3,<5`   |
|        `4.0.x`         |  `>=6,<7`  |   `>=3,<4`   |
|         `3.x`          |  `>=6,<7`  |   `>=2,<4`   |

## Disable

### Server extension

```bash
jupyter serverextension disable --sys-prefix jupyter_server_proxy
jupyter server extension disable --sys-prefix jupyter_server_proxy
```

### Notebook Classic extension

```bash
jupyter nbextension disable --sys-prefix --py jupyter_server_proxy
```

### JupyterLab extension

```bash
jupyter labextension disable @jupyterhub/jupyter-server-proxy
```

#### Local development

To setup a local development environment, see the [contributing guide](https://github.com/jupyterhub/jupyter-server-proxy/blob/main/CONTRIBUTING.md).


  <img src="https://img.shields.io/badge/Documentation-FFCD00?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgLTk2MCA5NjAgOTYwIiB3aWR0aD0iMjQiPjxwYXRoIGQ9Ik0zMjAtMjQwaDMyMHYtODBIMzIwdjgwWm0wLTE2MGgzMjB2LTgwSDMyMHY4MFpNMjQwLTgwcS0zMyAwLTU2LjUtMjMuNVQxNjAtMTYwdi02NDBxMC0zMyAyMy41LTU2LjVUMjQwLTg4MGgzMjBsMjQwIDI0MHY0ODBxMCAzMy0yMy41IDU2LjVUNzIwLTgwSDI0MFptMjgwLTUyMHYtMjAwSDI0MHY2NDBoNDgwdi00NDBINTIwWk0yNDAtODAwdjIwMC0yMDAgNjQwLTY0MFoiLz48L3N2Zz4="/>
  <img src="https://img.shields.io/badge/Live_Demo-C00A35?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgLTk2MCA5NjAgOTYwIiB3aWR0aD0iMjQiPjxwYXRoIGQ9Ik0yMDAtMTIwcS0zMyAwLTU2LjUtMjMuNVQxMjAtMjAwdi01NjBxMC0zMyAyMy41LTU2LjVUMjAwLTg0MGg1NjBxMzMgMCA1Ni41IDIzLjVUODQwLTc2MHY1NjBxMCAzMy0yMy41IDU2LjVUNzYwLTEyMEg2MDB2LTgwaDE2MHYtNDgwSDIwMHY0ODBoMTYwdjgwSDIwMFptMjQwIDB2LTI0NmwtNjQgNjQtNTYtNTggMTYwLTE2MCAxNjAgMTYwLTU2IDU4LTY0LTY0djI0NmgtODBaIi8+PC9zdmc+&logoColor=white"/>

## ASReview: Active learning for Systematic Reviews

Systematically screening large amounts of textual data is time-consuming and
often tiresome. The rapidly evolving field of Artificial Intelligence (AI) has
allowed the development of AI-aided pipelines that assist in finding relevant
texts for search tasks. A well-established approach to increasing efficiency
is screening prioritization via [Active
Learning](https://asreview.readthedocs.io/en/latest/guides/activelearning.html).

The Active learning for Systematic Reviews (ASReview) project, published in
[_Nature Machine Intelligence_](https://doi.org/10.1038/s42256-020-00287-7)
implements different machine learning algorithms that interactively query the
researcher. ASReview LAB is designed to accelerate the step of screening
textual data with a minimum of records to be read by a human with no or very
few false negatives. ASReview LAB will save time, increase the quality of
output and strengthen the transparency of work when screening large amounts of
textual data to retrieve relevant information. Active Learning will support
decision-making in any discipline or industry.

ASReview software implements three different modes:

- **Oracle** Screen textual data in
  interaction with the active learning model. The reviewer is the 'oracle',
  making the labeling decisions.
- **Exploration** Explore or
  demonstrate ASReview LAB with a completely labeled dataset. This mode is
  suitable for teaching purposes.
- **Simulation** Evaluate
  the performance of active learning models on fully labeled data. Simulations
  can be run in ASReview LAB or via the command line interface with more
  advanced options.

## Installation

The ASReview software requires Python 3.8 or later. Detailed step-by-step
instructions to install Python and ASReview are available for
[Windows](https://asreview.ai/installation-guide-windows/) and
[macOS](https://asreview.ai/installation-guide-macos/) users.

```bash
pip install asreview
```

Upgrade ASReview with the following command:

```bash
pip install --upgrade asreview
```

To install ASReview LAB with Docker, see [Install with Docker](https://asreview.readthedocs.io/en/latest/installation.html).

## How it works

[![ASReview LAB explained - animation](https://img.youtube.com/vi/k-a2SCq-LtA/0.jpg)](https://www.youtube.com/watch?v=k-a2SCq-LtA)

## Getting started

[Getting Started with ASReview
LAB](https://asreview.readthedocs.io/en/latest/about.html).

[![ASReview LAB](https://github.com/asreview/asreview/blob/main/images/ASReviewWebApp.png?raw=true)](https://asreview.readthedocs.io/en/latest/lab/overview_lab.html "ASReview LAB")

## Citation

If you wish to cite the underlying methodology of the ASReview software,
please use the following publication in Nature Machine Intelligence:

> van de Schoot, R., de Bruin, J., Schram, R. et al. An open source machine
> learning framework for efficient and transparent systematic reviews.
> Nat Mach Intell 3, 125–133 (2021). https://doi.org/10.1038/s42256-020-00287-7

For citing the software, please refer to the specific release of
the ASReview software on Zenodo https://doi.org/10.5281/zenodo.3345592. The menu on the
right can be used to find the citation format of prevalence.

For more scientific publications on the ASReview software, go to
[asreview.ai/papers](https://asreview.ai/papers/).

## Contact

For an overview of the team working on ASReview, see [ASReview Research Team](https://asreview.ai/about).
ASReview LAB is maintained by
[Jonathan de Bruin](https://github.com/J535D165) and [Yongchao Terry Ma](https://github.com/terrymyc).

The best resources to find an answer to your question or ways to get in
contact with the team are:

- Documentation - [asreview.readthedocs.io](https://asreview.readthedocs.io/)
- Newsletter - [asreview.ai/newsletter/subscribe](https://asreview.ai/newsletter/subscribe)
- Quick tour - [ASReview LAB quick tour](https://asreview.readthedocs.io/en/latest/lab/overview_lab.html)
- Issues or feature requests - [ASReview issue tracker](https://github.com/asreview/asreview/issues)
- FAQ - [ASReview Discussions](https://github.com/asreview/asreview/discussions?discussions_q=sort%3Atop)
- Donation - [asreview.ai/donate](https://asreview.ai/donate)
- Contact - [asreview@uu.nl](mailto:asreview@uu.nl)

[![PyPI version](https://badge.fury.io/py/asreview.svg)](https://badge.fury.io/py/asreview) [![DOI](https://zenodo.org/badge/164874894.svg)](https://zenodo.org/badge/latestdoi/164874894)
[![Downloads](https://static.pepy.tech/badge/asreview)](https://github.com/asreview/asreview#installation) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4755/badge)](https://bestpractices.coreinfrastructure.org/projects/4755)

## License

The ASReview software has an Apache 2.0 [LICENSE](LICENSE). The ASReview team
accepts no responsibility or liability for the use of the ASReview tool or any
direct or indirect damages arising out of the application of the tool.


# CDlib - Community Detection Library
[![codecov](https://codecov.io/gh/GiulioRossetti/cdlib/branch/master/graph/badge.svg?token=3YJOEVK02B)](https://codecov.io/gh/GiulioRossetti/cdlib)
[![Build](https://github.com/GiulioRossetti/cdlib/actions/workflows/python-package.yml/badge.svg)](https://github.com/GiulioRossetti/cdlib/actions/workflows/python-package.yml)
[![Documentation Status](https://readthedocs.org/projects/cdlib/badge/?version=latest)](http://cdlib.readthedocs.io/en/latest/?badge=latest)
[![CodeQL](https://github.com/GiulioRossetti/cdlib/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/GiulioRossetti/cdlib/actions/workflows/codeql-analysis.yml)
[![pyversions](https://img.shields.io/pypi/pyversions/cdlib.svg)](https://badge.fury.io/py/cdlib)
[![PyPI version](https://badge.fury.io/py/cdlib.svg)](https://badge.fury.io/py/cdlib)
[![Anaconda-Server Badge](https://anaconda.org/giuliorossetti/cdlib/badges/version.svg)](https://anaconda.org/giuliorossetti/cdlib)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Downloads](https://pepy.tech/badge/cdlib/month)](https://pepy.tech/project/cdlib)
[![Downloads](https://pepy.tech/badge/cdlib)](https://pepy.tech/project/cdlib)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4575156.svg)](https://doi.org/10.5281/zenodo.4575156)
[![SBD++](https://img.shields.io/badge/Available%20on-SoBigData%2B%2B-green)](https://sobigdata.d4science.org/group/sobigdata-gateway/explore?siteId=20371853)

[![Twitter](https://img.shields.io/twitter/url/https/twitter.com/_CDlib_.svg?style=social&label=Follow%20%40_CDlib_)](https://twitter.com/_CDlib_)

``CDlib`` is a meta-library for community detection in complex networks: it implements algorithms, clustering fitness functions as well as visualization facilities.

``CDlib`` is designed around the ``networkx`` python library: however, when needed, it takes care to automatically convert (from and to) ``igraph`` object so to provide an abstraction on specific algorithm implementations to the final user.

``CDlib`` provides a standardized input/output facilities for several Community Discovery algorithms: whenever possible, to guarantee literature coherent results, implementations of CD algorithms are inherited from their original projects (acknowledged on the [documentation](https://cdlib.readthedocs.io)).

If you use ``CDlib`` as support to your research consider citing:

> G. Rossetti, L. Milli, R. Cazabet.
> **CDlib: a Python Library to Extract, Compare and Evaluate Communities from Complex Networks.**
> Applied Network Science Journal. 2019.
> [DOI:10.1007/s41109-019-0165-9]()

## Tutorial and Online Environments

Check out the official [tutorial](https://colab.research.google.com/github/GiulioRossetti/cdlib/blob/master/docs/CDlib.ipynb) to get started!

If you would like to test ``CDlib`` functionalities without installing anything on your machine consider using the preconfigured Jupyter Hub instances offered by [SoBigData++](https://sobigdata.d4science.org/group/sobigdata-gateway/explore?siteId=20371853).

## Installation

``CDlib`` *requires* python>=3.8.

To install the latest version of our library just download (or clone) the current project, open a terminal and run the following commands:

```bash
pip install -r requirements.txt
pip install -r requirements_optional.txt # (Optional) this might not work in Windows systems due to C-based dependencies.
pip install .
```

Alternatively use pip
```bash
pip install cdlib
```

or conda
```bash
conda create -n cdlib python=3.9
conda config --add channels giuliorossetti
conda config --add channels conda-forge
conda install cdlib
```

### Optional Dependencies (pip package)
To simplify the installation process, the default installation does not include optional dependencies (e.g., ``graph-tool``). If you need them, you can install them manually or run the following command:

```bash
pip install 'cdlib[C]'
```

This option, safe for *nix users, will install all those optional dependencies that require C code compilation.

```bash
pip install 'cdlib[pypi]'
```

This option will install all those optional dependencies that are not available on conda/conda-forge.

```bash
pip install 'cdlib[all]'
```

This option will install all optional dependencies accessible with the flag ``C`` and ``pypi``.

#### (Advanced)

Due to some strict requirements, the installation of a subset of optional dependencies is left outside the previous procedures.

##### graph-tool
``CDlib`` integrates the support for SBM models offered by ``graph-tool``.
To install it refer to the official [documentation](https://git.skewed.de/count0/graph-tool/wikis/installation-instructions) and install the conda-forge version of the package (or the deb version if in a *nix system).

##### ASLPAw

Since its 2.1.0 release ``ASLPAw`` relies on ``gmpy2`` whose installation through pip is not easy to automatize due to some C dependencies.
To address such issue test the following recipe:

```bash
conda install gmpy2
pip install shuffle_graph>=2.1.0 similarity-index-of-label-graph>=2.0.1 ASLPAw>=2.1.0
```

In case this does not solve the issue, please refer to the official ``gmpy2`` [installation](https://gmpy2.readthedocs.io/en/latest/intro.html#installation) instructions.

### Optional Dependencies (Conda package)

``CDlib`` relies on a few packages not available through conda: to install them please use pip.

```bash
pip install pycombo
pip install GraphRicciCurvature

conda install gmpy2
pip install shuffle_graph>=2.1.0 similarity-index-of-label-graph>=2.0.1 ASLPAw>=2.1.0
```

In case ASLPAw installation fails, please refer to the official ``gmpy2`` [installation](https://gmpy2.readthedocs.io/en/latest/intro.html#installation) instructions.

## Collaborate with us!

``CDlib`` is an active project, any contribution is welcome!

If you like to include your model in CDlib feel free to fork the project, open an issue and contact us.

### How to contribute to this project?

Contributing is good, doing it correctly is better! Check out our [rules](https://github.com/GiulioRossetti/cdlib/blob/master/.github/CONTRIBUTING.md), issue a proper [pull request](https://github.com/GiulioRossetti/cdlib/blob/master/.github/PULL_REQUEST_TEMPLATE.md) /[bug report](https://github.com/GiulioRossetti/cdlib/blob/master/.github/ISSUE_TEMPLATE/bug_report.md) / [feature request](https://github.com/GiulioRossetti/cdlib/blob/master/.github/ISSUE_TEMPLATE/feature_request.md).

We are a welcoming community... just follow the [Code of Conduct](https://github.com/GiulioRossetti/cdlib/blob/master/.github/CODE_OF_CONDUCT.md).


Synapse
=======

Synapse is a versatile central intelligence system created to support analyst teams in every stage of the intelligence life cycle.

|codecov|_ |circleci|_ |rtd|_

Learn More About Synapse
------------------------

* Visit our website_ to learn more about Synapse.
* Watch `Synapse 101`_!

Installation & Documentation
----------------------------

* Synapse is available as a Python Package on pypi_ and as a Docker image on dockerhub_.

  * The Getting Started guide_ will help get you up and running!

* You can also use synapse_quickstart_ to get up and running quickly on Windows, Mac, or Linux and start using Synapse!
* User Guides and API Documentation can be found here_.

Connect With Us
---------------

* Join the Synapse user community on Slack_
* Follow us on Twitter_.
* Follow us on LinkedIn_.

.. |circleci| image:: https://circleci.com/gh/vertexproject/synapse/tree/master.svg?style=svg
..  _circleci: https://v.vtx.lk/synapse-circleci

.. |codecov| image:: https://codecov.io/gh/vertexproject/synapse/branch/master/graph/badge.svg?branch=master
.. _codecov: https://v.vtx.lk/synapse-codecov

.. |rtd| image:: https://readthedocs.com/projects/vertex-synapse/badge/?version=latest
.. _rtd: https://v.vtx.lk/synapse-docs

.. _website: https://v.vtx.lk/vertex

.. _Synapse 101: https://v.vtx.lk/new-syn101

.. _pypi: https://v.vtx.lk/synapse-pypi

.. _dockerhub: https://v.vtx.lk/synapse-dockerhub

.. _guide: https://synapse.docs.vertex.link/en/latest/synapse/quickstart.html

.. _synapse_quickstart: https://github.com/vertexproject/synapse-quickstart

.. _here: https://v.vtx.lk/synapse-docs

.. _slack: https://v.vtx.lk/join-slack

.. _Twitter: https://v.vtx.lk/twitter

.. _LinkedIn: https://www.linkedin.com/company/vertexproject


# The COMPAS framework

![build](https://github.com/compas-dev/compas/workflows/build/badge.svg)
[![GitHub - License](https://img.shields.io/github/license/compas-dev/compas.svg)](https://github.com/compas-dev/compas)
[![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/compas)](https://anaconda.org/conda-forge/compas)
[![pip downloads](https://img.shields.io/pypi/dm/compas)](https://pypi.python.org/project/COMPAS)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/COMPAS.svg)](https://pypi.python.org/project/COMPAS)
[![PyPI - Latest Release](https://img.shields.io/pypi/v/COMPAS.svg)](https://pypi.python.org/project/COMPAS)
[![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/compas)](https://anaconda.org/conda-forge/compas)
[![Conda - Platform](https://img.shields.io/conda/pn/conda-forge/compas)](https://anaconda.org/conda-forge/compas)
[![DOI](https://zenodo.org/badge/104857648.svg)](https://zenodo.org/badge/latestdoi/104857648)
[![Twitter Follow](https://img.shields.io/twitter/follow/compas_dev?style=social)](https://twitter.com/compas_dev)

The **COMPAS** framework is an open-source, Python-based framework for computational research and collaboration in architecture, engineering, digital fabrication and construction.

The framework consists of a general-purpose core library, written in pure Python, and a growing collection of extensions that provide easy access to peer-reviewed research, state-of-the-art external libraries such as CGAL, libigl and Triangle, and tools with specialized functionality for AEFC applications such as Abaqus, ANSYS, SOFISTIK, ROS, etc.

COMPAS has dedicated packages for working with Rhino, Grasshopper, and Blender, but it can be used in any environment that supports Python scripting. It is available on PyPI and conda-forge and can be easily installed using popular package managers on multiple platforms.

## COMPAS 2.0!

We are working on a new major release of the framework, COMPAS 2.0!
Therefore, be aware that the current version of the repository already contains some changes that are incompatible with the version 1 releases.
The documentation of the latest stable version (COMPAS 1.17.9) [is available here](https://compas.dev/compas/1.17.9).

## Installation

The recommended way to install **COMPAS** is to use [Anaconda/conda](https://conda.io/docs/):

```bash
conda config --add channels conda-forge
conda install compas
```

For other installation options, [see the user guide](https://compas.dev/compas/latest/userguide/installation.html)

## First Steps

* [First steps](https://compas.dev/compas/latest/userguide/firststeps.html)
* [Tutorials: geometry basics](https://compas.dev/compas/latest/userguide/basics.geometry.html)
* [Tutorials: datastructures](https://compas.dev/compas/latest/userguide/basics.datastructures.html)
* [API Reference](https://compas.dev/compas/latest/api/index.html)

## Questions and feedback

The **COMPAS** framework has a [forum for questions and discussions](https://forum.compas-framework.org/).

## Issue tracker

If you find a bug, please help us solve it by [filing a report](https://github.com/compas-dev/compas/issues).

## Contributing

If you want to contribute, check out the [contribution guidelines](https://compas.dev/compas/latest/devguide/index.html).

## Changelog

See changes between releases on the [changelog](https://github.com/compas-dev/compas/blob/main/CHANGELOG.md).

## License

The main library of **COMPAS** is [released under the MIT license](https://compas.dev/compas/latest/userguide/license.html).

## Credits

COMPAS is developed by a small team of core developers (`compas-dev`) and with the support of contributers from the open source community.
See the [list of authors](https://github.com/compas-dev/compas/blob/main/AUTHORS.md) for a complete overview...


# Keylime

[![Slack CNCF chat](https://img.shields.io/badge/Chat-CNCF%20Slack-informational)](https://cloud-native.slack.com/archives/C01ARE2QUTZ)
[![Docs Status](https://readthedocs.org/projects/keylime/badge/?version=latest)](https://keylime.readthedocs.io/en/latest/?badge=latest)

![Keylime](docs/keylime.png?raw=true "Title")

Keylime is an open-source scalable trust system harnessing TPM Technology.

Keylime provides an end-to-end solution for bootstrapping hardware rooted
cryptographic trust for remote machines, the provisioning of encrypted payloads,
and run-time system integrity monitoring. It also provides a flexible
framework for the remote attestation of any given `PCR` (Platform Configuration
Register). Users can create their own customized actions that will trigger when
a machine fails its attested measurements.

Keylime's mission is to make TPM Technology easily accessible to developers and
users alike, without the need for a deep understanding of the lower levels of a
TPM's operations. Amongst many scenarios, it well suited to tenants who need to
remotely attest machines not under their own full control (such as a consumer of
hybrid cloud or a remote Edge / IoT device in an insecure physical tamper prone
location.)

Keylime can be driven with a CLI application and a set of RESTful APIs.

Keylime consists of three main components; The Verifier, Registrar and the
Agent.

* The Verifier continuously verifies the integrity state of the machine that
the agent is running on.

* The Registrar is a database of all agents registered
with Keylime and hosts the public keys of the TPM vendors.

* The Agent is deployed to the remote machine that is to be measured or provisioned
with secrets stored within an encrypted payload released once trust is established.

### Rust based Keylime Agent

The verifier, registrar, and agent are all developed in Python and situated
in this repository `keylime`. The agent was ported to the
[Rust programming language](https://www.rust-lang.org). The code can be found
in the [rust-keylime repository](https://github.com/keylime/rust-keylime).

The decision was made to port the agent to Rust, as rust is a low-level
performant systems language designed with security as a central tenet, by means
of the rust compiler's ownership model.

Starting with the 0.1.0 release of the Rust based Keylime agent, this agent is now the official agent.

| IMPORTANT: The Python version is deprecated and will be removed with the next major version (7.0.0)! |
|------------------------------------------------------------------------------------------------------|

### TPM Support

Keylime supports TPM version *2.0*.

Keylime can be used with a hardware TPM, or a software TPM emulator for
development, testing, or demonstration purposes.  However, DO NOT USE Keylime in
production with a TPM emulator!  A software TPM emulator does not provide a
hardware root of trust and dramatically lowers the security benefits of using
Keylime.

A hardware TPM should always be used when real secrets and trust is required.

## Table of Contents

* [Installation](#installation)
* [Usage](#usage)
  * [Configuring Keylime](#configuring-keylime)
  * [Running Keylime](#running-keylime)
  * [Provisioning](#provisioning)
* [Request a Feature](#request-a-feature)
* [Security Vulnerability Management Policy](#security-vulnerability-management-policy)
* [Meeting Information](#project-meetings)
* [Contributing: First Timers Support](#contributing--first-timers-support)
* [Testing](#testing)
* [Additional Reading](#additional-reading)
* [Disclaimer](#disclaimer)

## Installation

To install Keylime refer to [the instructions found in the documentation](https://keylime.readthedocs.io/en/latest/installation.html).

## Usage

### Configuring Keylime

Keylime puts its configuration in `/etc/keylime/*.conf` or `/usr/etc/keylime/*.conf`.
It will also take an alternate location for the config in the environment var
`keylime_{VERIFIER,REGISTRAR,TENANT,CA,LOGGING}_CONFIG`.

Those files are documented with comments and should be self-explanatory in most cases.

### Running Keylime

Keylime has three major component services that run: the registrar, verifier, and the agent:

* The *registrar* is a simple HTTPS service that accepts TPM public keys.  It then
presents an interface to obtain these public keys for checking quotes.

* The *verifier* is the most important component in Keylime.  It does initial and
periodic checks of system integrity and supports bootstrapping a cryptographic key
securely with the agent.  The verifier uses mutual TLS for its control interface.

    By default, the verifier will create appropriate TLS certificates for itself
    in `/var/lib/keylime/cv_ca/`.  The registrar and tenant will use this as well.  If
    you use the generated TLS certificates then all the processes need to run as root
    to allow reading of private key files in `/var/lib/keylime/`.

* The *agent* is the target of bootstrapping and integrity measurements.  It puts
    its stuff into `/var/lib/keylime/`.

### Provisioning

To kick everything off you need to tell Keylime to provision a machine. This can be
done with the Keylime tenant.

#### Provisioning with keylime_tenant

The `keylime_tenant` utility can be used to provision your agent.

As an example, the following command tells Keylime to provision a new agent
at 127.0.0.1 with UUID d432fbb3-d2f1-4a97-9ef7-75bd81c00000 and talk to a
verifier at 127.0.0.1. Finally, it will encrypt a file called `filetosend`
and send it to the agent allowing it to decrypt it only if the configured TPM
policy is satisfied:

`keylime_tenant -c add -t 127.0.0.1 -v 127.0.0.1 -u D432fbb3-d2f1-4a97-9ef7-75bd81c00000 -f filetosend`

To stop Keylime from requesting attestations:

`keylime_tenant -c delete -t 127.0.0.1 -u d432fbb3-d2f1-4a97-9ef7-75bd81c00000`

For additional advanced options for the tenant utility run:

`keylime_tenant -h`

Documentation on how to create runtime and measured boot policies can be found in
the [Keylime User Guide](https://keylime.readthedocs.io/en/latest/user_guide.html).

## Systemd service support

The directory `services/` includes `systemd` service files for the verifier,
agent and registrar.

You can install the services with the following command:

`sudo ./services/installer.sh`

Once installed, you can run and inspect the services `keylime_verifier` and `keylime_registrar` via `systemctl`.
The Rust agent repository also contains a systemd service file for the agent.

## Request a feature

Keylime feature requests are tracked as enhancements in the [enhancements repository](https://github.com/keylime/enhancements)

The enhancement process has been implemented to provide a way to review and
assess the impact(s) of significant changes to Keylime.

## Security Vulnerability Management Policy

If you have found a security vulnerability in Keylime and would like to
report, first of all: thank you.

Please contact us directly at [security@keylime.groups.io](mailto:security@keylime.groups.io)
for any bug that might impact the security of this project. **Do not** use a
Github issue to report any potential security bugs.

## Project Meetings

We meet on the fourth Wednesday each month @ 15:30 GMT to 16:30. Anyone is welcome to join the meeting.

The meeting is normally announced on [CNCF chat (Slack)](https://cloud-native.slack.com/archives/C01ARE2QUTZ)

Meeting agenda are hosted and archived in the [meetings repo](https://github.com/keylime/meetings) as GitHub issues.

## Contributing: First Timers Support

We welcome new contributors to Keylime of any form, including those of you who maybe new to working in an open source project.

So if you are new to open source development, don't worry, there are a myriad of ways you can get involved in our open source project. As a start, try exploring issues with [`good first issue`](https://github.com/keylime/keylime/labels/good%20first%20issue) label.
We understand that the process of creating a Pull Request (PR) can be a barrier for new contributors. These issues are reserved for new contributors like you. If you need any help or advice in making the PR, feel free to jump into our [chat room](https://cloud-native.slack.com/archives/C01ARE2QUTZ) and ask for help there.

Your contribution is our gift to make our project even more robust. Check out [CONTRIBUTING.md](https://github.com/keylime/keylime/blob/master/CONTRIBUTING.md) to find out more about how to contribute to our project.

Keylime uses [Semantic Versioning](https://semver.org/). It is recommended you also read the [RELEASE.md](RELEASE.md)
file to learn more about it and familiarise yourself with simple of examples of using it.

## Testing

Please, see [TESTING.md](TESTING.md) for details.

## Additional Reading

* Executive summary Keylime slides: [docs/old/keylime-elevator-slides.pptx](https://github.com/keylime/keylime/raw/master/docs/old/keylime-elevator-slides.pptx)
* Detailed Keylime Architecture slides: [docs/old/keylime-detailed-architecture-v7.pptx](https://github.com/keylime/keylime/raw/master/docs/old/keylime-detailed-architecture-v7.pptx)
* See ACSAC 2016 paper in doc directory: [docs/old/tci-acm.pdf](https://github.com/keylime/keylime/blob/master/docs/old/tci-acm.pdf)
  * and the ACSAC presentation on Keylime: [docs/old/llsrc-keylime-acsac-v6.pptx](https://github.com/keylime/keylime/raw/master/docs/old/llsrc-keylime-acsac-v6.pptx)
* See the HotCloud 2018 paper: [docs/old/hotcloud18.pdf](https://github.com/keylime/keylime/blob/master/docs/old/hotcloud18.pdf)
* Details about Keylime REST API: [docs/old/keylime RESTful API.docx](https://github.com/keylime/keylime/raw/master/docs/old/keylime%20RESTful%20API.docx)
* [Demo files](demo/) - Some pre-packaged demos to show off what Keylime can do.
* [IMA stub service](https://github.com/keylime/rust-keylime/tree/master/keylime-ima-emulator) - Allows you to test IMA and Keylime on a machine without a TPM.  Service keeps emulated TPM synchronized with IMA.

#### Errata from the ACSAC Paper

We discovered a typo in Figure 5 of the published ACSAC paper. The final interaction
between the Tenant and Cloud Verifier showed an HMAC of the node's ID using the key
K_e.  This should be using K_b. The paper in this repository and the ACSAC presentation
have been updated to correct this typo.

The software that runs on the system with the TPM is now called the Keylime *agent* rather
than the *node*.  We have made this change in the documentation and code.  The ACSAC paper
will remain as it was published using *node*.

## Disclaimer

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.

This material is based upon work supported by the Assistant Secretary of Defense for
Research and Engineering under Air Force Contract No. FA8721-05-C-0002 and/or
FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this
material are those of the author(s) and do not necessarily reflect the views of the
Assistant Secretary of Defense for Research and Engineering.

Keylime's license was changed from BSD Clause-2 to Apache 2.0. The original BSD
Clause-2 licensed code can be found on the [MIT GitHub
organization](https://github.com/mit-ll/MIT-keylime).


# Datadog Agent and API Integrations

[![Build Status][1]][2]
[![Code style - black][9]][10]

Collecting data is cheap; not having it when you need it can be very expensive. So we recommend instrumenting as much of your systems and applications as possible. This integrations repository will help you do that by making it easier to create and share new integrations for [Datadog][3].

**Note:** Integrations in this repo are not included with the Agent, and are not currently packaged.

## Building Integrations

For more information about how to build a new integration, please see [the guide at docs.datadoghq.com][4].

Also see the [agent integrations developer documentation][11] for more information on guidelines, tutorials, and metadata.

## Community Maintenance

Please note that integrations in this repository are maintained by the community. The current maintainer is listed in `manifest.json` and will address the pull requests and issues opened for that integration. Additionally, Datadog will assist on a best-effort basis, and will support the current maintainer whenever possible. When submitting a new integration, please indicate in the PR that you're willing to become the maintainer. For current maintainers, we understand circumstances change. If you're no longer able to maintain an integration, please notify us so we can find a new maintainer or mark the integration as orphaned. If you have any questions about the process, don't hesitate to contact us.

## Submitting Your Integration

Once you have completed the development of your integration, submit a [pull request][5] to have Datadog review your integration. Once we've reviewed your integration, we will approve and merge your pull request or provide feedback and next steps required for approval.

## Reporting Issues

For more information on integrations, please reference our [documentation][6] and [knowledge base][7]. You can also visit our [help page][8] to connect with us.

[1]: https://raw.githubusercontent.com/DataDog/integrations-extras/badges/test-results.svg
[2]: https://github.com/DataDog/integrations-extras/actions/workflows/master.yml
[3]: https://www.datadoghq.com
[4]: https://docs.datadoghq.com/developers/integrations/
[5]: https://github.com/DataDog/integrations-extras/compare
[6]: http://docs.datadoghq.com
[7]: https://help.datadoghq.com/hc/en-us
[8]: http://docs.datadoghq.com/help/
[9]: https://img.shields.io/badge/code%20style-black-000000.svg
[10]: https://github.com/ambv/black
[11]: https://datadoghq.dev/integrations-core/


.. image:: https://img.shields.io/pypi/v/biotite.svg
   :target: https://pypi.python.org/pypi/biotite
   :alt: Biotite at PyPI
.. image:: https://img.shields.io/pypi/pyversions/biotite.svg
   :alt: Python version
.. image:: https://github.com/biotite-dev/biotite/actions/workflows/test_and_deploy.yml/badge.svg
   :target: https://github.com/biotite-dev/biotite/actions/workflows/test_and_deploy.yml
   :alt: Test status

.. image:: https://www.biotite-python.org/_static/assets/general/biotite_logo_m.png
   :alt: The Biotite Project

Biotite project
===============

*Biotite* is your Swiss army knife for bioinformatics.
Whether you want to identify homologous sequence regions in a protein family
or you would like to find disulfide bonds in a protein structure: *Biotite*
has the right tool for you.
This package bundles popular tasks in computational molecular biology
into a uniform *Python* library.
It can handle a major part of the typical workflow
for sequence and biomolecular structure data:

   - Searching and fetching data from biological databases
   - Reading and writing popular sequence/structure file formats
   - Analyzing and editing sequence/structure data
   - Visualizing sequence/structure data
   - Interfacing external applications for further analysis

*Biotite* internally stores most of the data as *NumPy* `ndarray` objects,
enabling

   - fast C-accelerated analysis,
   - intuitive usability through *NumPy*-like indexing syntax,
   - extensibility through direct access of the internal *NumPy* arrays.

As a result the user can skip writing code for basic functionality (like
file parsers) and can focus on what their code makes unique - from
small analysis scripts to entire bioinformatics software packages.

If you use *Biotite* in a scientific publication, please cite:

| Kunzmann, P. & Hamacher, K. BMC Bioinformatics (2018) 19:346.
| `<https://doi.org/10.1186/s12859-018-2367-z>`_

Installation
------------

*Biotite* requires the following packages:

   - **numpy**
   - **requests**
   - **msgpack**
   - **networkx**

Some functions require some extra packages:

   - **mdtraj** - Required for trajetory file I/O operations.
   - **matplotlib** - Required for plotting purposes.

*Biotite* can be installed via *Conda*...

.. code-block:: console

   $ conda install -c conda-forge biotite

... or *pip*

.. code-block:: console

   $ pip install biotite

Usage
-----

Here is a small example that downloads two protein sequences from the
*NCBI Entrez* database and aligns them:

.. code-block:: python

   import biotite.sequence.align as align
   import biotite.sequence.io.fasta as fasta
   import biotite.database.entrez as entrez

   # Download FASTA file for the sequences of avidin and streptavidin
   file_name = entrez.fetch_single_file(
       uids=["CAC34569", "ACL82594"], file_name="sequences.fasta",
       db_name="protein", ret_type="fasta"
   )

   # Parse the downloaded FASTA file
   # and create 'ProteinSequence' objects from it
   fasta_file = fasta.FastaFile.read(file_name)
   avidin_seq, streptavidin_seq = fasta.get_sequences(fasta_file).values()

   # Align sequences using the BLOSUM62 matrix with affine gap penalty
   matrix = align.SubstitutionMatrix.std_protein_matrix()
   alignments = align.align_optimal(
       avidin_seq, streptavidin_seq, matrix,
       gap_penalty=(-10, -1), terminal_penalty=False
   )
   print(alignments[0])

.. code-block::

   MVHATSPLLLLLLLSLALVAPGLSAR------KCSLTGKWDNDLGSNMTIGAVNSKGEFTGTYTTAV-TA
   -------------------DPSKESKAQAAVAEAGITGTWYNQLGSTFIVTA-NPDGSLTGTYESAVGNA

   TSNEIKESPLHGTQNTINKRTQPTFGFTVNWKFS----ESTTVFTGQCFIDRNGKEV-LKTMWLLRSSVN
   ESRYVLTGRYDSTPATDGSGT--ALGWTVAWKNNYRNAHSATTWSGQYV---GGAEARINTQWLLTSGTT

   DIGDDWKATRVGINIFTRLRTQKE---------------------
   -AANAWKSTLVGHDTFTKVKPSAASIDAAKKAGVNNGNPLDAVQQ

More documentation, including a tutorial, an example gallery and the API
reference is available at `<https://www.biotite-python.org/>`_.

Contribution
------------

Interested in improving *Biotite*?
Have a look at the
`contribution guidelines <https://www.biotite-python.org/contribute.html>`_.
Feel free to join or community chat on `Discord <https://discord.gg/cUjDguF>`_.


.. image:: docs/_static/logo.png

|

**OGGM is a modular open source model for glacier dynamics**

OGGM is able to simulate past and
future mass balance, volume and geometry of (almost) any glacier in the world,
in a fully automated and extensible workflow.

The model accounts for glacier geometry (including contributory branches) and
includes an explicit ice dynamics module. We rely exclusively on publicly
available data for calibration and validation. **OGGM is modular and
supports novel modelling workflows**: it LOVES to be remixed and reused!

.. image:: docs/_static/ex_tasman.jpg

Installation, documentation
---------------------------

The documentation is hosted on ReadTheDocs: http://docs.oggm.org

Get in touch
------------

- View the source code `on GitHub`_.
- Report bugs or share your ideas on the `issue tracker`_.
- Improve the model by submitting a `pull request`_.
- Follow us on `Twitter`_.
- Or you can always send us an `e-mail`_ the good old way.

.. _e-mail: info@oggm.org
.. _on GitHub: https://github.com/OGGM/oggm
.. _issue tracker: https://github.com/OGGM/oggm/issues
.. _pull request: https://github.com/OGGM/oggm/pulls
.. _Twitter: https://twitter.com/OGGM_org

About
-----

:Version:
    .. image:: https://img.shields.io/pypi/v/oggm.svg
        :target: https://pypi.python.org/pypi/oggm
        :alt: Pypi version

    .. image:: https://img.shields.io/pypi/pyversions/oggm.svg
        :target: https://pypi.python.org/pypi/oggm
        :alt: Supported python versions

:Citation:
    .. image:: https://img.shields.io/badge/Citation-GMD%20paper-orange.svg
        :target: https://www.geosci-model-dev.net/12/909/2019/
        :alt: GMD Paper

    .. image:: https://zenodo.org/badge/43965645.svg
        :target: https://zenodo.org/badge/latestdoi/43965645
        :alt: Zenodo

:Tests:
    .. image:: https://coveralls.io/repos/github/OGGM/oggm/badge.svg?branch=master
        :target: https://coveralls.io/github/OGGM/oggm?branch=master
        :alt: Code coverage

    .. image:: https://github.com/OGGM/oggm/actions/workflows/run-tests.yml/badge.svg?branch=master
        :target: https://github.com/OGGM/oggm/actions/workflows/run-tests.yml
        :alt: Linux build status

    .. image:: https://img.shields.io/badge/Cross-validation-blue.svg
        :target: https://cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.4/crossval.html
        :alt: Mass balance cross validation

    .. image:: https://readthedocs.org/projects/oggm/badge/?version=latest
        :target: http://docs.oggm.org/en/latest/
        :alt: Documentation status

    .. image:: https://img.shields.io/badge/benchmarked%20by-asv-green.svg?style=flat
        :target: https://cluster.klima.uni-bremen.de/~github/asv/
        :alt: Benchmark status

:License:
    .. image:: https://img.shields.io/pypi/l/oggm.svg
        :target: https://github.com/OGGM/oggm/blob/master/LICENSE.txt
        :alt: BSD-3-Clause License

:Authors:

    See the `version history`_ for a list of all contributors.

    .. _version history: http://docs.oggm.org/en/stable/whats-new.html


<!--
This file is part of BenchExec, a framework for reliable benchmarking:
https://github.com/sosy-lab/benchexec

SPDX-FileCopyrightText: 2007-2020 Dirk Beyer <https://www.sosy-lab.org>

SPDX-License-Identifier: Apache-2.0
-->

# BenchExec

## A Framework for Reliable Benchmarking and Resource Measurement

[![Build Status](https://gitlab.com/sosy-lab/software/benchexec/badges/main/pipeline.svg)](https://gitlab.com/sosy-lab/software/benchexec/pipelines)
[![Apache 2.0 License](https://img.shields.io/badge/license-Apache--2-brightgreen.svg)](https://www.apache.org/licenses/LICENSE-2.0)
[![PyPI version](https://img.shields.io/pypi/v/BenchExec.svg)](https://pypi.python.org/pypi/BenchExec)
[![DOI](https://zenodo.org/badge/30758422.svg)](https://zenodo.org/badge/latestdoi/30758422)

**News and Updates**:
- Two projects accepted for BenchExec as part of [Google Summer of Code](https://summerofcode.withgoogle.com/)!
  We are happy that [Haoran Yang](https://summerofcode.withgoogle.com/programs/2024/projects/UzhlnEel)
  and [Eshaan Aggarwal](https://summerofcode.withgoogle.com/programs/2024/projects/l7WEa0Gz)
  will work to improve BenchExec (click on their names for more information about their projects)
  and are thankful to them and to Google for providing the opportunity.
- BenchExec 3.18 brings support for systems with cgroups v2!
- Linux kernel 5.11 finally [makes it possible](https://github.com/sosy-lab/benchexec/blob/main/doc/INSTALL.md#kernel-requirements) to use all BenchExec features on distributions other than Ubuntu!
- We now provide an [Ubuntu PPA](https://launchpad.net/~sosy-lab/+archive/ubuntu/benchmarking) that makes installing and upgrading BenchExec easier ([docs](https://github.com/sosy-lab/benchexec/blob/main/doc/INSTALL.md#debianubuntu)).
- An extended version of our paper on BenchExec and its background was published as open access in the journal STTT,
  you can read [Reliable Benchmarking: Requirements and Solutions](https://doi.org/10.1007/s10009-017-0469-y) online.
  We also provide a set of [overview slides](https://www.sosy-lab.org/research/prs/Latest_ReliableBenchmarking.pdf).

> To help new or inexperienced users get started with reliable benchmarking
> right away, we offer a [quickstart guide](doc/quickstart.md) that contains
> a brief explanation of the issues of common setups as well as the (few)
> steps necessary to setup and use BenchExec instead.

BenchExec is a framework for reliable benchmarking and resource measurement
and provides a standalone solution for benchmarking
that takes care of important low-level details for accurate, precise, and reproducible measurements
as well as result handling and analysis for large sets of benchmark runs.
However, even users of other benchmarking frameworks or scripts
can benefit from BenchExec
by letting it perform the resource measurements and limits
instead of less reliable tools and techniques like `time` or `ulimit`,
and results produced by BenchExec can easily be exported for use with other tools.

In particular, BenchExec provides three major features:

- execution of arbitrary commands with precise and reliable measurement
  and limitation of resource usage (e.g., CPU time and memory),
  and isolation against other running processes
  (provided by [`runexec`](https://github.com/sosy-lab/benchexec/blob/main/doc/runexec.md),
  a replacement for `time` and similar tools)
- an easy way to define benchmarks with specific tool configurations
  and resource limits,
  and automatically executing them on large sets of input files
  (provided by [`benchexec`](https://github.com/sosy-lab/benchexec/blob/main/doc/benchexec.md)
  on top of `runexec`)
- generation of interactive tables and plots for the results
  (provided by [`table-generator`](https://github.com/sosy-lab/benchexec/blob/main/doc/table-generator.md)
  for results produced with `benchexec`)

Unlike other benchmarking frameworks,
BenchExec is able to reliably measure and limit resource usage
of the benchmarked tool even if the latter spawns subprocesses.
In order to achieve this,
it uses the [cgroups feature](https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt)
of the Linux kernel to correctly handle groups of processes.
For proper isolation of the benchmarks, it uses (if available)
Linux [user namespaces](http://man7.org/linux/man-pages/man7/namespaces.7.html)
and an [overlay filesystem](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt)
to create a [container](https://github.com/sosy-lab/benchexec/blob/main/doc/container.md)
that restricts interference of the executed tool with the benchmarking host.
More information on why this is necessary and the problems with other tools
can be found in our paper
[Reliable Benchmarking: Requirements and Solutions](https://doi.org/10.1007/s10009-017-0469-y) (open access)
and our [slides](https://www.sosy-lab.org/research/prs/Latest_ReliableBenchmarking.pdf)
(starting with slide "Checklist").

BenchExec is intended for benchmarking non-interactive tools on Linux systems.
It measures CPU time, wall time, and memory usage of a tool,
and allows to specify limits for these resources.
It also allows to limit the CPU cores and (on NUMA systems) memory regions,
and the container mode allows to restrict filesystem and network access.
In addition to measuring resource usage,
BenchExec can optionally verify that the result of the tool was as expected
and extract further statistical data from the output.
Results from multiple runs can be combined into CSV and interactive HTML tables,
of which the latter provide scatter and quantile plots
(have a look at our [demo table](https://sosy-lab.github.io/benchexec/example-table/svcomp-simple-cbmc-cpachecker.table.html)).

BenchExec works only on Linux and needs a one-time setup of cgroups by the machine's administrator.
The actual benchmarking can be done by any user and does not need root access.

BenchExec was originally developed for use with the software verification framework
[CPAchecker](https://cpachecker.sosy-lab.org)
and is now developed as an independent project
at the [Software Systems Lab](https://www.sosy-lab.org)
of the [Ludwig-Maximilians-Universität München (LMU Munich)](https://www.uni-muenchen.de).

### Links

- [Documentation](https://github.com/sosy-lab/benchexec/tree/main/doc/INDEX.md)
- [Demo](https://sosy-lab.github.io/benchexec/example-table/svcomp-simple-cbmc-cpachecker.table.html) of a result table
- [Downloads](https://github.com/sosy-lab/benchexec/releases)
- [Changelog](https://github.com/sosy-lab/benchexec/tree/main/CHANGELOG.md)
- [BenchExec GitHub Repository](https://github.com/sosy-lab/benchexec),
  use this for [reporting issues and asking questions](https://github.com/sosy-lab/benchexec/issues)
- [BenchExec at PyPI](https://pypi.python.org/pypi/BenchExec)
- [BenchExec at Zenodo](https://doi.org/10.5281/zenodo.1163552): Each release gets a DOI such that you can reference it from your publications and artifacts.

### Literature

- [Reliable Benchmarking: Requirements and Solutions](https://doi.org/10.1007/s10009-017-0469-y), by D. Beyer, S. Löwe, and P. Wendler.  International Journal on Software Tools for Technology Transfer (STTT), 21(1):1-29, 2019. [doi:10.1007/s10009-017-0469-y](https://doi.org/10.1007/s10009-017-0469-y). Journal paper about BenchExec (open access, also with a [supplementary webpage](https://www.sosy-lab.org/research/benchmarking/) and [overview slides](https://www.sosy-lab.org/research/prs/Latest_ReliableBenchmarking.pdf))
- [CPU Energy Meter: A Tool for Energy-Aware Algorithms Engineering](https://doi.org/10.1007/978-3-030-45237-7_8), by D. Beyer and P. Wendler. In Proc. TACAS 2020, part 2, LNCS 12079, pages 126-133, 2020. Springer. [doi:10.1007/978-3-030-45237-7_8](https://doi.org/10.1007/978-3-030-45237-7_8)
- [Benchmarking and Resource Measurement](https://doi.org/10.1007/978-3-319-23404-5_12), by D. Beyer, S. Löwe, and P. Wendler. In Proc. SPIN 2015, LNCS 9232, pages 160-178, 2015. Springer. [doi:10.1007/978-3-319-23404-5_12](https://doi.org/10.1007/978-3-319-23404-5_12)

### License and Copyright

BenchExec is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0),
copyright [Dirk Beyer](https://www.sosy-lab.org/people/beyer/).
Exceptions are some tool-info modules
and third-party code that is bundled in the HTML tables,
which are available under several other free licenses
(cf. [folder `LICENSES`](https://github.com/sosy-lab/benchexec/tree/main/LICENSES)).

### Authors
Maintainer: [Philipp Wendler](https://www.philippwendler.de)

Contributors:
- [Eshaan Aggarwal](https://github.com/EshaanAgg)
- [Aditya Arora](https://github.com/alohamora)
- [Levente Bajczi](https://github.com/leventeBajczi)
- [Dirk Beyer](https://www.sosy-lab.org/people/beyer/)
- [Laura Bschor](https://github.com/laurabschor)
- [Thomas Bunk](https://github.com/TBunk)
- [Montgomery Carter](https://github.com/MontyCarter)
- [Andreas Donig](https://github.com/adonig)
- [Karlheinz Friedberger](https://www.sosy-lab.org/people/friedberger)
- [Robin Gloster](https://github.com/globin)
- [Sam Grayson](https://github.com/charmoniumQ)
- Peter Häring
- [Florian Heck](https://github.com/fheck)
- [George Karpenkov](http://metaworld.me/)
- [Mike Kazantsev](http://fraggod.net/)
- [Hugo van Kemenade](https://github.com/hugovk)
- [Michael Lachner](https://github.com/lachnerm)
- [Thomas Lemberger](https://www.sosy-lab.org/people/lemberger/)
- [Lorenz Leutgeb](https://github.com/lorenzleutgeb)
- [Sebastian Ott](https://github.com/ottseb)
- Stefan Löwe
- [Stephan Lukasczyk](https://github.com/stephanlukasczyk)
- [Tobias Meggendorfer](https://github.com/incaseoftrouble)
- [Alexander von Rhein](http://www.infosun.fim.uni-passau.de/se/people-rhein.php)
- [Alexander Schremmer](https://www.xing.com/profile/Alexander_Schremmer)
- [Dennis Simon](https://github.com/DennisSimon)
- [Andreas Stahlbauer](http://stahlbauer.net/)
- [Thomas Stieglmaier](https://stieglmaier.me/)
- [Martin Yankov](https://github.com/marto97)
- [Hojan Young](https://github.com/younghojan)
- [Ilja Zakharov](https://github.com/IljaZakharov)
- and [lots of more people who integrated tools into BenchExec](https://github.com/sosy-lab/benchexec/graphs/contributors)

### Users of BenchExec

Several well-known international competitions use BenchExec,
such as [SMT-COMP](https://smt-comp.github.io/),
[SV-COMP](https://sv-comp.sosy-lab.org) (software verification),
the [Termination Competition](https://termination-portal.org/wiki/Termination_Competition),
and
[Test-Comp](https://test-comp.sosy-lab.org).
In particular in SV-COMP
BenchExec was used successfully for benchmarking in all instances of the competition
and with a wide variety of benchmarked tools and millions of benchmark runs per year.
BenchExec is also integrated into the cluster-based logic-solving service
[StarExec](https://www.starexec.org/starexec/public/about.jsp) ([GitHub](https://github.com/StarExec/StarExec)).

The developers of the following tools use BenchExec:

- [CPAchecker](https://cpachecker.sosy-lab.org), also for regression testing
- [Dartagnan](https://github.com/hernanponcedeleon/Dat3M)
- [ESBMC](https://github.com/esbmc/esbmc), also for regression testing and even with a [GitHub action](https://github.com/esbmc/esbmc/blob/master/.github/workflows/benchexec.yml) for BenchExec
- [SMACK](https://github.com/smackers/smack)
- [SMTInterpol](https://github.com/ultimate-pa/smtinterpol)
- [TriCera](https://github.com/uuverifiers/tricera)
- [Ultimate](https://github.com/ultimate-pa/ultimate)

If you would like to be listed here, [contact us](https://github.com/sosy-lab/benchexec/issues/new).


neptune.ai

  Quickstart
    •
  Website
    •
  Docs
    •
  Examples
    •
  Resource center
    •
  Blog

## What is neptune.ai?

Neptune is a lightweight experiment tracker for ML teams that struggle with debugging and reproducing experiments, sharing results, and messy model handover. It offers a single place to track, compare, store, and collaborate on experiments and models.

With Neptune, Data Scientists can develop production-ready models faster, and ML Engineers can access model artifacts instantly in order to deploy them to production.

Watch a 3min explainer video →

Watch a 20min product demo →

Play with a live example project in the Neptune app  →

## Getting started

**Step 1:** Create a **[free account](https://neptune.ai/register)**

**Step 2:** Install the Neptune client library

```bash
pip install neptune
```

**Step 3:** Add an experiment tracking snippet to your code

```python
import neptune

run = neptune.init_run(project="workspace-name/project-name")
run["parameters"] = {"lr": 0.1, "dropout": 0.4}
run["test_accuracy"] = 0.84
```

[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neptune-ai/examples/blob/master/how-to-guides/hello-neptune/notebooks/hello_neptune.ipynb)

## Core features

**Log and display**

Add a snippet to any step of your ML pipeline once. Decide what and how you want to log. Run a million times.

* Any framework: any code, fastai, PyTorch, Lightning, TensorFlow/Keras, scikit-learn, 🤗 Transformers, XGBoost, Optuna.

* Any metadata type: metrics, parameters, dataset and model versions, images, interactive plots, videos, hardware (GPU, CPU, memory), code state.

* From anywhere in your ML pipeline: multinode pipelines, distributed computing, log during or after execution, log offline, and sync when you are back online.

**Organize experiments**

Organize logs in a fully customizable nested structure. Display model metadata in user-defined dashboard templates.

* Nested metadata structure: the flexible API lets you customize the metadata logging structure however you want. Organize nested parameter configs or the results on k-fold validation splits the way they should be.

* Custom dashboards: combine different metadata types in one view. Define it for one run. Use anywhere. Look at GPU, memory consumption, and load times to debug training speed. See learning curves, image predictions, and confusion matrix to debug model quality.

* Table views: create different views of the runs table and save them for later. You can have separate table views for debugging, comparing parameter sets, or best experiments.

**Compare results**

Visualize training live in the neptune.ai web app. See how different parameters and configs affect the results. Optimize models quicker.

* Compare: learning curves, parameters, images, datasets.

* Search, sort, and filter: experiments by any field you logged. Use our query language to filter runs based on parameter values, metrics, execution times, or anything else.

* Visualize and display: runs table, interactive display, folder structure, dashboards.

* Monitor live: hardware consumption metrics, GPU, CPU, memory.

* Group by: dataset versions, parameters.

**Version models**

Version, review, and access production-ready models and metadata associated with them in a single place.

* Version models: register models, create model versions, version external model artifacts.

* Review and change stages: look at the validation, test metrics and other model metadata. You can move models between None/Staging/Production/Archived.

* Access and share models: every model and model version is accessible via the neptune.ai web app or through the API.

**Share results**

Have a single place where your team can see the results and access all models and experiments.

* Send a link: share every chart, dashboard, table view, or anything else you see in the neptune.ai app by copying and sending persistent URLs.

* Query API: access all model metadata via neptune.ai API. Whatever you logged, you can query in a similar way.

* Manage users and projects: create different projects, add users to them, and grant different permissions levels.

* Add your entire org: you can collaborate with a team on every plan, even the Free one. So, invite your entire organization, including product managers and subject matter experts, to increase the visibility from the very beginning.

## Integrate with any MLOps stack
neptune.ai integrates with 25+ frameworks: PyTorch, Lightning, TensorFlow/Keras, LightGBM, scikit-learn, XGBoost, Optuna, Kedro, 🤗 Transformers, fastai, Prophet, detectron2, Airflow, and more.

####

 PyTorch Lightning

Example:

```python
from pytorch_lightning import Trainer
from lightning.pytorch.loggers import NeptuneLogger

# Create NeptuneLogger instance
from neptune import ANONYMOUS_API_TOKEN

neptune_logger = NeptuneLogger(
    api_key=ANONYMOUS_API_TOKEN,
    project="common/pytorch-lightning-integration",
    tags=["training", "resnet"],  # optional
)

# Pass the logger to the Trainer
trainer = Trainer(max_epochs=10, logger=neptune_logger)

# Run the Trainer
trainer.fit(my_model, my_dataloader)
```

[![neptune-pl](https://img.shields.io/badge/PytorchLightning-experiment-success?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAgCAYAAABQISshAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAF6klEQVRYCa2YXWiWZRjH301nGWVTa36ESWqb1bTSorSTUg8kwg4iSIhOgo49yDoIOomgjiqQPCroIEGQqA6ighoIZYbZNEtnhU1rUc3U5fzY3Nbv//hcT//33rN37zu74Nr1fV33fd1fz95KpQaMjo7OHBsb2wy+A74HbgXbaoTUZSLHevBj8A/wGLiNWjfVFdyokxJT4CMwhb0obm40X/iT9zHiB9KkyF3Yrg+//4WSdBq4s6RYqN6GaWq0GDGLwePgRLCl0Zw1/amyDhyaqBr6U2B7zSQlRmJeLcvJSoR6P8w1JaGNq0jUBO6IzDm9CB1OdFsbyU7sQvDXJMfZRL6EvKGRvOHbHIzRpfAbTT4L/zj4FHjB9I/SyRkmT8Y+jIMf6N3Ia8HtFjgNfpPJU2fpyJakS7siG/ous52DXxW2WhS/ZvB9ixX7pGJoxhL4k2Y7At/woa9aEZKqI+qcw2cm7DF+Jvw6kydkGdgCjPebwwD815Kbmpp+CV4ysAxcnXEN/KmaCEl1rXqSIeRuy3cAfszkhxikJl8TyKuVu9GcjsD3SsY2CvlKfA7K92AI9dKqiRB0HzjHgvvgj4bMoA/BnwsZugrdPJMnYtdg8FoHmMBFc94PrwkFPFBPg8JZ1JNLVkGHHxBOhYLi6uLxkKFt6FaYPI7NB5SeJa1sAfiojrZbQCc6bce6oZgIgbqBfFspiTrnnRpEp6IBik9jwhZ0LsztIUAvgVrZAqjxG8LPhaJSuQG+ZoPMN2OLiSC1ge2Jw3cuU1Dnw8+MzKtHRkZqvfLKOV+OOZyEFttVOvLqWvfJNaNLV1GuE4JPZCles83zPHyPycFqIr5KnRSdFcYSuhJdi+k1iX6Tg00bpPNXq0ERl9F0ItPN+jv8MZODVedOhwBdxESWmJyydyWKQ/gPJzqJB8ER03fCX2dyFcskV4CvgM/xbMzxgS+u8ry8GmcSncS4yeJd0HtyB/itjA75yy+bQ9VBN4OuZK1U3IKL4G8Bx/kz+Dnod4CarLZmh6/IVVIapAc9MxGUvi3Sp10PX507PXABWgk/C6EX/RP0sxMNcp+MZyKq5w1a7xPZhfF4HnUC5505X0Z07zusxL/sYbwVJ3Uv4C/8fgzBKQ3Sbabt5VDaIBxuA/38/FNsLRLtYytsgN6N04Hm5uayg54VYTAH85WZkSkqlQ50s+G1NRzUtaIGfA9xf7uD8+Toxu4qnQPdYH65aCtlW8oce7xIhcGrW6UdsyAlko/ufu1hwXx0uvXSiejGctBBV+dLAZuu+4tgbPPlTKQVuZg8ssa8HHTY51vLDZPxp3HwN0Yr43u2QsFxOny6ayUmRg3qM58FeYNMlW3VZabQah2c0kRIruBvLJnYdD/rI9EL6tHzySumCnhY1aDDptTKVDUIWSuvSyRgAKZnShPJM2givnc7GYhv1XbsftBPINfcti0tLcqXXrd3onPQQddqB/Sykn1XMhHdMPrcCFjOSs0NAaoV8olpW50xeynLoNQg/1dBB95vxPSgf9/f33/+Siail9/fBH0J68YLuCcYUQbzJXYfoJsLHh9tP22XAH1wZv/LkENXWkcYcrq/ra1t3Gd84jOxSEHdPuk/RI8ogoLXQu4Vn4O+23aHMAntxe7nROchcrXC+4oMI/sYsuK+DbBPDrw7Gxn0CBign4meAV8LRU734nv15BkvexCjbygHvXGbULwpJXzYDsPPyqLQzANfB7vAlziwlw11VCVJKzGHQU8uMYUX60hXuJB3DQkuKEkMGpqNXiR0mN8oghDeVYDBy0ND+pyqD4h7wWLL2DMo0yu0ZnL8p4MfKpkNWqLL+q1tbZEIQdvBYQ/B/v9D4VvG4LuA4J+UIC2aJ902ODhY9d1RlifVkUv/t6c/4OUpM/IWf/+7rBA+ldoGsR2xocLEbiZGHfI84j8HsxsnHWg9MumeVUrlTeATZH8Usx/I2lF+ASrgAwovrKdI6kPsE2A3KOgD9TvvlCcR+cnxNKgf7QRHwec5x+N+wPsX+f7UoKzjPDEAAAAASUVORK5CYII=)](https://app.neptune.ai/common/pytorch-lightning-integration/experiments?split=tbl&dash=charts&viewId=faa75e77-5bd6-42b9-9379-863fe7a33227)

[![github-code](https://img.shields.io/badge/GitHub-code-informational?logo=github)](https://github.com/neptune-ai/examples/tree/main/integrations-and-supported-tools/pytorch-lightning/scripts)
[![jupyter-code](https://img.shields.io/badge/Jupyter-code-informational?logo=jupyter)](https://github.com/neptune-ai/examples/blob/main/integrations-and-supported-tools/pytorch-lightning/notebooks/Neptune_PyTorch_Lightning.ipynb)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neptune-ai/examples/blob/main/integrations-and-supported-tools/pytorch-lightning/notebooks/Neptune_PyTorch_Lightning.ipynb)
[](https://docs.neptune.ai/integrations/lightning/)

## neptune.ai is trusted by great companies

Read how various customers use Neptune to improve their workflow.

## Support

If you get stuck or simply want to talk to us about something, here are your options:
* Check our FAQ page.
* Take a look at our resource center.
* Chat! In the app, click the blue message icon in the bottom-right corner and send a message. A real person will talk to you ASAP (typically very ASAP).
* You can just shoot us an email at [support@neptune.ai](mailto:support@neptune.ai).

## People behind

Created with :heart: by the [neptune.ai team](https://neptune.ai/jobs#team)


# Welcome to the Tekton Community!

This is the central home for documentation on joining and contributing to the
Tekton community.

## Want to learn how the project works

- Take a look at our [mission and roadmaps](roadmap.md)
- Take a look at [our Governance docs](governance.md)

## Want to get involved

[Reach out](contact.md) and see what we're up to via:

- [slack](contact.md#slack)
- [our mailing list](contact.md#mailing-list)
- [our working group meetings](working-groups.md)
- [our shared document drive](contact.md#shared-drive)
- [our shared calendar](contact.md#calendar)

See our standards regarding:

- [Code of conduct](code-of-conduct.md)
- [Design principles](design-principles.md)
- [Commits](standards.md#commit-messages)
- [Code](standards.md#code)
- [User profiles](user-profiles.md)
- [Releases](releases.md)

Find out about our processes:

- [Find something to work on](./process/README.md#finding-something-to-work-on)
- [Propose features](./process/README.md#proposing-features)
- [Contributor ladder](./process/contributor-ladder.md)
- Pull request [reviews](./process/README.md#reviews) and [process](./process/README.md#pull-request-process)
- [Propose projects](./process/projects.md)
- [GitHub Org Management](org/README.md), including
  [requirements to join the org](org/README.md#requirements)
- [The CDF CLA](./process/README.md#cla)
- [Postmortems](./process/README.md#postmortems)

_For guidelines on how to contribute to `tektoncd/community` see
[CONTRIBUTING.md](CONTRIBUTING.md)._

## Learn about our users community

- Take a look at our [tekton friends](https://github.com/tektoncd/friends) repo
- Take a look at our [list of adopters](adopters.md)
- Join our [tekton-users@](https://groups.google.com/forum/#!forum/tekton-users) mailing list