.. image:: https://github.com/aws/sagemaker-python-sdk/raw/master/branding/icon/sagemaker-banner.png
    :height: 100px
    :alt: SageMaker

====================
SageMaker Python SDK
====================

.. image:: https://img.shields.io/pypi/v/sagemaker.svg
   :target: https://pypi.python.org/pypi/sagemaker
   :alt: Latest Version

.. image:: https://img.shields.io/conda/vn/conda-forge/sagemaker-python-sdk.svg
   :target: https://anaconda.org/conda-forge/sagemaker-python-sdk
   :alt: Conda-Forge Version

.. image:: https://img.shields.io/pypi/pyversions/sagemaker.svg
   :target: https://pypi.python.org/pypi/sagemaker
   :alt: Supported Python Versions

.. image:: https://img.shields.io/badge/code_style-black-000000.svg
   :target: https://github.com/python/black
   :alt: Code style: black

.. image:: https://readthedocs.org/projects/sagemaker/badge/?version=stable
   :target: https://sagemaker.readthedocs.io/en/stable/
   :alt: Documentation Status

.. image:: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml/badge.svg
    :target: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml
    :alt: CI Health

SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker.

With the SDK, you can train and deploy models using popular deep learning frameworks **Apache MXNet** and **TensorFlow**.
You can also train and deploy models with **Amazon algorithms**,
which are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training.
If you have **your own algorithms** built into SageMaker compatible Docker containers, you can train and host models using these as well.

For detailed documentation, including the API reference, see `Read the Docs <https://sagemaker.readthedocs.io>`_.

Table of Contents
-----------------

#. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__
#. `Using the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html>`__
#. `Using MXNet <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html>`__
#. `Using TensorFlow <https://sagemaker.readthedocs.io/en/stable/using_tf.html>`__
#. `Using Chainer <https://sagemaker.readthedocs.io/en/stable/using_chainer.html>`__
#. `Using PyTorch <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html>`__
#. `Using Scikit-learn <https://sagemaker.readthedocs.io/en/stable/using_sklearn.html>`__
#. `Using XGBoost <https://sagemaker.readthedocs.io/en/stable/using_xgboost.html>`__
#. `SageMaker Reinforcement Learning Estimators <https://sagemaker.readthedocs.io/en/stable/using_rl.html>`__
#. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__
#. `Amazon SageMaker Built-in Algorithm Estimators <src/sagemaker/amazon/README.rst>`__
#. `Using SageMaker AlgorithmEstimators <https://sagemaker.readthedocs.io/en/stable/overview.html#using-sagemaker-algorithmestimators>`__
#. `Consuming SageMaker Model Packages <https://sagemaker.readthedocs.io/en/stable/overview.html#consuming-sagemaker-model-packages>`__
#. `BYO Docker Containers with SageMaker Estimators <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-docker-containers-with-sagemaker-estimators>`__
#. `SageMaker Automatic Model Tuning <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-automatic-model-tuning>`__
#. `SageMaker Batch Transform <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-batch-transform>`__
#. `Secure Training and Inference with VPC <https://sagemaker.readthedocs.io/en/stable/overview.html#secure-training-and-inference-with-vpc>`__
#. `BYO Model <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-model>`__
#. `Inference Pipelines <https://sagemaker.readthedocs.io/en/stable/overview.html#inference-pipelines>`__
#. `Amazon SageMaker Operators in Apache Airflow <https://sagemaker.readthedocs.io/en/stable/using_workflow.html>`__
#. `SageMaker Autopilot <src/sagemaker/automl/README.rst>`__
#. `Model Monitoring <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html>`__
#. `SageMaker Debugger <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html>`__
#. `SageMaker Processing <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html>`__

Installing the SageMaker Python SDK
-----------------------------------

The SageMaker Python SDK is built to PyPI and the latest version of the SageMaker Python SDK can be installed with pip as follows
::

    pip install sagemaker==<Latest version from pyPI from https://pypi.org/project/sagemaker/>

You can install from source by cloning this repository and running a pip install command in the root directory of the repository:

::

    git clone https://github.com/aws/sagemaker-python-sdk.git
    cd sagemaker-python-sdk
    pip install .

Supported Operating Systems
~~~~~~~~~~~~~~~~~~~~~~~~~~~

SageMaker Python SDK supports Unix/Linux and Mac.

Supported Python Versions
~~~~~~~~~~~~~~~~~~~~~~~~~

SageMaker Python SDK is tested on:

- Python 3.8
- Python 3.9
- Python 3.10
- Python 3.11

Telemetry
~~~~~~~~~~~~~~~

The ``sagemaker`` library has telemetry enabled to help us better understand user needs, diagnose issues, and deliver new features. This telemetry tracks the usage of various SageMaker functions.

If you prefer to opt out of telemetry, you can easily do so by setting the ``TelemetryOptOut`` parameter to ``true`` in the SDK defaults configuration. For detailed instructions, please visit `Configuring and using defaults with the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html#configuring-and-using-defaults-with-the-sagemaker-python-sdk>`__.

AWS Permissions
~~~~~~~~~~~~~~~

As a managed service, Amazon SageMaker performs operations on your behalf on the AWS hardware that is managed by Amazon SageMaker.
Amazon SageMaker can perform only operations that the user permits.
You can read more about which permissions are necessary in the `AWS Documentation <https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html>`__.

The SageMaker Python SDK should not require any additional permissions aside from what is required for using SageMaker.
However, if you are using an IAM role with a path in it, you should grant permission for ``iam:GetRole``.

Licensing
~~~~~~~~~
SageMaker Python SDK is licensed under the Apache 2.0 License. It is copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at:
http://aws.amazon.com/apache2.0/

Running tests
~~~~~~~~~~~~~

SageMaker Python SDK has unit tests and integration tests.

You can install the libraries needed to run the tests by running :code:`pip install --upgrade .[test]` or, for Zsh users: :code:`pip install --upgrade .\[test\]`

**Unit tests**

We run unit tests with tox, which is a program that lets you run unit tests for multiple Python versions, and also make sure the
code fits our style guidelines. We run tox with `all of our supported Python versions <#supported-python-versions>`_, so to run unit tests
with the same configuration we do, you need to have interpreters for those Python versions installed.

To run the unit tests with tox, run:

::

    tox tests/unit

**Integration tests**

To run the integration tests, the following prerequisites must be met

1. AWS account credentials are available in the environment for the boto3 client to use.
2. The AWS account has an IAM role named :code:`SageMakerRole`.
   It should have the AmazonSageMakerFullAccess policy attached as well as a policy with `the necessary permissions to use Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei-setup.html>`__.
3. To run remote_function tests, dummy ecr repo should be created. It can be created by running -
    :code:`aws ecr create-repository --repository-name remote-function-dummy-container`

We recommend selectively running just those integration tests you'd like to run. You can filter by individual test function names with:

::

    tox -- -k 'test_i_care_about'

You can also run all of the integration tests by running the following command, which runs them in sequence, which may take a while:

::

    tox -- tests/integ

You can also run them in parallel:

::

    tox -- -n auto tests/integ

Git Hooks
~~~~~~~~~

to enable all git hooks in the .githooks directory, run these commands in the repository directory:

::

    find .git/hooks -type l -exec rm {} \;
    find .githooks -type f -exec ln -sf ../../{} .git/hooks/ \;

To enable an individual git hook, simply move it from the .githooks/ directory to the .git/hooks/ directory.

Building Sphinx docs
~~~~~~~~~~~~~~~~~~~~

Setup a Python environment, and install the dependencies listed in ``doc/requirements.txt``:

::

    # conda
    conda create -n sagemaker python=3.7
    conda activate sagemaker
    conda install sphinx=3.1.1 sphinx_rtd_theme=0.5.0

    # pip
    pip install -r doc/requirements.txt

Clone/fork the repo, and install your local version:

::

    pip install --upgrade .

Then ``cd`` into the ``sagemaker-python-sdk/doc`` directory and run:

::

    make html

You can edit the templates for any of the pages in the docs by editing the .rst files in the ``doc`` directory and then running ``make html`` again.

Preview the site with a Python web server:

::

    cd _build/html
    python -m http.server 8000

View the website by visiting http://localhost:8000

SageMaker SparkML Serving
-------------------------

With SageMaker SparkML Serving, you can now perform predictions against a SparkML Model in SageMaker.
In order to host a SparkML model in SageMaker, it should be serialized with ``MLeap`` library.

For more information on MLeap, see https://github.com/combust/mleap .

Supported major version of Spark: 3.3 (MLeap version - 0.20.0)

Here is an example on how to create an instance of  ``SparkMLModel`` class and use ``deploy()`` method to create an
endpoint which can be used to perform prediction against your trained SparkML Model.

.. code:: python

    sparkml_model = SparkMLModel(model_data='s3://path/to/model.tar.gz', env={'SAGEMAKER_SPARKML_SCHEMA': schema})
    model_name = 'sparkml-model'
    endpoint_name = 'sparkml-endpoint'
    predictor = sparkml_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', endpoint_name=endpoint_name)

Once the model is deployed, we can invoke the endpoint with a ``CSV`` payload like this:

.. code:: python

    payload = 'field_1,field_2,field_3,field_4,field_5'
    predictor.predict(payload)

For more information about the different ``content-type`` and ``Accept`` formats as well as the structure of the
``schema`` that SageMaker SparkML Serving recognizes, please see `SageMaker SparkML Serving Container`_.

.. _SageMaker SparkML Serving Container: https://github.com/aws/sagemaker-sparkml-serving-container


 :warning: :warning: :warning:

TensorFlow Addons (TFA) has ended development and introduction of new features.

TFA has entered a minimal maintenance and release mode until a planned end of life
in May 2024. Please modify downstream libraries to take dependencies from other
repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP)

For more information see: [https://github.com/tensorflow/addons/issues/2807](https://github.com/tensorflow/addons/issues/2807)

-----------------

-----------------

[![PyPI Status Badge](https://badge.fury.io/py/tensorflow-addons.svg)](https://pypi.org/project/tensorflow-addons/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/tensorflow-addons)](https://pypi.org/project/tensorflow-addons/)
[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://www.tensorflow.org/addons/api_docs/python/tfa)
[![Gitter chat](https://img.shields.io/badge/chat-on%20gitter-46bc99.svg)](https://gitter.im/tensorflow/sig-addons)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

### Continuous Build Status

| Build      | Status |
| ---             | ---    |
| **Ubuntu/macOS**   | [![Status](https://github.com/tensorflow/addons/actions/workflows/release.yml/badge.svg)](https://github.com/tensorflow/addons/actions?query=workflow%3Aaddons-release) |
| **Ubuntu GPU custom ops**   | [![Status](https://storage.googleapis.com/tensorflow-kokoro-build-badges/addons/ubuntu-gpu-py3.svg)](https://storage.googleapis.com/tensorflow-kokoro-build-badges/addons/ubuntu-gpu-py3.html) |

**TensorFlow Addons** is a repository of contributions that conform to
well-established API patterns, but implement new functionality
not available in core TensorFlow. TensorFlow natively supports
a large number of operators, layers, metrics, losses, and optimizers.
However, in a fast moving field like ML, there are many interesting new
developments that cannot be integrated into core TensorFlow
(because their broad applicability is not yet clear, or it is mostly
 used by a smaller subset of the community).

## Addons Subpackages

* [tfa.activations](https://www.tensorflow.org/addons/api_docs/python/tfa/activations)
* [tfa.callbacks](https://www.tensorflow.org/addons/api_docs/python/tfa/callbacks)
* [tfa.image](https://www.tensorflow.org/addons/api_docs/python/tfa/image)
* [tfa.layers](https://www.tensorflow.org/addons/api_docs/python/tfa/layers)
* [tfa.losses](https://www.tensorflow.org/addons/api_docs/python/tfa/losses)
* [tfa.metrics](https://www.tensorflow.org/addons/api_docs/python/tfa/metrics)
* [tfa.optimizers](https://www.tensorflow.org/addons/api_docs/python/tfa/optimizers)
* [tfa.rnn](https://www.tensorflow.org/addons/api_docs/python/tfa/rnn)
* [tfa.seq2seq](https://www.tensorflow.org/addons/api_docs/python/tfa/seq2seq)
* [tfa.text](https://www.tensorflow.org/addons/api_docs/python/tfa/text)

## Maintainership
The maintainers of TensorFlow Addons can be found in the [CODEOWNERS](.github/CODEOWNERS) file of the repo. This file
is parsed and pull requests will automatically tag the owners using a bot. If you would
like to maintain something, please feel free to submit a PR. We encourage multiple
owners for all submodules.

## Installation
#### Stable Builds
TensorFlow Addons is available on PyPI for Linux & macOS (Windows support was dropped
due to [inconsistent TF2.15 whl packaging](https://github.com/tensorflow/tensorflow/issues/61830)). To install the latest version,
run the following:
```
pip install tensorflow-addons
```

To ensure you have a version of TensorFlow that is compatible with TensorFlow Addons,
you can specify the `tensorflow` extra requirement during install:

```
pip install tensorflow-addons[tensorflow]
```

Similar extras exist for the `tensorflow-gpu` and `tensorflow-cpu` packages.

To use TensorFlow Addons:

```python
import tensorflow as tf
import tensorflow_addons as tfa
```

### Python Op Compatility
TensorFlow Addons is actively working towards forward compatibility with TensorFlow 2.x.
However, there are still a few private API uses within the repository so at the moment
we can only guarantee compatibility with the TensorFlow versions which it was tested against.
Warnings will be emitted when importing `tensorflow_addons` if your TensorFlow version does not match
what it was tested against.

#### Python Op Compatibility Matrix
| TensorFlow Addons | TensorFlow | Python  |
|:----------------------- |:---|:---------- |
| tfa-nightly | 2.12, 2.13, 2.14 |3.9, 3.10, 3.11 |
| tensorflow-addons-0.22.0 | 2.12, 2.13, 2.14 |3.9, 3.10, 3.11 |
| tensorflow-addons-0.21.0 | 2.11, 2.12, 2.13 |3.8, 3.9, 3.10, 3.11 |
| tensorflow-addons-0.20.0 | 2.10, 2.11, 2.12 |3.8, 3.9, 3.10, 3.11 |
| tensorflow-addons-0.19.0 | 2.9, 2.10, 2.11 |3.7, 3.8, 3.9, 3.10 |
| tensorflow-addons-0.18.0 | 2.8, 2.9, 2.10 |3.7, 3.8, 3.9, 3.10 |
| tensorflow-addons-0.17.1 | 2.7, 2.8, 2.9 |3.7, 3.8, 3.9, 3.10 |
| tensorflow-addons-0.16.1 | 2.6, 2.7, 2.8 |3.7, 3.8, 3.9, 3.10 |
| tensorflow-addons-0.15.0 | 2.5, 2.6, 2.7 |3.7, 3.8, 3.9 |
| tensorflow-addons-0.14.0 | 2.4, 2.5, 2.6 |3.6, 3.7, 3.8, 3.9 |
| tensorflow-addons-0.13.0 | 2.3, 2.4, 2.5 |3.6, 3.7, 3.8, 3.9 |
| tensorflow-addons-0.12.1 | 2.3, 2.4 |3.6, 3.7, 3.8 |
| tensorflow-addons-0.11.2 | 2.2, 2.3 |3.5, 3.6, 3.7, 3.8 |
| tensorflow-addons-0.10.0 | 2.2 |3.5, 3.6, 3.7, 3.8 |
| tensorflow-addons-0.9.1 | 2.1, 2.2 |3.5, 3.6, 3.7 |
| tensorflow-addons-0.8.3 | 2.1 |3.5, 3.6, 3.7 |
| tensorflow-addons-0.7.1 | 2.1 | 2.7, 3.5, 3.6, 3.7 |
| tensorflow-addons-0.6.0 | 2.0 | 2.7, 3.5, 3.6, 3.7 |

### C++ Custom Op Compatibility
TensorFlow C++ APIs are not stable and thus we can only guarantee compatibility with the
version TensorFlow Addons was built against. It is possible custom ops will work with multiple
versions of TensorFlow, but there is also a chance for segmentation faults or other problematic crashes.
Warnings will be emitted when loading a custom op if your TensorFlow version does not match
what it was built against.

Additionally, custom ops registration does not have a stable ABI interface so it is
required that users have a compatible installation of TensorFlow even if the versions
match what we had built against. A simplification of this is that **TensorFlow Addons
custom ops will work with `pip`-installed TensorFlow** but will have issues when TensorFlow
is compiled differently. A typical example of this would be `conda`-installed TensorFlow.
[RFC #133](https://github.com/tensorflow/community/pull/133) aims to fix this.

#### C++ Custom Op Compatibility Matrix
| TensorFlow Addons | TensorFlow | Compiler  | cuDNN | CUDA |
|:----------------------- |:---- |:---------|:---------|:---------|
| tfa-nightly | 2.14  | GCC 9.3.1 | 8.6 | 11.8 |
| tensorflow-addons-0.22.0 | 2.14  | GCC 9.3.1 | 8.6 | 11.8 |
| tensorflow-addons-0.21.0 | 2.13  | GCC 9.3.1 | 8.6 | 11.8 |
| tensorflow-addons-0.20.0 | 2.12  | GCC 9.3.1 | 8.6 | 11.8 |
| tensorflow-addons-0.19.0 | 2.11  | GCC 9.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.18.0 | 2.10  | GCC 9.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.17.1 | 2.9  | GCC 9.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.16.1 | 2.8  | GCC 7.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.15.0 | 2.7  | GCC 7.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.14.0 | 2.6  | GCC 7.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.13.0 | 2.5  | GCC 7.3.1 | 8.1 | 11.2 |
| tensorflow-addons-0.12.1 | 2.4  | GCC 7.3.1 | 8.0 | 11.0 |
| tensorflow-addons-0.11.2 | 2.3  | GCC 7.3.1 | 7.6 | 10.1 |
| tensorflow-addons-0.10.0 | 2.2  | GCC 7.3.1 | 7.6 | 10.1 |
| tensorflow-addons-0.9.1 | 2.1  | GCC 7.3.1 | 7.6 | 10.1 |
| tensorflow-addons-0.8.3 | 2.1  | GCC 7.3.1 | 7.6 | 10.1 |
| tensorflow-addons-0.7.1 | 2.1  | GCC 7.3.1 | 7.6 | 10.1 |
| tensorflow-addons-0.6.0 | 2.0  | GCC 7.3.1 | 7.4 | 10.0 |

#### Nightly Builds
There are also nightly builds of TensorFlow Addons under the pip package
`tfa-nightly`, which is built against **the latest stable version of TensorFlow**. Nightly builds
include newer features, but may be less stable than the versioned releases. Contrary to
what the name implies, nightly builds are not released every night, but at every commit
of the master branch. `0.9.0.dev20200306094440` means that the commit time was
2020/03/06 at 09:44:40 Coordinated Universal Time.

```
pip install tfa-nightly
```

#### Installing from Source
You can also install from source. This requires the [Bazel](
https://bazel.build/) build system (version >= 1.0.0).

##### CPU Custom Ops
```
git clone https://github.com/tensorflow/addons.git
cd addons

# This script links project with TensorFlow dependency
python3 ./configure.py

bazel build build_pip_pkg
bazel-bin/build_pip_pkg artifacts

pip install artifacts/tensorflow_addons-*.whl
```

##### GPU and CPU Custom Ops
```
git clone https://github.com/tensorflow/addons.git
cd addons

export TF_NEED_CUDA="1"

# Set these if the below defaults are different on your system
export TF_CUDA_VERSION="12"
export TF_CUDNN_VERSION="8"
export CUDA_TOOLKIT_PATH="/usr/local/cuda"
export CUDNN_INSTALL_PATH="/usr/lib/x86_64-linux-gnu"

# This script links project with TensorFlow dependency
python3 ./configure.py

bazel build build_pip_pkg
bazel-bin/build_pip_pkg artifacts

pip install artifacts/tensorflow_addons-*.whl
```

## Tutorials
See [`docs/tutorials/`](docs/tutorials/)
for end-to-end examples of various addons.

## Core Concepts

#### Standardized API within Subpackages
User experience and project maintainability are core concepts in
TensorFlow Addons. In order to achieve these we require that our additions
conform to established API patterns seen in core TensorFlow.

#### GPU and CPU Custom Ops
TensorFlow Addons supports precompiled custom ops for CPU and GPU. However,
GPU custom ops currently only work on Linux distributions. For this reason Windows and macOS
will fallback to pure TensorFlow Python implementations whenever possible.

The order of priority on macOS/Windows is:
1) Pure TensorFlow + Python implementation (works on CPU and GPU)
2) C++ implementation for CPU

The order of priority on Linux is:
1) CUDA implementation
2) C++ implementation
3) Pure TensorFlow + Python implementation (works on CPU and GPU)

If you want to change the default priority, "C++ and CUDA" VS "pure TensorFlow Python",
you can set the environment variable `TF_ADDONS_PY_OPS=1` from the command line or
run `tfa.options.disable_custom_kernel()` in your code.

For example, if you are on Linux and you have compatibility problems with the compiled ops,
you can give priority to the Python implementations:

From the command line:
```bash
export TF_ADDONS_PY_OPS=1
```

or in your code:

```python
import tensorflow_addons as tfa
tfa.options.disable_custom_kernel()
```

This variable defaults to `True` on Windows and macOS, and `False` on Linux.

#### Proxy Maintainership
TensorFlow Addons has been designed to compartmentalize submodules so
that they can be maintained by community users who have expertise, and a vested
interest in that component. We heavily encourage users to submit sign up to maintain a
submodule by submitting your username to the [CODEOWNERS](.github/CODEOWNERS) file.

Full write access will only be granted after substantial contribution
has been made in order to limit the number of users with write permission.
Contributions can come in the form of issue closings, bug fixes, documentation,
new code, or optimizing existing code. Submodule maintainership can be granted
with a lower barrier for entry as this will not include write permissions to
the repo.

For more information see [the RFC](https://github.com/tensorflow/community/blob/master/rfcs/20190308-addons-proxy-maintainership.md)
on this topic.

#### Periodic Evaluation of Subpackages
Given the nature of this repository, submodules may become less
and less useful to the community as time goes on. In order to keep the
repository sustainable, we'll be performing bi-annual reviews of our code to
ensure everything still belongs within the repo. Contributing factors to this
review will be:

1. Number of active maintainers
2. Amount of OSS use
3. Amount of issues or bugs attributed to the code
4. If a better solution is now available

Functionality within TensorFlow Addons can be categorized into three groups:

* **Suggested**: well-maintained API; use is encouraged.
* **Discouraged**: a better alternative is available; the API is kept for
historic reasons; or the API requires maintenance and is the waiting period
to be deprecated.
* **Deprecated**: use at your own risk; subject to be deleted.

The status change between these three groups is:
Suggested <-> Discouraged -> Deprecated.

The period between an API being marked as deprecated and being deleted will be
90 days. The rationale being:

1. In the event that TensorFlow Addons releases monthly, there will be 2-3
releases before an API is deleted. The release notes could give user enough
warning.

2. 90 days gives maintainers ample time to fix their code.

## Contributing
TensorFlow Addons is a community-led open source project (only a few maintainers work for Google!).
As such, the project depends on public contributions, bug fixes, and documentation.
This project adheres to [TensorFlow's code of conduct](CODE_OF_CONDUCT.md).
By participating, you are expected to uphold this code.

Do you want to contribute but are not sure of what? Here are a few suggestions:
1. Add a new tutorial. Located in [`docs/tutorials/`](docs/tutorials),
  these are a great way to familiarize yourself and others with TensorFlow Addons. See
  [the guidelines](docs/tutorials/README.md) for more information on how to add
  examples.
2. Improve the docstrings. The docstrings are fetched and then displayed in the documentation.
  Do a change and hundreds of developers will see it and benefit from it. Maintainers are often focused
  on making APIs, fixing bugs and other code related changes. The documentation will never
  be loved enough!
3. Solve an [existing issue](https://github.com/tensorflow/addons/issues).
  These range from low-level software bugs to higher-level design problems.
  Check out the label [help wanted](https://github.com/tensorflow/addons/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22). If you're a new contributor, the label [good first issue](https://github.com/tensorflow/addons/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) can be a good place to start.
4. Review a pull request. So you're not a software engineer but you know a lot
  about a certain field a research? That's awesome and we need your help! Many people
  are submitting pull requests to add layers/optimizers/functions taken from recent
  papers. Since TensorFlow Addons maintainers are not specialized in everything,
  you can imagine how hard it is to review. It takes very long to read the paper,
  understand it and check the math in the pull request. If you're specialized, look at
  the [list of pull requests](https://github.com/tensorflow/addons/pulls).
  If there is something from a paper you know, please comment on the pull request to
  check the math is ok. If you see that everything is good, say it! It will help
  the maintainers to sleep better at night knowing that he/she wasn't the only
  person to approve the pull request.
5. You have an opinion and want to share it? The docs are not very helpful for
  a function or a class? You tried to open a pull request but you didn't manage to
  install or test anything and you think it's too complicated? You made a pull request
  but you didn't find the process good enough and it made no sense to you? Please
  say it! We want feedback. Maintainers are too much the head into the code
  to understand what it's like for someone new to open source to come to this project.
  If you don't understand something, be aware there are no people who are
  bad at understanding, there are just bad tutorials and bad guides.

Please see [contribution guidelines](CONTRIBUTING.md) to get started (and remember,
if you don't understand something, open an issue, or even make a pull request to
improve the guide!).

## Community
* [Public Mailing List](https://groups.google.com/a/tensorflow.org/forum/#!forum/addons)
* [SIG Monthly Meeting Notes](https://docs.google.com/document/d/1kxg5xIHWLY7EMdOJCdSGgaPu27a9YKpupUz2VTXqTJg)
    * Join our mailing list and receive calendar invites to the meeting

## License
[Apache License 2.0](LICENSE)


Distributed
===========

|Test Status| |Longitudinal Report (full)| |Longitudinal Report (short)| |Coverage| |Doc Status| |Discourse| |Version Status| |NumFOCUS|

A library for distributed computation.  See documentation_ for more details.

.. _documentation: https://distributed.dask.org
.. |Test Status| image:: https://github.com/dask/distributed/workflows/Tests/badge.svg?branch=main
   :target: https://github.com/dask/distributed/actions?query=workflow%3ATests+branch%3Amain
.. |Longitudinal Report (full)| image:: https://github.com/dask/distributed/workflows/Test%20Report/badge.svg?branch=main
   :target: https://dask.github.io/distributed/test_report.html
   :alt: Longitudinal test report (full version)
.. |Longitudinal Report (short)| image:: https://github.com/dask/distributed/workflows/Test%20Report/badge.svg?branch=main
   :target: https://dask.github.io/distributed/test_short_report.html
   :alt: Longitudinal test report (short version)
.. |Coverage| image:: https://codecov.io/gh/dask/distributed/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/dask/distributed/branch/main
   :alt: Coverage status
.. |Doc Status| image:: https://readthedocs.org/projects/distributed/badge/?version=latest
   :target: https://distributed.dask.org
   :alt: Documentation Status
.. |Discourse| image:: https://img.shields.io/discourse/users?logo=discourse&server=https%3A%2F%2Fdask.discourse.group
   :alt: Discuss Dask-related things and ask for help
   :target: https://dask.discourse.group
.. |Version Status| image:: https://img.shields.io/pypi/v/distributed.svg
   :target: https://pypi.python.org/pypi/distributed/
.. |NumFOCUS| image:: https://img.shields.io/badge/powered%20by-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A
   :target: https://www.numfocus.org/


# Intel® X86 Encoder Decoder  (Intel® XED)

## Doxygen API manual and source build manual:

https://intelxed.github.io

## Bugs:

https://github.com/intelxed/xed/issues/new

## Abbreviated *GITHUB* building instructions:

```shell
git clone https://github.com/intelxed/xed.git xed
git clone https://github.com/intelxed/mbuild.git mbuild
cd xed
./mfile.py
```

Then get your libxed.a/libxed.so (Linux) or xed.lib (Windows) from the obj directory.
Add " --shared" if you want a shared object build.
Add " install" if you want the headers & libraries put in to a kit in the "kits" directory.
Add "C:/python3/python " before "./mfile.py" if on windows.

## How to build the examples:

There are two options:

1) When building libxed you can also build the examples, from the main directory (above examples):

```shell
./mfile.py examples
```

and the compiled examples will be in obj/examples.

2) Build a compiled "kit" and the build the examples from within the kit:

```shell
./mfile.py install
cd kits
cd <whatever the kit is called>
cd examples
./mfile.py
```

See source build documentation for more information.

## Binary size?

Concerned about large libraries or binaries? There are several options:

1. Consider building with "--limit-strings"
2. Strip the binaries
3. Consider doing an encoder-only or decoder-only build if you only need one or the other.

## Contribute to Intel® XED:

In order to contribute, please:

1. Read the [Contribution Agreement](CONTRIBUTING.md)
2. Sign your commit message to show your agreement
3. Submit a Pull-Request


# ArcGIS API for Python
ArcGIS API for Python is a Python library for working with maps and geospatial data, powered by web GIS. It provides simple and efficient tools for deep learning, sophisticated vector and raster analysis, geocoding, map making, routing and directions, as well as for organizing and managing a GIS with users, groups and information items. In addition to working with your own data, the library enables access to ready to use maps and curated geographic data from Esri and other authoritative sources. It also integrates well with the scientific Python ecosystem and includes rich support for Pandas, Scikit-Learn, Fast.ai, etc. and Jupyter notebook.

To learn more about the API, visit the product page [here](https://developers.arcgis.com/python/). You can get in touch with the developers of the API and other users like you at the community page [here](https://geonet.esri.com/groups/arcgis-python-api/).

## What's included
This SDK repository contains the following items:
* API Reference Documentation. A hosted version of this can be found [here](https://developers.arcgis.com/python/api-reference/).
* Samples as Jupyter Notebooks.
* Guides chapters as Jupyter Notebooks.
* API Reference Documentation can be found [here](https://developers.arcgis.com/python/api-reference/).  Downloads are available for current and previous versions [here](https://developers.arcgis.com/downloads/#python).

You have multiple ways of executing these notebooks as listed below:

 - Execute locally on your computer by installing anaconda and the API. See _Install and set up_ help [Package managers](https://developers.arcgis.com/python/guide/anaconda/)
 - Execute with ArcGIS Pro. See the _Install and set up_ [ArcGIS Pro](https://developers.arcgis.com/python/guide/arcgis-pro/) topic. For additional details, see [Notebooks in ArcGIS Pro](https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/pro-notebooks.htm).
 - Execute with ArcGIS Notebooks, hosted on ArcGIS Online. Checkout [this group](https://www.arcgis.com/home/group.html?id=2464da88f55e45d89aedcae843167f51#overview) with sample notebooks.
 - Execute in a Dockerised environment. See _Install and set up_ help [Installation as Docker image](https://developers.arcgis.com/python/guide/install-and-set-up/#installation-as-a-docker-image)
 - Execute with Binder. See help [here](https://mybinder.org/)

## Issues

Find a bug or want to request a new feature?  Please let us know by submitting an issue.  Thank you!

## Contributing

Anyone and everyone is welcome to contribute. Please see our contribution guideline [here](https://github.com/Esri/arcgis-python-api/wiki/How-to-contribute-samples-and-guide-chapters).

## Licensing
Copyright 2018-2022 Esri

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

A copy of the license is available in the repository's [license.txt](https://github.com/Esri/arcgis-python-api/blob/master/license.txt) file.


botocore
========

|Version| |Python| |License|

A low-level interface to a growing number of Amazon Web Services. The
botocore package is the foundation for the
`AWS CLI <https://github.com/aws/aws-cli>`__ as well as
`boto3 <https://github.com/boto/boto3>`__.

Botocore is maintained and published by `Amazon Web Services`_.

Notices
-------

On 2023-12-13, support was dropped for Python 3.7. This follows the
Python Software Foundation `end of support <https://www.python.org/dev/peps/pep-0537/#lifespan>`__
for the runtime which occurred on 2023-06-27.
For more information, see this `blog post <https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/>`__.

.. _`Amazon Web Services`: https://aws.amazon.com/what-is-aws/
.. |Python| image:: https://img.shields.io/pypi/pyversions/botocore.svg?style=flat
    :target: https://pypi.python.org/pypi/botocore/
    :alt: Python Versions
.. |Version| image:: http://img.shields.io/pypi/v/botocore.svg?style=flat
    :target: https://pypi.python.org/pypi/botocore/
    :alt: Package Version
.. |License| image:: http://img.shields.io/pypi/l/botocore.svg?style=flat
    :target: https://github.com/boto/botocore/blob/develop/LICENSE.txt
    :alt: License

Getting Started
---------------
Assuming that you have Python and ``virtualenv`` installed, set up your environment and install the required dependencies like this or you can install the library using ``pip``:

.. code-block:: sh

    $ git clone https://github.com/boto/botocore.git
    $ cd botocore
    $ virtualenv venv
    ...
    $ . venv/bin/activate
    $ pip install -r requirements.txt
    $ pip install -e .

.. code-block:: sh

    $ pip install botocore

Using Botocore
~~~~~~~~~~~~~~
After installing botocore

Next, set up credentials (in e.g. ``~/.aws/credentials``):

.. code-block:: ini

    [default]
    aws_access_key_id = YOUR_KEY
    aws_secret_access_key = YOUR_SECRET

Then, set up a default region (in e.g. ``~/.aws/config``):

.. code-block:: ini

   [default]
   region=us-east-1

Other credentials configuration method can be found `here <https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html>`__

Then, from a Python interpreter:

.. code-block:: python

    >>> import botocore.session
    >>> session = botocore.session.get_session()
    >>> client = session.create_client('ec2')
    >>> print(client.describe_instances())

Getting Help
------------

We use GitHub issues for tracking bugs and feature requests and have limited
bandwidth to address them. Please use these community resources for getting
help. Please note many of the same resources available for ``boto3`` are
applicable for ``botocore``:

* Ask a question on `Stack Overflow <https://stackoverflow.com/>`__ and tag it with `boto3 <https://stackoverflow.com/questions/tagged/boto3>`__
* Open a support ticket with `AWS Support <https://console.aws.amazon.com/support/home#/>`__
* If it turns out that you may have found a bug, please `open an issue <https://github.com/boto/botocore/issues/new/choose>`__

Contributing
------------

We value feedback and contributions from our community. Whether it's a bug report, new feature, correction, or additional documentation, we welcome your issues and pull requests. Please read through this `CONTRIBUTING <https://github.com/boto/botocore/blob/develop/CONTRIBUTING.rst>`__ document before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your contribution.

Maintenance and Support for SDK Major Versions
----------------------------------------------

Botocore was made generally available on 06/22/2015 and is currently in the full support phase of the availability life cycle.

For information about maintenance and support for SDK major versions and their underlying dependencies, see the following in the AWS SDKs and Tools Reference Guide:

* `AWS SDKs and Tools Maintenance Policy <https://docs.aws.amazon.com/sdkref/latest/guide/maint-policy.html>`__
* `AWS SDKs and Tools Version Support Matrix <https://docs.aws.amazon.com/sdkref/latest/guide/version-support-matrix.html>`__

More Resources
--------------

* `NOTICE <https://github.com/boto/botocore/blob/develop/NOTICE>`__
* `Changelog <https://github.com/boto/botocore/blob/develop/CHANGELOG.rst>`__
* `License <https://github.com/boto/botocore/blob/develop/LICENSE.txt>`__


Alembic is a database migrations tool written by the author
of `SQLAlchemy <http://www.sqlalchemy.org>`_.  A migrations tool
offers the following functionality:

* Can emit ALTER statements to a database in order to change
  the structure of tables and other constructs
* Provides a system whereby "migration scripts" may be constructed;
  each script indicates a particular series of steps that can "upgrade" a
  target database to a new version, and optionally a series of steps that can
  "downgrade" similarly, doing the same steps in reverse.
* Allows the scripts to execute in some sequential manner.

The goals of Alembic are:

* Very open ended and transparent configuration and operation.   A new
  Alembic environment is generated from a set of templates which is selected
  among a set of options when setup first occurs. The templates then deposit a
  series of scripts that define fully how database connectivity is established
  and how migration scripts are invoked; the migration scripts themselves are
  generated from a template within that series of scripts. The scripts can
  then be further customized to define exactly how databases will be
  interacted with and what structure new migration files should take.
* Full support for transactional DDL.   The default scripts ensure that all
  migrations occur within a transaction - for those databases which support
  this (Postgresql, Microsoft SQL Server), migrations can be tested with no
  need to manually undo changes upon failure.
* Minimalist script construction.  Basic operations like renaming
  tables/columns, adding/removing columns, changing column attributes can be
  performed through one line commands like alter_column(), rename_table(),
  add_constraint(). There is no need to recreate full SQLAlchemy Table
  structures for simple operations like these - the functions themselves
  generate minimalist schema structures behind the scenes to achieve the given
  DDL sequence.
* "auto generation" of migrations. While real world migrations are far more
  complex than what can be automatically determined, Alembic can still
  eliminate the initial grunt work in generating new migration directives
  from an altered schema.  The ``--autogenerate`` feature will inspect the
  current status of a database using SQLAlchemy's schema inspection
  capabilities, compare it to the current state of the database model as
  specified in Python, and generate a series of "candidate" migrations,
  rendering them into a new migration script as Python directives. The
  developer then edits the new file, adding additional directives and data
  migrations as needed, to produce a finished migration. Table and column
  level changes can be detected, with constraints and indexes to follow as
  well.
* Full support for migrations generated as SQL scripts.   Those of us who
  work in corporate environments know that direct access to DDL commands on a
  production database is a rare privilege, and DBAs want textual SQL scripts.
  Alembic's usage model and commands are oriented towards being able to run a
  series of migrations into a textual output file as easily as it runs them
  directly to a database. Care must be taken in this mode to not invoke other
  operations that rely upon in-memory SELECTs of rows - Alembic tries to
  provide helper constructs like bulk_insert() to help with data-oriented
  operations that are compatible with script-based DDL.
* Non-linear, dependency-graph versioning.   Scripts are given UUID
  identifiers similarly to a DVCS, and the linkage of one script to the next
  is achieved via human-editable markers within the scripts themselves.
  The structure of a set of migration files is considered as a
  directed-acyclic graph, meaning any migration file can be dependent
  on any other arbitrary set of migration files, or none at
  all.  Through this open-ended system, migration files can be organized
  into branches, multiple roots, and mergepoints, without restriction.
  Commands are provided to produce new branches, roots, and merges of
  branches automatically.
* Provide a library of ALTER constructs that can be used by any SQLAlchemy
  application. The DDL constructs build upon SQLAlchemy's own DDLElement base
  and can be used standalone by any application or script.
* At long last, bring SQLite and its inability to ALTER things into the fold,
  but in such a way that SQLite's very special workflow needs are accommodated
  in an explicit way that makes the most of a bad situation, through the
  concept of a "batch" migration, where multiple changes to a table can
  be batched together to form a series of instructions for a single, subsequent
  "move-and-copy" workflow.   You can even use "move-and-copy" workflow for
  other databases, if you want to recreate a table in the background
  on a busy system.

Documentation and status of Alembic is at https://alembic.sqlalchemy.org/

The SQLAlchemy Project
======================

Alembic is part of the `SQLAlchemy Project <https://www.sqlalchemy.org>`_ and
adheres to the same standards and conventions as the core project.

Development / Bug reporting / Pull requests
___________________________________________

Please refer to the
`SQLAlchemy Community Guide <https://www.sqlalchemy.org/develop.html>`_ for
guidelines on coding and participating in this project.

Code of Conduct
_______________

Above all, SQLAlchemy places great emphasis on polite, thoughtful, and
constructive communication between users and developers.
Please see our current Code of Conduct at
`Code of Conduct <https://www.sqlalchemy.org/codeofconduct.html>`_.

License
=======

Alembic is distributed under the `MIT license
<https://opensource.org/licenses/MIT>`_.


# napari

### multi-dimensional image viewer for python

[![napari on Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/napari/napari/main?urlpath=%2Fdesktop)
[![image.sc forum](https://img.shields.io/badge/dynamic/json.svg?label=forum&url=https%3A%2F%2Fforum.image.sc%2Ftags%2Fnapari.json&query=%24.topic_list.tags.0.topic_count&colorB=brightgreen&suffix=%20topics&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAABPklEQVR42m3SyyqFURTA8Y2BER0TDyExZ+aSPIKUlPIITFzKeQWXwhBlQrmFgUzMMFLKZeguBu5y+//17dP3nc5vuPdee6299gohUYYaDGOyyACq4JmQVoFujOMR77hNfOAGM+hBOQqB9TjHD36xhAa04RCuuXeKOvwHVWIKL9jCK2bRiV284QgL8MwEjAneeo9VNOEaBhzALGtoRy02cIcWhE34jj5YxgW+E5Z4iTPkMYpPLCNY3hdOYEfNbKYdmNngZ1jyEzw7h7AIb3fRTQ95OAZ6yQpGYHMMtOTgouktYwxuXsHgWLLl+4x++Kx1FJrjLTagA77bTPvYgw1rRqY56e+w7GNYsqX6JfPwi7aR+Y5SA+BXtKIRfkfJAYgj14tpOF6+I46c4/cAM3UhM3JxyKsxiOIhH0IO6SH/A1Kb1WBeUjbkAAAAAElFTkSuQmCC)](https://forum.image.sc/tag/napari)
[![License](https://img.shields.io/pypi/l/napari.svg)](https://github.com/napari/napari/raw/main/LICENSE)
[![Build Status](https://api.cirrus-ci.com/github/Napari/napari.svg)](https://cirrus-ci.com/napari/napari)
[![Code coverage](https://codecov.io/gh/napari/napari/branch/main/graph/badge.svg)](https://codecov.io/gh/napari/napari)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/napari.svg)](https://python.org)
[![Python package index](https://img.shields.io/pypi/v/napari.svg)](https://pypi.org/project/napari)
[![Python package index download statistics](https://img.shields.io/pypi/dm/napari.svg)](https://pypistats.org/packages/napari)
[![Development Status](https://img.shields.io/pypi/status/napari.svg)](https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
[![DOI](https://zenodo.org/badge/144513571.svg)](https://zenodo.org/badge/latestdoi/144513571)
[![NEP29](https://raster.shields.io/badge/Follows-NEP29-brightgreen.png)](https://numpy.org/neps/nep-0029-deprecation_policy.html)

**napari** is a fast, interactive, multi-dimensional image viewer for Python. It's designed for browsing, annotating, and analyzing large multi-dimensional images. It's built on top of Qt (for the GUI), vispy (for performant GPU-based rendering), and the scientific Python stack (numpy, scipy).

We're developing **napari** in the open! But the project is in an **alpha** stage, and there will still likely be **breaking changes** with each release. You can follow progress on [this repository](https://github.com/napari/napari), test out new versions as we release them, and contribute ideas and code.

If you want to refer to our documentation, please go to [napari.org](https://www.napari.org). If you want to contribute to it, please refer to the *contributing* section below.

We're working on [tutorials](https://napari.org/tutorials/), but you can also quickly get started by looking below.

## installation

It is recommended to install napari into a virtual environment, like this:

```sh
conda create -y -n napari-env -c conda-forge python=3.9
conda activate napari-env
python -m pip install "napari[all]"
```

If you prefer conda over pip, you can replace the last line with: `conda install -c conda-forge napari pyqt`

See here for the full [installation guide](https://napari.org/tutorials/fundamentals/installation.html).

## simple example

(The examples below require the `scikit-image` package to run. We just use data samples from this package for demonstration purposes. If you change the examples to use your own dataset, you may not need to install this package.)

From inside an IPython shell, you can open up an interactive viewer by calling

```python
from skimage import data
import napari

viewer = napari.view_image(data.cells3d(), channel_axis=1, ndisplay=3)
```

![napari viewer with a multichannel image of cells displayed as two image layers: nuclei and membrane.](https://github.com/napari/docs/blob/main/docs/images/multichannel_cells.png)

To use napari from inside a script, use `napari.run()`:

```python
from skimage import data
import napari

viewer = napari.view_image(data.cells3d(), channel_axis=1, ndisplay=3)
napari.run()  # start the "event loop" and show the viewer
```

## features

Check out the scripts in our [`examples` folder](examples) to see some of the functionality we're developing!

**napari** supports six main different layer types, `Image`, `Labels`, `Points`, `Vectors`, `Shapes`, and `Surface`, each corresponding to a different data type, visualization, and interactivity. You can add multiple layers of different types into the viewer and then start working with them, adjusting their properties.

All our layer types support n-dimensional data and the viewer provides the ability to quickly browse and visualize either 2D or 3D slices of the data.

**napari** also supports bidirectional communication between the viewer and the Python kernel, which is especially useful when launching from jupyter notebooks or when using our built-in console. Using the console allows you to interactively load and save data from the viewer and control all the features of the viewer programmatically.

You can extend **napari** using custom shortcuts, key bindings, and mouse functions.

## tutorials

For more details on how to use `napari` checkout our [tutorials](https://napari.org/tutorials/). These are still a work in progress, but we'll be updating them regularly.

## mission, values, and roadmap

For more information about our plans for `napari` you can read our [mission and values statement](https://napari.org/community/mission_and_values.html), which includes more details on our vision for supporting a plugin ecosystem around napari.
You can see details of [the project roadmap here](https://napari.org/roadmaps/index.html).

## contributing

Contributions are encouraged! Please read our [contributing guide](https://napari.org/developers/contributing.html) to get started. Given that we're in an early stage, you may want to reach out on our [Github Issues](https://github.com/napari/napari/issues) before jumping in.

If you want to contribute or edit to our documentation, please go to [napari/docs](https://github.com/napari/docs).

## code of conduct

`napari` has a [Code of Conduct](https://napari.org/community/code_of_conduct.html) that should be honored by everyone who participates in the `napari` community.

## governance

You can learn more about how the `napari` project is organized and managed from our [governance model](https://napari.org/community/governance.html), which includes information about, and ways to contact the [@napari/steering-council and @napari/core-devs](https://napari.org/community/team.html#current-core-developers).

## citing napari

If you find `napari` useful please cite [this repository](https://github.com/napari/napari) using its DOI as follows:

> napari contributors (2019). napari: a multi-dimensional image viewer for python. [doi:10.5281/zenodo.3555620](https://zenodo.org/record/3555620)

Note this DOI will resolve to all versions of napari. To cite a specific version please find the
DOI of that version on our [zenodo page](https://zenodo.org/record/3555620). The DOI of the latest version is in the badge at the top of this page.

## help

We're a community partner on the [image.sc forum](https://forum.image.sc/tags/napari) and all help and support requests should be posted on the forum with the tag `napari`. We look forward to interacting with you there.

Bug reports should be made on our [github issues](https://github.com/napari/napari/issues/new?template=bug_report.md) using
the bug report template. If you think something isn't working, don't hesitate to reach out - it is probably us and not you!

## institutional and funding partners

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://chanzuckerberg.com/wp-content/themes/czi/img/logo-white.svg">
  <img alt="CZI logo" src="https://chanzuckerberg.com/wp-content/themes/czi/img/logo.svg">
</picture>


.. image:: https://anaconda.org/conda-forge/control/badges/version.svg
   :target: https://anaconda.org/conda-forge/control

.. image:: https://img.shields.io/pypi/v/control.svg
   :target: https://pypi.org/project/control/

.. image:: https://github.com/python-control/python-control/actions/workflows/python-package-conda.yml/badge.svg
   :target: https://github.com/python-control/python-control/actions/workflows/python-package-conda.yml

.. image:: https://github.com/python-control/python-control/actions/workflows/install_examples.yml/badge.svg
   :target: https://github.com/python-control/python-control/actions/workflows/install_examples.yml

.. image:: https://github.com/python-control/python-control/actions/workflows/control-slycot-src.yml/badge.svg
   :target: https://github.com/python-control/python-control/actions/workflows/control-slycot-src.yml

.. image:: https://coveralls.io/repos/python-control/python-control/badge.svg
   :target: https://coveralls.io/r/python-control/python-control

Python Control Systems Library
==============================

The Python Control Systems Library is a Python module that implements basic
operations for analysis and design of feedback control systems.

Have a go now!
--------------
Try out the examples in the examples folder using the binder service.

.. image:: https://mybinder.org/badge_logo.svg
 :target: https://mybinder.org/v2/gh/python-control/python-control/HEAD

The package can also be installed on Google Colab using the commands::

  !pip install control
  import control as ct

Features
--------

- Linear input/output systems in state-space and frequency domain
- Block diagram algebra: serial, parallel, feedback, and other interconnections
- Time response: initial, step, impulse
- Frequency response: Bode, Nyquist, and Nichols plots
- Control analysis: stability, reachability, observability, stability margins, root locus
- Control design: eigenvalue placement, linear quadratic regulator, sisotool, hinfsyn, rootlocus_pid_designer
- Estimator design: linear quadratic estimator (Kalman filter)
- Nonlinear systems: optimization-based control, describing functions, differential flatness

Links
-----

- Project home page: http://python-control.org
- Source code repository: https://github.com/python-control/python-control
- Documentation: http://python-control.readthedocs.org/
- Issue tracker: https://github.com/python-control/python-control/issues
- Mailing list: http://sourceforge.net/p/python-control/mailman/

Dependencies
------------

The package requires numpy, scipy, and matplotlib.  In addition, some routines
use a module called slycot, that is a Python wrapper around some FORTRAN
routines.  Many parts of python-control will work without slycot, but some
functionality is limited or absent, and installation of slycot is recommended
(see below). The Slycot wrapper can be found at:

https://github.com/python-control/Slycot

Installation
============

Conda and conda-forge
---------------------

The easiest way to get started with the Control Systems library is
using `Conda <https://conda.io>`_.

The Control Systems library has packages available using the `conda-forge
<https://conda-forge.org>`_ Conda channel, and as of Slycot version
0.3.4, binaries for that package are available for 64-bit Windows,
OSX, and Linux.

To install both the Control Systems library and Slycot in an existing
conda environment, run::

  conda install -c conda-forge control slycot

Mixing packages from conda-forge and the default conda channel can
sometimes cause problems with dependencies, so it is usually best to
instally NumPy, SciPy, and Matplotlib from conda-forge as well.

Pip
---

To install using pip::

  pip install slycot   # optional; see below
  pip install control

If you install Slycot using pip you'll need a development environment
(e.g., Python development files, C and Fortran compilers).  Pip
installation can be particularly complicated for Windows.

Installing from source
----------------------

To install from source, get the source code of the desired branch or release
from the github repository or archive, unpack, and run from within the
toplevel `python-control` directory::

  pip install .

Article and Citation Information
================================

An `article <https://ieeexplore.ieee.org/abstract/document/9683368>`_ about
the library is available on IEEE Explore. If the Python Control Systems Library helped you in your research, please cite::

  @inproceedings{python-control2021,
    title={The Python Control Systems Library (python-control)},
    author={Fuller, Sawyer and Greiner, Ben and Moore, Jason and
            Murray, Richard and van Paassen, Ren{\'e} and Yorke, Rory},
    booktitle={60th IEEE Conference on Decision and Control (CDC)},
    pages={4875--4881},
    year={2021},
    organization={IEEE}
  }

or the GitHub site: https://github.com/python-control/python-control

Development
===========

Code
----

You can check out the latest version of the source code with the command::

  git clone https://github.com/python-control/python-control.git

Testing
-------

You can run the unit tests with `pytest`_ to make sure that everything is
working correctly.  Inside the source directory, run::

  pytest -v

or to test the installed package::

  pytest --pyargs control -v

.. _pytest: https://docs.pytest.org/

License
-------

This is free software released under the terms of `the BSD 3-Clause
License <http://opensource.org/licenses/BSD-3-Clause>`_.  There is no
warranty; not even for merchantability or fitness for a particular
purpose.  Consult LICENSE for copying conditions.

When code is modified or re-distributed, the LICENSE file should
accompany the code or any subset of it, however small.  As an
alternative, the LICENSE text can be copied within files, if so
desired.

Contributing
------------

Your contributions are welcome!  Simply fork the GitHub repository and send a
`pull request`_.

.. _pull request: https://github.com/python-control/python-control/pulls

Please see the `Developer's Wiki`_ for detailed instructions.

.. _Developer's Wiki: https://github.com/python-control/python-control/wiki


**cocotb** is a coroutine based cosimulation library for writing VHDL and Verilog testbenches in Python.

[![Documentation Status](https://readthedocs.org/projects/cocotb/badge/?version=latest)](https://docs.cocotb.org/en/latest/)
[![CI](https://github.com/cocotb/cocotb/actions/workflows/build-test-dev.yml/badge.svg?branch=master)](https://github.com/cocotb/cocotb/actions/workflows/build-test-dev.yml)
[![PyPI](https://img.shields.io/pypi/dm/cocotb.svg?label=PyPI%20downloads)](https://pypi.org/project/cocotb/)
[![Gitpod Ready-to-Code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/cocotb/cocotb)
[![codecov](https://codecov.io/gh/cocotb/cocotb/branch/master/graph/badge.svg)](https://codecov.io/gh/cocotb/cocotb)

* Read the [documentation](https://docs.cocotb.org)
* Get involved:
  * [Raise a bug / request an enhancement](https://github.com/cocotb/cocotb/issues/new) (Requires a GitHub account)
  * [Join the Gitter chat room](https://gitter.im/cocotb/Lobby)

**Note: The current `master` branch of the cocotb repository is expected to be released as cocotb 2.0, which contains API-breaking changes from previous 1.x releases. Please use the `stable/1.8` branch if you're building cocotb from source, or just [install it from PyPi](https://pypi.org/project/cocotb/).**

## Installation

The current stable version of cocotb requires:

- Python 3.6+
- GNU Make 3+
- An HDL simulator (such as [Icarus Verilog](https://docs.cocotb.org/en/stable/simulator_support.html#icarus-verilog),
[Verilator](https://docs.cocotb.org/en/stable/simulator_support.html#verilator),
[GHDL](https://docs.cocotb.org/en/stable/simulator_support.html#ghdl) or
[other simulator](https://docs.cocotb.org/en/stable/simulator_support.html))

After installing these dependencies, the latest stable version of cocotb can be installed with pip.

```command
pip install cocotb
```

For more details on installation, including prerequisites,
see [the documentation](https://docs.cocotb.org/en/stable/install.html).

For details on how to install the *development* version of cocotb,
see [the preliminary documentation of the future release](https://docs.cocotb.org/en/latest/install_devel.html#install-devel).

## Usage

As a first trivial introduction to cocotb, the following example "tests" a flip-flop.

First, we need a hardware design which we can test. For this example, create a file `dff.sv` with SystemVerilog code for a simple [D flip-flop](https://en.wikipedia.org/wiki/Flip-flop_(electronics)#D_flip-flop). You could also use any other language a [cocotb-supported simulator](https://docs.cocotb.org/en/stable/simulator_support.html) understands, e.g. VHDL.

```systemverilog
// dff.sv

`timescale 1us/1ns

module dff (
    output logic q,
    input logic clk, d
);

always @(posedge clk) begin
    q <= d;
end

endmodule
```

An example of a simple randomized cocotb testbench:

```python
# test_dff.py

import random

import cocotb
from cocotb.clock import Clock
from cocotb.triggers import RisingEdge
from cocotb.types import LogicArray

@cocotb.test()
async def dff_simple_test(dut):
    """Test that d propagates to q"""

    # Assert initial output is unknown
    assert LogicArray(dut.q.value) == LogicArray("X")
    # Set initial input value to prevent it from floating
    dut.d.value = 0

    clock = Clock(dut.clk, 10, units="us")  # Create a 10us period clock on port clk
    # Start the clock. Start it low to avoid issues on the first RisingEdge
    cocotb.start_soon(clock.start(start_high=False))

    # Synchronize with the clock. This will regisiter the initial `d` value
    await RisingEdge(dut.clk)
    expected_val = 0  # Matches initial input value
    for i in range(10):
        val = random.randint(0, 1)
        dut.d.value = val  # Assign the random value val to the input port d
        await RisingEdge(dut.clk)
        assert dut.q.value == expected_val, f"output q was incorrect on the {i}th cycle"
        expected_val = val # Save random value for next RisingEdge

    # Check the final input on the next clock
    await RisingEdge(dut.clk)
    assert dut.q.value == expected_val, "output q was incorrect on the last cycle"
```

A simple Makefile:

```make
# Makefile

TOPLEVEL_LANG = verilog
VERILOG_SOURCES = $(shell pwd)/dff.sv
TOPLEVEL = dff
MODULE = test_dff

include $(shell cocotb-config --makefiles)/Makefile.sim
```

In order to run the test with Icarus Verilog, execute:

```command
make SIM=icarus
```

[![asciicast](https://asciinema.org/a/317220.svg)](https://asciinema.org/a/317220)

For more information please see the [cocotb documentation](https://docs.cocotb.org/)
and [our wiki](https://github.com/cocotb/cocotb/wiki).

## Tutorials, examples and related projects

* the tutorial section [in the official documentation](https://docs.cocotb.org/)
* [cocotb-bus](https://github.com/cocotb/cocotb-bus) for pre-packaged testbenching tools and reusable bus interfaces.
* [cocotb-based USB 1.1 test suite](https://github.com/antmicro/usb-test-suite-build) for FPGA IP, with testbenches for a variety of open source USB cores
* [`cocotb-coverage`](https://github.com/mciepluc/cocotb-coverage), an extension for Functional Coverage and Constrained Randomization
* [`uvm-python`](https://github.com/tpoikela/uvm-python), an almost 1:1 port of UVM 1.2 to Python
* our wiki [on extension modules](https://github.com/cocotb/cocotb/wiki/Further-Resources#extension-modules-cocotbext)
* the list of [GitHub projects depending on cocotb](https://github.com/cocotb/cocotb/network/dependents)


# University Domains and Names Data List & API

Do you need a list of universities and their domain names? You found it!

This package includes a JSON file that contains domains, names and countries of most of the universities of the world.

Example usecases:
- You can create a validation script that checks the email domain.
- You can automatically generate a user's country and university by looking at their emails.

You can use this data source in three ways:

- Use the JSON file as your data source and do whatever you like with your favourite programming language.
- Use free hosted-API.
- Use the tiny Python app to serve a fast API that you can query data.

### 1 - Using the Data Source

The whole data source is located in the `world_universities_and_domains.json` file. It is just a list of dictionaries in the following format:

    [
    	...
    	{
    	    "alpha_two_code": "TR",
    	    "country": "Turkey",
    	    "state-province": null,
    	    "domains": [
    	        "sabanciuniv.edu",
    	        "sabanciuniv.edu.tr"
    	    ],
    	    "name": "Sabanci University",
    	    "web_pages": [
    	        "http://www.sabanciuniv.edu/",
    	        "http://www.sabanciuniv.edu.tr/"
    	    ],
    	},
    	...
    ]

If you want a smaller final payload and only need a subset of countries, run

```bash
filter.py $country1 [Optional: $country2]
```

from the root directory to return

```
filtered_world_universities_and_domains.json
```

NOTE: Some universities use a format like `[user]@[department].[domain]`, but this list only contains the `[domain]` portion.
For example, an email address might be `[student]@cs.usc.edu`, and this list will contain 'usc.edu', the domain for the
University of Southern California. Take this into consideration if using this list for email address validation.

### 2 - Using The Hosted API

This is the easiest method if you're making a small project or just want to discover the data without any hassle.
It is sponsored by [Hipo](http://www.hipolabs.com) and free. If you have a big project, please host it on your own server.

Some example searches:

- http://universities.hipolabs.com
- http://universities.hipolabs.com/search?name=middle
- http://universities.hipolabs.com/search?name=middle&country=turkey

The hosted API uses [university-domains-list-api](https://github.com/Hipo/university-domains-list-api) package.

### 3 - Using the built-in API on your server

You can access the python API via [university-domains-list-api](https://github.com/Hipo/university-domains-list-api)

# Contribution

Please contribute to this list! We need your support to keep this list up-to-date.
Do not hesitate to fix any wrong data. It is extremely easy. Just open a PR, or create an issue.

# Contributors

- Yiğit Güler
- Tuna Vargı
- Patrick Michelberger
- Rasim Demirbay
- Ryan White
- Bilal Arslan
- anwilli5
- Thomas Bauer
- Emin Mastizada
- Jai
- Jimi Ford
- Lars Schwegmann
- Sedat Karancı
- Charles Bedrosian
- Harrison Lo
- mattdfloyd
- Ender Ahmet Yurt
- Enis Behiç Tuysuz
- Syed Zakawat
- Daksh Shah
- Maizer Gomes
- Denys Vitali
- Ary Wibowo
- Matt Floyd
- Joris Boquet
- Konstantin Ladutenko
- Romain Odeval
- remediate
- Errorific
- summerplaybook
- hamedty
- Sedat
- Sotirios Roussis
- majilesh
- Itay Grudev
- luungoc2005
- Ajithkumar Sekar
- Christopher Chen
- Dimitris Karakostas
- Chun Fei Lung
- Mamat Rahmat
- Wisnu Adi Nurcahyo
- jvanstraten
- Ekin Dursun
- Kevin Bohinski
- Lachlan Marnham
- Baptiste Pellarin
- Kelian Baert
- [more](https://github.com/Hipo/university-domains-list/graphs/contributors)

### Created and maintained by [Hipo](http://www.hipolabs.com)


# FuzzBench: Fuzzer Benchmarking As a Service

FuzzBench is a free service that evaluates fuzzers on a wide variety of
real-world benchmarks, at Google scale. The goal of FuzzBench is to make it
painless to rigorously evaluate fuzzing research and make fuzzing research
easier for the community to adopt. We invite members of the research community
to contribute their fuzzers and give us feedback on improving our evaluation
techniques.

FuzzBench provides:

* An easy API for integrating fuzzers.
* Benchmarks from real-world projects. FuzzBench can use any
  [OSS-Fuzz](https://github.com/google/oss-fuzz) project as a benchmark.
* A reporting library that produces reports with graphs and statistical tests
  to help you understand the significance of results.

To participate, submit your fuzzer to run on the FuzzBench platform by following
[our simple guide](
https://google.github.io/fuzzbench/getting-started/).
After your integration is accepted, we will run a large-scale experiment using
your fuzzer and generate a report comparing your fuzzer to others.
See [a sample report](https://www.fuzzbench.com/reports/sample/index.html).

## Overview
<kbd>

![FuzzBench Service diagram](docs/images/FuzzBench-service.png)

</kbd>

## Sample Report

You can view our sample report
[here](https://www.fuzzbench.com/reports/sample/index.html) and
our periodically generated reports
[here](https://www.fuzzbench.com/reports/index.html).
The sample report is generated using 10 fuzzers against 24 real-world
benchmarks, with 20 trials each and over a duration of 24 hours.
The raw data in compressed CSV format can be found at the end of the report.

When analyzing reports, we recommend:
* Checking the strengths and weaknesses of a fuzzer against various benchmarks.
* Looking at aggregate results to understand the overall significance of the
  result.

Please provide feedback on any inaccuracies and potential improvements (such as
integration changes, new benchmarks, etc.) by opening a GitHub issue
[here](https://github.com/google/fuzzbench/issues/new).

## Documentation

Read our [detailed documentation](https://google.github.io/fuzzbench/) to learn
how to use FuzzBench.

## Contacts

Join our [mailing list](https://groups.google.com/forum/#!forum/fuzzbench-users)
for discussions and announcements, or send us a private email at
[fuzzbench@google.com](mailto:fuzzbench@google.com).


> *Tools like TorchIO are a symptom of the maturation of medical AI research using deep learning techniques*.

Jack Clark, Policy Director
at [OpenAI](https://openai.com/) ([link](https://jack-clark.net/2020/03/17/)).

---

            Package

            CI

            Code

            Tutorials

            Community

---

---

        Original

            Random blur

            Random flip

            Random noise

            Random affine transformation

            Random elastic transformation

            Random bias field artifact

            Random motion artifact

            Random spike artifact

            Random ghosting artifact

---

([Queue](https://torchio.readthedocs.io/patches/patch_training.html#queue)
for [patch-based training](https://torchio.readthedocs.io/patches/index.html))

---

TorchIO is a Python package containing a set of tools to efficiently
read, preprocess, sample, augment, and write 3D medical images in deep learning applications
written in [PyTorch](https://pytorch.org/),
including intensity and spatial transforms
for data augmentation and preprocessing.
Transforms include typical computer vision operations
such as random affine transformations and also domain-specific ones such as
simulation of intensity artifacts due to
[MRI magnetic field inhomogeneity](https://mriquestions.com/why-homogeneity.html)
or [k-space motion artifacts](http://proceedings.mlr.press/v102/shaw19a.html).

This package has been greatly inspired by NiftyNet,
[which is not actively maintained anymore](https://github.com/NifTK/NiftyNet/commit/935bf4334cd00fa9f9d50f6a95ddcbfdde4031e0).

## Credits

If you like this repository, please click on Star!

If you use this package for your research, please cite our paper:

[F. Pérez-García, R. Sparks, and S. Ourselin. *TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning*. Computer Methods and Programs in Biomedicine (June 2021), p. 106236. ISSN: 0169-2607.doi:10.1016/j.cmpb.2021.106236.](https://doi.org/10.1016/j.cmpb.2021.106236)

BibTeX entry:

```bibtex
@article{perez-garcia_torchio_2021,
    title = {{TorchIO}: a {Python} library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning},
    journal = {Computer Methods and Programs in Biomedicine},
    pages = {106236},
    year = {2021},
    issn = {0169-2607},
    doi = {https://doi.org/10.1016/j.cmpb.2021.106236},
    url = {https://www.sciencedirect.com/science/article/pii/S0169260721003102},
    author = {P{\'e}rez-Garc{\'i}a, Fernando and Sparks, Rachel and Ourselin, S{\'e}bastien},
}
```

This project is supported by the following institutions:

- [Engineering and Physical Sciences Research Council (EPSRC) & UK Research and Innovation (UKRI)](https://epsrc.ukri.org/)
- [EPSRC Centre for Doctoral Training in Intelligent, Integrated Imaging In Healthcare (i4health)](https://www.ucl.ac.uk/intelligent-imaging-healthcare/) (University College London)
- [Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS)](https://www.ucl.ac.uk/interventional-surgical-sciences/) (University College London)
- [School of Biomedical Engineering & Imaging Sciences (BMEIS)](https://www.kcl.ac.uk/bmeis) (King's College London)

## Getting started

See [Getting started](https://torchio.readthedocs.io/quickstart.html) for
[installation](https://torchio.readthedocs.io/quickstart.html#installation)
instructions
and a [Hello, World!](https://torchio.readthedocs.io/quickstart.html#hello-world)
example.

Longer usage examples can be found in the
[tutorials](https://github.com/fepegar/torchio/blob/main/tutorials/README.md).

All the documentation is hosted on
[Read the Docs](http://torchio.rtfd.io/).

Please
[open a new issue](https://github.com/fepegar/torchio/issues/new/choose)
if you think something is missing.

## Contributors

Thanks goes to all these people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

Fernando Pérez-García
💻 📖

valabregue
🤔 👀 💻 💬 🐛

GFabien
💻 👀 🤔

G.Reguig
💻

Niels Schurink
💻

Ibrahim Hadzic
🐛

ReubenDo
🤔

Julian Klug
🤔

David Völgyes
🤔 💻

Jean-Christophe Fillion-Robin
📖

Suraj Pai
🤔

Ben Darwin
🤔

Oeslle Lucena
🐛

Soumick Chatterjee
💻

neuronflow
📖

Jan Witowski
📖

Derk Mus
📖 💻 🐛

Christian Herz
🐛

Cory Efird
💻 🐛

Esteban Vaca C.
🐛

Ray Phan
🐛

Akis Linardos
🐛 💻

Nina Montana-Brown
📖 🚇

fabien-brulport
🐛

malteekj
🐛

Andres Diaz-Pinto
🐛

Sarthak Pati
📦 📖

GabriellaKamlish
🐛

Tyler Spears
🐛

DaGuT
📖

Xiangyu Zhao
🐛

siahuat0727
📖 🐛

Svdvoort
💻

Albans98
💻

Matthew T. Warkentin
💻

glupol
🐛

ramonemiliani93
📖 🐛 💻

Justus Schock
💻 🐛 🤔 👀

Stefan Milorad Radonjić
🐛

Sajan Gohil
🐛

Ikko Ashimine
📖

laynr
📖

Omar U. Espejel
🔊

James Butler
🐛

res191
🔍

nengwp
🐛 📖

susanveraclarke
🎨

nepersica
🐛

Sebastian Penhouet
🤔

Bigsealion
🐛

Dženan Zukić
👀

vasl12
✅ 🐛

François Rousseau
🐛

snavalm
💻

Jacob Reinhold
💻

Hsu
🐛

snipdome
🐛

SmallY
🐛

guigautier
🤔

AyedSamy
🐛

J. Miguel Valverde
🤔 💻 🐛

José Guilherme Almeida
🤔

Asim Usman
🐛

cbri92
🐛

Markus J. Ankenbrand
🐛

Ziv Yaniv
📖

Luca Lumetti
💻 📖

chagelo
🐛

mueller-franzes
💻 🐛

Abdelwahab Kawafi
🐛

Arthur Masson
🐛 📖

양현식
💻

nicoloesch
💻 🐛

Amund Vedal
📖

Alabamagan
🐛

sbdoherty
📖

Zhack47
🐛

Blake Dewey
📖

Doyeon Kim
🐛

KonoMaxi
🐛

Laurent Chauvin
🐛

Christian Hinge
🐛

zzz123xyz
🐛

Amin Alam
📖

marius-sm
🤔

This project follows the
[all-contributors](https://github.com/all-contributors/all-contributors)
specification. Contributions of any kind welcome!


CommCare HQ
===========

CommCare HQ is a web application for building complex, customizable, frontline worker solutions.

It includes multi-tenant application building, user management, data collection, and reporting.

CommCare HQ apps work with [CommCare mobile](https://github.com/dimagi/commcare-android) and its
bundled [web application platform](https://github.com/dimagi/formplayer).

### Key Components

+ CommCare application builder
+ OpenRosa compliant XForms designer
+ SMS integration
+ Domain/user/mobile worker management
+ XForms data collection
+ Case management
+ Over-the-air (ota) restore of user and cases
+ Integrated web and email reporting

### More Information

+ To try CommCare you can use [this production instance of hosted CommCare](https://www.commcarehq.org/).
+ For setting up a local CommCare HQ developer environment, see [Setting up CommCare HQ for Developers](https://github.com/dimagi/commcare-hq/blob/master/DEV_SETUP.md).
+ For setting up a production CommCare HQ environment, check out [CommCare Cloud](https://commcare-cloud.readthedocs.io/), our toolkit for deploying and maintaining CommCare servers.
+ Additional documentation is available on [ReadTheDocs](https://commcare-hq.readthedocs.io/).
+ We welcome contributions, see [Contributing](CONTRIBUTING.rst) for more.
+ Questions?  Contact the CommCare community at our [forum](https://forum.dimagi.com/).

*CommCare is built by [Dimagi](https://dimagi.com/) and our open-source contributors.*


# Kolibri

[![Python test status](https://github.com/learningequality/kolibri/actions/workflows/tox.yml/badge.svg?branch=develop)](https://github.com/learningequality/kolibri/actions/workflows/tox.yml)
[![JS test status](https://github.com/learningequality/kolibri/actions/workflows/yarn.yml/badge.svg?branch=develop)](https://github.com/learningequality/kolibri/actions/workflows/yarn.yml)
[![Developer docs](https://img.shields.io/badge/docs-dev-blue.svg)](http://kolibri-dev.readthedocs.org/en/develop/)
[![Developer chat](https://img.shields.io/badge/chat-dev-blue.svg)](http://webchat.freenode.net?channels=%23kolibri)
[![PyPI](https://img.shields.io/pypi/v/kolibri.svg?color=blue)](https://pypi.org/project/kolibri/)
[![Demo](https://img.shields.io/badge/demo-online-blue.svg)](http://kolibridemo.learningequality.org/)
[![User docs](https://img.shields.io/badge/docs-user-blue.svg)](http://kolibri.readthedocs.org/en/latest/)
[![Discourse topics](https://img.shields.io/discourse/https/community.learningequality.org/topics.svg?color=blue)](https://community.learningequality.org/)

This repository is for software developers wishing to contribute to Kolibri. If you are looking for help installing, configuring and using Kolibri, please refer to the [User Guide](https://kolibri.readthedocs.io/).

## What is Kolibri?

[Kolibri Learning Platform](https://learningequality.org/kolibri/): the offline-first platform for teaching and learning with technology without requiring the Internet.

Developed with and for the community by [Learning Equality](https://learningequality.org/).

## How can I use it?

Kolibri is [available for download](https://learningequality.org/download/) from our website.

## How do I get help or give feedback?

You can ask questions, make suggestions, and report issues in the [community forums](https://community.learningequality.org/).

If you have found a bug and are comfortable using Github and Markdown, you can create a [Github issue](https://github.com/learningequality/kolibri/issues) following the instructions in the issue template.

<!-- Also update CONTRIBUTING.md (duplicate) -->
## How can I contribute?

1. 📙 **Skim through the [Developer documentation](https://kolibri-dev.readthedocs.io)** to understand where to refer later on.
2. 💻 **Follow the [Getting started](https://kolibri-dev.readthedocs.io/en/develop/getting_started.html) to set up your development server.** Some of the [How To Guides](https://kolibri-dev.readthedocs.io/en/develop/howtos/index.html#howtos) may be handy too.
3. 🔍 **Search for issues tagged as [help wanted](https://github.com/learningequality/kolibri/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22+no%3Aassignee) or [good first issue](https://github.com/learningequality/kolibri/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22+no%3Aassignee).**
4. 🗣 **Ask us for an assignment in the comments of an issue you've chosen.** Please request assignment of a reasonable amount of issues at a time. Once you finish your current issue or two, you are welcome to ask for more.

**❓ Where to ask questions**

- For anything development related, refer to the [Developer documentation](https://kolibri-dev.readthedocs.io) at first. Some answers may already be there.
- For questions related to a specific issue or assignment requests, use the corresponding issue's comments section.
- Visit [GitHub Discussions](https://github.com/learningequality/kolibri/discussions) to ask about anything related to contributing or to troubleshoot development server issues.

**👥 How to connect**

- We encourage you to visit [GitHub Discussions](https://github.com/learningequality/kolibri/discussions) to connect with the Learning Equality team as well as with other contributors.
- If you'd like to contribute on a regular basis, we are happy to invite you to our open-source community Slack channel. Get in touch with us at info@learningequality.org to receive an invitation.

---

🕖 Please allow us a few days to reply to your comments. If you don't hear from us within a week, reach out via [GitHub Discussions](https://github.com/learningequality/kolibri/discussions).

As soon as you open a pull request, it may take us a week or two to review it as we're a small team. We appreciate your contribution and will provide feedback.

---

*Thank you for your interest in contributing! Learning Equality was founded by volunteers dedicated to helping make educational materials more accessible to those in need, and every contribution makes a difference.*


Chrome Platform Status
==================

### Mission

[chromestatus.com](https://chromestatus.com/) is the official tool used for for tracking feature launches in Blink (the browser engine that powers Chrome and many other web browsers).  This tool guides feature owners through our [launch process](https://www.chromium.org/blink/launching-features/) and serves as a primary source for developer information that then ripples throughout the web developer ecosystem.

### Get the code

For a one-click setup that leverages devcontainers, check out the devcontainer
[README](.devcontainer/README.md). Otherwise, to continue setting up locally:

    git clone https://github.com/GoogleChrome/chromium-dashboard

### Installation
1. Install gcloud and needed components:
    1.  Before you begin, make sure that you have a java JRE (version 8 or greater) installed. JRE is required to use the DataStore Emulator and [openapi-generator-cli](https://github.com/OpenAPITools/openapi-generator-cli).
    1. [Google App Engine SDK for Python](https://cloud.google.com/appengine/docs/standard/python3/setting-up-environment). Make sure to select Python 3.
    1. `gcloud init`
    1. `gcloud components install cloud-datastore-emulator`
    1. `gcloud components install beta`
1. Install other developer tools commands
    1. node and npm.
    1. Gulp: `npm install --global gulp-cli`
    1. Python virtual environment: `sudo apt install python3.11-venv`
1. We recommend using an older node version, e.g. node 18
    1. Use `node -v` to check the default node version
    2. `nvm use 18` to switch to node 18
1. `cd chromium-dashboard`
1. Install JS an python dependencies: `npm run setup`
    1. Note: Whenever we make changes to package.json or requirements.txt, you will need to run `npm run clean-setup`.

If you encounter any error during the installation process, the section **Notes** (later in this README.md) may help.

### Developing

To start the main server and the notifier backend, run:

```bash
npm start
```
Then visit `http://localhost:8080/`.

To start front end code watching (sass, js lint check, babel, minify files), run

```bash
npm run watch
```

To run lint & lit-analyzer:

```bash
npm run lint
```

To run unit tests:

```bash
npm test
```

This will start a local datastore emulator, run unit tests, and then shut down the emulator.

To update test_html_rendering.html, modify the `test_html_rendering` method in
the corresponding test file, uncomment the line that looks like:

```python
 # TESTDATA.make_golden(template_text, 'test_html_rendering.html')
```

Then run the test again (and maybe one more time), and then you can revert
your change of the test files.

To run the Playwright visual tests (aka end-to-end tests), the command to use is:
```bash
npm run pwtests --workspace=playwright
```

If there are errors, they will be displayed in the console.
If you need to update any of the screenshot images, you will see differences in
the `packages/playwright/test-results` directory, and if they look correct,
then you can update _all_ images for all tests with:
```bash
npm run pwtests-update --workspace=playwright
```

The updated images are also added to the __screenshots__ directory.  Images that
did not need to be updated do not show up as having been changed.
If you change the test file names, or the test method names, or the screenshot
image file names, then new files will be generated, and you will need to manually delete the old files.  You could simply delete all screenshots and
update all, but that will take a fairly long time.

You can update images for just one test file by adding `--filename=some_pwtest.js`
to the `pwtests-update` command.  The `some_pwtest.js` name does not need to be
a full path.

If there are error reported by the GitHub CI playwright action, you can look at
the error log, but if the problem is a difference in some of the images, you
should probably download the artifact `.zip` file containing all the differences.

There is some additional information for developers in developer-documentation.md.

### Origin Trials
To test the functionality of this application locally that interacts with data from the Origin Trials API, an API key will need to be acquired. To do this, run the following command:

```bash
npm run dev-ot-key
```

Note: *Only developers with access to the cr-status-staging GCP project will be able to successfully run this command. If you need to test this and you don't have access, open an issue.*

**Notes**

- If you get an error saying `No module named protobuf` or `No module named six` or `No module named enum` , try installing them locally with `pip install six enum34 protobuf`.

- When installing the GAE SDK, make sure to get the version for python 3.

- If you run the server locally, and then you are disconnected from your terminial window, the jobs might remain running which will prevent you from starting the server again.  To work around this, use `npm run stop-emulator; npm stop`.  Or, use `ps aux | grep gunicorn` and/or `ps aux | grep emulator`, then use the unix `kill -9` command to terminate those jobs.

- If you need to test or debug anything to do with dependencies, you can get a clean start by running `npm run clean-setup`.

- Occasionally, the Google Cloud CLI will requires an update, which will cause a failure when trying to run the development server with `npm start`. An unrelated error message `Failed to connect to localhost port 15606 after 0 ms: Connection refused` will appear. Running the `gcloud components update` command will update as needed and resolve this issue.

#### Blink components

Chromestatus currently gets the list of Blink components from the file `hack_components.py`.

#### Seed the blink component owners

Visit http://localhost:8080/admin/blink/populate_blink to see the list of Blink component owners.

#### Debugging / settings

[`settings.py`](https://github.com/GoogleChrome/chromium-dashboard/blob/master/settings.py) contains a list
of globals for debugging and running the site locally.

### Deploying

If you have uncommitted local changes, the appengine version name will end with `-tainted`.
It is OK to test on staging with tainted versions, but everything should be committed
(and thus not tainted) before staging a version that can later be pushed to prod.

**Note** you need to have admin privileges on the `cr-status-staging` and `cr-status`
cloud projects to be able to deploy the site.

Run the npm target:

    npm run staging

Open the [Google Developer
Console for the staging site](https://console.cloud.google.com/appengine/versions?project=cr-status-staging)
and flip to the new version by selecting from the list and clicking *MIGRATE TRAFFIC*. Make sure to do this for both the 'default' service as well as for the 'notifier' service.

Alternatively, run `npm run staging-rc` to  upload the same code to a version named `rc` for "Release candidate".  This is the only version that you can test using Google Sign-In at `https://rc-dot-cr-status-staging.appspot.com`.

If manual testing on the staging server looks good, then repeat the same steps to deploy to prod:

    npm run deploy

Open the [Google Developer
Console for the production site](https://console.cloud.google.com/appengine/versions?project=cr-status)

The production site should only have versions that match versions on staging.

### LICENSE

Copyright (c) 2013-2022 Google Inc. All rights reserved.

Apache2 License.

[![Analytics](https://ga-beacon.appspot.com/UA-39048143-2/GoogleChrome/chromium-dashboard/README)](https://github.com/igrigorik/ga-beacon)


[![Build Status](https://github.com/mlrun/mlrun/actions/workflows/build.yaml/badge.svg?branch=development)](https://github.com/mlrun/mlrun/actions/workflows/build.yaml?query=branch%3Adevelopment)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![PyPI version fury.io](https://badge.fury.io/py/mlrun.svg)](https://pypi.python.org/pypi/mlrun/)
[![Documentation](https://readthedocs.org/projects/mlrun/badge/?version=latest)](https://mlrun.readthedocs.io/en/latest/?badge=latest)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
![GitHub commit activity](https://img.shields.io/github/commit-activity/w/mlrun/mlrun)
![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/mlrun/mlrun?sort=semver)
[![Join MLOps Live](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://mlopslive.slack.com)

# Using MLRun

MLRun is an open MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications, significantly reducing engineering efforts, time to production, and computation resources.
With MLRun, you can choose any IDE on your local machine or on the cloud. MLRun breaks the silos between data, ML, software, and DevOps/MLOps teams, enabling collaboration and fast continuous improvements.

Get started with MLRun [**Tutorials and Examples**](https://docs.mlrun.org/en/latest/tutorials/index.html), [**Installation and setup guide**](https://docs.mlrun.org/en/latest/install.html), or read about [**MLRun Architecture**](https://docs.mlrun.org/en/latest/architecture.html).

This page explains how MLRun addresses the [**MLOps Tasks**](#mlops-tasks) and the [**MLRun core components**](#core-components).

## MLOps tasks

The [**MLOps development workflow**](https://docs.mlrun.org/en/latest/mlops-dev-flow.html) section describes the different tasks and stages in detail.
MLRun can be used to automate and orchestrate all the different tasks or just specific tasks (and integrate them with what you have already deployed).

### Project management and CI/CD automation

In MLRun the assets, metadata, and services (data, functions, jobs, artifacts, models, secrets, etc.) are organized into projects.
Projects can be imported/exported as a whole, mapped to git repositories or IDE projects (in PyCharm, VSCode, etc.), which enables versioning, collaboration, and CI/CD.
Project access can be restricted to a set of users and roles.

See: **Docs:** [Projects and Automation](https://docs.mlrun.org/en/latest/projects/project.html), [CI/CD Integration](https://docs.mlrun.org/en/latest/projects/ci-integration.html), **Tutorials:** [Quick start](https://docs.mlrun.org/en/latest/tutorials/01-mlrun-basics.html), [Automated ML Pipeline](https://docs.mlrun.org/en/latest/tutorials/04-pipeline.html), **Video:** [Quick start](https://youtu.be/xI8KVGLlj7Q).

### Ingest and process data

MLRun provides abstract interfaces to various offline and online [**data sources**](https://docs.mlrun.org/en/latest/store/datastore.html), supports batch or realtime data processing at scale, data lineage and versioning, structured and unstructured data, and more.
In addition, the MLRun [**Feature Store**](https://docs.mlrun.org/en/latest/feature-store/feature-store.html) automates the collection, transformation, storage, catalog, serving, and monitoring of data features across the ML lifecycle and enables feature reuse and sharing.

See: **Docs:** [Ingest and process data](https://docs.mlrun.org/en/latest/data-prep/index.html), [Feature Store](https://docs.mlrun.org/en/latest/feature-store/feature-store.html), [Data & Artifacts](https://docs.mlrun.org/en/latest/concepts/data.html); **Tutorials:** [Quick start](https://docs.mlrun.org/en/latest/tutorials/01-mlrun-basics.html), [Feature Store](https://docs.mlrun.org/en/latest/feature-store/basic-demo.html).

### Develop and train models

MLRun allows you to easily build ML pipelines that take data from various sources or the Feature Store and process it, train models at scale with multiple parameters, test models, tracks each experiments, register, version and deploy models, etc. MLRun provides scalable built-in or custom model training services, integrate with any framework and can work with 3rd party training/auto-ML services. You can also bring your own pre-trained model and use it in the pipeline.

See: **Docs:** [Develop and train models](https://docs.mlrun.org/en/latest/development/index.html), [Model Training and Tracking](https://docs.mlrun.org/en/latest/development/model-training-tracking.html), [Batch Runs and Workflows](https://docs.mlrun.org/en/latest/concepts/runs-workflows.html); **Tutorials:** [Train, compare, and register models](https://docs.mlrun.org/en/latest/tutorials/02-model-training.html), [Automated ML Pipeline](https://docs.mlrun.org/en/latest/tutorials/04-pipeline.html); **Video:** [Train and compare models](https://youtu.be/bZgBsmLMdQo).

### Deploy models and applications

MLRun rapidly deploys and manages production-grade real-time or batch application pipelines using elastic and resilient serverless functions. MLRun addresses the entire ML application: intercepting application/user requests, running data processing tasks, inferencing using one or more models, driving actions, and integrating with the application logic.

See: **Docs:** [Deploy models and applications](https://docs.mlrun.org/en/latest/deployment/index.html), [Realtime Pipelines](https://docs.mlrun.org/en/latest/serving/serving-graph.html), [Batch Inference](https://docs.mlrun.org/en/latest/deployment/batch_inference.html), **Tutorials:** [Realtime Serving](https://docs.mlrun.org/en/latest/tutorials/03-model-serving.html), [Batch Inference](https://docs.mlrun.org/en/latest/tutorials/07-batch-infer.html), [Advanced Pipeline](https://docs.mlrun.org/en/latest/tutorials/07-batch-infer.html); **Video:** [Serving pre-trained models](https://youtu.be/OUjOus4dZfw).

### Monitor and alert

Observability is built into the different MLRun objects (data, functions, jobs, models, pipelines, etc.), eliminating the need for complex integrations and code instrumentation. With MLRun, you can observe the application/model resource usage and model behavior (drift, performance, etc.), define custom app metrics, and trigger alerts or retraining jobs.

See: **Docs:** [Monitor and alert](https://docs.mlrun.org/en/latest/monitoring/index.html), [Model Monitoring Overview](https://docs.mlrun.org/en/latest/monitoring/model-monitoring-deployment.html), **Tutorials:** [Model Monitoring & Drift Detection](https://docs.mlrun.org/en/latest/tutorials/05-model-monitoring.html).

## MLRun core components

MLRun includes the following major components:

[**Project Management:**](https://docs.mlrun.org/en/latest/projects/project.html) A service (API, SDK, DB, UI) that manages the different project assets (data, functions, jobs, workflows, secrets, etc.) and provides central control and metadata layer.

[**Functions:**](https://docs.mlrun.org/en/latest/runtimes/functions.html) automatically deployed software package with one or more methods and runtime-specific attributes (such as image, libraries, command, arguments, resources, etc.).

[**Data & Artifacts:**](https://docs.mlrun.org/en/latest/concepts/data.html) Glueless connectivity to various data sources, metadata management, catalog, and versioning for structures/unstructured artifacts.

[**Feature Store:**](https://docs.mlrun.org/en/latest/feature-store/feature-store.html) automatically collects, prepares, catalogs, and serves production data features for development (offline) and real-time (online) deployment using minimal engineering effort.

[**Batch Runs & Workflows:**](https://docs.mlrun.org/en/latest/concepts/runs-workflows.html) Execute one or more functions with specific parameters and collect, track, and compare all their results and artifacts.

[**Real-Time Serving Pipeline:**](https://docs.mlrun.org/en/latest/serving/serving-graph.html) Rapid deployment of scalable data and ML pipelines using real-time serverless technology, including API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc.

[**Real-Time monitoring:**](https://docs.mlrun.org/en/latest/monitoring/index.html) monitors data, models, resources, and production components and provides a feedback loop for exploring production data, identifying drift, alerting on anomalies or data quality issues, triggering retraining jobs, measuring business impact, etc.