.. image:: https://github.com/buildtesters/buildtest/blob/devel/logos/BuildTest_Primary_Center_4x3.png
   :width: 400

.. |docs| image:: https://readthedocs.org/projects/buildtest/badge/?version=devel
    :alt: Documentation Status
    :scale: 100%
    :target: https://readthedocs.org/projects/buildtest/builds/

.. |license| image:: https://img.shields.io/github/license/buildtesters/buildtest.svg
   :target: https://github.com/buildtesters/buildtest/blob/devel/LICENSE

.. |release| image:: https://img.shields.io/github/v/release/buildtesters/buildtest.svg
   :target: https://github.com/buildtesters/buildtest/releases

.. |issues| image:: https://img.shields.io/github/issues/buildtesters/buildtest.svg
    :target: https://github.com/buildtesters/buildtest/issues

.. |open_pr| image:: https://img.shields.io/github/issues-pr/buildtesters/buildtest.svg
    :target: https://github.com/buildtesters/buildtest/pulls

.. |commit_activity_yearly| image:: https://img.shields.io/github/commit-activity/y/buildtesters/buildtest.svg
   :target: https://github.com/buildtesters/buildtest/pulse

.. |commit_activity_monthly| image:: https://img.shields.io/github/commit-activity/m/buildtesters/buildtest.svg
   :target: https://github.com/buildtesters/buildtest/pulse

.. |core_infrastructure| image:: https://bestpractices.coreinfrastructure.org/projects/3469/badge
   :target: https://bestpractices.coreinfrastructure.org/projects/3469

.. |codecov| image:: https://codecov.io/gh/buildtesters/buildtest/branch/devel/graph/badge.svg
    :target: https://codecov.io/gh/buildtesters/buildtest

.. |codefactor| image:: https://www.codefactor.io/repository/github/buildtesters/buildtest/badge
    :target: https://www.codefactor.io/repository/github/buildtesters/buildtest
    :alt: CodeFactor

.. |black| image:: https://img.shields.io/badge/code%20style-black-000000.svg
    :target: https://github.com/psf/black

.. |checkurls| image:: https://github.com/buildtesters/buildtest/workflows/Check%20URLs/badge.svg
    :target: https://github.com/buildtesters/buildtest/actions

.. |style| image:: https://github.com/buildtesters/buildtest/workflows/Style%20Checks/badge.svg
    :target: https://github.com/buildtesters/buildtest/actions

.. |isort| image:: https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336
    :target: https://pycqa.github.io/isort/

.. |installation| image:: https://github.com/buildtesters/buildtest/workflows/installation/badge.svg
   :target: https://github.com/buildtesters/buildtest/actions

.. |regressiontest| image:: https://github.com/buildtesters/buildtest/workflows/regressiontest/badge.svg
    :target: https://github.com/buildtesters/buildtest/actions

.. |nightly_regression_test| image:: https://github.com/buildtesters/buildtest/actions/workflows/nightly_regression.yml/badge.svg
    :target: https://github.com/buildtesters/buildtest/actions

.. |gh_pages_devel| image:: https://github.com/buildtesters/buildtest/workflows/Schema%20Documentation/badge.svg
    :target: https://github.com/buildtesters/buildtest/actions

.. |zenodo| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.3967143.svg
   :target: https://doi.org/10.5281/zenodo.3967143

buildtest
---------

| |license| |docs| |codecov| |release| |installation| |regressiontest| |nightly_regression_test| |gh_pages_devel| |checkurls| |codefactor| |style|  |black| |isort| |issues| |open_pr| |commit_activity_yearly| |commit_activity_monthly| |core_infrastructure| |zenodo|
=======

`buildtest <https://buildtest.rtfd.io/>`_ is a testing framework that automates build/execution of tests which allows HPC facilities to develop
test for their system. In buildtest, tests are written as test recipes called **Buildspecs** which is a `YAML <https://yaml.org/>`_
used by buildtest for generating shell-scripts and run them on your system. Buildtest will validate buildspecs using `jsonschema <https://json-schema.org/>`_
which helps define structure of buildspec when writing test. buildtest can support batch submission to job schedulers including: `IBM Spectrum LSF <https://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_welcome/lsf_welcome.html>`_,
`Slurm <https://slurm.schedmd.com/>`_, `PBS <https://www.openpbs.org/>`_ and `Cobalt <https://trac.mcs.anl.gov/projects/cobalt>`_ .

If you want to learn more about buildtest please read `Summary of Buildtest <https://buildtest.readthedocs.io/en/latest/what_is_buildtest.html>`_.

Installation
--------------

Installing buildtest, is relatively easy. Just clone this repo and source the setup script::

    git clone https://github.com/buildtesters/buildtest.git
    cd buildtest
    source setup.sh

For more details see `Installing buildtest <https://buildtest.readthedocs.io/en/latest/installing_buildtest.html>`_.

Schema Development
-------------------

The schemas are found in top-level folder `buildtest/schemas/ <https://github.com/buildtesters/buildtest/tree/devel/buildtest/schemas>`_
and published via Github Pages at https://buildtesters.github.io/buildtest/. Each schema has a unique URI defined
by `$id <https://json-schema.org/understanding-json-schema/structuring.html#the-id-property>`_.

For any issues with schema, please create an `issue <https://github.com/buildtesters/buildtest/issues/new/choose>`_ in buildtest.

References
------------

- Documentation: http://buildtest.rtfd.io/

- Schema Docs: https://buildtesters.github.io/buildtest/

- ReadTheDocs: https://readthedocs.org/projects/buildtest/

- CodeCov: https://codecov.io/gh/buildtesters/buildtest

- CodeFactor: https://www.codefactor.io/repository/github/buildtesters/buildtest

- Snyk: https://app.snyk.io/org/buildtesters/

Documentation
-------------

buildtest `documentation <http://buildtest.readthedocs.io/en/latest/>`_  is your
source for getting help with buildtest. If you get stuck check out the
`current issues <https://github.com/buildtesters/buildtest/issues>`_ to see
if you face similar issue. If all else fails please create an `issue <https://github.com/buildtesters/buildtest/issues/new/choose>`_.

Source Code
------------

buildtest source code is under `buildtest <https://github.com/buildtesters/buildtest/tree/devel/buildtest>`_
directory found in the root of this repository. The documentation pages are located in
`docs <https://github.com/buildtesters/buildtest/tree/devel/docs>`_ folder
which consist of `Makefile <https://github.com/buildtesters/buildtest/blob/devel/docs/Makefile>`_ and
`conf.py <https://github.com/buildtesters/buildtest/blob/devel/docs/conf.py>`_ to build the sphinx project along with documentation pages in
ReStructuredText (rst). The regression test are found in top-level directory
named `tests <https://github.com/buildtesters/buildtest/tree/devel/tests>`_ and the test suite is run via `pytest <https://docs.pytest.org/en/stable/>`_.

Slack
------

Slack is the **preferred** method for communication and user support with buildtest.
If you haven't joined slack click the `link <https://communityinviter.com/apps/hpcbuildtest/buildtest-slack-invitation>`_
to register your account. Once you have an account you can access slack channel at https://hpcbuildtest.slack.com and post
your questions in **#general** channel.

Contributing Back
-------------------

We would love to get your feedback and contribution, for more details see
`contribution guide <https://buildtest.readthedocs.io/en/latest/contributing.html>`_.

Author
-------

buildtest was founded by `Shahzeb Siddiqui <https://github.com/shahzebsiddiqui>`_.

LICENSE
--------

buildtest is released under the MIT License. See
`LICENSE <https://github.com/buildtesters/buildtest/blob/master/LICENSE>`_ for more details.


---

    Getting Started
      •
    API Documentation
      •
    Getting In Touch (GitHub Discussions)

  <img alt="Beta" src="https://img.shields.io/badge/status-beta-informational?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAIRlWElmTU0AKgAAAAgABQESAAMAAAABAAEAAAEaAAUAAAABAAAASgEbAAUAAAABAAAAUgEoAAMAAAABAAIAAIdpAAQAAAABAAAAWgAAAAAAAACQAAAAAQAAAJAAAAABAAOgAQADAAAAAQABAACgAgAEAAAAAQAAABigAwAEAAAAAQAAABgAAAAA8A2UOAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAAVlpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IlhNUCBDb3JlIDUuNC4wIj4KICAgPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4KICAgICAgPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIKICAgICAgICAgICAgeG1sbnM6dGlmZj0iaHR0cDovL25zLmFkb2JlLmNvbS90aWZmLzEuMC8iPgogICAgICAgICA8dGlmZjpPcmllbnRhdGlvbj4xPC90aWZmOk9yaWVudGF0aW9uPgogICAgICA8L3JkZjpEZXNjcmlwdGlvbj4KICAgPC9yZGY6UkRGPgo8L3g6eG1wbWV0YT4KTMInWQAABK5JREFUSA2dVm1sFEUYfmd2b/f2Pkqghn5eEQWKrRgjpkYgpoRCLC0oxV5apAiGUDEpJvwxEQ2raWPU+Kf8INU/RtEedwTCR9tYPloxGNJYTTQUwYqJ1aNpaLH3sXu3t7vjvFevpSqt7eSyM+/czvM8877PzB3APBoLgoDLsNePF56LBwqa07EKlDGg84CcWsI4CEbhNnDpAd951lXE2NkiNknCCTLv4HtzZuvPm1C/IKv4oDNXqNDHragety2XVzjECZsJARuBMyRzJrh1O0gQwLXuxofxsPSj4hG8fMLQo7bl9JJD8XZfC1E5yWFOMtd07dvX5kDwg6+2++Chq8txHGtfPoAp0gOFmhYoNFkHjn2TNUmrwRdna7W1QSkU8hvbGk4uThLrapaiLA2E6QY4u/lS9ItHfvJkxYsTMVtnAJLipYIWtVrcdX+8+b8IVnPl/R81prbuPZ1jpYw+0aEUGSkdFsgyBIaFTXCm6nyaxMtJ4n+TeDhJzGqZtQZcuYDgqDwDbqb0JF9oRpIG1Oea3bC1Y6N3x/WV8Zh83emhCs++hlaghDw+8w5UlYKq2lU7Pl8IkvS9KDqXmKmEwdMppVPKwGSEilmyAwJhRwWcq7wYC6z4wZ1rrEoMWxecdOjZWXeAQClBcYDN3NwVwD9pGwqUSyQgclcmxpNJqCuwLmDh3WtvPqXdlt+6Oz70HPGDNSNBee/EOen+rGbEFqDENBPDbtdCp0ukPANmzO0QQJYUpyS5IJJI3Hqt4maS+EB3199ozm8EDU/6fVNU2dQpdx3ZnKzeFXyaUTiasEV/gZMzJMjr3Z+WvAdQ+hs/zw9savimxUntDSaBdZ2f+Idbm1rlNY8esFffBit9HtK5/MejsrJVxikOXlb1Ukir2X+Rbdkd1KG2Ixfn2Ql4JRmELnYK9mEM8G36fAA3xEQ89fxXihC8q+sAKi9jhHxNqagY2hiaYgRCm0f0QP7H4Fp11LSXiuBY2aYFlh0DeDIVVFUJQn5rCnpiNI2gvLxHnASn9DIVHJJlm5rXvQAGEo4zvKq2w5G1NxENN7jrft1oxMdekETjxdH2Z3x+VTVYsPb+O0C/9/auN6v2hNZw5b2UOmSbG5/rkC3LBA+1PdxFxORjxpQ81GcxKc+ybVjEBvUJvaGJ7p7n5A5KSwe4AzkasA+crmzFtowoIVTiLjANm8GDsrWW35ScI3JY8Urv83tnkF8JR0yLvEt2hO/0qNyy3Jb3YKeHeHeLeOuVLRpNF+pkf85OW7/zJxWdXsbsKBUk2TC0BCPwMq5Q/CPvaJFkNS/1l1qUPe+uH3oD59erYGI/Y4sce6KaXYElAIOLt+0O3t2+/xJDF1XvOlWGC1W1B8VMszbGfOvT5qaRRAIFK3BCO164nZ0uYLH2YjNN8thXS2v2BK9gTfD7jHVxzHr4roOlEvYYz9QIz+Vl/sLDXInsctFsXjqIRnO2ZO387lxmIboLDZCJ59KLFliNIgh9ipt6tLg9SihpRPDO1ia5byw7de1aCQmF5geOQtK509rzfdwxaKOIq+73AvwCC5/5fcV4vo3+3LpMdtWHh0ywsJC/ZGoCb8/9D8F/ifgLLl8S8QWfU8cAAAAASUVORK5CYII=">

    Contributing
      •
    Examples

---

# OpenTelemetry Python Contrib

The Python auto-instrumentation libraries for [OpenTelemetry](https://opentelemetry.io/) (per [OTEP 0001](https://github.com/open-telemetry/oteps/blob/main/text/0001-telemetry-without-manual-instrumentation.md))

## Index

* [Installation](#installation)
* [Releasing](#releasing)
  * [Releasing a package as `1.0` stable](#releasing-a-package-as-10-stable)
* [Contributing](#contributing)
* [Running Tests Locally](#running-tests-locally)
* [Thanks to all the people who already contributed](#thanks-to-all-the-people-who-already-contributed)

## Installation

This repository includes installable packages for each instrumented library. Libraries that produce telemetry data should only depend on `opentelemetry-api`,
and defer the choice of the SDK to the application developer. Applications may
depend on `opentelemetry-sdk` or another package that implements the API.

**Please note** that these libraries are currently in _beta_, and shouldn't
generally be used in production environments.

Unless explicitly stated otherwise, any instrumentation here for a particular library is not developed or maintained by the authors of such library.

The
[`instrumentation/`](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation)
directory includes OpenTelemetry instrumentation packages, which can be installed
separately as:

```sh
pip install opentelemetry-instrumentation-{integration}
```

To install the development versions of these packages instead, clone or fork
this repo and do an [editable
install](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs):

```sh
pip install -e ./instrumentation/opentelemetry-instrumentation-{integration}
```

## Releasing

Maintainers release new versions of the packages in `opentelemetry-python-contrib` on a monthly cadence. See [releases](https://github.com/open-telemetry/opentelemetry-python-contrib/releases) for all previous releases.

Contributions that enhance OTel for Python are welcome to be hosted upstream for the benefit of group collaboration. Maintainers will look for things like good documentation, good unit tests, and in general their own confidence when deciding to release a package with the stability guarantees that are implied with a `1.0` release.

To resolve this, members of the community are encouraged to commit to becoming a CODEOWNER for packages in `-contrib` that they feel experienced enough to maintain. CODEOWNERS can then follow the checklist below to release `-contrib` packages as 1.0 stable:

### Releasing a package as `1.0` stable

To release a package as `1.0` stable, the package:

- SHOULD have a CODEOWNER. To become one, submit an issue and explain why you meet the responsibilities found in [CODEOWNERS](.github/CODEOWNERS).
- MUST have unit tests that cover all supported versions of the instrumented library.
  - e.g. Instrumentation packages might use different techniques to instrument different major versions of python packages
- MUST have clear documentation for non-obvious usages of the package
  - e.g. If an instrumentation package uses flags, a token as context, or parameters that are not typical of the `BaseInstrumentor` class, these are documented
- After the release of `1.0`, a CODEOWNER may no longer feel like they have the bandwidth to meet the responsibilities of maintaining the package. That's not a problem at all, life happens! However, if that is the case, we ask that the CODEOWNER please raise an issue indicating that they would like to be removed as a CODEOWNER so that they don't get pinged on future PRs. Ultimately, we hope to use that issue to find a new CODEOWNER.

## Semantic Convention status of instrumentations

In our efforts to maintain optimal user experience and prevent breaking changes for transitioning into stable semantic conventions, OpenTelemetry Python is adopting the [semantic convention migration plan](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/http/migration-guide.md) for several instrumentations. Currently this plan is only being adopted for HTTP-related instrumentations, but will eventually cover all types. Please refer to the `semconv status` column of the [instrumentation README](instrumentation/README.md) of the current status of instrumentations' semantic conventions. The possible values are `experimental`, `stable` and `migration` referring to [status](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.31.0/specification/document-status.md#lifecycle-status) of that particular semantic convention. `Migration` refers to an instrumentation that currently supports the migration plan.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md)

We meet weekly on Thursday at 9AM PT. The meeting is subject to change depending on contributors' availability. Check the [OpenTelemetry community calendar](https://calendar.google.com/calendar/embed?src=c_2bf73e3b6b530da4babd444e72b76a6ad893a5c3f43cf40467abc7a9a897f977%40group.calendar.google.com) for specific dates and for the Zoom link.

Meeting notes are available as a public [Google doc](https://docs.google.com/document/d/1CIMGoIOZ-c3-igzbd6_Pnxx1SjAkjwqoYSUWxPY8XIs/edit). For edit access, get in touch on [GitHub Discussions](https://github.com/open-telemetry/opentelemetry-python/discussions).

Approvers ([@open-telemetry/python-approvers](https://github.com/orgs/open-telemetry/teams/python-approvers)):

- [Aaron Abbott](https://github.com/aabmass), Google
- [Jeremy Voss](https://github.com/jeremydvoss), Microsoft
- [Owais Lone](https://github.com/owais), Splunk
- [Pablo Collins](https://github.com/pmcollins), Splunk
- [Riccardo Magliocchetti](https://github.com/xrmx), Elastic
- [Sanket Mehta](https://github.com/sanketmehta28), Cisco
- [Srikanth Chekuri](https://github.com/srikanthccv), signoz.io
- [Tammy Baylis](https://github.com/tammy-baylis-swi), SolarWinds

Emeritus Approvers:

- [Ashutosh Goel](https://github.com/ashu658), Cisco
- [Héctor Hernández](https://github.com/hectorhdzg), Microsoft
- [Nikolay Sokolik](https://github.com/oxeye-nikolay), Oxeye
- [Nikolay Sokolik](https://github.com/nikosokolik), Oxeye
- [Nathaniel Ruiz Nowell](https://github.com/NathanielRN), AWS

*Find more about the approver role in [community repository](https://github.com/open-telemetry/community/blob/main/community-membership.md#approver).*

Maintainers ([@open-telemetry/python-maintainers](https://github.com/orgs/open-telemetry/teams/python-maintainers)):

- [Diego Hurtado](https://github.com/ocelotl), Lightstep
- [Leighton Chen](https://github.com/lzchen), Microsoft
- [Shalev Roda](https://github.com/shalevr), Cisco

Emeritus Maintainers:

- [Alex Boten](https://github.com/codeboten), Lightstep
- [Owais Lone](https://github.com/owais), Splunk
- [Yusuke Tsutsumi](https://github.com/toumorokoshi), Google

*Find more about the maintainer role in [community repository](https://github.com/open-telemetry/community/blob/main/community-membership.md#maintainer).*

## Running Tests Locally

1. Go to your Contrib repo directory. `cd ~/git/opentelemetry-python-contrib`.
2. Create a virtual env in your Contrib repo directory. `python3 -m venv my_test_venv`.
3. Activate your virtual env. `source my_test_venv/bin/activate`.
4. Make sure you have `tox` installed. `pip install tox`.
5. Run tests for a package. (e.g. `tox -e test-instrumentation-flask`.)

### Thanks to all the people who already contributed


# The JWST Quicklook Application (`JWQL`)

[![Current Release](https://img.shields.io/github/release/spacetelescope/jwql.svg)](https://github.com/spacetelescope/jwql/releases/latest/)
[![PyPI - License](https://img.shields.io/pypi/l/Django.svg)](https://github.com/spacetelescope/jwql/blob/master/LICENSE)
[![Python](https://img.shields.io/badge/Python-3.7-blue.svg)](https://www.python.org/)
[![Build Status](https://github.com/spacetelescope/jwql/workflows/JWQL%20CI/badge.svg)](https://github.com/spacetelescope/jwql/actions)
[![Documentation Status](https://readthedocs.org/projects/jwql/badge/?version=latest)](https://jwql.readthedocs.io/en/latest/?badge=latest)
[![STScI](https://img.shields.io/badge/powered%20by-STScI-blue.svg?colorA=707170&colorB=3e8ddd&style=flat)](http://www.stsci.edu)
[![DOI](https://zenodo.org/badge/109727729.svg)](https://zenodo.org/badge/latestdoi/109727729)
[![codecov](https://codecov.io/gh/spacetelescope/jwql/branch/develop/graph/badge.svg)](https://codecov.io/gh/spacetelescope/jwql)

The JWST Quicklook Application (`JWQL`) is a database-driven web application and automation framework for use by the JWST instrument teams to monitor and trend the health, stability, and performance of the JWST instruments.  The system is comprised of the following:
1. A network file system that stores all uncalibrated and calibrated data products on disk in a centrally-located area, accessible to instrument team members (i.e. the MAST data cache)
2. A relational database that stores observational metadata allowing for data discovery via relational queries (MAST database API).
3. A software library that provides tools to support an automation framework in which to build automated instrument monitoring routines.
4. A web application that allows users to visually inspect new and archival JWST data as well as instrument-specific monitoring and performance results.

Official API documentation can be found on [ReadTheDocs](https://jwql.readthedocs.io)

The `jwql` application is available at [https://jwql.stsci.edu](https://jwql.stsci.edu).  Please note that the application is currently restricted to specific JWST instrument team members.

## Installation for Users

To install `jwql`, simply use `pip`:

```
pip install jwql
```

The section below describes a more detailed installation for users that wish to contribute to the `jwql` repository.

## Installation for Contributors

Getting `jwql` up and running on your own computer requires four steps, detailed below:
1. Cloning the GitHub repository
2. Installing the `conda`environment
3. Installing the python package
4. Setting up the configuration file

### Prerequisites

It is highly suggested that contributors have a working installation of `anaconda` or `miniconda` for Python 3.10+.  Downloads and installation instructions are  available here:

- [Miniconda](https://conda.io/miniconda.html)
- [Anaconda](https://www.continuum.io/downloads)

Requirements for contributing to the `jwql` package will be included in the `jwql` `conda` environment, which is included in our installation instructions below. Further package requirements will be provided for `jwql` by a `pyproject.toml` file included in the repository.

### Clone the `jwql` repo

You first need to clone the current version of `jwql`. The simplest way to do this is to go to the directory you want your copy of the repository to be in and clone the repository there. Once you are in the directory you can do the following:

```
git clone https://github.com/spacetelescope/jwql.git
cd jwql
```

or, if you would rather use `SSH` instead of `https`, type
```
git clone git@github.com:spacetelescope/jwql.git
cd jwql
```
instead, and then proceed as stated.

### Environment Installation

Following the download of the `jwql` repository, contributors can then install the `jwql` `conda` environment via the environment yaml file, which contains all of the dependencies for the project. First, if necessary, [install `conda`](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html). Next, ensure that your version of `conda` is up to date:

```
conda update conda
```

Next, activate the `base` or `root` environment (depending on your version of `conda`):

```
source activate base/root
```

**Note:** If you have added a step activating conda to your default terminal/shell (e.g. the `.bashrc`, `.zshrc`, or `.profile` file) then you don't need to do the above step.

Lastly, create the `jwql` environment via one of the `environment.yml` files (currently `environment_python_3.9.yml`, for python 3.9, and `environment_python_3.10.yml`, for python 3.10, are supported by `jwql`):

```
conda env create -f environment_python_3.10.yml
```

or

```
conda env create -f environment_python_3.11.yml
```

### Configuration File

Much of the `jwql` software depends on the existence of a `config.json` file within the `jwql` directory.  This file contains data that may be unique to users and/or contain sensitive information.  Please see the [Config File wiki page](https://github.com/spacetelescope/jwql/wiki/Config-file) for instructions on how to provide this file.

## Citation

If you use `JWQL` for work/research presented in a publication (whether directly,
or as a dependency to another package), we recommend and encourage the following acknowledgment:

```
  This research made use of the open source Python package 'jwql' (Bourque et al, 2020).
```

where (Bourque et al, 2020) is a citation of the Zenodo record available using the DOI badge above. By using the `Export` box in the lower right corner of the Zenodo page, you can export the citation in the format most convenient for you.

## Software Contributions

There are two current pages to review before you begin contributing to the `jwql` development. The first is our [style guide](https://github.com/spacetelescope/jwql/blob/main/style_guide/README.md) and the second is our [suggested git workflow page](https://github.com/spacetelescope/jwql/wiki/git-&-GitHub-workflow-for-contributing), which contains an in-depth explanation of the workflow.

Contributors are also encouraged to check out the [Checklist for Contributors Guide](https://github.com/spacetelescope/jwql/wiki/Checklist-for-Contributors-and-Reviewers-of-Pull-Requests) to ensure the pull request contains all of the necessary changes.

The following is a bare-bones example of a best work flow for contributing to the project:

1. Create a fork off of the `spacetelescope` `jwql` repository.
2. Make a local clone of your fork.
3. Ensure your personal fork is [pointing `upstream` properly](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/configuring-a-remote-for-a-fork).
4. Create a branch on that personal fork.
5. Make your software changes.
6. Push that branch to your personal GitHub repository (i.e. `origin`).
7. On the `spacetelescope` `jwql` repository, create a pull request that merges the branch into `spacetelescope:develop`.
8. Assign a reviewer from the team for the pull request.
9. Iterate with the reviewer over any needed changes until the reviewer accepts and merges your branch.
10. Delete your local copy of your branch.

## Issue Reporting / Feature Requests

Users who wish to report an issue or request a new feature may do so through the following channels:

1. Submit a new issue on GitHub (preferred method): https://github.com/spacetelescope/jwql/issues
2. Submit a new ticket on Jira: https://jira.stsci.edu/projects/JWQL/

## Code of Conduct

Users and contributors to the `jwql` repository should adhere to the [Code of Conduct](https://github.com/spacetelescope/jwql/blob/main/CODE_OF_CONDUCT.md).  Any issues or violations pertaining to the Code of Conduct should be brought to the attention of a `jwql` team member or to `jwql@stsci.edu`.

## Questions

Any questions about the `jwql` project or its software can be directed to `jwql@stsci.edu`.

## Current Development Team
- Bryan Hilbert (Project Manager, INS) [@bilhbert4](https://github.com/bhilbert4)
- Mees Fix (Technical Lead, INS) [@mfixstsci](https://github.com/mfixstsci)
- Misty Cracraft (INS) [@cracraft](https://github.com/cracraft)
- Mike Engesser (INS) [@mengesser](https://github.com/mengesser)
- Maria Pena-Guerrero [@penaguerrero](https://github.com/penaguerrero)
- Ben Sunnquist (INS) [@bsunnquist](https://github.com/bsunnquist)
- Brian York (INS) [@york-stsci](https://github.com/york-stsci)
- Bradley Sappington (INS) [@bradleysappington](https://github.com/bradleysappington)
- Melanie Clarke (INS) [@melanieclarke](https://github.com/melanieclarke)

## Past Development Team Members
- Matthew Bourque (INS) [@bourque](https://github.com/bourque)
- Lauren Chambers (INS) [@laurenmarietta](https://github.com/laurenmarietta)
- Joe Filippazzo (INS) [@hover2pi](https://github.com/hover2pi)
- Graham Kanarek (INS) [@gkanarek](https://github.com/gkanarek)
- Teagan King (INS) [@tnking97](https://github.com/tnking97)
- Sara Ogaz (DMD) [@SaOgaz](https://github.com/SaOgaz)
- Catherine Martlin (INS) [@catherine-martlin](https://github.com/catherine-martlin)
- Johannes Sahlmann (INS) [@Johannes-Sahlmann](https://github.com/johannes-sahlmann)
- Shannon Osborne (INS) [@shanosborne](https://github.com/shanosborne)

## Acknowledgments:
- Faith Abney (DMD)
- Joshua Alexander (DMD) [@obviousrebel](https://github.com/obviousrebel)
- Anastasia Alexov (DMD)
- Sara Anderson (DMD)
- Tracy Beck (INS)
- Francesca Boffi (INS) [@frboffi](https://github.com/frboffi)
- Clara Brasseur (DMD) [@ceb8](https://github.com/ceb8)
- Matthew Burger (DMD)
- Steven Crawford (DMD) [@stscicrawford](https://github.com/stscicrawford)
- James Davies (DMD) [@jdavies-st](https://github.com/jdavies-st)
- Rosa Diaz (INS) [@rizeladiaz](https://github.com/rizeladiaz)
- Van Dixon (INS)
- Larry Doering (ITSD)
- Tom Donaldson (DMD) [@tomdonaldson](https://github.com/tomdonaldson)
- Kim DuPrie (DMD)
- Jonathan Eisenhamer (DMD) [@stscieisenhamer](https://github.com/stscieisenhamer)
- Ben Falk (DMD) [@falkben](https://github.com/falkben)
- Ann Feild (OPO)
- Mike Fox (DSMO) [@mfox22](https://github.com/mfox22)
- Scott Friedman (INS)
- Alex Fullerton (INS) [@awfullerton](https://github.com/awfullerton)
- Macarena Garcia Marin (INS)
- Lisa Gardner (DMD)
- Vera Gibbs (ITSD)
- Catherine Gosmeyer (INS) [@cgosmeyer](https://github.com/cgosmeyer)
- Phil Grant (ITSD)
- Dean Hines (INS)
- Sherie Holfeltz (INS) [@stholfeltz](https://github.com/stholfeltz)
- Joe Hunkeler (DMD) [@jhunkeler](https://github.com/jhunkeler)
- Catherine Kaleida (DMD) [@ckaleida](https://github.com/ckaleida)
- Deborah Kenny (DMD)
- Jenn Kotler (DMD) [@jenneh](https://github.com/jenneh)
- Daniel Kühbacher (Goddard) [@DanielKuebi](https://github.com/DanielKuebi)
- Mark Kyprianou (DMD) [@mkyp](https://github.com/mkyp)
- Stephanie La Massa (INS)
- Matthew Lallo (INS)
- Karen Levay (DMD)
- Crystal Mannfolk (SCOPE) [@cmannfolk](https://github.com/cmannfolk)
- Greg Masci (ITSD)
- Jacob Matuskey (DMD) [@jmatuskey](https://github.com/jmatuskey)
- Margaret Meixner (INS)
- Christain Mesh (DMD) [@cam72cam](https://github.com/cam72cam)
- Prem Mishra (ITSD)
- Don Mueller (ITSD)
- Maria Antonia Nieto-Santisteban (SEITO)
- Brian O'Sullivan (INS)
- Joe Pollizzi (JWSTMO)
- Lee Quick (DMD)
- Anupinder Rai (ITSD)
- Matt Rendina (DMD) [@rendinam](https://github.com/rendinam)
- Massimo Robberto (INS) [@mrobberto](https://github.com/mrobberto)
- Mary Romelfanger (DMD)
- Elena Sabbi (INS)
- Bernie Shiao (DMD)
- Matthew Sienkiewicz (ITSD)
- Arfon Smith (DSMO) [@arfon](https://github.com/arfon)
- Linda Smith (INS)
- Patrick Taylor (ITSD)
- Dave Unger (ITSD)
- Jeff Valenti (JWSTMO) [@JeffValenti](https://github.com/JeffValenti)
- Jeff Wagner (ITSD)
- Thomas Walker (ITSD)
- Geoff Wallace (DMD)
- Lara Wilkinson (OPO)
- Alex Yermolaev (ITSD) [@alexyermolaev](https://github.com/alexyermolaev)
- Joe Zahn (ITSD)


[![Build Status](https://circleci.com/gh/DemocracyClub/UK-Polling-Stations.svg?style=svg)](https://circleci.com/gh/DemocracyClub/UK-Polling-Stations) [![Coverage Status](https://coveralls.io/repos/github/DemocracyClub/UK-Polling-Stations/badge.svg)](https://coveralls.io/github/DemocracyClub/UK-Polling-Stations) ![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)

# UK-Polling-Stations

This is a work in progress project that needs help in a number of ways:

1. Importing the data we have collected from councils ([See Below](https://github.com/DemocracyClub/UK-Polling-Stations#importing-the-data-we-have-from-councils))
2. If you are a developer (python, django, frontend, etc) or designer, we need help making the site itself.
3. If you are interested in helping us gather this data, or if you know a lot about the strange world of the UK geographic system.

If you are interested in helping out in any way at all, please contact sym@democracyclub.org.uk

## Getting Started

### Python
UK-Polling-Stations requires python 3.10

### Install system dependencies
UK-Polling-Stations requires Python 3.10, Postgres, PostGIS, libgeos, GDAL, Node JS and NPM.

On Mac OSX, run:
```
brew install postgresql
brew install postgis
brew install geos
brew install gdal
brew install node
```

From a clean install of Ubuntu 22.04 (Bionic):
```
sudo apt-get install postgresql-14 postgresql-server-dev-all python-psycopg2 python3-dev postgis postgresql-14-postgis-3 libxml2-dev libxslt1-dev nodejs npm

sudo npm install -g npm@latest-6
```

For other linux distributions, see [here](https://docs.djangoproject.com/en/2.2/ref/contrib/gis/install/geolibs/) for details on installing geospatial libraries for use with Django.

### Install python dependencies

- Local Dev without CDK libs
```
pip install -r requirements/base.txt -r requirements/testing.txt -r requirements/local.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt cdk/lambdas/wdiv-s3-trigger/requirements/testing.txt
```

- Local Dev with CDK libs
```
pip install -r requirements/base.txt -r requirements/testing.txt -r requirements/local.txt -r requirements/cdk.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt
```

- Just Running Tests
```
pip install -r requirements/base.txt -r requirements/testing.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt
```

- Update requirements
Bump version in relevant requirements file, e.g. ```base.in``` then run ```python -m invoke requirements --upgrade```.

### Install front-end dependencies
```
npm install
```

### Install testing python dependencies
```
pip install -r requirements/testing.txt
```

### Install testing system dependencies
We have a suite of end-to-end integration tests. We use [Playwright](https://playwright.dev/python/)
with `pytest` to run these.

To set up playwright, after running `pip install -r requirements/testing.txt`,
run `playwright install`. This will download the browser biniaries required
for your system.

The Playwright tests will run in a headless browser as part of a normal `pytest` run.
You may want to run the Playwright suite in headed mode for debugging purposes. To do that, run the following:

`pytest polling_stations/apps/data_finder/tests/playwright/ --headed`

### Create local config
```
cp polling_stations/settings/local.example.py polling_stations/settings/local.py
```

### Create database
```
sudo -u postgres createdb polling_stations
sudo -u postgres createuser dc -P -s
sudo -u postgres psql polling_stations
psql (9.3.6)
Type "help" for help.

polling_stations=# CREATE EXTENSION postgis;
CREATE EXTENSION
polling_stations=#
```

### Update local.py
Be sure that your database user name matches
To check, run:
`psql postgres`
then:
`\l`
### Run migrations
```
python manage.py migrate
```

### Import initial data

#### Import ONSPD

For development purposes, you can use the ONSPD for geocoding. Grab the latest release from https://geoportal.statistics.gov.uk/search?collection=Dataset&sort=-modified&tags=PRD_ONSPD unzip the data and import it using:

```
python manage.py import_onspd /path/to/data
```

N.B. Path is to data _directory_ which contains the csv, not the csv itself.

#### Import Councils

```
python manage.py import_councils
```

#### Import some Polling District/Station data

For development purposes, you will need to seed your database with some data.
We depend on Ordnance Survey Addressbase Plus, which is not publicly available.
To allow open source contributions we have prepared the sample data Ordnance Survey
provide (https://www.ordnancesurvey.co.uk/business-government/products/addressbase-plus#sample-data)
ready for import, and some sample import scripts for testing.

To prepare your database run the following commands:

```
./manage.py import_cleaned_addresses test_data/addressbase/
./manage.py create_uprn_council_lookup
./manage.py import_uprn_council_lookup uprn-to-councils.csv
```

You can clean up the look up with:

```
rm uprn-to-councils.csv
```

And finally you can import some dummy data with:

```
./manage.py import_fake_teignbridge
./manage.py import_fake_exeter
```

## Importing the data we have from councils

Each council that has unimported data has a Github Issue with the [Data Import](https://github.com/DemocracyClub/UK-Polling-Stations/issues?q=is%3Aissue+is%3Aopen+label%3A%22Data+Import%22) label.

We make a Django `manage.py` command in the data_importers app for each council which imports the raw data.
If you are interested in helping the project by writing an import script, see the issues tagged [recommended for beginners](https://github.com/DemocracyClub/UK-Polling-Stations/issues?q=is%3Aissue+is%3Aopen+label%3A%22recommended+for+beginners%22) for more info.

## Welsh Language translations
```
django-admin makemessages -l cy --ignore='env*'
./manage.py compilemessages
```

## Install git hooks

If you like you can use the commit hooks defined in `.pre-commit-config.yaml`. Run `pre-commit install && pre-commit install -t pre-push`.

## Custom council domains
If a user requests access to the council uploader using a custom domain, the domain should be added to the [`CUSTOM_DOMAINS` setting](https://github.com/DemocracyClub/UK-Polling-Stations/blob/master/polling_stations/settings/constants/uploads.py).
To remove a custom domain, follow these steps:
1) Delete the entry from the CUSTOM_DOMAIN list in `polling_stations/settings/constants/uploads.py`
2) In the admin panel, delete the UserCouncil object from the bottom of the Council profile page

## Election season
As part of preparations for upcoming elections, `NEXT_CHARISMATIC_ELECTION_DATES` will need to be updated in
`settings/constants/elections.py`. This var takes a ["YYYY-MM-DD",] format.

To show messaging about the 2022 GB voter ID legislation, update `SHOW_GB_ID_MESSAGING` to `True` in `settings/constants/elections.py`.

## API

This project has an API, however it's not designed for public consumption.

To authenticate against the API you need to pass a token in the `auth_token` GET parameter.

The main API endpoints are `postcode` and `address`. These endpoints return polling station information for the given location.

In turn, they request the EveryElection API.

```mermaid

sequenceDiagram
    AggregatorAPI ->> WDIV: Call from Aggregator API
    WDIV ->> EE: WDIV calls EE to see if there are elections
    EE ->> WDIV: Election data returned
    WDIV ->> AggregatorAPI: Return polling station and election data to AggregatorAPI

```
This workflow is the same for the `postcode` and `address` endpoints.

The documentation for the AggregatorAPI should explain the main concepts for this API.

### API development documentation

We use [`drf-spectacular`](https://github.com/tfranzel/drf-spectacular) to document the API.

The documentation is inline with the code, meaning that comments and annotations should be enough
in most case to understand what's going on.  `drf-spectacular` comes with some views for
`Redoc` and `Swagger`. To enable these, set `ENABLE_API_DOCS=True` in `local.py` and visit `http://localhost:8000/api/swagger/` or `http://localhost:8000/api/docs/`.

## Adding tokens

There are two types of token:

* `READ_ONLY_API_AUTH_TOKENS`
* `SUPERUSER_API_AUTH_TOKENS`

Both are lists that are defined in settings. To add a token, add a string to the correct list, or set environment variables as comma seperated values.

Superuser API tokens are designed for the Trigger function that posts information about newly updated files back to the uploads app.

Read only tokens are for the workflow outlined above.


devlib
======

``devlib`` exposes an interface for interacting with and collecting
measurements from a variety of devices (such as mobile phones, tablets and
development boards) running a Linux-based operating system.

Installation
------------

::

        sudo -H pip install devlib

Dependencies
------------

``devlib`` should install all dependencies automatically, however if you run
into issues please ensure you are using that latest version of pip.

On some systems there may additional steps required to install the dependency
``paramiko`` please consult the `module documentation <http://www.paramiko.org/installing.html>`_
for more information.

Usage
-----

Please refer  to the "Overview" section of the `documentation <http://devlib.readthedocs.io/en/latest/>`_.

License
-------

This package is distributed under `Apache v2.0 License <http://www.apache.org/licenses/LICENSE-2.0>`_.

Feedback, Contrubutions and Support
-----------------------------------

- Please use the GitHub Issue Tracker associated with this repository for
  feedback.
- ARM licensees may contact ARM directly via their partner managers.
- We welcome code contributions via GitHub Pull requests. Please try to
  stick to the style in the rest of the code for your contributions.


..
    Copyright Swiss Data Science Center (SDSC).
    A partnership between École Polytechnique Fédérale de Lausanne (EPFL) and
    Eidgenössische Technische Hochschule Zürich (ETHZ).

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.

======================================
 Renku Python Library, CLI and Service
======================================

.. image:: https://github.com/SwissDataScienceCenter/renku-python/workflows/Test,%20Integration%20Tests%20and%20Deploy/badge.svg
   :target: https://github.com/SwissDataScienceCenter/renku-python/actions?query=workflow%3A%22Test%2C+Integration+Tests+and+Deploy%22+branch%3Amaster

.. image:: https://img.shields.io/coveralls/SwissDataScienceCenter/renku-python.svg
   :target: https://coveralls.io/r/SwissDataScienceCenter/renku-python

.. image:: https://img.shields.io/github/tag/SwissDataScienceCenter/renku-python.svg
   :target: https://github.com/SwissDataScienceCenter/renku-python/releases

.. image:: https://img.shields.io/pypi/dm/renku.svg
   :target: https://pypi.python.org/pypi/renku

.. image:: http://readthedocs.org/projects/renku-python/badge/?version=latest
   :target: http://renku-python.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status

.. image:: https://img.shields.io/github/license/SwissDataScienceCenter/renku-python.svg
   :target: https://github.com/SwissDataScienceCenter/renku-python/blob/master/LICENSE

A Python library for the `Renku collaborative data science platform
<https://github.com/SwissDataScienceCenter/renku>`_. It includes a CLI and SDK
for end-users as well as a service backend. It provides functionality for the
creation and management of projects and datasets, and simple utilities to
capture data provenance while performing analysis tasks.

**NOTE**:
   ``renku-python`` is the python library and core service for Renku - it *does
   not* start the Renku platform itself - for that, refer to the Renku docs on
   `running the platform
   <https://renku.readthedocs.io/en/latest/developer/setup.html>`_.

Renku for Users
===============

Installation
------------

.. _installation-reference:

Renku releases and development versions are available from `PyPI
<https://pypi.org/project/renku/>`_. You can install it using any tool that
knows how to handle PyPI packages. Our recommendation is to use `:code:pipx
<https://github.com/pipxproject/pipx>`_.

.. note::

   We do not officially support Windows at this moment. The way Windows
   handles paths and symlinks interferes with some Renku functionality.
   We recommend using the Windows Subsystem for Linux (WSL) to use Renku
   on Windows.

Prerequisites
~~~~~~~~~~~~~

Renku depends on Git under the hood, so make sure that you have Git
`installed on your system <https://git-scm.com/downloads>`_.

Renku also offers support to store large files in `Git LFS
<https://git-lfs.github.com/>`_, which is used by default and should be
installed on your system. If you do not wish to use Git LFS, you can run
Renku commands with the `-S` flag, as in `renku -S <command>`.  More
information on Git LFS usage in renku can be found in the `Data in Renku
<https://renku.readthedocs.io/en/latest/user/data.html>`_ section of the docs.

Renku uses CWL to execute recorded workflows when calling `renku update`
or `renku rerun`. CWL depends on NodeJs to execute the workflows, so installing
`NodeJs <https://nodejs.org/en/download/package-manager/>`_ is required if
you want to use those features.

For development of the service, `Docker <https://docker.com>`_ is recommended.

.. _pipx-before-reference:

``pipx``
~~~~~~~~
.. _pipx-after-reference:

First, `install pipx <https://github.com/pipxproject/pipx#install-pipx>`_
and make sure that the ``$PATH`` is correctly configured.

::

    $ python3 -m pip install --user pipx
    $ python3 -m pipx ensurepath

Once ``pipx`` is installed use following command to install ``renku``.

::

    $ pipx install renku
    $ which renku
    ~/.local/bin/renku

``pipx`` installs Renku into its own virtual environment, making sure that it
does not pollute any other packages or versions that you may have already
installed.

.. note::

    If you install Renku as a dependency in a virtual environment and the
    environment is active, your shell will default to the version installed
    in the virtual environment, *not* the version installed by ``pipx``.

To install a development release:

::

    $ pipx install --pip-args pre renku

.. _pip-before-reference:

``pip``
~~~~~~~
.. _pip-after-reference:

::

    $ pip install renku

The latest development versions are available on PyPI or from the Git
repository:

::

    $ pip install --pre renku
    # - OR -
    $ pip install -e git+https://github.com/SwissDataScienceCenter/renku-python.git#egg=renku

Use following installation steps based on your operating system and preferences
if you would like to work with the command line interface and you do not need
the Python library to be importable.

.. _windows-before-reference:

Windows
~~~~~~~
.. _windows-after-reference:

.. note::

    We don't officially support Windows yet, but Renku works well in the Windows Subsystem for Linux (WSL).
    As such, the following can be regarded as a best effort description on how to get started with Renku on Windows.

Renku can be run using the Windows Subsystem for Linux (WSL). To install the WSL, please follow the
`official instructions <https://docs.microsoft.com/en-us/windows/wsl/install-win10#manual-installation-steps>`__.

We recommend you use the Ubuntu 20.04 image in the WSL when you get to that step of the installation.

Once WSL is installed, launch the WSL terminal and install the packages required by Renku with:

::

    $ sudo apt-get update && sudo apt-get install git python3 python3-pip python3-venv pipx

Since Ubuntu has an older version of git LFS installed by default which is known to have some bugs when cloning
repositories, we recommend you manually install the newest version by following
`these instructions <https://github.com/git-lfs/git-lfs/wiki/Installation#debian-and-ubuntu>`__.

Once all the requirements are installed, you can install Renku normally by running:

::

    $ pipx install renku
    $ pipx ensurepath

After this, Renku is ready to use. You can access your Windows in the various mount points in
``/mnt/`` and you can execute Windows executables (e.g. ``\*.exe``) as usual directly from the
WSL (so ``renku run myexecutable.exe`` will work as expected).

.. _docker-before-reference:

Docker
~~~~~~
.. _docker-after-reference:

The containerized version of the CLI can be launched using Docker command.

::

    $ docker run -it -v "$PWD":"$PWD" -w="$PWD" renku/renku-python renku

It makes sure your current directory is mounted to the same place in the
container.

CLI Example
-----------

Initialize a Renku project:

::

    $ mkdir -p ~/temp/my-renku-project
    $ cd ~/temp/my-renku-project
    $ renku init

Create a dataset and add data to it:

::

    $ renku dataset create my-dataset
    $ renku dataset add my-dataset https://raw.githubusercontent.com/SwissDataScienceCenter/renku-python/master/README.rst

Run an analysis:

::

    $ renku run --name my-workflow -- wc < data/my-dataset/README.rst > wc_readme

Trace the data provenance:

::

    $ renku workflow visualize wc_readme

These are the basics, but there is much more that Renku allows you to do with
your data analysis workflows. The full documentation will soon be available
at: https://renku-python.readthedocs.io/

Renku as a Service
==================

This repository includes a ``renku-core`` RPC service written as a `Flask
<https://flask.palletsprojects.com>`_ application that provides (almost) all of
the functionality of the Renku CLI. This is used to provide one of the backends
for the `RenkuLab <https://renkulab.io>`_ web UI. The service can be deployed in
production as a Helm chart (see `helm-chart <./helm-chart/README.rst>`_.

Deploying locally
-----------------

To test the service functionality you can deploy it quickly and easily using
``docker-compose up`` [docker-compose](https://pypi.org/project/docker-compose/).
Make sure to make a copy of the ``renku/service/.env-example`` file and configure it
to your needs. The setup here is to expose the service behind a traefik reverse proxy
to mimic an actual production deployment. You can access the proxied endpoints at
``http://localhost/api``. The service itself is exposed on port 8080 so its endpoints
are available directly under ``http://localhost:8080``.

API Documentation
-----------------

The renku core service implements the API documentation as an OpenAPI 3.0.x spec.
You can retrieve the yaml of the specification itself with

```
$ renku service apispec
```

If deploying the service locally with ``docker-compose`` you can find the swagger-UI
under ``localhost/api/swagger``. To send the proper authorization headers to the
service endpoints, click the ``Authorize`` button and enter a valid JWT token and
a gitlab token with read/write repository scopes. The JWT token can be obtained by
logging in to a renku instance with ``renku login`` and retrieving it from your local
renku configuration.

In a live deployment, the swagger documentation is available under ``https://<renku-endpoint>/swagger``.
You can authorize the API by first logging into renku normally, then going to the
swagger page, clicking ``Authorize`` and picking the ``oidc (OAuth2, authorization_code)``
option. Leave the ``client_id`` as ``swagger`` and the ``client_secret`` empty, select
all scopes and click ``Authorize``. You should now be logged in and you can send
requests using the ``Try it out`` buttons on individual requests.

Developing Renku
================

For testing the functionality from source it is convenient to install ``renku``
in editable mode using ``pipx``. Clone the repository and then do:

::

    $ pipx install \
        --editable \
        <path-to-renku-python>[all] \
        renku

This will install all the extras for testing and debugging.

If you already use `pyenv <https://github.com/pyenv/pyenv>`__ to manage different python versions,
you may be interested in installing `pyenv-virtualenv <https://github.com/pyenv/pyenv-virtualenv>`__ to
create an ad-hoc virtual environment for developing renku.

Once you have created and activated a virtual environment for renku-python, you can use the usual
`pip` commands to install the required dependencies.

::

    $ pip install -e .[all]  # use `.[all]` for zsh

Service
-------

Developing the service and testing its APIs can be done with ``docker compose`` (see
"Deploying Locally" above).

If you have a full RenkuLab deployment at your disposal, you can
use `telepresence <https://www.telepresence.io/>`__ v1 to develop and debug locally.
Just run the `start-telepresence.sh` script and follow the instructions.
Mind that the script doesn't work with telepresence v2.

Running tests
-------------

We use `pytest <https://docs.pytest.org>`__ for running tests.
You can use our `run-tests.sh` script for running specific set of tests.

::

    $ ./run-tests.sh -h

We lint the files using `black <https://github.com/psf/black>`__ and
`isort <https://github.com/PyCQA/isort>`__.

Using External Debuggers
------------------------

Local Machine
~~~~~~~~~~~~~

To run ``renku`` via e.g. the `Visual Studio Code debugger
<https://code.visualstudio.com/docs/python/debugging>`_ you need run it via
the python executable in whatever virtual environment was used to install ``renku``. If there is a package
needed for the debugger, you need to inject it into the virtual environment first, e.g.:

::

    $ pipx inject renku ptvsd

Finally, run ``renku`` via the debugger:

::

    $ ~/.local/pipx/venvs/renku/bin/python -m ptvsd --host localhost --wait -m renku.ui.cli <command>

If using Visual Studio Code, you may also want to set the ``Remote Attach`` configuration
``PathMappings`` so that it will find your source code, e.g.

::

    {
        "name": "Python: Remote Attach",
        "type": "python",
        "request": "attach",
        "port": 5678,
        "host": "localhost",
        "pathMappings": [
            {
                "localRoot": "<path-to-renku-python-source-code>",
                "remoteRoot": "<path-to-renku-python-source-code>"
            }
        ]
    }


# securesystemslib

[![CI](https://github.com/secure-systems-lab/securesystemslib/workflows/Run%20Securesystemslib%20tests/badge.svg)](https://github.com/secure-systems-lab/securesystemslib/actions?query=workflow%3A%22Run+Securesystemslib+tests%22+branch%3Amain)
[![Documentation Status](https://readthedocs.org/projects/python-securesystemslib/badge/?version=latest)](https://python-securesystemslib.readthedocs.io/en/latest/?badge=latest)

Securesystemslib is a cryptography interface for signing and verifying digital
signatures. It is developed for the [TUF](https://theupdateframework.io) and
[in-toto](https://in-toto.io) projects: the key and signature containers are
compatible with metadata formats from those projects.

Under the hood, Securesystemslib can use various digital signing systems
(e.g. [cryptography](https://pypi.org/project/cryptography/), PIV hardware keys
and multiple cloud-based key management systems).

## Installation

The default installation supports [pure-Python `ed25519` signature
verification](https://github.com/pyca/ed25519) only. To enable other schemes and
signature creation, `securesystemslib` can be installed with *extras*. See
[pyproject.toml](pyproject.toml) for available *optional dependencies*.

```bash
# Install with ed25519, RSA, ECDSA sign and verify support
pip install securesystemslib[crypto]
```

```bash
# ...or with HSM (e.g. Yubikey) support
pip install securesystemslib[hsm]
```

## Usage
[python-securesystemslib.readthedocs.io](https://python-securesystemslib.readthedocs.io)

## Contact
- Questions and discussions:
  [`#securesystemslib-python`](https://cloud-native.slack.com/archives/C05PF3GA7AL)
  on [CNCF Slack](https://communityinviter.com/apps/cloud-native/cncf)
- Security issues: see [Security policy](docs/SECURITY.md)
- Other issues and requests: [*Open a new
  issue*](https://github.com/secure-systems-lab/securesystemslib/issues/new)

## Contribute
See [Instructions for contributors](docs/CONTRIBUTING.md).

## Legacy key migration

Use
[`migrate_keys`](https://github.com/secure-systems-lab/securesystemslib/blob/v0.31.0/docs/migrate_key.py)
script to convert key pairs generated with legacy `keys` or `interface` modules
to a consistent standard format, which is compatible with
[`CryptoSigner`](docs/CRYPTO_SIGNER.md). The script requires
`securesystemslib~=0.31.0`.


# Indico Plugins [![CI Status][ci-badge]][ci-link] [![License][license-badge]][license-link] [![Available on PyPI][pypi-badge]][pypi-link] [![Made at CERN!][cern-badge]][cern-link]

This repository contains all the official plugins for [Indico][indico].

These plugins are licensed under the MIT license.

## Note

In applying the MIT license, CERN does not waive the privileges and immunities
granted to it by virtue of its status as an Intergovernmental Organization
or submit itself to any jurisdiction.

[ci-badge]: https://github.com/indico/indico-plugins/actions/workflows/ci.yml/badge.svg
[ci-link]: https://github.com/indico/indico-plugins/actions/workflows/ci.yml
[license-link]: https://github.com/indico/indico-plugins/blob/master/LICENSE
[license-badge]: https://img.shields.io/github/license/indico/indico.svg
[pypi-badge]: https://img.shields.io/pypi/v/indico.svg
[pypi-link]: https://pypi.python.org/pypi/indico-plugins/
[cern-badge]: https://img.shields.io/badge/CERN-Open%20Source-%232980b9.svg
[cern-link]: https://home.cern
[indico]: https://github.com/indico/indico
[indico-plugins]: https://github.com/indico/indico-plugins


# Shaken Fist: Opinionated to the point of being impolite
![Python application](https://github.com/shakenfist/shakenfist/workflows/Python%20application/badge.svg)

**Documentation:** https://shakenfist.com/
**Source Code:** https://github.com/shakenfist/shakenfist


Tornado Web Server
==================

.. image:: https://badges.gitter.im/Join%20Chat.svg
   :alt: Join the chat at https://gitter.im/tornadoweb/tornado
   :target: https://gitter.im/tornadoweb/tornado?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge

`Tornado <http://www.tornadoweb.org>`_ is a Python web framework and
asynchronous networking library, originally developed at `FriendFeed
<http://friendfeed.com>`_.  By using non-blocking network I/O, Tornado
can scale to tens of thousands of open connections, making it ideal for
`long polling <http://en.wikipedia.org/wiki/Push_technology#Long_Polling>`_,
`WebSockets <http://en.wikipedia.org/wiki/WebSocket>`_, and other
applications that require a long-lived connection to each user.

Hello, world
------------

Here is a simple "Hello, world" example web app for Tornado:

.. code-block:: python

    import asyncio
    import tornado

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("Hello, world")

    def make_app():
        return tornado.web.Application([
            (r"/", MainHandler),
        ])

    async def main():
        app = make_app()
        app.listen(8888)
        await asyncio.Event().wait()

    if __name__ == "__main__":
        asyncio.run(main())

This example does not use any of Tornado's asynchronous features; for
that see this `simple chat room
<https://github.com/tornadoweb/tornado/tree/stable/demos/chat>`_.

Documentation
-------------

Documentation and links to additional resources are available at
https://www.tornadoweb.org


# MkDocs

> *Project documentation with Markdown*

[![PyPI Version][pypi-v-image]][pypi-v-link]
[![Build Status][GHAction-image]][GHAction-link]
[![Coverage Status][codecov-image]][codecov-link]

MkDocs is a **fast**, **simple** and **downright gorgeous** static site
generator that's geared towards building project documentation. Documentation
source files are written in Markdown, and configured with a single YAML
configuration file. It is designed to be easy to use and can be extended with
third-party themes, plugins, and Markdown extensions.

Please see the [Documentation][mkdocs] for an introductory tutorial and a full
user guide.

## Features

- Build static HTML files from Markdown files.
- Use Plugins and Markdown Extensions to enhance MkDocs.
- Use the built-in themes, third party themes or create your own.
- Publish your documentation anywhere that static files can be served.
- Much more!

## Support

If you need help with MkDocs, do not hesitate to get in contact with us!

-   For questions and high-level discussions, use **[Discussions]** on GitHub.
    -   For small questions, a good alternative is the **[Chat room]** on
        Gitter/Matrix.
-   To report a bug or make a feature request, open an **[Issue]** on GitHub.

Please note that we may only provide
support for problems/questions regarding core features of MkDocs. Any
questions or bug reports about features of third-party themes, plugins,
extensions or similar should be made to their respective projects.
But, such questions are *not* banned from the [chat room].

Make sure to stick around to answer some questions as well!

## Links

- [Official Documentation][mkdocs]
- [Latest Release Notes][release-notes]
- [Catalog of third-party plugins, themes and recipes][catalog]

## Contributing to MkDocs

The MkDocs project welcomes, and depends on, contributions from developers and
users in the open source community. Please see the [Contributing Guide] for
information on how you can help.

## Code of Conduct

Everyone interacting in the MkDocs project's codebases, issue trackers, and
discussion forums is expected to follow the [PyPA Code of Conduct].

<!-- Badges -->
[codecov-image]: https://codecov.io/github/mkdocs/mkdocs/coverage.svg?branch=master
[codecov-link]: https://codecov.io/github/mkdocs/mkdocs?branch=master
[pypi-v-image]: https://img.shields.io/pypi/v/mkdocs.svg
[pypi-v-link]: https://pypi.org/project/mkdocs/
[GHAction-image]: https://github.com/mkdocs/mkdocs/workflows/CI/badge.svg?branch=master&event=push
[GHAction-link]: https://github.com/mkdocs/mkdocs/actions?query=event%3Apush+branch%3Amaster
<!-- Links -->
[mkdocs]: https://www.mkdocs.org
[Issue]: https://github.com/mkdocs/mkdocs/issues
[Discussions]: https://github.com/mkdocs/mkdocs/discussions
[Chat room]: https://gitter.im/mkdocs/community
[release-notes]: https://www.mkdocs.org/about/release-notes/
[Contributing Guide]: https://www.mkdocs.org/about/contributing/
[PyPA Code of Conduct]: https://www.pypa.io/en/latest/code-of-conduct/
[catalog]: https://github.com/mkdocs/catalog

## License

[BSD-2-Clause](https://github.com/mkdocs/mkdocs/blob/master/LICENSE)


[English](README_en.md) | 简体中文 | [हिन्दी](./doc/doc_i18n/README_हिन्द.md) | [日本語](./doc/doc_i18n/README_日本語.md) | [한국인](./doc/doc_i18n/README_한국어.md) | [Pу́сский язы́к](./doc/doc_i18n/README_Ру́сский_язы́к.md)

## 简介
PaddleOCR 旨在打造一套丰富、领先、且实用的 OCR 工具库，助力开发者训练出更好的模型，并应用落地。

## 🚀 社区
PaddleOCR 由 [PMC](https://github.com/PaddlePaddle/PaddleOCR/issues/12122) 监督。Issues 和 PRs 将在尽力的基础上进行审查。欲了解 PaddlePaddle 社区的完整概况，请访问 [community](https://github.com/PaddlePaddle/community)。

⚠注意：[Issues](https://github.com/PaddlePaddle/PaddleOCR/issues)模块仅用来报告程序🐞Bug，其余提问请移步[Discussions](https://github.com/PaddlePaddle/PaddleOCR/discussions)模块提问。如所提Issue不是Bug，会被移到Discussions模块，敬请谅解。

## 📣 近期更新
- **📚直播和OCR实战打卡营预告**：《PP-ChatOCRv2赋能金融报告信息智能化抽取，新金融效率再升级》课程上线，破解复杂版面、表格识别、信息抽取OCR解析难题，直播时间：6月6日（周四）19：00。并于6月11日启动【政务采购合同信息抽取】实战打卡营。报名链接：https://www.wjx.top/vm/eBcYmqO.aspx?udsid=197406
- **🔥2024.5.10 上线星河零代码产线(OCR 相关)**：全面覆盖了以下四大 OCR 核心任务，提供极便捷的 Badcase 分析和实用的在线体验：
  - [通用 OCR](https://aistudio.baidu.com/community/app/91660) (PP-OCRv4)。
  - [通用表格识别](https://aistudio.baidu.com/community/app/91661) (SLANet)。
  - [通用图像信息抽取](https://aistudio.baidu.com/community/app/91662) (PP-ChatOCRv2-common)。
  - [文档场景信息抽取](https://aistudio.baidu.com/community/app/70303) (PP-ChatOCRv2-doc)。

  同时采用了 **[全新的场景任务开发范式](https://aistudio.baidu.com/pipeline/mine)** ,将模型统一汇聚，实现训练部署的零代码开发，并支持在线服务化部署和导出离线服务化部署包。

- **🔥2023.8.7 发布 PaddleOCR [release/2.7](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.7)**
    - 发布[PP-OCRv4](./doc/doc_ch/PP-OCRv4_introduction.md)，提供 mobile 和 server 两种模型
      - PP-OCRv4-mobile：速度可比情况下，中文场景效果相比于 PP-OCRv3 再提升 4.5%，英文场景提升 10%，80 语种多语言模型平均识别准确率提升 8%以上
      - PP-OCRv4-server：发布了目前精度最高的 OCR 模型，中英文场景上检测模型精度提升 4.9%， 识别模型精度提升 2%
      可参考[快速开始](./doc/doc_ch/quickstart.md) 一行命令快速使用，同时也可在飞桨 AI 套件(PaddleX)中的[通用 OCR 产业方案](https://aistudio.baidu.com/aistudio/modelsdetail?modelId=286)中低代码完成模型训练、推理、高性能部署全流程
- 🔨**2022.11 新增实现[4 种前沿算法](doc/doc_ch/algorithm_overview.md)**：文本检测 [DRRG](doc/doc_ch/algorithm_det_drrg.md),  文本识别 [RFL](doc/doc_ch/algorithm_rec_rfl.md), 文本超分[Text Telescope](doc/doc_ch/algorithm_sr_telescope.md)，公式识别[CAN](doc/doc_ch/algorithm_rec_can.md)
- **2022.10 优化[JS 版 PP-OCRv3 模型](./deploy/paddlejs/README_ch.md)**：模型大小仅 4.3M，预测速度提升 8 倍，配套 web demo 开箱即用
- **💥 直播回放：PaddleOCR 研发团队详解 PP-StructureV2 优化策略**。微信扫描[下方二维码](#开源社区)，关注公众号并填写问卷后进入官方交流群，获取直播回放链接与 20G 重磅 OCR 学习大礼包（内含 PDF 转 Word 应用程序、10 种垂类模型、《动手学 OCR》电子书等）
- **🔥2022.8.24 发布 PaddleOCR [release/2.6](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.6)**
  - 发布[PP-StructureV2](./ppstructure/README_ch.md)，系统功能性能全面升级，适配中文场景，新增支持[版面复原](./ppstructure/recovery/README_ch.md)，支持**一行命令完成 PDF 转 Word**；
  - [版面分析](./ppstructure/layout/README_ch.md)模型优化：模型存储减少 95%，速度提升 11 倍，平均 CPU 耗时仅需 41ms；
  - [表格识别](./ppstructure/table/README_ch.md)模型优化：设计 3 大优化策略，预测耗时不变情况下，模型精度提升 6%；
  - [关键信息抽取](./ppstructure/kie/README_ch.md)模型优化：设计视觉无关模型结构，语义实体识别精度提升 2.8%，关系抽取精度提升 9.1%。
- 🔥**2022.8 发布 [OCR 场景应用集合](./applications)**：包含数码管、液晶屏、车牌、高精度 SVTR 模型、手写体识别等**9 个垂类模型**，覆盖通用，制造、金融、交通行业的主要 OCR 垂类应用。

> [更多](./doc/doc_ch/update.md)

## 🌟 特性

支持多种 OCR 相关前沿算法，在此基础上打造产业级特色模型[PP-OCR](./doc/doc_ch/ppocr_introduction.md)、[PP-Structure](./ppstructure/README_ch.md)和[PP-ChatOCRv2](https://aistudio.baidu.com/community/app/70303)，并打通数据生产、模型训练、压缩、预测部署全流程。

> 上述内容的使用方法建议从文档教程中的快速开始体验

## ⚡ 快速开始

- 在线免费体验：
    - PP-OCRv4 在线体验地址：https://aistudio.baidu.com/community/app/91660
    - SLANet 在线体验地址：https://aistudio.baidu.com/community/app/91661
    - PP-ChatOCRv2-common 在线体验地址：https://aistudio.baidu.com/community/app/91662
    - PP-ChatOCRv2-doc 在线体验地址：https://aistudio.baidu.com/community/app/70303

- 一行命令快速使用：[快速开始（中英文/多语言/文档分析）](./doc/doc_ch/quickstart.md)
- 移动端 demo 体验：[安装包 DEMO 下载地址](https://ai.baidu.com/easyedge/app/openSource?from=paddlelite)(基于 EasyEdge 和 Paddle-Lite, 支持 iOS 和 Android 系统)

## 📖 技术交流合作

- 飞桨低代码开发工具 PaddleX 官方交流频道：https://aistudio.baidu.com/community/channel/610

## 📚《动手学 OCR》电子书
- [《动手学 OCR》电子书](./doc/doc_ch/ocr_book.md)

## 🛠 PP-OCR 系列模型列表（更新中）

| 模型简介                              | 模型名称                | 推荐场景        | 检测模型                                                     | 方向分类器                                                   | 识别模型                                                     |
| ------------------------------------- | ----------------------- | --------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| 中英文超轻量 PP-OCRv4 模型（15.8M）     | ch_PP-OCRv4_xx          | 移动端&服务器端 | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_det_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_det_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_train.tar) |
| 中英文超轻量 PP-OCRv3 模型（16.2M）     | ch_PP-OCRv3_xx          | 移动端&服务器端 | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_train.tar) |
| 英文超轻量 PP-OCRv3 模型（13.4M）     | en_PP-OCRv3_xx          | 移动端&服务器端 | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_distill_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_infer.tar) / [训练模型](https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_train.tar) |

- 超轻量 OCR 系列更多模型下载（包括多语言），可以参考[PP-OCR 系列模型下载](./doc/doc_ch/models_list.md)，文档分析相关模型参考[PP-Structure 系列模型下载](./ppstructure/docs/models_list.md)

### PaddleOCR 场景应用模型

| 行业 | 类别         | 亮点                               | 文档说明                                                     | 模型下载                                      |
| ---- | ------------ | ---------------------------------- | ------------------------------------------------------------ | --------------------------------------------- |
| 制造 | 数码管识别   | 数码管数据合成、漏识别调优         | [光功率计数码管字符识别](./applications/光功率计数码管字符识别/光功率计数码管字符识别.md) | [下载链接](./applications/README.md#模型下载) |
| 金融 | 通用表单识别 | 多模态通用表单结构化提取           | [多模态表单识别](./applications/多模态表单识别.md)           | [下载链接](./applications/README.md#模型下载) |
| 交通 | 车牌识别     | 多角度图像处理、轻量模型、端侧部署 | [轻量级车牌识别](./applications/轻量级车牌识别.md)           | [下载链接](./applications/README.md#模型下载) |

- 更多制造、金融、交通行业的主要 OCR 垂类应用模型（如电表、液晶屏、高精度 SVTR 模型等），可参考[场景应用模型下载](./applications)

## 📖 文档教程

- [运行环境准备](./doc/doc_ch/environment.md)
- [PP-OCR 文本检测识别🔥](./doc/doc_ch/ppocr_introduction.md)
    - [快速开始](./doc/doc_ch/quickstart.md)
    - [模型库](./doc/doc_ch/models_list.md)
    - [模型训练](./doc/doc_ch/training.md)
        - [文本检测](./doc/doc_ch/detection.md)
        - [文本识别](./doc/doc_ch/recognition.md)
        - [文本方向分类器](./doc/doc_ch/angle_class.md)
    - 模型压缩
        - [模型量化](./deploy/slim/quantization/README.md)
        - [模型裁剪](./deploy/slim/prune/README.md)
        - [知识蒸馏](./doc/doc_ch/knowledge_distillation.md)
    - [推理部署](./deploy/README_ch.md)
        - [基于 Python 预测引擎推理](./doc/doc_ch/inference_ppocr.md)
        - [基于 C++预测引擎推理](./deploy/cpp_infer/readme_ch.md)
        - [服务化部署](./deploy/pdserving/README_CN.md)
        - [端侧部署](./deploy/lite/readme.md)
        - [Paddle2ONNX 模型转化与预测](./deploy/paddle2onnx/readme_ch.md)
        - [云上飞桨部署工具](./deploy/paddlecloud/README.md)
        - [Benchmark](./doc/doc_ch/benchmark.md)
- [PP-Structure 文档分析🔥](./ppstructure/README_ch.md)
    - [快速开始](./ppstructure/docs/quickstart.md)
    - [模型库](./ppstructure/docs/models_list.md)
    - [模型训练](./doc/doc_ch/training.md)
        - [版面分析](./ppstructure/layout/README_ch.md)
        - [表格识别](./ppstructure/table/README_ch.md)
        - [关键信息提取](./ppstructure/kie/README_ch.md)
    - [推理部署](./deploy/README_ch.md)
        - [基于 Python 预测引擎推理](./ppstructure/docs/inference.md)
        - [基于 C++预测引擎推理](./deploy/cpp_infer/readme_ch.md)
        - [服务化部署](./deploy/hubserving/readme.md)
- [前沿算法与模型🚀](./doc/doc_ch/algorithm_overview.md)
    - [文本检测算法](./doc/doc_ch/algorithm_overview.md)
    - [文本识别算法](./doc/doc_ch/algorithm_overview.md)
    - [端到端 OCR 算法](./doc/doc_ch/algorithm_overview.md)
    - [表格识别算法](./doc/doc_ch/algorithm_overview.md)
    - [关键信息抽取算法](./doc/doc_ch/algorithm_overview.md)
    - [使用 PaddleOCR 架构添加新算法](./doc/doc_ch/add_new_algorithm.md)
- [场景应用](./applications)
- 数据标注与合成
    - [半自动标注工具 PPOCRLabel](https://github.com/PFCCLab/PPOCRLabel/blob/main/README_ch.md)
    - [数据合成工具 Style-Text](https://github.com/PFCCLab/StyleText/blob/main/README_ch.md)
    - [其它数据标注工具](./doc/doc_ch/data_annotation.md)
    - [其它数据合成工具](./doc/doc_ch/data_synthesis.md)
- 数据集
    - [通用中英文 OCR 数据集](doc/doc_ch/dataset/datasets.md)
    - [手写中文 OCR 数据集](doc/doc_ch/dataset/handwritten_datasets.md)
    - [垂类多语言 OCR 数据集](doc/doc_ch/dataset/vertical_and_multilingual_datasets.md)
    - [版面分析数据集](doc/doc_ch/dataset/layout_datasets.md)
    - [表格识别数据集](doc/doc_ch/dataset/table_datasets.md)
    - [关键信息提取数据集](doc/doc_ch/dataset/kie_datasets.md)
- [代码组织结构](./doc/doc_ch/tree.md)
- [效果展示](#效果展示)
- [《动手学 OCR》电子书📚](./doc/doc_ch/ocr_book.md)
- [开源社区](#开源社区)
- FAQ
    - [通用问题](./doc/doc_ch/FAQ.md)
    - [PaddleOCR 实战问题](./doc/doc_ch/FAQ.md)
- [参考文献](./doc/doc_ch/reference.md)
- [许可证书](#许可证书)

## 👀 效果展示 [more](./doc/doc_ch/visualization.md)

PP-OCRv3 中文模型

PP-OCRv3 英文模型

PP-OCRv3 多语言模型

PP-Structure 文档分析

- 版面分析+表格识别

- SER（语义实体识别）

- RE（关系提取）

## 许可证书
本项目的发布受Apache 2.0 license许可认证。


##########
django CMS
##########
.. image:: https://img.shields.io/pypi/v/django-cms.svg
    :target: https://pypi.python.org/pypi/django-cms/
.. image:: https://img.shields.io/badge/wheel-yes-green.svg
    :target: https://pypi.python.org/pypi/django-cms/
.. image:: https://img.shields.io/pypi/l/django-cms.svg
    :target: https://pypi.python.org/pypi/django-cms/
.. image:: https://codeclimate.com/github/divio/django-cms/badges/gpa.svg
   :target: https://codeclimate.com/github/divio/django-cms
   :alt: Code Climate

Open source enterprise content management system based on the Django framework and backed by the non-profit django CMS Association (`Sponsor us! <https://www.django-cms.org/en/memberships/>`_).

*******************************************
Contribute to this project and win rewards
*******************************************

Because django CMS is a community-driven project, we welcome everyone to `get involved in the project <https://www.django-cms.org/en/contribute/>`_. Become part of a fantastic community and help us make django CMS the best open source CMS in the world.

.. ATTENTION::

    Please use the ``develop-4`` branch as the target for pull requests for on-going development.

    Security fixes will be backported to older branches by the core team as appropriate.

********
Features
********

* hierarchical pages
* extensive built-in support for multilingual websites
* multi-site support
* draft/publish workflows
* version control
* a sophisticated publishing architecture, that's also usable in your own applications
* frontend content editing
* a hierarchical content structure for nested plugins
* an extensible navigation system that your own applications can hook into
* SEO-friendly URLs
* designed to integrate thoroughly into other applications

Developing applications that integrate with and take advantage of django CMS features is easy and well-documented.

More information on `our website <https://www.django-cms.org>`_.

************
Requirements
************

See the `Python/Django requirements for the current release version
<http://docs.django-cms.org/en/latest/#software-version-requirements-and-release-notes>`_ in our documentation.

See the `installation how-to guide for an overview of some other requirements and dependencies of the current release
<https://docs.django-cms.org/en/latest/introduction/01-install.html>`_.

***************
Getting started
***************

These `tutorials <http://docs.django-cms.org/en/latest/introduction/index.html>`_ take you step-by-step through some key aspects of django CMS.

*************
Documentation
*************

Our documentation working group maintains documentation for several versions of the project. Key versions are:

* `stable <http://docs.django-cms.org>`_ (default), for the **current release** version
* `latest <http://docs.django-cms.org/en/latest/>`_, representing the latest build of the **develop-4 branch**

For more information about our branch policy, see `Branches
<http://docs.django-cms.org/en/latest/contributing/development-policies.html>`_.

Our documentation is hosted courtesy of `Read the Docs <https://readthedocs.org>`_.

The dependencies for the docs are compiled by `pip-tools <https://github.com/jazzband/pip-tools>`_.

***************************
Test django CMS in our demo
***************************

The demo platform is kindly provided by Divio, platinum member of the django CMS Association.

.. image:: https://raw.githubusercontent.com/django-cms/django-cms/develop/docs/images/try-with-divio.png
   :target: https://www.django-cms.org/en/django-cms-demo/
   :alt: Try demo with Divio Cloud

************
Getting Help
************

Please head over to our `Discord Server <https://discord-support-channel.django-cms.org>`_ or Stackoverflow for support.

********************
Professional support
********************

Choose from a list of `trusted tech partner <https://www.django-cms.org/en/tech-partners/>`_ of the django CMS Association to get your website project delivered successfully.

Choose a `trusted web host <https://www.django-cms.org/en/hosting-services/>`_ for your django CMS project and get your website online today.

**************************
The django CMS Association
**************************

The django CMS Association is a non-profit organization that was founded in 2020 with the goal to drive the success of django CMS, by increasing customer happiness, market share and open-source contributions. We provide infrastructure and guidance for the django CMS project.

The non-profit django CMS Association is dependent on donations to fulfill its purpose. The best way to donate is to become a member of the association and pay membership fees. The funding will be funneled back into core development and community projects.

`Join the django CMS Association <https://www.django-cms.org/en/contribute/>`_.

*******
Credits
*******

* Includes icons and adapted icons from `Bootstrap <https://icons.getbootstrap.com>`_.
* Includes icons from `FamFamFam <http://www.famfamfam.com>`_.
* Python tree engine powered by
  `django-treebeard <https://tabo.pe/projects/django-treebeard/>`_.
* JavaScript tree in admin uses `jsTree <https://www.jstree.com>`_.
* Many thanks to
  `all the contributors <https://github.com/django-cms/django-cms/graphs/contributors>`_
  to django CMS!