Distributed Multi-Agent Reinforcement Learning in JAX

## Welcome to Mava! 🦁

[**Installation**](#installation-) | [**Quickstart**](#quickstart-)

Mava provides simplified code for quickly iterating on ideas in multi-agent reinforcement learning (MARL) with useful implementations of MARL algorithms in JAX allowing for easy parallelisation across devices with JAX's `pmap`. Mava is a project originating in the Research Team at [InstaDeep](https://www.instadeep.com/).

To join us in these efforts, please feel free to reach out, raise issues or read our [contribution guidelines](#contributing-) (or just star 🌟 to stay up to date with the latest developments)!

## Overview 🦜

Mava currently offers the following building blocks for MARL research:

- 🥑 **Implementations of MARL algorithms**: Implementations of multi-agent PPO systems that follow both the Centralised Training with Decentralised Execution (CTDE) and Decentralised Training with Decentralised Execution (DTDE) MARL paradigms.
- 🍬 **Environment Wrappers**: Example wrappers for mapping Jumanji environments to an environment that is compatible with Mava. At the moment, we support [Robotic Warehouse][jumanji_rware] and [Level-Based Foraging][jumanji_lbf] with plans to support more environments soon. We have also recently added support for the SMAX environment from [JaxMARL][jaxmarl].
- 🎓 **Educational Material**: [Quickstart notebook][quickstart] to demonstrate how Mava can be used and to highlight the added value of JAX-based MARL.
- 🧪 **Statistically robust evaluation**: Mava natively supports logging to json files which adhere to the standard suggested by [Gorsane et al. (2022)][toward_standard_eval]. This enables easy downstream experiment plotting and aggregation using the tools found in the [MARL-eval][marl_eval] library.

## Performance and Speed 🚀

### SMAX
For comparing Mava’s stability to other JAX-based baseline algorithms, we train Mava’s recurrent IPPO and MAPPO systems on a broad range of [SMAX][smax] tasks. In all cases we do not rerun baselines but instead take results for final win rates from the [JaxMARL technical report](https://arxiv.org/pdf/2311.10090.pdf). For the full SMAX experiments results, please see the following [page](docs/smax_benchmark.md).

 Mava Recurrent IPPO and MAPPO performance on the 3s5z, 6h_vs_8z and 3s5z_vs_3s6z SMAX tasks.

### Robotic Warehouse

All of the experiments below were performed using an NVIDIA Quadro RTX 4000 GPU with 8GB Memory.

In order to show the utility of end-to-end JAX-based MARL systems and JAX-based environments we compare the speed of Mava against [EPyMARL][epymarl] as measured in total training wallclock time on simple [Robotic Warehouse][rware] (RWARE) tasks with 2 and 4 agents. Our aim is to illustrate the speed increases that are possible with using end-to-end Jax-based systems and we do not necessarily make an effort to achieve optimal performance. For EPyMARL, we use the hyperparameters as recommended by [Papoudakis et al. (2020)](https://arxiv.org/pdf/2006.07869.pdf) and for Mava we performed a basic grid search. In both cases, systems were trained up to 20 million total environment steps using 16 vectorised environments.

 Mava feedforward MAPPO performance on the tiny-2ag, tiny-4ag and small-4ag RWARE tasks.

### 📌 An important note on the differences in converged performance

In order to benefit from the wallclock speed-ups afforded by JAX-based systems it is required that environments also be written in JAX. It is for this reason that Mava does not use the exact same version of the RWARE environment as EPyMARL but instead uses a JAX-based implementation of RWARE found in [Jumanji][jumanji_rware], under the name RobotWarehouse. One of the notable differences in the underlying environment logic is that RobotWarehouse will not attempt to resolve agent collisions but will instead terminate an episode when agents do collide. In our experiments, this appeared to make the environment more challenging. For this reason we show the performance of Mava on Jumanji with and without termination upon collision indicated with `w/o collision` in the figure legends. For a more detailed discussion, please see the following [page](docs/jumanji_rware_comparison.md).

### Level-Based Foraging
Mava also supports [Jumanji][jumanji_lbf]'s LBF. We evaluate Mava's recurrent MAPPO system on LBF, against [EPyMARL][epymarl] (we used original [LBF](https://github.com/semitable/lb-foraging) for EPyMARL) in 2 and 4 agent settings up to 20 million timesteps. Both systems were trained using 16 vectorized environments. For the EPyMARL systems we use a NVIDIA A100 GPU and for the Mava systems we use a GeForce RTX 3050 laptop GPU with 4GB of memory. To show how Mava can generalise to different hardware, we also train the Mava systems on a TPU v3-8. We plan to publish comprehensive performance benchmarks for all Mava's algorithms across various LBF scenarios soon.

 Mava Recurrent MAPPO performance on the 2s-8x8-2p-2f-coop, and 15x15-4p-3fz Level-Based Foraging tasks.

### 🧨 Steps per second experiments using vectorised environments

Furthermore, we illustrate the speed of Mava by showing the steps per second as the number of parallel environments is increased. These steps per second scaling plots were computed using a standard laptop GPU, specifically an RTX-3060 GPU with 6GB memory.

 Mava steps per second scaling with increased vectorised environments and total training run time for 20M environment steps.

## Code Philosophy 🧘

The current code in Mava is adapted from [PureJaxRL][purejaxrl] which provides high-quality single-file implementations with research-friendly features. In turn, PureJaxRL is inspired by the code philosophy from [CleanRL][cleanrl]. Along this vein of easy-to-use and understandable RL codebases, Mava is not designed to be a modular library and is not meant to be imported. Our repository focuses on simplicity and clarity in its implementations while utilising the advantages offered by JAX such as `pmap` and `vmap`, making it an excellent resource for researchers and practitioners to build upon.

## Installation 🎬

At the moment Mava is not meant to be installed as a library, but rather to be used as a research tool.

You can use Mava by cloning the repo and pip installing as follows:

```bash
git clone https://github.com/instadeepai/mava.git
cd mava
pip install -e .
```

We have tested `Mava` on Python 3.9. Note that because the installation of JAX differs depending on your hardware accelerator,
we advise users to explicitly install the correct JAX version (see the [official installation guide](https://github.com/google/jax#installation)). For more in-depth installation guides including Docker builds and virtual environments, please see our [detailed installation guide](docs/DETAILED_INSTALL.md).

## Quickstart ⚡

To get started with training your first Mava system, simply run one of the system files. e.g.,

```bash
python mava/systems/ff_ippo.py
```

Mava makes use of Hydra for config management. In order to see our default system configs please see the `mava/configs/` directory. A benefit of Hydra is that configs can either be set in config yaml files or overwritten from the terminal on the fly. For an example of running a system on the LBF environment, the above code can simply be adapted as follows:

```bash
python mava/systems/ff_ippo.py env=lbf
```

Different scenarios can also be run by making the following config updates from the terminal:

```bash
python mava/systems/ff_ippo.py env=rware env/scenario=tiny-4ag
```

Additionally, we also have a [Quickstart notebook][quickstart] that can be used to quickly create and train your first Multi-agent system.

## Advanced Usage 👽

Mava can be used in a wide array of advanced systems. As an example, we demonstrate recording experience data from one of our PPO systems into a [Flashbax](https://github.com/instadeepai/flashbax) `Vault`. This vault can then easily be integrated into offline MARL systems, such as those found in [OG-MARL](https://github.com/instadeepai/og-marl). See the [Advanced README](./mava/advanced_usage/) for more information.

## Contributing 🤝

Please read our [contributing docs](docs/CONTRIBUTING.md) for details on how to submit pull requests, our Contributor License Agreement and community guidelines.

## Roadmap 🛤

We plan to iteratively expand Mava in the following increments:

- 🌴 Support for more environments.
- 🔁 More robust recurrent systems.
- 🌳 Support for non JAX-based environments.
- 🦾 Support for off-policy algorithms.
- 🎛 Continuous action space environments and algorithms.

Please do follow along as we develop this next phase!

## TensorFlow 2 Mava:
Originally Mava was written in Tensorflow 2. Support for the TF2-based framework and systems has now been fully **deprecated**. If you would still like to use it, please install `v0.1.3` of Mava (i.e. `pip install id-mava==0.1.3`).

## See Also 🔎

**InstaDeep's MARL ecosystem in JAX.** In particular, we suggest users check out the following sister repositories:

- 🔌 [OG-MARL](https://github.com/instadeepai/og-marl): datasets with baselines for offline MARL in JAX.
- 🌴 [Jumanji](https://github.com/instadeepai/jumanji): a diverse suite of scalable reinforcement learning environments in JAX.
- 😎 [Matrax](https://github.com/instadeepai/matrax): a collection of matrix games in JAX.
- ⚡ [Flashbax](https://github.com/instadeepai/flashbax): accelerated replay buffers in JAX.
- 📈 [MARL-eval][marl_eval]: standardised experiment data aggregation and visualisation for MARL.

**Related.** Other libraries related to accelerated MARL in JAX.

- 🦊 [JaxMARL](https://github.com/flairox/jaxmarl): accelerated MARL environments with baselines in JAX.
- 🌀 [DeepMind Anakin][anakin_paper] for the Anakin podracer architecture to train RL agents at scale.
- ♟ [Pgx](https://github.com/sotetsuk/pgx): JAX implementations of classic board games, such as Chess, Go and Shogi.
- 🔼 [Minimax](https://github.com/facebookresearch/minimax/): JAX implementations of autocurricula baselines for RL.

## Citing Mava 📚

If you use Mava in your work, please cite the accompanying
[technical report][Paper]:

```bibtex
@article{dekock2023mava,
    title={Mava: a research library for distributed multi-agent reinforcement learning in JAX},
    author={Ruan de Kock and Omayma Mahjoub and Sasha Abramowitz and Wiem Khlifi and Callum Rhys Tilbury
    and Claude Formanek and Andries P. Smit and Arnu Pretorius},
    year={2023},
    journal={arXiv preprint arXiv:2107.01460},
    url={https://arxiv.org/pdf/2107.01460.pdf},
}
```

## Acknowledgements 🙏

We would like to thank all the authors who contributed to the previous TF version of Mava: Kale-ab Tessera, St John Grimbly, Kevin Eloff, Siphelele Danisa, Lawrence Francis, Jonathan Shock, Herman Kamper, Willie Brink, Herman Engelbrecht, Alexandre Laterre, Karim Beguir. Their contributions can be found in our [TF technical report](https://arxiv.org/pdf/2107.01460v1.pdf).

The development of Mava was supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC) 🌤.

[Paper]: https://arxiv.org/pdf/2107.01460.pdf
[quickstart]: https://github.com/instadeepai/Mava/blob/develop/examples/Quickstart.ipynb
[jumanji]: https://github.com/instadeepai/jumanji
[cleanrl]: https://github.com/vwxyzjn/cleanrl
[purejaxrl]: https://github.com/luchris429/purejaxrl
[jumanji_rware]: https://instadeepai.github.io/jumanji/environments/robot_warehouse/
[jumanji_lbf]: https://github.com/sash-a/jumanji/tree/feat/lbf-truncate
[epymarl]: https://github.com/uoe-agents/epymarl
[anakin_paper]: https://arxiv.org/abs/2104.06272
[rware]: https://github.com/semitable/robotic-warehouse
[jaxmarl]: https://github.com/flairox/jaxmarl
[toward_standard_eval]: https://arxiv.org/pdf/2209.10485.pdf
[marl_eval]: https://github.com/instadeepai/marl-eval
[smax]: https://github.com/FLAIROx/JaxMARL/tree/main/jaxmarl/environments/smax


# IoGT

Internet of Good Things ([IoGT][2]) is developed as a public good under a [BSD-2][3] open-source license.

The development uses the Python programming language, building on the [Django][4] server-side web framework combined with the [Wagtail][5] content management system.

IoGT 2.x requires:
- Wagtail 3.0.3
- Django 3.2
- Python 3.9+
- PostgreSQL (14+), MySQL and SQLite (3.38+) as database backends

## Getting started

The easiest way to get started is to run IoGT using Docker. Make sure you have Docker Desktop installed, or at the very least the Docker Engine and Docker Compose (which are included in Docker Desktop).

Check out the code.
```
git clone https://github.com/unicef/iogt.git
cd iogt
```

Create the database. By default, an SQLite database will be created. This will take a few minutes on the first run because the IoGT container image needs to be created from scratch.
```
docker compose run --rm django python manage.py migrate
```

Create a superuser account for administration purposes.
```
docker compose run --rm django python manage.py createsuperuser
```

Compile .po language files stored in locale/
```
docker compose run --rm django python manage.py compilemessages
```

Start the server.
```
docker compose up -d
```

Check the logs.
```
docker compose logs -f
```

Once running, navigate to http://localhost:8000/ in your browser. Log in to the admin UI at http://localhost:8000/admin/ - use the superuser name and password you created earlier.

Shut everything down.
```
docker compose down
```

# Optional

## Setting up test data

It is possible to automatically populate the database with example data for a basic test site.

```sh
docker compose run --rm django python manage.py create_initial_data
```

Optionally, create the main menu automatically as well.

```sh
docker compose run --rm django python manage.py autopopulate_main_menus
```

## Configuring the Chatbot

Follow instructions [here](messaging/README.md)

## Content transfer between sites

Articles can be transferred from other IoGT sites - [details][12].

## Adding new localizable strings to the code base

After adding new strings to the code base that are user-facing (see https://docs.djangoproject.com/en/4.0/topics/i18n/translation/ ), follow the following process:
1. Run `./manage.py translation_tracking`
2. Review and update `common/translation_utils/translation_status.csv`. Strings that only appear in the admin backend don't need to be tagged as `translate`, only those that face users. See [here](common/translation_utils/README.md) for details.
3. Rerun `./manage.py translation_tracking`

This process updates PO files as necessary and compiles a list of strings that appear in the translation manager in the admin backend.

**Remark:** The file `common/translation_utils/translations.csv` is NOT updated in the process.
Even though it has a column "is in use", its data is currently not updated automatically, and similarly, new strings are not added automatically.

## Apache Superset Setup and Configuration
- See [here](questionnaires/superset/README.md)

## Page caching

Page caching is provided by Wagtail Cache and is deactivated by default. To start caching pages, follow the [activation intructions][7].

## Elasticsearch

To use Elasticsearch as a search backend instead of the relational database see the [setup instructions][11].

## Migrating content from IoGT v1
Follow instructions [here](iogt_content_migration/README.md)

## Add/Update Package
Follow instructions [here][10]

# Running Tests

Run the following commands:
```
make tests
make selenium-test
```

In parallel, for example, with 4 processes:
```
IOGT_TEST_PARALLEL=4 make test
IOGT_TEST_PARALLEL=4 make selenium-test
```

More details of the Selenium tests can be found in the [Selenium test README][9].

[1]: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment
[2]: https://www.unicef.org/innovation/IoGT
[3]: https://github.com/unicef/iogt/blob/develop/LICENSE
[4]: https://www.djangoproject.com/
[5]: https://wagtail.io/
[6]: https://github.com/wagtail/wagtail/wiki/Release-schedule
[7]: ./docs/cache.md
[8]: ./docs/troubleshooting.md
[9]: ./selenium_tests/README.md
[10]: ./docs/dependency-management.md
[11]: ./docs/elasticsearch.md
[12]: docs/content-transfer.md


# cports

Cports is a collection of source package ports for Chimera. The system has been
written specifically for the distribution using the Python scripting language.

The system is largely inspired by `xbps-src` from Void Linux, but should not be
considered a variant of it, nor should it be expected that the options and
behaviors are the same.

There are two authoritative documents on the system:

* [`Usage.md`](Usage.md) is the reference for users. It covers usage of `cbuild` and its
  basic and advanced options as well as concepts and requirements.
* [`Packaging.md`](Packaging.md) is the reference manual for packagers. It covers the API of the
  system and guidelines for creating and modifying templates, but not usage.

Most people looking to get involved with the project should read both.

To get started, read [`Usage.md`](Usage.md) first.

## Using cports with Chimera

You might want to test your built packages in an actual Chimera system. Since
`cbuild` creates a regular `apk` repository for you, this is as simple as
adding the repositories in your system.

Consider path to `cports` at `/home/user/cports`. The default repository path
for `cbuild` is the `packages` directory directly in `cports`. This is not
the actual repo yet, as there are multiple categories. The actual repositories
are those that have a directory named like your architecture (e.g. `x86_64`)
with the file `APKINDEX.tar.gz` in them.

Create a file `/etc/apk/repositories.d/00-cports.list`. The file must have
the `.list` extension. Put something like this in there:

```
/home/user/cports/packages/main
/home/user/cports/packages/contrib
```

This will give `apk` acccess to the `main` and `contrib` packages of your local
repository. You might want to restrict this list to only the repositories that
you have.

If you want access to local `-dbg` packages, you will also want to add the `debug`
sub-repositories, e.g. `/home/user/cports/packages/main/debug`.

You will also want to drop your signing public key in `/etc/apk/keys`. The key
can be located in `etc/keys` in the `cports` directory, with the `.pub` extension
(do not put in the private key).

### Pinning the repositories

You might also want to pin the local repository. This will effectively make `apk`
prefer your pinned repository even if a newer version if available in remote
repos. This is done by adding a prefix such as `@cports` before the repository
line, e.g. `@cports /home/user/cports/packages/main`. Then you can install things
from the repository like `apk add foo@cports`. If you just `apk add foo`, the
tagged repositories will be ignored.

Note that dependencies of packages from pinned repositories will still be pulled
from unpinned repositories preferentially, but pinned repositories will be used
if necessary. This is not the case for dependencies of packages from unpinned
repositories, which will only ever be pulled from unpinned repositories.

## Bootstrapping installations from repositories

For instructions on how to bootstrap the system into a target root as well as
some more advanced tooling for e.g. creation of actual images, check out the
[chimera-live](https://github.com/chimera-linux/chimera-live) repository.


---

# [ggshield](https://github.com/GitGuardian/ggshield): protect your code with GitGuardian

[![PyPI](https://img.shields.io/pypi/v/ggshield?color=%231B2D55&style=for-the-badge)](https://pypi.org/project/ggshield/)
[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/gitguardian/ggshield?color=1B2D55&sort=semver&style=for-the-badge&label=Docker)](https://hub.docker.com/r/gitguardian/ggshield)
[![License](https://img.shields.io/github/license/GitGuardian/ggshield?color=%231B2D55&style=for-the-badge)](LICENSE)
[![GitHub stars](https://img.shields.io/github/stars/gitguardian/ggshield?color=%231B2D55&style=for-the-badge)](https://github.com/GitGuardian/ggshield/stargazers)
[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/GitGuardian/ggshield/main.yml?branch=main&style=for-the-badge)](https://github.com/GitGuardian/ggshield/actions)
[![Codecov](https://img.shields.io/codecov/c/github/GitGuardian/ggshield?style=for-the-badge)](https://codecov.io/gh/GitGuardian/ggshield/)

`ggshield` is a CLI application that runs in your local environment or in a CI environment to help you detect more than 400+ types of secrets, as well as other potential security vulnerabilities or policy breaks affecting your codebase.

`ggshield` uses our [public API](https://api.gitguardian.com/doc) through [py-gitguardian](https://github.com/GitGuardian/py-gitguardian) to scan and detect potential vulnerabilities in files and other text content.

Only metadata such as call time, request size and scan mode is stored from scans using `ggshield`, therefore secrets and policy breaks incidents will not be displayed on your dashboard and **your files and secrets won't be stored**.

# Table of Contents

- [Installation](#installation)
  - [Requirements](#requirements)
  - [macOS](#macos)
    - [Using Homebrew](#using-homebrew)
    - [Using a standalone .pkg package](#using-a-standalone-pkg-package)
  - [Linux](#linux)
    - [Deb and RPM packages](#deb-and-rpm-packages)
  - [All operating systems](#all-operating-systems)
    - [Using pipx](#using-pipx)
    - [Using pip](#using-pip)
- [Initial setup](#initial-setup)
  - [Using `ggshield auth login`](#using-ggshield-auth-login)
  - [Manual setup](#manual-setup)
- [Getting started](#getting-started)
  - [Secrets](#secrets)
  - [Infra as Code Security (IaC)](#infra-as-code-security-iac)
- [Integrations](#integrations)
- [Learn more](#learn-more)
- [Output](#output)
- [Related open source projects](#related-open-source-projects)
- [License](#license)

# Installation

## Requirements

`ggshield` works on macOS, Linux and Windows.

It requires **Python 3.8 and newer** (except for standalone packages) and git.

Some commands require additional programs:

- docker: to scan docker images.
- pip: to scan pypi packages.

## macOS

### Using Homebrew

You can install `ggshield` using Homebrew:

```shell
$ brew install gitguardian/tap/ggshield
```

Upgrading is handled by Homebrew.

### Using a standalone .pkg package

Alternatively, you can download and install a standalone .pkg package from [`ggshield` release page](https://github.com/GitGuardian/ggshield/releases).

This package _does not_ require installing Python, but you have to manually download new versions.

## Linux

### Deb and RPM packages

Deb and RPM packages are available on [Cloudsmith](https://cloudsmith.io/~gitguardian/repos/ggshield/packages/).

Setup instructions:

- [Deb packages](https://cloudsmith.io/~gitguardian/repos/ggshield/setup/#formats-deb)
- [RPM packages](https://cloudsmith.io/~gitguardian/repos/ggshield/setup/#formats-rpm)

Upgrading is handled by the package manager.

## All operating systems

`ggshield` can be installed on all supported operating systems via its [PyPI package](https://pypi.org/project/ggshield).

### Using pipx

The recommended way to install `ggshield` from PyPI is to use [pipx](https://pypa.github.io/pipx/), which will install it in an isolated environment:

```shell
$ pipx install ggshield
```

To upgrade your installation, run:

```shell
$ pipx upgrade ggshield
```

### Using pip

You can also install `ggshield` from PyPI using pip, but this is not recommended because the installation is not isolated, so other applications or packages installed this way may affect your `ggshield` installation. This method will also not work if your Python installation is declared as externally managed (for example when using the system Python on operating systems like Debian 12):

```shell
$ pip install --user ggshield
```

To upgrade your installation, run:

```shell
$ pip install --user --upgrade ggshield
```

# Initial setup

## Using `ggshield auth login`

To use `ggshield` you need to authenticate against GitGuardian servers. To do so, use the `ggshield auth login` command. This command automates the provisioning of a personal access token and its configuration on the local workstation.

You can learn more about it from [`ggshield auth login` documentation](https://docs.gitguardian.com/internal-repositories-monitoring/ggshield/reference/auth/login).

## Manual setup

You can also create your personal access token manually and store it in the `GITGUARDIAN_API_KEY` environment variable to complete the setup.

# Getting started

## Secrets

You can now use `ggshield` to search for secrets:

- in files: `ggshield secret scan path -r .`
- in repositories: `ggshield secret scan repo .`
- in Docker images: `ggshield secret scan docker ubuntu:22.04`
- in Pypi packages: `ggshield secret scan pypi flask`
- and more, have a look at `ggshield secret scan --help` output for details.

## Infra as Code Security (IaC)

You can also search for vulnerabilities in your IaC files using the following command:

```
ggshield iac scan all .
```

However, if you are only interested in _new_ potential IaC vulnerabilities, you can run:

```
ggshield iac scan diff --ref=HEAD~1 .
```

Have a look at `ggshield iac scan --help` for more details.

# Integrations

You can integrate `ggshield` in your [CI/CD workflow](https://docs.gitguardian.com/ggshield-docs/integrations/overview#cicd-integrations-secrets-detection-in-your-cicd-workflow).

To catch errors earlier, use `ggshield` as a [pre-commit, pre-push or pre-receive Git hook](https://docs.gitguardian.com/ggshield-docs/integrations/overview#git-hooks-prevent-secrets-from-reaching-your-vcs).

# Learn more

For more information, have a look at [the documentation](https://docs.gitguardian.com/ggshield-docs/getting-started)

# Output

If no secrets or policy breaks have been found, the exit code will be 0:

```bash
$ ggshield secret scan pre-commit
```

If a secret or other issue is found in your staged code or in your CI, you will have an alert giving you the type of policy break, the filename where the policy break has been found and a patch giving you the position of the policy break in the file:

```shell
$ ggshield secret scan pre-commit

🛡  ⚔  🛡  2 policy breaks have been found in file production.rb

11 | config.paperclip_defaults = {
12 |     :s3_credentials => {
13 |     :bucket => "XXX",
14 |     :access_key_id => "XXXXXXXXXXXXXXXXXXXX",
                            |_____AWS Keys_____|

15 |     :secret_access_key => "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
                                |_______________AWS Keys_______________|

16 |     }
17 | }
```

Lines that are too long are truncated to match the size of the terminal, unless the verbose mode is used (`-v` or `--verbose`).

# Related open source projects

- [truffleHog](https://github.com/dxa4481/truffleHog)
- [gitleaks](https://github.com/zricethezav/gitleaks)
- [gitrob](https://github.com/michenriksen/gitrob)
- [git-hound](https://github.com/tillson/git-hound)
- [AWS git-secrets](https://github.com/awslabs/git-secrets)
- [detect-secrets](https://github.com/Yelp/detect-secrets)

# License

`ggshield` is MIT licensed.


[(en-US)](./readme-en.md) | [(ko-KR)](./readme.md)

# 오픈나무
[![Python 3.8 이상](https://img.shields.io/badge/python->=%203.8-blue.svg)](https://python.org)
[![라이선스](https://img.shields.io/badge/license-BSD%203--Clause-lightgrey.svg)](./LICENSE)

![](https://raw.githubusercontent.com/openNAMU/openNAMU/beta/.github/logo.png)

오픈나무는 파이썬 기반의 위키 엔진입니다.

## 시작하기
오픈나무는 파이썬 환경에서 동작하는 파이썬 애플리케이션으로, 파이썬 환경을 필요로 합니다.

[여기](https://2du.pythonanywhere.com/w/설치법)를 눌러 설치 가이드를 볼 수 있습니다.

### 클론
아래 명령을 터미널(명령 프롬프트)에 입력하여 본 리포지토리를 클론할 수 있습니다.
 * 일반: `git clone -b stable https://github.com/openNAMU/openNAMU.git`
 * 베타: `git clone -b beta https://github.com/openNAMU/openNAMU.git`
 * 개발: `git clone -b dev https://github.com/openNAMU/openNAMU.git`

## 기여
오픈나무에는 확인되지 않은 버그가 존재할 수 있습니다. 이를 보고해주시면 오픈나무의 발전을 도울 수 있습니다. [여기](https://github.com/openNAMU/openNAMU/issues/new)를 눌러 버그를 보고해주세요.

오픈나무는 오픈소스 프로젝트입니다. 원한다면 직접 코드를 수정하고 [Pull Request](https://github.com/openNAMU/openNAMU/compare)를 보낼 수 있습니다.

## 라이선스
오픈나무 프로젝트는 [BSD 3-Clause License](./LICENSE) [(ko-KR)](https://www.olis.or.kr/license/Detailselect.do?lId=1092)이며, 오픈나무 프로젝트를 사용하고자 한다면 라이선스를 준수해야 합니다. 자세한 내용은 문서를 참고하세요.

### 포함된 외부 프로젝트
 * [Quotes icon - Dave Gandy](http://www.flaticon.com/free-icon/quote-left_25672)
 * [highlight.js](https://highlightjs.org/)
 * [KaTeX](https://katex.org/)
 * [Feather](https://feathericons.com/)
 * [go.mod](https://github.com/openNAMU/openNAMU/blob/beta/route_go/go.mod)

### 도움을 주신 분들
 * [Team Croatia](https://github.com/TeamCroatia)
 * Basix
 * Efrit
 * 기타 여러 사람들

## 지원 문법
 * 나무마크 (NamuMark)
 * 마크다운 (Markdown) (Beta)

## 기타
 * 첫 가입자에게 소유자 권한이 부여됩니다.
 * [테스트 서버](http://2du.pythonanywhere.com)
 * [기여자 목록](https://github.com/openNAMU/openNAMU/graphs/contributors)


# F-UJI (FAIRsFAIR Research Data Object Assessment Service)
Developers: [Robert Huber](mailto:rhuber@marum.de), [Anusuriya Devaraju](mailto:anusuriya.devaraju@googlemail.com)

Thanks to [Heinz-Alexander Fuetterer](https://github.com/afuetterer) for his contributions and his help in cleaning up the code.

[![CI](https://github.com/pangaea-data-publisher/fuji/actions/workflows/ci.yml/badge.svg)](https://github.com/pangaea-data-publisher/fuji/actions/workflows/ci.yml)
[![Coverage](https://pangaea-data-publisher.github.io/fuji/coverage/coveragebadge.svg)](https://pangaea-data-publisher.github.io/fuji/coverage/)

[![Publish Docker image](https://github.com/pangaea-data-publisher/fuji/actions/workflows/publish-docker.yml/badge.svg)](https://github.com/pangaea-data-publisher/fuji/actions/workflows/publish-docker.yml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.11084909.svg)](https://doi.org/10.5281/zenodo.11084909)

## Overview

F-UJI is a web service to programmatically assess FAIRness of research data objects based on [metrics](https://doi.org/10.5281/zenodo.3775793) developed by the [FAIRsFAIR](https://www.fairsfair.eu/) project.
The service will be applied to demonstrate the evaluation of objects in repositories selected for in-depth collaboration with the project.

The '__F__' stands for FAIR (of course) and '__UJI__' means 'Test' in Malay. So __F-UJI__ is a FAIR testing tool.

**Cite as**

Devaraju, A. and Huber, R. (2021). An automated solution for measuring the progress toward FAIR research data. Patterns, vol 2(11), https://doi.org/10.1016/j.patter.2021.100370

### Clients and User Interface

A web demo using F-UJI is available at <https://www.f-uji.net>.

An R client package that was generated from the F-UJI OpenAPI definition is available from <https://github.com/NFDI4Chem/rfuji>.

An open source web client for F-UJI is available at <https://github.com/MaastrichtU-IDS/fairificator>.

## Assessment Scope, Constraint and Limitation
The service is **in development** and its assessment depends on several factors.
- In the FAIR ecosystem, FAIR assessment must go beyond the object itself. FAIR enabling services and repositories are vital to ensure that research data objects remain FAIR over time. Importantly, machine-readable services (e.g., registries) and documents (e.g., policies) are required to enable automated tests.
- In addition to repository and services requirements, automated testing depends on clear machine assessable criteria. Some aspects (rich, plurality, accurate, relevant) specified in FAIR principles still require human mediation and interpretation.
- The tests must focus on generally applicable data/metadata characteristics until domain/community-driven criteria have been agreed (e.g., appropriate schemas and required elements for usage/access control, etc.). For example, for some metrics (i.e., on I and R principles), the automated tests we proposed only inspect the ‘surface’ of criteria to be evaluated. Therefore, tests are designed in consideration of generic cross-domain metadata standards such as Dublin Core, DCAT, DataCite, schema.org, etc.
- FAIR assessment is performed based on aggregated metadata; this includes metadata embedded in the data (landing) page, metadata retrieved from a PID provider (e.g., DataCite content negotiation) and other services (e.g., re3data).

![alt text](https://github.com/pangaea-data-publisher/fuji/blob/master/fuji_server/static/main.png?raw=true)

## Requirements
[Python](https://www.python.org/downloads/) `3.11`

### Google Dataset Search
* Download the latest Dataset Search corpus file from: <https://www.kaggle.com/googleai/dataset-search-metadata-for-datasets>
* Open file `fuji_server/helper/create_google_cache_db.py` and set variable 'google_file_location' according to the file location of the corpus file
* Run `create_google_cache_db.py` which creates a SQLite database in the data directory. From root directory run `python3 -m fuji_server.helper.create_google_cache_db`.

The service was generated by the [swagger-codegen](https://github.com/swagger-api/swagger-codegen) project. By using the
[OpenAPI-Spec](https://github.com/swagger-api/swagger-core/wiki) from a remote server, you can easily generate a server stub.
The service uses the [Connexion](https://github.com/spec-first/connexion) library on top of Flask.

## Usage
Before running the service, please set user details in the configuration file, see config/users.py.

To install F-UJI, you may execute the following Python-based or docker-based installation commands from the root directory:

### Python module-based installation

From the fuji source folder run:
```bash
python -m pip install .
```
The F-UJI server can now be started with:
```bash
python -m fuji_server -c fuji_server/config/server.ini
```

The OpenAPI user interface is then available at <http://localhost:1071/fuji/api/v1/ui/>.

### Docker-based installation

```bash
docker run -d -p 1071:1071 ghcr.io/pangaea-data-publisher/fuji
```

To access the OpenAPI user interface, open the URL below in the browser:
<http://localhost:1071/fuji/api/v1/ui/>

Your OpenAPI definition lives here:

<http://localhost:1071/fuji/api/v1/openapi.json>

You can provide a different server config file this way:

```bash
docker run -d -p 1071:1071 -v server.ini:/usr/src/app/fuji_server/config/server.ini ghcr.io/pangaea-data-publisher/fuji
```

You can also build the docker image from the source code:

```bash
docker build -t <tag_name> .
docker run -d -p 1071:1071 <tag_name>
```

### Notes

To avoid Tika startup warning message, set environment variable `TIKA_LOG_PATH`. For more information, see [https://github.com/chrismattmann/tika-python](https://github.com/chrismattmann/tika-python)

If you receive the exception `urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]` on macOS, run the install command shipped with Python:
`./Install\ Certificates.command`.

F-UJI is using [basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), so username and password have to be provided for each REST call which can be configured in `fuji_server/config/users.py`.

## Development

First, make sure to read the [contribution guidelines](./CONTRIBUTING.md).
They include instructions on how to set up your environment with `pre-commit` and how to run the tests.

The repository includes a [simple web client](./simpleclient/) suitable for interacting with the API during development.
One way to run it would be with a LEMP stack (Linux, Nginx, MySQL, PHP), which is described in the following.

First, install the necessary packages:

```bash
sudo apt-get update
sudo apt-get install nginx
sudo ufw allow 'Nginx HTTP'
sudo service mysql start  # expects that mysql is already installed, if not run sudo apt install mysql-server
sudo service nginx start
sudo apt install php8.1-fpm php-mysql
sudo apt install php8.1-curl
sudo phpenmod curl
```

Next, configure the service by running `sudo vim /etc/nginx/sites-available/fuji-dev` and paste:

```php
server {
    listen 9000;
    server_name fuji-dev;
    root /var/www/fuji-dev;

    index index.php;

    location / {
        try_files $uri $uri/ =404;
    }

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock;
     }

    location ~ /\.ht {
        deny all;
    }
}
```

Link `simpleclient/index.php` and `simpleclient/icons/` to `/var/www/fuji-dev` by running `sudo ln <path_to_fuji>/fuji/simpleclient/* /var/www/fuji-dev/`. You might need to adjust the file permissions to allow non-root writes.

Next,
```bash
sudo ln -s /etc/nginx/sites-available/fuji-dev /etc/nginx/sites-enabled/
sudo nginx -t
sudo service nginx reload
sudo service php8.1-fpm start
```

The web client should now be available at <http://localhost:9000/>. Make sure to adjust the username and password in [`simpleclient/index.php`](./simpleclient/index.php).

After a restart, it may be necessary to start the services again:

```bash
sudo service php8.1-fpm start
sudo service nginx start
python -m fuji_server -c fuji_server/config/server.ini
```

### Component interaction (walkthrough)

This walkthrough can guide you through the comprehensive codebase.

A good starting point is [`fair_object_controller/assess_by_id`](fuji_server/controllers/fair_object_controller.py#36).
Here, we create a [`FAIRCheck`](fuji_server/controllers/fair_check.py) object called `ft`.
This reads the metrics YAML file during initialisation and will provide all the `check` methods.

Next, several harvesting methods are called, first [`harvest_all_metadata`](fuji_server/controllers/fair_check.py#329), followed by [`harvest_re3_data`](fuji_server/controllers/fair_check.py#345) (Datacite) and [`harvest_github`](fuji_server/controllers/fair_check.py#366) and finally [`harvest_all_data`](fuji_server/controllers/fair_check.py#359).
The harvesters are implemented separately in [`harvester/`](./fuji_server/harvester/), and each of them collects different kinds of data.
This is regardless of the defined metrics, the harvesters always run.
- The metadata harvester looks through HTML markup following schema.org, Dublincore etc., through signposting/typed links.
Ideally, it can find things like author information or license names that way.
- The data harvester is only run if the metadata harvester finds an `object_content_identifier` pointing at content files.
Then, the data harvester runs over the files and checks things like the file format.
- The Github harvester connects with the GitHub API to retrieve metadata and data from software repositories.
It relies on an access token being defined in [`config/github.cfg`](./fujji_server/config/github.cfg).

After harvesting, all evaluators are called.
Each specific evaluator, e.g. [`FAIREvaluatorLicense`](fuji_server/evaluators/fair_evaluator_license.py), is associated with a specific FsF and/or FAIR4RS metric.
Before the evaluator runs any checks on the harvested data, it asserts that its associated metric is listed in the metrics YAML file.
Only if it is, the evaluator runs through and computes a local score.

In the end, all scores are aggregated into F, A, I, R scores.

### Adding support for new metrics

Start by adding a new metrics YAML file in [`yaml/`](./fuji_server/yaml).
Its name has to match the following regular expression: `(metrics_v)?([0-9]+\.[0-9]+)(_[a-z]+)?(\.yaml)`,
and the content should be structured similarly to the existing metric files.

Metric names are tested for validity using regular expressions throughout the code.
If your metric names do not match those, not all components of the tool will execute as expected, so make sure to adjust the expressions.
Regular expression groups are also used for mapping to F, A, I, R categories for scoring, and debug messages are only displayed if they are associated with a valid metric.

Evaluators are mapped to metrics in their `__init__` methods, so adjust existing evaluators to associate with your metric as well or define new evaluators if needed.
The multiple test methods within an evaluator also check whether their specific test is defined.
[`FAIREvaluatorLicense`](fuji_server/evaluators/fair_evaluator_license.py) is an example of an evaluator corresponding to metrics from different sources.

For each metric, the maturity is determined as the maximum of the maturity associated with each passed test.
This means that if a test indicating maturity 3 is passed and one indicating maturity 2 is not passed, the metric will still be shown to be fulfilled with maturity 3.

### Community specific metrics

Some, not all, metrics can be configured using the following guidelines:
[Metrics configuration guide](https://github.com/pangaea-data-publisher/fuji/blob/master/metrics_configuration.md)

### Updates to the API

Making changes to the API requires re-generating parts of the code using Swagger.
First, edit [`fuji_server/yaml/openapi.yaml`](fuji_server/yaml/openapi.yaml).
Then, use the [Swagger Editor](https://editor.swagger.io/) to generate a python-flask server.
The zipped files should be automatically downloaded.
Unzip it.

Next:
1. Place the files in `swagger_server/models` into `fuji_server/models`, except `swagger_server/models/__init__.py`.
2. Rename all occurrences of `swagger_server` to `fuji_server`.
3. Add the content of `swagger_server/models/__init__.py` into `fuji_server/__init__.py`.

Unfortunately, the Swagger Editor doesn't always produce code that is compliant with PEP standards.
Run `pre-commit run` (or try to commit) and fix any errors that cannot be automatically fixed.

## License
This project is licensed under the MIT License; for more details, see the [LICENSE](https://github.com/pangaea-data-publisher/fuji/blob/master/LICENSE) file.

## Acknowledgements

F-UJI is a result of the [FAIRsFAIR](https://www.fairsfair.eu/) “Fostering FAIR Data Practices In Europe” project which received funding from the European Union’s Horizon 2020 project call H2020-INFRAEOSC-2018-2020 (grant agreement 831558).

The project was also supported through our contributors by the [Helmholtz Metadata Collaboration (HMC)](https://www.helmholtz-metadaten.de/en), an incubator-platform of the Helmholtz Association within the framework of the Information and Data Science strategic initiative.


  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://github.com/RapidAI/RapidOCR/releases/download/v1.1.0/RapidOCR_LOGO_white.png"  width="55%" height="55%">
    <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/RapidAI/RapidOCR/main/assets/RapidOCR_LOGO.png"  width="55%" height="55%">

  </picture>

    Open source OCR for the security of the digital world

=3.6,<3.13-aff.svg">

[简体中文](./docs/README_zh.md) | English

### Introduction
- The fastest running, most widely supported, completely open source and free multi-platform, multi-language OCR known to support rapid offline deployment.
- **Supported Languages**: The default is Chinese and English, other language recognition requires self-service conversion. For specific reference [here](https://rapidai.github.io/RapidOCRDocs/blog/2022/09/28/%E6%94%AF%E6%8C%81%E8%AF%86%E5%88%AB%E8%AF%AD%E8%A8%80/).
- **Cause**: [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) is not well engineered, and to make it easier for people to do OCR inference on various ends, we converted the model in PaddleOCR to ONNX format and ported it to various platforms using `Python/C++/Java/C#`.
- **Name Source**: Light, fast, economical and smart. OCR technology based on deep learning technology focuses on artificial intelligence advantages and small models, with speed as the mission and effect as the leading role.
- **Usage**:
  - If the existing model in the repo meets the requirements → RapidOCR deployment can be used.
  - Not meeting requirements → Based on PaddleOCR. Fine-tune your own data → RapidOCR deployment.
-If this repo is helpful to you, please click on a small star ⭐ Bah!

### [Demo](https://huggingface.co/spaces/SWHL/RapidOCRDemo)

### Installation
```bash
pip install rapidocr_onnxruntime
```

### Usage
```bash
rapidocr_onnxruntime -img 1.jpg
```

### Related projects overview
> In the table below, except for the **Evaluation Collection** part which is hosted under the [Hugging Face](https://huggingface.co/SWHL) Community, the rest are all under Github. The details are as follows:
>
> The first line is the function introduction.
>
> The second line is the corresponding warehouse name. You can directly search for the name on Github.

### Documentation
Full documentation can be found on [docs](https://rapidai.github.io/RapidOCRDocs/), in Chinese.

### Acknowledgements
- Many thanks to [DeliciaLaniD](https://github.com/DeliciaLaniD) for fixing the misplaced start position of scan animation in ocrweb.
- Many thanks to [zhsunlight](https://github.com/zhsunlight) for the suggestion about parameterized call GPU reasoning and the careful and thoughtful testing.
- Many thanks to [lzh111222334](https://github.com/lzh111222334) for fixing some bugs of rec preprocessing under python version.
- Many thanks to [AutumnSun1996](https://github.com/AutumnSun1996) for the suggestion in the [#42](https://github.com/RapidAI/RapidOCR/issues/42).
- Many thanks to [DeadWood8](https://github.com/DeadWood8) for providing the [document](https://rapidai.github.io/RapidOCRDocs/install_usage/rapidocr_web/nuitka_package) which packages rapidocr_web to exe by Nuitka.
- Many thanks to [Loovelj](https://github.com/Loovelj) for fixing the bug of sorting the text boxes. For details see [issue 75](https://github.com/RapidAI/RapidOCR/issues/75).

### Code Contributors

### Contributing
- Pull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.
- Please make sure to update tests as appropriate.

### [Sponsor](https://rapidai.github.io/RapidOCRDocs/sponsor/)

> [!IMPORTANT]
>
> If you want to sponsor the project, you can directly click the **Buy me a coffee** image, please write a note (e.g. your github account name) to facilitate adding to the sponsorship list below.
>
>

>
>

|                                                                    Sponsor                                                                     |                                                                       Applied Products                                                                        |
| :--------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------: |
|   |  |
|  |                                                                               -                                                                               |

### Citation
If you find this project useful in your research, please consider cite:
```bibtex
@misc{RapidOCR 2021,
    title={{Rapid OCR}: OCR Toolbox},
    author={RapidAI Team},
    howpublished = {\url{https://github.com/RapidAI/RapidOCR}},
    year={2021}
}
```

### Stargazers over time

[![Stargazers over time](https://starchart.cc/RapidAI/RapidOCR.svg)](https://starchart.cc/RapidAI/RapidOCR)

### License
The copyright of the OCR model is held by Baidu, while the copyrights of all other engineering scripts are retained by the repository's owner.

This project is released under the [Apache 2.0 license](./LICENSE).


# Mattermost Documentation

This repository generates the documentation available at https://docs.mattermost.com/. All documentation is available under the terms of a [Creative Commons License](https://creativecommons.org/licenses/by-nc-sa/3.0/).

If you have any questions, create an account on the [Mattermost Community server](https://community.mattermost.com/signup_user_complete/?id=f1924a8db44ff3bb41c96424cdc20676), then join the writing team in the [Documentation Working Group](https://community.mattermost.com/core/channels/dwg-documentation-working-group) channel. We look forward to working with you!

# Table of Contents

 * [Contributing](#contribute-to-mattermost-product-documentation)
     * [Get started](#get-started)
     * [Edit content directly on GitHub](#edit-content-directly-on-github)
     * [Create documentation PRs](#create-documentation-pull-requests)
     * [Use GitHub PR Labels](#use-github-pr-labels)
     * [Comment on Pull Requests](#comment-on-pull-requests)
     * [Review Pull Requests](#review-pull-requests)
 * [Build Mattermost product documentation locally](#build-locally)

## Contribute to Mattermost product documentation

### Get started

You can edit or create Mattermost documentation directly in GitHub, or by downloading the `mattermost/docs` repository onto your machine and using an IDE such as VS Code. Consult the Mattermost [Documentation Style Guide](https://handbook.mattermost.com/operations/research-and-development/product/technical-writing-team-handbook/documentation-style-guide) and [reStructuredText Markup](https://handbook.mattermost.com/operations/research-and-development/product/technical-writing-team-handbook/documentation-style-guide#using-restructuredtext-markup-rst) section for stylistic and technical guidance.

If this is your first time contributing to Mattermost, first read and sign the [Mattermost Contributor Agreement](https://mattermost.com/mattermost-contributor-agreement/), so you can be added to the Mattermost [Approved Contributor List](https://docs.google.com/spreadsheets/d/1NTCeG-iL_VS9bFqtmHSfwETo5f-8MQ7oMDE5IUYJi_Y/pubhtml?gid=0&single=true).

### Edit content directly on GitHub

The quickest way to begin is editing directly on GitHub on your fork of the Mattermost docs repo. Select the **Edit** icon on the top right corner of the page you want to edit in the Mattermost documentation.

If this is the first time you're contributing, follow these steps:
1. Select **Fork** in the top-right corner of the GitHub repository page to fork the current repository.
2. Navigate to file you want to edit, then select the Pencil icon (**Edit the file**) to open the editing interface.

### Create Documentation pull requests

1. When you're ready to submit your changes, add a descriptive title and comments to summarize the changes made.
2. Select **Create a new branch for this commit and start a pull request**.
3. Check the **Propose file change** button.
4. Scroll down to compare changes with the original document.
5. Select **Create pull request**.

### Use GitHub PR labels

GitHub PR labels are used to track the lifecycle and status of a pull request. Using the correct labels helps with managing workflows and ensuring that content is edited, merged and released at the correct time. For example, PRs that include an **Editor Review** label will be processed by an editor on the writing team to ensure the documentation is correctly formatted at https://docs.mattermost.com/ based on guidelines outlined in the style guide.

Take a look at the [Labels](https://developers.mattermost.com/contribute/getting-started/labels/) page for information about how and when to use which labels.

### Comment on pull requests

Once a pull request is submitted, multiple committers may comment on it and provide edits or suggestions which you can commit directly. You can also add line comments. Take a look at [Commenting on pull requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/commenting-on-a-pull-request) for more details.

### Review pull requests

Once a pull request has been submitted and the correct label assigned, the review process begins. This includes aligning the content with the Style Guide, validating processes, and tagging any other relevant committers. Read more about the review process and expectations [in the Mattermost Developer documentation](https://developers.mattermost.com/contribute/getting-started/code-review/).

Once the review process is complete, and depending on the type of issue it is (e.g., a typo fix vs. a new feature), the change is either merged into master and pushed immediately or merged into the release branch and pushed in alignment with a future release. The branch is then deleted.

## Build locally

If you've downloaded the `mattermost/docs` repository and are editing Mattermost documentation on your local machine, you can generate the HTML files from markdown in the `source` directory. You can review your changes before you commit them or create pull requests.

**Note:** Terminal commands can be executed on Linux, Mac, and Windows (using PowerShell).

### Build prerequisites

The following software is required to build the documentation:

- Git [[download]](https://git-scm.com/downloads)
- Python 3.9 or later [[download]](https://www.python.org/downloads)
- Pipenv [[download]](https://pipenv.pypa.io)

### Build instructions

1. Open a terminal window, then clone a forked copy of the documentation repository:
    ```shell
    git clone https://github.com/mattermost/docs.git
    ```

2. In the terminal window, navigate into the cloned repository:
    ```shell
    cd docs
    ```

3. Install [pipenv](https://docs.pipenv.org/) by using one of the following commands based on your operating system:

    For Mac users where Homebrew is installed:
    ```shell
    brew install pipenv
    ```

    For other operating systems:
    ```shell
    pip install --user pipenv
    ```

4. Install required Python packages:
    ```shell
    pipenv install --dev
    ```

5. Build the documentation set. You have three build commands available at the terminal:

    - Use `make html` to generate HTML files in the `/build` directory. Only file you've modified are re-built.
    - Use `make clean html` to delete all static HTML output in the `/build` directory and re-build all files. This command is particularly useful when you're making changes to the LHS navigation pane and want to ensure you're not reviewing cached results.
    - Use `make livehtml` to review a live preview published to `http://127.0.0.1:8000` that automatically updates as new changes are saved in your local IDE.

   Windows users will require [GNU Make](https://gnuwin32.sourceforge.net/packages/make.htm) installed for the above commands to work correctly. If GNU Make is not installed, please substitute `CMD /C make.bat` for `make` in the above commands to use the Windows command interpreter. For example `make html` will become `CMD /C make.bat html`.

   Note: When using the `CMD /C make.bat` substitution, only a single target may be specified. Instead of running `CMD /C make.bat clean html`, each target must be run seperately. For example, `CMD /C make.bat clean` followed by `CMD /C make.bat html`.

6. When working with static build results, navigate to the `build` directory:
    ```sh
    cd build
    ```

7. Then, preview your changes by opening the `build/html/index.html` file.

Build errors are written to the `build/warnings.log` file.


[![CircleCI](https://circleci.com/gh/spcl/serverless-benchmarks.svg?style=shield)](https://circleci.com/gh/spcl/serverless-benchmarks)
![Release](https://img.shields.io/github/v/release/spcl/serverless-benchmarks)
![License](https://img.shields.io/github/license/spcl/serverless-benchmarks)
![GitHub issues](https://img.shields.io/github/issues/spcl/serverless-benchmarks)
![GitHub pull requests](https://img.shields.io/github/issues-pr/spcl/serverless-benchmarks)

# SeBS: Serverless Benchmark Suite

**FaaS benchmarking suite for serverless functions with automatic build, deployment, and measurements.**

![Overview of SeBS features and components.](docs/overview.png)

SeBS is a diverse suite of FaaS benchmarks that allows automatic performance analysis of
commercial and open-source serverless platforms. We provide a suite of
[benchmark applications](docs/benchmarks.md) and [experiments](docs/experiments.md)
and use them to test and evaluate different components of FaaS systems.
See the [installation instructions](#installation) to learn how to configure SeBS to use selected
commercial and open-source serverless systems.
Then, take a look at [usage instructions](docs/usage.md) to see how
SeBS can automatically launch serverless functions and entire experiments in the cloud!

SeBS provides support for **automatic deployment** and invocation of benchmarks on
commercial and black-box platforms
[AWS Lambda](https://aws.amazon.com/lambda/),
[Azure Functions](https://azure.microsoft.com/en-us/services/functions/),
and [Google Cloud Functions](https://cloud.google.com/functions).
Furthermore, we support the open-source platform [OpenWhisk](https://openwhisk.apache.org/)
and offer a custom, Docker-based local evaluation platform.
See the [documentation on cloud providers](docs/platforms.md)
for details on configuring each platform in SeBS.
The documentation describes in detail [the design and implementation of our
tool](docs/design.md), and see the [modularity](docs/modularity.md)
section to learn how SeBS can be extended with new platforms, benchmarks, and experiments.
Find out more about our project in [a paper summary](https://mcopik.github.io/projects/sebs/).

Do you have further questions not answered by our documentation?
Did you encounter troubles with installing and using SeBS?
Or do you want to use SeBS in your work and you need new features?
Feel free to reach us through GitHub issues or by writing to <marcin.copik@inf.ethz.ch>.

For more information on how to configure, use, and extend SeBS, see our
documentation:

* [How to use SeBS?](docs/usage.md)
* [Which benchmark applications are offered?](docs/benchmarks.md)
* [Which experiments can be launched to evaluate FaaS platforms?](docs/experiments.md)
* [How to configure serverless platforms?](docs/platforms.md)
* [How SeBS builds and deploys functions?](docs/build.md)
* [How SeBS package is designed?](docs/design.md)
* [How to extend SeBS with new benchmarks, experiments, and platforms?](docs/modularity.md)

### Publication

When using SeBS, please cite our [Middleware '21 paper](https://dl.acm.org/doi/abs/10.1145/3464298.3476133).
An extended version of our paper is [available on arXiv](https://arxiv.org/abs/2012.14132), and you can
find more details about research work [in this paper summary](https://mcopik.github.io/projects/sebs/).
You can cite our software repository as well, using the citation button on the right.

```
@inproceedings{copik2021sebs,
  author = {Copik, Marcin and Kwasniewski, Grzegorz and Besta, Maciej and Podstawski, Michal and Hoefler, Torsten},
  title = {SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing},
  year = {2021},
  isbn = {9781450385343},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3464298.3476133},
  doi = {10.1145/3464298.3476133},
  booktitle = {Proceedings of the 22nd International Middleware Conference},
  pages = {64–78},
  numpages = {15},
  keywords = {benchmark, serverless, FaaS, function-as-a-service},
  location = {Qu\'{e}bec city, Canada},
  series = {Middleware '21}
}
```

## Installation

Requirements:
- Docker (at least 19)
- Python 3.7+ with:
    - pip
    - venv
- `libcurl` and its headers must be available on your system to install `pycurl`
- Standard Linux tools and `zip` installed

... and that should be all. We currently support Linux and other POSIX systems with Bash available.
On Windows, we recommend using WSL.

To install the benchmarks with a support for all platforms, use:

```
./install.py --aws --azure --gcp --openwhisk --local
```

It will create a virtual environment in `python-venv`, and install necessary Python
dependencies and third-party dependencies. To use SeBS, you must first active the new Python
virtual environment:

```
. python-venv/bin/activate
```

Now you can deploy serverless experiments :-)

The installation of additional platforms is controlled with the `--{platform}` and `--no-{platform}`
switches. Currently, the default behavior for `install.py` is to install only the
local environment.

To verify the correctness of installation, you can use [our regression testing](docs/usage.md#regression).

> [!WARNING]
> We use libcurl to make HTTP requests. `pycurl` will attempt to build its bindings and needs headers for that - make sure you have all development packages installed. If you see an error like this one: `src/pycurl.h:206:13: fatal error: gnutls/gnutls.h: No such file or directory`, it means that you are missing some of the dependencies.

> [!WARNING]
> **Make sure** that your Docker daemon is running and your user has sufficient permissions to use it. Otherwise, you might see many "Connection refused" and "Permission denied" errors when using SeBS.

## Authors

* [Marcin Copik (ETH Zurich)](https://github.com/mcopik/) - main author.
* [Michał Podstawski (Future Processing SA)](https://github.com/micpod/) - contributed graph and DNA benchmarks, and worked on Google Cloud support.
* [Laurin Brandner (ETH Zurich)](https://github.com/lbrndnr) - contributed serverless workflows.
* [Nico Graf (ETH Zurich)](https://github.com/ncograf/) - contributed to the implementation of regression tests and bugfixes and helped with testing and documentation.
* [Kacper Janda](https://github.com/Kacpro), [Mateusz Knapik](https://github.com/maknapik), [JmmCz](https://github.com/JmmCz), AGH University of Science and Technology - contributed together Google Cloud support.
* [Grzegorz Kwaśniewski (ETH Zurich)](https://github.com/gkwasniewski) - worked on the modeling experiments.
* [Paweł Żuk (University of Warsaw)](https://github.com/pmzuk) - contributed OpenWhisk support.
* [Sascha Kehrli (ETH Zurich)](https://github.com/skehrli) - contributed local measurement of Docker containers.
* [Kaleab](https://github.com/Kaleab-git) - contributed to SeBS local backend to make it portable between platforms and more robust on non-Linux systems.
* [lawrence910426](https://github.com/lawrence910426) - contributed color-coded output to SeBS CLI.


# Home Assistant integration for Husqvarna Automower

![Maintenance](https://img.shields.io/maintenance/yes/2023.svg)
[![buy me a coffee](https://img.shields.io/badge/If%20you%20like%20it-Buy%20me%20a%20coffee-orange.svg)](https://www.buymeacoffee.com/Thomas55555)
[![downloads](https://img.shields.io/github/downloads/Thomas55555/husqvarna_automower/total.svg)](https://img.shields.io/github/downloads/Thomas55555/husqvarna_automower/total.svg)

Custom component to support Automower.

- [Home Assistant integration for Husqvarna Automower](#home-assistant-integration-for-husqvarna-automower)
  - [About](#about)
  - [Supported devices](#supported-devices)
  - [Installation](#installation)
    - [Installation through HACS](#installation-through-hacs)
    - [Manual installation](#manual-installation)
  - [Configuration](#configuration)
    - [Husqvarna API-Key](#husqvarna-api-key)
    - [Home Assistant](#home-assistant)
    - [Image Sensor](#cimage-sensor)
      - [Example of map image](#example-of-map-image)
      - [Example of map image with zones enabled](#example-of-map-image-with-zones-enabled)
    - [Zone Sensor](#zone-sensor)
  - [Usage](#usage)
    - [Services](#services)
  - [Debugging](#debugging)
  - [Troubleshooting](#troubleshooting)
    - [Remove Credentials](#remove-credentials)
    - [Error: The component is not configured...](#error-the-component-is-not-configured)

## About

This Home Assistant integration provides status and control of supported Husqvarna  Automowers.  The official Husqvarna  [API](https://developer.husqvarnagroup.cloud/) uses websocket connection for pushed updates, so no polling is performed.  Park and Start commands including schedule overrides are supported by the integration allowing for robust automations to be implemented in Home Assistant.  Diagnostic and statics provided by the API are included with the integration for monitoring mower usage and performance.

![Screenshot of the integration](/images/screenshot_husqvarna_automower.png)

## Supported devices

Husqvarna Automowers with built-in *Automower® Connect* or with the *Automower® Connect Module* are supported.  This integration does not support Bluetooth connectivity with Automowers.

## Installation

Requires Home Assistant 2023.7.0 or newer.

### Installation through HACS

Installation using Home Assistant Community Store (HACS) is recommended.

1. If HACS is not installed, follow HACS installation and configuration at https://hacs.xyz/.

2. In HACS, search under integrations for Husqvarna Automower and install.

3. Restart Home Assistant!

### Manual installation

1. Download the `husqvarna_automower.zip` file from the repository [release section](https://github.com/Thomas55555/husqvarna_automower/releases).

2. Extract and copy the content into the path `/config/custom_components/husqvarna_automower` of your HA installation.

   Do **not** download directly from the `main` branch.

3. Restart Home Assistant!

## Configuration

### Husqvarna API-Key

In order to use this integration you must properly configure OAuth2 credentials using your Husqvarna account.  Refer to [this guide](https://developer.husqvarnagroup.cloud/docs/get-started) for general overview of the process.  Username/password authentication for this integration is no longer supported as of version 2022.7.0.

Your Husqvarna account username/password used for the *Automower® Connect*  phone app is required.  Most users probably created a Husqvarna account during initial mower setup.

1. Make sure, that the external url on the page `Settings -> Network` in the field `Internet` is set to your external url.

2. Go to <https://developer.husqvarnagroup.cloud/> and sign in with Husqvarna account.  Sign in page has password recovery/reset using registered email address if needed.  Authorize *Developer Porthole* to access Husqvarna account when prompted.

3. After signing in you will be automatically redirected to "Your applications". (Otherwise go to: <https://developer.husqvarnagroup.cloud/applications>)

4. Create a new application:

   * Name is required but can be anything, for example "My Home Assistant"

   * Description is optional

   * Redirect URL:

     ```
     https://my.home-assistant.io/redirect/oauth
     ```

     Confirm no extra spaces were appended at end of URL from copy and paste.

   * Click **CREATE**.  *Application Key* and *Application Secret* will be generated and shown.  Protect these like a username and password.

5. Click on **CONNECT NEW API** and connect the **Authentication API**.

6. Click on **CONNECT NEW API** again and connect the **Husqvarna Automower API**.

7. Leave this tab open in browser and continue with Home Assistant configuration.

### Home Assistant

The My Home Assistant redirect feature needs to be setup to redirect to your home assistant installation.  See https://my.home-assistant.io/faq/ for additional information.

1. Add the integration to your home assistant installation and test the redirect feature by following below link:
   [![my_button](https://my.home-assistant.io/badges/config_flow_start.svg)](https://my.home-assistant.io/redirect/config_flow_start/?domain=husqvarna_automower)

2. Acknowledge prompts to open link, install Husqvarna Automower integration.

3. Acknowledge prompt to setup application credentials.

4. Enter the following from the Husqvrana developer tab:

   * The name of the application assigned in Step 3 above
   * Copy and paste the *Application Key* into the *OAuth Client ID* field
   * Copy and paste the *Application Secret* into the *OAuth Client Secret* field

5. Click **Create**

6. Browser will be redirected to Husqvarna Developer site.  Sign in and Authorize the integration to connect with your Husqvarna account

7. After authorizing the integration the browser will show the my Home Assistant redirect link to link this account.  Click on **Link Account**.

8. Confirm successful connection of mower and assign to an HA area if desired.

### Image Sensor

#### [Detailed Instruction for creating and configuring the map image](MAP_GUIDE.md)

#### Example of map image
![Example of image](/images/map_image.png)

#### Example of map image with zones enabled
![Example of image with zones enabled](/images/map_image_zone.png)

The image entity is disabled by default.  The image entity will plot the current coordinates and location history of the mower on a user provided image. To configure the entity you need to upload your desired map image and determine the coordinates of the top left corner and the bottom right corner of your selected image.

The image entity is configured via the configure option on the integration. To enter the coordinates, ensure that they are in Signed Degree format and separated by a comma for example `40.689209, -74.044661`

You can then provide the path to the image you would like to use for the map and mower.  This has been tested with the PNG format, other formats may work.  The `.../resources/map_image.png` default image is over written when the integration is updated, store the custom image in another location.

The path color can be changes by providing an RGB value such as (255,0,0).

### Zone Sensor

The optional zone sensor allows zones to be designated by coordinates, this sensor will then return the name of the zone the mower is currently located.

To create a Zone, select new then enter a name for the zone and the coordinates of the zone.  Coordinates are entered in Signed Degree format with latitude and longitude separated by a comma and each coordinate separated by a semi colon. You must enter at least three coordinates to define a zone. For example: ```40.689209, -74.044661; 40.689210, -74.044652; 40.689211, -74.044655``` You must select save and then submit, exiting the flow in another manner will cause any entered zones to be lost.

If display zone is selected the zone will be drawn as an overlay on the map image in the provided RGB color.  To change the color provide an RGB string such as (255,255,255).

If a Home Zone is set, the sensor will return Home and the image will display the mower at the home location, when the mower is charging or at the docking station.

## Usage

* `vacuum.start`
  The mower continues to mow, within the specified schedule

* `vacuum.pause`
  Pauses the mower until a new command

* `vacuum.stop`
  The mower returns to the base and parks there until the next schedule starts

* `vacuum.return_to_base`
  The mower returns to the base and parks there until it gets a new start command

* `number.automower_park_for`
  Override schedule to park mower for specified number of minutes.

* `number.automower_mow_for`
  Override schedule to mow for specified number of minutes.

* `number.automower_cutting_height`
  Set mower cutting height.

* `select.automower_headlight_mode`
  Set the mower headlight operating mode

### Services

* `husqvarna_automower.calendar`
  Allows mower schedule to be revised.  Supports single schedule per day, this will override existing schedule.

  ```
  service: husqvarna_automower.calendar
  data:
    start: '11:45:00'
    end: '21:30:00'
    monday: true
    tuesday: true
    wednesday: true
    thursday: true
    friday: true
    saturday: false
    sunday: false
  target:
    entity_id: vacuum.automower
  ```
  `start` must be less than `end`.  Seconds are ignored.

* `husqvarna_automower.custom_command`
  Allows custom JSON formatted commands to be sent.

  Example equivalent to `vacuum.start`
  ```
  service: husqvarna_automower.custom_command
  data:
    command_type: actions
    json_string: >-
      {
       "data": {"type": "ResumeSchedule"}
      }
  target:
    entity_id: vacuum.automower
  ```
  See Husqvarna [API reference](https://developer.husqvarnagroup.cloud/apis/Automower+Connect+API#/swagger) for additional details.

## Debugging

To enable debug logging for this integration and related libraries you can control this in your Home Assistant `configuration.yaml` file.

Example:

```
logger:
  default: info
  logs:
    custom_components.husqvarna_automower: debug
    aioautomower: debug
```

After a restart detailed log entries will appear in `/config/home-assistant.log`.

## Troubleshooting

### Remove Credentials

The OAuth2 credentials can be removed from the home assistant user interface.  Navigate to the Integrations tab under settings.  Access the *Application Credentials* menu  by clicking on the Kebab (3 vertical dot menu icon)  or [direct link](https://my.home-assistant.io/redirect/application_credentials/).

### Error: The component is not configured...

[Dialog box](https://user-images.githubusercontent.com/28894450/178799277-ac61b8c7-4f53-4371-9afe-7ee9ee7e9a49.png) stating the component is not configured.  Please follow the documentation.  This error is resolved by removing OAuth2 credentials


# DaCy: An efficient and unified framework for danish NLP

[![PyPI](https://img.shields.io/pypi/v/dacy.svg)][pypi status]
[![pip downloads](https://img.shields.io/pypi/dm/dacy.svg)](https://pypi.org/project/dacy/)
[![Python Version](https://img.shields.io/pypi/pyversions/dacy)][pypi status]
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)]([ruff])
[![documentation](https://github.com/centre-for-humanities-computing/dacy/actions/workflows/documentation.yml/badge.svg)][documentation]
[![Tests](https://github.com/centre-for-humanities-computing/dacy/actions/workflows/tests.yml/badge.svg)][tests]

[pypi status]: https://pypi.org/project/dacy/
[documentation]: https://centre-for-humanities-computing.github.io/DaCy/
[tests]: https://github.com/centre-for-humanities-computing/dacy/actions?workflow=Tests
[ruff]: https://github.com/astral-sh/ruff

DaCy is a Danish natural language preprocessing framework made with SpaCy. Its largest pipeline has achieved State-of-the-Art performance on Named entity recognition, part-of-speech tagging and dependency parsing for Danish. Feel free to try out the [demo](https://huggingface.co/chcaa/da_dacy_medium_trf?text=DaCy+er+en+pipeline+til+anvendelse+af+dansk+sprogteknologi+lavet+af+K.+Enevoldsen%2C+L.+Hansen+og+K.+Nielbo+fra+Center+for+Humanities+Computing.). This repository contains material for using DaCy, reproducing the results and guides on the usage of the package. Furthermore, it also contains behavioral tests for biases and robustness of Danish NLP pipelines.

## 🔧 Installation

You can install `dacy` via [pip] from [PyPI]:

```bash
pip install dacy
```

## 👩💻 Usage
To use the model you first have to download either the small, medium, or large model. To see a list of all available models:

```python
import dacy
for model in dacy.models():
    print(model)
# ...
# da_dacy_small_trf-0.2.0
# da_dacy_medium_trf-0.2.0
# da_dacy_large_trf-0.2.0
```

To download and load a model simply execute:
```python
nlp = dacy.load("da_dacy_medium_trf-0.2.0")
# or equivalently (always loads the latest version)
nlp = dacy.load("medium")
```

To see more examples, see the [documentation].

# 📖 Documentation

| Documentation              |                                                                                |
| -------------------------- | ------------------------------------------------------------------------------------------- |
| 📚 **[Getting started]**    | Guides and instructions on how to use DaCy and its features.                                |
| 🦾 **[Performance]**        | A detailed description of the performance of DaCy and comparison with similar Danish models |
| 📰 **[News and changelog]** | New additions, changes and version history.                                                 |
| 🎛 **[API References]**     | The detailed reference for DaCy's API. Including function documentation                     |
| 🙋 **[FAQ]**                | Frequently asked questions                                                                  |

[Installation]: https://centre-for-humanities-computing.github.io/DaCy/installation.html
[Getting started]: https://centre-for-humanities-computing.github.io/DaCy/tutorials/basic.html
[api references]: https://centre-for-humanities-computing.github.io/DaCy/
[News and changelog]: https://centre-for-humanities-computing.github.io/DaCy/news.html
[FAQ]: https://centre-for-humanities-computing.github.io/DaCy/faq.html
[Performance]: https://centre-for-humanities-computing.github.io/DaCy/performance.html

   Training and reproduction

The folder `training` contains a range of folders with a SpaCy project for each model version. This allows for the reproduction of the results.

Want to learn more about how DaCy initially came to be, check out this [blog post](https://www.centre-for-humanities-computing.com/post/new-fast-and-efficient-state-of-the-art-in-danish-nlp/).

# 💬 Where to ask questions
To report issues or request features, please use the [GitHub Issue Tracker](https://github.com/centre-for-humanities-computing/DaCy/issues).
Questions related to SpaCy are kindly referred to the SpaCy GitHub or forum. Otherwise, please use the Discussion Forums.

| Type                           |                        |
| ------------------------------ | ---------------------- |
| 📚 **FAQ**                      | [FAQ]                  |
| 🚨 **Bug Reports**              | [GitHub Issue Tracker] |
| 🎁 **Feature Requests & Ideas** | [GitHub Issue Tracker] |
| 👩💻 **Usage Questions**          | [GitHub Discussions]   |
| 🗯 **General Discussion**       | [GitHub Discussions]   |

[Documentation]: https://centre-for-humanities-computing.github.io/DaCy/
[Installation]: https://centre-for-humanities-computing.github.io/dacy/installation.html
[Tutorials]: https://centre-for-humanities-computing.github.io/dacy/tutorials.html
[API Reference]: https://centre-for-humanities-computing.github.io/dacy/references.html
[FAQ]: https://centre-for-humanities-computing.github.io/dacy/faq.html
[github issue tracker]: https://github.com/centre-for-humanities-computing/dacy/issues
[github discussions]: https://github.com/centre-for-humanities-computing/dacy/discussions
[pip]: https://pypi.org/project/pip/
[PyPI]: https://pypi.org/project/dacy/


![title](https://github.com/bczsalba/pytermgui/raw/master/assets/readme/screenshot.png)

> Python TUI framework with mouse support, modular widget system, customizable and rapid terminal markup language and more!

```bash
pip3 install pytermgui
```

## Notice

A much better, more complete version of PTG's core ideas now exists over at [Shade 40](https://github.com/shade40). While PTG is not yet fully obsolete,
those libraries will be the primary focus of development going forward.

## Why?

Mostly because terminals are cool, but creating terminal apps has historically been difficult. PyTermGUI aims to provide a simple, readable and modular way to make the app of your dreams!

Terminal apps are (often):

- Easier to install
- Faster & more resource efficient
- Less prone to differences between environments (no IE7 here!)

...than their web or native counterparts.

## How?

We provide a couple of things to make your life easier:

- Sensible abstractions over most terminal standards
- A fully fledged, desktop-inspired window manager system with modals and completely customizable windows
- Mouse support out of the box with **0** configuration
- YAML (or Python) based styling engines
- TIM, our markup language for creating styled terminal text with expressive text, including systems for aliases & macros
- A bunch of things I can't think of right now :slightly_smiling_face:

Additionally, there are a couple of neat tools to make your general Python development easier:

- An inspection utility
- A pretty printer for both the REPL and IPython
- A way to create SVG and HTML screenshots of your terminal

## Examples

> All images below are generated directly from the source displayed by a PyTermGUI-powered SVG exporter tool, [Termage](https://github.com/bczsalba/termage).

Your first application, a simple clock:

```python3
import time

import pytermgui as ptg

def macro_time(fmt: str) -> str:
    return time.strftime(fmt)

ptg.tim.define("!time", macro_time)

with ptg.WindowManager() as manager:
    manager.layout.add_slot("Body")
    manager.add(
        ptg.Window("[bold]The current time is:[/]\n\n[!time 75]%c", box="EMPTY")
    )
```

Since strings are converted into the `Label` widget, and all widgets use markup for styling, we can use a custom-defined TIM macro function to return the current time. After running the above, you should see something like:

For something a bit more in-depth, see this contact form inspired by [asciimatics' example](https://github.com/peterbrittain/asciimatics#how-to-use-it):

```python3
import pytermgui as ptg

CONFIG = """
config:
    InputField:
        styles:
            prompt: dim italic
            cursor: '@72'
    Label:
        styles:
            value: dim bold

    Window:
        styles:
            border: '60'
            corner: '60'

    Container:
        styles:
            border: '96'
            corner: '96'
"""

with ptg.YamlLoader() as loader:
    loader.load(CONFIG)

with ptg.WindowManager() as manager:
    window = (
        ptg.Window(
            "",
            ptg.InputField("Balazs", prompt="Name: "),
            ptg.InputField("Some street", prompt="Address: "),
            ptg.InputField("+11 0 123 456", prompt="Phone number: "),
            "",
            ptg.Container(
                "Additional notes:",
                ptg.InputField(
                    "A whole bunch of\nMeaningful notes\nand stuff", multiline=True
                ),
                box="EMPTY_VERTICAL",
            ),
            "",
            ["Submit", lambda *_: submit(manager, window)],
            width=60,
            box="DOUBLE",
        )
        .set_title("[210 bold]New contact")
        .center()
    )

    manager.add(window)
```

This showcases the YAML-based config system, as well as some additional API. I recommended checking out the [source file](https://github.com/bczsalba/pytermgui/blob/master/utils/readme_scripts/contact.py) to see how the `submit` callback works.

## Not a fan of colors? We've got you!

PyTermGUI is one of the only TUI libraries that offers [NO_COLOR](https://no-color.org) support that doesn't ~~suck~~ ruin the usability & design of your apps.

This is how the above example looks like with the environment variable `NO_COLOR` set to anything. Note how contrast between colors is retained, as well as the inclusion of background colors:

## Older terminals? No problem!

We use algorithms based on human vision to convert and downgrade colors when the current terminal emulator doesn't support them. Here is a cool screenshot:

Disclaimer: Termage currently doesn't play nicely to changing colorsystems during runtime, so this image had to be captured natively :(

## Questions? See the docs!

Pretty much every single name in the library, private or public, has an insightful dockstring attached to it, and we are accumulating a growing amount of walkthrough-based documentations articles. See 'em all on the [doc website](https://ptg.bczsalba.com)!

## Contributions, issues et al.

If you have any problems using the library, feel free to open up a discussion or raise an issue ticket. If you would prefer to hack on the library yourself, see the [contribution guidelines](https://github.com/bczsalba/pytermgui/blob/master/CONTRIBUTING.md). Pull requests are encouraged, but make sure you aren't trying to fix an issue that others are already working on, for your own sake. :slightly_smiling_face:


.. image:: https://raw.githubusercontent.com/PlasmaControl/DESC/master/docs/_static/images/logo_med_clear.png

.. inclusion-marker-do-not-remove

################################
Stellarator Optimization Package
################################
|License| |DOI| |Issues| |Pypi|

|Docs| |UnitTests| |RegressionTests| |Codecov|

DESC solves for and optimizes 3D MHD equilibria using pseudo-spectral numerical methods
and automatic differentiation.

The theoretical approach and implementation details used by DESC are presented in the
following papers and documented at Theory_. Please cite our work if you use DESC!

- Dudt, D. & Kolemen, E. (2020). DESC: A Stellarator Equilibrium Solver.
  [`Physics of Plasmas <https://doi.org/10.1063/5.0020743>`__]
  [`pdf <https://github.com/PlasmaControl/DESC/blob/master/publications/dudt2020/dudt2020desc.pdf>`__]
- Panici, D. et al (2023). The DESC Stellarator Code Suite Part I: Quick and accurate equilibria computations.
  [`Journal of Plasma Physics <https://doi.org/10.1017/S0022377823000272>`__]
  [`pdf <https://github.com/PlasmaControl/DESC/blob/master/publications/panici2022/Panici_DESC_Stellarator_suite_part_I_quick_accurate_equilibria.pdf>`__]
- Conlin, R. et al. (2023). The DESC Stellarator Code Suite Part II: Perturbation and continuation methods.
  [`Journal of Plasma Physics <https://doi.org/10.1017/S0022377823000399>`__]
  [`pdf <https://github.com/PlasmaControl/DESC/blob/master/publications/conlin2022/conlin2022perturbations.pdf>`__]
- Dudt, D. et al. (2023). The DESC Stellarator Code Suite Part III: Quasi-symmetry optimization.
  [`Journal of Plasma Physics <https://doi.org/10.1017/S0022377823000235>`__]
  [`pdf <https://github.com/PlasmaControl/DESC/blob/master/publications/dudt2022/dudt2022optimization.pdf>`__]

.. _Theory: https://desc-docs.readthedocs.io/en/latest/theory_general.html

Quick Start
===========

The easiest way to install DESC is from PyPI: ``pip install desc-opt``

For more detailed instructions on installing DESC and its dependencies, see Installation_.

The best place to start learning about DESC is our tutorials:

- `Basic fixed boundary equilibrium`_: running from a VMEC input, creating an equilibrium from scratch
- `Advanced equilibrium`_: continuation and perturbation methods.
- `Free boundary equilibrium`_: vacuum and or finite beta with external field.
- `Using DESC outputs`_: analysis, plotting, saving to VMEC format.
- `Basic optimization`_: specifying objectives, fixing degrees of freedom.
- `Advanced optimization`_: advanced constraints, precise quasi-symmetry, constrained optimization.
- `Near axis constraints`_: loading solutions from QSC/QIC and fixing near axis expansion.

For details on the various objectives, constraints, optimizable objects and more, see
the full `api documentation`_.

If all you need is an equilibrium solution, the simplest method is through the command
line by giving an input file specifying the equilibrium and solver options, this
way can also can also accept VMEC input files.

The code is run using the syntax ``desc <path/to/inputfile>`` and the full list
of command line options are given in `Command Line Interface`_. (Note that you may have
to prepend the command with ``python -m``)

Refer to `Inputs`_ for documentation on how to format the input file.

The equilibrium solution is output in a HDF5 binary file, whose format is detailed in `Outputs`_.

.. _Installation: https://desc-docs.readthedocs.io/en/latest/installation.html
.. _Command Line Interface: https://desc-docs.readthedocs.io/en/latest/command_line.html
.. _Inputs: https://desc-docs.readthedocs.io/en/latest/input.html
.. _Outputs: https://desc-docs.readthedocs.io/en/latest/output.html
.. _Basic fixed boundary equilibrium: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/basic_equilibrium.html
.. _Advanced equilibrium: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/advanced_equilibrium_continuation.html
.. _Free boundary equilibrium: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/free_boundary_equilibrium.html
.. _Using DESC outputs: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/use_outputs.html
.. _Basic optimization: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/basic_optimization.html
.. _Advanced optimization: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/advanced_optimization.html
.. _Near axis constraints: https://desc-docs.readthedocs.io/en/latest/notebooks/tutorials/nae_constraint.html
.. _api documentation: https://desc-docs.readthedocs.io/en/latest/api.html

Repository Contents
===================

- desc_ contains the source code including the main script and supplemental files. Refer to the API_ documentation for details on all of the available functions and classes.
- docs_ contains the documentation files.
- tests_ contains routines for automatic testing.
- publications_ contains PDFs of publications by the DESC group, as well as scripts and data to reproduce the results of these papers.

.. _desc: https://github.com/PlasmaControl/DESC/tree/master/desc
.. _docs: https://github.com/PlasmaControl/DESC/tree/master/docs
.. _tests: https://github.com/PlasmaControl/DESC/tree/master/tests
.. _publications: https://github.com/PlasmaControl/DESC/tree/master/publications
.. _API: https://desc-docs.readthedocs.io/en/latest/api.html

Contribute
==========

- `Contributing guidelines <https://github.com/PlasmaControl/DESC/blob/master/CONTRIBUTING.rst>`_
- `Issue Tracker <https://github.com/PlasmaControl/DESC/issues>`_
- `Source Code <https://github.com/PlasmaControl/DESC/>`_
- `Documentation <https://desc-docs.readthedocs.io/>`_

.. |License| image:: https://img.shields.io/github/license/PlasmaControl/desc?color=blue&logo=open-source-initiative&logoColor=white
    :target: https://github.com/PlasmaControl/DESC/blob/master/LICENSE
    :alt: License

.. |DOI| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4876504.svg
   :target: https://doi.org/10.5281/zenodo.4876504
   :alt: DOI

.. |Docs| image:: https://img.shields.io/readthedocs/desc-docs?logo=Read-the-Docs
    :target: https://desc-docs.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation

.. |UnitTests| image:: https://github.com/PlasmaControl/DESC/actions/workflows/unittest.yml/badge.svg
    :target: https://github.com/PlasmaControl/DESC/actions/workflows/unittest.yml
    :alt: UnitTests

.. |RegressionTests| image:: https://github.com/PlasmaControl/DESC/actions/workflows/regression_test.yml/badge.svg
    :target: https://github.com/PlasmaControl/DESC/actions/workflows/regression_test.yml
    :alt: RegressionTests

.. |Codecov| image:: https://codecov.io/gh/PlasmaControl/DESC/branch/master/graph/badge.svg?token=5LDR4B1O7Z
    :target: https://codecov.io/github/PlasmaControl/DESC
    :alt: Coverage

.. |Issues| image:: https://img.shields.io/github/issues/PlasmaControl/DESC
    :target: https://github.com/PlasmaControl/DESC/issues
    :alt: GitHub issues

.. |Pypi| image:: https://img.shields.io/pypi/v/desc-opt
    :target: https://pypi.org/project/desc-opt/
    :alt: Pypi


.. image:: https://github.com/materialsvirtuallab/pymatgen-diffusion/actions/workflows/testing.yml/badge.svg
      :alt: CI Status
      :target: https://github.com/materialsvirtuallab/pymatgen-diffusion/actions/workflows/testing.yml

.. image:: https://coveralls.io/repos/github/materialsvirtuallab/pymatgen-diffusion/badge.svg?branch=master
      :target: https://coveralls.io/github/materialsvirtuallab/pymatgen-diffusion?branch=master

pymatgen-analysis-diffusion
===========================

Formerly pymatgen-diffusion, this is an add-on to pymatgen for diffusion
analysis that is developed by the Materials Virtual Lab. Note that it relies on
pymatgen for structural manipulations, file io, and preliminary analyses. This is
and will always be, a scientific work in progress. Pls check back regularly for
more details.

Major Update (v2021.3.5)
========================

pymatgen-analysis-diffusion is now released as a namespace package `pymatgen-analysis-diffusion` on PyPI. It should be
imported via `pymatgen.analysis.diffusion` instead `pymatgen_diffusion`. To install this package via pip::

    pip install pymatgen-analysis-diffusion

Features (non-exhaustive!)
==========================

1. Van-Hove analysis
2. Probability density
3. Clustering (e.g., k-means with periodic boundary conditions).
4. Migration path finding and IDPP.

Citing
======

If you use pymatgen-diffusion in your research, please cite the following
work::

    Deng, Z.; Zhu, Z.; Chu, I.H.; Ong, S. P. Data-Driven First-Principles
    Methods for the Study and Design of Alkali Superionic Conductors,
    Chem. Mater., 2016, acs.chemmater.6b02648, doi:10.1021/acs.chemmater.6b02648.

You should also include the following citation for the pymatgen core package
given that it forms the basis for most of the analyses::

    Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier,
    Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent Chevrier, Kristin A.
    Persson, Gerbrand Ceder. *Python Materials Genomics (pymatgen) : A Robust,
    Open-Source Python Library for Materials Analysis.* Computational
    Materials Science, 2013, 68, 314-319. doi:10.1016/j.commatsci.2012.10.028.

In addition, some of the analyses may also have relevant publications that
you should cite. Please consult the documentation of each module.

Contributing
============

We welcome contributions in all forms. If you'd like to contribute, please
fork this repository, make changes and send us a pull request!

Acknowledgments
===============

We gratefully acknowledge funding from the following agencies for the
development of this code:

1. US National Science Foundation’s Designing Materials to Revolutionize and
   Engineer our Future (DMREF) program under Grant No. 1436976 for the AIMD
   analysis package.
2. US Department of Energy, Oﬃce of Science, Basic Energy Sciences under
   Award No. DE-SC0012118 for the NEB analysis package.