**Status:** In Development. All tasks are subject to changes right now.

# NeuroGym

NeuroGym is a curated collection of neuroscience tasks with a common interface.
The goal is to facilitate training of neural network models on neuroscience tasks.

Documentation: https://neurogym.github.io/
- [Installation](#Installation)
- [Tasks](#Tasks)
- [Wrappers](#Wrappers)
- [Examples](#Examples)
- [Contributing](#Contributing)
- [Authors](#Authors)

NeuroGym inherits from the machine learning toolkit [Gym](https://github.com/openai/gym) by OpenAI,
and thus allows a wide range of well established machine learning algorithms to be easily trained on behavioral paradigms relevant for the neuroscience community.
NeuroGym also incorporates several properties and functions (e.g. continuous-time and trial-based tasks) that are important for neuroscience applications.
The toolkit also includes various modifier functions that allow easy configuration of new tasks.

![alt tag](docs/pipeline.png)

### Installation

You can perform a minimal install of ``neurogym`` with:

    git clone https://github.com/neurogym/neurogym.git
    cd neurogym
    pip install -e .

Or a full install by replacing the last command with ``pip install -e '.[all]'``

### Tasks
Currently implemented tasks can be found [here](https://github.com/gyyang/neurogym/blob/master/docs/envs.md).

### Wrappers
Wrappers (see [list](https://github.com/gyyang/neurogym/blob/master/docs/wrappers.md))
are short scripts that allow introducing modifications the original tasks. For instance, the Random Dots Motion task can be transformed into a reaction time task by passing it through the *reaction_time* wrapper. Alternatively, the *combine* wrapper allows training an agent in two different tasks simultaneously.

### Examples

NeuroGym is compatible with most packages that use OpenAI gym.
In this [example](https://github.com/gyyang/neurogym/blob/master/examples/example_neurogym_rl.ipynb) jupyter notebook we show how to train
a neural network with reinforcement learning algorithms using the
[Stable Baselines](https://github.com/hill-a/stable-baselines) toolbox.

### Contributing
Contributing new tasks should be easy. You can contribute tasks using the regular OpenAI gym format. If your task has a trial/period structure,
this [template](https://github.com/gyyang/neurogym/blob/master/examples/template.py) provides the basic structure that we recommend a task to have:

```
from gym import spaces
import neurogym as ngym

class YourTask(ngym.PeriodEnv):
    metadata = {}

    def __init__(self, dt=100, timing=None, extra_input_param=None):
        super().__init__(dt=dt)

    def new_trial(self, **kwargs):
        """
        new_trial() is called when a trial ends to generate the next trial.
        Here you have to set:
        The trial periods: fixation, stimulus...
        Optionally, you can set:
        The ground truth: the correct answer for the created trial.
        """

    def _step(self, action):
        """
        _step receives an action and returns:
            a new observation, obs
            reward associated with the action, reward
            a boolean variable indicating whether the experiment has end, done
            a dictionary with extra information:
                ground truth correct response, info['gt']
                boolean indicating the end of the trial, info['new_trial']
        """

        return obs, reward, done, {'new_trial': new_trial, 'gt': gt}

```

### Authors
* Contact

    [Manuel Molano](https://github.com/manuelmolano) (manuelmolanomazon@gmail.com).
    [Guangyu Robert Yang](https://github.com/gyyang) (gyyang.neuro@gmail.com).

* Contributors (listed in chronological order)

    [Marta Fradera](https://github.com/martafradera),
    [Jordi Pastor](https://github.com/pastorjordi),
    [Jeremy Forest](https://github.com/jeremyforest),
    [Ru-Yuan Zhang](https://github.com/ruyuanzhang)


# ndindex

![ndindex logo](docs/_static/ndindex_logo_white_bg.svg)

A Python library for manipulating indices of ndarrays.

The documentation for ndindex can be found at https://quansight-labs.github.io/ndindex/

ndindex is a library that allows representing and manipulating objects that
can be valid indices to numpy arrays, i.e., slices, integers, ellipses,
None, integer and boolean arrays, and tuples thereof. The goals of the library
are

- Provide a uniform API to manipulate these objects. Unlike the standard index
  objects themselves like `slice`, `int`, and `tuple`, which do not share any
  methods in common related to being indices, ndindex classes can all be
  manipulated uniformly. For example, `idx.args` always gives the arguments
  used to construct `idx`.

- Give 100% correct semantics as defined by numpy's ndarray. This means that
  ndindex will not make a transformation on an index object unless it is
  correct for all possible input array shapes. The only exception to this rule
  is that ndindex assumes that any given index will not raise IndexError (for
  instance, from an out of bounds integer index or from too few dimensions).
  For those operations where the array shape is known, there is a `reduce()`
  method to reduce an index to a simpler index that is equivalent for the
  given shape.

- Enable useful transformation and manipulation functions on index objects.

## Examples

**Canonicalize a slice (over a given shape, or independent of array shape)**

```py
>>> from ndindex import *
>>> Slice(-2, 10, 3).reduce()
Slice(-2, 10, 2)
>>> Slice(-2, 10, 3).reduce(5)
Slice(3, 4, 1)
```

**Compute the maximum length of a sliced axis**

```py
>>> import numpy as np
>>> len(Slice(2, 10, 3))
3
>>> len(np.arange(10)[2:10:3])
3
```

**Compute the shape of an array of shape `(10, 20)` indexed by `[0, 0:10]`**

```py
>>> Tuple(0, slice(0, 10)).newshape((10, 20))
(10,)
>>> np.ones((10, 20))[0, 0:10].shape
(10,)
```

**Check if an indexed array would be empty**

```py
>>> Tuple(0, ..., Slice(10, 20)).isempty((3, 4, 5))
True
>>> np.ones((3, 4, 5))[0,...,10:20]
array([], shape=(4, 0), dtype=float64)
```

See the [documentation](https://quansight-labs.github.io/ndindex/) for full details
on what ndindex can do.

## License

[MIT License](LICENSE)

## Acknowledgments

ndindex development is supported by [Quansight
Labs](https://labs.quansight.org/) and is sponsored in part by [the D. E.
Shaw group](https://www.deshaw.com/). The D. E. Shaw group collaborates with
Quansight on numerous open source projects, including Numba, Dask and Project
Jupyter.


.. image:: https://img.shields.io/pypi/v/dwave-system.svg
   :target: https://pypi.org/project/dwave-system

.. image:: https://img.shields.io/pypi/pyversions/dwave-system.svg?style=flat
    :target: https://pypi.org/project/dwave-system
    :alt: PyPI - Python Version

.. image:: https://codecov.io/gh/dwavesystems/dwave-system/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/dwavesystems/dwave-system

.. image:: https://circleci.com/gh/dwavesystems/dwave-system.svg?style=shield
   :target: https://circleci.com/gh/dwavesystems/dwave-system

.. index-start-marker

dwave-system
============

`dwave-system` is a basic API for easily incorporating the D-Wave system as a
sampler in the
`D-Wave Ocean software stack <https://docs.ocean.dwavesys.com/en/stable/overview/stack.html>`_,
directly or through `Leap <https://cloud.dwavesys.com/leap/>`_\ 's cloud-based
hybrid solvers. It includes ``DWaveSampler``, a dimod sampler that accepts and
passes system parameters such as system identification and authentication down
the stack, ``LeapHybridSampler``, for Leap's hybrid solvers, and other. It also
includes several useful composites---layers of pre- and post-processing---that
can be used with ``DWaveSampler`` to handle minor-embedding,
optimize chain strength, etc.

.. index-end-marker

Installation
------------

.. installation-start-marker

**Installation from PyPI:**

.. code-block:: bash

    pip install dwave-system

**Installation from PyPI with drivers:**

.. note::
    Prior to v0.3.0, running ``pip install dwave-system`` installed a driver dependency called ``dwave-drivers``
    (previously also called ``dwave-system-tuning``). This dependency has a restricted license and has been made optional
    as of v0.3.0, but is highly recommended. To view the license details:

    .. code-block:: python

        from dwave.drivers import __license__
        print(__license__)

To install with optional dependencies:

.. code-block:: bash

    pip install dwave-system[drivers] --extra-index-url https://pypi.dwavesys.com/simple

**Installation from source:**

.. code-block:: bash

    pip install -r requirements.txt
    python setup.py install

Note that installing from source installs ``dwave-drivers``. To uninstall the proprietary components:

.. code-block:: bash

    pip uninstall dwave-drivers

.. installation-end-marker

License
-------

Released under the Apache License 2.0. See LICENSE file.

Contributing
============

Ocean's `contributing guide <https://docs.ocean.dwavesys.com/en/stable/contributing.html>`_
has guidelines for contributing to Ocean packages.


===============
WESTPA 2.0
===============

|ghactions| |anaconda| |tutorial|

|docs| |usersgroup| |develgroup|

.. |ghactions| image:: https://github.com/westpa/westpa/actions/workflows/test.yaml/badge.svg?branch=westpa2
              :target: https://github.com/westpa/westpa/actions/workflows/test.yaml
              :alt: GitHub Actions

.. |docs| image:: https://readthedocs.org/projects/westpa/badge/?version=latest
         :target: https://westpa.readthedocs.io/en/latest/?badge=latest
         :alt: Documentation Status

.. |tutorial| image:: https://img.shields.io/badge/WESTPA-tutorials-blueviolet.svg
        :target: https://github.com/westpa/tutorials
        :alt: WESTPA Tutorials GitHub

.. |usersgroup| image:: https://img.shields.io/badge/Google%20Group-Users-lightgrey.svg
        :target: http://groups.google.com/group/westpa-users
        :alt: Users Google Group

.. |develgroup| image:: https://img.shields.io/badge/Google%20Group-Developers-lightgrey.svg
        :target: https://groups.google.com/g/westpa-devel
        :alt: Developers Google Group

.. |anaconda| image:: https://anaconda.org/conda-forge/westpa/badges/version.svg
   :alt: Anaconda
   :target: https://anaconda.org/conda-forge/westpa

--------
Overview
--------

WESTPA is a package for constructing and running stochastic simulations using the "weighted ensemble" approach
of Huber and Kim (1996). For use of WESTPA please cite the following:

Zwier, M.C., Adelman, J.L., Kaus, J.W., Pratt, A.J., Wong, K.F., Rego, N.B., Suarez, E., Lettieri, S.,
Wang, D.W., Grabe, M., Zuckerman, D.M., and Chong, L.T. "WESTPA: An Interoperable, Highly
Scalable Software Package For Weighted Ensemble Simulation and Analysis," J. Chem. Theory Comput., 11: 800−809 (2015).

Russo, J. D., Zhang, S., Leung, J.M.G., Bogetti, A.T., Thompson, J.P., DeGrave, A.J., Torrillo, P.A., Pratt, A.J.,
Wong, K.F., Xia, J., Copperman, J., Adelman, J.L., Zwier, M.C., LeBard, D.N., Zuckerman, D.M., Chong, L.T.
WESTPA 2.0: High-Performance Upgrades for Weighted Ensemble Simulations and Analysis of Longer-Timescale Applications.
J. Chem. Theory Comput., 18 (2): 638–649 (2022).

See this page_ and this powerpoint_ for an overview of weighted ensemble simulation.

To help us fund development and improve WESTPA please fill out a one-minute survey_ and consider
contributing documentation or code to the WESTPA community.

WESTPA is free software, licensed under the terms of the MIT License. See the file ``LICENSE`` for more information.

.. _survey: https://docs.google.com/forms/d/e/1FAIpQLSfWaB2aryInU06cXrCyAFmhD_gPibgOfFk-dspLEsXuS9-RGQ/viewform
.. _page: https://westpa.github.io/westpa/overview.html
.. _powerpoint: https://docs.google.com/presentation/d/1PKsklN5aYLlLSTDOLdNGti6pcGs4Rgxm/edit?usp=sharing&ouid=117278498028744119802&rtpof=true&sd=true

------------
Requirements
------------

WESTPA is written in Python and requires version 3.7 or later. WESTPA also requires a number of Python scientific software packages.
The simplest way to meet these requirements is to download the
Anaconda Python distribution from www.anaconda.com (free for all users).

WESTPA currently runs on Unix-like operating systems, including Linux and
Mac OS X. It is developed and tested on x86_64 machines running Linux.

--------------------------------
Obtaining and Installing WESTPA
--------------------------------

WESTPA is developed and tested on Unix-like operating systems, including Linux and Mac OS X.

Regardless of the chosen method of installation, before installing WESTPA, we recommend you to first install the Python 3 version provided by the latest free `Anaconda Python distribution`_. After installing Anaconda, create a new python environment for the WESTPA install with the following::

    conda create -n westpa python=3.11
    conda activate westpa

Then, we recommend installing WESTPA through conda or pip. Execute either of the following::

    conda install -c conda-forge westpa

or::

    python -m pip install westpa

See the install instructions on our `wiki`_ for more detailed information.

To install from source (**not recommended**), start by downloading the corresponding tar.gz file from the `releases page`_. After downloading the file, unpack the file and install WESTPA by executing the following::

    tar xvzf westpa-2022.09.tar.gz
    cd westpa
    python -m pip install -e .

.. _`releases page`: https://github.com/westpa/westpa/releases
.. _`Anaconda Python distribution`: https://www.anaconda.com/download
.. _`wiki`: https://github.com/westpa/westpa/wiki/Installing-WESTPA

---------------
Getting started
---------------

High-level tutorials of how to use the WESTPA software can be found here_.
Further, all WESTPA command-line tools provide detailed help when
given the -h/--help option.

Finally, while WESTPA is a powerful tool that enables expert simulators to access much longer
timescales than is practical with standard simulations, there can be a steep learning curve to
figuring out how to effectively run the simulations on your computing resource of choice.
For serious users who have completed the online tutorials and are ready for production simulations
of their system, we invite you to contact Lillian Chong (ltchong AT pitt DOT edu) about spending
a few days with her lab and/or setting up video conferencing sessions to help you get your
simulations off the ground.

.. _here: https://github.com/westpa/westpa/wiki/Tutorials

------------
Getting help
------------

WESTPA FAQ_

A mailing list for WESTPA is available, at which one can ask questions (or see
if a question one has was previously addressed). This is the preferred means
for obtaining help and support. See http://groups.google.com/group/westpa-users
to sign up or search archived messages.

.. _FAQ: https://github.com/westpa/westpa/wiki/Frequently-Asked-Questions

----------
Developers
----------

Search archived messages or post to the westpa-devel Google group: https://groups.google.com/group/westpa-devel.


# Git Gud

![Demonstration](./preview.gif)

## What is it?
Welcome to Git Gud, a command line game designed to help you learn how to use the popular version control system known as Git!
As levels progress, you will know more and more about git, and eventually become a git grandmaster!

If something's missing feel free to add an issue, or if you're interested, view the [contributors file](https://github.com/benthayer/git-gud/blob/main/CONTRIBUTING.md) and add something yourself! The project is intentially structured to make it very easy to add new levels!

If you're more of a visual learner, you should start with ["Learn Git Branching"](https://learngitbranching.js.org), and and then give Git Gud a try. Learn Git Branching is more visual, but with Git Gud, you're actually using git to complete the levels.

## How do I use it?
For install instructions, see below.

Once Git Gud is installed, typing "git gud" will produce output and will start telling you what to do.
Git Gud is meant to be like a game, and like a game, it has levels.
The levels are divided up into skills, each of which will introduce you to a new topic in Git.
It start off, assuming you have zero knowledge, and then builds up.
For each level, it will give you a goal and will explain what's going on.
Ideally, the game will teach you everything you need to know to beat it, but you're still encouraged to use other resources to learn as much as you want.

The beginning levels of the game start by getting you accustomed to the Git Gud interface, but later on, the training wheels come off, and you'll have to remember to type in the commands.
If you ever forget which commands there are, or if you want to start on a later level, you can always run "git gud help"
The most important commands are `git gud goal`, `git gud status`, `git gud explain` `git gud test`, and `git gud load next`.
Other commands are also useful, but the output of those commands should be enough to guide you through the level.

To get started, you need to initialize Git Gud in an empty directory.
Once Git Gud is initialized, it'll have full control over that directory, and it will start adding/removing commits and files.
There will normally be multiple branches, and you'll be expected to use Git commands to solve each level.
The levels range in difficulty, and require you to do different things.
Some levels are really easy and only require you to read the explanation, but others just give you a situation and you'll need to use what you've learned to solve the level.

### How to install
Git Gud is written in Python 3.
You'll need to have Python >=3.6 installed in your system for Git Gud to work.
I prefer using [Anaconda](https://docs.anaconda.com/anaconda/install/) to make sure everything works correctly, but you can also install with pip if you now what you're doing.

Once your environment is set up with Python >=3.6, installing is simple:
```
pip3 install git-gud
```
Getting started is also simple:
```
git gud
```
Git Gud will guide you through what to do

If either of those command don't work, there are verious things you can try:
 - Use `pip` instead of `pip3`
 - Make sure your PATH variable includes Python executables
 - User install: `pip3 install --user git-gud`
 - Use Anaconda


# OBO PURLs

This repository provides tools for managing OBO Foundry Permanent URLs (PURLs). Like <https://github.com/perma-id/w3id.org> we use per-directory Apache configuration files (`.htaccess` files), each of which uses `RedirectMatch` directives to redirect PURL requests to their proper targets. Unlike w3id.org, we do not edit the Apache configuration files by hand. Instead we have a simple YAML configuration format, and scripts to translate the YAML configuration into Apache configuration. The YAML files are easier to read and write, and allow us to validate and test PURLs automatically.

## Status

All http://purl.obolibrary.org/obo/ PURLs are now handled by this system, with the exception of some [open issues](https://github.com/OBOFoundry/purl.obolibrary.org/issues). PURLs that do not match any rule in this system will fall back to the old [PURL.org](http://purl.org) system.

## Adding and Updating PURLs

Please use one of these four options to make changes to the PURLs:

1. [Create a new issue](https://github.com/OBOFoundry/purl.obolibrary.org/issues/new) describing the change you require.

2. [Browse to the configuration file you want to change](https://github.com/OBOFoundry/purl.obolibrary.org/tree/master/config) and click the "pencil" icon to edit it.

3. [Add a new configuration file](https://github.com/OBOFoundry/purl.obolibrary.org/new/master/config).

4. [Fork this repository](https://help.github.com/articles/fork-a-repo/) and [make a pull request](https://help.github.com/articles/using-pull-requests/).

All changes are reviewed before they are merged into the `master` branch. Once merged, updated PURLs will be active within 20 minutes.

## Configuration Format

Each OBO project using this service gets a [YAML](http://yaml.org) configuration file in `config/`. That YAML configuration file is used to generate an Apache `.htaccess` file for that ontology. That Apache configuration will apply to all PURLs for that project.

Every YAML configuration file must have these fields:

- `idspace:` the project's [IDSPACE](http://obofoundry.org/id-policy.html), case sensitive, usually uppercase
- `base_url:` the part of a PURL that comes after the domain, usually lowercase
- `term_browser:` usually [`ontobee`](http://ontobee.org) or [`ols`](https://www.ebi.ac.uk/ols/index) but can be `custom` (see below)
- `products:` a list of primary files for the ontology and the URLs to redirect them to; an `.owl` file is required, and an `.obo` file is optional

Optional fields include:

- `example_terms:` a list of one or more term IDs for automated testing
- `base_redirect:` If your project redirects its `base_url`, then you will need a `base_redirect:` entry. So `base_redirect: http://obi-ontology.org` will redirect <http://purl.obolibrary.org/obo/obi> to <http://obi-ontology.org>.
- `entries:` a list of other PURLs under the `base_url`, see below

Here's an example adapted from the [`config/obi.yml`](config/obi.yml) file:

    idspace: OBI
    base_url: /obo/obi

    products:
    - obi.owl: http://svn.code.sf.net/p/obi/code/releases/2015-09-15/obi.owl

    term_browser: ontobee
    example_terms:
    - OBI_0000070

    entries:
    - exact: /wiki
      replacement: http://obi-ontology.org

Most of these fields are straightforward, but the `entries:` need some more explanation.

### Entries

Each YAML configuration file contains the keyword `entries:` followed by a list of entries. Each entry defines an Apache [RedirectMatch](https://httpd.apache.org/docs/2.4/mod/mod_alias.html#redirectmatch) directive for matching URLs and redirecting to new URLs. Every entry begins with a `- `, followed by keywords and values on indented lines. There are three types of entries:

1. **exact**: The simplest entry matches an exact URL and returns an exact replacement
2. **prefix**: These entries match the first part of a URL and replace just that prefix part
3. **regex**: These entries use powerful regular expressions, and should be avoided unless absolutely necessary.

The `#` character indicates a comment, which is not considered part of the configuration.

See the [`tools/examples/test2/test2.yml`](tools/examples/test2/test2.yml) and [`tools/examples/test2/test2.htaccess`](tools/examples/test2/test2.htaccess) for examples.

#### Exact

In the most common case, your PURL should match a unique URL and redirect to a unique URL. Here's an example from the `config/obi.yml` file:

    - exact: /obi.owl
      replacement: http://svn.code.sf.net/p/obi/code/releases/2015-10-20/obi.owl

This entry will match exactly the URL `http://purl.obolibrary.org/obo/obi/obi.owl`, and it will redirect to exactly `http://svn.code.sf.net/p/obi/code/releases/2015-10-20/obi.owl`. The matched domain name is fixed `http://purl.obolibrary.org`; the next part is project-specific `/obo/obi/`; the final part is taken from the entry `/obi.owl`. The replacement is expected to be a valid, absolute URL, starting with `http`.

Behind the scenes, the entry is translated into a case insensitive Apache RedirectMatch directive in `obo/obi/.htaccess` by escaping special characters and "anchoring" with initial `^`, the project's base URL, and final `$`:

    RedirectMatch temp "(?i)^/2015\-09\-15/obi\.owl$" "http://svn.code.sf.net/p/obi/code/releases/2015-09-15/obi.owl"

#### Prefix

You can also match and replace just the first part of a URL, leaving the rest unchanged. This allows you to define one entry that redirects many URLs matching a common prefix. Another example from `config/obi.yml`:

    - prefix: /branches/
      replacement: http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/

This entry will match the URL `http://purl.obolibrary.org/obo/obi/branches/obi.owl` (for example), replace the first part `http://purl.obolibrary.org/obo/obi/branches/` with `http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/`, resulting in `http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/obi.owl`. Effectively, the `obi.owl` is appended to the replacement.

The translation is similar, with the addition of `(.*)` wildcard and a `$1` "backreference" at the ends of the given strings:

    RedirectMatch temp "(?i)^/branches/(.*)$" "http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/$1"

#### Regex

Regular expression entries should only be needed very rarely, and should always be used very carefully.

For the regular expression type, the value of the `regex:` and `replacement:` keywords should contain regular expressions in exactly the format expected by Apache [RedirectMatch](https://httpd.apache.org/docs/2.4/mod/mod_alias.html#redirectmatch). The values will be quoted, but no other changes will be made to them. Consider using `(?i)` to make the match case insensitive.

#### Tests

Every `prefix` or `regex` entry should also have a `tests:` keyword, with a list of additional URLs to check. Each test requires a `from:` value (like `exact:`) and a `to:` value (like `replacement:`). Here's an example:

    - prefix: /branches/
      replacement: http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/
      tests:
      - from: /branches/obi.owl
        to: http://obi.svn.sourceforge.net/svnroot/obi/trunk/src/ontology/branches/obi.owl

#### Order of Entries

Apache RedirectMatch directives are processed in the [order that they appear](https://httpd.apache.org/docs/2.4/mod/mod_alias.html#order) in the configuration file. Be careful that your `prefix` and `regex` entries do not conflict with your other entries. The YAML-to-Apache translation preserves the order of entries, so you can control the order of processing, but it's best to avoid conflicts.

## Custom Term Browsers

If your project does not use Ontobee or OLS as a term browser, you must specify `term_browser: custom` in your project's YAML configuration file, and provide a `regex` entry in the [`config/obo.yml`](config/obo.yml) configuration file. Here's an example for [ChEBI](https://www.ebi.ac.uk/chebi/):

    # Terms for CHEBI
    - regex: ^/obo/CHEBI_(\d+)$
      replacement: http://www.ebi.ac.uk/chebi/searchId.do?chebiId=CHEBI:$1
      tests:
      - from: /CHEBI_15377
        to: http://www.ebi.ac.uk/chebi/searchId.do?chebiId=CHEBI:15377

Note that term redirect rules are case sensitive.

Since these are `regex` entries, and could affect multiple projects, we prefer that OBO admins are the only ones to edit `obo.yml`. If you need a change to the term redirect entry for your project, please [create a new issue](https://github.com/OBOFoundry/purl.obolibrary.org/issues/new).

## Development and Testing

The PURL system runs on Ubuntu Linux, but you can test your changes using a virtual machine (VM) that runs on Windows, Mac, or Linux. Your local development machine (Windows, Mac, or Linux) will be the "host" machine. The VM will be a copy of Ubuntu Linux that runs on your host, and can be thrown away when you're done testing.

You'll have to install these three tools on your host machine:

1. [VirtualBox](https://www.virtualbox.org) to run the VM
2. [Vagrant](https://www.vagrantup.com) to setting up the VM
3. [Ansible](https://) to provisioning the VM with the right software

All of these tools are free for you to use. If you're using macOS with [Homebrew](http://brew.sh), then you can install the three tools like this:

    brew install --cask Caskroom/cask/vagrant
    brew install --cask Caskroom/cask/virtualbox
    brew install ansible

Once the three tools are installed, check out a copy of this repository and start the VM:

    git clone https://github.com/OBOFoundry/purl.obolibrary.org.git
    cd purl.obolibrary.org/tools
    vagrant up

This will:

1. download an Ubuntu Linux virtual machine (using Vagrant and the `tools/Vagrantfile`)
2. run it (using VirtualBox)
3. configure it as a web server (using Ansible and the `tools/site.yml` file)

If something goes wrong with step 3, the `vagrant provision` command will run Ansible again. Please [report any issues](https://github.com/OBOFoundry/purl.obolibrary.org/issues) that you run into.

Use your favourite text editor on your host machine to make your changes to the files in the `purl.obolibrary.org` directory. That directory will be synchronized with the `/var/www/purl.obolibrary.org` directory inside the VM. When you're ready to test your changes, log in to the VM and rebuild the `.htaccess` files:

    vagrant ssh
    cd /var/www/purl.obolibrary.org
    make clean all

You can use the web browser on the host machine to see the results, using URLs starting with `http://172.16.100.10/obo/`, such as [`http://172.16.100.10/obo/OBI_0000070`](http://172.16.100.10/obo/OBI_0000070). You can also run an automated tests. To check a single `config/foo.yml` configuration file, run one of these commands

    make clean validate-foo
    make clean build-foo

To update and test the whole system, run

    make clean all test

Detailed test results will be listed in `tests/development/*.tsv` files, with their expected and actual values. If you're making changes to the project tools, you can test them against the `tools/examples/` files with:

    make clean test-examples

Expert users who have to run more extensive tests can consider (temporarily) modifying their `hosts` file to redirect `purl.obolibrary.org` to the test server.

When you're done with the VM, log out with `exit`. Then you can choose to suspend the VM with

    vagrant suspend

or delete the VM with

    vagrant destroy

You can test against the production PURL server using `make test-production`. We only make one request per second, to avoid abusing the server, so this can take along time.

### Optional: Sync VirtualBox Guest Additions

If you keep your development VM for any length of time you may be presented with this message upon starting your VM:
```
==> default: A newer version of the box 'ubuntu/trusty64' is available! You currently
==> default: have version '20190122.1.1'. The latest is version '20190206.0.0'. Run
==> default: `vagrant box update` to update.
```
If you upgrade, then the next time you resume your box you may receive the warning:
```
[default] The guest additions on this VM do not match the install version of
VirtualBox! This may cause things such as forwarded ports, shared
folders, and more to not work properly. If any of those things fail on
this machine, please update the guest additions and repackage the
box.
```

To automatically sync with VirtualBox's Guest Additions at startup (and thus avoid this warning) you can install `vagrant-vbguest` like so:

- `vagrant plugin install vagrant-vbguest` (in the tools directory on the host machine)

Now, whenever you bring up your VM, it will check the version of the VM's guest additions and automatically bring them up to date whenever this is needed.

## Deployment

Deployment is automated using [Ansible](http://ansible.com), and targets a stock Ubuntu Linux server with Python installed. You should install on a **fresh** server, not one that's running other applications, unless you **really** know what you're doing.

Install Ansible on your **local** machine, add the IP address or hostname of your **target** server to `tools/hosts`, then run:

    cd tools
    ansible-playbook -i hosts site.yml

Ansible uses SSH to connect to the target server an execute the tasks defined in [`tools/site.yml`](tools/site.yml). If you have trouble connecting, you may have to adjust your SSH configuration to be more automatic, say by editing your `.ssh/config`.

You can re-run Ansible as you make changes. Once the system is running, it will fetch changes from the master Git repository every 10 minutes. From your local machine, you can test all URLs against any target server, e.g.:

    export PRODUCTION=url.ontodev.org; make clean test-production

The `make safe-update` task will check [Travis-CI](https://travis-ci.org/OBOFoundry/purl.obolibrary.org) to ensure that the latest build on the master branch passed all automated tests, and that it is newer than the last time `safe-update` completed. Then it will pull from the Git repository and rebuild the site. This *should* be safe for a `cron` task to synchronize PURLs with the repository.

## Copyright

The copyright for the OBO PURL code and documentation belongs to the respective authors. The code is distributed under a [BSD3 license](https://github.com/OBOFoundry/purl.obolibrary.org/blob/master/LICENSE).


Anytask [![Build Status](https://github.com/znick/anytask/actions/workflows/anytask.yml/badge.svg)](https://github.com/znick/anytask/actions)
=======

Used Python3.8

local install
-------------

Development installation commands:

    # ... clone ...
    # ... cd ...
    . deploy_local_beta/run.sh
    ./anytask/manage.py runserver 127.0.0.1:8019 -v 3 --traceback

To activate environment in already deployed project run

    . deploy_local_beta/activate.sh

Run deploy_local_beta/run.sh -h for more information.


# Open Raadsinformatie API
Master: [![Build Status](https://semaphoreci.com/api/v1/jurrian/open-raadsinformatie/branches/master/shields_badge.svg)](https://semaphoreci.com/jurrian/open-raadsinformatie)
Develop: [![Build Status](https://semaphoreci.com/api/v1/jurrian/open-raadsinformatie/branches/develop/shields_badge.svg)](https://semaphoreci.com/jurrian/open-raadsinformatie)

Open Raadsinformatie (ORI) aims to collect and standardize governmental decision making data of Dutch municipalities.
This includes Meetings, Agenda Items, Documents, Motions and more.
Open Raadsinformatie is a collaborative effort of the [Open State Foundation](https://openstate.eu/), [Argu](https://argu.co) / [Ontola](https://ontola.io) and [VNG Realisatie](https://vngrealisatie.nl/).

## Important links

 - [Open Raadsinformatie homepage](http://www.openraadsinformatie.nl/)
 - [Official source code repository](https://github.com/openstate/open-raadsinformatie/)
 - [Issue tracker](https://github.com/openstate/open-raadsinformatie/issues)
 - [Search engine](http://zoek.openraadsinformatie.nl/)
 - [New search engine (beta)](https://ori.argu.co)
 - [Docs for V1 API](http://docs.openraadsinformatie.nl)

## Installation and usage

See this guide to [install the Open Raadsinformatie API](https://github.com/openstate/open-raadsinformatie/blob/master/docs/installation.rst) using Docker, Vagrant or manually. There are also a few usage commands to get you started. Check out the [maintenance guide](maintenance_guide.md) for info on how to manage this project in production.

## Supported Sources

Data extraction support is available for the following source systems:

- [GO GemeenteOplossingen](https://www.gemeenteoplossingen.nl/)
- [Visma Roxit / GreenValley](https://www.greenvalley.nl/)
- [Notubiz](https://notubiz.nl/)
- [iBabs](https://www.ibabs.eu/nl/)
- [Parlaeus](https://parlaeus.nl/)
- [Tweede Kamer](https://opendata.tweedekamer.nl/documentatie/api-documentatie-20/)

## How to add your municipality

Get in touch with [Sander Bakker](sander.bakker@vng.nl) from VNG Realisatie.
Your griffie (municipality clerk) needs to formally agree that the data becomes open data, and the source system might need some configuration.

## Related repositories

- [ori-search](https://github.com/ontola/ori-search/), the new search interface.
- [ori_api](https://github.com/ontola/ori_api/), the `id.` REST api.

## Contributing

Please read through our [contributing guidelines](https://github.com/openstate/open-raadsinformatie/blob/master/CONTRIBUTING.rst).
Included are directions for opening issues, coding standards, and notes on development.

## Bugs and feature requests

Have a bug or a feature request? Please first read through and search for existing and closed issues. If your problem
or idea is not addressed yet, [please open a new issue](https://github.com/openstate/open-raadsinformatie/issues/new).

## Authors and contributors

The Open Raadsinformatie API was originally based on the
[Open Cultuur Data API](https://github.com/openstate/open-cultuur-data/).

Authors and contributors of both projects are:

* Bart de Goede ([@bartdegoede](https://twitter.com/bartdegoede))
* Justin van Wees ([@justin_v_w](https://twitter.com/justin_v_w))
* Breyten Ernsting ([@breyten](https://twitter.com/breyten))
* Sicco van Sas ([@siccovansas](https://twitter.com/siccovansas))
* Jurrian Tromp ([@jurrian](https://github.com/jurrian), [@ontola](https://github.com/ontola))
* Jorrit van Belzen ([@jorritb](https://github.com/jorritb), [@ontola](https://github.com/ontola))

Contributors:

* [DutchCoders](http://dutchcoders.io/)
* [Benno Kruit](https://github.com/bennokr)

## Copyright and license

The Open Raadsinformatie project is distributed under the [MIT license](https://opensource.org/licenses/MIT).


|pic1| |pic2| |pic3|

.. |pic1| image:: https://github.com/scaleoutsystems/fedn/actions/workflows/integration-tests.yaml/badge.svg
   :target: https://github.com/scaleoutsystems/fedn/actions/workflows/integration-tests.yaml

.. |pic2| image:: https://badgen.net/badge/icon/discord?icon=discord&label
   :target: https://discord.gg/KMg4VwszAd

.. |pic3| image:: https://readthedocs.org/projects/fedn/badge/?version=latest&style=flat
   :target: https://fedn.readthedocs.io

FEDn
--------

FEDn empowers its users to create federated learning applications that seamlessly transition from local proofs-of-concept to secure distributed deployments.

Leverage a flexible pseudo-local sandbox to rapidly transition your existing ML project to a federated setting. Test and scale in real-world scenarios using FEDn Studio - a fully managed, secure deployment of all server-side components (SaaS).

We develop the FEDn framework following these core design principles:

-  **Seamless transition from proof-of-concepts to real-world FL**. FEDn has been designed to make the journey from R&D to real-world deployments as smooth as possibe. Develop your federated learning use case in a pseudo-local environment, then deploy it to FEDn Studio (cloud or on-premise) for real-world scenarios. No code change is required to go from development and testing to production.

-  **Designed for scalability and resilience.** FEDn enables model aggregation through multiple aggregation servers sharing the workload. A hierarchical architecture makes the framework well suited borh for cross-silo and cross-device use-cases. FEDn seamlessly recover from failures in all critical components, and manages intermittent client-connections, ensuring robust deployment in production environments.

-  **Secure by design.** FL clients do not need to open any ingress ports, facilitating distributed deployments across a wide variety of settings. Additionally, FEDn utilizes secure, industry-standard communication protocols and supports token-based authentication and RBAC for FL clients (JWT), providing flexible integration in production environments.

-  **Developer and data scientist friendly.** Extensive event logging and distributed tracing enables developers to monitor experiments in real-time, simplifying troubleshooting and auditing. Machine learning metrics can be accessed via both a Python API and visualized in an intuitive UI that helps the data scientists analyze and communicate ML-model training progress.

Features
=========

Core FL framework (this repository):

- Tiered federated learning architecture enabling massive scalability and resilience.
- Support for any ML framework (examples for PyTorch, Tensforflow/Keras and Scikit-learn)
- Extendable via a plug-in architecture (aggregators, load balancers, object storage backends, databases  etc.)
- Built-in federated algorithms (FedAvg, FedAdam, FedYogi, FedAdaGrad, etc.)
- CLI and Python API.
- Implement clients in any language (Python, C++, Kotlin etc.)
- No open ports needed client-side.
- Flexible deployment of server-side components using Docker / docker compose.

FEDn Studio - From development to FL in production:

-  Secure deployment of server-side / control-plane on Kubernetes.
-  UI with dashboards for orchestrating experiments and visualizing results
-  Team features - collaborate with other users in shared project workspaces.
-  Features for the trusted-third party: Manage access to the FL network, FL clients and training progress.
-  REST API for handling experiments/jobs.
-  View and export logging and tracing information.
-  Public cloud, dedicated cloud and on-premise deployment options.

Available clients:

- Python client (this repository)
- C++ client (`FEDn C++ client <https://github.com/scaleoutsystems/fedn-cpp-client>`__)
- Android Kotlin client (`FEDn Kotlin client <https://github.com/scaleoutsystems/fedn-android-client>`__)

Getting started
============================

Get started with FEDn in two steps:

1. Sign up for a `Free FEDn Studio account <https://fedn.scaleoutsystems.com/signup>`__
2. Take the `Quickstart tutorial <https://fedn.readthedocs.io/en/stable/quickstart.html>`__

FEDn Studio (SaaS) is free for academic use and personal development / small-scale testing and exploration. For users and teams requiring
additional project resources, dedicated support or other hosting options, `explore our plans <https://www.scaleoutsystems.com/start#pricing>`__.

Documentation
=============

More details about the architecture, deployment, and how to develop your own application and framework extensions are found in the documentation:

-  `Documentation <https://fedn.readthedocs.io>`__

FEDn Studio Deployment options
==============================

Several hosting options are available to suit different project settings.

-  `Public cloud (multi-tenant) <https://fedn.scaleoutsystems.com>`__: Managed multi-tenant deployment in public cloud.
-   Dedicated cloud (single-tenant): Managed, dedicated deployment in a cloud region of your choice (AWS, GCP, Azure, managed Kubernetes)
-   Self-managed: Set up a self-managed deployment in your VPC or on-premise Kubernets cluster using Helm Chart and container images provided by Scaleout.

Contact the Scaleout team for information.

Support
=================

Community support is available in our `Discord
server <https://discord.gg/KMg4VwszAd>`__.

Options are available for `Dedicated/custom support <https://www.scaleoutsystems.com/start#pricing>`__.

Making contributions
====================

All pull requests will be considered and are much appreciated. For
more details please refer to our `contribution
guidelines <https://github.com/scaleoutsystems/fedn/blob/master/CONTRIBUTING.md>`__.

Citation
========

If you use FEDn in your research, please cite:

::

   @article{ekmefjord2021scalable,
     title={Scalable federated machine learning with FEDn},
     author={Ekmefjord, Morgan and Ait-Mlouk, Addi and Alawadi, Sadi and {\AA}kesson, Mattias and Stoyanova, Desislava and Spjuth, Ola and Toor, Salman and Hellander, Andreas},
     journal={arXiv preprint arXiv:2103.00148},
     year={2021}
   }

License
=======

FEDn is licensed under Apache-2.0 (see `LICENSE <LICENSE>`__ file for
full information).

Use of FEDn Studio is subject to the `Terms of Use <https://www.scaleoutsystems.com/terms>`__.


<!--
SPDX-FileCopyrightText: 2009 Fermi Research Alliance, LLC
SPDX-License-Identifier: Apache-2.0
-->

# GlideinWMS

[![DOI](https://zenodo.org/badge/1833401.svg)](https://zenodo.org/badge/latestdoi/1833401)

![.github/workflows/pyunittest.yaml](https://github.com/GlideinWMS/glideinwms/workflows/.github/workflows/pyunittest.yaml/badge.svg)
![.github/workflows/pylint.yaml](https://github.com/GlideinWMS/glideinwms/workflows/.github/workflows/pylint.yaml/badge.svg)
![.github/workflows/bats.yaml](https://github.com/GlideinWMS/glideinwms/workflows/.github/workflows/bats.yaml/badge.svg)
![.github/workflows/pycodestyle.yaml](https://github.com/GlideinWMS/glideinwms/workflows/PyCodeStyle/badge.svg)
![.github/workflows/build.yaml](https://github.com/GlideinWMS/glideinwms/workflows/.github/workflows/build.yaml/badge.svg)
![.github/workflows/reuse.yaml](https://github.com/GlideinWMS/glideinwms/workflows/REUSE%20Licensing%20test/badge.svg)
![.github/workflows/docker_build.yml](<https://github.com/GlideinWMS/glideinwms/workflows/Build%20and%20Push%20Docker%20image%20(CI)/badge.svg>)

The purpose of the GlideinWMS is to provide a simple way to access distributed computing resources, including Grid resources, institutional clusters, commercial clouds (AWS and GCE) and HPC resources.
GlideinWMS is a Glidein Based WMS (Workload Management System) that works on top of HTCondor. Glideins, aka pilot jobs, are like placeholders, a mechanism by which one or more remote resources temporarily join a local HTCondor pool. The HTCondor system is used for scheduling and job control.

The code is available as RPM via the Open Science Grid yum repository
and is distributed under the Apache 2.0 license, see the [LICENSE](LICENSE) file.

For release notes, a detailed description and installation, configuration and use instructions see:
https://glideinwms.fnal.gov/

For build instruction and development guidlines see:
https://cdcvs.fnal.gov/redmine/projects/glideinwms/wiki


psd-tools
=========

``psd-tools`` is a Python package for working with Adobe Photoshop PSD files
as described in specification_.

|pypi| |build| |docs|

.. _specification: https://www.adobe.com/devnet-apps/photoshop/fileformatashtml/

.. |pypi| image:: https://img.shields.io/pypi/v/psd-tools.svg
    :alt: PyPI Version
    :target: https://pypi.python.org/pypi/psd-tools

.. |build| image:: https://github.com/psd-tools/psd-tools/actions/workflows/ci.yml/badge.svg
    :alt: Build
    :target: https://github.com/psd-tools/psd-tools/actions/workflows/ci.yml

.. |docs| image:: https://readthedocs.org/projects/psd-tools/badge/
    :alt: Document Status
    :target: http://psd-tools.readthedocs.io/en/latest/

Features
--------

Supported:

* Read and write of the low-level PSD/PSB file structure
* Raw layer image export in NumPy and PIL format

Limited support:

* Composition of basic pixel-based layers
* Composition of fill layer effects
* Vector masks
* Editing of some layer attributes such as layer name
* Blending modes except for dissolve
* Drawing of bezier curves

Not supported:

* Editing of layer structure, such as adding or removing a layer
* Composition of adjustment layers
* Composition of many layer effects
* Font rendering

Installation
------------

Use ``pip`` to install the package::

    pip install psd-tools

Getting started
---------------

.. code-block:: python

    from psd_tools import PSDImage

    psd = PSDImage.open('example.psd')
    psd.composite().save('example.png')

    for layer in psd:
        print(layer)
        layer_image = layer.composite()
        layer_image.save('%s.png' % layer.name)

Check out the documentation_ for features and details.

.. _documentation: https://psd-tools.readthedocs.io/

Contributing
------------

See contributing_ page.

.. _contributing: https://github.com/psd-tools/psd-tools/blob/master/docs/contributing.rst

.. note::

    PSD specification_ is far from complete. If you cannot find a desired
    information in the documentation_, you should inspect the low-level
    data structure.


![logo](collateral/logos/indy-logo.png)

[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/hyperledger/indy-node/tree/main)
# Indy Node
- [Indy Node](#indy-node)
  - [Announcements](#announcements)
    - [April 12 2023](#april-12-2023)
  - [About Indy Node](#about-indy-node)
        - [Relationship with Sovrin](#relationship-with-sovrin)
        - [Getting Started Guide](#getting-started-guide)
        - [Hyperledger Wiki-Indy](#hyperledger-wiki-indy)
  - [Technical Overview of Indy Blockchain](#technical-overview-of-indy-blockchain)
  - [Indy Node Repository Structure](#indy-node-repository-structure)
  - [Dependent Projects](#dependent-projects)
  - [Contact us](#contact-us)
  - [How to Contribute](#how-to-contribute)
  - [How to Install a Test Network](#how-to-install-a-test-network)
  - [How to Start Working with the Code](#how-to-start-working-with-the-code)
  - [Continuous Integration and Delivery](#continuous-integration-and-delivery)
  - [How to send a PR](#how-to-send-a-pr)
      - [How to send a PR to both plenum and node](#how-to-send-a-pr-to-both-plenum-and-node)
  - [Docs and links](#docs-and-links)

## Announcements

### April 12 2023

**_The project branches have changed._**

The `main` branch now contains the Ubuntu 20.04 work stream, and the previous `main` branch containing the Ubuntu 16.04 work stream has been moved to the `ubuntu-16.04` branch.  We encourage everyone to switch to using the new code and appreciate your patience while we stabilize the work flows and documentation on this new branch.

The following changes were made to the branches:
- `main` (default) renamed to `ubuntu-16.04`
  - This retargeted the associated PRs.
- `ubuntu-20.04-upgrade` set as the default branch.
- `ubuntu-20.04-upgrade` (default) renamed to `main`

## About Indy Node

This codebase embodies all the functionality to run nodes (validators and/or observers)
that provide a [self-sovereign identity ecosystem](https://sovrin.org) on top of a
distributed ledger. It is the core project for Indy; over time, all other indy-\* projects may
collapse into this one, except for [indy-sdk](https://github.com/hyperledger/indy-sdk).

Indy has its own distributed ledger based on RBFT.

##### Relationship with Sovrin
This code is independent from but commonly associated with [The Sovrin Foundation](https://sovrin.org). The Sovrin Foundation is a public utility
for identity, built on top of this codebase. People who install sovrin packages (e.g., with
`sudo apt install sovrin`) get prepackaged genesis transactions that integrate
with an Indy validator pool using [Sovrin's governance and trust framework](https://sovrin.org/wp-content/uploads/2018/03/Sovrin-Provisional-Trust-Framework-2017-06-28.pdf). However, it is possible to use Indy Node
with a different network, using whatever conventions a community chooses.

##### Getting Started Guide

We recommend that developers should explore
[Indy Walk through](https://github.com/hyperledger/indy-sdk/blob/master/docs/getting-started/indy-walkthrough.md) to learn about Indy basics
or [Getting Started Guide with VCX](https://github.com/hyperledger/indy-sdk/blob/master/vcx/docs/getting-started/getting-started.md) and
[Getting Started Notebook](https://github.com/hyperledger/indy-sdk/blob/master/docs/getting-started/getting-started.ipynb).

##### Hyperledger Wiki-Indy

If you haven't done so already, please visit the main resource for all things "Indy" to get acquainted with the code base, helpful resources, and up-to-date information: [Hyperledger Wiki-Indy](https://wiki.hyperledger.org/display/indy/Hyperledger+Indy).

## Technical Overview of Indy Blockchain
- Refer to our documentation site at `indy.readthedocs.io` for the most current documentation and walkthroughs
  - [indy.readthedocs.io/projects/plenum](https://indy.readthedocs.io/projects/plenum/en/latest/index.html)
  - [indy.readthedocs.io/projects/node](https://hyperledger-indy.readthedocs.io/projects/node/en/latest/index.html)
- Please visit [Technical Overview of Plenum](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/main.md).
- More documentation can be found in [indy-plenum-docs](https://github.com/hyperledger/indy-plenum/blob/master/docs)
and [indy-node-docs](https://github.com/hyperledger/indy-node/blob/master/docs/source).

## Indy Node Repository Structure

Indy Node repo consists of the following parts:
- indy-node:
    - [indy-plenum](https://github.com/hyperledger/indy-plenum)-based implementation of distributed ledger
    - Extends plenum's base pool functionality with specific transactions support (CLAIM_DEF, SCHEMA, POOL_UPGRADE, etc.)
- indy-common
    - Common code for indy-node
- scripts
    - Some scripts that can be run for installed Node (in particular, scripts to start Nodes, generate keys, prepare test Network, etc.)
- doc
    - A folder with documentation
- dev-setup
    - A folder with scripts helping to configure development environment (python, dependencies, projects, virtual environment)

## Dependent Projects

- [indy-plenum](https://github.com/hyperledger/indy-plenum)
    - The heart of the distributed ledger technology inside Hyperledger Indy.
    - Most probably you will need to make changes in Plenum if you want to contribute to Indy.
      So, if you want to work with Indy Node, you will need to have the Plenum code as well in most of the cases
      and work with two projects at the same time
      (see [How to Start Working with the Code](#how-to-start-working-with-the-code) below).
- [indy-sdk](https://github.com/hyperledger/indy-sdk)
    - An official SDK for Indy.
    - It contains client and anoncreds implementation
    - You don't need it to contribute to Indy-Node. But please use indy-sdk for your own applications dealing with Indy ecosystem.
- [ursa](https://github.com/hyperledger/ursa)
    - Hyperledger's shared crypto library
    - In particular, it contains BLS multi-signature crypto needed for state proofs support in Indy.

## Contact us

- Bugs, stories, and backlog for this codebase are managed in [Hyperledger's Jira](https://jira.hyperledger.org).
Use project name `INDY`.

[chat-url-indy]:https://chat.hyperledger.org/channel/indy
[chat-url-indy-node]:https://chat.hyperledger.org/channel/indy-node
[chat-image-indy]:https://raw.githubusercontent.com/hyperledger/chat-assets/master/indy.svg?sanitize=true
[chat-image-indy-node]:https://raw.githubusercontent.com/hyperledger/chat-assets/master/indy-node.svg?sanitize=true

- Join us on [Hyperledger's chat](https://chat.hyperledger.org/) in [![indy][chat-image-indy]][chat-url-indy] or [![indy-node][chat-image-indy-node]][chat-url-indy-node] to discuss.

## How to Contribute

- We'd love your help; see these [instructions on how to contribute](https://wiki.hyperledger.org/display/indy/How+to+Contribute).
- You may also want to read this info about [maintainers](MAINTAINERS.md).
- See [How to send a PR](#how-to-send-a-pr) below.

## How to Install a Test Network

You can have a look at [Start Nodes](docs/source/start-nodes.md)
to understand what needs to be done to create a Network, initialize and start Nodes, and what scripts are provided for this.

The described process is automated in one of the ways below (it allow to install a test Network):

 - **Docker** [Start Pool with Docker](environment/docker/pool/README.md)

 - **Docker-based pool using with new libindy-based CLI**:
   - [Start Pool Locally](https://github.com/hyperledger/indy-sdk/blob/master/README.md#how-to-start-local-nodes-pool-with-docker)
   - [Get Started with Libindy](https://github.com/hyperledger/indy-sdk/blob/master/doc/getting-started/getting-started.md)

 - **Also coming soon:** Create virtual machines in AWS.

## How to Start Working with the Code

Please have a look at [Dev Setup](docs/source/setup-dev.md)

## Continuous Integration and Delivery

Please have a look at [Continuous integration/delivery](docs/source/ci-cd.md)

## How to send a PR

- Make sure that you followed [write code guideline](docs/source/write-code-guideline.md) before sending a PR
- Do not create big PRs; send a PR for one feature or bug fix only.
 If a feature is too big, consider splitting a big PR to a number of small ones.
- Consider sending a design doc into `design` folder (as markdown or PlantUML diagram) for a new feature  before implementing it
- Make sure that a new feature or fix is covered by tests (try following TDD)
- Make sure that documentation is updated according to your changes
- Provide a full description of changes in the PR including Jira ticket number if any
- Make sure that all your commits have a DCO sign-off from the author
- Make sure that static code validation passed
(you can run `flake8 .` on the project root to check it; you can install flake8 from pypi: `pip install flake8`)
- Put the link to the PR into `#indy-pr-review` channel in Rocket.Chat
- A reviewer needs to start your tests first (add `test this please` comment to the PR)
- You need to make sure that all the tests pass
- A reviewer needs to review the code and approve the PR. If there are review comments, they will be put into the PR itself.
- You must process them (feel free to reply in the PR threads, or have a discussion in Rocket.Chat if needed)
- A reviewer or maintainer will merge the PR (we usually use Squash)

#### How to send a PR to both plenum and node
If you made changes in both indy-plenum and indy-node, you need to do the following:
- Raise a PR to indy-plenum's master and wait until code is reviewed and merged (see above)
    - So, a new build of indy-plenum is created
- Note a just built version `X.Y.Z.devB` of indy-plenum (you can check it in [pypi](https://pypi.python.org/pypi/indy-plenum) or on CI server).
- Change indy-plenum's dependency version to the new one in indy-node's [setup.py](https://github.com/hyperledger/indy-node/blob/master/setup.py).
- Raise PR to indy-node's master and wait until code is reviewed and merged (see above)
    - So, a new build of indy-node is created

## Docs and links

- Indy-plenum is based on [RBFT](https://pakupaku.me/plaublin/rbft/5000a297.pdf) protocol
- Please have a look at documents and diagrams in [docs/source](docs/source) folder
- Please have a look at documents and diagrams in Plenum's [docs](https://github.com/hyperledger/indy-plenum/tree/master/docs) folder, or on https://indy.readthedocs.io/projects/plenum :
    - [Technical Overview of Plenum](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/main.md)
    - [Plenum Consensus Algorithm Diagram](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/diagrams/consensus-protocol.png)
    - [Glossary](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/glossary.md)
    - [Storages](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/storage.md)
    - [Request Handling](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/request_handling.md)
    - [Catchup](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/catchup.md)
    - [Catchup Diagram](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/diagrams/catchup-procedure.png)
    - [Audit Ledger](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/audit_ledger.md)
    - [Plugins](https://github.com/hyperledger/indy-plenum/blob/master/docs/source/plugins.md)
- Relationship between Entities and Transactions: [relationship diagram](docs/source/relationship-diagram.png)
- Supported transactions and their format: [transactions](docs/source/transactions.md)
- Supported requests (write, read) and their format: [requests](docs/source/requests.md)
- [Network roles and permissions](https://github.com/hyperledger/indy-node/blob/master/docs/source/auth_rules.md)
- [Indy file folder structure guideline](docs/source/indy-file-structure-guideline.md)
- [Helper Scripts](docs/source/helper-scripts.md)
- [Pool Upgrade](docs/source/pool-upgrade.md)
- [Node Addition](docs/source/add-node.md)


[![PyPI](https://img.shields.io/pypi/v/whatshap.svg)](https://pypi.python.org/pypi/whatshap)
![CI](https://github.com/whatshap/whatshap/workflows/CI/badge.svg)
[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](http://bioconda.github.io/recipes/whatshap/README.html)

![WhatsHap logo](https://github.com/whatshap/whatshap/raw/main/logo/whatshap_logo.png)

WhatsHap is a software for phasing genomic variants using DNA sequencing
reads, also called *read-based phasing* or *haplotype assembly*. It is
especially suitable for long reads, but works also well with short reads.

For documentation and information on how to cite WhatsHap, please visit the [WhatsHap Homepage](https://whatshap.readthedocs.io/)


  ROS Wrapper for Intel(R) RealSense(TM) Cameras

  Latest release notes

[![rolling][rolling-badge]][rolling]
[![iron][iron-badge]][iron]
[![humble][humble-badge]][humble]
[![foxy][foxy-badge]][foxy]
[![ubuntu22][ubuntu22-badge]][ubuntu22]
[![ubuntu20][ubuntu20-badge]][ubuntu20]

![GitHubWorkflowStatus](https://img.shields.io/github/actions/workflow/status/IntelRealSense/realsense-ros/main.yml?logo=github&style=flat-square)
[![GitHubcontributors](https://img.shields.io/github/contributors/IntelRealSense/realsense-ros?style=flat-square)](CONTRIBUTING.md)
[![License](https://img.shields.io/github/license/IntelRealSense/realsense-ros?style=flat-square)](LICENSE)

## Table of contents
  * [ROS1 and ROS2 legacy](#ros1-and-ros2-legacy)
  * [Installation on Ubuntu](#installation-on-ubuntu)
  * [Installation on Windows](#installation-on-windows)
  * [Usage](#usage)
     * [Starting the camera node](#start-the-camera-node)
     * [Camera name and namespace](#camera-name-and-camera-namespace)
     * [Parameters](#parameters)
     * [ROS2-vs-Optical Coordination Systems](#ros2robot-vs-opticalcamera-coordination-systems)
     * [TF from coordinate A to coordinate B](#tf-from-coordinate-a-to-coordinate-b)
     * [Extrinsics from sensor A to sensor B](#extrinsics-from-sensor-a-to-sensor-b)
     * [Topics](#published-topics)
     * [RGBD Topic](#rgbd-topic)
     * [Metadata Topic](#metadata-topic)
     * [Post-Processing Filters](#post-processing-filters)
     * [Available Services](#available-services)
     * [Efficient intra-process communication](#efficient-intra-process-communication)
  * [Contributing](CONTRIBUTING.md)
  * [License](LICENSE)

# ROS1 and ROS2 Legacy

    ROS1 Wrapper for Intel® RealSense™ cameras

    ROS1 Wrapper for Intel® RealSense™ cameras is not supported anymore, since our developers team are focusing on ROS2 distro.

    For ROS1 wrapper, go to ros1-legacy branch

     Moving from ros2-legacy to ros2-development

* Changed Parameters:
    - **"stereo_module"**, **"l500_depth_sensor"** are replaced by **"depth_module"**
    - For video streams: **\<module>.profile** replaces **\<stream>_width**, **\<stream>_height**, **\<stream>_fps**
        - **ROS2-legacy (Old)**:
          - ros2 launch realsense2_camera rs_launch.py depth_width:=640 depth_height:=480 depth_fps:=30.0 infra1_width:=640 infra1_height:=480 infra1_fps:=30.0
        - **ROS2-development (New)**:
          - ros2 launch realsense2_camera rs_launch.py depth_module.profile:=640x480x30
    - Removed paramets **\<stream>_frame_id**, **\<stream>_optical_frame_id**. frame_ids are now defined by camera_name
    - **"filters"** is removed. All filters (or post-processing blocks) are enabled/disabled using **"\<filter>.enable"**
    - **"align_depth"** is now a regular processing block and as such the parameter for enabling it is replaced with **"align_depth.enable"**
    - **"allow_no_texture_points"**, **"ordered_pc"** are now belong to the pointcloud filter and as such are replaced by **"pointcloud.allow_no_texture_points"**, **"pointcloud.ordered_pc"**
    - **"pointcloud_texture_stream"**, **"pointcloud_texture_index"** belong now to the pointcloud filter and were renamed to match their librealsense' names: **"pointcloud.stream_filter"**, **"pointcloud.stream_index_filter"**
- Allow enable/disable of sensors in runtime (parameters **\<stream>.enable**)
- Allow enable/disable of filters in runtime (parameters **\<filter_name>.enable**)
- **unite_imu_method** parameter is now changeable in runtime.
- **enable_sync** parameter is now changeable in runtime.

# Installation on Ubuntu

    Step 1: Install the ROS2 distribution

- #### Ubuntu 22.04:
  - [ROS2 Iron](https://docs.ros.org/en/iron/Installation/Ubuntu-Install-Debians.html)
  - [ROS2 Humble](https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html)
  #### Ubuntu 20.04
	- [ROS2 Foxy](https://docs.ros.org/en/foxy/Installation/Ubuntu-Install-Debians.html)

    Step 2: Install latest Intel® RealSense™ SDK 2.0

  **Please choose only one option from the 3 options below (in order to prevent multiple versions installation and workspace conflicts)**

- #### Option 1: Install librealsense2 debian package from Intel servers
  - Jetson users - use the [Jetson Installation Guide](https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md)
  - Otherwise, install from [Linux Debian Installation Guide](https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md#installing-the-packages)
    - In this case treat yourself as a developer: make sure to follow the instructions to also install librealsense2-dev and librealsense2-dkms packages

- #### Option 2: Install librealsense2 (without graphical tools and examples) debian package from ROS servers (Foxy EOL distro is not supported by this option):
  - [Configure](http://wiki.ros.org/Installation/Ubuntu/Sources) your Ubuntu repositories
  - Install all realsense ROS packages by ```sudo apt install ros-<ROS_DISTRO>-librealsense2*```
    - For example, for Humble distro: ```sudo apt install ros-humble-librealsense2*```

- #### Option 3: Build from source
  - Download the latest [Intel® RealSense™ SDK 2.0](https://github.com/IntelRealSense/librealsense/releases/tag/v2.53.1)
  - Follow the instructions under [Linux Installation](https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md)

    Step 3: Install ROS Wrapper for Intel® RealSense™ cameras

#### Option 1: Install debian package from ROS servers (Foxy EOL distro is not supported by this option):
  - [Configure](http://wiki.ros.org/Installation/Ubuntu/Sources) your Ubuntu repositories
  - Install all realsense ROS packages by ```sudo apt install ros-<ROS_DISTRO>-realsense2-*```
  - For example, for Humble distro: ```sudo apt install ros-humble-realsense2-*```

#### Option 2: Install from source

  - Create a ROS2 workspace
      ```bash
      mkdir -p ~/ros2_ws/src
      cd ~/ros2_ws/src/
      ```

  - Clone the latest ROS Wrapper for Intel® RealSense™ cameras from [here](https://github.com/IntelRealSense/realsense-ros.git) into '~/ros2_ws/src/'
      ```bashrc
      git clone https://github.com/IntelRealSense/realsense-ros.git -b ros2-development
      cd ~/ros2_ws
      ```

  - Install dependencies
   ```bash
   sudo apt-get install python3-rosdep -y
   sudo rosdep init # "sudo rosdep init --include-eol-distros" for Foxy and earlier
   rosdep update # "sudo rosdep update --include-eol-distros" for Foxy and earlier
   rosdep install -i --from-path src --rosdistro $ROS_DISTRO --skip-keys=librealsense2 -y
   ```

  - Build
   ```bash
   colcon build
   ```

  -  Source environment
   ```bash
   ROS_DISTRO=<YOUR_SYSTEM_ROS_DISTRO>  # set your ROS_DISTRO: iron, humble, foxy
   source /opt/ros/$ROS_DISTRO/setup.bash
   cd ~/ros2_ws
   . install/local_setup.bash
   ```

# Installation on Windows
  **PLEASE PAY ATTENTION: ROS Wrapper for Intel® RealSense™ cameras is not meant to be supported on Windows by our team, since ROS2 and its packages are still not fully supported over Windows. We added these installation steps below in order to try and make it easier for users who already started working with ROS2 on Windows and want to take advantage of the capabilities of our RealSense cameras**

    Step 1: Install the ROS2 distribution

- #### Windows 10/11

  **Please choose only one option from the two options below (in order to prevent multiple versions installation and workspace conflicts)**

  - Manual install from ROS2 formal documentation:
    - [ROS2 Iron](https://docs.ros.org/en/iron/Installation/Windows-Install-Binary.html)
    - [ROS2 Humble](https://docs.ros.org/en/humble/Installation/Windows-Install-Binary.html)
    - [ROS2 Foxy](https://docs.ros.org/en/foxy/Installation/Windows-Install-Binary.html)
  - Microsoft IOT binary installation:
    - https://ms-iot.github.io/ROSOnWindows/GettingStarted/SetupRos2.html
    - Pay attention that the examples of install are for Foxy distro (which is not supported anymore by ROS Wrapper for Intel® RealSense™ cameras)
	- Please replace the word "Foxy" with Humble or Iron, depends on the chosen distro.

    Step 2: Download RealSense™ ROS2 Wrapper and RealSense™ SDK 2.0 source code from github:

- Download ROS Wrapper for Intel® RealSense™ cameras source code from [ROS Wrapper for Intel® RealSense™ cameras releases](https://github.com/IntelRealSense/realsense-ros/releases)
- Download the corrosponding supported Intel® RealSense™ SDK 2.0 source code from the **"Supported RealSense SDK" section** of the specific release you chose fronm the link above
- Place the librealsense folder inside the realsense-ros folder, to make the librealsense package set beside realsense2_camera, realsense2_camera_msgs and realsense2_description packages

    Step 3: Build

1. Before starting building of our packages, make sure you have OpenCV for Windows installed on your machine. If you choose the Microsoft IOT way to install it, it will be installed automatically. Later, when colcon build, you might need to expose this installation folder by setting CMAKE_PREFIX_PATH, PATH, or OpenCV_DIR environment variables
2. Run "x64 Native Tools Command Prompt for VS 2019" as administrator
3. Setup ROS2 Environment (Do this for every new terminal/cmd you open):
    - If you choose the Microsoft IOT Binary option for installation
      ```
	  > C:\opt\ros\humble\x64\setup.bat
	  ```

    - If you choose the ROS2 formal documentation:
      ```
	  > call C:\dev\ros2_iron\local_setup.bat
	  ```
4.  Change directory to realsense-ros folder
      ```bash
      > cd C:\ros2_ws\realsense-ros
      ```
5. Build librealsense2 package only
      ```bash
      > colcon build --packages-select librealsense2 --cmake-args -DBUILD_EXAMPLES=OFF -DBUILD_WITH_STATIC_CRT=OFF -DBUILD_GRAPHICAL_EXAMPLES=OFF
      ```
	  - User can add `--event-handlers console_direct+` parameter to see more debug outputs of the colcon build
6. Build the other packages
	```bash
	> colcon build --packages-select realsense2_camera_msgs realsense2_description realsense2_camera
	```
	- User can add `--event-handlers console_direct+` parameter to see more debug outputs of the colcon build

7. Setup environment with new installed packages (Do this for every new terminal/cmd you open):
      ```bash
      > call install\setup.bat
      ```

# Usage

## Start the camera node

  #### with ros2 run:
    ros2 run realsense2_camera realsense2_camera_node
    # or, with parameters, for example - temporal and spatial filters are enabled:
    ros2 run realsense2_camera realsense2_camera_node --ros-args -p enable_color:=false -p spatial_filter.enable:=true -p temporal_filter.enable:=true

  #### with ros2 launch:
    ros2 launch realsense2_camera rs_launch.py
    ros2 launch realsense2_camera rs_launch.py depth_module.depth_profile:=1280x720x30 pointcloud.enable:=true

## Camera Name And Camera Namespace

User can set the camera name and camera namespace, to distinguish between cameras and platforms, which helps identifying the right nodes and topics to work with.

### Example
- If user have multiple cameras (might be of the same model) and multiple robots then user can choose to launch/run his nodes on this way.
- For the first robot and first camera he will run/launch it with these parameters:
  - camera_namespace:
    - robot1
  - camera_name
    - D455_1

  - With ros2 launch (via command line or by editing these two parameters in the launch file):

  ```ros2 launch realsense2_camera rs_launch.py camera_namespace:=robot1 camera_name:=D455_1```

  - With ros2 run (using remapping mechanisim [Reference](https://docs.ros.org/en/humble/How-To-Guides/Node-arguments.html)):

  ```ros2 run realsense2_camera realsense2_camera_node --ros-args -r __node:=D455_1 -r __ns:=robot1```

  - Result
  ```
  > ros2 node list
  /robot1/D455_1

  > ros2 topic list
  /robot1/D455_1/color/camera_info
  /robot1/D455_1/color/image_raw
  /robot1/D455_1/color/metadata
  /robot1/D455_1/depth/camera_info
  /robot1/D455_1/depth/image_rect_raw
  /robot1/D455_1/depth/metadata
  /robot1/D455_1/extrinsics/depth_to_color
  /robot1/D455_1/imu

  > ros2 service list
  /robot1/D455_1/device_info
  ```

### Default behavior if non of these parameters are given:
  - camera_namespace:=camera
  - camera_name:=camera

```
> ros2 node list
/camera/camera

> ros2 topic list
/camera/camera/color/camera_info
/camera/camera/color/image_raw
/camera/camera/color/metadata
/camera/camera/depth/camera_info
/camera/camera/depth/image_rect_raw
/camera/camera/depth/metadata
/camera/camera/extrinsics/depth_to_color
/camera/camera/imu

> ros2 service list
/camera/camera/device_info
```

## Parameters

### Available Parameters:
- For the entire list of parameters type `ros2 param list`.
- For reading a parameter value use `ros2 param get <node> <parameter_name>`
  - For example: `ros2 param get /camera/camera depth_module.emitter_enabled`
- For setting a new value for a parameter use `ros2 param set <node> <parameter_name> <value>`
  - For example: `ros2 param set /camera/camera depth_module.emitter_enabled 1`

#### Parameters that can be modified during runtime:
- All of the filters and sensors inner parameters.
- Video Sensor Parameters: (```depth_module``` and ```rgb_camera```)
  - They have, at least, the **<stream_type>_profile** parameter.
    - The profile parameter is a string of the following format: \<width>X\<height>X\<fps> (The dividing character can be X, x or ",". Spaces are ignored.)
    - For example: ```depth_module.depth_profile:=640x480x30 depth_module.infra_profile:=640x480x30 rgb_camera.color_profile:=1280x720x30```
    - Note: The param **depth_module.infra_profile** is common for all infra streams. i.e., infra 0, 1 & 2.
    - If the specified combination of parameters is not available by the device, the default or previously set configuration will be used.
      - Run ```ros2 param describe <your_node_name> <param_name>``` to get the list of supported profiles.
    - Note: Should re-enable the stream for the change to take effect.
  - ***<stream_name>*_format**
    - This parameter is a string used to select the stream format.
    - <stream_name> can be any of *infra, infra1, infra2, color, depth*.
    - For example: ```depth_module.depth_format:=Z16 depth_module.infra1_format:=y8 rgb_camera.color_format:=RGB8```
    - This parameter supports both lower case and upper case letters.
    - If the specified parameter is not available by the stream, the default or previously set configuration will be used.
      - Run ```ros2 param describe <your_node_name> <param_name>``` to get the list of supported formats.
    - Note: Should re-enable the stream for the change to take effect.
  - If the stream doesn't support the user selected profile \<width>X\<height>X\<fps> + \<format>, it will not be opened and a warning message will be shown.
    - Should update the profile settings and re-enable the stream for the change to take effect.
    - Run ```rs-enumerate-devices``` command to know the list of profiles supported by the connected sensors.
- **enable_*<stream_name>***:
  - Choose whether to enable a specified stream or not. Default is true for images and false for orientation streams.
  - <stream_name> can be any of *infra, infra1, infra2, color, depth, gyro, accel*.
  - For example: ```enable_infra1:=true enable_color:=false```
- **enable_sync**:
  - gathers closest frames of different sensors, infra red, color and depth, to be sent with the same timetag.
  - This happens automatically when such filters as pointcloud are enabled.
- ***<stream_type>*_qos**:
  - Sets the QoS by which the topic is published.
  - <stream_type> can be any of *infra, infra1, infra2, color, depth, gyro, accel*.
  -  Available values are the following strings: `SYSTEM_DEFAULT`, `DEFAULT`, `PARAMETER_EVENTS`, `SERVICES_DEFAULT`, `PARAMETERS`, `SENSOR_DATA`.
  - For example: ```depth_qos:=SENSOR_DATA```
  - Pointcloud QoS is controlled with the `pointcloud.pointcloud_qos` parameter in the pointcloud filter, refer to the Post-Processing Filters section for details.
  - Reference: [ROS2 QoS profiles formal documentation](https://docs.ros.org/en/rolling/Concepts/About-Quality-of-Service-Settings.html#qos-profiles)
- **Notice:** ***<stream_type>*_info_qos** refers to both camera_info topics and metadata topics.
- **tf_publish_rate**:
  - double, rate (in Hz) at which dynamic transforms are published
  - Default value is 0.0 Hz *(means no dynamic TF)*
  - This param also depends on **publish_tf** param
    - If **publish_tf:=false**, then no TFs will be published, even if **tf_publish_rate** is >0.0 Hz
    - If **publish_tf:=true** and **tf_publish_rate** set to >0.0 Hz, then dynamic TFs will be published at the specified rate
- **unite_imu_method**:
  - For the D400 cameras with built in IMU components, below 2 unrelated streams (each with it's own frequency) will be created:
    - *gyro* - which shows angular velocity
    - *accel* - which shows linear acceleration.
  - Both streams will publish data to its corresponding topics:
    - '/camera/camera/gyro/sample' & '/camera/camera/accel/sample'
    - Though both topics are of same message type 'sensor_msgs::Imu', only their relevant fields are filled out.
  - A new topic called **imu** will be created, when both *accel* and *gyro* streams are enabled and the param *unite_imu_method* set to > 0.
    - Data from both accel and gyro are combined and published to this topic
    - All the fields of the Imu message are filled out.
    - It will be published at the rate of the gyro.
  - `unite_imu_method` param supports below values:
    - 0 -> **none**: no imu topic
    - 1 -> **copy**: Every gyro message will be attached by the last accel message.
    - 2 -> **linear_interpolation**: Every gyro message will be attached by an accel message which is interpolated to gyro's timestamp.
  - Note: When the param *unite_imu_method* is dynamically updated, re-enable either gyro or accel stream for the change to take effect.
- **accelerate_gpu_with_glsl**:
  - Boolean: GPU accelerated with GLSL for processing PointCloud and Colorizer filters.
  - Note:
    - To have smooth transition between the processing blocks when this parameter is updated dynamically, the node will:
      - Stop the video sensors
      - Do necessary GLSL configuration
      - And then, start the video sensors
    - To enable GPU acceleration, turn ON `BUILD_ACCELERATE_GPU_WITH_GLSL` during build:
    ```bash
    colcon build --cmake-args '-DBUILD_ACCELERATE_GPU_WITH_GLSL=ON'
    ```

#### Parameters that cannot be changed in runtime:
- **serial_no**:
  - will attach to the device with the given serial number (*serial_no*) number.
  - Default, attach to the first (in an inner list) RealSense device.
  - Note: serial number should be defined with "_" prefix.
    - That is a workaround until a better method will be found to ROS2's auto conversion of strings containing only digits into integers.
  - Example: serial number 831612073525 can be set in command line as `serial_no:=_831612073525`.
- **usb_port_id**:
  - will attach to the device with the given USB port (*usb_port_id*).
  - For example: `usb_port_id:=4-1` or `usb_port_id:=4-2`
  - Default, ignore USB port when choosing a device.
- **device_type**:
  - will attach to a device whose name includes the given *device_type* regular expression pattern.
  - Default, ignore device type.
  - For example:
    - `device_type:=d435` will match d435 and d435i.
    - `device_type=d435(?!i)` will match d435 but not d435i.
- **reconnect_timeout**:
  - When the driver cannot connect to the device try to reconnect after this timeout (in seconds).
  - For Example: `reconnect_timeout:=10`
- **wait_for_device_timeout**:
  - If the specified device is not found, will wait *wait_for_device_timeout* seconds before exits.
  - Defualt, *wait_for_device_timeout < 0*, will wait indefinitely.
  - For example: `wait_for_device_timeout:=60`
- **rosbag_filename**:
  - Publish topics from rosbag file. There are two ways for loading rosbag file:
   * Command line - ```ros2 run realsense2_camera realsense2_camera_node -p rosbag_filename:="/full/path/to/rosbag.bag"```
   * Launch file - set ```rosbag_filename``` parameter with rosbag full path (see ```realsense2_camera/launch/rs_launch.py``` as reference)
- **initial_reset**:
  - On occasions the device was not closed properly and due to firmware issues needs to reset.
  - If set to true, the device will reset prior to usage.
  - For example: `initial_reset:=true`
- **base_frame_id**: defines the frame_id all static transformations refers to.
- **clip_distance**:
  - Remove from the depth image all values above a given value (meters). Disable by giving negative value (default)
  - For example: `clip_distance:=1.5`
- **linear_accel_cov**, **angular_velocity_cov**: sets the variance given to the Imu readings.
- **hold_back_imu_for_frames**: Images processing takes time. Therefor there is a time gap between the moment the image arrives at the wrapper and the moment the image is published to the ROS environment. During this time, Imu messages keep on arriving and a situation is created where an image with earlier timestamp is published after Imu message with later timestamp. If that is a problem, setting *hold_back_imu_for_frames* to *true* will hold the Imu messages back while processing the images and then publish them all in a burst, thus keeping the order of publication as the order of arrival. Note that in either case, the timestamp in each message's header reflects the time of it's origin.
- **publish_tf**:
  - boolean, enable/disable publishing static and dynamic TFs
  - Defaults to True
    - So, static TFs will be published by default
    - If dynamic TFs are needed, user should set the param **tf_publish_rate** to >0.0 Hz
  - If set to false, both static and dynamic TFs won't be published, even if the param **tf_publish_rate** is set to >0.0 Hz
- **diagnostics_period**:
  - double, positive values set the period between diagnostics updates on the `/diagnostics` topic.
  - 0 or negative values mean no diagnostics topic is published. Defaults to 0.

The `/diagnostics` topic includes information regarding the device temperatures and actual frequency of the enabled streams.

## ROS2(Robot) vs Optical(Camera) Coordination Systems:

- Point Of View:
  - Imagine we are standing behind of the camera, and looking forward.
  - Always use this point of view when talking about coordinates, left vs right IRs, position of sensor, etc..

![image](https://user-images.githubusercontent.com/99127997/230150735-bc31fedf-d715-4e35-b462-fe2c338832c3.png)

- ROS2 Coordinate System: (X: Forward, Y:Left, Z: Up)
- Camera Optical Coordinate System: (X: Right, Y: Down, Z: Forward)
- References: [REP-0103](https://www.ros.org/reps/rep-0103.html#coordinate-frame-conventions), [REP-0105](https://www.ros.org/reps/rep-0105.html#coordinate-frames)
- All data published in our wrapper topics is optical data taken directly from our camera sensors.
- static and dynamic TF topics publish optical CS and ROS CS to give the user the ability to move from one CS to other CS.

## TF from coordinate A to coordinate B:

- TF msg expresses a transform from coordinate frame "header.frame_id" (source) to the coordinate frame child_frame_id (destination) [Reference](http://docs.ros.org/en/noetic/api/geometry_msgs/html/msg/Transform.html)
- In RealSense cameras, the origin point (0,0,0) is taken from the left IR (infra1) position and named as "camera_link" frame
- Depth, left IR and "camera_link" coordinates converge together.
- Our wrapper provide static TFs between each sensor coordinate to the camera base (camera_link)
- Also, it provides TFs from each sensor ROS coordinates to its corrosponding optical coordinates.
- Example of static TFs of RGB sensor and Infra2 (right infra) sensor of D435i module as it shown in rviz2:
![example](https://user-images.githubusercontent.com/99127997/230148106-0f79cbdb-c401-4d09-b386-a366af18e5f7.png)

## Extrinsics from sensor A to sensor B:

- Extrinsic from sensor A to sensor B means the position and orientation of sensor A relative to sensor B.
- Imagine that B is the origin (0,0,0), then the Extrensics(A->B) describes where is sensor A relative to sensor B.
- For example, depth_to_color, in D435i:
  - If we look from behind of the D435i, extrinsic from depth to color, means, where is the depth in relative to the color.
  - If we just look at the X coordinates, in the optical coordiantes (again, from behind) and assume that COLOR(RGB) sensor is (0,0,0), we can say that DEPTH sensor is on the right of RGB by 0.0148m (1.48cm).

![d435i](https://user-images.githubusercontent.com/99127997/230220297-e392f0fc-63bf-4bab-8001-af1ddf0ed00e.png)

```
administrator@perclnx466 ~/ros2_humble $ ros2 topic echo /camera/camera/extrinsics/depth_to_color
rotation:
- 0.9999583959579468
- 0.008895332925021648
- -0.0020127370953559875
- -0.008895229548215866
- 0.9999604225158691
- 6.045500049367547e-05
- 0.0020131953060626984
- -4.254872692399658e-05
- 0.9999979734420776
translation:
- 0.01485931035131216
- 0.0010161789832636714
- 0.0005317096947692335
---
```

- Extrinsic msg is made up of two parts:
  - float64[9] rotation  (Column - major 3x3 rotation matrix)
  - float64[3] translation  (Three-element translation vector, in meters)

## Published Topics

The published topics differ according to the device and parameters.
After running the above command with D435i attached, the following list of topics will be available (This is a partial list. For full one type `ros2 topic list`):
- /camera/camera/aligned_depth_to_color/camera_info
- /camera/camera/aligned_depth_to_color/image_raw
- /camera/camera/color/camera_info
- /camera/camera/color/image_raw
- /camera/camera/color/metadata
- /camera/camera/depth/camera_info
- /camera/camera/depth/color/points
- /camera/camera/depth/image_rect_raw
- /camera/camera/depth/metadata
- /camera/camera/extrinsics/depth_to_color
- /camera/camera/imu
- /diagnostics
- /parameter_events
- /rosout
- /tf_static

This will stream relevant camera sensors and publish on the appropriate ROS topics.

  Enabling accel and gyro is achieved either by adding the following parameters to the command line:

`ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true enable_gyro:=true enable_accel:=true`

or in runtime using the following commands:
```
ros2 param set /camera/camera enable_accel true
ros2 param set /camera/camera enable_gyro true
```

Enabling stream adds matching topics. For instance, enabling the gyro and accel streams adds the following topics:
- /camera/camera/accel/imu_info
- /camera/camera/accel/metadata
- /camera/camera/accel/sample
- /camera/camera/extrinsics/depth_to_accel
- /camera/camera/extrinsics/depth_to_gyro
- /camera/camera/gyro/imu_info
- /camera/camera/gyro/metadata
- /camera/camera/gyro/sample

## RGBD Topic

RGBD new topic, publishing [RGB + Depth] in the same message (see RGBD.msg for reference). For now, works only with depth aligned to color images, as color and depth images are synchronized by frame time tag.

These boolean paramters should be true to enable rgbd messages:

- `enable_rgbd`: new paramter, to enable/disable rgbd topic, changeable during runtime
- `align_depth.enable`: align depth images to rgb images
- `enable_sync`: let librealsense sync between frames, and get the frameset with color and depth images combined
- `enable_color` + `enable_depth`: enable both color and depth sensors

The current QoS of the topic itself, is the same as Depth and Color streams (SYSTEM_DEFAULT)

Example:
```
ros2 launch realsense2_camera rs_launch.py enable_rgbd:=true enable_sync:=true align_depth.enable:=true enable_color:=true enable_depth:=true
```

## Metadata topic

The metadata messages store the camera's available metadata in a *json* format. To learn more, a dedicated script for echoing a metadata topic in runtime is attached. For instance, use the following command to echo the camera/depth/metadata topic:
```
python3 src/realsense-ros/realsense2_camera/scripts/echo_metadada.py /camera/camera/depth/metadata
```

## Post-Processing Filters

The following post processing filters are available:
 - ```align_depth```: If enabled, will publish the depth image aligned to the color image on the topic `/camera/camera/aligned_depth_to_color/image_raw`.
   - The pointcloud, if created, will be based on the aligned depth image.
 - ```colorizer```: will color the depth image. On the depth topic an RGB image will be published, instead of the 16bit depth values .
 - ```pointcloud```: will add a pointcloud topic `/camera/camera/depth/color/points`.
    * The texture of the pointcloud can be modified using the `pointcloud.stream_filter` parameter.

    * The depth FOV and the texture FOV are not similar. By default, pointcloud is limited to the section of depth containing the texture. You can have a full depth to pointcloud, coloring the regions beyond the texture with zeros, by setting `pointcloud.allow_no_texture_points` to true.
    * pointcloud is of an unordered format by default. This can be changed by setting `pointcloud.ordered_pc` to true.
    * The QoS of the pointcloud topic is independent from depth and color streams and can be controlled with the `pointcloud.pointcloud_qos` parameter.
      - The same set of QoS values are supported as other streams, refer to <stream_type>_qos in the Parameters section of this page.
      - The launch file should include the parameter with initial QoS value, for example,`{'name': 'pointcloud.pointcloud_qos',    'default': 'SENSOR_DATA', 'description': 'pointcloud qos'}`
      - The QoS value can also be overridden at launch with command option, for example, `pointcloud.pointcloud_qos:=SENSOR_DATA`
      - At runtime, the QoS can be changed dynamically but require the filter re-enable for the change to take effect, for example,
        ```bash
        ros2 param set /camera/camera pointcloud.pointcloud_qos SENSOR_DATA
        ros2 param set /camera/camera pointcloud.enable false
        ros2 param set /camera/camera pointcloud.enable true
        ```
 - ```hdr_merge```: Allows depth image to be created by merging the information from 2 consecutive frames, taken with different exposure and gain values.
 - `depth_module.hdr_enabled`: to enable/disable HDR. The way to set exposure and gain values for each sequence:
   -  during Runtime:
      - is by first selecting the sequence id, using the `depth_module.sequence_id` parameter and then modifying the `depth_module.gain`, and `depth_module.exposure`.
      - From FW versions 5.14.x.x and above, if HDR is enabled, the preset configs (like exposure, gain, etc.,) cannot be updated.
        - Disable the HDR first using `depth_module.hdr_enabled` parameter and then, update the required presets.
    - during Launch time of the node:
      - is by setting below parameters
        - `depth_module.exposure.1`
        - `depth_module.gain.1`
        - `depth_module.exposure.2`
        - `depth_module.gain.2`
      - Make sure to set `depth_module.hdr_enabled` to true, otherwise these parameters won't be considered.
    - To view the effect on the infrared image for each sequence id use the `filter_by_sequence_id.sequence_id` parameter.
    - For in-depth review of the subject please read the accompanying [white paper](https://dev.intelrealsense.com/docs/high-dynamic-range-with-stereoscopic-depth-cameras).
    - **Note**: Auto exposure functionality is not supported when HDR is enabled. i.e., Auto exposure will be auto-disabled if HDR is enabled.

  - The following filters have detailed descriptions in : https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md
    - ```disparity_filter``` - convert depth to disparity before applying other filters and back.
    - ```spatial_filter``` - filter the depth image spatially.
    - ```temporal_filter``` - filter the depth image temporally.
    - ```hole_filling_filter``` - apply hole-filling filter.
    - ```decimation_filter``` - reduces depth scene complexity.

Each of the above filters have it's own parameters, following the naming convention of `<filter_name>.<parameter_name>` including a `<filter_name>.enable` parameter to enable/disable it.

## Available services

### device_info:
  - retrieve information about the device - serial_number, firmware_version etc.
  - Type `ros2 interface show realsense2_camera_msgs/srv/DeviceInfo` for the full list.
  - Call example: `ros2 service call /camera/camera/device_info realsense2_camera_msgs/srv/DeviceInfo`

### calib_config_read (for specific camera modules):
  - Read calibration config.
  - Note that reading calibration config is applicable only in Safey Service Mode
  - Type `ros2 interface show realsense2_camera_msgs/srv/CalibConfigRead` for the full request/response fields.
  - Call example: `ros2 service call /camera/camera/calib_config_read realsense2_camera_msgs/srv/CalibConfigRead`

    Click to see the full response of the call example

    `response: realsense2_camera_msgs.srv.CalibConfigRead_Response(success=True, error_message='', calib_config='{"calibration_config":{"calib_roi_num_of_segments":0,"camera_position":{"rotation":[[0,0,0],[0,0,0],[0,0,0]],"translation":[0,0,0]},"crypto_signature":[0, 0 ,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],"roi":[[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]]]}}')`

  - [JSON safety interface config example](realsense2_camera/examples/jsons/calib_config_example.json)

### calib_config_write (for specific camera modules):
  - Write calibration config.
  - Note that writing calibration config is applicable only in Safey Service Mode
  - Type `ros2 interface show realsense2_camera_msgs/srv/CalibConfigWrite` for the full request/response fields.
    - Only for commnad line usage, user should escape all " with \\". Using ros2 services API from rclcpp/rclpy doesn't need escaping. e.g.,:

    Click to see full call example

    `ros2 service call /camera/camera/calib_config_write realsense2_camera_msgs/srv/CalibConfigWrite "{calib_config: '{\"calibration_config\":{\"calib_roi_num_of_segments\":0,\"camera_position\":{\"rotation\":[[0,0,0],[0,0,0],[0,0,0]],\"translation\":[0,0,0]},\"crypto_signature\":[0, 0 ,0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\"roi\":[[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]],[[0,0],[0,0],[0,0],[0,0]]]}}' }"`

  - [JSON safety interface config example](realsense2_camera/examples/jsons/calib_config_example.json)
  - Result example: `realsense2_camera_msgs.srv.CalibConfigWrite_Response(success=True, error_message='')`

## Efficient intra-process communication:

Our ROS2 Wrapper node supports zero-copy communications if loaded in the same process as a subscriber node. This can reduce copy times on image/pointcloud topics, especially with big frame resolutions and high FPS.

You will need to launch a component container and launch our node as a component together with other component nodes. Further details on "Composing multiple nodes in a single process" can be found [here](https://docs.ros.org/en/rolling/Tutorials/Composition.html).

Further details on efficient intra-process communication can be found [here](https://docs.ros.org/en/humble/Tutorials/Intra-Process-Communication.html#efficient-intra-process-communication).

  ### Example
#### Manually loading multiple components into the same process
* Start the component:
  ```bash
  ros2 run rclcpp_components component_container
  ```

* Add the wrapper:
  ```bash
  ros2 component load /ComponentManager realsense2_camera realsense2_camera::RealSenseNodeFactory -e use_intra_process_comms:=true
  ```
  Load other component nodes (consumers of the wrapper topics) in the same way.

### Limitations

* Node components are currently not supported on RCLPY
* Compressed images using `image_transport` will be disabled as this isn't supported with intra-process communication

### Latency test tool and launch file

For getting a sense of the latency reduction, a frame latency reporter tool is available via a launch file.
The launch file loads the wrapper and a frame latency reporter tool component into a single container (so the same process).
The tool prints out the frame latency (`now - frame.timestamp`) per frame.

The tool is not built unless asked for. Turn on `BUILD_TOOLS` during build to have it available:
```bash
colcon build --cmake-args '-DBUILD_TOOLS=ON'
```

The launch file accepts a parameter, `intra_process_comms`, controlling whether zero-copy is turned on or not. Default is on:
```bash
ros2 launch realsense2_camera rs_intra_process_demo_launch.py intra_process_comms:=true
```

[rolling-badge]: https://img.shields.io/badge/-ROLLING-orange?style=flat-square&logo=ros
[rolling]: https://docs.ros.org/en/rolling/index.html
[foxy-badge]: https://img.shields.io/badge/-foxy-orange?style=flat-square&logo=ros
[foxy]: https://docs.ros.org/en/foxy/index.html
[humble-badge]: https://img.shields.io/badge/-HUMBLE-orange?style=flat-square&logo=ros
[humble]: https://docs.ros.org/en/humble/index.html
[iron-badge]: https://img.shields.io/badge/-IRON-orange?style=flat-square&logo=ros
[iron]: https://docs.ros.org/en/iron/index.html
[ubuntu22-badge]: https://img.shields.io/badge/-UBUNTU%2022%2E04-blue?style=flat-square&logo=ubuntu&logoColor=white
[ubuntu22]: https://releases.ubuntu.com/jammy/
[ubuntu20-badge]: https://img.shields.io/badge/-UBUNTU%2020%2E04-blue?style=flat-square&logo=ubuntu&logoColor=white
[ubuntu20]: https://releases.ubuntu.com/focal/