<img src="./docs/_static/logo/biopsykit_logo.png" height="200">

# BioPsyKit

[![PyPI](https://img.shields.io/pypi/v/biopsykit)](https://pypi.org/project/biopsykit/)
[![status](https://joss.theoj.org/papers/4769dbce3a25db943d7e3a23578becd1/status.svg)](https://joss.theoj.org/papers/4769dbce3a25db943d7e3a23578becd1)
![GitHub](https://img.shields.io/github/license/mad-lab-fau/biopsykit)
[![Documentation Status](https://readthedocs.org/projects/biopsykit/badge/?version=latest)](https://biopsykit.readthedocs.io/en/latest/?badge=latest)
[![Test and Lint](https://github.com/mad-lab-fau/BioPsyKit/actions/workflows/test-and-lint.yml/badge.svg)](https://github.com/mad-lab-fau/BioPsyKit/actions/workflows/test-and-lint.yml)
[![codecov](https://codecov.io/gh/mad-lab-fau/BioPsyKit/branch/main/graph/badge.svg?token=IK0QBHQKCO)](https://codecov.io/gh/mad-lab-fau/BioPsyKit)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
![PyPI - Downloads](https://img.shields.io/pypi/dm/biopsykit)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/mad-lab-fau/biopsykit)

A Python package for the analysis of biopsychological data.

With this package you have everything you need for analyzing biopsychological data, including:
* Data processing pipelines for various physiological signals (ECG, EEG, Respiration, Motion, ...).
* Algorithms and data processing pipelines for sleep/wake prediction and computation of sleep endpoints
  based on activity or IMU data.
* Functions to import and process data from sleep trackers (e.g., Withings Sleep Analyzer)
* Functions for processing and analysis of salivary biomarker data (cortisol, amylase).
* Implementation of various psychological and HCI-related questionnaires.
* Implementation of classes representing different psychological protocols
  (e.g., TSST, MIST, Cortisol Awakening Response Assessment, etc.)
* Functions for easily setting up statistical analysis pipelines.
* Functions for setting up and evaluating machine learning pipelines.
* Plotting wrappers optimized for displaying biopsychological data.

## Details

### Analysis of Physiological Signals
#### ECG Processing
`BioPsyKit` provides a whole ECG data processing pipeline, consisting of:
* Loading ECG data from:
    * Generic `.csv` files
    * NilsPod binary (`.bin`) files (requires [`NilsPodLib`](https://github.com/mad-lab-fau/NilsPodLib))
    * Other sensor types (_coming soon_)
* Splitting data into single study parts (based on time intervals) that will be analyzed separately
* Performing ECG processing, including:
    * R peak detection (using [`Neurokit`](https://github.com/neuropsychology/NeuroKit))
    * R peak outlier removal and interpolation
    * HRV feature computation
    * ECG-derived respiration (EDR) estimation for respiration rate and respiratory sinus arrhythmia (RSA)
      (_experimental_)
    * Instantaneous heart rate resampling
    * Computing aggregated results (e.g., mean and standard error) per study part
* Creating plots for visualizing processing results

#### Quick Example
```python
from biopsykit.signals.ecg import EcgProcessor
from biopsykit.example_data import get_ecg_example

ecg_data, sampling_rate = get_ecg_example()

ep = EcgProcessor(ecg_data, sampling_rate)
ep.ecg_process()

print(ep.ecg_result)
```

... more biosignals coming soon!

### Sleep/Wake Prediction
`BioPsyKit` allows to process sleep data collected from IMU or activity sensors (e.g., Actigraphs). This includes:
* Detection of wear periods
* Detection of time spent in bed
* Detection of sleep and wake phases
* Computation of sleep endpoints (e.g., sleep and wake onset, net sleep duration wake after sleep onset, etc.)

#### Quick Example
```python
import biopsykit as bp
from biopsykit.example_data import get_sleep_imu_example

imu_data, sampling_rate = get_sleep_imu_example()

sleep_results = bp.sleep.sleep_processing_pipeline.predict_pipeline_acceleration(imu_data, sampling_rate)
sleep_endpoints = sleep_results["sleep_endpoints"]

print(sleep_endpoints)
```

### Salivary Biomarker Analysis
`BioPsyKit` provides several methods for the analysis of salivary biomarkers (e.g. cortisol and amylase), such as:
* Import data from Excel and csv files into a standardized format
* Compute standard features (maximum increase, slope, area-under-the-curve, mean, standard deviation, ...)

#### Quick Example
```python
import biopsykit as bp
from biopsykit.example_data import get_saliva_example

saliva_data = get_saliva_example(sample_times=[-20, 0, 10, 20, 30, 40, 50])

max_inc = bp.saliva.max_increase(saliva_data)
# remove the first saliva sample (t=-20) from computing the AUC
auc = bp.saliva.auc(saliva_data, remove_s0=True)

print(max_inc)
print(auc)
```

### Questionnaires
`BioPsyKit` implements various established psychological (state and trait) questionnaires, such as:
* Perceived Stress Scale (PSS)
* Positive and Negative Affect Schedule (PANAS)
* Self-Compassion Scale (SCS)
* Big Five Inventory (BFI)
* State Trait Depression and Anxiety Questionnaire (STADI)
* Trier Inventory for Chronic Stress (TICS)
* Primary Appraisal Secondary Appraisal Scale (PASA)
* ...

#### Quick Example
```python
import biopsykit as bp
from biopsykit.example_data import get_questionnaire_example

data = get_questionnaire_example()

pss_data = data.filter(like="PSS")
pss_result = bp.questionnaires.pss(pss_data)

print(pss_result)
```

#### List Supported Questionnaires
```python
import biopsykit as bp

print(bp.questionnaires.utils.get_supported_questionnaires())
```

### Psychological Protocols
`BioPsyKit` implements methods for easy handling and analysis of data recorded with several established psychological
protocols, such as:
* Montreal Imaging Stress Task (MIST)
* Trier Social Stress Test (TSST)
* Cortisol Awakening Response Assessment (CAR)
* ...

#### Quick Example
```python
from biopsykit.protocols import TSST
from biopsykit.example_data import get_saliva_example
from biopsykit.example_data import get_hr_subject_data_dict_example
# specify TSST structure and the durations of the single phases
structure = {
   "Pre": None,
   "TSST": {
       "Preparation": 300,
       "Talk": 300,
       "Math": 300
   },
   "Post": None
}
tsst = TSST(name="TSST", structure=structure)

saliva_data = get_saliva_example(sample_times=[-20, 0, 10, 20, 30, 40, 50])
hr_subject_data_dict = get_hr_subject_data_dict_example()
# add saliva data collected during the whole TSST procedure
tsst.add_saliva_data(saliva_data, saliva_type="cortisol")
# add heart rate data collected during the "TSST" study part
tsst.add_hr_data(hr_subject_data_dict, study_part="TSST")
# compute heart rate results: normalize ECG data relative to "Preparation" phase; afterwards, use data from the
# "Talk" and "Math" phases and compute the average heart rate for each subject and study phase, respectively
tsst.compute_hr_results(
    result_id="hr_mean",
    study_part="TSST",
    normalize_to=True,
    select_phases=True,
    mean_per_subject=True,
    params={
        "normalize_to": "Preparation",
        "select_phases": ["Talk", "Math"]
    }
)
```

### Statistical Analysis
`BioPsyKit` implements methods for simplified statistical analysis of biopsychological data by offering an
object-oriented interface for setting up statistical analysis pipelines, displaying the results, and adding
statistical significance brackets to plots.

#### Quick Example

```python
import matplotlib.pyplot as plt
from biopsykit.stats import StatsPipeline
from biopsykit.plotting import multi_feature_boxplot
from biopsykit.example_data import get_stats_example

data = get_stats_example()

# configure statistical analysis pipeline which consists of checking for normal distribution and performing paired
# t-tests (within-variable: time) on each questionnaire subscale separately (grouping data by subscale).
pipeline = StatsPipeline(
    steps=[("prep", "normality"), ("test", "pairwise_ttests")],
    params={"dv": "PANAS", "groupby": "subscale", "subject": "subject", "within": "time"}
)

# apply statistics pipeline on data
pipeline.apply(data)

# plot data and add statistical significance brackets from statistical analysis pipeline
fig, axs = plt.subplots(ncols=3)
features = ["NegativeAffect", "PositiveAffect", "Total"]
# generate statistical significance brackets
box_pairs, pvalues = pipeline.sig_brackets(
    "test", stats_effect_type="within", plot_type="single", x="time", features=features, subplots=True
)
# plot data
multi_feature_boxplot(
    data=data, x="time", y="PANAS", features=features, group="subscale", order=["pre", "post"],
    stats_kwargs={"box_pairs": box_pairs, "pvalues": pvalues}, ax=axs
)
```

### Machine Learning Analysis
`BioPsyKit` implements methods for simplified and systematic evaluation of different machine learning pipelines.

#### Quick Example

```python
# Utils
from sklearn.datasets import load_breast_cancer
# Preprocessing & Feature Selection
from sklearn.feature_selection import SelectKBest
from sklearn.preprocessing import MinMaxScaler, StandardScaler
# Classification
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
# Cross-Validation
from sklearn.model_selection import KFold

from biopsykit.classification.model_selection import SklearnPipelinePermuter

# load example dataset
breast_cancer = load_breast_cancer()
X = breast_cancer.data
y = breast_cancer.target

# specify estimator combinations
model_dict = {
  "scaler": {
    "StandardScaler": StandardScaler(),
    "MinMaxScaler": MinMaxScaler()
  },
  "reduce_dim": {
    "SelectKBest": SelectKBest(),
  },
  "clf": {
    "KNeighborsClassifier": KNeighborsClassifier(),
    "DecisionTreeClassifier": DecisionTreeClassifier(),
  }
}
# specify hyperparameter for grid search
params_dict = {
  "StandardScaler": None,
  "MinMaxScaler": None,
  "SelectKBest": {"k": [2, 4, "all"]},
  "KNeighborsClassifier": {"n_neighbors": [2, 4], "weights": ["uniform", "distance"]},
  "DecisionTreeClassifier": {"criterion": ['gini', 'entropy'], "max_depth": [2, 4]},
}

pipeline_permuter = SklearnPipelinePermuter(model_dict, params_dict)
pipeline_permuter.fit(X, y, outer_cv=KFold(5), inner_cv=KFold(5))

# print summary of all relevant metrics for the best pipeline for each evaluated pipeline combination
print(pipeline_permuter.metric_summary())
```

## Installation

``BioPsyKit`` requires Python >=3.8. First, install a compatible version of Python. Then install ``BioPsyKit`` via pip.

Installation from [PyPi](https://pypi.org/):
```bash
pip install biopsykit
```

Installation from [PyPi](https://pypi.org/) with extras
(e.g., `jupyter` to directly install all required dependencies for the use with Jupyter Lab):
```bash
pip install "biopsykit[jupyter]"
```

Installation from local repository copy:
```bash
git clone https://github.com/mad-lab-fau/BioPsyKit.git
cd BioPsyKit
pip install .
```

### For Developer

If you are a developer and want to contribute to ``BioPsyKit`` you can install an editable version of the package from
a local copy of the repository.

BioPsyKit uses [poetry](https://python-poetry.org) to manage dependencies and packaging. Once you installed poetry,
run the following commands to clone the repository, initialize a virtual env and install all development dependencies:

#### Without Extras
```bash
git clone https://github.com/mad-lab-fau/BioPsyKit.git
cd BioPsyKit
poetry install
```

#### With all Extras (e.g., extended functionalities for IPython/Jupyter Notebooks)
```bash
git clone https://github.com/mad-lab-fau/BioPsyKit.git
cd BioPsyKit
poetry install -E mne -E jupyter
```

To run any of the tools required for the development workflow, use the `poe` commands of the
[poethepoet](https://github.com/nat-n/poethepoet) task runner:

```bash
$ poe
docs                 Build the html docs using Sphinx.
format               Reformat all files using black.
format_check         Check, but not change, formatting using black.
lint                 Lint all files with Prospector.
test                 Run Pytest with coverage.
update_version       Bump the version in pyproject.toml and biopsykit.__init__ .
register_ipykernel   Register a new IPython kernel named `biopsykit` linked to the virtual environment.
remove_ipykernel     Remove the associated IPython kernel.
```

----
#### Some Notes

* The `poe` commands are only available if you are in the virtual environment associated with this project.
You can either activate the virtual environment manually (e.g., `source .venv/bin/activate`) or use the `poetry shell`
command to spawn a new shell with the virtual environment activated.

* In order to use jupyter notebooks with the project you need to register a new IPython
kernel associated with the venv of the project (`poe register_ipykernel` - see below).
When creating a notebook, make to sure to select this kernel (top right corner of the notebook).

* In order to build the documentation, you need to additionally install [pandoc](https://pandoc.org/installing.html).
----

See the [Contributing Guidelines](https://biopsykit.readthedocs.io/en/latest/source/contributing/CONTRIBUTING.html) for further information.

## Examples
See the [Examples Gallery](https://biopsykit.readthedocs.io/en/latest/examples/index.html) for example on how to use BioPsyKit.

## Citing BioPsyKit

If you use `BioPsyKit` in your work, please report the version you used in the text. Additionally, please also cite the corresponding paper:

```
Richer et al., (2021). BioPsyKit: A Python package for the analysis of biopsychological data. Journal of Open Source Software, 6(66), 3702, https://doi.org/10.21105/joss.03702
```

If you use a specific algorithm please also to make sure you cite the original paper of the algorithm! We recommend the following citation style:

```
We used the algorithm proposed by Author et al. [paper-citation], implemented by the BioPsykit package [biopsykit-citation].
```


==========
jsonschema
==========

|PyPI| |Pythons| |CI| |ReadTheDocs| |Precommit| |Zenodo|

.. |PyPI| image:: https://img.shields.io/pypi/v/jsonschema.svg
   :alt: PyPI version
   :target: https://pypi.org/project/jsonschema/

.. |Pythons| image:: https://img.shields.io/pypi/pyversions/jsonschema.svg
   :alt: Supported Python versions
   :target: https://pypi.org/project/jsonschema/

.. |CI| image:: https://github.com/python-jsonschema/jsonschema/workflows/CI/badge.svg
  :alt: Build status
  :target: https://github.com/python-jsonschema/jsonschema/actions?query=workflow%3ACI

.. |ReadTheDocs| image:: https://readthedocs.org/projects/python-jsonschema/badge/?version=stable&style=flat
   :alt: ReadTheDocs status
   :target: https://python-jsonschema.readthedocs.io/en/stable/

.. |Precommit| image:: https://results.pre-commit.ci/badge/github/python-jsonschema/jsonschema/main.svg
   :alt: pre-commit.ci status
   :target: https://results.pre-commit.ci/latest/github/python-jsonschema/jsonschema/main

.. |Zenodo| image:: https://zenodo.org/badge/3072629.svg
   :alt: Zenodo DOI
   :target: https://zenodo.org/badge/latestdoi/3072629

``jsonschema`` is an implementation of the `JSON Schema <https://json-schema.org>`_ specification for Python.

.. code:: python

    >>> from jsonschema import validate

    >>> # A sample schema, like what we'd get from json.load()
    >>> schema = {
    ...     "type" : "object",
    ...     "properties" : {
    ...         "price" : {"type" : "number"},
    ...         "name" : {"type" : "string"},
    ...     },
    ... }

    >>> # If no exception is raised by validate(), the instance is valid.
    >>> validate(instance={"name" : "Eggs", "price" : 34.99}, schema=schema)

    >>> validate(
    ...     instance={"name" : "Eggs", "price" : "Invalid"}, schema=schema,
    ... )                                   # doctest: +IGNORE_EXCEPTION_DETAIL
    Traceback (most recent call last):
        ...
    ValidationError: 'Invalid' is not of type 'number'

It can also be used from the command line by installing `check-jsonschema <https://github.com/python-jsonschema/check-jsonschema>`_.

Features
--------

* Full support for `Draft 2020-12 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft202012Validator>`_, `Draft 2019-09 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft201909Validator>`_, `Draft 7 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft7Validator>`_, `Draft 6 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft6Validator>`_, `Draft 4 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft4Validator>`_ and `Draft 3 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft3Validator>`_

* `Lazy validation <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/protocols/#jsonschema.protocols.Validator.iter_errors>`_ that can iteratively report *all* validation errors.

* `Programmatic querying <https://python-jsonschema.readthedocs.io/en/latest/errors/>`_ of which properties or items failed validation.

Installation
------------

``jsonschema`` is available on `PyPI <https://pypi.org/project/jsonschema/>`_. You can install using `pip <https://pip.pypa.io/en/stable/>`_:

.. code:: bash

    $ pip install jsonschema

Extras
======

Two extras are available when installing the package, both currently related to ``format`` validation:

    * ``format``
    * ``format-nongpl``

They can be used when installing in order to include additional dependencies, e.g.:

.. code:: bash

    $ pip install jsonschema'[format]'

Be aware that the mere presence of these dependencies – or even the specification of ``format`` checks in a schema – do *not* activate format checks (as per the specification).
Please read the `format validation documentation <https://python-jsonschema.readthedocs.io/en/latest/validate/#validating-formats>`_ for further details.

.. start cut from PyPI

Running the Test Suite
----------------------

If you have ``nox`` installed (perhaps via ``pipx install nox`` or your package manager), running ``nox`` in the directory of your source checkout will run ``jsonschema``'s test suite on all of the versions of Python ``jsonschema`` supports.
If you don't have all of the versions that ``jsonschema`` is tested under, you'll likely want to run using ``nox``'s ``--no-error-on-missing-interpreters`` option.

Of course you're also free to just run the tests on a single version with your favorite test runner.
The tests live in the ``jsonschema.tests`` package.

Benchmarks
----------

``jsonschema``'s benchmarks make use of `pyperf <https://pyperf.readthedocs.io>`_.
Running them can be done via::

      $ nox -s perf

Community
---------

The JSON Schema specification has `a Slack <https://json-schema.slack.com>`_, with an `invite link on its home page <https://json-schema.org/>`_.
Many folks knowledgeable on authoring schemas can be found there.

Otherwise, opening a `GitHub discussion <https://github.com/python-jsonschema/jsonschema/discussions>`_ or asking questions on Stack Overflow are other means of getting help if you're stuck.

.. end cut from PyPI

About
-----

I'm Julian Berman.

``jsonschema`` is on `GitHub <https://github.com/python-jsonschema/jsonschema>`_.

Get in touch, via GitHub or otherwise, if you've got something to contribute, it'd be most welcome!

You can also generally find me on Libera (nick: ``Julian``) in various channels, including ``#python``.

If you feel overwhelmingly grateful, you can also `sponsor me <https://github.com/sponsors/Julian/>`_.

And for companies who appreciate ``jsonschema`` and its continued support and growth, ``jsonschema`` is also now supportable via `TideLift <https://tidelift.com/subscription/pkg/pypi-jsonschema?utm_source=pypi-jsonschema&utm_medium=referral&utm_campaign=readme>`_.


# Open Source Vizier: Reliable and Flexible Black-Box Optimization.
[![PyPI version](https://badge.fury.io/py/google-vizier.svg)](https://badge.fury.io/py/google-vizier)
[![Continuous Integration](https://github.com/google/vizier/actions/workflows/ci.yml/badge.svg)](https://github.com/google/vizier/actions/workflows/ci.yml?query=branch%3Amain)
![Docs](https://github.com/google/vizier/workflows/docs_test/badge.svg)

  [**Google AI Blog**](https://ai.googleblog.com/2023/02/open-source-vizier-towards-reliable-and.html)
| [**Getting Started**](#getting_started)
| [**Documentation**](#documentation)
| [**Installation**](#installation)
| [**Citing and Highlights**](#citing_vizier)

## What is Open Source (OSS) Vizier?
[OSS Vizier](https://arxiv.org/abs/2207.13676) is a Python-based service for black-box optimization and research, based on [Google Vizier](https://dl.acm.org/doi/10.1145/3097983.3098043), one of the first hyperparameter tuning services designed to work at scale.

  OSS Vizier's distributed client-server system. Animation by Tom Small.

## Getting Started
As a basic example for users, below shows how to tune a simple objective using all flat search space types:

```python
from vizier.service import clients
from vizier.service import pyvizier as vz

# Objective function to maximize.
def evaluate(w: float, x: int, y: float, z: str) -> float:
  return w**2 - y**2 + x * ord(z)

# Algorithm, search space, and metrics.
study_config = vz.StudyConfig(algorithm='DEFAULT')
study_config.search_space.root.add_float_param('w', 0.0, 5.0)
study_config.search_space.root.add_int_param('x', -2, 2)
study_config.search_space.root.add_discrete_param('y', [0.3, 7.2])
study_config.search_space.root.add_categorical_param('z', ['a', 'g', 'k'])
study_config.metric_information.append(vz.MetricInformation('metric_name', goal=vz.ObjectiveMetricGoal.MAXIMIZE))

# Setup client and begin optimization. Vizier Service will be implicitly created.
study = clients.Study.from_study_config(study_config, owner='my_name', study_id='example')
for i in range(10):
  suggestions = study.suggest(count=2)
  for suggestion in suggestions:
    params = suggestion.parameters
    objective = evaluate(params['w'], params['x'], params['y'], params['z'])
    suggestion.complete(vz.Measurement({'metric_name': objective}))
```

## Documentation
OSS Vizier's interface consists of [three main APIs](https://oss-vizier.readthedocs.io/en/latest/guides/index.html):

* [**User API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-users) Allows a user to optimize their blackbox objective and optionally setup a server for distributed multi-client settings.
* [**Developer API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-developers) Defines abstractions and utilities for implementing new optimization algorithms for research and to be hosted in the service.
* [**Benchmarking API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-benchmarking) A wide collection of objective functions and methods to benchmark and compare algorithms.

Additionally, it contains [advanced API](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html) for:

* [**Tensorflow Probability:**](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html#tensorflow-probability) For writing Bayesian Optimization algorithms using Tensorflow Probability and Flax.
* [**PyGlove:**](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html#pyglove) For large-scale evolutionary experimentation and program search using OSS Vizier as a distributed backend.

Please see OSS Vizier's [ReadTheDocs documentation](https://oss-vizier.readthedocs.io/) for detailed information.

## Installation
**Quick start:** For tuning objectives using our state-of-the-art JAX-based Bayesian Optimizer, run:

```bash
pip install google-vizier[jax]
```

### Advanced Installation
**Minimal version:** To install only the core service and client APIs from `requirements.txt`, run:

```bash
pip install google-vizier
```

**Full installation:** To support all algorithms and benchmarks, run:

```bash
pip install google-vizier[all]
```

**Specific installation:** If you only need a specific part "X" of OSS Vizier, run:

```bash
pip install google-vizier[X]
```

which installs add-ons from `requirements-X.txt`. Possible options:

* `requirements-jax.txt`: Jax libraries shared by both algorithms and benchmarks.
* `requirements-tf.txt`: Tensorflow libraries used by benchmarks.
* `requirements-algorithms.txt`: Additional repositories (e.g. EvoJAX) for algorithms.
* `requirements-benchmarks.txt`: Additional repositories (e.g. NASBENCH-201) for benchmarks.
* `requirements-test.txt`: Libraries needed for testing code.

Check if all unit tests work by running `run_tests.sh` after a full installation. OSS Vizier requires Python 3.10+, while client-only packages require Python 3.8+.

## Citing and Highlights
**Citing Vizier:** If you found this code useful, please consider citing the [OSS Vizier paper](https://arxiv.org/abs/2207.13676) as well as the [Google Vizier paper](https://dl.acm.org/doi/10.1145/3097983.3098043).

**Highlights:** We track [notable users](https://oss-vizier.readthedocs.io/en/latest/highlights/applications.html) and [media attention](https://oss-vizier.readthedocs.io/en/latest/highlights/media.html) - let us know if OSS Vizier was helpful for your work.

Thanks!

```bibtex
@inproceedings{oss_vizier,
  author    = {Xingyou Song and
               Sagi Perel and
               Chansoo Lee and
               Greg Kochanski and
               Daniel Golovin},
  title     = {Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Black-box Optimization},
  booktitle = {Automated Machine Learning Conference, Systems Track (AutoML-Conf Systems)},
  year      = {2022},
}

@inproceedings{google_vizier,
  author    = {Daniel Golovin and
               Benjamin Solnik and
               Subhodeep Moitra and
               Greg Kochanski and
               John Karro and
               D. Sculley},
  title     = {Google Vizier: {A} Service for Black-Box Optimization},
  booktitle = {Proceedings of the 23rd {ACM} {SIGKDD} International Conference on
               Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13
               - 17, 2017},
  pages     = {1487--1495},
  publisher = {{ACM}},
  year      = {2017},
  url       = {https://doi.org/10.1145/3097983.3098043},
  doi       = {10.1145/3097983.3098043},
}
```


# bancho.py

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/osuAkatsuki/bancho.py/master.svg)](https://results.pre-commit.ci/latest/github/osuAkatsuki/bancho.py/master)
[![Discord](https://discordapp.com/api/guilds/748687781605408908/widget.png?style=shield)](https://discord.gg/ShEQgUx)

**bancho.py** is an in-progress osu! server implementation for developers of all levels
of experience interested in hosting their own osu private server instance(s).

the project is developed primarily by the [Akatsuki](https://akatsuki.gg/) team,
and our aim is to create the most easily maintainable, reliable, and feature-rich
osu! server implementation available.

If you are interested in running or contributing to **bancho.py**, you should head over to the **[bancho.py wiki](https://github.com/osuAkatsuki/bancho.py/wiki)**.

## License

**bancho.py** is licensed under the [MIT License](https://opensource.org/license/mit/). Please see the [LICENSE](https://github.com/osuAkatsuki/bancho.py/blob/master/LICENSE) file for more information.


[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![MIT License][license-shield]][license-url]
[![Coverage Information][coveralls-shield]][coveralls-url]
[![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/8NTPMVPaKy)

Gefyra

    Blazingly-fast, rock-solid, local application development with Kubernetes!

    Explore the docs »

    Try it yourself
    ·
    Report Bug
    ·
    Request Feature

  Table of Contents

      Quick Start

Installation

Your First Bridge

      About The Project

Built With

      Getting Started

Prerequisites

Running Gefyra

Cleaning up

Why "Gefyra"

License

Acknowledgments

## Quick Start
Short manual on where and how to start. You can find detailed information
[here (installation)](https://gefyra.dev/installation/) and [here (usage)](https://gefyra.dev/getting-started/).

### Installation

#### CLI

We offer platform specific installations:

  Linux/MacOS via script/cURL

```shell
curl -sSL https://raw.githubusercontent.com/gefyrahq/gefyra/main/install.sh | sh -
```

  MacOS via Homebrew

```shell
brew tap gefyrahq/gefyra
brew install gefyra
```

  Windows (Manual)

Download the latest binary for Windows under [here](https://github.com/gefyrahq/gefyra/releases/).

#### Docker Desktop Extension

Working with Docker Desktop? We offer an [extension](https://open.docker.com/extensions/marketplace?extensionId=gefyra/docker-desktop-extension) to operate Gefyra through a UI on Docker Desktop.

### Your First Bridge
Make sure Gefyra is installed on your cluster (`gefyra up`). Some details of the installation depend on your Kubernetes' platform.
Check out our [docs](https://gefyra.dev) for more details.

Bridge a local container into an existing cluster. For a detailed guide please
check out this [article](https://gefyra.dev/getting-started/k3d/#running-gefyra).
1. Run a local available image with Gefyra:
```shell
gefyra run -i <image_name> -N <container_name> -n default
```
2. Create a bridge:
```shell
gefyra bridge -N <container_name> -n <k8s_namespace> --target deployment/<k8s_deployment>/<k8s_deployment_container>
```
Explanation for placeholders:
- `container_name` the name of the container you created in the previous step
- `k8s_namespace` the namespace your target workload runs in
- `k8s_deployment` the name of your target deployment
- `k8s_deployment_container` the name of the container within `k8s_deployment`
- `bridge_name` the name for the bridge being created

All available `bridge` flags are listed [here](https://gefyra.dev/reference/cli/#bridge).

## About the project
Gefyra gives Kubernetes-("cloud-native")-developers a completely new way of writing and testing their applications.
Gone are the times of custom `docker-compose` setups, Vagrants, custom scripts or other scenarios in order to develop (micro-)services
for Kubernetes.

Gefyra offers you to:
- run services locally on a developer machine
- operate feature-branches in a production-like Kubernetes environment with all adjacent services
- write code in the IDE you already love, be fast, be confident
- leverage all the neat development features, such as debugger, code-hot-reloading, overriding environment variables
- run high-level integration tests against all dependent services
- keep peace-of-mind when pushing new code to the integration environment

(back to top)

### Built with
Gefyra builds on top of the following popular open-source technologies:

### Docker
[*Docker*](https://docker.io) is currently used in order to manage the local container-based development setup, including the
host, networking and container management procedures.

### Wireguard
[*Wireguard*](https://wireguard.com) is used to establish the connection tunnel between the two ends. It securely encrypts the UDP-based traffic
and allows to create a _site-to-site_ network for Gefyra. That way, the development setup becomes part of the cluster and containers running locally
are actually able to reach cluster-based resources, such as databases, other (micro)services and so on.

### CoreDNS
[*CoreDNS*](https://coredns.io) provides local DNS functionality. It allows resolving resources running within the Kubernetes cluster.

### Nginx
[*Nginx*](https://www.nginx.com/) is used for all kinds of proxying and reverse-proxying traffic, including the interceptions of already running containers
in the cluster.

(back to top)

## Getting Started
You can easily try Gefyra yourself following this small example.

### Prerequisites
1) Follow the [installation](https://gefyra.dev/installation) for your preferred platform.

2) Create a local Kubernetes cluster with `k3d` like so:
**< v5** `k3d cluster create mycluster --agents 1 -p 8080:80@agent[0] -p 31820:31820/UDP@agent[0]`
**>= v5** `k3d cluster create mycluster --agents 1 -p 8080:80@agent:0 -p 31820:31820/UDP@agent:0`
This creates a Kubernetes cluster that binds port 8080 and 31820 to localhost. `kubectl` context is immediately set to this cluster.
3) Apply some workload, for example from the testing directory of this repo:
`kubectl apply -f testing/workloads/hello.yaml`
Check out this workload running under: http://hello.127.0.0.1.nip.io:8080/

### Running Gefyra
4) Set up Gefyra with `gefyra up`
5) Run a local Docker image with Gefyra in order to  make it part of the cluster.
  a) Build your Docker image with a local tag, for example from the testing directory:
   `cd testing/images/ && docker build -f Dockerfile.local . -t pyserver`
  b) Execute Gefyra's run command:
   `gefyra run -i pyserver -N mypyserver -n default`
  c) _Exec_ into the running container and look around. You will find the container to run within your Kubernetes cluster.
   `docker exec -it mypyserver bash`
   `wget -O- hello-nginx` will print out the website of the cluster service _hello-nginx_ from within the cluster.
6) Create a bridge in order to intercept the traffic to the cluster application with the one running locally:
`gefyra bridge -N mypyserver -n default --target deployment/hello-nginxdemo/hello-nginx --port 80:8000`
Check out the locally running server comes up under: http://hello.127.0.0.1.nip.io:8080/
7) List all running _bridges_:
`gefyra list --bridges`
8) _Unbridge_ the local container and reset the cluster to its original state:
`gefyra unbridge -N mypybridge`
Check out the initial response from: http://hello.127.0.0.1.nip.io:8080/

### Cleaning up
9) Remove Gefyra's components from the cluster with `gefyra down`
10) Remove the locally running Kubernetes cluster with `k3d cluster delete mycluster`

(back to top)

## Usage
Checkout [Gefyra's CLI](https://gefyra.dev/docs/cli) or [Guides](https://gefyra.dev/docs/getting-started-with-gefyra).

(back to top)

## Why "Gefyra"
"Gefyra" is the Greek word for "Bridge" and fits nicely with Kubernetes' nautical theme.

(back to top)

## License
Distributed under the Apache License 2.0. See `LICENSE` for more information.

(back to top)

## Reporting Bugs
If you encounter issues, please create a new issue on GitHub or talk to us on the
[Unikube Slack channel](https://unikubeworkspace.slack.com/).
When reporting a bug please include the following information:

Gefyra version or Git commit that you're running (gefyra version),
description of the bug and logs from the relevant `gefyra` command (if applicable),
steps to reproduce the issue, expected behavior.
If you're reporting a security vulnerability, please follow the process for reporting security issues.

## Acknowledgments
Gefyra is based on well-crafted open source software. Special credits go to the teams of
[https://www.linuxserver.io/](linxuserver.io) and [https://git.zx2c4.com/wireguard-go/about/](Wireguard(-go)). Please
be sure to check out their awesome work.
Gefyra was heavily inspired by the free part of Telepresence2.

Doge is excited about that.

(back to top)

[contributors-shield]: https://img.shields.io/github/contributors/gefyrahq/gefyra.svg?style=for-the-badge
[contributors-url]: https://github.com/gefyrahq/gefyra/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/gefyrahq/gefyra.svg?style=for-the-badge
[forks-url]: https://github.com/gefyrahq/gefyra/network/members
[stars-shield]: https://img.shields.io/github/stars/gefyrahq/gefyra.svg?style=for-the-badge
[stars-url]: https://github.com/gefyrahq/gefyra/stargazers
[issues-shield]: https://img.shields.io/github/issues/gefyrahq/gefyra.svg?style=for-the-badge
[issues-url]: https://github.com/gefyrahq/gefyra/issues
[license-shield]: https://img.shields.io/github/license/gefyrahq/gefyra.svg?style=for-the-badge
[license-url]: https://github.com/gefyrahq/gefyra/blob/master/LICENSE.txt
[coveralls-shield]: https://img.shields.io/coveralls/github/gefyrahq/gefyra/main?style=for-the-badge
[coveralls-url]: https://coveralls.io/github/gefyrahq/gefyra


EPSchedule

A schedule app for Eastside Preparatory School, built with [Google App Engine](https://cloud.google.com/appengine) and [Polymer](https://polymer-project.org).

## Running EPSchedule locally

EPSchedule is designed to be *extremely* easy to get running locally - all you need are `git`, Python 3.7+, and `pip`. To set it up, just run:

```
git clone https://github.com/guberti/epschedule.git
cd epschedule
pip install -r requirements.txt
```

You'll then need to get service account credentials to access sensitive user data, like schedules, photos, and crypto keys. Email Justin Uberti (justin@uberti.name) and we'll send you a file - `epschedule-v2-xxxx.json` - which should be renamed to `service_account.json` and put in your `epschedule` folder. Then, just run

```python main.py```

and navigate to http://localhost:8080/, where a local version of EPSchedule will be running!

## Development

### Code style
We try to follow the Black [Python](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html) and Google [JavaScript](https://google.github.io/styleguide/javascriptguide.xml) style guides. To auto-format your Python code, run

```
black .
isort . --profile black
```

from the command line.

### Running unit tests
We use the [pytest](https://docs.pytest.org/en/stable/index.html) framework to test our Python code, and tests are run against all pull requests. To run the tests locally, simply run

```pytest```

from the command line.

### Contributing a patch
From your fork of the epschedule repo, submit a pull request. Be sure that all tests pass, and you have followed the code style guides mentioned above.

### Updating the master schedule
To recrawl or manually adjust the master period schedule (master_schedule.json), follow the instructions at [this link](https://github.com/guberti/epschedule/wiki/How-to-Crawl-schedules).

### More details
For more details, see the EPSchedule [wiki](https://github.com/guberti/epschedule/wiki).


[![Release](https://img.shields.io/github/release/ietf-tools/datatracker.svg?style=flat&maxAge=300)](https://github.com/ietf-tools/datatracker/releases)
[![License](https://img.shields.io/github/license/ietf-tools/datatracker)](https://github.com/ietf-tools/datatracker/blob/main/LICENSE)
[![Code Coverage](https://codecov.io/gh/ietf-tools/datatracker/branch/feat/bs5/graph/badge.svg?token=V4DXB0Q28C)](https://codecov.io/gh/ietf-tools/datatracker)
[![Python Version](https://img.shields.io/badge/python-3.9-blue?logo=python&logoColor=white)](#prerequisites)
[![Django Version](https://img.shields.io/badge/django-4.x-51be95?logo=django&logoColor=white)](#prerequisites)
[![Node Version](https://img.shields.io/badge/node.js-16.x-green?logo=node.js&logoColor=white)](#prerequisites)
[![MariaDB Version](https://img.shields.io/badge/postgres-14-blue?logo=postgresql&logoColor=white)](#prerequisites)

##### The day-to-day front-end to the IETF database for people who work on IETF standards.

- [**Production Website**](https://datatracker.ietf.org)
- [Changelog](https://github.com/ietf-tools/datatracker/releases)
- [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md)
- [Getting Started](#getting-started) - *[ tl;dr ](#the-tldr-to-get-going)*
    - [Creating a Fork](#creating-a-fork)
    - [Git Cloning Tips](#git-cloning-tips)
    - [Docker Dev Environment](docker/README.md)
- [Database & Assets](#database--assets)
- [Old Datatracker Branches](https://github.com/ietf-tools/old-datatracker-branches/branches/all)
- [Frontend Development](#frontend-development)
    - [Intro](#intro)
    - [Vite](#vite-vue-3)
    - [Parcel](#parcel-legacyjquery)
    - [Bootstrap](#bootstrap)
    - [Serving Static Files via CDN](#serving-static-files-via-cdn)
    - [Handling of External Javascript and CSS Components](#handling-of-external-javascript-and-css-components)
    - [Handling of Internal Static Files](#handling-of-internal-static-files)
    - [Changes to Template Files](#changes-to-template-files)
    - [Deployment](#deployment)
- [Running Tests](#running-tests)
    - [Python](#python-tests)
    - [Frontend](#frontend-tests)
    - [Diff Tool](#diff-tool)

---

### Getting Started

This project is following the standard **Git Feature Workflow** development model. Learn about all the various steps of the development workflow, from creating a fork to submitting a pull request, in the [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md) guide.

> Make sure to read the [Styleguides](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md#styleguides) section to ensure a cohesive code format across the project.

You can submit bug reports, enhancement and new feature requests in the [discussions](https://github.com/ietf-tools/datatracker/discussions) area. Accepted tickets will be converted to issues.

#### Creating a Fork

Click the Fork button in the top-right corner of the repository to create a personal copy that you can work on.

> Note that some GitHub Actions might be enabled by default in your fork. You should disable them by going to **Settings** > **Actions** > **General** and selecting **Disable actions** (then Save).

#### Git Cloning Tips

As outlined in the [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md) guide, you will first want to create a fork of the datatracker project in your personal GitHub account before cloning it.

Windows developers: [Start with WSL2 from the beginning](https://github.com/ietf-tools/.github/blob/main/docs/windows-dev.md).

Because of the extensive history of this project, cloning the datatracker project locally can take a long time / disk space. You can speed up the cloning process by limiting the history depth, for example *(replace `USERNAME` with your GitHub username)*:

- To fetch only up to the 10 latest commits:
    ```sh
    git clone --depth=10 https://github.com/USERNAME/datatracker.git
    ```
- To fetch only up to a specific date:
    ```sh
    git clone --shallow-since=DATE https://github.com/USERNAME/datatracker.git
    ```

#### The tl;dr to get going

Note that you will have to have cloned the datatracker code locally - please read the above sections.

Datatracker development is performed using Docker containers. You will need to be able to run docker (and docker-compose) on your machine to effectively develop. It is possible to get a purely native install working, but it is _very complicated_ and typically takes a first time datatracker developer a full day of setup, where the docker setup completes in a small number of minutes.

Many developers are using [VS Code](https://code.visualstudio.com/) and taking advantage of VS Code's ability to start a project in a set of containers. If you are using VS Code, simply start VS Code in your clone and inside VS Code choose `Restart in container`.

If VS Code is not available to you, in your clone, type `cd docker; ./run`

Once the containers are started, run the tests to make sure your checkout is a good place to start from (all tests should pass - if any fail, ask for help at tools-develop@). Inside the app container's shell type:
```sh
ietf/manage.py test --settings=settings_test
```

Note that we recently moved the datatracker onto PostgreSQL - you may still find older documentation that suggests testing with settings_sqlitetest. That will no longer work.

For a more detailed description of getting going, see [docker/README.md](docker/README.md).

#### Overview of the datatracker models

A beginning of a [walkthrough of the datatracker models](https://notes.ietf.org/iab-aid-datatracker-database-overview) was prepared for the IAB AID workshop.

#### Docker Dev Environment

In order to simplify and reduce the time required for setup, a preconfigured docker environment is available.

Read the [Docker Dev Environment](docker/README.md) guide to get started.

### Database & Assets

Nightly database dumps of the datatracker are available as Docker images: `ghcr.io/ietf-tools/datatracker-db:latest`

> Note that to update the database in your dev environment to the latest version, you should run the `docker/cleandb` script.

### Frontend Development

#### Intro

We now use `yarn` to manage assets for the Datatracker, and `vite`/`parcel` to package them. `yarn` maintains its `node` packages under the `.yarn` directory.

The datatracker uses 2 different build systems, depending on the use case:
- [**Vite**](https://vitejs.dev/) for Vue 3 pages / components
- [**Parcel**](https://parceljs.org/) for legacy pages / jQuery

#### Vite *(Vue 3)*

Pages will gradually be updated to Vue 3 components. These components are located under the `/client` directory.

Each Vue 3 app has its own sub-directory. For example, the agenda app is located under `/client/agenda`.

The datatracker makes use of the Django-Vite plugin to point to either the Vite.js server or the precompiled production files. The `DJANGO_VITE_DEV_MODE` flag, found in the `ietf/settings_local.py` file determines whether the Vite.js server is used or not.

In development mode, you must start the Vite.js development server, in addition to the usual Datatracker server:

```sh
yarn dev
```

Any changes made to the files under `/client` will automatically trigger a hot-reload of the modified components.

To generate production assets, run the build command:

```sh
yarn build
```

This will create packages under `ietf/static/dist-neue`, which are then served by the Django development server, and which must be uploaded to the CDN.

#### Parcel *(Legacy/jQuery)*

The Datatracker includes these packages from the various Javascript and CSS files in `ietf/static/js` and `ietf/static/css` respectively, bundled using Parcel.
Static images are likewise in `ietf/static/images`.

Whenever changes are made to the files under `ietf/static`, you must re-run the build command to package them:

``` shell
yarn legacy:build
```

This will create packages under `ietf/static/dist/ietf`, which are then served by the Django development server, and which must be uploaded to the CDN.

#### Bootstrap

The "new" datatracker uses Twitter Bootstrap for the UI.

Get familiar with <https://getbootstrap.com/getting-started/> and use those UI elements, CSS classes, etc. instead of cooking up your own.

Some ground rules:

-   Think hard before tweaking the bootstrap CSS, it will make it harder to upgrade to future releases.
-   No `<style>` tags in the HTML! Put CSS into the "morecss" block of a template instead.
-   CSS that is used by multiple templates goes into static/css/ietf.css or a new CSS file.
-   Javascript that is only used on one template goes into the "js" block of that template.
-   Javascript that is used by multiple templates goes into static/js/ietf.js or a new js file.
-   Avoid CSS, HTML styling or Javascript in the python code!

#### Serving Static Files via CDN

##### Production Mode

If resources served over a CDN and/or with a high max-age don't have different URLs for different versions, then any component upgrade which is accompanied by a change in template functionality will have a long transition time during which the new pages are served with old components, with possible breakage. We want to avoid this.

The intention is that after a release has been checked out, but before it is deployed, the standard django `collectstatic` management command will be run, resulting in all static files being collected from their working directory location and placed in an appropriate location for serving via CDN. This location will have the datatracker release version as part of its URL, so that after the deployment of a new release, the CDN will be forced to fetch the appropriate static files for that release.

An important part of this is to set up the `STATIC_ROOT` and `STATIC_URL` settings appropriately. In 6.4.0, the setting is as follows in production mode:

```
STATIC_URL = "https://www.ietf.org/lib/dt/%s/"%__version__
STATIC_ROOT = CDN_ROOT + "/a/www/www6s/lib/dt/%s/"%__version__
```

The result is that all static files collected via the `collectstatic` command will be placed in a location served via CDN, with the release version being part of the URL.

##### Development Mode

In development mode, `STATIC_URL` is set to `/static/`, and Django's `staticfiles` infrastructure makes the static files available under that local URL root (unless you set `settings.SERVE_CDN_FILES_LOCALLY_IN_DEV_MODE` to `False`). It is not necessary to actually populate the `static/` directory by running `collectstatic` in order for static files to be served when running `ietf/manage.py runserver` -- the `runserver` command has extra support for finding and serving static files without running collectstatic.

In order to work backwards from a file served in development mode to the location from which it is served, the mapping is as follows:

| Development URL | Working copy location |
| --------------- | --------------------- |
| localhost:8000/static/ietf/*  |  ietf/static/ietf/* |
| localhost:8000/static/secr/*  |  ietf/secr/static/secr/*|

#### Handling of External Javascript and CSS Components

In order to make it easy to keep track of and upgrade external components, these are now handled by a tool called `yarn` via the configuration in `package.json`.

To add a new package, simply run (replace `<package-name>` with the NPM module name):
```sh
yarn add <package-name>
```

#### Handling of Internal Static Files

Previous to this release, internal static files were located under `static/`, mixed together with the external components. They are now located under `ietf/static/ietf/` and `ietf/secr/static/secr`, and will be collected for serving via CDN by the `collectstatic` command. Any static files associated with a particular app will be handled the same way (which means that all `admin/` static files automatically will be handled correctly, too).

#### Changes to Template Files

In order to make the template files refer to the correct versioned CDN URL (as given by the STATIC_URL root) all references to static files in the templates have been updated to use the `static` template tag when referring to static files. This will automatically result in both serving static files from the right place in development mode, and referring to the correct versioned URL in production mode and the simpler `/static/` URLs in development mode.

#### Deployment

During deployment, it is now necessary to run the management command:

```sh
ietf/manage.py collectstatic
````
before activating a new release.

## Running Tests

### Python Tests

From a datatracker container, run the command:
```sh
./ietf/manage.py test --settings=settings_test
```

> You can limit the run to specific tests using the `--pattern` argument.

### Frontend Tests

Frontend tests are done via Playwright. There're 2 different type of tests:

- Tests that test Vue pages / components and run natively without any external dependency.
- Tests that require a running datatracker instance to test against (usually legacy views).

> Make sure you have Node.js 16.x or later installed on your machine.

#### Run Vue Tests

> :warning: All commands below **MUST** be run from the `./playwright` directory, unless noted otherwise.

1. Run **once** to install dependencies on your system:
    ```sh
    npm install
    npm run install-deps
    ```

2. Run in a **separate process**, from the **project root directory**:
    ```sh
    yarn preview
    ```

3. Run the tests, in of these 3 modes, from the `./playwright` directory:

    3.1 To run the tests headlessly (command line mode):
    ```sh
    npm test
    ```
    3.2 To run the tests visually **(CANNOT run in docker)**:
    ```sh
    npm run test:visual
    ```

    3.3 To run the tests in debug mode **(CANNOT run in docker)**:
    ```sh
    npm run test:debug
    ```

#### Run Legacy Views Tests

First, you need to start a datatracker instance (dev or prod), ideally from a docker container, exposing the 8000 port.

> :warning: All commands below **MUST** be run from the `./playwright` directory.

1. Run **once** to install dependencies on your system:
```sh
npm install
npm run install-deps
```

2. Run the tests headlessly (command line mode):
```sh
npm run test:legacy
```

### Diff Tool

To compare 2 different datatracker instances and look for diff, read the [diff tool instructions](dev/diff).


# axonius_api_client

[![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://lbesson.mit-license.org/)
[![Made With Python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)
[![PyPi package](https://badge.fury.io/py/axonius-api-client.svg)](https://pypi.org/project/axonius-api-client/)
[![GitHub Repository](https://badge.fury.io/gh/Axonius%2Faxonius_api_client.svg)](https://github.com/Axonius/axonius_api_client)
[![Documentation Status](https://readthedocs.org/projects/axonius-api-client/badge/?version=latest)](https://axonius-api-client.readthedocs.io/en/latest/?badge=latest)
[![Downloads](https://pepy.tech/badge/axonius-api-client)](https://pepy.tech/project/axonius-api-client)

axonius-api-client is an API client for [Axonius](https://axonius.com) that provides:

* A fully modeled python API library that makes using the Axonius REST API easy.
* A powerful command line interface that exposes most of the functionality of the
  underlying python library.

Resources:

* [Documentation](https://axonius-api-client.readthedocs.io/)
* [PyPi package listing](https://pypi.org/project/axonius-api-client/)
* [GitHub repository](https://github.com/Axonius/axonius_api_client)


# flūmine

![Build Status](https://github.com/betcode-org/flumine/actions/workflows/test.yml/badge.svg) [![Coverage Status](https://coveralls.io/repos/github/liampauling/flumine/badge.svg?branch=master)](https://coveralls.io/github/liampauling/flumine?branch=master) [![PyPI version](https://badge.fury.io/py/flumine.svg)](https://pypi.python.org/pypi/flumine) [![Downloads](https://pepy.tech/badge/flumine)](https://pepy.tech/project/flumine)

Betting trading framework with a focus on:

- simplicity
- modular
- pythonic
- rock-solid
- safe

![Backtesting Analysis](docs/images/jupyterloggingcontrol-screenshot.png?raw=true "Jupyter Logging Control Screenshot")

Support for market, order and custom streaming data.

[docs](https://betcode-org.github.io/flumine/)

[join betcode slack group](https://join.slack.com/t/betcode-org/shared_invite/zt-25yz6dt1y-LHya5VzHLOzN3RZEQrSnrA)

Tested on Python 3.8, 3.9, 3.10, 3.11 and 3.12.

## installation

```
$ pip install flumine
```

flumine requires Python 3.8+

## setup

Get started...

```python
import betfairlightweight
from flumine import Flumine, clients

trading = betfairlightweight.APIClient("username")
client = clients.BetfairClient(trading)

framework = Flumine(
    client=client,
)
```

Example strategy:

```python
from flumine import BaseStrategy
from flumine.order.trade import Trade
from flumine.order.order import LimitOrder, OrderStatus
from flumine.markets.market import Market
from betfairlightweight.filters import streaming_market_filter
from betfairlightweight.resources import MarketBook

class ExampleStrategy(BaseStrategy):
    def start(self) -> None:
        print("starting strategy 'ExampleStrategy'")

    def check_market_book(self, market: Market, market_book: MarketBook) -> bool:
        # process_market_book only executed if this returns True
        if market_book.status != "CLOSED":
            return True

    def process_market_book(self, market: Market, market_book: MarketBook) -> None:
        # process marketBook object
        for runner in market_book.runners:
            if runner.status == "ACTIVE" and runner.last_price_traded < 1.5:
                trade = Trade(
                    market_id=market_book.market_id,
                    selection_id=runner.selection_id,
                    handicap=runner.handicap,
                    strategy=self
                )
                order = trade.create_order(
                    side="LAY",
                    order_type=LimitOrder(price=1.01, size=2.00)
                )
                market.place_order(order)

    def process_orders(self, market: Market, orders: list) -> None:
        for order in orders:
            if order.status == OrderStatus.EXECUTABLE:
                if order.size_remaining == 2.00:
                    market.cancel_order(order, 0.02)  # reduce size to 1.98
                if order.order_type.persistence_type == "LAPSE":
                    market.update_order(order, "PERSIST")
                if order.size_remaining > 0:
                    market.replace_order(order, 1.02)  # move

strategy = ExampleStrategy(
    market_filter=streaming_market_filter(
        event_type_ids=["7"],
        country_codes=["GB"],
        market_types=["WIN"],
    )
)

framework.add_strategy(strategy)
```

Run framework:

```python
framework.run()
```

## Features

- Streaming
- Multiple strategies
- Multiple clients
- Order execution
- Paper trading
- Simulation
- Event simulation (multi market)
- Middleware and background workers to enable Scores / RaceCard / InPlayService

## Dependencies

flumine relies on these libraries:

* `betfairlightweight` - Betfair API support
* `tenacity` - Used for connection retrying (streaming)
* `python-json-logger` - JSON logging
* `requests` - HTTP support
* `smart-open` - Efficient streaming of very large files from/to storages such as S3, including (de)compression


![Tests](https://img.shields.io/github/actions/workflow/status/aphp/edsnlp/tests.yml?branch=master&label=tests&style=flat-square)
[![Documentation](https://img.shields.io/github/actions/workflow/status/aphp/edsnlp/documentation.yml?branch=master&label=docs&style=flat-square)](https://aphp.github.io/edsnlp/latest/)
[![PyPI](https://img.shields.io/pypi/v/edsnlp?color=blue&style=flat-square)](https://pypi.org/project/edsnlp/)
[![Demo](https://img.shields.io/badge/demo%20%F0%9F%9A%80-streamit-grean?style=flat-square)](https://aphp.github.io/edsnlp/demo/)
[![Coverage](https://raw.githubusercontent.com/aphp/edsnlp/coverage/coverage.svg)](https://raw.githubusercontent.com/aphp/edsnlp/coverage/coverage.txt)
[![DOI](https://zenodo.org/badge/467585436.svg)](https://zenodo.org/badge/latestdoi/467585436)

EDS-NLP
=======

EDS-NLP is a collaborative NLP framework that aims primarily at extracting information from French clinical notes.
At its core, it is a collection of components or pipes, either rule-based functions or
deep learning modules. These components are organized into a novel efficient and modular pipeline system, built for hybrid and multitask models. We use [spaCy](https://spacy.io) to represent documents and their annotations, and [Pytorch](https://pytorch.org/) as a deep-learning backend for trainable components.

EDS-NLP is versatile and can be used on any textual document. The rule-based components are fully compatible with spaCy's components, and vice versa. This library is a product of collaborative effort, and we encourage further contributions to enhance its capabilities.

Check out our interactive [demo](https://aphp.github.io/edsnlp/demo/) !

## Features

- [Rule-based components](https://aphp.github.io/edsnlp/latest/pipes/) for French clinical notes
- [Trainable components](https://aphp.github.io/edsnlp/latest/pipes/trainable): NER, Span classification
- Support for multitask deep-learning models with [weights sharing](https://aphp.github.io/edsnlp/latest/concepts/torch-component/#sharing-subcomponents)
- [Fast inference](https://aphp.github.io/edsnlp/latest/concepts/inference/), with multi-GPU support out of the box
- Easy to use, with a spaCy-like API
- Compatible with rule-based spaCy components
- Support for various io formats like [BRAT](https://aphp.github.io/edsnlp/latest/data/standoff/), [JSON](https://aphp.github.io/edsnlp/latest/data/json/), [Parquet](https://aphp.github.io/edsnlp/latest/data/parquet/), [Pandas](https://aphp.github.io/edsnlp/latest/data/pandas/) or [Spark](https://aphp.github.io/edsnlp/latest/data/spark/)

## Quick start

### Installation

You can install EDS-NLP via `pip`. We recommend pinning the library version in your projects, or use a strict package manager like [Poetry](https://python-poetry.org/).

```shell
pip install edsnlp==0.12.3
```

or if you want to use the trainable components (using pytorch)

```shell
pip install "edsnlp[ml]==0.12.3"
```

### A first pipeline

Once you've installed the library, let's begin with a very simple example that extracts mentions of COVID19 in a text, and detects whether they are negated.

```python
import edsnlp, edsnlp.pipes as eds

nlp = edsnlp.blank("eds")

terms = dict(
    covid=["covid", "coronavirus"],
)

# Split the documents into sentences, this isneeded for negation detection
nlp.add_pipe(eds.sentences())
# Matcher component
nlp.add_pipe(eds.matcher(terms=terms))
# Negation detection (we also support spacy-like API !)
nlp.add_pipe("eds.negation")

# Process your text in one call !
doc = nlp("Le patient n'est pas atteint de covid")

doc.ents
# Out: (covid,)

doc.ents[0]._.negation
# Out: True
```

## Documentation & Tutorials

Go to the [documentation](https://aphp.github.io/edsnlp) for more information.

## Disclaimer

The performances of an extraction pipeline may depend on the population and documents that are considered.

## Contributing to EDS-NLP

We welcome contributions ! Fork the project and propose a pull request.
Take a look at the [dedicated page](https://aphp.github.io/edsnlp/latest/contributing/) for detail.

## Citation

If you use EDS-NLP, please cite us as below.

```bibtex
@misc{edsnlp,
  author = {Wajsburt, Perceval and Petit-Jean, Thomas and Dura, Basile and Cohen, Ariel and Jean, Charline and Bey, Romain},
  doi    = {10.5281/zenodo.6424993},
  title  = {EDS-NLP: efficient information extraction from French clinical notes},
  url    = {https://aphp.github.io/edsnlp}
}
```

## Acknowledgement

We would like to thank [Assistance Publique – Hôpitaux de Paris](https://www.aphp.fr/), [AP-HP Foundation](https://fondationrechercheaphp.fr/) and [Inria](https://www.inria.fr) for funding this project.


## Links

- Documentation: [warn-scraper.readthedocs.io](https://warn-scraper.readthedocs.io)
- Repository: [github.com/biglocalnews/warn-scraper](https://github.com/biglocalnews/warn-scraper)
- Packaging: [pypi.org/project/warn-scraper](https://pypi.org/project/warn-scraper/)

## About

The project is sponsored by [Big Local News](https://biglocalnews.org/#/about), a program at Stanford University that collects data for impactful journalism. The code is maintained by Stanford Lecturer [Serdar Tumgoren](https://comm.stanford.edu/faculty-tumgoren/) and [Ben Welsh](https://palewi.re/who-is-ben-welsh/), a visiting data journalist from the Los Angeles Times.


# betfairlightweight

![Build Status](https://github.com/betcode-org/betfair/actions/workflows/test.yml/badge.svg) [![Coverage Status](https://coveralls.io/repos/github/liampauling/betfair/badge.svg?branch=master)](https://coveralls.io/github/liampauling/betfair?branch=master) [![PyPI version](https://badge.fury.io/py/betfairlightweight.svg)](https://pypi.python.org/pypi/betfairlightweight) [![Downloads](https://pepy.tech/badge/betfairlightweight)](https://pepy.tech/project/betfairlightweight)

Lightweight, super fast (uses C and Rust libraries) pythonic wrapper for [Betfair API-NG](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni) allowing all betting operations (including market and order streaming) and account operations, see [examples](https://github.com/betcode-org/betfair/tree/master/examples).

[docs](https://betcode-org.github.io/betfair/)

[join betcode slack group](https://join.slack.com/t/betcode-org/shared_invite/zt-25yz6dt1y-LHya5VzHLOzN3RZEQrSnrA)

Currently tested on Python 3.8, 3.9, 3.10, 3.11 and 3.12.

# installation

```bash
$ pip install betfairlightweight
```

To use C/Rust libraries install with

```bash
$ pip install betfairlightweight[speed]
```

# setup

In order to connect to the Betfair API you will need an App Key, SSL Certificates and a username/password.

### App Key
Follow [these](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Application+Keys) instructions to get your app key, you can either use a delayed or live key.

### SSL certificates
Follow [these](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Non-Interactive+%28bot%29+login) instructions to set up your SSL certificates. Save your .ctr and .key files to a local directory. The default directory where the library is looking for the keys is '/certs' but you can specify any other directory.

### Using the library

The library can then be used as follows:

```python
import betfairlightweight

trading = betfairlightweight.APIClient('username', 'password', app_key='app_key', certs='/certs')

trading.login()
```

or the following for interactive login with no certs (not as secure)

```python
import betfairlightweight

trading = betfairlightweight.APIClient('username', 'password', app_key='app_key')

trading.login_interactive()
```

```python
event_types = trading.betting.list_event_types()

[<EventTypeResult>, <EventTypeResult>, ..]
```

Following endpoints are available:

- trading.[login](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Non-Interactive+%28bot%29+login)
- trading.[login_interactive](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Interactive+Login+-+API+Endpoint)
- trading.[keep_alive](https://docs.developer.betfair.com/pages/viewpage.action?pageId=3834909#Login&SessionManagement-KeepAlive)
- trading.[logout](https://docs.developer.betfair.com/pages/viewpage.action?pageId=3834909#Login&SessionManagement-Logout)

- trading.[betting](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Betting+API)
- trading.[account](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Accounts+API)
- trading.[navigation](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Navigation+Data+For+Applications)
- trading.[scores](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Race+Status+API)
- trading.[streaming](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Exchange+Stream+API)
- trading.[historical](https://historicdata.betfair.com/#/apidocs)

- trading.in_play_service
- trading.race_card

# streaming

Currently two listeners available, below will run the base listener which prints anything it receives. Stream listener is able to hold an order stream or a market stream (one per listener). The listener can hold a cache and push market_books/order_books out via a queue.

[Exchange Stream API](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Exchange+Stream+API)

```python
from betfairlightweight.filters import (
    streaming_market_filter,
    streaming_market_data_filter,
)

betfair_socket = trading.streaming.create_stream()

market_filter = streaming_market_filter(
    event_type_ids=['7'],
    country_codes=['IE'],
    market_types=['WIN'],
)
market_data_filter = streaming_market_data_filter(
    fields=['EX_ALL_OFFERS', 'EX_MARKET_DEF'],
    ladder_levels=3
)

betfair_socket.subscribe_to_markets(
    market_filter=market_filter,
    market_data_filter=market_data_filter,
)

betfair_socket.start()  # blocking
```

# historic data

The historic endpoint provides some basic abstraction for the historicdata api:

[Historic Data API](https://historicdata.betfair.com/#/apidocs)

```python

trading.historic.get_my_data()

[{'plan': 'Basic Plan', 'purchaseItemId': 1343, 'sport': 'Cricket', 'forDate': '2017-06-01T00:00:00'}]
```

Taking advantage of the streaming code lightweight can parse/output historical data in the same way it process streaming data allowing backtesting or with a custom listener, csv creation (see [examples](https://github.com/betcode-org/betfair/tree/master/examples)).

[Historic Data](https://historicdata.betfair.com/#/home)

```python

stream = trading.streaming.create_historical_stream(
    file_path='horse-racing-pro-sample',
)

stream.start()
```

or use the  stream generator:

```python

stream = trading.streaming.create_historical_generator_stream(
    file_path='horse-racing-pro-sample',
)

g = stream.get_generator()

for market_books in g():
    print(market_books)
```


# Python Implementation of the KERI Core Libraries

Project Name:  keripy

[![PyPi](https://img.shields.io/pypi/v/keri.svg)](https://pypi.org/project/keri/)
[![GitHub Actions](https://github.com/webOfTrust/keripy/actions/workflows/python-app-ci.yml/badge.svg)](https://github.com/WebOfTrust/keripy/actions)
[![codecov](https://codecov.io/gh/WebOfTrust/keripy/branch/main/graph/badge.svg?token=FR5CB2TPYG)](https://codecov.io/gh/WebOfTrust/keripy)
[![https://pypi.org/project/keri/](https://img.shields.io/pypi/pyversions/keri.svg)](https://pypi.org/project/keri/)
[![Documentation Status](https://readthedocs.org/projects/keripy/badge/?version=latest)](https://keripy.readthedocs.io/en/latest/?badge=latest)

## Installation

### Local installation - build from source
Once all dependencies are installed and working then run:

`$ python3 -m pip install -e ./`

Then you can run

`$ kli version`

to get a version string similar to the following:

`1.0.0`

### Local installation - Docker build
Run `make build-keri` to build your docker image.

Then run `docker run --pull=never -it --entrypoint /bin/bash weboftrust/keri:1.1.10` and you can run `kli version` from within the running container to play with KERIpy.

Make sure the image tag matches the version used in the `Makefile`.
We use `--pull=never` to ensure that docker does not implicitly pull a remote image and relies on the local image tagged during `make build-keri`.

### Dependencies
#### Binaries

python 3.12.1+
libsodium 1.0.18+

#### python packages
lmdb 0.98+
pysodium 0.7.5+
blake3 0.1.5+
msgpack 1.0.0+
simplejson 3.17.0+
cbor2 5.1.0+

```shell
$ pip3 install -U lmdb pysodium blake3 msgpack simplejson cbor2
```

or separately

```shell
$ pip3 install -U lmdb
$ pip3 install -U pysodium
$ pip3 install -U blake3
$ pip3 install -U msgpack
$ pip3 install -U simplejson
$ pip3 install -U cbor2
```

## Development

### Setup
* Ensure Python 3.12.1 is present along with venv and dev header files;
* Setup virtual environment: `python3 -m venv keripy`
* Activate virtual environment: `source keripy/bin/activate`
* Setup dependencies: `pip install -r requirements.txt`

### Testing
* Install pytest: `pip install pytest`

* Run the test suites:

```shell
pytest tests/ --ignore tests/demo/
pytest tests/demo/
```

## Building Documentation in `/docs`
* Install sphinx:
  * `$ pip install sphinx`
  * `$ pip install myst-parser`
* Build with Sphinx in `/docs`:
  * `$ make html`


[![](https://github.com/canonical/microk8s/actions/workflows/build-snap.yml/badge.svg)](https://github.com/canonical/microk8s/actions/workflows/build-snap.yml)
[![](https://snapcraft.io/microk8s/badge.svg)](https://snapcraft.io/microk8s)
![](https://img.shields.io/badge/Kubernetes-1.30-326de6.svg)

## The smallest, fastest Kubernetes

Single-package fully conformant lightweight Kubernetes that works on [42
flavours of Linux](https://snapcraft.io/microk8s). Perfect for:

- Developer workstations
- IoT
- Edge
- CI/CD

 > Canonical might have assembled the easiest way to provision a single node Kubernetes cluster - [Kelsey Hightower](https://twitter.com/kelseyhightower/status/1120834594138406912)

## Why MicroK8s?

- **Small**. Developers want the smallest K8s for laptop and workstation
  development.  MicroK8s provides a standalone K8s compatible with Azure
  AKS, Amazon EKS, Google GKE when you run it on Ubuntu.

- **Simple**. Minimize administration and operations with a single-package
  install that has no moving parts for simplicity and certainty. All
  dependencies and batteries included.

- **Secure**. Updates are available for all security issues and can be
  applied immediately or scheduled to suit your maintenance cycle.

- **Current**. MicroK8s tracks upstream and releases beta, RC and final bits
  the same day as upstream K8s. You can track latest K8s or stick to any
  release version from 1.10 onwards.

- **Comprehensive**. MicroK8s includes a curated collection of manifests for
  common K8s capabilities and services:

  - Service Mesh: Istio, Linkerd
  - Serverless: Knative
  - Monitoring: Fluentd, Prometheus, Grafana, Metrics
  - Ingress, DNS, Dashboard, Clustering
  - Automatic updates to the latest Kubernetes version
  - GPGPU bindings for AI/ML

Drop us a line at [MicroK8s in the Wild](docs/community.md) if you are
doing something fun with MicroK8s!

## Quickstart

Install MicroK8s with:

```
snap install microk8s --classic
```

MicroK8s includes a `microk8s kubectl` command:

```
sudo microk8s kubectl get nodes
sudo microk8s kubectl get services
```

To use MicroK8s with your existing kubectl:

```
sudo microk8s kubectl config view --raw > $HOME/.kube/config
```

#### User access without sudo
The *microk8s* user group is created during the snap installation. Users in that group
are granted access to `microk8s` commands. To add a user to that group:
```
sudo usermod -a -G microk8s <username>
```

#### Kubernetes add-ons

MicroK8s installs a barebones upstream Kubernetes. Additional services like dns and the Kubernetes dashboard can be enabled using the `microk8s enable` command.

```
sudo microk8s enable dns
sudo microk8s enable dashboard
```

Use `microk8s status` to see a list of enabled and available addons. You can find the addon manifests and/or scripts under `${SNAP}/actions/`, with `${SNAP}` pointing by default to `/snap/microk8s/current`.

## Documentation

The [official docs](https://microk8s.io/docs/) are maintained in the
Kubernetes upstream Discourse.

Take a look at the [build instructions](docs/build.md) if you want to
contribute to MicroK8s.


# Flask-SQLAlchemy

Flask-SQLAlchemy is an extension for [Flask][] that adds support for
[SQLAlchemy][] to your application. It aims to simplify using SQLAlchemy
with Flask by providing useful defaults and extra helpers that make it
easier to accomplish common tasks.

[Flask]: https://flask.palletsprojects.com
[SQLAlchemy]: https://www.sqlalchemy.org

## Pallets Community Ecosystem

> [!IMPORTANT]\
> This project is part of the Pallets Community Ecosystem. Pallets is the open
> source organization that maintains Flask; Pallets-Eco enables community
> maintenance of Flask extensions. If you are interested in helping maintain
> this project, please reach out on [the Pallets Discord server][discord].
>
> [discord]: https://discord.gg/pallets

## A Simple Example

```python
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///example.sqlite"

class Base(DeclarativeBase):
  pass

db = SQLAlchemy(app, model_class=Base)

class User(db.Model):
    id: Mapped[int] = mapped_column(primary_key=True)
    username: Mapped[str] = mapped_column(unique=True)

with app.app_context():
    db.create_all()

    db.session.add(User(username="example"))
    db.session.commit()

    users = db.session.execute(db.select(User)).scalars()
```


# TorchRec (Beta Release)
[Docs](https://pytorch.org/torchrec/)

TorchRec is a PyTorch domain library built to provide common sparsity & parallelism primitives needed for large-scale recommender systems (RecSys). It allows authors to train models with large embedding tables sharded across many GPUs.

## TorchRec contains:
- Parallelism primitives that enable easy authoring of large, performant multi-device/multi-node models using hybrid data-parallelism/model-parallelism.
- The TorchRec sharder can shard embedding tables with different sharding strategies including data-parallel, table-wise, row-wise, table-wise-row-wise, column-wise, table-wise-column-wise sharding.
- The TorchRec planner can automatically generate optimized sharding plans for models.
- Pipelined training overlaps dataloading device transfer (copy to GPU), inter-device communications (input_dist), and computation (forward, backward) for increased performance.
- Optimized kernels for RecSys powered by FBGEMM.
- Quantization support for reduced precision training and inference.
- Common modules for RecSys.
- Production-proven model architectures for RecSys.
- RecSys datasets (criteo click logs and movielens)
- Examples of end-to-end training such the dlrm event prediction model trained on criteo click logs dataset.

# Installation

Torchrec requires Python >= 3.8 and CUDA >= 11.8 (CUDA is highly recommended for performance but not required). The example below shows how to install with Python 3.8 and CUDA 12.1. This setup assumes you have conda installed.

## Binaries

Experimental binary on Linux for Python 3.8, 3.9, 3.10, 3.11 and 3.12 (experimental), and CPU, CUDA 11.8 and CUDA 12.1 can be installed via pip wheels from [download.pytorch.org](download.pytorch.org) and PyPI (only for CUDA 12.1).

Below we show installations for CUDA 12.1 as an example. For CPU or CUDA 11.8, swap "cu121" for "cpu" or "cu118".

### Installations
```
Nightly

pip install torch --index-url https://download.pytorch.org/whl/nightly/cu121
pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu121
pip install torchmetrics==1.0.3
pip install torchrec --index-url https://download.pytorch.org/whl/nightly/cu121

Stable via pytorch.org

pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/cu121
pip install torchmetrics==1.0.3
pip install torchrec --index-url https://download.pytorch.org/whl/cu121

Stable via PyPI (only for CUDA 12.1)

pip install torch
pip install fbgemm-gpu
pip install torchrec

```

### Colab example: introduction + install
See our colab notebook for an introduction to torchrec which includes runnable installation.
    - [Tutorial Source](https://github.com/pytorch/torchrec/blob/main/Torchrec_Introduction.ipynb)
    - Open in [Google Colab](https://colab.research.google.com/github/pytorch/torchrec/blob/main/Torchrec_Introduction.ipynb)

## From Source

We are currently iterating on the setup experience. For now, we provide manual instructions on how to build from source. The example below shows how to install with CUDA 12.1. This setup assumes you have conda installed.

1. Install pytorch. See [pytorch documentation](https://pytorch.org/get-started/locally/).
   ```
   CUDA 12.1

   pip install torch --index-url https://download.pytorch.org/whl/nightly/cu121

   CUDA 11.8

   pip install torch --index-url https://download.pytorch.org/whl/nightly/cu118

   CPU

   pip install torch --index-url https://download.pytorch.org/whl/nightly/cpu
   ```

2. Clone TorchRec.
   ```
   git clone --recursive https://github.com/pytorch/torchrec
   cd torchrec
   ```

3. Install FBGEMM.
   ```
   CUDA 12.1

   pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu121

   CUDA 11.8

   pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cu118

   CPU

   pip install fbgemm-gpu --index-url https://download.pytorch.org/whl/nightly/cpu
   ```

4. Install other requirements.
   ```
   pip install -r requirements.txt
   ```

4. Install TorchRec.
   ```
   python setup.py install develop
   ```

5. Test the installation (use torchx-nightly for 3.11; for 3.12, torchx currently doesn't work).
   ```
   GPU mode

   torchx run -s local_cwd dist.ddp -j 1x2 --gpu 2 --script test_installation.py

   CPU Mode

   torchx run -s local_cwd dist.ddp -j 1x2 --script test_installation.py -- --cpu_only
   ```
   See [TorchX](https://pytorch.org/torchx/) for more information on launching distributed and remote jobs.

5. If you want to run a more complex example, please take a look at the torchrec [DLRM example](https://github.com/facebookresearch/dlrm/blob/main/torchrec_dlrm/dlrm_main.py).

## Contributing

### Pyre and linting

Before landing, please make sure that pyre and linting look okay. To run our linters, you will need to
```
pip install pre-commit
```

, and run it.

For Pyre, you will need to
```
cat .pyre_configuration
pip install pyre-check-nightly==<VERSION FROM CONFIG>
pyre check
```

We will also check for these issues in our GitHub actions.

## License
TorchRec is BSD licensed, as found in the [LICENSE](LICENSE) file.


# Qiskit Runtime IBM Client
[![License](https://img.shields.io/github/license/Qiskit/qiskit-ibm-runtime.svg?style=popout-square)](https://opensource.org/licenses/Apache-2.0)
[![CI](https://github.com/Qiskit/qiskit-ibm-runtime/actions/workflows/ci.yml/badge.svg)](https://github.com/Qiskit/qiskit-ibm-runtime/actions/workflows/ci.yml)
[![](https://img.shields.io/github/release/Qiskit/qiskit-ibm-runtime.svg?style=popout-square)](https://github.com/Qiskit/qiskit-ibm-runtime/releases)
[![](https://img.shields.io/pypi/dm/qiskit-ibm-runtime.svg?style=popout-square)](https://pypi.org/project/qiskit-ibm-runtime/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Coverage Status](https://coveralls.io/repos/github/Qiskit/qiskit-ibm-runtime/badge.svg?branch=main)](https://coveralls.io/github/Qiskit/qiskit-ibm-runtime?branch=main)

**Qiskit** is an open-source SDK for working with quantum computers at the level of extended quantum circuits, operators, and primitives.

**Qiskit IBM Runtime** is a new environment offered by IBM Quantum that streamlines quantum computations and provides optimal
implementations of the Qiskit primitives `sampler` and `estimator` for IBM Quantum hardware. It is designed to use additional classical compute resources to execute quantum circuits with more efficiency on quantum processors, by including near-time computations such as error suppression and error mitigation. Examples of error suppression include dynamical decoupling, noise-aware compilation, error mitigation including readout mitigation, zero-noise extrapolation (ZNE), and probabilistic error cancellation (PEC).

Using the runtime service, a research team at IBM Quantum was able to achieve a 120x speedup
in their lithium hydride simulation. For more information, see the
[IBM Research blog](https://research.ibm.com/blog/120x-quantum-speedup).

This module provides the interface to access the Qiskit Runtime service on IBM Quantum Platform or IBM Cloud.

## Installation

You can install this package using pip:

```bash
pip install qiskit-ibm-runtime
```

## Account setup

### Qiskit Runtime service on IBM Quantum Platform

You will need your IBM Quantum API token to authenticate with the runtime service:

1. Create an IBM Quantum account or log in to your existing account by visiting the [IBM Quantum login page].

1. Copy (and optionally regenerate) your API token from your
   [IBM Quantum account page].

### Qiskit Runtime service on IBM Cloud

The runtime service is now part of the IBM Quantum Services on IBM Cloud. To use this service, you'll
need to create an IBM Cloud account and a quantum service instance.
[This guide](https://cloud.ibm.com/docs/quantum-computing?topic=quantum-computing-get-started)
contains step-by-step instructions, including how to find your
IBM Cloud API key and Cloud Resource Name (CRN), which you will need for authentication.

### Save your account on disk

Once you have the account credentials, you can save them on disk, so you won't have to input
them each time. The credentials are saved in the `$HOME/.qiskit/qiskit-ibm.json` file, where `$HOME` is your home directory.

| :warning: Account credentials are saved in plain text, so only do so if you are using a trusted device. |
|:---------------------------|

 ```python
from qiskit_ibm_runtime import QiskitRuntimeService

# Save an IBM Cloud account.
QiskitRuntimeService.save_account(channel="ibm_cloud", token="MY_IBM_CLOUD_API_KEY", instance="MY_IBM_CLOUD_CRN")

# Save an IBM Quantum account.
QiskitRuntimeService.save_account(channel="ibm_quantum", token="MY_IBM_QUANTUM_TOKEN")
```

Once the account is saved on disk, you can instantiate the service without any arguments:

```python
from qiskit_ibm_runtime import QiskitRuntimeService
service = QiskitRuntimeService()
```

### Loading account from environment variables

Alternatively, the service can discover credentials from environment variables:
```bash
export QISKIT_IBM_TOKEN="MY_IBM_CLOUD_API_KEY"
export QISKIT_IBM_INSTANCE="MY_IBM_CLOUD_CRN"
export QISKIT_IBM_CHANNEL="ibm_cloud"
```

Then instantiate the service without any arguments:
```python
from qiskit_ibm_runtime import QiskitRuntimeService
service = QiskitRuntimeService()
```

### Enabling account for current Python session

As another alternative, you can also enable an account just for the current session by instantiating the
service with your credentials.

```python
from qiskit_ibm_runtime import QiskitRuntimeService

# For an IBM Cloud account.
ibm_cloud_service = QiskitRuntimeService(channel="ibm_cloud", token="MY_IBM_CLOUD_API_KEY", instance="MY_IBM_CLOUD_CRN")

# For an IBM Quantum account.
ibm_quantum_service = QiskitRuntimeService(channel="ibm_quantum", token="MY_IBM_QUANTUM_TOKEN")
```

## Primitives

All quantum applications and algorithms level are fundamentally built using these steps:
1. Map classical inputs to a quantum problem
2. Translate problem for optimized quantum execution.
3. Execute the quantum circuits by using a primitive (Estimator or Sampler).
4. Post-process, return result in classical format.

**Primitives** are base-level functions that serve as building blocks for many quantum algorithms and applications.
Primitives accept vectorized inputs, where single circuits can be grouped with array-valued specifications. That is, one circuit can be executed for arrays of n parameter sets, n observables, or both (in the case of the estimator). Each group is called a Primitive Unified Bloc (PUB), and can be represented as a tuple.

The [primitive interfaces](https://docs.quantum.ibm.com/api/qiskit/primitives) are defined in Qiskit.

The IBM Runtime service offers these primitives with additional features, such as built-in error suppression and mitigation.

There are several different options you can specify when calling the primitives. See [Primitive options](https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/options) for more information.

### Primitive versions

Version 2 of the primitives is introduced by `qiskit-ibm-runtime` release 0.21.0. If you are using V1 primitives, refer to [Migrate to the V2 primitives](https://docs.quantum.ibm.com/api/migration-guides/v2-primitives) on how to migratie to V2 primitives. The examples below all use V2 primitives.

### Sampler

This primitive takes a list of user circuits (including measurements) as input and returns the sampling output. The type of the output is defined by the program (typically bit-arrays), and the output data is separated by the classical register names.

To invoke the `Sampler` primitive

```python
from qiskit import QuantumCircuit
from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager
from qiskit_ibm_runtime import QiskitRuntimeService, SamplerV2 as Sampler

service = QiskitRuntimeService()

# 1. A quantum circuit for preparing the quantum state (|00> + |11>)/rt{2}
bell = QuantumCircuit(2)
bell.h(0)
bell.cx(0, 1)
bell.measure_all()

# 2: Optimize problem for quantum execution.
backend = service.least_busy(operational=True, simulator=False)
pm = generate_preset_pass_manager(backend=backend, optimization_level=1)
isa_circuit = pm.run(bell)

# 3. Execute using the Sampler primitive
sampler = Sampler(backend=backend)
sampler.options.default_shots = 1024  # Options can be set using auto-complete.
job = sampler.run([isa_circuit])
print(f"Job ID is {job.job_id()}")
pub_result = job.result()[0]
print(f"Counts for the meas output register: {pub_result.data.meas.get_counts()}")
```

### Estimator

This primitive takes circuits and observables as input, to evaluate expectation values and standard error for a given parameter input. This Estimator allows users to efficiently calculate and interpret expectation values of quantum operators required for many algorithms.

To invoke the `Estimator` primitive:

```python
from qiskit_ibm_runtime import QiskitRuntimeService, EstimatorV2 as Estimator
from qiskit.quantum_info import SparsePauliOp
from qiskit import QuantumCircuit
from qiskit.circuit import Parameter
import numpy as np

service = QiskitRuntimeService()

# 1. A quantum circuit for preparing the quantum state (|000> + e^{itheta} |111>)/rt{2}
theta = Parameter('θ')
circuit = QuantumCircuit(3)
circuit.h(0) # generate superposition
circuit.p(theta, 0) # add quantum phase
circuit.cx(0, 1) # condition 1st qubit on 0th qubit
circuit.cx(0, 2) # condition 2nd qubit on 0th qubit

# The observable to be measured
M1 = SparsePauliOp.from_list([("XXY", 1), ("XYX", 1), ("YXX", 1), ("YYY", -1)])

# batch of theta parameters to be executed
points = 50
theta1 = []
for x in range(points):
    theta = [x*2.0*np.pi/50]
    theta1.append(theta)

# 2: Optimize problem for quantum execution.
backend = service.least_busy(operational=True, simulator=False)
pm = generate_preset_pass_manager(backend=backend, optimization_level=1)
isa_circuit = pm.run(circuit)
isa_observables = M1.apply_layout(isa_circuit.layout)

# 3. Execute using the Estimator primitive
estimator = Estimator(backend)
estimator.options.resilience_level = 1  # Options can be set using auto-complete.
job = estimator.run([(isa_circuit, isa_observables, theta1)])
print(f"Job ID is {job.job_id()}")
pub_result = job.result()[0]
print(f"Expectation values: {pub_result.data.evs}")
```

This code batches together 50 parameters to be executed in a single job. If a user wanted to find the `theta` that optimized the observable, they could plot and observe it occurs at `theta=np.pi/2`. For speed we recommend batching results together (note that depending on your access, there may be limits on the number of circuits, objects, and parameters that you can send).

## Session

In many algorithms and applications, an Estimator needs to be called iteratively without incurring queuing delays on each iteration. To solve this, the IBM Runtime service provides a **Session**. A session starts when the first job within the session is started, and subsequent jobs within the session are prioritized by the scheduler.

You can use the [`qiskit_ibm_runtime.Session`](https://github.com/Qiskit/qiskit-ibm-runtime/blob/main/qiskit_ibm_runtime/session.py) class to start a
session. Consider the same example above and try to find the optimal `theta`. The following example uses the [golden search method](https://en.wikipedia.org/wiki/Golden-section_search) to iteratively find the optimal theta that maximizes the observable.

To invoke the `Estimator` primitive within a session:

```python
from qiskit_ibm_runtime import QiskitRuntimeService, Session, EstimatorV2 as Estimator
from qiskit.quantum_info import SparsePauliOp
from qiskit import QuantumCircuit
from qiskit.circuit import Parameter
import numpy as np

service = QiskitRuntimeService()

# 1. A quantum circuit for preparing the quantum state (|000> + e^{itheta} |111>)/rt{2}
theta = Parameter('θ')
circuit = QuantumCircuit(3)
circuit.h(0) # generate superpostion
circuit.p(theta,0) # add quantum phase
circuit.cx(0, 1) # condition 1st qubit on 0th qubit
circuit.cx(0, 2) # condition 2nd qubit on 0th qubit

# The observable to be measured
M1 = SparsePauliOp.from_list([("XXY", 1), ("XYX", 1), ("YXX", 1), ("YYY", -1)])

gr = (np.sqrt(5) + 1) / 2 # golden ratio
thetaa = 0 # lower range of theta
thetab = 2*np.pi # upper range of theta
tol = 1e-1 # tol

# 2: Optimize problem for quantum execution.
backend = service.least_busy(operational=True, simulator=False)
pm = generate_preset_pass_manager(backend=backend, optimization_level=1)
isa_circuit = pm.run(circuit)
isa_observables = M1.apply_layout(isa_circuit.layout)

# 3. Execute iteratively using the Estimator primitive
with Session(service=service, backend=backend) as session:
    estimator = Estimator(session=session)
    estimator.options.default_precision = 0.03  # Options can be set using auto-complete.
    #next test range
    thetac = thetab - (thetab - thetaa) / gr
    thetad = thetaa + (thetab - thetaa) / gr
    while abs(thetab - thetaa) > tol:
        print(f"max value of M1 is in the range theta = {[thetaa, thetab]}")
        job = estimator.run([(isa_circuit, isa_observables, [[thetac],[thetad]])])
        test = job.result()[0].data.evs
        if test[0] > test[1]:
            thetab = thetad
        else:
            thetaa = thetac
        thetac = thetab - (thetab - thetaa) / gr
        thetad = thetaa + (thetab - thetaa) / gr

    # Final job to evaluate Estimator at midpoint found using golden search method
    theta_mid = (thetab + thetaa) / 2
    job = estimator.run([(isa_circuit, isa_observables, theta_mid)])
    print(f"Session ID is {session.session_id}")
    print(f"Final Job ID is {job.job_id()}")
    print(f"Job result is {job.result()[0].data.evs} at theta = {theta_mid}")
```

This code returns `Job result is [4.] at theta = 1.575674623307102` using only nine iterations. This is a very powerful extension to the primitives. However, using too much code between iterative calls can lock the QPU and use excessive QPU time, which is expensive. We recommend only using sessions when needed. The Sampler can also be used within a session, but there are not any well-defined examples for this.

## Instances

Access to IBM Quantum Platform channel is controlled by the instances (previously called providers) to which you are assigned. An instance is defined by a hierarchical organization of hub, group, and project. A hub is the top level of a given hierarchy (organization) and contains within it one or more groups. These groups are in turn populated with projects. The combination of hub/group/project is called an instance. Users can belong to more than one instance at any time.

> **_NOTE:_** IBM Cloud instances are different from IBM Quantum Platform instances.  IBM Cloud does not use the hub/group/project structure for user management. To view and create IBM Cloud instances, visit the [IBM Cloud Quantum Instances page](https://cloud.ibm.com/quantum/instances).

To view a list of your instances, visit your [account settings page](https://www.quantum-computing.ibm.com/account) or use the `instances()` method.

You can specify an instance when initializing the service or provider, or when picking a backend:

```python
# Optional: List all the instances you can access.
service = QiskitRuntimeService(channel='ibm_quantum')
print(service.instances())

# Optional: Specify the instance at service level. This becomes the default unless overwritten.
service = QiskitRuntimeService(channel='ibm_quantum', instance="hub1/group1/project1")
backend1 = service.backend("ibmq_manila")

# Optional: Specify the instance at the backend level, which overwrites the service-level specification when this backend is used.
backend2 = service.backend("ibmq_manila", instance="hub2/group2/project2")

sampler1 = Sampler(backend=backend1)    # this will use hub1/group1/project1
sampler2 = Sampler(backend=backend2)    # this will use hub2/group2/project2
```

If you do not specify an instance, then the code will select one in the following order:

1. If your account only has access to one instance, it is selected by default.
2. If your account has access to multiple instances, but only one can access the requested backend, the instance with access is selected.
3. In all other cases, the code selects the first instance other than ibm-q/open/main that has access to the backend.

## Access your IBM Quantum backends

A **backend** is a quantum device or simulator capable of running quantum circuits or pulse schedules.

You can query for the backends you have access to. Attributes and methods of the returned instances
provide information, such as qubit counts, error rates, and statuses, of the backends.

```python
from qiskit_ibm_runtime import QiskitRuntimeService
service = QiskitRuntimeService()

# Display all backends you have access.
print(service.backends())

# Get a specific backend.
backend = service.backend('ibm_brisbane')

# Print backend coupling map.
print(backend.coupling_map)
```

## Next Steps

Now you're set up and ready to check out some of the [tutorials].

## Contribution guidelines

If you'd like to contribute to qiskit-ibm-runtime, please take a look at our
[contribution guidelines]. This project adheres to Qiskit's [code of conduct].
By participating, you are expected to uphold to this code.

We use [GitHub issues] for tracking requests and bugs. Please use our [slack]
for discussion and simple questions. To join our Slack community use the
invite link at [ibm.com/quantum/qiskit]. For questions that are more suited for a forum we
use the `Qiskit` tag in [Stack Exchange].

## Authors and Citation

Qiskit Runtime IBM Client is the work of [many people](https://github.com/Qiskit/qiskit-ibm-runtime/graphs/contributors) who contribute to the project at different levels.
If you use Qiskit, please cite as per the included [BibTeX file](https://github.com/Qiskit/qiskit/blob/main/CITATION.bib).

## License

[Apache License 2.0].

[IBM Quantum]: https://www.ibm.com/quantum-computing/
[IBM Quantum login page]:  https://quantum-computing.ibm.com/login
[IBM Quantum account page]: https://quantum-computing.ibm.com/account
[contribution guidelines]: https://github.com/Qiskit/qiskit-ibm-runtime/blob/main/CONTRIBUTING.md
[code of conduct]: https://github.com/Qiskit/qiskit-ibm-runtime/blob/main/CODE_OF_CONDUCT.md
[GitHub issues]: https://github.com/Qiskit/qiskit-ibm-runtime/issues
[slack]: https://qiskit.slack.com
[ibm.com/quantum/qiskit]: https://www.ibm.com/quantum/qiskit
[Stack Exchange]: https://quantumcomputing.stackexchange.com/questions/tagged/qiskit
[many people]: https://github.com/Qiskit/qiskit-ibm-runtime/graphs/contributors
[BibTeX file]: https://github.com/Qiskit/qiskit/blob/master/Qiskit.bib
[Apache License 2.0]: https://github.com/Qiskit/qiskit-ibm-runtime/blob/main/LICENSE.txt
[tutorials]: https://learning.quantum.ibm.com/catalog/tutorials