# Lingvo

[![PyPI](https://badge.fury.io/py/lingvo.svg)](https://badge.fury.io/py/lingvo)
[![Python](https://img.shields.io/pypi/pyversions/lingvo)](https://badge.fury.io/py/tensorflow)

[![Documentation](https://img.shields.io/badge/api-reference-blue.svg)](https://tensorflow.github.io/lingvo)

[![License](https://img.shields.io/github/license/tensorflow/lingvo)](LICENSE)

## What is it?

Lingvo is a framework for building neural networks in Tensorflow, particularly
sequence models.

A list of publications using Lingvo can be found [here](PUBLICATIONS.md).

## Table of Contents

*   [Releases](#releases)
    *   [Major breaking changes](#major-breaking-changes)
*   [Quick start](#quick-start)
    *   [Installation](#installation)
    *   [Running the MNIST image model](#running-the-mnist-image-model)
    *   [Running the machine translation model](#running-the-machine-translation-model)
    *   [Running the GShard transformer based giant language model](#running-the-gshard-transformer-based-giant-language-model)
    *   [Running the 3d object detection model](#running-the-3d-object-detection-model)
*   [Models](#models)
    *   [Automatic Speech Recognition](#automatic-speech-recognition)
    *   [Car](#car)
    *   [Image](#image)
    *   [Language Modelling](#language-modelling)
    *   [Machine Translation](#machine-translation)
*   [References](#references)
*   [License](#license)

## Releases

PyPI Version | Commit
------------ | ----------------------------------------
0.12.4       | --
0.11.0       | 6fae10077756f54beacd5c454959f20b33fd65e2
0.10.0       | 075fd1d88fa6f92681f58a2383264337d0e737ee
0.9.1        | c1124c5aa7af13d2dd2b6d43293c8ca6d022b008
0.9.0        | f826e99803d1b51dccbbbed1ef857ba48a2bbefe

Older releases

PyPI Version | Commit
------------ | ----------------------------------------
0.8.2        | 93e123c6788e934e6b7b1fd85770371becf1e92e
0.7.2        | b05642fe386ee79e0d88aa083565c9a93428519e

Details for older releases are unavailable.

### Major breaking changes

**NOTE: this is not a comprehensive list. Lingvo releases do not offer any
guarantees regarding backwards compatibility.**

#### HEAD

Nothing here.

#### 0.12.0

*   **General**
    *   Tensorflow 2.9 is now required.
    *   Python 3.7 support has been removed.
    *   Compatible with (up to) Tensorflow 2.10 and Python 3.10

#### 0.11.0

*   **General**
    *   Tensorflow 2.7 is now the required version.
    *   Python 3.6 support has been removed.

#### 0.10.0

*   **General**
    *   Tensorflow 2.6 is now the required version.
    *   The theta_fn arg to CreateVariable() has been removed.

#### 0.9.1

*   **General**
    *   Python 3.9 is now supported.
    *   ops.beam_search_step now takes and returns an additional arg
        `beam_done`.
    *   The namedtuple beam_search_helper.BeamSearchDecodeOutput now removes the
        field `done_hyps`.

#### 0.9.0

*   **General**
    *   Tensorflow 2.5 is now the required version.
    *   Python 3.5 support has been removed.
    *   py_utils.AddGlobalVN and py_utils.AddPerStepVN have been combined into
        py_utils.AddVN.
    *   BaseSchedule().Value() no longer takes a step arg.
    *   Classes deriving from BaseSchedule should implement Value() not FProp().
    *   theta.global_step has been removed in favor of py_utils.GetGlobalStep().
    *   py_utils.GenerateStepSeedPair() no longer takes a global_step arg.
    *   PostTrainingStepUpdate() no longer takes a global_step arg.
    *   The fatal_errors argument to custom input ops now takes error message
        substrings rather than integer error codes.

Older releases

#### 0.8.2

*   **General**
    *   NestedMap Flatten/Pack/Transform/Filter etc now expand descendent dicts
        as well.
    *   Subclasses of BaseLayer extending from `abc.ABCMeta` should now extend
        `base_layer.ABCLayerMeta` instead.
    *   Trying to call self.CreateChild outside of `__init__` now raises an
        error.
    *   `base_layer.initializer` has been removed. Subclasses no longer need to
        decorate their `__init__` function.
    *   Trying to call self.CreateVariable outside of `__init__` or
        `_CreateLayerVariables` now raises an error.
    *   It is no longer possible to access self.vars or self.theta inside of
        `__init__`. Refactor by moving the variable creation and access to
        `_CreateLayerVariables`. The variable scope is set automatically
        according to the layer name in `_CreateLayerVariables`.

Details for older releases are unavailable.

## Quick start

### Installation

There are two ways to set up Lingvo: installing a fixed version through pip, or
cloning the repository and building it with bazel. Docker configurations are
provided for each case.

If you would just like to use the framework as-is, it is easiest to just install
it through pip. This makes it possible to develop and train custom models using
a frozen version of the Lingvo framework. However, it is difficult to modify the
framework code or implement new custom ops.

If you would like to develop the framework further and potentially contribute
pull requests, you should avoid using pip and clone the repository instead.

**pip:**

The [Lingvo pip package](https://pypi.org/project/lingvo) can be installed with
`pip3 install lingvo`.

See the
[codelab](https://colab.research.google.com/github/tensorflow/lingvo/blob/master/codelabs/introduction.ipynb)
for how to get started with the pip package.

**From sources:**

The prerequisites are:

*   a TensorFlow 2.7 [installation](https://www.tensorflow.org/install/),
*   a `C++` compiler (only g++ 7.3 is officially supported), and
*   the bazel build system.

Refer to [docker/dev.Dockerfile](docker/dev.Dockerfile) for a set of working
requirements.

`git clone` the repository, then use bazel to build and run targets directly.
The `python -m module` commands in the codelab need to be mapped onto `bazel
run` commands.

**docker:**

Docker configurations are available for both situations. Instructions can be
found in the comments on the top of each file.

*   [lib.dockerfile](docker/lib.dockerfile) has the Lingvo pip package
    preinstalled.
*   [dev.Dockerfile](docker/dev.Dockerfile) can be used to build Lingvo from
    sources.

[How to install docker.](https://docs.docker.com/install/linux/docker-ce/ubuntu/)

### Running the MNIST image model

#### Preparing the input data

**pip:**

```shell
mkdir -p /tmp/mnist
python3 -m lingvo.tools.keras2ckpt --dataset=mnist
```

**bazel:**

```shell
mkdir -p /tmp/mnist
bazel run -c opt //lingvo/tools:keras2ckpt -- --dataset=mnist
```

The following files will be created in `/tmp/mnist`:

*   `mnist.data-00000-of-00001`: 53MB.
*   `mnist.index`: 241 bytes.

#### Running the model

**pip:**

```shell
cd /tmp/mnist
curl -O https://raw.githubusercontent.com/tensorflow/lingvo/master/lingvo/tasks/image/params/mnist.py
python3 -m lingvo.trainer --run_locally=cpu --mode=sync --model=mnist.LeNet5 --logdir=/tmp/mnist/log
```

**bazel:**

```shell
(cpu) bazel build -c opt //lingvo:trainer
(gpu) bazel build -c opt --config=cuda //lingvo:trainer
bazel-bin/lingvo/trainer --run_locally=cpu --mode=sync --model=image.mnist.LeNet5 --logdir=/tmp/mnist/log --logtostderr
```

After about 20 seconds, the loss should drop below 0.3 and a checkpoint will be
saved, like below. Kill the trainer with Ctrl+C.

```
trainer.py:518] step:   205, steps/sec: 11.64 ... loss:0.25747201 ...
checkpointer.py:115] Save checkpoint
checkpointer.py:117] Save checkpoint done: /tmp/mnist/log/train/ckpt-00000205
```

Some artifacts will be produced in `/tmp/mnist/log/control`:

*   `params.txt`: hyper-parameters.
*   `model_analysis.txt`: model sizes for each layer.
*   `train.pbtxt`: the training `tf.GraphDef`.
*   `events.*`: a tensorboard events file.

As well as in `/tmp/mnist/log/train`:

*   `checkpoint`: a text file containing information about the checkpoint files.
*   `ckpt-*`: the checkpoint files.

Now, let's evaluate the model on the "Test" dataset. In the normal training
setup the trainer and evaler should be run at the same time as two separate
processes.

**pip:**

```shell
python3 -m lingvo.trainer --job=evaler_test --run_locally=cpu --mode=sync --model=mnist.LeNet5 --logdir=/tmp/mnist/log
```

**bazel:**

```shell
bazel-bin/lingvo/trainer --job=evaler_test --run_locally=cpu --mode=sync --model=image.mnist.LeNet5 --logdir=/tmp/mnist/log --logtostderr
```

Kill the job with Ctrl+C when it starts waiting for a new checkpoint.

```
base_runner.py:177] No new check point is found: /tmp/mnist/log/train/ckpt-00000205
```

The evaluation accuracy can be found slightly earlier in the logs.

```
base_runner.py:111] eval_test: step:   205, acc5: 0.99775392, accuracy: 0.94150388, ..., loss: 0.20770954, ...
```

### Running the machine translation model

To run a more elaborate model, you'll need a cluster with GPUs. Please refer to
[`third_party/py/lingvo/tasks/mt/README.md`](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/mt/README.md)
for more information.

### Running the GShard transformer based giant language model

To train a GShard language model with one trillion parameters on GCP using
CloudTPUs v3-512 using 512-way model parallelism, please refer to
[`third_party/py/lingvo/tasks/lm/README.md`](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/lm/README.md)
for more information.

### Running the 3d object detection model

To run the StarNet model using CloudTPUs on GCP, please refer to
[`third_party/py/lingvo/tasks/car/README.md`](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/README.md).

## Models

### Automatic Speech Recognition

*   [Listen, Attend and Spell](https://arxiv.org/pdf/1508.01211.pdf).

    William Chan, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. ICASSP 2016.

    [End-to-end Continuous Speech Recognition using Attention-based Recurrent
    NN: First Results](https://arxiv.org/pdf/1412.1602.pdf).
Jan Chorowski,
    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. arXiv 2014.

    *   [asr.librispeech.Librispeech960Grapheme](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/asr/params/librispeech.py)
    *   [asr.librispeech.Librispeech960Wpm](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/asr/params/librispeech.py)

### Car
*   [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://arxiv.org/pdf/2203.08195.pdf).

    Yingwei Li, Adams Wei Yu, Tianjian Meng, Ben Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Bo Wu, Yifeng Lu, Denny
    Zhou, Quoc V. Le, Alan Yuille, Mingxing Tan. CVPR 2022.
    *   [car.waymo_deepfusion.DeepFusionCenterPointPed](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/waymo_deepfusion.py)

*   [StarNet: Targeted Computation for Object Detection in Point Clouds](https://arxiv.org/pdf/1908.11069.pdf).

    Jiquan Ngiam, Benjamin Caine, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin
    Zhou, Xi Yi, Ouais Alsharif, Patrick Nguyen, Zhifeng Chen, Jonathon Shlens,
    and Vijay Vasudevan. arXiv 2019.

    *   [car.kitti.StarNetCarModel0701](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py)
    *   [car.kitti.StarNetPedCycModel0704](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/kitti.py)
    *   [car.waymo.StarNetVehicle](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/waymo.py)
    *   [car.waymo.StarNetPed](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/car/params/waymo.py)

### Image

*   [Gradient-based learning applied to document recognition](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf).

    Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. IEEE 1998.

    *   [image.mnist.LeNet5](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/image/params/mnist.py)

### Language Modelling

*   [Exploring the Limits of Language Modeling](https://arxiv.org/pdf/1602.02410.pdf).

    Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui
    Wu. arXiv, 2016.

    *   [lm.one_billion_wds.WordLevelOneBwdsSimpleSampledSoftmax](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/lm/params/one_billion_wds.py)

*   [GShard: Scaling Giant Models with Conditional Computation and Automatic
    Sharding](https://arxiv.org/pdf/2006.16668.pdf).

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun,
    Noam Shazeer and Zhifeng Chen arXiv, 2020.

    *   [lm.synthetic_packed_input.DenseLm1T16x16](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/lm/params/synthetic_packed_input.py)

### Machine Translation

*   [The Best of Both Worlds: Combining Recent Advances in Neural Machine
    Translation](http://aclweb.org/anthology/P18-1008).

    Mia X. Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion
    Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob
    Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes.
    ACL 2018.

    *   [mt.wmt14_en_de.WmtEnDeTransformerBase](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/mt/params/wmt14_en_de.py)
    *   [mt.wmt14_en_de.WmtEnDeRNMT](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/mt/params/wmt14_en_de.py)
    *   [mt.wmtm16_en_de.WmtCaptionEnDeTransformer](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/mt/params/wmtm16_en_de.py)

*   [Self-supervised and Supervised Joint Training for Resource-rich Neural
    Machine Translation](https://arxiv.org/pdf/2106.04060.pdf).

    Yong Cheng, Wei Wang, Lu Jiang, and Wolfgang Macherey. ICML 2021.

    *   [mt.xendec.wmt14_en_de.WmtEnDeXEnDec](https://github.com/tensorflow/lingvo/blob/master/lingvo/tasks/mt/params/xendec/wmt14_en_de.py)

## References

*   [API Docs](https://tensorflow.github.io/lingvo/)
*   [Codelab](https://colab.research.google.com/github/tensorflow/lingvo/blob/master/codelabs/introduction.ipynb)

Please cite this [paper](https://arxiv.org/abs/1902.08295) when referencing
Lingvo.

```
@misc{shen2019lingvo,
    title={Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling},
    author={Jonathan Shen and Patrick Nguyen and Yonghui Wu and Zhifeng Chen and others},
    year={2019},
    eprint={1902.08295},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

## License

[Apache License 2.0](LICENSE)


Nikola, a Static Site and Blog Generator
========================================

In goes content, out comes a website, ready to deploy.

.. image:: http://img.shields.io/pypi/v/Nikola.svg
   :target: https://pypi.python.org/pypi/Nikola

.. image:: https://github.com/getnikola/nikola/workflows/Nikola%20CI/badge.svg
   :target: https://github.com/getnikola/nikola/actions

.. image:: http://img.shields.io/badge/license-MIT-green.svg
   :target: https://github.com/getnikola/nikola/blob/master/LICENSE.txt

Why Static Websites?
--------------------

Static websites are safer, use fewer resources, and avoid vendor and platform lock-in.
You can read more about this in the `Nikola Handbook`_

What Can Nikola Do?
-------------------

It has many features, but here are some of the nicer ones:

* `Blogs, with tags, feeds, archives, comments, etc.`__
* `Themable`_
* Fast builds, thanks to `doit`_
* Flexible, extensible via the dozens of `available plugins`_
* Small codebase (programmers can understand all of Nikola core in a day)
* `reStructuredText`_ or Markdown as input language (also Wiki, BBCode, Textile, and HTML)
* Easy `image galleries`_ (just drop files in a folder!)
* Syntax highlighting for almost any programming language or markup
* Multilingual sites, `translated to 50 languages.`__
* Doesn't reinvent wheels, leverages existing tools.
* Python 3.8+ compatible.

.. _Nikola Handbook: https://getnikola.com/handbook.html#why-static
__ https://users.getnikola.com/
.. _Themable: https://themes.getnikola.com
.. _doit: https://pydoit.org
.. _available plugins: https://plugins.getnikola.com/
.. _reStructuredText: https://getnikola.com/quickstart.html
.. _image galleries: https://getnikola.com/galleries/demo/
__ https://www.transifex.com/projects/p/nikola/

Nikola Architecture
-------------------

.. image:: https://getnikola.com/images/architecture.png

Installation Instructions
-------------------------

Assuming you have pip installed::

    pip install Nikola

For optional features::

    pip install "Nikola[extras]"

For tests::

    pip install "Nikola[extras,tests]"

For more information, see https://getnikola.com/


![Captum Logo](./website/static/img/captum_logo.png)

<hr/>

<!--- BADGES: START --->
[![GitHub - License](https://img.shields.io/github/license/pytorch/captum?logo=github&style=flat&color=green)][#github-license]
[![Conda](https://img.shields.io/conda/vn/pytorch/captum?logo=anaconda&style=flat&color=orange)](https://anaconda.org/pytorch/captum)
[![PyPI](https://img.shields.io/pypi/v/captum.svg)][#pypi-package]
[![Conda - Platform](https://img.shields.io/conda/pn/conda-forge/captum?logo=anaconda&style=flat)][#conda-forge-package]
[![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/captum?logo=anaconda&style=flat&color=orange)][#conda-forge-package]
[![Conda Recipe](https://img.shields.io/static/v1?logo=conda-forge&style=flat&color=green&label=recipe&message=captum)][#conda-forge-feedstock]
[![Docs - GitHub.io](https://img.shields.io/static/v1?logo=captum&style=flat&color=pink&label=docs&message=captum)][#docs-package]

[#github-license]: https://github.com/pytorch/captum/blob/master/LICENSE
[#pypi-package]: https://pypi.org/project/captum/
[#conda-forge-package]: https://anaconda.org/conda-forge/captum
[#conda-forge-feedstock]: https://github.com/conda-forge/captum-feedstock
[#docs-package]: https://captum.ai/
<!--- BADGES: END --->

Captum is a model interpretability and understanding library for PyTorch.
Captum means comprehension in Latin and contains general purpose implementations
of integrated gradients, saliency maps, smoothgrad, vargrad and others for
PyTorch models. It has quick integration for models built with domain-specific
libraries such as torchvision, torchtext, and others.

*Captum is currently in beta and under active development!*

#### About Captum

With the increase in model complexity and the resulting lack of transparency, model interpretability methods have become increasingly important. Model understanding is both an active area of research as well as an area of focus for practical applications across industries using machine learning. Captum provides state-of-the-art algorithms such as Integrated Gradients, Testing with Concept Activaton Vectors (TCAV), TracIn influence functions, just to name a few, that provide researchers and developers with an easy way to understand which features, training examples or concepts contribute to a models' predictions and in general what and how the model learns. In addition to that, Captum also provides adversarial attacks and minimal input perturbation capabilities that can be used both for generating counterfactual explanations and adversarial perturbations.

<!--For model developers, Captum can be used to improve and troubleshoot models by facilitating the identification of different features that contribute to a model’s output in order to design better models and troubleshoot unexpected model outputs. -->

Captum helps ML researchers more easily implement interpretability algorithms that can interact with PyTorch models. Captum also allows researchers to quickly benchmark their work against other existing algorithms available in the library.

![Overview of Attribution Algorithms](./docs/Captum_Attribution_Algos.png)

#### Target Audience

The primary audiences for Captum are model developers who are looking to improve their models and understand which concepts, features or training examples are important and interpretability researchers focused on identifying algorithms that can better interpret many types of models.

Captum can also be used by application engineers who are using trained models in production. Captum provides easier troubleshooting through improved model interpretability, and the potential for delivering better explanations to end users on why they’re seeing a specific piece of content, such as a movie recommendation.

## Installation

**Installation Requirements**
- Python >= 3.8
- PyTorch >= 1.10

##### Installing the latest release

The latest release of Captum is easily installed either via
[Anaconda](https://www.anaconda.com/distribution/#download-section) (recommended) or via `pip`.

**with `conda`**

You can install captum from any of the following supported conda channels:

- channel: `pytorch`

  ```sh
  conda install captum -c pytorch
  ```

- channel: `conda-forge`

  ```sh
  conda install captum -c conda-forge
  ```

**With `pip`**

```bash
pip install captum
```

**Manual / Dev install**

If you'd like to try our bleeding edge features (and don't mind potentially
running into the occasional bug here or there), you can install the latest
master directly from GitHub. For a basic install, run:
```bash
git clone https://github.com/pytorch/captum.git
cd captum
pip install -e .
```

To customize the installation, you can also run the following variants of the
above:
* `pip install -e .[insights]`: Also installs all packages necessary for running Captum Insights.
* `pip install -e .[dev]`: Also installs all tools necessary for development
  (testing, linting, docs building; see [Contributing](#contributing) below).
* `pip install -e .[tutorials]`: Also installs all packages necessary for running the tutorial notebooks.

To execute unit tests from a manual install, run:
```bash
# running a single unit test
python -m unittest -v tests.attr.test_saliency
# running all unit tests
pytest -ra
```

## Getting Started
Captum helps you interpret and understand predictions of PyTorch models by
exploring features that contribute to a prediction the model makes.
It also helps understand which neurons and layers are important for
model predictions.

Let's apply some of those algorithms to a toy model we have created for
demonstration purposes.
For simplicity, we will use the following architecture, but users are welcome
to use any PyTorch model of their choice.

```python
import numpy as np

import torch
import torch.nn as nn

from captum.attr import (
    GradientShap,
    DeepLift,
    DeepLiftShap,
    IntegratedGradients,
    LayerConductance,
    NeuronConductance,
    NoiseTunnel,
)

class ToyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin1 = nn.Linear(3, 3)
        self.relu = nn.ReLU()
        self.lin2 = nn.Linear(3, 2)

        # initialize weights and biases
        self.lin1.weight = nn.Parameter(torch.arange(-4.0, 5.0).view(3, 3))
        self.lin1.bias = nn.Parameter(torch.zeros(1,3))
        self.lin2.weight = nn.Parameter(torch.arange(-3.0, 3.0).view(2, 3))
        self.lin2.bias = nn.Parameter(torch.ones(1,2))

    def forward(self, input):
        return self.lin2(self.relu(self.lin1(input)))
```

Let's create an instance of our model and set it to eval mode.
```python
model = ToyModel()
model.eval()
```

Next, we need to define simple input and baseline tensors.
Baselines belong to the input space and often carry no predictive signal.
Zero tensor can serve as a baseline for many tasks.
Some interpretability algorithms such as `IntegratedGradients`, `Deeplift` and `GradientShap` are designed to attribute the change
between the input and baseline to a predictive class or a value that the neural
network outputs.

We will apply model interpretability algorithms on the network
mentioned above in order to understand the importance of individual
neurons/layers and the parts of the input that play an important role in the
final prediction.

To make computations deterministic, let's fix random seeds.

```python
torch.manual_seed(123)
np.random.seed(123)
```

Let's define our input and baseline tensors. Baselines are used in some
interpretability algorithms such as `IntegratedGradients, DeepLift,
GradientShap, NeuronConductance, LayerConductance, InternalInfluence` and
`NeuronIntegratedGradients`.

```python
input = torch.rand(2, 3)
baseline = torch.zeros(2, 3)
```
Next we will use `IntegratedGradients` algorithms to assign attribution
scores to each input feature with respect to the first target output.
```python
ig = IntegratedGradients(model)
attributions, delta = ig.attribute(input, baseline, target=0, return_convergence_delta=True)
print('IG Attributions:', attributions)
print('Convergence Delta:', delta)
```
Output:
```
IG Attributions: tensor([[-0.5922, -1.5497, -1.0067],
                         [ 0.0000, -0.2219, -5.1991]])
Convergence Delta: tensor([2.3842e-07, -4.7684e-07])
```
The algorithm outputs an attribution score for each input element and a
convergence delta. The lower the absolute value of the convergence delta the better
is the approximation. If we choose not to return delta,
we can simply not provide the `return_convergence_delta` input
argument. The absolute value of the returned deltas can be interpreted as an
approximation error for each input sample.
It can also serve as a proxy of how accurate the integral approximation for given
inputs and baselines is.
If the approximation error is large, we can try a larger number of integral
approximation steps by setting `n_steps` to a larger value. Not all algorithms
return approximation error. Those which do, though, compute it based on the
completeness property of the algorithms.

Positive attribution score means that the input in that particular position
positively contributed to the final prediction and negative means the opposite.
The magnitude of the attribution score signifies the strength of the contribution.
Zero attribution score means no contribution from that particular feature.

Similarly, we can apply `GradientShap`, `DeepLift` and other attribution algorithms to the model.

`GradientShap` first chooses a random baseline from baselines' distribution, then
 adds gaussian noise with std=0.09 to each input example `n_samples` times.
Afterwards, it chooses a random point between each example-baseline pair and
computes the gradients with respect to target class (in this case target=0). Resulting
attribution is the mean of gradients * (inputs - baselines)
```python
gs = GradientShap(model)

# We define a distribution of baselines and draw `n_samples` from that
# distribution in order to estimate the expectations of gradients across all baselines
baseline_dist = torch.randn(10, 3) * 0.001
attributions, delta = gs.attribute(input, stdevs=0.09, n_samples=4, baselines=baseline_dist,
                                   target=0, return_convergence_delta=True)
print('GradientShap Attributions:', attributions)
print('Convergence Delta:', delta)
```
Output
```
GradientShap Attributions: tensor([[-0.1542, -1.6229, -1.5835],
                                   [-0.3916, -0.2836, -4.6851]])
Convergence Delta: tensor([ 0.0000, -0.0005, -0.0029, -0.0084, -0.0087, -0.0405,  0.0000, -0.0084])

```
Deltas are computed for each `n_samples * input.shape[0]` example. The user can,
for instance, average them:
```python
deltas_per_example = torch.mean(delta.reshape(input.shape[0], -1), dim=1)
```
in order to get per example average delta.

Below is an example of how we can apply `DeepLift` and `DeepLiftShap` on the
`ToyModel` described above. The current implementation of DeepLift supports only the
`Rescale` rule.
For more details on alternative implementations, please see the [DeepLift paper](https://arxiv.org/abs/1704.02685).

```python
dl = DeepLift(model)
attributions, delta = dl.attribute(input, baseline, target=0, return_convergence_delta=True)
print('DeepLift Attributions:', attributions)
print('Convergence Delta:', delta)
```
Output
```
DeepLift Attributions: tensor([[-0.5922, -1.5497, -1.0067],
                               [ 0.0000, -0.2219, -5.1991])
Convergence Delta: tensor([0., 0.])
```
`DeepLift` assigns similar attribution scores as `IntegratedGradients` to inputs,
however it has lower execution time. Another important thing to remember about
DeepLift is that it currently doesn't support all non-linear activation types.
For more details on limitations of the current implementation, please see the
[DeepLift paper](https://arxiv.org/abs/1704.02685).

Similar to integrated gradients, DeepLift returns a convergence delta score
per input example. The approximation error is then the absolute
value of the convergence deltas and can serve as a proxy of how accurate the
algorithm's approximation is.

Now let's look into `DeepLiftShap`. Similar to `GradientShap`, `DeepLiftShap` uses
baseline distribution. In the example below, we use the same baseline distribution
as for `GradientShap`.

```python
dl = DeepLiftShap(model)
attributions, delta = dl.attribute(input, baseline_dist, target=0, return_convergence_delta=True)
print('DeepLiftSHAP Attributions:', attributions)
print('Convergence Delta:', delta)
```
Output
```
DeepLiftShap Attributions: tensor([[-5.9169e-01, -1.5491e+00, -1.0076e+00],
                                   [-4.7101e-03, -2.2300e-01, -5.1926e+00]], grad_fn=<MeanBackward1>)
Convergence Delta: tensor([-4.6120e-03, -1.6267e-03, -5.1045e-04, -1.4184e-03, -6.8886e-03,
                           -2.2224e-02,  0.0000e+00, -2.8790e-02, -4.1285e-03, -2.7295e-02,
                           -3.2349e-03, -1.6265e-03, -4.7684e-07, -1.4191e-03, -6.8889e-03,
                           -2.2224e-02,  0.0000e+00, -2.4792e-02, -4.1289e-03, -2.7296e-02])
```
`DeepLiftShap` uses `DeepLift` to compute attribution score for each
input-baseline pair and averages it for each input across all baselines.

It computes deltas for each input example-baseline pair, thus resulting to
`input.shape[0] * baseline.shape[0]` delta values.

Similar to GradientShap in order to compute example-based deltas we can average them per example:
```python
deltas_per_example = torch.mean(delta.reshape(input.shape[0], -1), dim=1)
```
In order to smooth and improve the quality of the attributions we can run
`IntegratedGradients` and other attribution methods through a `NoiseTunnel`.
`NoiseTunnel` allows us to use `SmoothGrad`, `SmoothGrad_Sq` and `VarGrad` techniques
to smoothen the attributions by aggregating them for multiple noisy
samples that were generated by adding gaussian noise.

Here is an example of how we can use `NoiseTunnel` with `IntegratedGradients`.

```python
ig = IntegratedGradients(model)
nt = NoiseTunnel(ig)
attributions, delta = nt.attribute(input, nt_type='smoothgrad', stdevs=0.02, nt_samples=4,
      baselines=baseline, target=0, return_convergence_delta=True)
print('IG + SmoothGrad Attributions:', attributions)
print('Convergence Delta:', delta)
```
Output
```
IG + SmoothGrad Attributions: tensor([[-0.4574, -1.5493, -1.0893],
                                      [ 0.0000, -0.2647, -5.1619]])
Convergence Delta: tensor([ 0.0000e+00,  2.3842e-07,  0.0000e+00, -2.3842e-07,  0.0000e+00,
        -4.7684e-07,  0.0000e+00, -4.7684e-07])

```
The number of elements in the `delta` tensor is equal to: `nt_samples * input.shape[0]`
In order to get an example-wise delta, we can, for example, average them:
```python
deltas_per_example = torch.mean(delta.reshape(input.shape[0], -1), dim=1)
```

Let's look into the internals of our network and understand which layers
and neurons are important for the predictions.

We will start with the `NeuronConductance`. `NeuronConductance` helps us to identify
input features that are important for a particular neuron in a given
layer. It decomposes the computation of integrated gradients via the chain rule by
defining the importance of a neuron as path integral of the derivative of the output
with respect to the neuron times the derivatives of the neuron with respect to the
inputs of the model.

In this case, we choose to analyze the first neuron in the linear layer.

```python
nc = NeuronConductance(model, model.lin1)
attributions = nc.attribute(input, neuron_selector=1, target=0)
print('Neuron Attributions:', attributions)
```
Output
```
Neuron Attributions: tensor([[ 0.0000,  0.0000,  0.0000],
                             [ 1.3358,  0.0000, -1.6811]])
```

Layer conductance shows the importance of neurons for a layer and given input.
It is an extension of path integrated gradients for hidden layers and holds the
completeness property as well.

It doesn't attribute the contribution scores to the input features
but shows the importance of each neuron in the selected layer.
```python
lc = LayerConductance(model, model.lin1)
attributions, delta = lc.attribute(input, baselines=baseline, target=0, return_convergence_delta=True)
print('Layer Attributions:', attributions)
print('Convergence Delta:', delta)
```
Outputs
```
Layer Attributions: tensor([[ 0.0000,  0.0000, -3.0856],
                            [ 0.0000, -0.3488, -4.9638]], grad_fn=<SumBackward1>)
Convergence Delta: tensor([0.0630, 0.1084])
```

Similar to other attribution algorithms that return convergence delta, `LayerConductance`
returns the deltas for each example. The approximation error is then the absolute
value of the convergence deltas and can serve as a proxy of how accurate integral
approximation for given inputs and baselines is.

More details on the list of supported algorithms and how to apply
Captum on different types of models can be found in our tutorials.

## Captum Insights

Captum provides a web interface called Insights for easy visualization and
access to a number of our interpretability algorithms.

To analyze a sample model on CIFAR10 via Captum Insights run

```
python -m captum.insights.example
```

and navigate to the URL specified in the output.

![Captum Insights Screenshot](./website/static/img/captum_insights_screenshot.png)

To build Insights you will need [Node](https://nodejs.org/en/) >= 8.x
and [Yarn](https://yarnpkg.com/en/) >= 1.5.

To build and launch from a checkout in a conda environment run

```
conda install -c conda-forge yarn
BUILD_INSIGHTS=1 python setup.py develop
python captum/insights/example.py
```

### Captum Insights Jupyter Widget
Captum Insights also has a Jupyter widget providing the same user interface as the web app.
To install and enable the widget, run

```
jupyter nbextension install --py --symlink --sys-prefix captum.insights.attr_vis.widget
jupyter nbextension enable captum.insights.attr_vis.widget --py --sys-prefix
```

To build the widget from a checkout in a conda environment run

```
conda install -c conda-forge yarn
BUILD_INSIGHTS=1 python setup.py develop
```

## FAQ
If you have questions about using Captum methods, please check this [FAQ](docs/faq.md), which addresses many common issues.

## Contributing
See the [CONTRIBUTING](CONTRIBUTING.md) file for how to help out.

## Talks and Papers
**NeurIPS 2019:**
The slides of our presentation  can be found [here](docs/presentations/Captum_NeurIPS_2019_final.key)

**KDD 2020:**
The slides of our presentation from KDD 2020 tutorial can be found [here](https://pytorch-tutorial-assets.s3.amazonaws.com/Captum_KDD_2020.pdf).
You can watch the recorded talk [here](https://www.youtube.com/watch?v=hY_XzglTkak)

**GTC 2020:**
Opening Up the Black Box: Model Understanding with Captum and PyTorch.
You can watch the recorded talk [here](https://www.youtube.com/watch?v=0QLrRyLndFI)

**XAI Summit 2020:**
Using Captum and Fiddler to Improve Model Understanding with Explainable AI.
You can watch the recorded talk [here](https://www.youtube.com/watch?v=dvuVld5Hyc8)

**PyTorch Developer Day 2020**
Model Interpretability.
You can watch the recorded talk [here](https://www.youtube.com/watch?v=Lj5hHBGue58)

**NAACL 2021**
Tutorial on Fine-grained Interpretation and Causation Analysis in Deep NLP Models.
You can watch the recorded talk [here](https://www.youtube.com/watch?v=ayhBHZYjeqs
)

**ICLR 2021 workshop on Responsible AI**:
- [Paper](https://arxiv.org/abs/2009.07896) on the Captum Library
- [Paper](https://arxiv.org/abs/2106.07475) on Invesitgating Sanity Checks for Saliency Maps

Summer school on medical imaging at University of Lyon. A class on model explainability (link to the video)
https://www.youtube.com/watch?v=vn-jLzY67V0

## References of Algorithms

* `IntegratedGradients`, `LayerIntegratedGradients`: [Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017](https://arxiv.org/abs/1703.01365) and [Did the Model Understand the Question?, Pramod K. Mudrakarta, et al. 2018](https://arxiv.org/abs/1805.05492)
* `InputXGradient`: [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences, Avanti Shrikumar et al. 2016](https://arxiv.org/abs/1605.01713)
* `SmoothGrad`: [SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017](https://arxiv.org/abs/1706.03825)
* `NoiseTunnel`: [Sanity Checks for Saliency Maps, Julius Adebayo et al. 2018](https://arxiv.org/abs/1810.03292)
* `NeuronConductance`: [How Important is a neuron?, Kedar Dhamdhere et al. 2018](https://arxiv.org/abs/1805.12233)
* `LayerConductance`: [Computationally Efficient Measures of Internal Neuron Importance, Avanti Shrikumar et al. 2018](https://arxiv.org/abs/1807.09946)
* `DeepLift`, `NeuronDeepLift`, `LayerDeepLift`: [Learning Important Features Through Propagating Activation Differences, Avanti Shrikumar et al. 2017](https://arxiv.org/abs/1704.02685) and [Towards better understanding of gradient-based attribution methods for deep neural networks, Marco Ancona et al. 2018](https://openreview.net/pdf?id=Sy21R9JAW)
* `NeuronIntegratedGradients`: [Computationally Efficient Measures of Internal Neuron Importance, Avanti Shrikumar et al. 2018](https://arxiv.org/abs/1807.09946)
* `GradientShap`, `NeuronGradientShap`, `LayerGradientShap`, `DeepLiftShap`, `NeuronDeepLiftShap`, `LayerDeepLiftShap`: [A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg et al. 2017](http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions)
* `InternalInfluence`: [Influence-Directed Explanations for Deep Convolutional Networks, Klas Leino et al. 2018](https://arxiv.org/abs/1802.03788)
* `Saliency`, `NeuronGradient`: [Deep Inside Convolutional Networks: Visualising
Image Classification Models and Saliency Maps, K. Simonyan, et. al. 2014](https://arxiv.org/abs/1312.6034)
* `GradCAM`, `Guided GradCAM`: [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017](https://arxiv.org/abs/1610.02391)
* `Deconvolution`, `Neuron Deconvolution`: [Visualizing and Understanding Convolutional Networks, Matthew D Zeiler et al. 2014](https://arxiv.org/abs/1311.2901)
* `Guided Backpropagation`, `Neuron Guided Backpropagation`: [Striving for Simplicity: The All Convolutional Net, Jost Tobias Springenberg et al. 2015](https://arxiv.org/abs/1412.6806)
* `Feature Permutation`: [Permutation Feature Importance](https://christophm.github.io/interpretable-ml-book/feature-importance.html)
* `Occlusion`: [Visualizing and Understanding Convolutional Networks](https://arxiv.org/abs/1311.2901)
* `Shapley Value`: [A value for n-person games. Contributions to the Theory of Games 2.28 (1953): 307-317](https://apps.dtic.mil/dtic/tr/fulltext/u2/604084.pdf)
* `Shapley Value Sampling`: [Polynomial calculation of the Shapley value based on sampling](https://www.sciencedirect.com/science/article/pii/S0305054808000804)
* `Infidelity and Sensitivity`: [On the (In)fidelity and Sensitivity for Explanations](https://arxiv.org/abs/1901.09392)
* `TracInCP, TracInCPFast, TracInCPRandProj`: [Estimating Training Data Influence by Tracing Gradient Descent](https://arxiv.org/abs/2002.08484)
* `SimilarityInfluence`: [Pairwise similarities between train and test examples based on predefined similarity metrics]
* `BinaryConcreteStochasticGates`: [Stochastic Gates with Binary Concrete Distribution](https://arxiv.org/abs/1712.01312)
* `GaussianStochasticGates`: [Stochastic Gates with Gaussian Distribution](https://arxiv.org/abs/1810.04247)

More details about the above mentioned [attribution algorithms](https://captum.ai/docs/attribution_algorithms) and their pros and cons can be found on our [web-site](https://captum.ai/docs/algorithms_comparison_matrix).

## License
Captum is BSD licensed, as found in the [LICENSE](LICENSE) file.


# Donkeycar: a python self driving library

![Build Status](https://github.com/autorope/donkeycar/actions/workflows/python-package-conda.yml/badge.svg?branch=main)
![Lint Status](https://github.com/autorope/donkeycar/actions/workflows/superlinter.yml/badge.svg?branch=main)
![Release](https://img.shields.io/github/v/release/autorope/donkeycar)

[![All Contributors](https://img.shields.io/github/contributors/autorope/donkeycar)](#contributors-)
![Issues](https://img.shields.io/github/issues/autorope/donkeycar)
![Pull Requests](https://img.shields.io/github/issues-pr/autorope/donkeycar?)
![Forks](https://img.shields.io/github/forks/autorope/donkeycar)
![Stars](https://img.shields.io/github/stars/autorope/donkeycar)
![License](https://img.shields.io/github/license/autorope/donkeycar)

![Discord](https://img.shields.io/discord/662098530411741184.svg?logo=discord&colorB=7289DA)

Donkeycar is minimalist and modular self driving library for Python. It is developed for hobbyists and students with a focus on allowing fast experimentation and easy community contributions.  It is being actively used at the high school and university level for learning and research.  It offers a [rich graphical interface](https://docs.donkeycar.com/utility/ui/) and includes a [simulator](https://docs.donkeycar.com/guide/deep_learning/simulator/) so you can experiment with self-driving even before you build a robot.

#### Quick Links
* [Donkeycar Updates & Examples](http://donkeycar.com)
* [Build instructions and Software documentation](http://docs.donkeycar.com)
* [Discord / Chat](https://discord.gg/PN6kFeA)

![donkeycar](https://github.com/autorope/donkeydocs/blob/master/docs/assets/build_hardware/donkey2.png)

### Use Donkeycar if you want to:
* Build a robot and teach it to drive itself.
* Experiment with [autopilots](https://docs.donkeycar.com/guide/train_autopilot/), gps, computer vision and neural networks.
* Compete in self driving races like [DIY Robocars](http://diyrobocars.com), including [online simulator races](https://docs.donkeycar.com/guide/deep_learning/virtual_race_league/) against competitors from around the world.
* Participate in a vibrant online community learning cutting edge techology and having fun doing it.

### What do you need to know before starting? (TL;DR nothing)
Donkeycar is designed to be the 'Hello World' of automomous driving; it is simple yet flexible and powerful.  No specific prequisite knowledge is required, but it helps if you have some knowledge of:
- [Python](https://docs.python.org/3.11/) programming.  You do not have to do any programming to use Donkeycar.  The file that you edit to configure your car, `myconfig.py`, is a Python file.  You mostly just uncomment the sections you want to change and edit them; you can avoid common mistakes if you know how Python [comments](https://www.w3schools.com/python/python_comments.asp) and [indentation](https://www.w3schools.com/python/python_syntax.asp) works.
- Raspberry Pi.  The Raspberry Pi is the preferred on-board computer for a Donkeycar.  It is helpful to have setup and used a Raspberry Pi, but it is not necessary.  The Donkeycar documentation describes how to install the software on a RaspberryPi OS, but the specifics of how to install the RaspberryPi OS using [Raspberry Pi Imager](https://www.raspberrypi.com/software/) and how to configure the Raspberry Pi using [raspi-config](https://www.raspberrypi.com/documentation/computers/configuration.html) is left to the Raspberry Pi documentation, which is extensive and quite good. I would recommend setting up your Raspberry Pi using the Raspberry Pi documentation and then play with it a little; use the browser to visit websites and watch YouTube videos, like this one taken at the [very first outdoor race](https://youtu.be/tjWmrCIKgnE) for a Donkeycar.  Use a text editor to write and save a file.  Open a terminal and learn how to navigate the file system (see below). If you are comfortable with the Raspberry Pi then you won't have to learn it and Donkeycar at the same time.
- The Linux [command line shell](https://magpi.raspberrypi.com/articles/terminal-help).  The command line shell is also often called the terminal.  You will type commands into the terminal to install and start the Donkeycar software.  The Donkeycar documentation describes how this works.  It is also helpful to know how navigate the file system and how to list, copy and delete files and directories/folders. You may also access your car [remotely](https://www.raspberrypi.com/documentation/computers/remote-access.html); so you will want to know how to enable and connect WIFI and how to enable and start an [SSH](https://www.raspberrypi.com/documentation/computers/remote-access.html#ssh) terminal or [VNC](https://www.raspberrypi.com/documentation/computers/remote-access.html#vnc) session from your host computer to get a command line on your car.

## Get driving.
After [building a Donkeycar](https://docs.donkeycar.com/guide/build_hardware/) and [installing](https://docs.donkeycar.com/guide/install_software/) the Donkeycar software you can choose your autopilot [template](https://docs.donkeycar.com/guide/create_application/) and [calibrate](https://docs.donkeycar.com/guide/calibrate/) your car and [get driving](https://docs.donkeycar.com/guide/get_driving/)!

## Modify your car's behavior.
Donkeycar includes a number of pre-built [templates](https://docs.donkeycar.com/guide/create_application/) that make it easy to get started by just changing configuration. The pre-built templates are all you may ever need, but if you want to go farther you can change a template or make your own. A Donkeycar template is organized as a pipeline of software [parts](https://docs.donkeycar.com/parts/about/) that run in order on each pass through the vehicle loop, reading inputs and writing outputs to the vehicle's software memory as they run.  A typical car has a parts that:
- Get images from a camera. Donkeycar supports lots of different kinds of [cameras](https://docs.donkeycar.com/parts/cameras/), including 3D cameras and [lidar](https://docs.donkeycar.com/parts/lidar/).
- Get position readings from a GPS receiver.
- Get steering and throttle inputs from a [game controller](https://docs.donkeycar.com/parts/controllers/) or RC controller.  Donkeycar support PS3, PS4, XBox, WiiU, Nimbus and Logitech Bluetooth game controllers and any game controller that works with RaspberryPi.  Donkeycar also implements a WebUI that allows any browser compatible game controller to be connected and also offers an onscreen touch controller that works with phones.
- Control the car's drivetrain [motors](https://docs.donkeycar.com/parts/actuators/) for acceleration and steering. Donkeycar supports various drivetrains including the ESC/Steering-servo configuration that is common to most RC cars and Differential Drive configurations.
- Save telemetry [data](https://docs.donkeycar.com/parts/stores/) such as camera images, steering and throttle inputs, lidar data, etc.
- Drive the car on autopilot.  Donkey supports three kinds of [autopilots](https://docs.donkeycar.com/guide/train_autopilot/); a [deep-learning](https://docs.donkeycar.com/guide/deep_learning/train_autopilot/) autopilot, a [gps autopilot](https://docs.donkeycar.com/guide/path_follow/path_follow/) and a [computer vision](https://docs.donkeycar.com/guide/computer_vision/computer_vision/) autopilot.  The Deep Learning autopilot supports Tensorflow, Tensorflow Lite, and Pytorch and many model [architectures](https://docs.donkeycar.com/parts/keras/).

If there isn't a Donkeycar part that does what you want then write your own [part](https://docs.donkeycar.com/parts/about/#parts) and add it to a vehicle [template](https://docs.donkeycar.com/parts/about/).

```python
#Define a vehicle to take and record pictures 10 times per second.

import time
from donkeycar import Vehicle
from donkeycar.parts.cv import CvCam
from donkeycar.parts.tub_v2 import TubWriter
V = Vehicle()

IMAGE_W = 160
IMAGE_H = 120
IMAGE_DEPTH = 3

#Add a camera part
cam = CvCam(image_w=IMAGE_W, image_h=IMAGE_H, image_d=IMAGE_DEPTH)
V.add(cam, outputs=['image'], threaded=True)

#warmup camera
while cam.run() is None:
    time.sleep(1)

#add tub part to record images
tub = TubWriter(path='./dat', inputs=['image'], types=['image_array'])
V.add(tub, inputs=['image'], outputs=['num_records'])

#start the drive loop at 10 Hz
V.start(rate_hz=10)
```

See [home page](http://donkeycar.com), [docs](http://docs.donkeycar.com)
or join the [Discord server](http://www.donkeycar.com/community.html) to learn more.


django-helpdesk - A Django powered ticket tracker for small businesses.
=======================================================================

.. image:: https://dev.azure.com/django-helpdesk/django-helpdesk/_apis/build/status/django-helpdesk.django-helpdesk?branchName=master
  :target: https://dev.azure.com/django-helpdesk/django-helpdesk/_build/latest?definitionId=1&branchName=master
  :alt: Build Status

.. image:: https://codecov.io/gh/django-helpdesk/django-helpdesk/branch/develop/graph/badge.svg
  :target: https://codecov.io/gh/django-helpdesk/django-helpdesk

Copyright 2009-2023 Ross Poulton and django-helpdesk contributors. All Rights Reserved.
See LICENSE for details.

django-helpdesk was formerly known as Jutda Helpdesk, named after the
company which originally created it. As of January 2011 the name has been
changed to reflect what it really is: a Django-powered ticket tracker with
contributors reaching far beyond Jutda.

Complete documentation is available in the docs/ directory,
or online at http://django-helpdesk.readthedocs.org/.

Demo Quickstart
---------------

`django-helpdesk` includes a basic demo Django project so that you may easily
get started with testing or developing `django-helpdesk`. The demo project
resides in the `demo/` top-level folder.

It's likely that you can start up a demo project server by running
only the command (consider creating a virtualenv before):

    make rundemo

or with docker::

    docker build . -t demodesk
    docker run --rm -v "$PWD:/app" -p 8080:8080 demodesk

then pointing your web browser at http://localhost:8080 (log in as user
`admin`` with password `Test1234`).

For more information and options, please read the `demo/README.rst` file.

**NOTE REGARDING SQLITE AND SEARCHING:**
The demo project uses `sqlite` as its database. Sqlite does not allow
case-insensitive searches and so the search function may not work as
effectively as it would on other database such as PostgreSQL or MySQL
that does support case-insensitive searches.
For more information, see this note_ in the Django documentation.

When you try to do a keyword search using `sqlite`, a message will be displayed
to alert you to this shortcoming. There is no way around it, sorry.

Installation
------------

* |standalone_icon| For **standalone** installation, refer to `standalone documentation <./docs/standalone.rst>`_.

* |django_icon| To **integrate** with an existing Django application, follow the guidelines in `installation documentation <./docs/install.rst>`_ and `configuration documentation <./docs/configuration.rst>`_.

.. |standalone_icon| image:: helpdesk/static/helpdesk/img/icon512.png
   :height: 24px
   :width: 24px
   :align: middle

.. |django_icon| image:: helpdesk/static/helpdesk/img/django-logo-positive.png
   :height: 24px
   :width: 60px
   :align: middle

Developer Environment
---------------------

Follow these steps to set up your development environment to contribute to helpdesk:
 - check out the helpdesk app to your local file system::

    git clone https://github.com/django-helpdesk/django-helpdesk.git

 - install a virtual environment and activate it::

    python -m venv .venv && source .venv/bin/activate

 - install the requirements for development::

    pip install -r requirements.txt -r requirements-dev.txt

 - you can install the requirements for testing as well::

    pip install -r requirements-testing.txt

To deactivate the virtual environment, use ``deactivate``. Then to reactivate it, just run::

    source .venv/bin/activate

To see option for the Makefile run: `make`

The project enforces a standardized formatting in the CI/CD pipeline. To ensure you have the correct formatting run::

    make checkformat

To auto format any code use this::

    make format

Testing
-------

From the command line you can run the tests using: `make test`

See `quicktest.py` for usage details.

If you need to create tests for new features, add your tests in a test file to the `tests` module and call them in the test VENV with::
    python quicktest.py helpdesk.tests.test_my_new_features -v 2

Upgrading from previous versions
--------------------------------

If you are upgrading from a previous version of `django-helpdesk` that used
migrations, get an up to date version of the code base (eg by using
`git pull` or `pip install --upgrade django-helpdesk`) then migrate the database::

    python manage.py migrate helpdesk --db-dry-run # DB untouched
    python manage.py migrate helpdesk

Lastly, restart your web server software (eg Apache) or FastCGI instance, to
ensure the latest changes are in use.

Unfortunately we are unable to assist if you are upgrading from a
version of `django-helpdesk` prior to migrations (ie pre-2011).

You can continue to the 'Initial Configuration' area, if needed.

Contributing
------------

We're happy to include any type of contribution! This can be:

* back-end python/django code development
* front-end web development (HTML/Javascript, especially jQuery)
* language translations
* writing improved documentation and demos

For more information on contributing, please see the `CONTRIBUTING.rst` file.

Licensing
---------

django-helpdesk is licensed under terms of the BSD 3-clause license.
See the `LICENSE` file for full licensing terms.

Note that django-helpdesk is distributed with 3rd party products which
have their own licenses. See LICENSE.3RDPARTY for license terms for
included packages.

.. _note: https://docs.djangoproject.com/en/dev/ref/databases/#substring-matching-and-case-sensitivity


.. image:: https://github.com/thonny/thonny/blob/master/thonny/res/Ukraine.png

`Support Ukraine! <https://github.com/thonny/thonny/wiki/Support-Ukraine>`_

======
Thonny
======

Thonny is a Python `IDE <https://en.wikipedia.org/wiki/Integrated_development_environment>`_ meant for learning programming.

End users
---------
See https://thonny.org and `wiki <https://github.com/thonny/thonny/wiki>`_ for more info.

Contributors
------------
Contributions are welcome! See `CONTRIBUTING.rst <https://github.com/thonny/thonny/blob/master/CONTRIBUTING.rst>`_ for more info.

Sponsors
----------
You can sponsor development of Thonny by sending a donation to Thonny's main author Aivar Annamaa: https://github.com/thonny/thonny/wiki/Sponsors


<img src="https://github.com/strawberry-graphql/strawberry/raw/main/.github/logo.png" width="124" height="150">

# Strawberry GraphQL

> Python GraphQL library based on dataclasses

[![CircleCI](https://img.shields.io/circleci/token/307b40d5e152e074d34f84d30d226376a15667d5/project/github/strawberry-graphql/strawberry/main.svg?style=for-the-badge)](https://circleci.com/gh/strawberry-graphql/strawberry/tree/main)
[![Discord](https://img.shields.io/discord/689806334337482765?label=discord&logo=discord&logoColor=white&style=for-the-badge&color=blue)](https://discord.gg/ZkRTEJQ)
[![PyPI](https://img.shields.io/pypi/v/strawberry-graphql?logo=pypi&logoColor=white&style=for-the-badge)](https://pypi.org/project/strawberry-graphql/)

## Installation ( Quick Start )

The quick start method provides a server and CLI to get going quickly. Install
with:

```shell
pip install "strawberry-graphql[debug-server]"
```

## Getting Started

Create a file called `app.py` with the following code:

```python
import strawberry

@strawberry.type
class User:
    name: str
    age: int

@strawberry.type
class Query:
    @strawberry.field
    def user(self) -> User:
        return User(name="Patrick", age=100)

schema = strawberry.Schema(query=Query)
```

This will create a GraphQL schema defining a `User` type and a single query
field `user` that will return a hardcoded user.

To run the debug server run the following command:

```shell
strawberry server app
```

Open the debug server by clicking on the following link:
[http://0.0.0.0:8000/graphql](http://0.0.0.0:8000/graphql)

This will open GraphiQL where you can test the API.

### Type-checking

Strawberry comes with a [mypy] plugin that enables statically type-checking your
GraphQL schema. To enable it, add the following lines to your `mypy.ini`
configuration:

```ini
[mypy]
plugins = strawberry.ext.mypy_plugin
```

[mypy]: http://www.mypy-lang.org/

### Django Integration

A Django view is provided for adding a GraphQL endpoint to your application.

1. Add the app to your `INSTALLED_APPS`.

```python
INSTALLED_APPS = [
    ...,  # your other apps
    "strawberry.django",
]
```

2. Add the view to your `urls.py` file.

```python
from strawberry.django.views import GraphQLView
from .schema import schema

urlpatterns = [
    ...,
    path("graphql", GraphQLView.as_view(schema=schema)),
]
```

## WebSockets

To support graphql Subscriptions over WebSockets you need to provide a WebSocket
enabled server. The debug server can be made to support WebSockets with these
commands:

```shell
pip install 'strawberry-graphql[debug-server]'
pip install 'uvicorn[standard]'
```

## Examples

* [Various examples on how to use Strawberry](https://github.com/strawberry-graphql/examples)
* [Full stack example using Starlette, SQLAlchemy, Typescript codegen and Next.js](https://github.com/jokull/python-ts-graphql-demo)
* [Quart + Strawberry tutorial](https://github.com/rockyburt/Ketchup)

## Contributing

We use [poetry](https://github.com/sdispater/poetry) to manage dependencies, to
get started follow these steps:

```shell
git clone https://github.com/strawberry-graphql/strawberry
cd strawberry
poetry install --with integrations
poetry run pytest
```

For all further detail, check out the [Contributing Page](CONTRIBUTING.md)

### Pre commit

We have a configuration for
[pre-commit](https://github.com/pre-commit/pre-commit), to add the hook run the
following command:

```shell
pre-commit install
```

## Links

- Project homepage: https://strawberry.rocks
- Repository: https://github.com/strawberry-graphql/strawberry
- Issue tracker: https://github.com/strawberry-graphql/strawberry/issues
  - In case of sensitive bugs like security vulnerabilities, please contact
    patrick.arminio@gmail.com directly instead of using the issue tracker. We
    value your effort to improve the security and privacy of this project!

## Licensing

The code in this project is licensed under MIT license. See [LICENSE](./LICENSE)
for more information.

![Recent Activity](https://images.repography.com/0/strawberry-graphql/strawberry/recent-activity/d751713988987e9331980363e24189ce.svg)


    OpenMMLab website

        HOT

    OpenMMLab platform

        TRY IT OUT

[![PyPI](https://img.shields.io/pypi/v/mmdet3d)](https://pypi.org/project/mmdet3d)
[![docs](https://img.shields.io/badge/docs-latest-blue)](https://mmdetection3d.readthedocs.io/en/latest/)
[![badge](https://github.com/open-mmlab/mmdetection3d/workflows/build/badge.svg)](https://github.com/open-mmlab/mmdetection3d/actions)
[![codecov](https://codecov.io/gh/open-mmlab/mmdetection3d/branch/main/graph/badge.svg)](https://codecov.io/gh/open-mmlab/mmdetection3d)
[![license](https://img.shields.io/github/license/open-mmlab/mmdetection3d.svg)](https://github.com/open-mmlab/mmdetection3d/blob/main/LICENSE)
[![open issues](https://isitmaintained.com/badge/open/open-mmlab/mmdetection3d.svg)](https://github.com/open-mmlab/mmdetection3d/issues)
[![issue resolution](https://isitmaintained.com/badge/resolution/open-mmlab/mmdetection3d.svg)](https://github.com/open-mmlab/mmdetection3d/issues)

[📘Documentation](https://mmdetection3d.readthedocs.io/en/latest/) |
[🛠Installation](https://mmdetection3d.readthedocs.io/en/latest/get_started.html) |
[👀Model Zoo](https://mmdetection3d.readthedocs.io/en/latest/model_zoo.html) |
[🆕Update News](https://mmdetection3d.readthedocs.io/en/latest/notes/changelog.html) |
[🚀Ongoing Projects](https://github.com/open-mmlab/mmdetection3d/projects) |
[🤔Reporting Issues](https://github.com/open-mmlab/mmdetection3d/issues/new/choose)

English | [简体中文](README_zh-CN.md)

## Introduction

MMDetection3D is an open source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D detection. It is a part of the [OpenMMLab](https://openmmlab.com/) project.

The main branch works with **PyTorch 1.8+**.

![demo image](resources/mmdet3d_outdoor_demo.gif)

Major features

- **Support multi-modality/single-modality detectors out of box**

  It directly supports multi-modality/single-modality detectors including MVXNet, VoteNet, PointPillars, etc.

- **Support indoor/outdoor 3D detection out of box**

  It directly supports popular indoor and outdoor 3D detection datasets, including ScanNet, SUNRGB-D, Waymo, nuScenes, Lyft, and KITTI. For nuScenes dataset, we also support [nuImages dataset](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/nuimages).

- **Natural integration with 2D detection**

  All the about **300+ models, methods of 40+ papers**, and modules supported in [MMDetection](https://github.com/open-mmlab/mmdetection/blob/3.x/docs/en/model_zoo.md) can be trained or used in this codebase.

- **High efficiency**

  It trains faster than other codebases. The main results are as below. Details can be found in [benchmark.md](./docs/en/notes/benchmarks.md). We compare the number of samples trained per second (the higher, the better). The models that are not supported by other codebases are marked by `✗`.

  |       Methods       | MMDetection3D | [OpenPCDet](https://github.com/open-mmlab/OpenPCDet) | [votenet](https://github.com/facebookresearch/votenet) | [Det3D](https://github.com/poodarchu/Det3D) |
  | :-----------------: | :-----------: | :--------------------------------------------------: | :----------------------------------------------------: | :-----------------------------------------: |
  |       VoteNet       |      358      |                          ✗                           |                           77                           |                      ✗                      |
  |  PointPillars-car   |      141      |                          ✗                           |                           ✗                            |                     140                     |
  | PointPillars-3class |      107      |                          44                          |                           ✗                            |                      ✗                      |
  |       SECOND        |      40       |                          30                          |                           ✗                            |                      ✗                      |
  |       Part-A2       |      17       |                          14                          |                           ✗                            |                      ✗                      |

Like [MMDetection](https://github.com/open-mmlab/mmdetection) and [MMCV](https://github.com/open-mmlab/mmcv), MMDetection3D can also be used as a library to support different projects on top of it.

## What's New

### Highlight

In version 1.4, MMDetecion3D refactors the Waymo dataset and accelerates the preprocessing, training/testing setup, and evaluation of Waymo dataset. We also extends the support for camera-based, such as Monocular and BEV, 3D object detection models on Waymo. A detailed description of the Waymo data information is provided [here](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html).

Besides, in version 1.4, MMDetection3D provides [Waymo-mini](https://download.openmmlab.com/mmdetection3d/data/waymo_mmdet3d_after_1x4/waymo_mini.tar.gz) to help community users get started with Waymo and use it for quick iterative development.

**v1.4.0** was released in 8/1/2024：

- Support the training of [DSVT](<(https://arxiv.org/abs/2301.06051)>) in `projects`
- Support [Nerf-Det](https://arxiv.org/abs/2307.14620) in `projects`
- Refactor Waymo dataset

**v1.3.0** was released in 18/10/2023:

- Support [CENet](https://arxiv.org/abs/2207.12691) in `projects`
- Enhance demos with new 3D inferencers

**v1.2.0** was released in 4/7/2023

- Support [New Config Type](https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#a-pure-python-style-configuration-file-beta) in `mmdet3d/configs`
- Support the inference of [DSVT](<(https://arxiv.org/abs/2301.06051)>) in `projects`
- Support downloading datasets from [OpenDataLab](https://opendatalab.com/) using `mim`

**v1.1.1** was released in 30/5/2023:

- Support [TPVFormer](https://arxiv.org/pdf/2302.07817.pdf) in `projects`
- Support the training of BEVFusion in `projects`
- Support lidar-based 3D semantic segmentation benchmark

## Installation

Please refer to [Installation](https://mmdetection3d.readthedocs.io/en/latest/get_started.html) for installation instructions.

## Getting Started

For detailed user guides and advanced guides, please refer to our [documentation](https://mmdetection3d.readthedocs.io/en/latest/):

User Guides

- [Train & Test](https://mmdetection3d.readthedocs.io/en/latest/user_guides/index.html#train-test)
  - [Learn about Configs](https://mmdetection3d.readthedocs.io/en/latest/user_guides/config.html)
  - [Coordinate System](https://mmdetection3d.readthedocs.io/en/latest/user_guides/coord_sys_tutorial.html)
  - [Dataset Preparation](https://mmdetection3d.readthedocs.io/en/latest/user_guides/dataset_prepare.html)
  - [Customize Data Pipelines](https://mmdetection3d.readthedocs.io/en/latest/user_guides/data_pipeline.html)
  - [Test and Train on Standard Datasets](https://mmdetection3d.readthedocs.io/en/latest/user_guides/train_test.html)
  - [Inference](https://mmdetection3d.readthedocs.io/en/latest/user_guides/inference.html)
  - [Train with Customized Datasets](https://mmdetection3d.readthedocs.io/en/latest/user_guides/new_data_model.html)
- [Useful Tools](https://mmdetection3d.readthedocs.io/en/latest/user_guides/index.html#useful-tools)

Advanced Guides

- [Datasets](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#datasets)
  - [KITTI Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/kitti.html)
  - [NuScenes Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/nuscenes.html)
  - [Lyft Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/lyft.html)
  - [Waymo Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html)
  - [SUN RGB-D Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/sunrgbd.html)
  - [ScanNet Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/scannet.html)
  - [S3DIS Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/s3dis.html)
  - [SemanticKITTI Dataset](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/semantickitti.html)
- [Supported Tasks](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#supported-tasks)
  - [LiDAR-Based 3D Detection](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_det3d.html)
  - [Vision-Based 3D Detection](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/vision_det3d.html)
  - [LiDAR-Based 3D Semantic Segmentation](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_sem_seg3d.html)
- [Customization](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#customization)
  - [Customize Datasets](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html)
  - [Customize Models](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_models.html)
  - [Customize Runtime Settings](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html)

## Overview of Benchmark and Model Zoo

Results and models are available in the [model zoo](docs/en/model_zoo.md).

  Components

        Backbones

        Heads

        Features

PointNet (CVPR'2017)

PointNet++ (NeurIPS'2017)

RegNet (CVPR'2020)

DGCNN (TOG'2019)

DLA (CVPR'2018)

MinkResNet (CVPR'2019)

MinkUNet (CVPR'2019)

Cylinder3D (CVPR'2021)

FreeAnchor (NeurIPS'2019)

Dynamic Voxelization (CoRL'2019)

  Architectures

        LiDAR-based 3D Object Detection

        Camera-based 3D Object Detection

        Multi-modal 3D Object Detection

        3D Semantic Segmentation

Outdoor

SECOND (Sensor'2018)

PointPillars (CVPR'2019)

SSN (ECCV'2020)

3DSSD (CVPR'2020)

SA-SSD (CVPR'2020)

PointRCNN (CVPR'2019)

Part-A2 (TPAMI'2020)

CenterPoint (CVPR'2021)

PV-RCNN (CVPR'2020)

CenterFormer (ECCV'2022)

Indoor

VoteNet (ICCV'2019)

H3DNet (ECCV'2020)

Group-Free-3D (ICCV'2021)

FCAF3D (ECCV'2022)

TR3D (ArXiv'2023)

Outdoor

ImVoxelNet (WACV'2022)

SMOKE (CVPRW'2020)

FCOS3D (ICCVW'2021)

PGD (CoRL'2021)

MonoFlex (CVPR'2021)

DETR3D (CoRL'2021)

PETR (ECCV'2022)

Indoor

ImVoxelNet (WACV'2022)

Outdoor

MVXNet (ICRA'2019)

BEVFusion (ICRA'2023)

Indoor

ImVoteNet (CVPR'2020)

Outdoor

MinkUNet (CVPR'2019)

SPVCNN (ECCV'2020)

Cylinder3D (CVPR'2021)

TPVFormer (CVPR'2023)

Indoor

PointNet++ (NeurIPS'2017)

PAConv (CVPR'2021)

DGCNN (TOG'2019)

|               | ResNet | VoVNet | Swin-T | PointNet++ | SECOND | DGCNN | RegNetX | DLA | MinkResNet | Cylinder3D | MinkUNet |
| :-----------: | :----: | :----: | :----: | :--------: | :----: | :---: | :-----: | :-: | :--------: | :--------: | :------: |
|    SECOND     |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
| PointPillars  |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✓    |  ✗  |     ✗      |     ✗      |    ✗     |
|  FreeAnchor   |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✓    |  ✗  |     ✗      |     ✗      |    ✗     |
|    VoteNet    |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    H3DNet     |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|     3DSSD     |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    Part-A2    |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    MVXNet     |   ✓    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|  CenterPoint  |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|      SSN      |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✓    |  ✗  |     ✗      |     ✗      |    ✗     |
|   ImVoteNet   |   ✓    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    FCOS3D     |   ✓    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|  PointNet++   |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
| Group-Free-3D |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|  ImVoxelNet   |   ✓    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    PAConv     |   ✗    |   ✗    |   ✗    |     ✓      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|     DGCNN     |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✓   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|     SMOKE     |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✓  |     ✗      |     ✗      |    ✗     |
|      PGD      |   ✓    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|   MonoFlex    |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✓  |     ✗      |     ✗      |    ✗     |
|    SA-SSD     |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|    FCAF3D     |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✓      |     ✗      |    ✗     |
|    PV-RCNN    |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|  Cylinder3D   |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✓      |    ✗     |
|   MinkUNet    |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✓     |
|    SPVCNN     |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✓     |
|   BEVFusion   |   ✗    |   ✗    |   ✓    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
| CenterFormer  |   ✗    |   ✗    |   ✗    |     ✗      |   ✓    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|     TR3D      |   ✗    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✓      |     ✗      |    ✗     |
|    DETR3D     |   ✓    |   ✓    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|     PETR      |   ✗    |   ✓    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |
|   TPVFormer   |   ✓    |   ✗    |   ✗    |     ✗      |   ✗    |   ✗   |    ✗    |  ✗  |     ✗      |     ✗      |    ✗     |

**Note:** All the about **500+ models, methods of 90+ papers** in 2D detection supported by [MMDetection](https://github.com/open-mmlab/mmdetection/blob/3.x/docs/en/model_zoo.md) can be trained or used in this codebase.

## FAQ

Please refer to [FAQ](docs/en/notes/faq.md) for frequently asked questions.

## Contributing

We appreciate all contributions to improve MMDetection3D. Please refer to [CONTRIBUTING.md](docs/en/notes/contribution_guides.md) for the contributing guideline.

## Acknowledgement

MMDetection3D is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new 3D detectors.

## Citation

If you find this project useful in your research, please consider cite:

```latex
@misc{mmdet3d2020,
    title={{MMDetection3D: OpenMMLab} next-generation platform for general {3D} object detection},
    author={MMDetection3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmdetection3d}},
    year={2020}
}
```

## License

This project is released under the [Apache 2.0 license](LICENSE).

## Projects in OpenMMLab

- [MMEngine](https://github.com/open-mmlab/mmengine): OpenMMLab foundational library for training deep learning models.
- [MMCV](https://github.com/open-mmlab/mmcv): OpenMMLab foundational library for computer vision.
- [MMEval](https://github.com/open-mmlab/mmeval): A unified evaluation library for multiple machine learning libraries.
- [MIM](https://github.com/open-mmlab/mim): MIM installs OpenMMLab packages.
- [MMPreTrain](https://github.com/open-mmlab/mmpretrain): OpenMMLab pre-training toolbox and benchmark.
- [MMDetection](https://github.com/open-mmlab/mmdetection): OpenMMLab detection toolbox and benchmark.
- [MMDetection3D](https://github.com/open-mmlab/mmdetection3d): OpenMMLab's next-generation platform for general 3D object detection.
- [MMRotate](https://github.com/open-mmlab/mmrotate): OpenMMLab rotated object detection toolbox and benchmark.
- [MMYOLO](https://github.com/open-mmlab/mmyolo): OpenMMLab YOLO series toolbox and benchmark.
- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation): OpenMMLab semantic segmentation toolbox and benchmark.
- [MMOCR](https://github.com/open-mmlab/mmocr): OpenMMLab text detection, recognition, and understanding toolbox.
- [MMPose](https://github.com/open-mmlab/mmpose): OpenMMLab pose estimation toolbox and benchmark.
- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d): OpenMMLab 3D human parametric model toolbox and benchmark.
- [MMSelfSup](https://github.com/open-mmlab/mmselfsup): OpenMMLab self-supervised learning toolbox and benchmark.
- [MMRazor](https://github.com/open-mmlab/mmrazor): OpenMMLab model compression toolbox and benchmark.
- [MMFewShot](https://github.com/open-mmlab/mmfewshot): OpenMMLab fewshot learning toolbox and benchmark.
- [MMAction2](https://github.com/open-mmlab/mmaction2): OpenMMLab's next-generation action understanding toolbox and benchmark.
- [MMTracking](https://github.com/open-mmlab/mmtracking): OpenMMLab video perception toolbox and benchmark.
- [MMFlow](https://github.com/open-mmlab/mmflow): OpenMMLab optical flow toolbox and benchmark.
- [MMagic](https://github.com/open-mmlab/mmagic): Open**MM**Lab **A**dvanced, **G**enerative and **I**ntelligent **C**reation toolbox.
- [MMGeneration](https://github.com/open-mmlab/mmgeneration): OpenMMLab image and video generative models toolbox.
- [MMDeploy](https://github.com/open-mmlab/mmdeploy): OpenMMLab model deployment framework.


Algo-trading was 😵💫, we made it 🤩

# Jesse
[![PyPI](https://img.shields.io/pypi/v/jesse)](https://pypi.org/project/jesse)
[![Downloads](https://pepy.tech/badge/jesse)](https://pepy.tech/project/jesse)
[![Docker Pulls](https://img.shields.io/docker/pulls/salehmir/jesse)](https://hub.docker.com/r/salehmir/jesse)
[![GitHub](https://img.shields.io/github/license/jesse-ai/jesse)](https://github.com/jesse-ai/jesse)
[![coverage](https://codecov.io/gh/jesse-ai/jesse/graph/badge.svg)](https://codecov.io/gh/jesse-ai/jesse)

---

Jesse is an advanced crypto trading framework that aims to **simplify** **researching** and defining **YOUR OWN trading strategies**.

## Why Jesse?
In short, Jesse is more **accurate** than other solutions, and way more **simple**.
In fact, it is so simple that in case you already know Python, you can get started today, in **matter of minutes**, instead of **weeks and months**.

**[Here](https://docs.jesse.trade)** you can read more about Jesse's features and why you should use it.

## Getting Started
Head over to the "getting started" section of the [documentation](https://docs.jesse.trade/docs/getting-started). The
documentation is **short yet very informative**.

## Resources

- [⚡ Website](https://jesse.trade)
- [🎓 Documentation](https://docs.jesse.trade)
- [🛟 Help center](https://jesse.trade/help)
- [🎥 Youtube channel (screencast tutorials)](https://jesse.trade/youtube)
- [💬 Discord community](https://jesse.trade/discord)

## Screenshots

Check out Jesse's [Youtube channel](https://jesse.trade/youtube) for tutorials that go through example strategies step by step.

Here are a few screenshots just to get you excited:

**Backtest results:**

![image](assets/screenshots/quantstats.png)

**Example strategy code:**

![image](assets/screenshots/strategy.jpg)

**Live trading (requires [live plugin](https://docs.jesse.trade/docs/livetrade.html)):**

![image](assets/screenshots/live-mode.jpg)

## What's next?

You can see the project's **[roadmap here](https://docs.jesse.trade/docs/roadmap.html)**. **Subscribe** to our mailing list at [jesse.trade](https://jesse.trade) to get the good stuff as soon they're released. Don't worry, We won't send you spam—Pinky promise.

## How to contribute

Thank you for your interest in contributing to the project. The best way to contribute is by **participating in the community** and **helping other users**.

You can also contribute by submitting **bug reports** and **feature requests** or writing code (submitting PRs) which can be incorporated into Jesse itself.

In that case, here's what you need to know:

- Before starting to work on a PR, please **reach out** to make sure it **aligns** with the **project's roadmap** and **vision**.
- If your PR makes changes to the source code, please make sure to **add unit tests**. If you're not sure how to do that, just check out some of the already existing [tests](https://github.com/jesse-ai/jesse/tree/master/tests).

First, you need to install Jesse from the repository instead of PyPi:

```sh
# first, make sure that the PyPi version is not installed
pip uninstall jesse

# now install Jesse from the repository
git clone https://github.com/jesse-ai/jesse.git
cd jesse
pip install -e .
```

Now every change you make to the code will be affected immediately.

After every change, **make sure** your changes did not **break** any functionality by **running tests**:
```
pytest
```

## Disclaimer
This software is for educational purposes only. USE THE SOFTWARE AT **YOUR OWN RISK**. THE AUTHORS AND ALL AFFILIATES ASSUME **NO RESPONSIBILITY FOR YOUR TRADING RESULTS**. **Do not risk money that you are afraid to lose**. There might be **bugs** in the code - this software DOES NOT come with **ANY warranty**.


#######
PyVista
#######

    3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK)

.. image:: https://github.com/pyvista/pyvista/raw/main/doc/source/_static/pyvista_banner_small.png
   :target: https://docs.pyvista.org/examples/index.html
   :alt: pyvista

PyVista is:

* *Pythonic VTK*: a high-level API to the `Visualization Toolkit`_ (VTK)
* mesh data structures and filtering methods for spatial datasets
* 3D plotting made simple and built for large/complex data geometries

.. _Visualization Toolkit: https://vtk.org

.. image:: https://github.com/pyvista/pyvista/raw/main/assets/pyvista_ipython_demo.gif
   :alt: pyvista ipython demo

PyVista is a helper module for the Visualization Toolkit (VTK) that wraps the VTK library
through NumPy and direct array access through a variety of methods and classes.
This package provides a Pythonic, well-documented interface exposing
VTK's powerful visualization backend to facilitate rapid prototyping, analysis,
and visual integration of spatially referenced datasets.

This module can be used for scientific plotting for presentations and research
papers as well as a supporting module for other mesh 3D rendering dependent
Python modules; see Connections for a list of projects that leverage
PyVista.

.. |tweet| image:: https://img.shields.io/twitter/url.svg?style=social&url=http%3A%2F%2Fshields.io
   :target: https://twitter.com/intent/tweet?text=Check%20out%20this%20project%20for%203D%20visualization%20in%20Python&url=https://github.com/pyvista/pyvista&hashtags=3D,visualization,Python,vtk,mesh,plotting,PyVista

Share this project on X: |tweet|

PyVista is a NumFOCUS affiliated project

.. image:: https://raw.githubusercontent.com/numfocus/templates/master/images/numfocus-logo.png
   :target: https://numfocus.org/sponsored-projects/affiliated-projects
   :alt: NumFOCUS affiliated projects
   :height: 60px

Status badges
=============

.. |zenodo| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.8415866.svg
   :target: https://zenodo.org/records/8415866

.. |joss| image:: http://joss.theoj.org/papers/10.21105/joss.01450/status.svg
   :target: https://doi.org/10.21105/joss.01450

.. |pypi| image:: https://img.shields.io/pypi/v/pyvista.svg?logo=python&logoColor=white
   :target: https://pypi.org/project/pyvista/

.. |conda| image:: https://img.shields.io/conda/vn/conda-forge/pyvista.svg?logo=conda-forge&logoColor=white
   :target: https://anaconda.org/conda-forge/pyvista

.. |GH-CI| image:: https://github.com/pyvista/pyvista/actions/workflows/testing-and-deployment.yml/badge.svg
   :target: https://github.com/pyvista/pyvista/actions/workflows/testing-and-deployment.yml

.. |codecov| image:: https://codecov.io/gh/pyvista/pyvista/branch/main/graph/badge.svg
   :target: https://app.codecov.io/gh/pyvista/pyvista

.. |codacy| image:: https://app.codacy.com/project/badge/Grade/779ac6aed37548839384acfc0c1aab44
   :target: https://app.codacy.com/gh/pyvista/pyvista/dashboard

.. |MIT| image:: https://img.shields.io/badge/License-MIT-yellow.svg
   :target: https://opensource.org/license/mit/

.. |slack| image:: https://img.shields.io/badge/Slack-pyvista-green.svg?logo=slack
   :target: https://communityinviter.com/apps/pyvista/pyvista

.. |PyPIact| image:: https://img.shields.io/pypi/dm/pyvista.svg?label=PyPI%20downloads
   :target: https://pypi.org/project/pyvista/

.. |condaact| image:: https://img.shields.io/conda/dn/conda-forge/pyvista.svg?label=Conda%20downloads
   :target: https://anaconda.org/conda-forge/pyvista

.. |discuss| image:: https://img.shields.io/badge/GitHub-Discussions-green?logo=github
   :target: https://github.com/pyvista/pyvista/discussions

.. |prettier| image:: https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat
  :target: https://github.com/prettier/prettier
  :alt: prettier

.. |python| image:: https://img.shields.io/badge/python-3.8+-blue.svg
   :target: https://www.python.org/downloads/

.. |NumFOCUS Affiliated| image:: https://img.shields.io/badge/affiliated-NumFOCUS-orange.svg?style=flat&colorA=E1523D&colorB=007D8A
   :target: https://numfocus.org/sponsored-projects/affiliated-projects

.. |pre-commit.ci status| image:: https://results.pre-commit.ci/badge/github/pyvista/pyvista/main.svg
   :target: https://results.pre-commit.ci/latest/github/pyvista/pyvista/main

.. |Ruff| image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json
   :target: https://github.com/astral-sh/ruff
   :alt: Ruff

.. |Awesome Scientific Computing| image:: https://awesome.re/mentioned-badge.svg
   :target: https://github.com/nschloe/awesome-scientific-computing

+----------------------+------------------------------------------------+
| Deployment           | |pypi| |conda|                                 |
+----------------------+------------------------------------------------+
| Build Status         | |GH-CI| |python| |pre-commit.ci status|        |
+----------------------+------------------------------------------------+
| Metrics              | |codacy| |codecov|                             |
+----------------------+------------------------------------------------+
| Activity             | |PyPIact| |condaact|                           |
+----------------------+------------------------------------------------+
| Citation             | |joss| |zenodo|                                |
+----------------------+------------------------------------------------+
| License              | |MIT|                                          |
+----------------------+------------------------------------------------+
| Community            | |slack| |discuss|                              |
+----------------------+------------------------------------------------+
| Formatter            | |prettier|                                     |
+----------------------+------------------------------------------------+
| Linter               | |Ruff|                                         |
+----------------------+------------------------------------------------+
| Affiliated           | |NumFOCUS Affiliated|                          |
+----------------------+------------------------------------------------+
| Mentioned            | |Awesome Scientific Computing|                 |
+----------------------+------------------------------------------------+

Highlights
==========

.. |binder| image:: https://static.mybinder.org/badge_logo.svg
   :target: https://mybinder.org/v2/gh/pyvista/pyvista-examples/master
   :alt: Launch on Binder

Head over to the `Quick Examples`_ page in the docs to explore our gallery of
examples showcasing what PyVista can do. Want to test-drive PyVista?
All of the examples from the gallery are live on MyBinder for you to test
drive without installing anything locally: |binder|

.. _Quick Examples: http://docs.pyvista.org/examples/index.html

Overview of Features
--------------------

* Extensive gallery of examples (see `Quick Examples`_)
* Interactive plotting in Jupyter Notebooks with server-side and client-side
  rendering with `trame`_.
* Filtering/plotting tools built for interactivity (see `Widgets`_)
* Direct access to mesh analysis and transformation routines (see Filters_)
* Intuitive plotting routines with ``matplotlib`` similar syntax (see Plotting_)
* Import meshes from many common formats (use ``pyvista.read()``). Support for all formats handled by `meshio`_ is built-in.
* Export meshes as VTK, STL, OBJ, or PLY (``mesh.save()``) file types or any formats supported by meshio_ (``pyvista.save_meshio()``)

.. _trame: https://github.com/Kitware/trame
.. _Widgets: https://docs.pyvista.org/api/plotting/index.html#widget-api
.. _Filters: https://docs.pyvista.org/api/core/filters.html
.. _Plotting: https://docs.pyvista.org/api/plotting/index.html
.. _meshio: https://github.com/nschloe/meshio

Documentation
=============

Refer to the `documentation <http://docs.pyvista.org/>`_ for detailed
installation and usage details.

For general questions about the project, its applications, or about software
usage, please create a discussion in `pyvista/discussions`_
where the community can collectively address your questions. You are also
welcome to join us on Slack_.

.. _pyvista/discussions: https://github.com/pyvista/pyvista/discussions
.. _Slack: https://communityinviter.com/apps/pyvista/pyvista

Installation
============

PyVista can be installed from `PyPI <https://pypi.org/project/pyvista/>`_
using ``pip`` on Python >= 3.8::

    pip install pyvista

You can also visit `PyPI <https://pypi.org/project/pyvista/>`_,
`Anaconda <https://anaconda.org/conda-forge/pyvista>`_, or
`GitHub <https://github.com/pyvista/pyvista>`_ to download the source.

See the `Installation <http://docs.pyvista.org/getting-started/installation.html#install-ref.>`_
for more details regarding optional dependencies or if the installation through pip doesn't work out.

Connections
===========

PyVista is a powerful tool that researchers can harness to create compelling,
integrated visualizations of large datasets in an intuitive, Pythonic manner.

Learn more about how PyVista is used across science and engineering disciplines
by a diverse community of users on our `Connections page`_.

.. _Connections page: https://docs.pyvista.org/getting-started/connections.html

Authors
=======

.. |contrib.rocks| image:: https://contrib.rocks/image?repo=pyvista/pyvista
   :target: https://github.com/pyvista/pyvista/graphs/contributors
   :alt: contrib.rocks

Please take a look at the `contributors page`_ and the active `list of authors`_
to learn more about the developers of PyVista.

|contrib.rocks|

Made with `contrib rocks`_.

.. _contributors page: https://github.com/pyvista/pyvista/graphs/contributors/
.. _list of authors: https://docs.pyvista.org/getting-started/authors.html#authors
.. _contrib rocks: https://contrib.rocks

Contributing
============

.. |Contributor Covenant| image:: https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg
   :target: CODE_OF_CONDUCT.md

.. |codetriage| image:: https://www.codetriage.com/pyvista/pyvista/badges/users.svg
   :target: https://www.codetriage.com/pyvista/pyvista
   :alt: Code Triage

.. |Open in GitHub Codespaces| image:: https://github.com/codespaces/badge.svg
   :target: https://codespaces.new/pyvista/pyvista
   :alt: Open in GitHub Codespaces

|Contributor Covenant|
|codetriage|
|Open in GitHub Codespaces|

We absolutely welcome contributions and we hope that our `Contributing Guide`_
will facilitate your ability to make PyVista better. PyVista is mostly
maintained on a volunteer basis and thus we need to foster a community that can
support user questions and develop new features to make this software a useful
tool for all users while encouraging every member of the community to share
their ideas. To learn more about contributing to PyVista, please see the
`Contributing Guide`_ and our `Code of Conduct`_.

.. _Contributing Guide: https://github.com/pyvista/pyvista/blob/main/CONTRIBUTING.rst
.. _Code of Conduct: https://github.com/pyvista/pyvista/blob/main/CODE_OF_CONDUCT.md

Citing PyVista
==============

There is a `paper about PyVista <https://doi.org/10.21105/joss.01450>`_.

If you are using PyVista in your scientific research, please help our scientific
visibility by citing our work.

    Sullivan and Kaszynski, (2019). PyVista: 3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK). Journal of Open Source Software, 4(37), 1450, https://doi.org/10.21105/joss.01450

BibTex:

.. code::

    @article{sullivan2019pyvista,
      doi = {10.21105/joss.01450},
      url = {https://doi.org/10.21105/joss.01450},
      year = {2019},
      month = {May},
      publisher = {The Open Journal},
      volume = {4},
      number = {37},
      pages = {1450},
      author = {Bane Sullivan and Alexander Kaszynski},
      title = {{PyVista}: {3D} plotting and mesh analysis through a streamlined interface for the {Visualization Toolkit} ({VTK})},
      journal = {Journal of Open Source Software}
    }

Professional Support
====================

While PyVista is an Open Source project with a big community, you might be looking for professional support.
This section aims to list companies with VTK/PyVista expertise who can help you with your software project.

+---------------+-----------------------------------------+
| Company Name  | Kitware Inc.                            |
+---------------+-----------------------------------------+
| Description   | Kitware is dedicated to build solutions |
|               | for our customers based on our          |
|               | well-established open source platforms. |
+---------------+-----------------------------------------+
| Expertise     | CMake, VTK, PyVista, ParaView, Trame    |
+---------------+-----------------------------------------+
| Contact       | https://www.kitware.com/contact/        |
+---------------+-----------------------------------------+


# ursina    ʕ •ᴥ•ʔゝ□
An easy to use game engine/framework for python.

![Banner](/docs/made_with_ursina.jpg)

[![Trailer](/docs/ursina_trailer_preview.webp)](https://youtu.be/j71j88oCTNo)

## Getting Started
1) Install Python 3.6 or newer. https://www.python.org/downloads/
2) Open cmd/terminal and type:

```
pip install ursina
```

If you want to install the newest version from git, you can install like this:

```
pip install git+https://github.com/pokepetter/ursina.git
```

If you want to easily edit the source, it's recommended to clone the git
repo and install as develop like this. Make sure you have git installed. https://git-scm.com/

```
git clone https://github.com/pokepetter/ursina.git
python setup.py develop
```

Also install any of the optional dependencies you want from the list below,
or install them all with:

```
pip install ursina[extras]
```

On some systems you might have to use pip3 instead of pip in order to use Python 3 and not the old Python 2.

## Dependencies
  * python 3.6+
  * panda3d
  * pillow, for texture manipulation
  * psd-tools, for converting .psd files
  * blender, for converting .blend files
  * pyperclip, for copy/pasting

## Examples
``` python
from ursina import *            # this will import everything we need from ursina with just one line.

app = Ursina()
ground = Entity(
    model = 'cube',
    color = color.magenta,
    z = -.1,
    y = -3,
    origin = (0, .5),
    scale = (50, 1, 10),
    collider = 'box',
    )

app.run()                       # opens a window and starts the game.
```

* [Minecraft Clone](/samples/minecraft_clone.py)

* [Platformer Game](/samples/platformer.py)

## How do I make a game?
Ursina games are made by writing Python code. You can use any text editor you want, but personally I like to use Atom.
1) Create an empty .py file called `ursina_game.py`
2) Copy this text into your new file:
``` python
from ursina import *           # this will import everything we need from ursina with just one line.

app = Ursina()

player = Entity(
    model = 'cube' ,           # finds a 3d model by name
    color = color.orange,
    scale_y = 2
    )

def update():                  # update gets automatically called by the engine.
    player.x += held_keys['d'] * .1
    player.x -= held_keys['a'] * .1

app.run()                     # opens a window and starts the game.
```

3) Type this in the terminal to start the game:

       python ursina_game.py
   If you use Atom, I recommend installing the package atom-python-run to run your scripts with the press of a button.

4) You can now move the orange box around with 'a' and 'd'!

   To close the window, you can by default, press shift+q or press the red x. to disable this, write `window.exit_button.enabled = False` somewhere in your code.

## Project Structure
```
## Project Structure

📁docs
    📃index.txt
    📃documentation.txt
    📃inventory_tutorial.txt
    ...
        # text files for the website. gets turned into .html files with sswg.

    📃cheat_sheet.html
        # auto generated documentation made with documentation_generator.py.

    📃tutorial_generator.py
        # turns specific .py files into .txt files, which can then be turned into .html by sswg.
        # this extracts the comments from the source files into description for that step and the code after into code blocks.
        # see platformer_tutorial.py for an example.

📁samples               # small example games.

📁ursina                # the actual ursina module.
    📁audio                 # built-in audio clips.
    📁editor                # the 3d level editor for ursina.
    📁fonts                 # built-in fonts.
    📁models                # .blend files, source files, for built-in 3d models.
        📁procedural            # classes for generating 3d models, like Cylinder, Quad and Terrain.
    📁models_compressed     # .blend files converted to .ursinamesh.
    📁prefabs               # higher level classes like Draggable, Slider, Sprite, etc.

    📃__init__.py
    📃application.py
    📃audio.py
    ...
        # ursina base modules, like code for Entity, input_handler, Text, window and so on.

```