[//]: # (Logo:)

PySR searches for symbolic expressions which optimize a particular objective.

https://github.com/MilesCranmer/PySR/assets/7593028/c8511a49-b408-488f-8f18-b1749078268f

# PySR: High-Performance Symbolic Regression in Python and Julia

| **Docs** | **Forums** | **Paper** | **colab demo** |
|:---:|:---:|:---:|:---:|
|[![Documentation](https://github.com/MilesCranmer/PySR/actions/workflows/docs.yml/badge.svg)](https://astroautomata.com/PySR/)|[![Discussions](https://img.shields.io/badge/discussions-github-informational)](https://github.com/MilesCranmer/PySR/discussions)|[![Paper](https://img.shields.io/badge/arXiv-2305.01582-b31b1b)](https://arxiv.org/abs/2305.01582)|[![Colab](https://img.shields.io/badge/colab-notebook-yellow)](https://colab.research.google.com/github/MilesCranmer/PySR/blob/master/examples/pysr_demo.ipynb)|

| **pip** | **conda** | **Stats** |
| :---: | :---: | :---: |
|[![PyPI version](https://badge.fury.io/py/pysr.svg)](https://badge.fury.io/py/pysr)|[![Conda Version](https://img.shields.io/conda/vn/conda-forge/pysr.svg)](https://anaconda.org/conda-forge/pysr)|
pip: [![Downloads](https://static.pepy.tech/badge/pysr)](https://pypi.org/project/pysr/)
conda: [![Anaconda-Server Badge](https://anaconda.org/conda-forge/pysr/badges/downloads.svg)](https://anaconda.org/conda-forge/pysr)
|

If you find PySR useful, please cite the paper [arXiv:2305.01582](https://arxiv.org/abs/2305.01582).
If you've finished a project with PySR, please submit a PR to showcase your work on the [research showcase page](https://astroautomata.com/PySR/papers)!

**Contents**:

- [Why PySR?](#why-pysr)
- [Installation](#installation)
- [Quickstart](#quickstart)
- [→ Documentation](https://astroautomata.com/PySR)
- [Contributors](#contributors-)

### Test status

| **Linux** | **Windows** | **macOS** |
|---|---|---|
|[![Linux](https://github.com/MilesCranmer/PySR/actions/workflows/CI.yml/badge.svg)](https://github.com/MilesCranmer/PySR/actions/workflows/CI.yml)|[![Windows](https://github.com/MilesCranmer/PySR/actions/workflows/CI_Windows.yml/badge.svg)](https://github.com/MilesCranmer/PySR/actions/workflows/CI_Windows.yml)|[![macOS](https://github.com/MilesCranmer/PySR/actions/workflows/CI_mac.yml/badge.svg)](https://github.com/MilesCranmer/PySR/actions/workflows/CI_mac.yml)|
| **Docker** | **Conda** | **Coverage** |
|[![Docker](https://github.com/MilesCranmer/PySR/actions/workflows/CI_docker.yml/badge.svg)](https://github.com/MilesCranmer/PySR/actions/workflows/CI_docker.yml)|[![conda-forge](https://github.com/MilesCranmer/PySR/actions/workflows/CI_conda_forge.yml/badge.svg)](https://github.com/MilesCranmer/PySR/actions/workflows/CI_conda_forge.yml)|[![Coverage Status](https://coveralls.io/repos/github/MilesCranmer/PySR/badge.svg?branch=master&service=github)](https://coveralls.io/github/MilesCranmer/PySR)|

## Why PySR?

PySR is an open-source tool for *Symbolic Regression*: a machine learning
task where the goal is to find an interpretable symbolic expression that optimizes some objective.

Over a period of several years, PySR has been engineered from the ground up
to be (1) as high-performance as possible,
(2) as configurable as possible, and (3) easy to use.
PySR is developed alongside the Julia library [SymbolicRegression.jl](https://github.com/MilesCranmer/SymbolicRegression.jl),
which forms the powerful search engine of PySR.
The details of these algorithms are described in the [PySR paper](https://arxiv.org/abs/2305.01582).

Symbolic regression works best on low-dimensional datasets, but
one can also extend these approaches to higher-dimensional
spaces by using "*Symbolic Distillation*" of Neural Networks, as explained in
[2006.11287](https://arxiv.org/abs/2006.11287), where we apply
it to N-body problems. Here, one essentially uses
symbolic regression to convert a neural net
to an analytic equation. Thus, these tools simultaneously present
an explicit and powerful way to interpret deep neural networks.

## Installation

### Pip

You can install PySR with pip:

```bash
pip install pysr
```

Julia dependencies will be installed at first import.

### Conda

Similarly, with conda:

```bash
conda install -c conda-forge pysr
```

### Dockerfile

You can also use the `Dockerfile` to install PySR in a docker container

1. Clone this repo.
2. Within the repo's directory, build the docker container:
```bash
docker build -t pysr .
```
3. You can then start the container with an IPython execution with:
```bash
docker run -it --rm pysr ipython
```

For more details, see the [docker section](#docker).

---

### Troubleshooting

One issue you might run into can result in a hard crash at import with
a message like "`GLIBCXX_...` not found". This is due to another one of the Python dependencies
loading an incorrect `libstdc++` library. To fix this, you should modify your
`LD_LIBRARY_PATH` variable to reference the Julia libraries. For example, if the Julia
version of `libstdc++.so` is located in `$HOME/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/`
(which likely differs on your system!), you could add:

```
export LD_LIBRARY_PATH=$HOME/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/:$LD_LIBRARY_PATH
```

to your `.bashrc` or `.zshrc` file.

## Quickstart

You might wish to try the interactive tutorial [here](https://colab.research.google.com/github/MilesCranmer/PySR/blob/master/examples/pysr_demo.ipynb), which uses the notebook in `examples/pysr_demo.ipynb`.

In practice, I highly recommend using IPython rather than Jupyter, as the printing is much nicer.
Below is a quick demo here which you can paste into a Python runtime.
First, let's import numpy to generate some test data:

```python
import numpy as np

X = 2 * np.random.randn(100, 5)
y = 2.5382 * np.cos(X[:, 3]) + X[:, 0] ** 2 - 0.5
```

We have created a dataset with 100 datapoints, with 5 features each.
The relation we wish to model is $2.5382 \cos(x_3) + x_0^2 - 0.5$.

Now, let's create a PySR model and train it.
PySR's main interface is in the style of scikit-learn:

```python
from pysr import PySRRegressor

model = PySRRegressor(
    niterations=40,  # < Increase me for better results
    binary_operators=["+", "*"],
    unary_operators=[
        "cos",
        "exp",
        "sin",
        "inv(x) = 1/x",
        # ^ Custom operator (julia syntax)
    ],
    extra_sympy_mappings={"inv": lambda x: 1 / x},
    # ^ Define operator for SymPy as well
    elementwise_loss="loss(prediction, target) = (prediction - target)^2",
    # ^ Custom loss function (julia syntax)
)
```

This will set up the model for 40 iterations of the search code, which contains hundreds of thousands of mutations and equation evaluations.

Let's train this model on our dataset:

```python
model.fit(X, y)
```

Internally, this launches a Julia process which will do a multithreaded search for equations to fit the dataset.

Equations will be printed during training, and once you are satisfied, you may
quit early by hitting 'q' and then \<enter\>.

After the model has been fit, you can run `model.predict(X)`
to see the predictions on a given dataset using the automatically-selected expression,
or, for example, `model.predict(X, 3)` to see the predictions of the 3rd equation.

You may run:

```python
print(model)
```

to print the learned equations:

```python
PySRRegressor.equations_ = [
	   pick     score                                           equation       loss  complexity
	0        0.000000                                          4.4324794  42.354317           1
	1        1.255691                                          (x0 * x0)   3.437307           3
	2        0.011629                          ((x0 * x0) + -0.28087974)   3.358285           5
	3        0.897855                              ((x0 * x0) + cos(x3))   1.368308           6
	4        0.857018                ((x0 * x0) + (cos(x3) * 2.4566472))   0.246483           8
	5  >>>>       inf  (((cos(x3) + -0.19699033) * 2.5382123) + (x0 *...   0.000000          10
]
```

This arrow in the `pick` column indicates which equation is currently selected by your
`model_selection` strategy for prediction.
(You may change `model_selection` after `.fit(X, y)` as well.)

`model.equations_` is a pandas DataFrame containing all equations, including callable format
(`lambda_format`),
SymPy format (`sympy_format` - which you can also get with `model.sympy()`), and even JAX and PyTorch format
(both of which are differentiable - which you can get with `model.jax()` and `model.pytorch()`).

Note that `PySRRegressor` stores the state of the last search, and will restart from where you left off the next time you call `.fit()`, assuming you have set `warm_start=True`.
This will cause problems if significant changes are made to the search parameters (like changing the operators). You can run `model.reset()` to reset the state.

You will notice that PySR will save two files:
`hall_of_fame...csv` and `hall_of_fame...pkl`.
The csv file is a list of equations and their losses, and the pkl file is a saved state of the model.
You may load the model from the `pkl` file with:

```python
model = PySRRegressor.from_file("hall_of_fame.2022-08-10_100832.281.pkl")
```

There are several other useful features such as denoising (e.g., `denoise=True`),
feature selection (e.g., `select_k_features=3`).
For examples of these and other features, see the [examples page](https://astroautomata.com/PySR/examples).
For a detailed look at more options, see the [options page](https://astroautomata.com/PySR/options).
You can also see the full API at [this page](https://astroautomata.com/PySR/api).
There are also tips for tuning PySR on [this page](https://astroautomata.com/PySR/tuning).

### Detailed Example

The following code makes use of as many PySR features as possible.
Note that is just a demonstration of features and you should not use this example as-is.
For details on what each parameter does, check out the [API page](https://astroautomata.com/PySR/api/).

```python
model = PySRRegressor(
    procs=4,
    populations=8,
    # ^ 2 populations per core, so one is always running.
    population_size=50,
    # ^ Slightly larger populations, for greater diversity.
    ncycles_per_iteration=500,
    # ^ Generations between migrations.
    niterations=10000000,  # Run forever
    early_stop_condition=(
        "stop_if(loss, complexity) = loss < 1e-6 && complexity < 10"
        # Stop early if we find a good and simple equation
    ),
    timeout_in_seconds=60 * 60 * 24,
    # ^ Alternatively, stop after 24 hours have passed.
    maxsize=50,
    # ^ Allow greater complexity.
    maxdepth=10,
    # ^ But, avoid deep nesting.
    binary_operators=["*", "+", "-", "/"],
    unary_operators=["square", "cube", "exp", "cos2(x)=cos(x)^2"],
    constraints={
        "/": (-1, 9),
        "square": 9,
        "cube": 9,
        "exp": 9,
    },
    # ^ Limit the complexity within each argument.
    # "inv": (-1, 9) states that the numerator has no constraint,
    # but the denominator has a max complexity of 9.
    # "exp": 9 simply states that `exp` can only have
    # an expression of complexity 9 as input.
    nested_constraints={
        "square": {"square": 1, "cube": 1, "exp": 0},
        "cube": {"square": 1, "cube": 1, "exp": 0},
        "exp": {"square": 1, "cube": 1, "exp": 0},
    },
    # ^ Nesting constraints on operators. For example,
    # "square(exp(x))" is not allowed, since "square": {"exp": 0}.
    complexity_of_operators={"/": 2, "exp": 3},
    # ^ Custom complexity of particular operators.
    complexity_of_constants=2,
    # ^ Punish constants more than variables
    select_k_features=4,
    # ^ Train on only the 4 most important features
    progress=True,
    # ^ Can set to false if printing to a file.
    weight_randomize=0.1,
    # ^ Randomize the tree much more frequently
    cluster_manager=None,
    # ^ Can be set to, e.g., "slurm", to run a slurm
    # cluster. Just launch one script from the head node.
    precision=64,
    # ^ Higher precision calculations.
    warm_start=True,
    # ^ Start from where left off.
    turbo=True,
    # ^ Faster evaluation (experimental)
    extra_sympy_mappings={"cos2": lambda x: sympy.cos(x)**2},
    # extra_torch_mappings={sympy.cos: torch.cos},
    # ^ Not needed as cos already defined, but this
    # is how you define custom torch operators.
    # extra_jax_mappings={sympy.cos: "jnp.cos"},
    # ^ For JAX, one passes a string.
)
```

### Docker

You can also test out PySR in Docker, without
installing it locally, by running the following command in
the root directory of this repo:

```bash
docker build -t pysr .
```

This builds an image called `pysr` for your system's architecture,
which also contains IPython. You can select a specific version
of Python and Julia with:

```bash
docker build -t pysr --build-arg JLVERSION=1.10.0 --build-arg PYVERSION=3.11.6 .
```

You can then run with this dockerfile using:

```bash
docker run -it --rm -v "$PWD:/data" pysr ipython
```

which will link the current directory to the container's `/data` directory
and then launch ipython.

If you have issues building for your system's architecture,
you can emulate another architecture by including `--platform linux/amd64`,
before the `build` and `run` commands.

### Contributors ✨

We are eager to welcome new contributors! Check out our contributors [guide](https://github.com/MilesCranmer/PySR/blob/master/CONTRIBUTORS.md) for tips 🚀.
If you have an idea for a new feature, don't hesitate to share it on the [issues](https://github.com/MilesCranmer/PySR/issues) or [discussions](https://github.com/MilesCranmer/PySR/discussions) page.

Mark Kittisopikul
💻 💡 🚇 📦 📣 👀 🔧 ⚠

T Coxon
🐛 💻 🔌 💡 🚇 🚧 👀 🔧 ⚠ 📓

Dhananjay Ashok
💻 🌍 💡 🚧 ⚠

Johan Blåbäck
🐛 💻 💡 🚧 📣 👀 ⚠ 📓

JuliusMartensen
🐛 💻 📖 🔌 💡 🚇 🚧 📦 📣 👀 🔧 📓

ngam
💻 🚇 📦 👀 🔧 ⚠

Christopher Rowley
💻 💡 🚇 📦 👀

Kaze Wong
🐛 💻 💡 🚇 🚧 📣 👀 🔬 📓

Christopher Rackauckas
🐛 💻 🔌 💡 🚇 📣 👀 🔬 🔧 ⚠ 📓

Patrick Kidger
🐛 💻 📖 🔌 💡 🚧 📣 👀 🔬 🔧 ⚠ 📓

Okon Samuel
🐛 💻 📖 🚧 💡 🚇 👀 ⚠ 📓

William Booth-Clibborn
💻 🌍 📖 📓 🚧 👀 🔧 ⚠

Pablo Lemos
🐛 💡 📣 👀 🔬 📓

Jerry Ling
🐛 💻 📖 🌍 💡 📣 👀 📓

Charles Fox
🐛 💻 💡 🚧 📣 👀 🔬 📓

Johann Brehmer
💻 📖 💡 📣 👀 🔬 ⚠ 📓

Marius Millea
💻 💡 📣 👀 📓

Coba
🐛 💻 💡 👀 📓

foxtran
💻 💡 🚧 🔧 📓

Shah Mahdi Hasan
🐛 💻 👀 📓

Pietro Monticone
🐛 📖 💡

Mateusz Kubica
📖 💡

Jay Wadekar
🐛 💡 📣 🔬

Anthony Blaom, PhD
🚇 💡 👀

Jgmedina95
🐛 💡 👀

Michael Abbott
💻 💡 👀 🔧

Oscar Smith
💻 💡

Eric Hanson
💡 📣 📓

Henrique Becker
💻 💡 👀

qwertyjl
🐛 📖 💡 📓

Rik Huijzer
💡 🚇

Hongyu Wang
💡 📣 🔬

Zehao Jin
🔬 📣

Tanner Mengel
🔬 📣

Arthur Grundner
🔬 📣

sjwetzel
🔬 📣 📓

Saurav Maheshkar
🔧


# Managarm: Distribution Build Scripts

![Sanity Checks](https://github.com/managarm/bootstrap-managarm/workflows/Sanity%20Checks/badge.svg)

This repository contains patches and build script to build a [Managarm](https://github.com/managarm/managarm) kernel and userspace.

## Building a Managarm Distribution from Source

For build instructions, refer to the [Managarm Handbook](https://docs.managarm.org/handbook/building/index.html).


# ![Aries RFCs](collateral/aries-rfcs-logo.png)

This repo holds Request for Comment (RFCs) for the Aries project. They describe important
topics ([not minor details](contributing.md#do-you-need-an-RFC)) that we want to
standardize across the Aries ecosystem.

If you are here to learn about Aries, we recommend you use the [the RFC Index](index.md) for a current listing of all RFCs and their statuses.

There are 2 types of Aries RFCs:

* RFCs that describe individual features (in the `./features` folder)
* RFCs that explain concepts underpinning many features (in the `./concepts` folder)

RFCs are for developers *building on* Aries. They don't provide guidance on how Aries components
implement features internally; individual Aries repos have design docs for that. Each
Aries RFC includes an "implementations" section and all RFCs with a status greater than
`Proposed` should have at least one listed implementation.

## RFC Lifecycle

RFCs go through a standard lifecycle.

<!-- To edit this lifecycle drawing:
 - Copy the URL below
 - Navigate to https://www.plantuml.com/plantuml/uml/
 - Paste the URL into the text area in the middle of the screen beside the "Decode URL" button.
 - Click the "Decode URL" button and the editable Plantuml definition of the drawing will be in the top of the box.
 - Edit the lifecycle drawing as desired.
 - When complete, copy the resulting URL from the box. Note that the copied URL is missing the leading "https:".
 - Replace the URL below, ensuring to add the `https:` at the beginning.
 - Done!
 -->

![lifecycle](https://www.plantuml.com/plantuml/png/TP1DZeCm38NtEKK6rbnXHAKUe6gNg8iKN43AJ-GOUlsIoa60mYQs_Db-UQu3AQJ9QF570nYGy_X2PKayI178uWuq8dI5L45oBilbINm9MZFdVCjlwBmBiQR7Vg0U0IoZAnXd0w6YBBwqBVWJv3sw-O1MfQefVvLdzR_J43l1gdCVk-bCSeAJJ0Uh7lPeU5CJ7SSUlX0lESUyAeytLZ3wMpdVLt1Cq-iFqvoemNQJqLy0)

#### PROPOSED
To __propose__ an RFC, [use these instructions to raise a PR](
contributing.md#how-to-propose-an-RFC) against the repo. Proposed
RFCs are considered a "work in progress", even after they are merged. In other words, they
haven't been endorsed by the community yet, but they seem like reasonable ideas worth
exploring.

#### DEMONSTRATED
__Demonstrated__ RFCs have one or more implementations available, listed in the "Implementations" section of the RFC document. As with the PROPOSED status, demonstrated RFCs haven't been endorsed by the community, but the ideas put forth have been more thoroughly explored through the implementation(s). The demonstrated status is an optional step in the lifecycle. For protocol-related RFCs, work on protocol tests SHOULD begin in the [test suite repo](https://github.com/hyperledger/aries-protocol-test-suite) by the time this status is assigned.

#### ACCEPTED
To get an RFC __accepted__, [build consensus](contributing.md#how-to-get-an-RFC-accepted) for your RFC on [chat](https://chat.hyperledger.org/channel/aries) and in community meetings. If your RFC is a feature that's protocol- or decorator-related, it MUST have reasonable tests in the [test suite repo](https://github.com/hyperledger/aries-agent-test-harness), it MUST list the test suite in the protocol RFC's [Implementations section](/0000-template.md#implementations), at least one other implementation must have passed the relevant portions of the test suite, and all implementations listed in this section of the RFC MUST hyperlink to their test results. An accepted RFC is incubating on a standards track; the community has decided to polish it and is exploring or pursuing implementation.

#### ADOPTED
To get an RFC __adopted__, [socialize and implement](contributing.md#how-to-get-an-rfc-adopted). An RFC gets this status once it has significant momentum--when implementations accumulate, or when the mental model it advocates has begun to permeate our discourse. In other words, adoption is acknowledgment of a _de facto_ standard.

To __refine__ an RFC, propose changes to it through additional PRs. Typically these changes are driven by experience that accumulates during or after adoption. Minor refinements that just improve clarity can happen inline with lightweight review. Status is still ADOPTED.

#### STALLED
An RFC is __stalled__ when a [proposed](#proposed) RFC makes
no progress towards implementation such that it is extremely unlikely it will ever move forward. The __stalled__ state differs from [retired](#retired) in that it is an RFC that has never been implemented or superseded. Like the [retired](#retired) state, it is (likely) an end state and the RFC will not proceed further. Such an RFC remains in the repository on the off chance it will ring a chord with others, be returned to the [proposed](#proposed) state, and continue to evolve.

#### RETIRED
An RFC is __retired__ when it is withdrawn from community consideration by its authors, when implementation seems permanently stalled, or when significant refinements require a superseding document. If a retired RFC has been superseded, its `Superseded By` field should contain a link to the newer spec, and the newer spec's `Supersedes` field should contain a link to the older spec. Permalinks are not broken.

### Changing an RFC Status

See notes about this in [Contributing](contributing.md#changing-an-rfc-status).

## About

#### License

This repository is licensed under an [Apache 2 License](LICENSE). It is protected
by a [Developer Certificate of Origin](https://developercertificate.org/) on every commit.
This means that any contributions you make must be licensed in an Apache-2-compatible
way, and must be free from patent encumbrances or additional terms and conditions. By
raising a PR, you certify that this is the case for your contribution.

For more instructions about contributing, see [Contributing](contributing.md).

#### Acknowledgement

The structure and a lot of the initial language of this repository was borrowed from [Indy HIPEs](
https://github.com/hyperledger/indy-hipe), which borrowed it from [Rust RFC](https://github.com/rust-lang/rfcs).
Their good work has made the setup of this repository much quicker and better than it otherwise would have been.
If you are not familiar with the Rust community, you should check them out.


#

[![GitHub release (latest by date)](https://img.shields.io/github/v/release/meisnate12/Plex-Meta-Manager?style=plastic)](https://github.com/meisnate12/Plex-Meta-Manager/releases)
[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/meisnate12/plex-meta-manager?label=docker&sort=semver&style=plastic)](https://hub.docker.com/r/meisnate12/plex-meta-manager)
[![Docker Pulls](https://img.shields.io/docker/pulls/meisnate12/plex-meta-manager?style=plastic)](https://hub.docker.com/r/meisnate12/plex-meta-manager)
[![Develop GitHub commits since latest stable release (by SemVer)](https://img.shields.io/github/commits-since/meisnate12/plex-meta-manager/latest/develop?label=Commits%20in%20Develop&style=plastic)](https://github.com/meisnate12/Plex-Meta-Manager/tree/develop)
[![Nightly GitHub commits since latest stable release (by SemVer)](https://img.shields.io/github/commits-since/meisnate12/plex-meta-manager/latest/nightly?label=Commits%20in%20Nightly&style=plastic)](https://github.com/meisnate12/Plex-Meta-Manager/tree/nightly)

[![Discord](https://img.shields.io/discord/822460010649878528?color=%2300bc8c&label=Discord&style=plastic)](https://metamanager.wiki/en/latest/discord/)
[![Reddit](https://img.shields.io/badge/%2Fr%2Fplexmetamanager-e05d44?style=plastic&logo=Reddit&logoColor=white&labelColor=0e8a6a&color=00bc8c)](https://www.reddit.com/r/PlexMetaManager/)
[![Wiki](https://img.shields.io/readthedocs/plex-meta-manager?color=%2300bc8c&style=plastic)](https://metamanager.wiki)
[![Translations](https://img.shields.io/weblate/progress/plex-meta-manager?color=00bc8c&server=https%3A%2F%2Ftranslations.metamanager.wiki&style=plastic)](https://translations.metamanager.wiki/projects/plex-meta-manager/#languages)
[![GitHub Sponsors](https://img.shields.io/github/sponsors/meisnate12?color=%238a2be2&style=plastic)](https://github.com/sponsors/meisnate12)
[![Sponsor or Donate](https://img.shields.io/badge/-Sponsor%2FDonate-blueviolet?style=plastic)](https://github.com/sponsors/meisnate12)
[![Feature Requests](https://img.shields.io/badge/Feature%20Requests-blueviolet?style=plastic)](https://features.metamanager.wiki/)

Plex Meta Manager is a powerful tool designed to give you complete control over your media libraries. With Plex Meta Manager, you can take your customization to the next level, with granular control over metadata, collections, overlays, and much more.

Transform your media library with Plex Meta Manager and discover its full potential! Connect to third-party services like TMDb, Trakt, and IMDb, among others, to create one-of-a-kind collections, overlays and more. Your media library will stand out and be tailored to your specific needs.

## What Can Plex-Meta-Manager Do?

### Overhaul Your Media Libraries

-  Elevate your library with beautifully crafted metadata - customize artwork, titles, summaries, and more to create a stunning library.

### PMM Defaults

-  Take advantage of pre-made modular Collections & Overlays to reduce the manual effort and get to the good stuff with less effort!

### Third-Party Integrations

-  Harness the power of Trakt, TMDb, IMDb and more to create collections and overlays!
-  Integrate with Sonarr and Radarr to automate your library growth.

### And More!

-  We're constantly working on new features to take your library management experience to the next level.
-  Consider sponsoring the project to allow us to continue building great features for you!

## Demo Video

The below YouTube video has been created by one of our community members to showcase some of the things that Plex Meta Manager can do for you.

[![Plex Meta Manager](https://img.youtube.com/vi/nTfCUtKWTYI/0.jpg)](https://www.youtube.com/watch?v=nTfCUtKWTYI "Plex Meta Manager")

## Example Plex Meta Manager Libraries

Here are some examples of the things you can achieve using Plex Meta Manager!

**Example Movie Collections using the [Plex Meta Manager Defaults](https://metamanager.wiki/en/latest/defaults/collections/)** (click to enlarge):

![Movie Collection Preview](https://metamanager.wiki/en/latest/images/movie-collection-preview.png)

**Example Show Overlays using the [Plex Meta Manager Defaults](https://metamanager.wiki/en/latest/defaults/overlays)** (click to enlarge):

![Show Library Preview](https://metamanager.wiki/en/latest/images/show-library-preview.png)

## Plex Meta Manager Defaults

Want your library to look like the above images?  With the [PMM Defaults](https://metamanager.wiki/en/latest/defaults/guide/) you can! These powerful and modular files were designed by the Plex Meta Manager team to make it simple to create a personalized, one-of-a-kind media collection without the hassle of manually defining each one.

Want to see what the community has to offer? Check out the [Plex Meta Manager Configs](https://github.com/meisnate12/Plex-Meta-Manager-Configs) repository on GitHub to see user-submitted configuration files, or even add your own to the mix!

With Plex Meta Manager, you can also manage metadata for all your media types, from movies and shows to music and more. And since your metadata is managed outside your libraries, you'll never have to worry about losing your customizations in the event of a media server database loss, you can simply reapply them! It is also easy to move your customizations between servers if you need to.

## Getting Started

To get started with Plex Meta Manager, follow these simple steps:

1. Install Plex Meta Manager on your device. You can find the installation instructions for a variety of platforms [here](https://metamanager.wiki/en/latest/pmm/install/overview/).

2. Once you have installed Plex Meta Manager, create your [Configuration File](https://metamanager.wiki/en/latest/config/overview/). This file contains important information such as URLs and credentials needed to connect to services like Plex and TMDb

3. After creating the Configuration File, you can start updating Metadata and building automatic Collections by creating a [Collection File](https://metamanager.wiki/en/latest/files/collections/) for each Library you want to work with. If you'd rather use some of our pre-made Collection Files, take a look at the [Plex Meta Manager Defaults](https://metamanager.wiki/en/latest/defaults/guide/)

4. Finally, check out the [Wiki](https://metamanager.wiki/), you'll find new and exciting ways to truly unlock the potential of your libraries.

## Step-by-Step Guides

If you're a beginner to the concepts of Python, Git and/or Plex Meta Manager and find the above steps challenging, don't worry. We've got some step-by-step guides that can help you get started. These guides will take you through the process of installing Plex Meta Manager, creating your Configuration File and getting some basic Collections up and running.

For those who need full installation walkthroughs, please refer to the following walkthrough guides:

  * [Local Walkthrough](https://metamanager.wiki/en/latest/pmm/install/local/) - follow this if you are running the script directly on Windows, OS X, or Linux
  * [Docker Walkthrough](https://metamanager.wiki/en/latest/pmm/install/docker/) - this discusses using Docker at the command line

If you are using unRAID, Kubernetes, QNAP, or Synology refer to the following basic guide to Docker container setup for each system:

**this doesn't cover the PMM setup specifics found in the guides above with regard to creating the config file and collection file, so you may want to go through the [Docker Walkthrough](https://metamanager.wiki/en/latest/pmm/install/docker/) first on your computer to gain that understanding.**

  * [unRAID Walkthrough](https://metamanager.wiki/en/latest/pmm/install/unraid/)
  * [Kubernetes Walkthrough](https://metamanager.wiki/en/latest/pmm/install/kubernetes/)
  * [QNAP Walkthrough](https://metamanager.wiki/en/latest/pmm/install/qnap/)
  * [Synology Walkthrough](https://metamanager.wiki/en/latest/pmm/install/synology/)

## Example Usage

Plex Meta Manager puts you in control of your media library by letting you create custom Collections that make discovering and organizing your content a breeze. With powerful search and filtering options, you can build Collections based on popular builders like TMDb, IMDb, Trakt, and many more.

Imagine having Collections like these at your fingertips:

  * Trending and Popular (based on TMDb, IMDb, Trakt, etc.)
  * Streaming Services (like Netflix, Disney+, and more)
  * Networks
  * Studios
  * Genres
  * Actors
  * Decades

Plex Meta Manager gives you endless possibilities to curate and organize your media library any way you want. Create custom Collections and Overlays that fit your unique preferences and make discovering your content effortless.

But if you don't want to spend time manually creating Collections and Overlays, we've got you covered. Check out the [PMM Defaults](https://metamanager.wiki/en/latest/defaults/guide/) - a handcrafted selection of tried-and-tested Collections and Overlays made by the Plex Meta Manager team.

## Develop & Nightly Branches

The Develop and Nightly branches are "beta" versions of Plex Meta Manager that are updated more frequently than the stable version (Master branch). These branches are where bug fixes, new features, and other changes are added before being released to the Master branch.

However, these branches (especially Nightly) are recommended for more technical users who don't mind updating frequently to get the latest changes. Keep in mind that these beta branches may have bugs or other issues that could cause problems with Plex Meta Manager or your media server. So, if you're not comfortable with technical issues, it's best to stick with the Master branch.

  Develop Branch (click to expand)

[![Develop GitHub commits since latest stable release (by SemVer)](https://img.shields.io/github/commits-since/meisnate12/plex-meta-manager/latest/develop?label=Commits%20in%20Develop&style=plastic)](https://github.com/meisnate12/Plex-Meta-Manager/tree/develop)

The [develop](https://github.com/meisnate12/Plex-Meta-Manager/tree/develop) branch has the most updated **documented** fixes and enhancements to Plex Meta Manager.  This version is tested and documented to some degree, but it is still an active Develop branch, so there may be rough edges.

Switching to `develop`:
````{tab} Running in Docker
Add ":develop" to the image name in your run command or configuration:
```
meisnate12/plex-meta-manager:develop
```
````
````{tab} Running on the Host
In the directory where you cloned PMM:
```bash
git checkout develop
```
To switch back:
```bash
git checkout master
```
````

If switching to the develop branch, it is recommended to also use the [develop branch of the wiki](https://metamanager.wiki/en/develop/), which documents any changes made from the Master branch.

  Nightly Branch  (click to expand)

[![Nightly GitHub commits since latest stable release (by SemVer)](https://img.shields.io/github/commits-since/meisnate12/plex-meta-manager/latest/nightly?label=Commits%20in%20Nightly&style=plastic)](https://github.com/meisnate12/Plex-Meta-Manager/tree/nightly)

**This branch will have squashed commits which can cause `git pull`/`git fetch` to error you can use `git reset origin/nightly --hard` to fix the branch.**

There is also a [nightly](https://github.com/meisnate12/Plex-Meta-Manager/tree/nightly) branch which will have the absolute latest version of the script, but it could easily break, there is no guarantee that it even works, and any new features will not be documented until they have progressed enough to reach the develop branch.

Switching to `nightly`:

````{tab} Running in Docker
Add ":nightly" to the image name in your run command or configuration:
```
meisnate12/plex-meta-manager:nightly
```
````
````{tab} Running on the Host
In the directory where you cloned PMM:
```bash
git checkout nightly
```
To switch back:
```bash
git checkout master
```
````

As this branch is subject to extreme change, there is no promise of the feature being documented in the [nightly](https://metamanager.wiki/en/nightly/) branch of the wiki and all discussions relating to changes made in the nightly branch will be held within the [Plex Meta Manager Discord Server](https://metamanager.wiki/en/latest/discord/).

## Discord Support Server

If you're looking for support for any questions or issues you might have, or if you just want to be a part of our growing community, Join the [Plex Meta Manager Discord Server](https://metamanager.wiki/en/latest/discord/).

## Feature Requests

At Plex Meta Manager, we value our community's input and actively seek feedback to drive the evolution of our product. We want to hear your ideas on how to enhance Plex Meta Manager, and we encourage you to visit our [Feature Request](https://features.metamanager.wiki/features) page to share your thoughts or vote on what features you would like to see added next. Your voice matters and helps shape the future of Plex Meta Manager, so please don't hesitate to join in the conversation and be a part of our community-driven development process.

## Errors and Configuration Questions

If you're having trouble, we recommend first joining the [Plex Meta Manager Discord Server](https://metamanager.wiki/en/latest/discord/) and seeking support there. If that isn't possible for you, here's what you can do:

* If you get an error, update to the latest version and check if the issue persists. If it does, report the bug by filling out the [Bug Report](https://github.com/meisnate12/Plex-Meta-Manager/issues/new?assignees=meisnate12&labels=status%3Anot-yet-viewed%2C+bug&template=bug_report.md&title=Bug%3A+) template.
* If you spot a mistake or have an idea to improve the [Plex Meta Manager Wiki](https://metamanager.wiki/), submit a request using the [Wiki Request](https://github.com/meisnate12/Plex-Meta-Manager/issues/new?assignees=meisnate12&labels=status%3Anot-yet-viewed%2C+documentation&template=3.docs_request.yml&title=%5BDocs%5D%3A+) template.
* If you have a question about metadata configuration, start a discussion on the [Discussions](https://github.com/meisnate12/Plex-Meta-Manager/discussions). Remember, the community helps shape the future of Plex Meta Manager, so your input is valuable!

For support on any of the above, visit the [Discord server](https://metamanager.wiki/en/latest/discord/).

## Contributing
* Pull Requests are greatly encouraged, please submit all Pull Requests to the nightly branch.


  CloudFront Extensions

Amazon CloudFront operation toolkit

CloudFront Extensions GitHub project offers an open source package consisting of a set of applications and solution templates that covers various user scenarios for using [Amazon CloudFront](https://aws.amazon.com/cloudfront/), [Lambda@Edge](https://aws.amazon.com/lambda/edge/) and CloudFront Functions.

      Home Page

     |

      Usage

     |

      Monitoring

     |

      Config Snapshot

     |

      SSL Certificates

     |

      Extensions Repo

     |

      Workshop

     |

      Blog

## Feature

### Monitoring
The solution provides two kinds of monitoring: non-real time monitoring and real time monitoring. You can choose one of them to deploy. You can use the solution to view CloudFront traffic metrics via built-in dashboard or via API calls. It analysis CloudFront logs to get the metric data and you only need to modify MetricsCollector, MetricsHelper and MetricsManager to add new metrics.

### Config Snapshot
A Snapshot is a CloudFront Distribution configuration. By saving a complex CloudFront Distribution configuration to a Snapshot, the configuration (snapshot) can be reused to current or other CloudFront Distributions in your AWS account.

### SSL Certificates

You can use this solution to automatically create or import SSL Certificates and associate them to new CloudFront distributions. The solution provides two kinds of jobs: Create Certificates Job and Import Certificate Job. You can use them on web console or via API calls.

### Extensions Repository

The solution provides a set of ready-to-use extensions (Lambda@Edge functions, CloudFront functions, CloudFormation templates) that could be useful when you use CloudFront. You can deploy them from the solution web console. After deployment, you can directly use it without programming or customize it when needed.

## Deployment

**Time to deploy:** Approximately 15 minutes.

Follow the step-by-step instructions in [implementation guide](https://awslabs.github.io/aws-cloudfront-extensions/en/deployment/) to configure and deploy CloudFront Extensions into your account.

1. Make sure you have signed in AWS Console.
2. Click the following button to launch the CloudFormation Stack in your account.

   | Region        | Launch Button  |
   | ------------- | -------------  |
   | Global Region      |    [![Launch Stack](./docs-ig/images/deploy_button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=cloudFrontExtensionsConsole&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/Aws-cloudfront-extensions/latest/custom-domain/CloudFrontExtnConsoleStack.template.json) |

## Architecture diagram

Deploying CloudFront Extensions solution with the default parameters builds the following environment in the AWS Cloud.

1. Amazon CloudFront distributes the solution frontend web UI assets hosted in Amazon S3 bucket.
2. Amazon Cognito user pool provides authentication for backend.
3. Amazon AppSync provides the backend GraphQL APIs.
4. Amazon API Gateway provides the backend RESTful APIs for SSL certificates and Monitoring features.
5. Amazon DynamoDB stores the solution related information as backend database.
6. Amazon Lambda interacts with other Amazon Services to process core logic of monitoring, SSL certificates and extensions repository, and obtains information updated in DynamoDB tables.
7. AWS Step Functions orchestrate workflows for creating ACM certificates, importing existed certificates and creating CloudFront distributions.
8. Extensions are shown in Extensions repository. AWS CloudFormation and AWS Serverless Application Repository will be triggered if you want to deploy an extension into your AWS account.
9. AWS Lambda stores CloudFront configuration changes into S3 bucket, and you can view the difference between two CloudFront configuration versions and apply the configuration.
10. Amazon Athena queries CloudFront standard logs or real-time logs to get CloudFront metrics and output it by API Gateway. You can also view the metrics by monitoring dashboard.

## Contribution

See [CONTRIBUTING](./CONTRIBUTING.md) for more information.

## License

This project is licensed under the Apache-2.0 License.


# EDDN - Elite Dangerous Data Network

## About EDDN
Elite Dangerous Data Network is a tool that facilitates players of the game
[Elite Dangerous](https://www.elitedangerous.com/), including its
expansions, sharing data about the game galaxy with others.
By pooling data in a common format, tools and analyses can be produced that add
an even greater depth and vibrancy to the in-game universe.

EDDN is not run by or affiliated with the developer of the game - [Frontier
Developments](http://www.frontier.co.uk/).

The live EDDN service itself does not store any data, and thus makes no
archive or "current state" available to anyone.  What it provides is a
stream of live data to any interested parties.  Some of those then make
aggregated data available for general use.

If you want an archive of messages, or a dump of the current known state of
the game galaxy, check ["Archives and data dumps"](#archives-and-data-dumps)
below.

---
---

## Using EDDN
### Game players
It might be useful to consult the [EDCD Cmdr's Guide](https://edcd.github.io/cmdrs-guide.html)
for a general overview of how players can contribute and use game data.

---

#### Contributing data
There are a variety of tools available to players in order for them to help
out by contributing data, and to then utilise that data to enhance their
gameplay experience.

For the most part any player who wishes to share data
will need to be playing the game on a PC, as that gives direct access to
"Journal" files written by the game client.  These are the best source of
game data.
There are, however, some tools that utilise an API provided by the game
developer that can supply some data if you are playing on a console.

So, on PC, look into installing one of the following tools:

- [E:D Market Connector](https://github.com/EDCD/EDMarketConnector/wiki)
- [EDDI](https://github.com/EDCD/EDDI)
- [EDDiscovery](https://github.com/EDDiscovery/EDDiscovery)
- [Elite Log Agent](https://github.com/DarkWanderer/Elite-Log-Agent)

This list is not exhaustive, or intended to particular endorse any of these
projects over another, listed here or not.

If you are playing on console some options are:

- The 'console updater', available in a user's 'dashboard' on
  [EDSM](https://www.edsm.net).
- [Journal Limpet](https://journal-limpet.com/).

---

#### Utilising data
If you're looking for tools that utilise EDDN data to enhance your experience
then you're probably looking for one of the sites listed below.  NB: These are
listed in name-alphabetical order and no particular ranking or endorsement is
intended.

- [EDSM](https://www.edsm.net/) - originally focused on being a 'Star Map',
  but has since expanded its functionality.  Of particular interest to
  in-game explorers.
- [Inara](https://inara.cz/) - a popular alternative to the now defunct EDDB,
  with a lot of its own unique functionality.
- [Spansh](https://www.spansh.co.uk/plotter) - originally this had one tool,
  a 'Neutron Star' route plotter, but has since expanded into offering many
  other route plotting tools and general data searching.

##### Archives and data dumps

Alternatively if you want an archive of past EDDN messages, or a data dump to
use:

- [edgalaxydata](https://edgalaxydata.space/) has various data captures,
    including 'all' (some listener downtime is inevitable) EDDN messages for
    many years.

- [spansh dumps](https://www.spansh.co.uk/dumps) are a "whole galaxy" data set,
    of systems, bodies and stations.  The full `galaxy.json.gz` is **very**
    large, but is currently the only source of an "all known bodies" dump.
    Pay attention to the 'Generated' "time ago" column.

- [EDSM nightly dumps](https://www.edsm.net/en/nightly-dumps) represent a
    snapshot of the data EDSM uses.  NB: there's only a "last 7 days" bodies
    dump as the full data proved too large to dump in a timely manner.

---

There are many other third-party tools for Elite Dangerous, both for
contributing data and utilising it, listed on
[Elite: Dangerous Codex](https://edcodex.info/).  Some of them
interact with EDDN - check the [EDDN tag](https://edcodex.info/?m=tools&cat=9).

---

### Developers
If you are a developer of a third-party tool that could be enhanced by
uploading data to EDDN then please consult
[the live branch Developers' documentation](https://github.com/EDCD/EDDN/blob/live/docs/Developers.md)
.
**DO NOT** assume that any code or documentation in the `master` (or
any other) branch on GitHub truly reflects the current live service!

Anyone planning to send data too EDDN **MUST** comply with all the advice in
that document, and the individual schema README files as applicable.  It's
also the best resource for those listening to the EDDN data stream.

#### EDDN endpoints

There are up to three separate services which might be running.

| Service | Upload |           Listeners           | Notes                                                                                |
| ------: | :-----: |:-----------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Live | `https://eddn.edcd.io:4430/upload/` |   `tcp://eddn.edcd.io:9500/`    | The actual live service, which should always be running. It is automatically restarted every Thursday at 07:06:00 UTC                                                                 |
| Beta | `https://beta.eddn.edcd.io:4431/upload/` | `tcp://beta.eddn.edcd.io:9510/` | The beta service, which should be running the current state of the `beta` branch.  Usually only active when either new schemas or forthcoming code changes are being actively tested. |
| Dev | `https://dev.eddn.edcd.io:4432/upload/` | `tcp://dev.eddn.edcd.io:9520/`  | The dev service, which could be running any public branch of the code *or* a private branch.                                                                                |

In general the Beta and Dev services will only be running so as to aid the core
development team in testing changes.  Don't expect them to be generally
available.

You **MUST** use the correct hostname in the Upload URLs as these are
TLS/HTTPS connections terminated on a Reverse Proxy.

The Listener URLs are ZeroMQ endpoints, no TLS.  But whilst this means you
don't strictly need to use the correct hostname there is no guarantee that the
beta and dev hostnames won't be pointing at, or hosted on, a different IP.

If you need to test some of your own changes then please read
[Running this software](docs/Running-this-software.md) for how to instantiate
your own test service.  It is hoped that in the future the code will allow for
easily running in a "local only" mode, not requiring any reverse proxy or
internet-valid TLS certificates.

#### New Schemas
*All* new Schema proposals **MUST** be started by opening
[an issue on GitHub](https://github.com/EDCD/EDDN/issues/new/choose).  Please
consult
[docs/Contributing 'Adding a New Schema'](docs/Contributing.md#adding-a-new-schema)
for further guidelines.

---
---

## Misc

### Service Status
Consult [EDDN Status](https://eddn.edcd.io/) for some information about,
and statistics for, the live service.

### Hosting of the live service

Hosting is currently provided by the
[Elite: Dangerous Community Developers](https://edcd.github.io/).

### Contacting the EDDN team

* [EDCD Discord - #eddn channel](https://discord.gg/DdqJ8nWVGc)
* [E:D forum thread](https://forums.frontier.co.uk/threads/elite-dangerous-data-network-eddn.585701/#post-9400060)


Lightkurve
==========

**A friendly package for Kepler & TESS time series analysis in Python.**

**Documentation: https://docs.lightkurve.org**

|test-badge| |conda-badge| |pypi-badge| |pypi-downloads| |doi-badge| |astropy-badge|

.. |conda-badge| image:: https://img.shields.io/conda/vn/conda-forge/lightkurve.svg
                 :target: https://anaconda.org/conda-forge/lightkurve
.. |pypi-badge| image:: https://img.shields.io/pypi/v/lightkurve.svg
                :target: https://pypi.python.org/pypi/lightkurve
.. |pypi-downloads| image:: https://pepy.tech/badge/lightkurve
                :target: https://pepy.tech/project/lightkurve
.. |test-badge| image:: https://github.com/lightkurve/lightkurve/workflows/Lightkurve-tests/badge.svg
                 :target: https://github.com/lightkurve/lightkurve/actions?query=branch%3Amain
.. |astropy-badge| image:: https://img.shields.io/badge/powered%20by-AstroPy-orange.svg?style=flat
                   :target: http://www.astropy.org
.. |doi-badge| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.1181928.svg
              :target: https://docs.lightkurve.org/about/citing.html

**Lightkurve** is a community-developed, open-source Python package which offers a beautiful and user-friendly way
to analyze astronomical flux time series data,
in particular the pixels and lightcurves obtained by
**NASA's Kepler and TESS exoplanet missions**.

.. image:: https://raw.githubusercontent.com/lightkurve/lightkurve/main/docs/source/_static/images/lightkurve-teaser.gif

This package aims to lower the barrier for students, astronomers,
and citizen scientists interested in analyzing Kepler and TESS space telescope data.
It does this by providing **high-quality building blocks and tutorials**
which enable both hand-tailored data analyses and advanced automated pipelines.

Documentation
-------------

Read the documentation at `https://docs.lightkurve.org <https://docs.lightkurve.org>`_.

Quickstart and Installation
---------------------------

Please visit our quickstart guide at `https://docs.lightkurve.org/quickstart.html <https://docs.lightkurve.org/quickstart.html>`_.

The easiest way to install *Lightkurve* and all of its dependencies is to use the ``pip`` command,
which is a standard part of all Python distributions.
To install *Lightkurve*, run the following command in a terminal window::

    $ python -m pip install lightkurve --upgrade

The ``--upgrade`` flag is optional, but recommended if you already
have *Lightkurve* installed and want to upgrade to the latest version.

Depending on the specific Python environment, you may need to replace ``python``
with the correct Python interpreter, e.g., ``python3``.

If you want to experiment with the latest development version of
*Lightkurve*, you can install it straight from the main branch on GitHub:

.. code-block:: bash

    $ git clone https://github.com/lightkurve/lightkurve.git
    $ cd lightkurve
    $ python -m pip install .

If you want to have a so-called editable install which enables the installed
version to immediately reflect changes made in the source tree, you can use:

.. code-block:: bash

    $ python -m pip install poetry
    $ poetry install

Please see our guide on `https://docs.lightkurve.org/development/index.html <https://docs.lightkurve.org/development/index.html>`_
for additional instructions.

Contributing
------------

We welcome community contributions!
Please read the  guidelines at `https://docs.lightkurve.org/development/contributing.html <https://docs.lightkurve.org/development/contributing.html>`_.

Citing
------

If you find Lightkurve useful in your research, please cite it and give us a GitHub star!
Please read the citation instructions at `https://docs.lightkurve.org/about/citing.html <https://docs.lightkurve.org/about/citing.html>`_.

Contact
-------
Lightkurve is an open source community project created by `the authors <AUTHORS.rst>`_.
The best way to contact us is to `open an issue <https://github.com/lightkurve/lightkurve/issues/new>`_ or to e-mail tesshelp@bigbang.gsfc.nasa.gov.
Please include a self-contained example that fully demonstrates your problem or question.


# Sollumz
Blender plugin to import, export, or create GTA V assets in the Codewalker XML format.

By Skylumz, Colton Fox and alexguirre.

## Join our Discord
Join the Sollumz [discord server](https://discord.gg/bZuWBWaQBg) to ask questions and chat with the community!
## Using the plugin
See the [wiki](https://github.com/Skylumz/Sollumz/wiki) to get started. Make sure to read the wiki before posting an issue or asking questions! Also, see the Tutorials and Resources channel on the Discord server.

**Note:** The issue tracker should be for bug reports and feature requests only. If you are having an issue and you are not sure if it is a bug or not, ask on the Discord server first!

## Features ##

**Supported Formats (Import)**
  * YDD
  * YDR
  * YBN
  * YCD
  * YMAP
  * YTYP
  * YNV (Partial)
  * YFT

**Supported Formats (Export)**
  * YDD
  * YDR
  * YBN
  * YCD
  * YMAP
  * YTYP
  * YFT

**General Features**
  * Import / Export
  * Automatic texture loading
  * Vertex colors
  * Bones and weights
  * Animations
  * Custom normals
  * Shader editing
  * Vehicle creation/editing
  * Dynamic prop creation/editing

## Requirements ##
  * Latest Codewalker release from [their discord](https://discord.gg/codewalker)
  * [Blender 4.0 or later](http://www.blender.org/download/)

## Installation ##
  1. Download the [latest release](https://github.com/Skylumz/Sollumz/releases/latest)
  2. Start Blender
  3. Click menu _`Edit > Preferences...`_
  4. Go to the  _`Addons`_ tab in the middle
  5. Click _`Install...`_ button at the top
  6. Select the Downloaded ZIP
  7. The following entry should be shown: _`Import-Export: Sollumz`_
  8. Enable the addon
  9. Restart Blender

---
### Supporters ❤ ###
- [GitBook](https://www.gitbook.com/)
- dexyfex and the [CodeWalker](https://github.com/dexyfex/CodeWalker) team


# plenoptic

[![PyPI Version](https://img.shields.io/pypi/v/plenoptic.svg)](https://pypi.org/project/plenoptic/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/LabForComputationalVision/plenoptic/blob/main/LICENSE)
![Python version](https://img.shields.io/badge/python-3.10|3.11|3.12-blue.svg)
[![Build Status](https://github.com/LabForComputationalVision/plenoptic/workflows/build/badge.svg)](https://github.com/LabForComputationalVision/plenoptic/actions?query=workflow%3Abuild)
[![Documentation Status](https://readthedocs.org/projects/plenoptic/badge/?version=latest)](https://plenoptic.readthedocs.io/en/latest/?badge=latest)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10151131.svg)](https://doi.org/10.5281/zenodo.10151131)
[![codecov](https://codecov.io/gh/LabForComputationalVision/plenoptic/branch/main/graph/badge.svg?token=EDtl5kqXKA)](https://codecov.io/gh/LabForComputationalVision/plenoptic)
[![Binder](http://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/LabForComputationalVision/plenoptic/1.0.1?filepath=examples)
[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)

![](docs/images/plenoptic_logo_wide.svg)

`plenoptic` is a python library for model-based synthesis of perceptual stimuli.
For `plenoptic`, models are those of visual[^1] information processing: they
accept an image as input, perform some computations, and return some output,
which can be mapped to neuronal firing rate, fMRI BOLD response, behavior on
some task, image category, etc. The intended audience is researchers in
neuroscience, psychology, and machine learning. The generated stimuli enable
interpretation of model properties through examination of features that are
enhanced, suppressed, or discarded. More importantly, they can facilitate the
scientific process, through use in further perceptual or neural experiments
aimed at validating or falsifying model predictions.

## Getting started

-   If you are unfamiliar with stimulus synthesis, see the [conceptual
    introduction](https://plenoptic.readthedocs.io/en/latest/conceptual_intro.html)
    for an in-depth introduction.
-   If you understand the basics of synthesis and want to get started
    using `plenoptic` quickly, see the
    [Quickstart](examples/00_quickstart.ipynb) tutorial.

### Installation

The best way to install `plenoptic` is via `pip`.

``` bash
$ pip install plenoptic
```

Our dependencies include [pytorch](https://pytorch.org/) and
[pyrtools](https://pyrtools.readthedocs.io/en/latest/). Installation should take
care of them (along with our other dependencies) automatically, but if you have
an installation problem (especially on a non-Linux operating system), it is
likely that the problem lies with one of those packages. [Open an
issue](https://github.com/LabForComputationalVision/plenoptic/issues) and we'll
try to help you figure out the problem!

See the [installation
page](https://plenoptic.readthedocs.io/en/latest/install.html) for more details,
including how to set up a virtual environment and jupyter.

### ffmpeg and videos

Several methods in this package generate videos. There are several backends
possible for saving the animations to file, see [matplotlib
documentation](https://matplotlib.org/stable/api/animation_api.html#writer-classes)
for more details. In order convert them to HTML5 for viewing (and thus, to view
in a jupyter notebook), you'll need [ffmpeg](https://ffmpeg.org/download.html)
installed and on your path as well. Depending on your system, this might already
be installed, but if not, the easiest way is probably through [conda]
(https://anaconda.org/conda-forge/ffmpeg): `conda install -c conda-forge
ffmpeg`.

To change the backend, run `matplotlib.rcParams['animation.writer'] = writer`
before calling any of the animate functions. If you try to set that `rcParam`
with a random string, `matplotlib` will tell you the available choices.

## Contents

### Synthesis methods

![](docs/images/example_synth.svg)

-   [Metamers](examples/06_Metamer.ipynb): given a model and a
    reference image, stochastically generate a new image whose model
    representation is identical to that of the reference image. This
    method investigates what image features the model disregards
    entirely.
-   [Eigendistortions](examples/02_Eigendistortions.ipynb): given a
    model and a reference image, compute the image perturbation that
    produces the smallest and largest changes in the model response
    space. This method investigates the image features the model
    considers the least and most important.
-   [Maximal differentiation (MAD)
    competition](examples/07_MAD_Competition.ipynb): given two metrics
    that measure distance between images and a reference image, generate
    pairs of images that optimally differentiate the models.
    Specifically, synthesize a pair of images that the first model says
    are equi-distant from the reference while the second model says they
    are maximally/minimally distant from the reference. Then synthesize
    a second pair with the roles of the two models reversed. This method
    allows for efficient comparison of two metrics, highlighting the
    aspects in which their sensitivities differ.
-   [Geodesics](examples/05_Geodesics.ipynb): given a model and two
    images, synthesize a sequence of images that lie on the shortest
    ("geodesic") path in the model's representation space. This
    method investigates how a model represents motion and what changes
    to an image it consider reasonable.

### Models, Metrics, and Model Components

-   Portilla-Simoncelli texture model, which measures the statistical properties
    of visual textures, here defined as "repeating visual patterns."
-   Steerable pyramid, a multi-scale oriented image decomposition. The basis are
    oriented (steerable) filters, localized in space and frequency. Among other
    uses, the steerable pyramid serves as a good representation from which to
    build a primary visual cortex model. See the [pyrtools
    documentation](https://pyrtools.readthedocs.io/en/latest/index.html) for
    more details on image pyramids in general and the steerable pyramid in
    particular.
-   Structural Similarity Index (SSIM), is a perceptual similarity metric,
    returning a number between -1 (totally different) and 1 (identical)
    reflecting how similar two images are. This is based on the images'
    luminance, contrast, and structure, which are computed convolutionally
    across the images.
-   Multiscale Structrual Similarity Index (MS-SSIM), is a perceptual similarity
    metric similar to SSIM, except it operates at multiple scales (i.e.,
    spatial frequencies).
-   Normalized Laplacian distance, is a perceptual distance metric based on
    transformations associated with the early visual system: local luminance
    subtraction and local contrast gain control, at six scales.

## Getting help

We communicate via several channels on Github:

-   [Discussions](https://github.com/LabForComputationalVision/plenoptic/discussions)
    is the place to ask usage questions, discuss issues too broad for a
    single issue, or show off what you've made with plenoptic.
-   If you've come across a bug, open an
    [issue](https://github.com/LabForComputationalVision/plenoptic/issues).
-   If you have an idea for an extension or enhancement, please post in the
    [ideas
    section](https://github.com/LabForComputationalVision/plenoptic/discussions/categories/ideas)
    of discussions first. We'll discuss it there and, if we decide to pursue it,
    open an issue to track progress.
-   See the [contributing guide](CONTRIBUTING.md) for how to get involved.

In all cases, please follow our [code of conduct](CODE_OF_CONDUCT.md).

## Citing us

If you use `plenoptic` in a published academic article or presentation, please
cite both the code by the DOI as well the JOV paper. If you are not using the
code, but just discussing the project, please cite the paper. You can click on
`Cite this repository` on the right side of the GitHub page to get a copyable
citation for the code, or use the following:

- Code: [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10151131.svg)](https://doi.org/10.5281/zenodo.10151131)
- Paper:
  ``` bibtex
  @article{duong2023plenoptic,
    title={Plenoptic: A platform for synthesizing model-optimized visual stimuli},
    author={Duong, Lyndon and Bonnen, Kathryn and Broderick, William and Fiquet, Pierre-{\'E}tienne and Parthasarathy, Nikhil and Yerxa, Thomas and Zhao, Xinyuan and Simoncelli, Eero},
    journal={Journal of Vision},
    volume={23},
    number={9},
    pages={5822--5822},
    year={2023},
    publisher={The Association for Research in Vision and Ophthalmology}
  }
  ```

See the [citation
guide](https://plenoptic.readthedocs.io/en/latest/citation.html) for more
details, including citations for the different synthesis methods and
computational moels included in plenoptic.

## Support

This package is supported by the Simons Foundation Flatiron Institute's Center
for Computational Neuroscience.

![](docs/images/CCN-logo-wText.png)

[^1]: These methods also work with auditory models, such as in [Feather et al.,
    2019](https://proceedings.neurips.cc/paper_files/paper/2019/hash/ac27b77292582bc293a51055bfc994ee-Abstract.html),
    though we haven't yet implemented examples. If you're interested, please
    post in
    [Discussions](https://github.com/LabForComputationalVision/plenoptic/discussions)!


  Leon

_
Your open-source personal assistant.
_

  Website ::
  Documentation ::
  Roadmap ::
  Contributing ::
  Story

---

## Important Notice (as of 2024-06-18)

> [!IMPORTANT]
> Due to all the new major changes coming to Leon AI, the development branch might be unstable. It is recommended to use the older version under the master branch.
>
> Please note that older versions do not make use of any foundation model, which will be introduced in upcoming versions.

**Outdated Documentation**

Please note that the documentation and this README are not up to date. We've made significant changes to Leon over the past few months, including the introduction of new TTS and ASR engines, and a hybrid approach that balances LLM, simple classification, and multiple NLP techniques to achieve optimal speed, customization, and accuracy. We'll update the documentation for the official release.

**Project History and Future Plans**

Since its inception in 2017, Leon has undergone significant transformations. Although we've been inconsistent in shipping updates over the years, we're now focused on maturing the project. With the recent integration of transformers-based models, we're prepared to unlock Leon's full potential.

Our next step is to finalize the latest features for the official release.
Then we'll be establishing a group of active contributors to work together, develop new skills, and share them with the community. A skill registry platform will be built (see it as the npm or pip registry but for skills).

- Check out [the roadmap](http://roadmap.getleon.ai/) for more information on our upcoming plans.
- Watch a [preview of our last progress](https://www.youtube.com/watch?v=6CInSt6pTVA) to see what we've been working on.

**Get Involved**

[Join us on Discord](https://discord.gg/MNQqqKg) to ask questions, or express interest in becoming an active contributor.

---

### Why is there a small amount of contributors?

I'm taking a lot of time to work on the new core of Leon due to personal reasons. I can only work on it during my spare time. Hence, I'm blocking any contribution as the whole core of Leon is coming with many breaking changes. Many of you are willing to contribute in Leon (create new skills, help to improve the core, translations and so on...), a big thanks to every one of you!

While I would love to devote more time to Leon, I'm currently unable to do so because I have bills to pay. I have some ideas about how to monetize Leon in the future (Leon's core will always remain open source), but before to get there there is still a long way to go.

Until then, any financial support by [sponsoring Leon](http://sponsor.getleon.ai) is much appreciated 🙂

---

## Latest Release

Check out the [latest release blog post](https://blog.getleon.ai/binaries-and-typescript-rewrite-1-0-0-beta-8/).

---

## 👋 Introduction

**Leon** is an **open-source personal assistant** who can live **on your server**.

He **does stuff** when you **ask him to**.

You can **talk to him** and he can **talk to you**.
You can also **text him** and he can also **text you**.
If you want to, Leon can communicate with you by being **offline to protect your privacy**.

### Why?

> 1. If you are a developer (or not), you may want to build many things that could help in your daily life.
>    Instead of building a dedicated project for each of those ideas, Leon can help you with his
>    Skills structure.
> 2. With this generic structure, everyone can create their own skills and share them with others.
>    Therefore there is only one core (to rule them all).
> 3. Leon uses AI concepts, which is cool.
> 4. Privacy matters, you can configure Leon to talk with him offline. You can already text with him without any third party services.
> 5. Open source is great.

### What is this repository for?

> This repository contains the following nodes of Leon:
>
> - The server
> - Skills
> - The web app
> - The hotword node
> - The TCP server (for inter-process communication between Leon and third-party nodes such as spaCy)
> - The Python bridge (the connector between the core and skills made with Python)

### What is Leon able to do?

> Today, the most interesting part is about his core and the way he can scale up. He is pretty young but can easily scale to have new features (skills).
> You can find what he is able to do by browsing the [skills list](https://github.com/leon-ai/leon/tree/develop/skills).

> Please do know that after the official release, we will build many skills along with the community. Feel free to [join us on Discord](https://discord.gg/MNQqqKg) to be part of the journey.

Sounds good to you? Then let's get started!

## ☁ Try with a Single-Click

Gitpod will automatically set up an environment and run an instance for you.

[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/leon-ai/leon)

## 🚀 Getting Started

### Prerequisites

- [Node.js](https://nodejs.org/) >= 16
- [npm](https://npmjs.com/) >= 8
- Supported OSes: Linux, macOS and Windows

To install these prerequisites, you can follow the [How To section](https://docs.getleon.ai/how-to/) of the documentation.

### Installation

```sh
# Install the Leon CLI
npm install --global @leon-ai/cli

# Install Leon (stable branch)
leon create birth
# OR install from the develop branch: leon create birth --develop
```

### Usage

```sh
# Check the setup went well
leon check

# Run
leon start

# Go to http://localhost:1337
# Hooray! Leon is running
```

## 📚 Documentation

For full documentation, visit [docs.getleon.ai](https://docs.getleon.ai).

## 📺 Video

[Watch a demo](https://www.youtube.com/watch?v=p7GRGiicO1c).

## 🧭 Roadmap

To know what is going on, follow [roadmap.getleon.ai](http://roadmap.getleon.ai).

## ❤ Contributing

If you have an idea for improving Leon, do not hesitate.

**Leon needs open source to live**, the more skills he has, the more skillful he becomes.

## 📖 The Story Behind Leon

You'll find a write-up on this [blog post](https://blog.getleon.ai/the-story-behind-leon/).

## 🔔 Stay Tuned

- [Twitter](https://twitter.com/grenlouis)
- [Newsletter](https://newsletter.getleon.ai/subscription/form)
- [Blog](https://blog.getleon.ai)
- [GitHub issues](https://github.com/leon-ai/leon/issues)
- [YouTube](https://www.youtube.com/channel/UCW6mk6j6nQUzFYY97r47emQ)
- [#LeonAI](<https://twitter.com/search?f=live&q=%23LeonAI%20(from%3Agrenlouis%20OR%20from%3Alouistiti_fr)&src=typed_query>)

## 👨 Author

**Louis Grenard** ([@grenlouis](https://twitter.com/grenlouis))

## 👍 Sponsors

          Appwrite

        250 USD / month

        Anonymous

        100 USD / month

          herbundkraut

        10 USD / month

          You?

You can also contribute by [sponsoring Leon](http://sponsor.getleon.ai).

Please note that I dedicate most of my free time to Leon.

By sponsoring the project you make the project sustainable and faster to develop features.

The focus is not only limited to the activity you see on GitHub but also a lot of thinking about the direction of the project. Which is naturally related to the overall design, architecture, vision, learning process and so on...

### Special Thanks

## 📝 License

[MIT License](https://github.com/leon-ai/leon/blob/develop/LICENSE.md)

Copyright (c) 2019-present, Louis Grenard <louis@getleon.ai>

## Cheers!

![Cheers!](https://github.githubassets.com/images/icons/emoji/unicode/1f379.png 'Cheers!')


# Via

An app that proxies web pages and PDF files and injects the Hypothesis client so you can annotate them.

## Setting up Your Via Development Environment

First you'll need to install:

* [Git](https://git-scm.com/).
  On Ubuntu: `sudo apt install git`, on macOS: `brew install git`.
* [GNU Make](https://www.gnu.org/software/make/).
  This is probably already installed, run `make --version` to check.
* [pyenv](https://github.com/pyenv/pyenv).
  Follow the instructions in pyenv's README to install it.
  The **Homebrew** method works best on macOS.
  The **Basic GitHub Checkout** method works best on Ubuntu.
  You _don't_ need to set up pyenv's shell integration ("shims"), you can
  [use pyenv without shims](https://github.com/pyenv/pyenv#using-pyenv-without-shims).
* [Docker Desktop](https://www.docker.com/products/docker-desktop/).
  On Ubuntu follow [Install on Ubuntu](https://docs.docker.com/desktop/install/ubuntu/).
  On macOS follow [Install on Mac](https://docs.docker.com/desktop/install/mac-install/).
* [Node](https://nodejs.org/) and npm.
  On Ubuntu: `sudo snap install --classic node`.
  On macOS: `brew install node`.
* [Yarn](https://yarnpkg.com/): `sudo npm install -g yarn`.

Then to set up your development environment:

```terminal
git clone https://github.com/hypothesis/via.git
cd via
make services
make devdata
make help
```

To run Via locally run `make dev` and visit http://localhost:9083.

## Changing the Project's Python Version

To change what version of Python the project uses:

1. Change the Python version in the
   [cookiecutter.json](.cookiecutter/cookiecutter.json) file. For example:

   ```json
   "python_version": "3.10.4",
   ```

2. Re-run the cookiecutter template:

   ```terminal
   make template
   ```

3. Re-compile the `requirements/*.txt` files.
   This is necessary because the same `requirements/*.in` file can compile to
   different `requirements/*.txt` files in different versions of Python:

   ```terminal
   make requirements
   ```

4. Commit everything to git and send a pull request

## Changing the Project's Python Dependencies

### To Add a New Dependency

Add the package to the appropriate [`requirements/*.in`](requirements/)
file(s) and then run:

```terminal
make requirements
```

### To Remove a Dependency

Remove the package from the appropriate [`requirements/*.in`](requirements)
file(s) and then run:

```terminal
make requirements
```

### To Upgrade or Downgrade a Dependency

We rely on [Dependabot](https://github.com/dependabot) to keep all our
dependencies up to date by sending automated pull requests to all our repos.
But if you need to upgrade or downgrade a package manually you can do that
locally.

To upgrade a package to the latest version in all `requirements/*.txt` files:

```terminal
make requirements --always-make args='--upgrade-package <FOO>'
```

To upgrade or downgrade a package to a specific version:

```terminal
make requirements --always-make args='--upgrade-package <FOO>==<X.Y.Z>'
```

To upgrade **all** packages to their latest versions:

```terminal
make requirements --always-make args=--upgrade
```

Configuration
-------------

Environment variables:

| Name                       | Purpose                                                                                | Example                             |
|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|
| `CHECKMATE_URL`            | The URL of the URL Checkmate instance to use                                                                                | `https://checkmate.example.com`     |
| `CHECKMATE_API_KEY`        | API key to authenticate with Checkmate                                                                                |                                     |
| `CHECKMATE_ALLOW_ALL`      | Whether to bypass Checkmate's allow-list (and use only the blocklist)                                                                                | `true`                              |
| `CHECKMATE_IGNORE_REASONS` | Comma-separated list of Checkmate block reasons to ignore                                                                                | `publisher-blocked,high-io`         |
| `CLIENT_EMBED_URL`         | The URL of the client's embed script                                                                                | `https://hypothes.is/embed.js`      |
| `DATA_DIRECTORY`           | Directory for externally provided data                                                                                | `/via-data`                         |
| `ENABLE_FRONT_PAGE`        | Show a front page at the root URL                                                                                | `true`                              |
| `NEW_RELIC_*`              | Various New Relic settings. See New Relic's docs for details                                                                                |                                     |
| `NGINX_SECURE_LINK_SECRET` | The NGINX secure links signing secret. This is used by Via's Python endpoints to generate the signed URLs required by its NGINX-implemented `/proxy/static/` endpoint. All instances of Via must have this setting                                                  |                                     |
| `NGINX_SERVER`             | The URL of Via's NGINX server for proxying PDF files                                                                                | `https://via.hypothes.is`           |
| `SENTRY_*`                 | Various Sentry settings. See Sentry's docs for details                                                                                |                                     |
| `SIGNED_URLS_REQUIRED`     | Require URLs to Via's Python endpoints to be signed so that Via can only be used by something that has the URL signing secret. Public instances of Via should _not_ enable this. Private instances of Via (e.g. the LMS app's instance of Via) _should_ enable this | `true`                              |
| `VIA_HTML_URL`             | The URL of the Via HTML instance to redirect to for proxying HTML pages                                                                                | `https://viahtml.hypothes.is/proxy` |
| `VIA_SECRET`               | The secret that must be used to sign URLs to Via's Python endpoints if `SIGNED_URLS_REQUIRED` is on                                                                                |                                     |

Expected data:

The following data is expected to be provided in the `DATA_DIRECTORY`:

 * `google_drive_credentials.json` - A list of credential JSON objects provided by the Google API console
 * `google_drive_resource_keys.json` - A dict of file ids to resource keys

Error codes
-----------

Most error codes have their natural meanings, but there are some special cases
that we catch. Those marged with **(error)** are from our point of view
something which is worth investigation. Others are a normal part of the
day-to-day.

### All end-points:

 * 401 - The user needs a secure token and either they don't have one, or it's
   invalid

### General end-points:

 * 400 - The user has make a mistake in calling us or given us a bad URL
 * 408 - We timed out accessing a website
 * 409 - A website we have called gave us a conventional error like a
   connection error. Trying again could help.
 * 417 - A website has given us an unexpected response. This could be anything.
   Trying again could help.

### `/google_drive/*`

 * 403 - The user has given us a URL which doesn't grant us permission to
   download it
 * 404 - The user has given us an invalid URL or one we _really_ don't have
   permission to see
 * **408 (error)** - We timed out trying to make a connection to Google
 * **409 (error)** - We had a conventional error like a connection error
 * **417 (error)** - We got an unexpected response from Google
 * **423 (error)** - Google has blocked the file as malicous
 * **429 (error)** - We have been rate limited by Google

Updating the PDF viewer
-----------------------

Via serves PDFs using [PDF.js](https://mozilla.github.io/pdf.js/). PDF.js is
vendored into the source tree and the viewer HTML is patched to load the Hypothesis
client. To update the PDF viewer, run `make update-pdfjs`.

How Via works
-------------

Via allows users to annotate arbitrary web pages or PDF files by proxying the
page or file and injecting the Hypothesis client. Users go to
<https://via.hypothes.is/> and paste in a PDF or HTML URL (or visit
`https://via.hypothes.is/<SOME_URL>` directly) and Via responds with an
annotatable version.

### Via's architecture

Via is composed of four separable components:

1. A **top-level component** that responds to requests to the top-level
   `/<THIRD_PARTY_URL>` endpoint by deciding whether the URL is a PDF file or
   not and redirecting the browser to either the PDF viewer component or the
   HTML proxying component accordingly.

   This component is implemented in Python / Pyramid.

   The Pyramid app sends a GET request to the third-party URL but only
   downloads the response headers not the body. It looks at the Content-Type
   header to determine whether the body is a PDF file or not.

   If it's a PDF file then it redirects to the PDF viewer component:
   `/pdf/<THIRD_PARTY_URL>`.

   If it's an HTML file then it redirects to the HTML proxy component.

   The Pyramid app also handles various other bits and bobs such as serving up
   the front page, handling special `via.*` query params, serving static files
   such as PDF.js's assets, etc etc.

2. A **PDF viewer component** that renders a modified version of PDF.js with the Hypothesis client embedded.

   This is what enables users to annotate PDF files.

   The PDF viewer is also implemented in Python / Pyramid (and JavaScript served by the Pyramid app).

   The PDF viewer responds to requests to `/pdf/<THIRD_PARTY_URL>` by rendering
   a version of [PDF.js](https://mozilla.github.io/pdf.js/) with the Hypothesis
   client embedded, and configuring PDF.js to download the PDF file from the
   static file proxy component.

3. A **static files proxy component** that simply proxies static files to get around CORS.

   This component is implemented in NGINX (in the [nginx.conf](nginx/nginx.conf) file) for efficiency.

   This component responds to requests to the `/proxy/static/<THIRD_PARTY_URL>`
   endpoint, such as PDF.js's download requests for PDF files.

   Many PDF hosts use
   [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) headers to
   prevent JavaScript cross-origin requests (such as requests from our copy of
   PDF.js) from downloading the file.
   See [Can I load a PDF from another server (cross domain request)?](https://github.com/mozilla/pdf.js/wiki/Frequently-Asked-Questions#can-i-load-a-pdf-from-another-server-cross-domain-request)
   in the PDF.js FAQ.

   To get around this we proxy the PDF file through our own server so that
   browsers no longer see PDF.js's download request as a cross-origin request.

   In the future we'll also use this component to proxy some static resources
   of web pages for the same reason.

4. A **rewriting HTML proxy component** that proxies HTML pages and injects the Hypothesis client.

   This is what enables users to annotate web pages.

   The HTML proxy isn't implemented yet. Via currently redirects to legacy Via for HTML proxying.

   The HTML proxy's job is to enable annotating of HTML pages by proxying the
   page and injecting the Hypothesis client into it.

   It also has to rewrite various elements of the page that would otherwise
   break because the page is being proxied.

### How Via works in production

In production both NGINX and Gunicorn (the WSGI server for the Python / Pyramid
app) run inside a single Docker container defined by the app's `Dockerfile`.

NGINX runs on port 9083 in the Docker container, which is exposed to the
outside world.

Gunicorn runs on a UNIX socket that is accessible to NGINX within the Docker
container but is not directly accessible to the outside world.

NGINX is "in front of" Gunicorn in production:

1. All requests from user's browsers first go to NGINX on the Docker container's port 9083.

2. If the request is to a URL that NGINX handles directly (such as a
   `/proxy/static/*` URL) then NGINX just responds directly.

3. If the request is to one of the URLs that should be handled by the Pyramid
   app then NGINX proxies to Gunicorn on a UNIX socket.

### How Via works in development

In development NGINX runs in Docker Compose and is exposed at
http://localhost:9083/. This is defined in `docker-compose.yml`. The app's
`Dockerfile` isn't used in development, but the NGINX running in Docker Compose
in development does use the same `nginx.conf` file as the NGINX running in
Docker in production.

The Python WSGI server (Gunicorn) runs on the host (no Docker) and is exposed
at http://localhost:9082/. The NGINX running on `:9083` proxies to the Gunicorn
on `:9082`.

### WhiteNoise

The Pyramid app uses [WhiteNoise](http://whitenoise.evans.io/) to serve static
files in a CDN-friendly (caching-friendly) way. WhiteNoise serves the Python
app's static files in an efficient way and with the appropriate caching headers,
compression, etc.

WhiteNoise is a piece of [WSGI middleware](https://www.python.org/dev/peps/pep-3333/#middleware-components-that-play-both-sides)
that wraps our Pyramid WSGI app. Rather than proxying to Pyramid directly
Gunicorn actually proxies to WhiteNoise which either responds directly (if the
request is for a static file) or proxies to Pyramid.

### See also

* [Caching strategy](docs/caching-strategy.md)


# Hackerspace Website
[![Python Integration](https://github.com/hackerspace-ntnu/website/actions/workflows/integration.yml/badge.svg)](https://github.com/hackerspace-ntnu/website/actions/workflows/integration.yml)

The code running [hackerspace-ntnu.no](http://hackerspace-ntnu.no).

## Prerequisites

To get started developing, make sure you have the correct software installed.

This projects uses Django with Python 3. Make sure you have Python3 and pip3 installed.

### For Linux

```
sudo apt install python3 python3-pip python3-venv
```

### For Windows
Alternatives:
1. Install Python3 manually and use CMD

2. Use an environment such as Cygwin64, select pip3 and python3 during install.

3. Install Linux Subsystem for Windows and install Ubuntu (or other prefered distro) from Windows Store

## Set up a virtual environment with virtualenv

Instead of installing a bunch of python files system-wide, we will use a virtual environment to install the packages in a single folder instead.

### For Linux/Mac

1. Create virtualenv:
   ```
   python3 -m venv NAME_OF_VENV
   ```

2. Activate virtualenv:
   ```
   source NAME_OF_VENV/bin/activate
   ```

### For Windows
1. Create virtualenv:
   ```
   python -m venv NAME_OF_VENV
   ```

2. Activate virtualenv:
   ```
   .\NAME_OF_VENV\Scripts\activate
   ```

## Download and initialize the project

Clone the project:
```
git clone git@github.com:hackerspace-ntnu/website.git
```

Go into the project:
```
cd website/
```

Install required python packages:
```
pip install -r requirements.txt
```

Install pre-commit hooks for automatic code formatting and various code style checks:
```
pre-commit install
```

Migrate the database:
```
python manage.py migrate
```

After installing the required packages and initializing the database, you can run the server with the following command:
```
python manage.py runserver
```

## Fixtures

Use fixtures to add test data to you local development

Load the fixtures:
```
python manage.py loaddata fixtures.json
```

Create new fixtures:
(Create fixtures if model updates make the fixtures unusable)
```
python manage.py dumpdata -e admin -e auth.Permission -e contenttypes -e sessions --indent=4 > fixtures.json
```

All users have the password: `adminpassword`

## Reset database

You can delete the database locally by deleting the db.sqlite3 file from the root directory

After that you need to migrate the database with
```
python manage.py migrate
```

Follow the above step about fixtures if you want your test data back

## Translations

We have locality for Norwegian and English
To generate new translations in `.po` files run
```
python manage.py makemessages -l nb -l en
```

Add the correct translations in the `msgstr` quotes.
If a translation is not given and the `msgstr` quote is empty, the msgid will be used

To compile the translations and make the translations available run
```
python manage.py compilemessages
```