Interactive Redis: A Cli for Redis with AutoCompletion and Syntax Highlighting.

IRedis is a terminal client for redis with auto-completion and syntax
highlighting. IRedis lets you type Redis commands smoothly, and displays results
in a user-friendly format.

IRedis is an alternative for redis-cli. In most cases, IRedis behaves exactly
the same as redis-cli. Besides, it is safer to use IRedis on production servers
than redis-cli: IRedis will prevent accidentally running dangerous commands,
like `KEYS *` (see
[Redis docs / Latency generated by slow commands](https://redis.io/topics/latency#latency-generated-by-slow-commands)).

## Features

- Advanced code completion. If you run command `KEYS` then run `DEL`, IRedis
  will auto-complete your command based on `KEYS` result.
- Command validation. IRedis will validate command while you are typing, and
  highlight errors. E.g. try `CLUSTER MEET IP PORT`, IRedis will validate IP and
  PORT for you.
- Command highlighting, fully based on redis grammar. Any valid command in
  IRedis shell is a valid redis command.
- Human-friendly result display.
- _pipeline_ feature, you can use your favorite shell tools to parse redis'
  response, like `get json | jq .`.
- Support pager for long output.
- Support connection via URL, `iredis --url redis://example.com:6379/1`.
- Support cluster, IRedis will auto reissue command for `MOVED` response in
  cluster mode.
- Store server configuration: `iredis -d prod-redis` (see [dsn](#using-dsn) for
  more).
- `peek` command to check the key's type then automatically call
  `get`/`lrange`/`sscan`, etc, depending on types. You don't need to call the
  `type` command then type another command to get the value. `peek` will also
  display the key's length and memory usage.
- Ctrl + C to cancel the current typed command, this won't
  exit IRedis, exactly like bash behaviour. Use Ctrl + D
  to send a EOF to exit IRedis.
- Ctrl + R to open **reverse-i-search** to search through
  your command history.
- Auto suggestions. (Like [fish shell](http://fishshell.com/).)
- Support `--encode=utf-8`, to decode Redis' bytes responses.
- Command hint on bottom, include command syntax, supported redis version, and
  time complexity.
- Official docs with built-in `HELP` command, try `HELP SET`!
- Written in pure Python, but IRedis was packaged into a single binary with
  [PyOxidizer](https://github.com/indygreg/PyOxidizer), you can use cURL to
  download and run, it just works, even you don't have a Python interpreter.
- You can change the cli prompt using `--prompt` option or set via `~/.iredisrc`
  config file.
- Hide password for `AUTH` command.
- Says "Goodbye!" to you when you exit!
- For full features, please see: [iredis.xbin.io](https://www.iredis.xbin.io)

## Install

### Pip

Install via pip:

```
pip install iredis
```

[pipx](https://github.com/pipxproject/pipx) is recommended:

```
pipx install iredis
```

### Brew

For Mac users, you can install iredis via brew 🍻

```
brew install iredis
```

### Linux

You can also use your Linux package manager to install IRedis, like `apt` in
Ubuntu (Only available on Ubuntu 21.04+).

```shell
apt install iredis
```

[![Packaging status](https://repology.org/badge/vertical-allrepos/iredis.svg)](https://repology.org/project/iredis/versions)

### Download Binary

Or you can download the executable binary with cURL(or wget), untar, then run.
It is especially useful when you don't have a python interpreter(E.g. the
[official Redis docker image](https://hub.docker.com/_/redis/) which doesn't
have Python installed.):

```
wget  https://github.com/laixintao/iredis/releases/latest/download/iredis.tar.gz \
 && tar -xzf iredis.tar.gz \
 && ./iredis
```

(Check the [release page](https://github.com/laixintao/iredis/releases) if you
want to download an old version of IRedis.)

## Usage

Once you install IRedis, you will know how to use it. Just remember, IRedis
supports similar options like redis-cli, like `-h` for redis-server's host and
`-p` for port.

```
$ iredis --help

Usage: iredis [OPTIONS] [CMD]...

  IRedis: Interactive Redis

  When no command is given, IRedis starts in interactive mode.

  Examples:
    - iredis
    - iredis -d dsn
    - iredis -h 127.0.0.1 -p 6379
    - iredis -h 127.0.0.1 -p 6379 -a <password>
    - iredis --url redis://localhost:7890/3

  Type "help" in interactive mode for information on available commands and
  settings.

Options:
  -h TEXT                         Server hostname (default: 127.0.0.1).
  -p TEXT                         Server port (default: 6379).
  -s, --socket TEXT               Server socket (overrides hostname and port).
  -n INTEGER                      Database number.(overwrites dsn/url's db
                                  number)

  -u, --username TEXT             User name used to auth, will be ignore for
                                  redis version < 6.

  -a, --password TEXT             Password to use when connecting to the
                                  server.

  --url TEXT                      Use Redis URL to indicate connection(Can set
                                  with env `IREDIS_URL`), Example:     redis:/
                                  /[[username]:[password]]@localhost:6379/0
                                  rediss://[[username]:[password]]@localhost:6
                                  379/0     unix://[[username]:[password]]@/pa
                                  th/to/socket.sock?db=0

  -d, --dsn TEXT                  Use DSN configured into the [alias_dsn]
                                  section of iredisrc file. (Can set with env
                                  `IREDIS_DSN`)

  --newbie / --no-newbie          Show command hints and useful helps.
  --iredisrc TEXT                 Config file for iredis, default is
                                  ~/.iredisrc.

  --decode TEXT                   decode response, default is No decode, which
                                  will output all bytes literals.

  --client_name TEXT              Assign a name to the current connection.
  --raw / --no-raw                Use raw formatting for replies (default when
                                  STDOUT is not a tty). However, you can use
                                  --no-raw to force formatted output even when
                                  STDOUT is not a tty.

  --rainbow / --no-rainbow        Display colorful prompt.
  --shell / --no-shell            Allow to run shell commands, default to
                                  True.

  --pager / --no-pager            Using pager when output is too tall for your
                                  window, default to True.

  --verify-ssl [none|optional|required]
                                  Set the TLS certificate verification
                                  strategy

  --prompt TEXT                   Prompt format (supported interpolations:
                                  {client_name}, {db}, {host}, {path}, {port},
                                  {username}, {client_addr}, {client_id}).

  --version                       Show the version and exit.
  --help                          Show this message and exit.

```

### Using DSN

IRedis support storing server configuration in config file. Here is a DSN
config:

```
[alias_dsn]
dev=redis://localhost:6379/4
staging=redis://username:password@staging-redis.example.com:6379/1
```

Put this in your `iredisrc` then connect via `iredis -d staging` or
`iredis -d dev`.

### Change The Default Prompt

You can change the prompt str, the default prompt is:

```shell
127.0.0.1:6379>
```

Which is rendered by `{host}:{port}[{db}]> `, you can change this via `--prompt`
option or change
[iredisrc](https://github.com/laixintao/iredis/blob/master/iredis/data/iredisrc)
config file. The prompwt string uses python string format engine, supported
interpolations:

- `{client_name}`
- `{db}`
- `{host}`
- `{path}`
- `{port}`
- `{username}`
- `{client_addr}`
- `{client_id}`

The `--prompt` utilize
[Python String format engine](https://docs.python.org/3/library/string.html#formatstrings),
so as long as it is a valid string formatter, it will work( anything that
`"<your prompt>".format(...)` accepts). For example, you can limit your Redis
server host name's length to 5 by setting `--prompt` to
`iredis --prompt '{host:.5s}'`.

### Configuration

IRedis supports config files. Command-line options will always take precedence
over config. Configuration resolution from highest to lowest precedence is:

- _Options from command line_
- `$PWD/.iredisrc`
- `~/.iredisrc` (this path can be changed with `iredis --iredisrc $YOUR_PATH`)
- `/etc/iredisrc`
- default config in IRedis package.

You can copy the _self-explained_ default config here:

https://raw.githubusercontent.com/laixintao/iredis/master/iredis/data/iredisrc

And then make your own changes.

(If you are using an old versions of IRedis, please use the config file below,
and change the version in URL):

https://raw.githubusercontent.com/laixintao/iredis/v1.0.4/iredis/data/iredisrc

### Keys

IRedis support unix/readline-style REPL keyboard shortcuts, which means keys
like Ctrl + F to forward work.

Also:

- Ctrl + D (i.e. EOF) to exit; you can also use the `exit`
  command.
- Ctrl + L to clear screen; you can also use the `clear`
  command.
- Ctrl + X Ctrl + E to open an
  editor to edit command, or V in vi-mode.

## Development

### Release Strategy

IRedis is built and released by `GitHub Actions`. Whenever a tag is pushed to
the `master` branch, a new release is built and uploaded to pypi.org, it's very
convenient.

Thus, we release as often as possible, so that users can always enjoy the new
features and bugfixes quickly. Any bugfix or new feature will get at least a
patch release, whereas big features will get a minor release.

### Setup Environment

IRedis favors [poetry](https://github.com/sdispater/poetry) as package
management tool. To setup a develop environment on your computer:

First, install poetry (you can do it in a python's virtualenv):

```
pip install poetry
```

Then run (which is similar to `pip install -e .`):

```
poetry install
```

**Be careful running testcases locally, it may flush you db!!!**

### Development Logs

This is a command-line tool, so we don't write logs to stdout.

You can `tail -f ~/.iredis.log` to see logs, the log is pretty clear, you can
see what actually happens from log files.

### Catch Up with Latest Redis-doc

IRedis use a git submodule to track current-up-to-date redis-doc version. To
catch up with latest:

1. Git pull in redis-doc
2. Copy doc files to `/data`: `cp -r redis-doc/commands* iredis/data`
3. Prettier
   markdown`prettier --prose-wrap always iredis/data/commands/*.md --write`
4. Check the diff, update IRedis' code if needed.

## Related Projects

- [redis-tui](https://github.com/mylxsw/redis-tui)

If you like iredis, you may also like other cli tools by
[dbcli](https://www.dbcli.com/):

- [pgcli](https://www.pgcli.com) - Postgres Client with Auto-completion and
  Syntax Highlighting
- [mycli](https://www.mycli.net) - MySQL/MariaDB/Percona Client with
  Auto-completion and Syntax Highlighting
- [litecli](https://litecli.com) - SQLite Client with Auto-completion and Syntax
  Highlighting
- [mssql-cli](https://github.com/dbcli/mssql-cli) - Microsoft SQL Server Client
  with Auto-completion and Syntax Highlighting
- [athenacli](https://github.com/dbcli/athenacli) - AWS Athena Client with
  Auto-completion and Syntax Highlighting
- [vcli](https://github.com/dbcli/vcli) - VerticaDB client
- [iredis](https://github.com/laixintao/iredis/) - Client for Redis with
  AutoCompletion and Syntax Highlighting

IRedis is build on the top of
[prompt_toolkit](https://github.com/jonathanslenders/python-prompt-toolkit), a
Python library (by [Jonathan Slenders](https://twitter.com/jonathan_s)) for
building rich commandline applications.


*******
webargs
*******

|pypi| |build-status| |docs| |marshmallow3|

.. |pypi| image:: https://badgen.net/pypi/v/webargs
    :target: https://pypi.org/project/webargs/
    :alt: PyPI package

.. |build-status| image:: https://github.com/marshmallow-code/webargs/actions/workflows/build-release.yml/badge.svg
    :target: https://github.com/marshmallow-code/webargs/actions/workflows/build-release.yml
    :alt: Build status

.. |docs| image:: https://readthedocs.org/projects/webargs/badge/
   :target: https://webargs.readthedocs.io/
   :alt: Documentation

.. |marshmallow3| image:: https://badgen.net/badge/marshmallow/3
    :target: https://marshmallow.readthedocs.io/en/latest/upgrading.html
    :alt: marshmallow 3 compatible

Homepage: https://webargs.readthedocs.io/

webargs is a Python library for parsing and validating HTTP request objects, with built-in support for popular web frameworks, including Flask, Django, Bottle, Tornado, Pyramid, Falcon, and aiohttp.

.. code-block:: python

    from flask import Flask
    from webargs import fields
    from webargs.flaskparser import use_args

    app = Flask(__name__)

    @app.route("/")
    @use_args({"name": fields.Str(required=True)}, location="query")
    def index(args):
        return "Hello " + args["name"]

    if __name__ == "__main__":
        app.run()

    # curl http://localhost:5000/\?name\='World'
    # Hello World

Install
=======

::

    pip install -U webargs

webargs supports Python >= 3.8.

Documentation
=============

Full documentation is available at https://webargs.readthedocs.io/.

Support webargs
===============

webargs is maintained by a group of
`volunteers <https://webargs.readthedocs.io/en/latest/authors.html>`_.
If you'd like to support the future of the project, please consider
contributing to our Open Collective:

.. image:: https://opencollective.com/marshmallow/donate/button.png
    :target: https://opencollective.com/marshmallow
    :width: 200
    :alt: Donate to our collective

Professional Support
====================

Professionally-supported webargs is available through the
`Tidelift Subscription <https://tidelift.com/subscription/pkg/pypi-webargs?utm_source=pypi-webargs&utm_medium=referral&utm_campaign=readme>`_.

Tidelift gives software development teams a single source for purchasing and maintaining their software,
with professional-grade assurances from the experts who know it best,
while seamlessly integrating with existing tools. [`Get professional support`_]

.. _`Get professional support`: https://tidelift.com/subscription/pkg/pypi-webargs?utm_source=pypi-webargs&utm_medium=referral&utm_campaign=readme

.. image:: https://user-images.githubusercontent.com/2379650/45126032-50b69880-b13f-11e8-9c2c-abd16c433495.png
    :target: https://tidelift.com/subscription/pkg/pypi-webargs?utm_source=pypi-webargs&utm_medium=referral&utm_campaign=readme
    :alt: Get supported marshmallow with Tidelift

Security Contact Information
============================

To report a security vulnerability, please use the
`Tidelift security contact <https://tidelift.com/security>`_.
Tidelift will coordinate the fix and disclosure.

Project Links
=============

- Docs: https://webargs.readthedocs.io/
- Changelog: https://webargs.readthedocs.io/en/latest/changelog.html
- Contributing Guidelines: https://webargs.readthedocs.io/en/latest/contributing.html
- PyPI: https://pypi.python.org/pypi/webargs
- Issues: https://github.com/marshmallow-code/webargs/issues
- Ecosystem / related packages: https://github.com/marshmallow-code/webargs/wiki/Ecosystem

License
=======

MIT licensed. See the `LICENSE <https://github.com/marshmallow-code/webargs/blob/dev/LICENSE>`_ file for more details.


# python.org

[![CI](https://github.com/python/pythondotorg/actions/workflows/ci.yml/badge.svg)](https://github.com/python/pythondotorg/actions/workflows/ci.yml)
[![Documentation Status](https://readthedocs.org/projects/pythondotorg/badge/?version=latest)](https://pythondotorg.readthedocs.io/?badge=latest)

### General information

This is the repository and issue tracker for [python.org](https://www.python.org).
The repository for CPython itself is at https://github.com/python/cpython, and the
issue tracker is at https://github.com/python/cpython/issues/.

Issues related to [Python's documentation](https://docs.python.org) can be filed in
https://github.com/python/cpython/issues/.

### Contributing

* Source code: https://github.com/python/pythondotorg
* Issue tracker: https://github.com/python/pythondotorg/issues
* Documentation: https://pythondotorg.readthedocs.org/
* Mailing list: [pydotorg-www](https://mail.python.org/mailman/listinfo/pydotorg-www)
* IRC: `#pydotorg` on Freenode
* Staging site: https://staging.python.org/ (`main` branch)
* License: Apache License


# Yeti Platform

Yeti aims to bridge the gap between CTI and DFIR practitioners by providing a
Forensics Intelligence platform and pipeline for DFIR teams. It was born out of frustration
of having to answer the question "where have I seen this artifact before?" or
"how do I search for IOCs related to this threat (or all threats?) in my timeline?"

Documentation links:

* Main website: https://yeti-platform.io/
* [Documentation](https://yeti-platform.io/docs)
* [Guides](https://yeti-platform.io/guides)

![](https://yeti-platform.io/logo.png)

## What is Yeti?

In a nutshell, Yeti allows you to:

- Bulk search observables and get a pretty good guess on the nature of the
  threat, and how to find it on a system.
- Inversely, focus on a threat and quickly list all TTPs, malware, and related
  DFIR artifacts.
- Let CTI analysts focus on adding intelligence rather than worrying about
  machine-readable export formats.
- Incorporate your own data sources, analytics, and logic very easily.

This is done by:

- Storing technical and tactical CTI (observables, TTPs, campagins, etc.) from
  internal or external systems.
- Being a backend for DFIR-related queries: Yara signatures, Sigma rules, DFIQ.
- Providing a web API to automate queries (think incident management platform)
  and enrichment (think malware sandbox).
- Export the data in user-defined formats so that they can be ingested by
  third-party applications (SIEM, DFIR platforms).

## Some screenshots

![](https://yeti-platform.io/scattered.png)

![](https://yeti-platform.io/attack.png)

![](https://yeti-platform.io/vuln.png)

![](https://yeti-platform.io/intrusionset.png)


![vlogo](https://user-images.githubusercontent.com/32848391/110344277-9bc20700-802d-11eb-8c0d-2e97226a9a32.png)

[![lics](https://img.shields.io/badge/license-MIT-blue.svg)](https://en.wikipedia.org/wiki/MIT_License)
[![Anaconda-Server Badge](https://anaconda.org/conda-forge/vedo/badges/version.svg)](https://anaconda.org/conda-forge/vedo)
[![Ubuntu 23.10 package](https://repology.org/badge/version-for-repo/ubuntu_23_10/vedo.svg)](https://repology.org/project/vedo/versions)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4587871.svg)](https://doi.org/10.5281/zenodo.4587871)
[![Downloads](https://static.pepy.tech/badge/vedo)](https://pepy.tech/project/vedo)
[![CircleCI](https://circleci.com/gh/marcomusy/vedo.svg?style=svg)](https://circleci.com/gh/marcomusy/vedo)

A lightweight and powerful python module
for scientific analysis and **v**isualization of **3d** **o**bjects.

## 💾  Installation
```bash
pip install vedo
```

additional installation details [click to expand]

- To install the latest _dev_ version of `vedo`:

```bash
pip install -U git+https://github.com/marcomusy/vedo.git
```

- To install from the conda-forge channel:

```bash
conda install -c conda-forge vedo
```

## 📙  Documentation
The webpage of the library with documentation is available [**here**](https://vedo.embl.es).

📌 **Need help? Have a question, or wish to ask for a missing feature?**
Do not hesitate to ask any questions on the [**image.sc** forum](https://forum.image.sc/)
or by opening a [**github issue**](https://github.com/marcomusy/vedo/issues).

## 🎨  Features
The library includes a [large set of working examples](https://github.com/marcomusy/vedo/tree/master/examples)
for a wide range of functionalities

working with polygonal meshes and point clouds [click to expand]

- Import meshes from VTK format, STL, Wavefront OBJ, 3DS, Dolfin-XML, Neutral, GMSH, OFF, PCD (PointCloud),
- Export meshes as ASCII or binary to VTK, STL, OBJ, PLY ... formats.
- Analysis tools like Moving Least Squares, mesh morphing and more..
- Tools to visualize and edit meshes (cutting a mesh with another mesh, slicing, normalizing, moving vertex positions, etc..).
- Split mesh based on surface connectivity. Extract the largest connected area.
- Calculate areas, volumes, center of mass, average sizes etc.
- Calculate vertex and face normals, curvatures, feature edges. Fill mesh holes.
- Subdivide faces of a mesh, increasing the number of vertex points. Mesh simplification.
- Coloring and thresholding of meshes based on associated scalar or vectorial data.
- Point-surface operations: find nearest points, determine if a point lies inside or outside of a mesh.
- Create primitive shapes: spheres, arrows, cubes, torus, ellipsoids...
- Generate glyphs (associate a mesh to every vertex of a source mesh).
- Create animations easily by just setting the position of the displayed objects in the 3D scene. Add trailing lines and shadows to moving objects is supported.
- Straightforward support for multiple sync-ed or independent renderers in  the same window.
- Registration (alignment) of meshes with different techniques.
- Mesh smoothing.
- Delaunay triangulation in 2D and 3D.
- Generate meshes by joining nearby lines in space.
- Find the closest path from one point to another, traveling along the edges of a mesh.
- Find the intersection of a mesh with lines, planes or other meshes.
- Interpolate scalar and vectorial fields with Radial Basis Functions and Thin Plate Splines.
- Add sliders and buttons to interact with the scene and the individual objects.
- Visualization of tensors.
- Analysis of Point Clouds
- Moving Least Squares smoothing of 2D, 3D and 4D clouds
- Fit lines, planes, spheres and ellipsoids in space
- Identify outliers in a distribution of points
- Decimate a cloud to a uniform distribution.

working with volumetric data and tetrahedral meshes

- Import data from VTK format volumetric TIFF stacks, DICOM, SLC, MHD and more
- Import 2D images as PNG, JPEG, BMP
- Isosurfacing of volumes
- Composite and maximum projection volumetric rendering
- Generate volumetric signed-distance data from an input surface mesh
- Probe volumes with lines and planes
- Generate stream-lines and stream-tubes from vectorial fields
- Slice and crop volumes
- Support for other volumetric structures (structured and grid data)

plotting and histogramming in 2D and 3D

- Polygonal 3D text rendering with Latex-like syntax and unicode characters, with 30 different fonts.
- Fully customizable axis styles
- donut plots and pie charts
- Scatter plots in 2D and 3D
- Surface function plotting
- 1D customizable histograms
- 2D hexagonal histograms
- Polar plots, spherical plots and histogramming
- Draw latex-formatted formulas in the rendering window.
- Quiver, violin, whisker and stream-line plots
- Graphical markers analogous to matplotlib

integration with other libraries

- Integration with the [Qt5](https://www.qt.io/) framework.
- Support for [FEniCS/Dolfin](https://fenicsproject.org/) platform for visualization of PDE/FEM solutions.
- Interoperability with the [trimesh](https://trimsh.org/), [pyvista](https://github.com/pyvista/pyvista) and [pymeshlab](https://github.com/cnr-isti-vclab/PyMeshLab) libraries.
- Export 3D scenes and embed them into a [web page](https://vedo.embl.es/examples/fenics_elasticity.html).
- Embed 3D scenes in *jupyter* notebooks with [K3D](https://github.com/K3D-tools/K3D-jupyter) (can export an interactive 3D-snapshot page [here](https://vedo.embl.es/examples/geo_scene.html)).

### ⌨  Command Line Interface
Visualize a polygonal mesh or a volume from a terminal window simply with:
```bash
vedo https://vedo.embl.es/examples/data/embryo.tif
```

volumetric files (slc, tiff, DICOM...) can be visualized in different modes [click to expand]

|Volume 3D slicing
`vedo --slicer embryo.slc`| Ray-casting
`vedo -g`| 2D slicing
`vedo --slicer2d`|
|:--------|:-----|:--------|
| ![slicer](https://user-images.githubusercontent.com/32848391/80292484-50757180-8757-11ea-841f-2c0c5fe2c3b4.jpg) | ![isohead](https://user-images.githubusercontent.com/32848391/58336107-5a09a180-7e43-11e9-8c4e-b50e4e95ae71.gif) | ![viz_slicer](https://user-images.githubusercontent.com/32848391/90966778-fc955200-e4d6-11ea-8e29-215f7aea3860.png)  |

Type `vedo -h` for the complete list of options.

## 🐾  Gallery
`vedo` currently includes 300+ working [examples](https://github.com/marcomusy/vedo/tree/master/examples) and [notebooks](https://github.com/marcomusy/vedo/tree/master/examples/notebooks).

Run any of the built-in examples. In a terminal type: `vedo -r warp2`

Check out the example galleries organized by subject here:

![](https://user-images.githubusercontent.com/32848391/104370203-d1aba900-551e-11eb-876c-41e0961fcdb5.jpg)

## ✏  Contributing

Any contributions are **greatly appreciated**!
If you have a suggestion that would make this better, please fork the repo and create a pull request.
You can also simply open an issue with the tag "enhancement".

## 📜  References

**Scientific publications leveraging `vedo`:**

- X. Diego *et al.*:
*"Key features of Turing systems are determined purely by network topology"*,
Phys. Rev. X 8, 021071,
[DOI](https://journals.aps.org/prx/abstract/10.1103/PhysRevX.8.021071).
- M. Musy, K. Flaherty *et al.*:
*"A Quantitative Method for Staging Mouse Limb Embryos based on Limb Morphometry"*,
Development (2018) 145 (7): dev154856,
[DOI](http://dev.biologists.org/content/145/7/dev154856).
- F. Claudi, A. L. Tyson, T. Branco, *"Brainrender. A python based software for visualisation
of neuroanatomical and morphological data."*,
eLife 2021;10:e65751,
[DOI](https://doi.org/10.7554/eLife.65751).
- J. S. Bennett, D. Sijacki,
*"Resolving shocks and filaments in galaxy formation simulations: effects on gas properties and
star formation in the circumgalactic medium"*,
Monthly Notices of the Royal Astronomical Society, Volume 499, Issue 1,
[DOI](https://doi.org/10.1093/mnras/staa2835).
- J.D.P. Deshapriya *et al.*,
*"Spectral analysis of craters on (101955) Bennu"*.
Icarus 2020,
[DOI](https://doi.org/10.1016/j.icarus.2020.114252).
- A. Pollack *et al.*,
*"Stochastic inversion of gravity, magnetic, tracer, lithology, and fault data
for geologically realistic structural models: Patua Geothermal Field case study"*,
Geothermics, Volume 95, September 2021,
[DOI](https://doi.org/10.1016/j.geothermics.2021.102129).
- X. Lu *et al.*,
*"3D electromagnetic modeling of graphitic faults in the Athabasca
 Basin using a finite-volume time-domain approach with unstructured grids"*,
Geophysics,
[DOI](https://doi.org/10.1190/geo2020-0657.1).
- M. Deepa Maheshvare *et al.*,
*"A Graph-Based Framework for Multiscale Modeling of Physiological Transport"*,
Front. Netw. Physiol. 1:802881,
[DOI](https://www.frontiersin.org/articles/10.3389/fnetp.2021.802881/full).
- F. Claudi, T. Branco,
*"Differential geometry methods for constructing manifold-targeted recurrent neural networks"*,
bioRxiv 2021.10.07.463479,
[DOI](https://doi.org/10.1101/2021.10.07.463479).
- J. Klatzow, G. Dalmasso, N. Martínez-Abadías, J. Sharpe, V. Uhlmann,
*"µMatch: 3D shape correspondence for microscopy data"*,
Front. Comput. Sci., 15 February 2022.
[DOI](https://doi.org/10.3389/fcomp.2022.777615)
- G. Dalmasso *et al.*, *"4D reconstruction of murine developmental trajectories using spherical harmonics"*,
Developmental Cell 57, 1–11 September 2022,
[DOI](https://doi.org/10.1016/j.devcel.2022.08.005).
- D.J.E Waibel *et al.*, *"Capturing Shape Information with Multi-scale Topological Loss Terms for 3D Reconstruction"*,
Lecture Notes in Computer Science, vol 13434. Springer, Cham.
[DOI](https://doi.org/10.1007/978-3-031-16440-8_15).
- N. Lamb *et al.*, *"DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair"*,
ACM Transactions on Graphics (TOG), vol 41, 6, 2022.
[DOI](https://dl.acm.org/doi/abs/10.1145/3550454.3555470)

**Have you found this software useful for your research? Star ✨ the project and cite it as:**

M. Musy  et al.,
"vedo, a python module for scientific analysis and visualization of 3D objects and point clouds",
Zenodo, 2021, doi: 10.5281/zenodo.7019968.

[![embl_logo](https://user-images.githubusercontent.com/32848391/58046204-e9157180-7b44-11e9-81c9-e916cdf9ba84.gif)](https://www.embl.es)


django-tenants
==============
.. image:: https://badge.fury.io/py/django-tenants.svg
    :target: http://badge.fury.io/py/django-tenants

.. image:: https://github.com/tomturner/django-tenants/workflows/code/badge.svg
    :alt: Build status
    :target: https://github.com/tomturner/django-tenants/actions

.. image:: https://readthedocs.org/projects/pip/badge/?version=latest
    :target: https://django-tenants.readthedocs.io/en/latest/

.. image:: https://img.shields.io/pypi/dm/django-tenants.svg?maxAge=180
   :target: https://pypi.org/project/django-tenants/

.. image:: https://codecov.io/gh/django-tenants/django-tenants/branch/master/graph/badge.svg?token=wCNgSgTTR8
   :target: https://codecov.io/gh/django-tenants/django-tenants

This application enables `django`_ powered websites to have multiple
tenants via `PostgreSQL schemas`_. A vital feature for every
Software-as-a-Service (SaaS) website.

    Read the full documentation here: `django-tenants.readthedocs.org`_

Django provides currently no simple way to support multiple tenants
using the same project instance, even when only the data is different.
Because we don’t want you running many copies of your project, you’ll be
able to have:

-  Multiple customers running on the same instance
-  Shared and Tenant-Specific data
-  Tenant View-Routing

What are schemas
----------------

A schema can be seen as a directory in an operating system, each
directory (schema) with its own set of files (tables and objects). This
allows the same table name and objects to be used in different schemas
without conflict. For an accurate description on schemas, see
`PostgreSQL’s official documentation on schemas`_.

Why schemas
-----------

There are typically three solutions for solving the multitenancy
problem.

1. Isolated Approach: Separate Databases. Each tenant has its own
   database.

2. Semi Isolated Approach: Shared Database, Separate Schemas. One
   database for all tenants, but one schema per tenant.

3. Shared Approach: Shared Database, Shared Schema. All tenants share
   the same database and schema. There is a main tenant-table, where all
   other tables have a foreign key pointing to.

This application implements the second approach, which in our opinion,
represents the ideal compromise between simplicity and performance.

-  Simplicity: barely make any changes to your current code to support
   multitenancy. Plus, you only manage one database.
-  Performance: make use of shared connections, buffers and memory.

Each solution has its up and down sides. For a more in-depth
discussion, see Microsoft’s excellent article on `Multi-Tenant Data
Architecture`_.

How it works
------------

Tenants are identified via their host name (i.e tenant.domain.com). This
information is stored on a table on the ``public`` schema. Whenever a
request is made, the host name is used to match a tenant in the
database. If there’s a match, the search path is updated to use this
tenant’s schema. So from now on all queries will take place at the
tenant’s schema. For example, suppose you have a tenant ``customer`` at
http://customer.example.com. Any request incoming at
``customer.example.com`` will automatically use ``customer``\ ’s schema
and make the tenant available at the request. If no tenant is found, a
404 error is raised. This also means you should have a tenant for your
main domain, typically using the ``public`` schema. For more information
please read the `setup`_ section.

What can this app do?
---------------------

As many tenants as you want
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Each tenant has its data on a specific schema. Use a single project
instance to serve as many as you want.

Tenant-specific and shared apps
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Tenant-specific apps do not share their data between tenants, but you
can also have shared apps where the information is always available and
shared between all.

Tenant View-Routing
~~~~~~~~~~~~~~~~~~~

You can have different views for ``http://customer.example.com/`` and
``http://example.com/``, even though Django only uses the string after
the host name to identify which view to serve.

Magic
~~~~~

Everyone loves magic! You’ll be able to have all this barely having to
change your code!

Setup & Documentation
---------------------

**This is just a short setup guide**. It is **strongly** recommended
that you read the complete version at
`django-tenants.readthedocs.org`_.

Your ``DATABASE_ENGINE`` setting needs to be changed to

.. code-block:: python

    DATABASES = {
        'default': {
            'ENGINE': 'django_tenants.postgresql_backend',
            # ..
        }
    }

Add the middleware ``django_tenants.middleware.main.TenantMainMiddleware`` to the
top of ``MIDDLEWARE``, so that each request can be set to use
the correct schema.

.. code-block:: python

    MIDDLEWARE = (
        'django_tenants.middleware.main.TenantMainMiddleware',
        #...
    )

Add ``django_tenants.routers.TenantSyncRouter`` to your `DATABASE_ROUTERS`
setting, so that the correct apps can be synced depending on what's
being synced (shared or tenant).

.. code-block:: python

    DATABASE_ROUTERS = (
        'django_tenants.routers.TenantSyncRouter',
    )

Add ``django_tenants`` to your ``INSTALLED_APPS``.

Create your tenant model
~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from django.db import models
    from django_tenants.models import TenantMixin, DomainMixin

    class Client(TenantMixin):
        name = models.CharField(max_length=100)
        paid_until = models.DateField()
        on_trial = models.BooleanField()
        created_on = models.DateField(auto_now_add=True)

    class Domain(DomainMixin):
        pass

Define on ``settings.py`` which model is your tenant model. Assuming you
created ``Client`` inside an app named ``customers``, your
``TENANT_MODEL`` should look like this:

.. code-block:: python

    TENANT_MODEL = "customers.Client" # app.Model
    TENANT_DOMAIN_MODEL = "customers.Domain" # app.Model

Now run ``migrate_schemas``. This will sync your apps to the ``public``
schema.

.. code-block:: bash

    python manage.py migrate_schemas --shared

Create your tenants just like a normal django model. Calling ``save``
will automatically create and sync the schema.

.. code-block:: python

    from customers.models import Client, Domain

    # create your public tenant
    tenant = Client(schema_name='tenant1',
                    name='My First Tenant',
                    paid_until='2014-12-05',
                    on_trial=True)
    tenant.save()

    # Add one or more domains for the tenant
    domain = Domain()
    domain.domain = 'tenant.my-domain.com'
    domain.tenant = tenant
    domain.is_primary = True
    domain.save()

Any request made to ``tenant.my-domain.com`` will now automatically set
your PostgreSQL’s ``search_path`` to ``tenant1`` and ``public``, making
shared apps available too. This means that any call to the methods
``filter``, ``get``, ``save``, ``delete`` or any other function
involving a database connection will now be done at the tenant’s schema,
so you shouldn’t need to change anything at your views.

You’re all set, but we have left key details outside of this short
tutorial, such as creating the public tenant and configuring shared and
tenant specific apps. Complete instructions can be found at
`django-tenants.readthedocs.org`_.

Running the example project
---------------------------

django-tenants comes with an example project please see

`examples`_.

Credits
-------

I would like to thank two of the original authors of this project.

1. Bernardo Pires under the name `django-tenant-schemas`_.

2. Vlada Macek under the name of `django-schemata`_.

Requirements
------------

 - Django 2 if you want to use Django 1.11 or lower please use version 1 of django-tenants
 - PostgreSQL

Testing
-------

If you want to run tests, you can either run ``run_tests.sh`` (which requires access to
a PostgreSQL instance, location of which you can customize using the ``DATABASE_HOST``
env variable) or use `docker-compose`_ like this:

.. code-block:: bash

    ## Start Docker service
    # start docker   # with Upstart
    # systemctl start docker  # with systemd

    ## Install docker-compose (you might want to do this in Python virtualenv)
    # pip install docker-compose

    ## In main directory of this repo do:
    docker-compose run --rm django-tenants-test  # runs django-tenants tests.
    # dockerized PostgreSQL service is started implicitly

(note that upon first run the ``Dockerfile`` will be built).

Video Tutorial
--------------

An online video tutorial is available on `youtube`_.

Donation
--------

If this project helped you reduce development time, you can give me cake :)

.. image:: https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif
  :target: https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=QU8BGC7DWB9G6&source=ur

.. _youtube: https://youtu.be/TWF7okf5Xoo
.. _django: https://www.djangoproject.com/
.. _PostgreSQL schemas: http://www.postgresql.org/docs/9.1/static/ddl-schemas.html
.. _PostgreSQL’s official documentation on schemas: http://www.postgresql.org/docs/9.1/static/ddl-schemas.html
.. _Multi-Tenant Data Architecture: https://web.archive.org/web/20160311212239/https://msdn.microsoft.com/en-us/library/aa479086.aspx
.. _setup: https://django-tenants.readthedocs.org/en/latest/install.html
.. _examples: https://django-tenants.readthedocs.org/en/latest/examples.html
.. _django-tenants.readthedocs.org: https://django-tenants.readthedocs.org/en/latest/
.. _django-tenant-schemas: http://github.com/bernardopires/django-tenant-schemas
.. _django-schemata: https://github.com/tuttle/django-schemata
.. _docker-compose: https://docs.docker.com/engine/reference/run/


yarl
====

The module provides handy URL class for URL parsing and changing.

.. image:: https://github.com/aio-libs/yarl/workflows/CI/badge.svg
  :target: https://github.com/aio-libs/yarl/actions?query=workflow%3ACI
  :align: right

.. image:: https://codecov.io/gh/aio-libs/yarl/branch/master/graph/badge.svg
  :target: https://codecov.io/gh/aio-libs/yarl

.. image:: https://badge.fury.io/py/yarl.svg
    :target: https://badge.fury.io/py/yarl

.. image:: https://readthedocs.org/projects/yarl/badge/?version=latest
    :target: https://yarl.aio-libs.org

.. image:: https://img.shields.io/pypi/pyversions/yarl.svg
    :target: https://pypi.python.org/pypi/yarl

.. image:: https://img.shields.io/matrix/aio-libs:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat
   :target: https://matrix.to/#/%23aio-libs:matrix.org
   :alt: Matrix Room — #aio-libs:matrix.org

.. image:: https://img.shields.io/matrix/aio-libs-space:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs-space%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat
   :target: https://matrix.to/#/%23aio-libs-space:matrix.org
   :alt: Matrix Space — #aio-libs-space:matrix.org

Introduction
------------

Url is constructed from ``str``:

.. code-block:: pycon

   >>> from yarl import URL
   >>> url = URL('https://www.python.org/~guido?arg=1#frag')
   >>> url
   URL('https://www.python.org/~guido?arg=1#frag')

All url parts: *scheme*, *user*, *password*, *host*, *port*, *path*,
*query* and *fragment* are accessible by properties:

.. code-block:: pycon

   >>> url.scheme
   'https'
   >>> url.host
   'www.python.org'
   >>> url.path
   '/~guido'
   >>> url.query_string
   'arg=1'
   >>> url.query
   <MultiDictProxy('arg': '1')>
   >>> url.fragment
   'frag'

All url manipulations produce a new url object:

.. code-block:: pycon

   >>> url = URL('https://www.python.org')
   >>> url / 'foo' / 'bar'
   URL('https://www.python.org/foo/bar')
   >>> url / 'foo' % {'bar': 'baz'}
   URL('https://www.python.org/foo?bar=baz')

Strings passed to constructor and modification methods are
automatically encoded giving canonical representation as result:

.. code-block:: pycon

   >>> url = URL('https://www.python.org/шлях')
   >>> url
   URL('https://www.python.org/%D1%88%D0%BB%D1%8F%D1%85')

Regular properties are *percent-decoded*, use ``raw_`` versions for
getting *encoded* strings:

.. code-block:: pycon

   >>> url.path
   '/шлях'

   >>> url.raw_path
   '/%D1%88%D0%BB%D1%8F%D1%85'

Human readable representation of URL is available as ``.human_repr()``:

.. code-block:: pycon

   >>> url.human_repr()
   'https://www.python.org/шлях'

For full documentation please read https://yarl.aio-libs.org.

Installation
------------

::

   $ pip install yarl

The library is Python 3 only!

PyPI contains binary wheels for Linux, Windows and MacOS.  If you want to install
``yarl`` on another operating system (like *Alpine Linux*, which is not
manylinux-compliant because of the missing glibc and therefore, cannot be
used with our wheels) the the tarball will be used to compile the library from
the source code. It requires a C compiler and and Python headers installed.

To skip the compilation you must explicitly opt-in by using a PEP 517
configuration setting ``pure-python``, or setting the ``YARL_NO_EXTENSIONS``
environment variable to a non-empty value, e.g.:

.. code-block:: console

   $ pip install yarl --config-settings=pure-python=false

Please note that the pure-Python (uncompiled) version is much slower. However,
PyPy always uses a pure-Python implementation, and, as such, it is unaffected
by this variable.

Dependencies
------------

YARL requires multidict_ library.

API documentation
------------------

The documentation is located at https://yarl.aio-libs.org.

Why isn't boolean supported by the URL query API?
-------------------------------------------------

There is no standard for boolean representation of boolean values.

Some systems prefer ``true``/``false``, others like ``yes``/``no``, ``on``/``off``,
``Y``/``N``, ``1``/``0``, etc.

``yarl`` cannot make an unambiguous decision on how to serialize ``bool`` values because
it is specific to how the end-user's application is built and would be different for
different apps.  The library doesn't accept booleans in the API; a user should convert
bools into strings using own preferred translation protocol.

Comparison with other URL libraries
------------------------------------

* furl (https://pypi.python.org/pypi/furl)

  The library has rich functionality but the ``furl`` object is mutable.

  I'm afraid to pass this object into foreign code: who knows if the
  code will modify my url in a terrible way while I just want to send URL
  with handy helpers for accessing URL properties.

  ``furl`` has other non-obvious tricky things but the main objection
  is mutability.

* URLObject (https://pypi.python.org/pypi/URLObject)

  URLObject is immutable, that's pretty good.

  Every URL change generates a new URL object.

  But the library doesn't do any decode/encode transformations leaving the
  end user to cope with these gory details.

Source code
-----------

The project is hosted on GitHub_

Please file an issue on the `bug tracker
<https://github.com/aio-libs/yarl/issues>`_ if you have found a bug
or have some suggestion in order to improve the library.

Discussion list
---------------

*aio-libs* google group: https://groups.google.com/forum/#!forum/aio-libs

Feel free to post your questions and ideas here.

Authors and License
-------------------

The ``yarl`` package is written by Andrew Svetlov.

It's *Apache 2* licensed and freely available.

.. _GitHub: https://github.com/aio-libs/yarl

.. _multidict: https://github.com/aio-libs/multidict


![Pyproj logo](https://raw.githubusercontent.com/pyproj4/pyproj/main/docs/media/logo.png)

# pyproj

Python interface to [PROJ](http://proj.org) (cartographic projections and coordinate transformations library).

## Documentation

- Stable: http://pyproj4.github.io/pyproj/stable/
- Latest: https://pyproj4.github.io/pyproj/latest/

## Bugs/Questions

- Report bugs/feature requests: https://github.com/pyproj4/pyproj/issues
- Ask questions: https://github.com/pyproj4/pyproj/discussions
- Ask developer questions: https://gitter.im/pyproj4-pyproj/community
- Ask the GIS community: https://gis.stackexchange.com/questions/tagged/pyproj

## Contributors ✨

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

Jeff Whitaker
📖 ⚠ 💻 💡 🤔 👀 💬 🚧 🚇 🐛

Alan D. Snow
📖 ⚠ 💻 💡 🚧 🚇 🤔 👀 💬 🐛

Micah Cochran
📖 ⚠ 💻 🚧 🚇 👀 💬 🐛

Joris Van den Bossche
📖 💻 🤔 👀 💬 🐛 ⚠

Chris Mayo
⚠

Charles Karney
💻 ⚠

Justin Dearing
🚇

Jos de Kloe
💻 ⚠ 🐛

George Ouzounoudis
💻 🤔

David Hoese
👀 🤔 📦 📖 ⚠ 💻

Mikhail Itkin
💻

Ryan May
💻

artttt
🤔

Filipe
🚇 💻 📦 📖

Heitor
📖

Bas Couwenberg
💻 📦 ⚠

Nick Eubank
💻

Michael Dunphy
📖

Matthew Brett
🚇 📦

Jakob de Maeyer
💻

The Gitter Badger
📖

Bernhard M. Wiedemann
💻

Marco Aurélio da Costa
💻

Christopher H. Barker
💻

Kristian Evers
💬 🤔 📖

Even Rouault
💬

Christoph Gohlke
📦 💬 🐛 ⚠

Chris Willoughby
💻

Guillaume Lostis
📖

Eduard Popov
📖

Joe Ranalli
🐛 💻 ⚠

Greg Berardinelli
🐛 💻 🤔 ⚠

Martin Raspaud
🐛 💻 ⚠ 🤔

Mike Taves
⚠

David Haberthür
📖

mmodenesi
🐛 💻 ⚠

jacob-indigo
🐛 💻

Poruri Sai Rahul
⚠

Yann-Sebastien Tremblay-Johnston
📖

odidev
📦

Idan Miara
💻 📖 💡 ⚠

Brendan Jurd
📖 🎨

Bill Little
📖

Gerrit Holl
📖

Kirill Kouzoubov
💻

Dan Hemberger
🐛 💻

Martin Fleischmann
🐛 💻 ⚠

Matthias Meulien
💻 🐛

Isaac Boates
💻 🐛 ⚠

Kyle Penner
💻 🐛 📖

paulcochrane
💻 📖 ⚠ 🐛

Antonio Ettorre
📦

DWesl
💻

Víctor Molina García
📦

Samuel Kogler
🐛 💻

Alexander Shadchin
🐛 💻

Greg Lucas
💻 🤔

Dan Mahr
💻 📖 ⚠

Romain Hugonnet
💻 📖 ⚠

Javier Jimenez Shaw
💻 📖 ⚠

Daniel McDonald
📖

Cora Schneck
📖 ⚠

zanejgr
📖

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!


See the [docassemble web site] for a description of **docassemble**
and installation instructions.

To get help with using **docassemble**, join the [docassemble Slack
group].

[docassemble web site]: https://docassemble.org
[docassemble Slack group]: https://docassemble.org/docs/support.html


# PySAML2 - SAML2 for Python

[![Version](https://img.shields.io/pypi/v/pysaml2)](https://pypi.org/project/pysaml2/)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/pysaml2)](https://pypi.org/project/pysaml2/)
[![Total downloads](https://pepy.tech/badge/pysaml2)](https://pepy.tech/project/pysaml2)
[![Weekly downloads](https://pepy.tech/badge/pysaml2/week)](https://pepy.tech/project/pysaml2)
[![License](https://img.shields.io/github/license/IdentityPython/pysaml2)](https://github.com/IdentityPython/pysaml2/blob/master/LICENSE)

PySAML2 is a pure python implementation of SAML Version 2 Standard.
It contains all necessary pieces for building a SAML2 service provider
or an identity provider. The distribution contains examples of both.
Originally written to work in a WSGI environment
there are extensions that allow you to use it with other frameworks.

**Website**: https://idpy.org/

**Documentation**: https://pysaml2.readthedocs.io/

**Contribution guidelines**: [CONTRIBUTING.md][contributing]

**Security policies**: [SECURITY.md][sec]

**Source code**: https://github.com/IdentityPython/pysaml2/

**Developer guidelines**: [DEVELOPERS.md][dev]

**PyPI project**: https://pypi.org/project/pysaml2/

**License**: [LICENSE][license]

## Specifications

Retrieved from https://wiki.oasis-open.org/security/FrontPage

#### SAML V2.0 Standard

- SAML2 Core (aka Assertions and Protocols): http://www.oasis-open.org/committees/download.php/56776/sstc-saml-core-errata-2.0-wd-07.pdf
  - Assertion schema: http://docs.oasis-open.org/security/saml/v2.0/saml-schema-assertion-2.0.xsd
  - Protocols schema: http://docs.oasis-open.org/security/saml/v2.0/saml-schema-protocol-2.0.xsd
- Bindings: http://www.oasis-open.org/committees/download.php/56779/sstc-saml-bindings-errata-2.0-wd-06.pdf
- Profiles: http://www.oasis-open.org/committees/download.php/56782/sstc-saml-profiles-errata-2.0-wd-07.pdf
- Metadata: http://www.oasis-open.org/committees/download.php/56785/sstc-saml-metadata-errata-2.0-wd-05.pdf
  - Metadata schema: http://docs.oasis-open.org/security/saml/v2.0/saml-schema-metadata-2.0.xsd
- Authentication Context: http://docs.oasis-open.org/security/saml/v2.0/saml-authn-context-2.0-os.pdf
- Conformance Requirements: https://docs.oasis-open.org/security/saml/v2.0/saml-conformance-2.0-os.pdf
- Security and Privacy Considerations: http://docs.oasis-open.org/security/saml/v2.0/saml-sec-consider-2.0-os.pdf
- Glossary: http://docs.oasis-open.org/security/saml/v2.0/saml-glossary-2.0-os.pdf

#### Profiles and extensions

- Metadata Extension for SAML V2.0 and V1.x Query Requesters: http://docs.oasis-open.org/security/saml/Post2.0/sstc-saml-metadata-ext-query-os.pdf
- SAML V2.0 Metadata Interoperability Profile: https://docs.oasis-open.org/security/saml/Post2.0/sstc-metadata-iop-os.pdf
- SAML V2.0 Metadata Extensions for Login and Discovery User Interface Version 1.0: https://docs.oasis-open.org/security/saml/Post2.0/sstc-saml-metadata-ui/v1.0/os/sstc-saml-metadata-ui-v1.0-os.pdf
- SAML V2.0 LDAP/X.500 Attribute Profile: http://www.oasis-open.org/committees/download.php/28042/sstc-saml-attribute-x500-cs-01.pdf
- SAML V2.0 Enhanced Client or Proxy Profile Version 2.0: https://docs.oasis-open.org/security/saml/Post2.0/saml-ecp/v2.0/cs01/saml-ecp-v2.0-cs01.pdf

#### Committee Specifications

- SAML V2.0 Subject Identifier Attributes Profile Version 1.0: https://docs.oasis-open.org/security/saml-subject-id-attr/v1.0/cs01/saml-subject-id-attr-v1.0-cs01.pdf

## Installation

You can install PySAML2 through pip:

```shell
pip install pysaml2
```

### External dependencies

PySAML2 works with the [`xmlsec`][xmlsec] binary.
Notice that support for xmlsec `1 1.3` was added with `v7.4.2`.

`xmlsec` should be readily available in most Linux distributions:

```shell
$ apt-get install xmlsec1
$ dnf install xmlsec1-openssl
$ yum install xmlsec1-openssl
$ pacman -S xmlsec
...
```

and on MacOS through [`homebrew`][brew]

```shell
$ brew install libxmlsec1
```

## Changelog

See the [CHANGELOG][clog] to learn about the latest developments.

## Contributing

We've set up a separate document for our [contribution guidelines][contributing].

## Community

[IdentityPython][idpy] is a community around
a collection of libraries and tools to manage identity related concepts with Python code.
You can interact with the community though the [mailing list](https://lists.sunet.se/postorius/lists/idpy-discuss.lists.sunet.se/)
or on the [Slack workspace](https://identity-python.slack.com/) ([invitation](https://join.slack.com/t/identity-python/shared_invite/enQtNzEyNjU1NDI1MjUyLTM2MWI5ZGNhMTk1ZThiOTIxNWY2OTY1ODVmMWNjMzUzMTYxNTY5MzE5N2RlYjExZTIyM2MwYjBjZGE4MGVlMTM)).

## Development

We've set up a separate document for [developers][dev].

### Releasing

We've set up a separate document for our [release process][rel].

### Pre-commit

(TODO)

  [idpy]: https://idpy.org/
  [docs]: https://pysaml2.readthedocs.io/
  [contributing]: https://github.com/IdentityPython/pysaml2/blob/master/CONTRIBUTING.md
  [sec]: https://github.com/IdentityPython/pysaml2/blob/master/SECURITY.md
  [repo]: https://github.com/IdentityPython/pysaml2/
  [dev]: https://github.com/IdentityPython/pysaml2/blob/master/DEVELOPERS.md
  [pypi]: https://pypi.org/project/pysaml2/
  [license]: https://github.com/IdentityPython/pysaml2/blob/master/LICENSE
  [clog]: https://github.com/IdentityPython/pysaml2/blob/master/CHANGELOG.md
  [rel]: https://github.com/IdentityPython/pysaml2/blob/master/RELEASE.md
  [xmlsec]: http://www.aleksey.com/xmlsec/
  [brew]: https://brew.sh/


.. contents::
   :backlinks: top

========
 Status
========
|VersionsBadge| |ImplementationBadge| |LicenseBadge|

|PyPI| |RTD| |Codecov| |Pyup| |CommitsSinceLastRelease|

=======
 About
=======
This is a Python cffi_ port of libtcod_.

This library is `hosted on GitHub <https://github.com/libtcod/python-tcod>`_.

Any issues you have with this module can be reported at the
`GitHub issue tracker <https://github.com/libtcod/python-tcod/issues>`_.

=======
 Usage
=======
This module was designed to be backward compatible with the original libtcodpy
module distributed with libtcod.
If you had code that runs on libtcodpy then you can use this library as a
drop-in replacement.  This installs a libtcodpy module so you'll only need to
delete the libtcodpy/ folder that's usually bundled in an older libtcodpy
project.

Guides and Tutorials for libtcodpy should work with the tcod module.

The latest documentation can be found here:
https://python-tcod.readthedocs.io/en/latest/

==============
 Installation
==============
Detailed installation instructions are here:
https://python-tcod.readthedocs.io/en/latest/installation.html

For the most part it's just::

    pip3 install tcod

==============
 Requirements
==============
* Python 3.8+
* Windows, Linux, or MacOS X 10.9+.
* On Linux, requires libsdl2 (2.0.10+).

===========
 Changelog
===========

You can find the most recent changelog
`here <https://github.com/libtcod/python-tcod/blob/main/CHANGELOG.md>`_.

=========
 License
=========
python-tcod is distributed under the `Simplified 2-clause FreeBSD license
<https://github.com/HexDecimal/python-tdl/blob/master/LICENSE.txt>`_.

.. _LICENSE.txt: https://github.com/libtcod/python-tcod/blob/master/LICENSE.txt

.. _python-tdl: https://github.com/libtcod/python-tcod/

.. _cffi: https://cffi.readthedocs.io/en/latest/

.. _numpy: https://docs.scipy.org/doc/numpy/user/index.html

.. _libtcod: https://github.com/libtcod/libtcod

.. _pip: https://pip.pypa.io/en/stable/installing/

.. |VersionsBadge| image:: https://img.shields.io/pypi/pyversions/tcod.svg?maxAge=2592000
    :target: https://pypi.python.org/pypi/tcod

.. |ImplementationBadge| image:: https://img.shields.io/pypi/implementation/tcod.svg?maxAge=2592000
    :target: https://pypi.python.org/pypi/tcod

.. |LicenseBadge| image:: https://img.shields.io/pypi/l/tcod.svg?maxAge=2592000
    :target: https://github.com/HexDecimal/tcod/blob/master/LICENSE.txt

.. |PyPI| image:: https://img.shields.io/pypi/v/tcod.svg?maxAge=10800
    :target: https://pypi.python.org/pypi/tcod

.. |RTD| image:: https://readthedocs.org/projects/python-tcod/badge/?version=latest
    :target: http://python-tcod.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

.. |Codecov| image:: https://codecov.io/gh/libtcod/python-tcod/branch/master/graph/badge.svg
    :target: https://codecov.io/gh/libtcod/python-tcod

.. |Issues| image:: https://img.shields.io/github/issues/libtcod/python-tcod.svg?maxAge=3600
    :target: https://github.com/libtcod/python-tcod/issues

.. |Pyup| image:: https://pyup.io/repos/github/libtcod/python-tcod/shield.svg
    :target: https://pyup.io/repos/github/libtcod/python-tcod/
    :alt: Updates

.. |CommitsSinceLastRelease| image:: https://img.shields.io/github/commits-since/libtcod/python-tcod/latest
    :target: https://github.com/libtcod/python-tcod/blob/main/CHANGELOG.md


# Python recommendation tools

![Test Suite](https://github.com/lenskit/lkpy/workflows/Test%20Suite/badge.svg)
[![codecov](https://codecov.io/gh/lenskit/lkpy/branch/master/graph/badge.svg)](https://codecov.io/gh/lenskit/lkpy)

LensKit is a set of Python tools for experimenting with and studying recommender
systems.  It provides support for training, running, and evaluating recommender
algorithms in a flexible fashion suitable for research and education.

LensKit for Python (LKPY) is the successor to the Java-based LensKit project.

> [!IMPORTANT]
> If you use LensKit for Python in published research, please cite:
>
> > Michael D. Ekstrand. 2020.
> > LensKit for Python: Next-Generation Software for Recommender Systems Experiments.
> > In <cite>Proceedings of the 29th ACM International Conference on Information and Knowledge Management</cite> (CIKM '20).
> > DOI:[10.1145/3340531.3412778](https://dx.doi.org/10.1145/3340531.3412778).
> > arXiv:[1809.03125](https://arxiv.org/abs/1809.03125) [cs.IR].

> [!WARNING]
> This is the `main` branch of LensKit, following new development in preparation
> for the 2024 release.  It will be changing frequently and incompatibly. You
> probably want to use a [stable release][release].

[release]: https://lkpy.lenskit.org/en/stable/

## Installing

To install the current release with Anaconda (recommended):

    conda install -c conda-forge lenskit

Or you can use `pip`:

    pip install lenskit

To use the latest development version, install directly from GitHub:

    pip install -U git+https://github.com/lenskit/lkpy

Then see [Getting Started](https://lkpy.lenskit.org/en/latest/GettingStarted.html)

## Developing

[issues]: https://github.com/lenskit/lkpy/issues
[workflow]: https://github.com/lenskit/lkpy/wiki/DevWorkflow

To contribute to LensKit, clone or fork the repository, get to work, and submit
a pull request.  We welcome contributions from anyone; if you are looking for a
place to get started, see the [issue tracker][issues].

Our development workflow is documented in [the wiki][workflow]; the wiki also
contains other information on *developing* LensKit. User-facing documentation is
at <https://lkpy.lenskit.org>.

[conda-lock]: https://github.com/conda-incubator/conda-lock
[lkbuild]: https://github.com/lenskit/lkbuild

We recommend using an Anaconda environment for developing LensKit.  We provide a
tool to automate setting up Conda environments from the LensKit dependencies; to
create a dev environment, checkout LensKit, then run:

    pipx ./utils/conda-tool.py --env -n lkpy pyproject.toml dev-requirements.txt
    conda activate lkpy

That will create and activate an environment named `lkpy` with all the LensKit
dependencies. You will also need to install LensKit in editable mode to do
things like run the tests:

    pip install -e lenskit

Each LensKit subpackage you want to work on will also need to be installed.

### Developing with Standard Virtual Environments

[uv]: https://github.com/astral-sh/uv

You can also use a standard virtual environment and vanilla Python to develop LensKit.
To do this, the easiest way is to use [uv][]:

    uv venv -p python3.11
    uv pip install -r full-dev-requirements.txt

You can also use traditional Pip:

    python -m venv .venv
    . .venv/bin/activate
    python -m pip install -r requirements-full-dev.txt

## Testing Changes

You should always test your changes by running the LensKit test suite:

    python -m pytest

If you want to use your changes in a LensKit experiment, you can locally install
your modified LensKit into your experiment's environment.  We recommend using
separate environments for LensKit development and for each experiment; you will
need to install the modified LensKit into your experiment's repository:

    conda activate my-exp
    conda install -c conda-forge
    cd /path/to/lkpy
    pip install -e . --no-deps

You may need to first uninstall LensKit from your experiment repo; make sure that
LensKit's dependencies are all still installed.

Once you have pushed your code to a GitHub branch, you can use a Git repository as
a Pip dependency in an `environment.yml` for your experiment, to keep using the
correct modified version of LensKit until your changes make it in to a release.

## Resources

- [Documentation](https://lkpy.lenskit.org)
- [Mailing list, etc.](https://lenskit.org/connect)

## Acknowledgements

This material is based upon work supported by the National Science Foundation
under Grant No. IIS 17-51278. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the author(s) and do not
necessarily reflect the views of the National Science Foundation.


<picture >
  <source media="(prefers-color-scheme: dark)" srcset="https://i.imgur.com/QE6SIrs.png"  style="width:300px">

</picture>

![phase: beta](https://img.shields.io/badge/phase-beta-C41E3A) [![license-badge]](LICENSE) [![docs-badge]][docs] ![PRs: welcome](https://img.shields.io/badge/PRs-Welcome-08A04B) [](https://t.me/lnbits) [](https://opensats.org)
![Lightning network wallet](https://i.imgur.com/DeIiO0y.png)

# The world's most powerful suite of bitcoin tools.

## Run for yourself, for others, or as part of a stack.

LNbits is beta, for responsible disclosure of any concerns please contact an admin in the community chat.

LNbits is a Python server that sits on top of any funding source. It can be used as:

- Accounts system to mitigate the risk of exposing applications to your full balance via unique API keys for each wallet
- Extendable platform for exploring Lightning network functionality via the LNbits extension framework
- Part of a development stack via LNbits API
- Fallback wallet for the LNURL scheme
- Instant wallet for LN demonstrations

LNbits can run on top of almost all Lightning funding sources.

See [LNbits manual](https://docs.lnbits.org/guide/wallets.html) for more detailed documentation about each funding source.

Checkout the LNbits [YouTube](https://www.youtube.com/playlist?list=PLPj3KCksGbSYG0ciIQUWJru1dWstPHshe) video series.

LNbits is inspired by all the great work of [opennode.com](https://www.opennode.com/), and in particular [lnpay.co](https://lnpay.co/). Both work as funding sources for LNbits.

## Running LNbits

Test on our demo server [demo.lnbits.com](https://demo.lnbits.com), or on [lnbits.com](https://lnbits.com) software as a service, where you can spin up an LNbits instance for 21sats per hr.

See the [install guide](https://github.com/lnbits/lnbits/blob/main/docs/guide/installation.md) for details on installation and setup.

## LNbits account system

LNbits is packaged with tools to help manage funds, such as a table of transactions, line chart of spending, export to csv. Each wallet also comes with its own API keys, to help partition the exposure of your funding source.

## LNbits extension universe

Extend YOUR LNbits to meet YOUR needs.

All non-core features are installed as extensions, reducing your code base and making your LNbits unique to you. Extend your LNbits install in any direction, and even create and share your own extensions.

## LNbits API

LNbits has a powerful API, many projects use LNbits to do the heavy lifting for their bitcoin/lightning services.

## LNbits node manager

LNbits comes packaged with a light node management UI, to make running your node that much easier.

## LNbits across all your devices

As well as working great in a browser, LNbits has native IoS and Android apps as well as a chrome extension. So you can enjoy the same UI across ALL your devices.

## Tip us

If you like this project [send some tip love](https://demo.lnbits.com/lnurlp/link/fH59GD)!

[docs]: https://github.com/lnbits/lnbits/wiki
[docs-badge]: https://img.shields.io/badge/docs-lnbits.org-673ab7.svg
[github-mypy]: https://github.com/lnbits/lnbits/actions?query=workflow%3Amypy
[github-mypy-badge]: https://github.com/lnbits/lnbits/workflows/mypy/badge.svg
[github-tests]: https://github.com/lnbits/lnbits/actions?query=workflow%3Atests
[github-tests-badge]: https://github.com/lnbits/lnbits/workflows/tests/badge.svg
[codecov]: https://codecov.io/gh/lnbits/lnbits
[codecov-badge]: https://codecov.io/gh/lnbits/lnbits/branch/master/graph/badge.svg
[license-badge]: https://img.shields.io/badge/license-MIT-blue.svg


[![Build Status](https://github.com/openvax/mhcflurry/actions/workflows/ci.yml/badge.svg)](https://github.com/openvax/mhcflurry/actions/workflows/ci.yml)
[![Coverage Status](https://coveralls.io/repos/github/openvax/mhcflurry/badge.svg?branch=master)](https://coveralls.io/github/openvax/mhcflurry?branch=master)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvax/mhcflurry/blob/master/notebooks/mhcflurry-colab.ipynb)

# mhcflurry
[MHC I](https://en.wikipedia.org/wiki/MHC_class_I) ligand
prediction package with competitive accuracy and a fast and
[documented](http://openvax.github.io/mhcflurry/) implementation.

MHCflurry implements class I peptide/MHC binding affinity prediction.
The current version provides pan-MHC I predictors supporting any MHC
allele of known sequence. MHCflurry runs on Python 3.4+ using the
[tensorflow](https://www.tensorflow.org/) neural network library.
It exposes [command-line](http://openvax.github.io/mhcflurry/commandline_tutorial.html)
and [Python library](http://openvax.github.io/mhcflurry/python_tutorial.html)
interfaces.

MHCflurry also includes two expermental predictors,
an "antigen processing" predictor that attempts to model MHC allele-independent
effects such as proteosomal cleavage and a "presentation" predictor that
integrates processing predictions with binding affinity predictions to give a
composite "presentation score." Both models are trained on mass spec-identified
MHC ligands.

If you find MHCflurry useful in your research please cite:

> T. O'Donnell, A. Rubinsteyn, U. Laserson. "MHCflurry 2.0: Improved pan-allele prediction of MHC I-presented peptides by incorporating antigen processing," *Cell Systems*, 2020. https://doi.org/10.1016/j.cels.2020.06.010

> T. O’Donnell, A. Rubinsteyn, M. Bonsack, A. B. Riemer, U. Laserson, and J. Hammerbacher, "MHCflurry: Open-Source Class I MHC Binding Affinity Prediction," *Cell Systems*, 2018. https://doi.org/10.1016/j.cels.2018.05.014

Please file an issue if you have questions or encounter problems.

Have a bugfix or other contribution? We would love your help. See our [contributing guidelines](CONTRIBUTING.md).

## Try it now

You can generate MHCflurry predictions without any setup by running our Google colaboratory [notebook](https://colab.research.google.com/github/openvax/mhcflurry/blob/master/notebooks/mhcflurry-colab.ipynb).

## Installation (pip)

Install the package:

```
$ pip install mhcflurry
```

Download our datasets and trained models:

```
$ mhcflurry-downloads fetch
```

You can now generate predictions:

```
$ mhcflurry-predict \
       --alleles HLA-A0201 HLA-A0301 \
       --peptides SIINFEKL SIINFEKD SIINFEKQ \
       --out /tmp/predictions.csv

Wrote: /tmp/predictions.csv
```

Or scan protein sequences for potential epitopes:

```
$ mhcflurry-predict-scan \
        --sequences MFVFLVLLPLVSSQCVNLTTRTQLPPAYTNSFTRGVYYPDKVFRSSVLHS \
        --alleles HLA-A*02:01 \
        --out /tmp/predictions.csv

Wrote: /tmp/predictions.csv
```

See the [documentation](http://openvax.github.io/mhcflurry/) for more details.

## Docker
You can also try the latest (GitHub master) version of MHCflurry using the Docker
image hosted on [Dockerhub](https://hub.docker.com/r/openvax/mhcflurry) by
running:

```
$ docker run -p 9999:9999 --rm openvax/mhcflurry:latest
```

This will start a [jupyter](https://jupyter.org/) notebook server in an
environment that has MHCflurry installed. Go to `http://localhost:9999` in a
browser to use it.

To build the Docker image yourself, from a checkout run:

```
$ docker build -t mhcflurry:latest .
$ docker run -p 9999:9999 --rm mhcflurry:latest
```
## Predicted sequence motifs
Sequence logos for the binding motifs learned by MHCflurry BA are available [here](https://openvax.github.io/mhcflurry-motifs/).

## Common issues and fixes

### Problems downloading data and models
Some users have reported HTTP connection issues when using `mhcflurry-downloads fetch`. As a workaround, you can download the data manually (e.g. using `wget`) and then use `mhcflurry-downloads` just to copy the data to the right place.

To do this, first get the URL(s) of the downloads you need using `mhcflurry-downloads url`:

```
$ mhcflurry-downloads url models_class1_presentation
https://github.com/openvax/mhcflurry/releases/download/1.6.0/models_class1_presentation.20200205.tar.bz2```
```

Then make a directory and download the needed files to this directory:

```
$ mkdir downloads
$ wget  --directory-prefix downloads https://github.com/openvax/mhcflurry/releases/download/1.6.0/models_class1_presentation.20200205.tar.bz2```

HTTP request sent, awaiting response... 200 OK
Length: 72616448 (69M) [application/octet-stream]
Saving to: 'downloads/models_class1_presentation.20200205.tar.bz2'
```

Now call `mhcflurry-downloads fetch` with the `--already-downloaded-dir` option to indicate that the downloads should be retrived from the specified directory:

```
$ mhcflurry-downloads fetch models_class1_presentation --already-downloaded-dir downloads
```


PyAutoLens: Open-Source Strong Lensing
======================================

.. |nbsp| unicode:: 0xA0
    :trim:

.. |binder| image:: https://mybinder.org/badge_logo.svg
   :target: https://mybinder.org/v2/gh/Jammy2211/autolens_workspace/HEAD

.. |RTD| image:: https://readthedocs.org/projects/pyautolens/badge/?version=latest
    :target: https://pyautolens.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

.. |Tests| image:: https://github.com/Jammy2211/PyAutoLens/actions/workflows/main.yml/badge.svg
   :target: https://github.com/Jammy2211/PyAutoLens/actions

.. |Build| image:: https://github.com/Jammy2211/PyAutoBuild/actions/workflows/release.yml/badge.svg
   :target: https://github.com/Jammy2211/PyAutoBuild/actions

.. |code-style| image:: https://img.shields.io/badge/code%20style-black-000000.svg
    :target: https://github.com/psf/black

.. |JOSS| image:: https://joss.theoj.org/papers/10.21105/joss.02825/status.svg
   :target: https://doi.org/10.21105/joss.02825

.. |arXiv| image:: https://img.shields.io/badge/arXiv-1708.07377-blue
    :target: https://arxiv.org/abs/1708.07377

|binder| |RTD| |Tests| |Build| |code-style| |JOSS| |arXiv|

`Installation Guide <https://pyautolens.readthedocs.io/en/latest/installation/overview.html>`_ |
`readthedocs <https://pyautolens.readthedocs.io/en/latest/index.html>`_ |
`Introduction on Binder <https://mybinder.org/v2/gh/Jammy2211/autolens_workspace/release?filepath=introduction.ipynb>`_ |
`HowToLens <https://pyautolens.readthedocs.io/en/latest/howtolens/howtolens.html>`_

.. image:: https://github.com/Jammy2211/PyAutoLogo/blob/main/gifs/pyautolens.gif?raw=true
  :width: 900

When two or more galaxies are aligned perfectly down our line-of-sight, the background galaxy appears multiple times.

This is called strong gravitational lensing and **PyAutoLens** makes it simple to model strong gravitational lenses.

Getting Started
---------------

The following links are useful for new starters:

- `The PyAutoLens readthedocs <https://pyautolens.readthedocs.io/en/latest>`_: including `an installation guide <https://pyautolens.readthedocs.io/en/latest/installation/overview.html>`_ and `overview of lens analysis features <https://pyautolens.readthedocs.io/en/latest/overview/overview_1_lensing.html>`_.

- `The introduction Jupyter Notebook on Binder <https://mybinder.org/v2/gh/Jammy2211/autolens_workspace/release?filepath=introduction.ipynb>`_: try **PyAutoLens** in a web browser (without installation).

- `The autolens_workspace GitHub repository <https://github.com/Jammy2211/autolens_workspace>`_: example scripts and the HowToLens Jupyter notebook lectures.

Support
-------

Support for installation issues, help with lens modeling and using **PyAutoLens** is available by
`raising an issue on the GitHub issues page <https://github.com/Jammy2211/PyAutoLens/issues>`_.

We also offer support on the **PyAutoLens** `Slack channel <https://pyautolens.slack.com/>`_, where we also provide the
latest updates on **PyAutoLens**. Slack is invitation-only, so if you'd like to join send
an `email <https://github.com/Jammy2211>`_ requesting an invite.

HowToLens
---------

For users less familiar with gravitational lensing, Bayesian inference and scientific analysis
you may wish to read through the **HowToLens** lectures. These teach you the basic principles of gravitational lensing
and Bayesian inference, with the content pitched at undergraduate level and above.

A complete overview of the lectures `is provided on the HowToLens readthedocs page <https://pyautolens.readthedocs.io/en/latest/howtolens/howtolens.html>`_

API Overview
------------

Lensing calculations are performed in **PyAutoLens** by building a ``Tracer`` object from ``LightProfile``,
``MassProfile`` and ``Galaxy`` objects. Below, we create a simple strong lens system where a redshift 0.5
lens ``Galaxy`` with an ``Isothermal`` ``MassProfile`` lenses a background source at redshift 1.0 with an
``Exponential`` ``LightProfile`` representing a disk.

.. code-block:: python

    import autolens as al
    import autolens.plot as aplt
    from astropy import cosmology as cosmo

    """
    To describe the deflection of light by mass, two-dimensional grids of (y,x) Cartesian
    coordinates are used.
    """
    grid = al.Grid2D.uniform(
        shape_native=(50, 50),
        pixel_scales=0.05,  # <- Conversion from pixel units to arc-seconds.
    )

    """
    The lens galaxy has an elliptical isothermal mass profile and is at redshift 0.5.
    """
    mass = al.mp.Isothermal(
        centre=(0.0, 0.0), ell_comps=(0.1, 0.05), einstein_radius=1.6
    )

    lens_galaxy = al.Galaxy(redshift=0.5, mass=mass)

    """
    The source galaxy has an elliptical exponential light profile and is at redshift 1.0.
    """
    disk = al.lp.Exponential(
        centre=(0.3, 0.2),
        ell_comps=(0.05, 0.25),
        intensity=0.05,
        effective_radius=0.5,
    )

    source_galaxy = al.Galaxy(redshift=1.0, disk=disk)

    """
    We create the strong lens using a Tracer, which uses the galaxies, their redshifts
    and an input cosmology to determine how light is deflected on its path to Earth.
    """
    tracer = al.Tracer(
        galaxies=[lens_galaxy, source_galaxy],
        cosmology = al.cosmo.Planck15()
    )

    """
    We can use the Grid2D and Tracer to perform many lensing calculations, for example
    plotting the image of the lensed source.
    """
    tracer_plotter = aplt.TracerPlotter(tracer=tracer, grid=grid)
    tracer_plotter.figures_2d(image=True)

With **PyAutoLens**, you can begin modeling a lens in minutes. The example below demonstrates a simple analysis which
fits the lens galaxy's mass with an ``Isothermal`` and the source galaxy's light with a ``Sersic``.

.. code-block:: python

    import autofit as af
    import autolens as al
    import autolens.plot as aplt

    """
    Load Imaging data of the strong lens from the dataset folder of the workspace.
    """
    dataset = al.Imaging.from_fits(
        data_path="/path/to/dataset/image.fits",
        noise_map_path="/path/to/dataset/noise_map.fits",
        psf_path="/path/to/dataset/psf.fits",
        pixel_scales=0.1,
    )

    """
    Create a mask for the imaging data, which we setup as a 3.0" circle, and apply it.
    """
    mask = al.Mask2D.circular(
        shape_native=dataset.shape_native,
        pixel_scales=dataset.pixel_scales,
        radius=3.0
    )
    dataset = dataset.apply_mask(mask=mask)

    """
    We model the lens galaxy using an elliptical isothermal mass profile and
    the source galaxy using an elliptical sersic light profile.

    To setup these profiles as model components whose parameters are free & fitted for
    we set up each Galaxy as a `Model` and define the model as a `Collection` of all galaxies.
    """
    # Lens:

    mass = af.Model(al.mp.Isothermal)
    lens = af.Model(al.Galaxy, redshift=0.5, mass=lens_mass_profile)

    # Source:

    disk = af.Model(al.lp.Sersic)
    source = af.Model(al.Galaxy, redshift=1.0, disk=disk)

    # Overall Lens Model:
    model = af.Collection(galaxies=af.Collection(lens=lens, source=source))

    """
    We define the non-linear search used to fit the model to the data (in this case, Dynesty).
    """
    search = af.Nautilus(name="search[example]", n_live=50)

    """
    We next set up the `Analysis`, which contains the `log likelihood function` that the
    non-linear search calls to fit the lens model to the data.
    """
    analysis = al.AnalysisImaging(dataset=dataset)

    """
    To perform the model-fit we pass the model and analysis to the search's fit method. This will
    output results (e.g., dynesty samples, model parameters, visualization) to hard-disk.
    """
    result = search.fit(model=model, analysis=analysis)

    """
    The results contain information on the fit, for example the maximum likelihood
    model from the Dynesty parameter space search.
    """
    print(result.samples.max_log_likelihood())


[![Continuous Integration](https://github.com/MolSSI-BSE/basis_set_exchange/workflows/Continuous%20Integration/badge.svg)](https://github.com/MolSSI-BSE/basis_set_exchange/actions?query=workflow%3A%22Continuous+Integration%22)
[![PyPI version](https://img.shields.io/pypi/v/basis_set_exchange.svg)](https://pypi.org/project/basis_set_exchange/)

## Basis Set Exchange Website

If you are looking for the Basis Set Exchange website (which can be
used to browse and download this data in a more user-friendly way),
visit https://www.basissetexchange.org

## Overview

This project is a library containing basis sets for use in quantum
chemistry calculations.  In addition, this library has functionality
for manipulation of basis set data.

The goal of this project is to create a consistent, thoroughly curated
database of basis sets, and to provide a standard nomenclature for
quantum chemistry.

The data contained within this library is being thoroughly evaluated
and checked against relevant literature, software implementations, and
other databases when available. The original data from the PNNL Basis
Set Exchange is also available.

This library is used to form the backend of the new Basis Set Exchange
website.

This project is a collaboration between the Molecular Sciences Software
Institute (https://molssi.org) and the Environmental Molecular Sciences
Laboratory (https://www.emsl.pnl.gov)

## Citation

When publishing results obtained from use of the Basis Set Exchange software, please cite:

 * *A New Basis Set Exchange: An Open, Up-to-date Resource for the Molecular Sciences Community* Benjamin P. Pritchard, Doaa Altarawy, Brett Didier, Tara D. Gibson, and Theresa L. Windus *J. Chem. Inf. Model.* **2019**, 59(11), 4814-4820 doi:[10.1021/acs.jcim.9b00725](https://doi.org/10.1021/acs.jcim.9b00725)

For citing the previous EMSL/PNNL Basis Set Exchange, please cite the following references:

 * *The Role of Databases in Support of Computational Chemistry Calculations,* Feller, D., *J. Comp. Chem.* **1996**, 17(13), 1571-1586, doi:[10.1002/(SICI)1096-987X(199610)17:13<1571::AID-JCC9>3.0.CO;2-P](https://doi.org/10.1002/(SICI)1096-987X(199610)17:13<1571::AID-JCC9>3.0.CO;2-P)
 * *Basis Set Exchange: A Community Database for Computational Sciences Schuchardt,* K.L., Didier, B.T., Elsethagen, T., Sun, L., Gurumoorthi, V., Chase, J., Li, J., and Windus, T.L. *J. Chem. Inf. Model.* **2007**, 47(3), 1045-1052, doi:[10.1021/ci600510j](https://doi.org/10.1021/ci600510j)

## Documentation

Full user and developer documentation can be found at
https://molssi-bse.github.io/basis_set_exchange

An overview of the project and its design is also available at
https://molssi-bse.github.io/basis_set_exchange/project_doc.html

## Command line interface

This library also includes a command line interface.
See https://molssi-bse.github.io/basis_set_exchange/bse_cli.html for how to use it.

## Installation
This project can be installed via pip/PyPI.
```
pip install basis_set_exchange
```

If checking out from github, you can do a local install of the Python
directory,
```
pip install -e .
```

## Testing

Tests can be run using `py.test -v` once installed. Thorough (but very
long) tests can be run with `py.test --runslow`.

## Examples
```python
import basis_set_exchange as bse

# Obtain the STO-3G basis set in nwchem format (as a string) for hydrogen and carbon
bse.get_basis('STO-3G', elements=[1,6], fmt='nwchem')

# Obtain the references for the above
bse.get_references('STO-3G', elements=[1,6], fmt='txt')
```

For more documentation, see https://molssi-bse.github.io/basis_set_exchange

## Command line

Same as above, but using the command line

``$ bse get-basis sto-3g nwchem --elements=1,6``

``$ bse get-refs sto-3g txt --elements=1,6``

## License

This project is released under the BSD 3-Clause license. See [LICENSE](LICENSE) for details.


# PowerGenome

[![The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)
[![code style black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4426097.svg)](https://doi.org/10.5281/zenodo.4426096)
[![pytest](https://github.com/PowerGenome/PowerGenome/actions/workflows/pytest.yml/badge.svg)](https://github.com/PowerGenome/PowerGenome/actions/workflows/pytest.yml)
[![codecov](https://codecov.io/gh/PowerGenome/PowerGenome/branch/master/graph/badge.svg?token=7KJYLE3jOW)](https://codecov.io/gh/PowerGenome/PowerGenome)
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/PowerGenome/PowerGenome/master.svg)](https://results.pre-commit.ci/latest/github/PowerGenome/PowerGenome/master)

The code and data for PowerGenome are under active development and some changes may break existing functions. Keep up to date with major code and data releases by joining [PowerGenome on groups.io](https://groups.io/g/powergenome). And **check out the growing documentation on the [Wiki](https://github.com/PowerGenome/PowerGenome/wiki)** for helpful background information.

Power system optimization models can be used to explore the cost and emission implications of different regulations in future energy systems. One of the most difficult parts of running these models is assembling all the data. A typical model will define several regions, each of which need data such as:

- All existing generating units (perhaps grouped into a few discrete clusters within each region)
- Transmission constraints between regions
- Hourly load profiles (including new loads from vehicle and building electrification)
- Hourly generation profiles for wind & solar
- Cost estimates for new generating units

Because computational complexity and run times increase as the number of regions and generating unit clusters increases, a user might want only want to disaggregate regions and generating units close to the primary region of interest. For example, a study focused on clean electricity regulations in New Mexico might combine several states in the Pacific Northwest into a single region while also splitting Arizona combined cycle units into multiple clusters.

The goal of PowerGenome is to let a user make all of these choices in a settings file and then run a single script that generates input files for the power system model. PowerGenome currently generates input files for [GenX](https://energy.mit.edu/wp-content/uploads/2017/10/Enhanced-Decision-Support-for-a-Changing-Electricity-Landscape.pdf), and we hope to expand to other models in the near future.

## Data

PowerGenome uses data from a number of different sources, including EIA, NREL, and EPA. The data are accessed through a combination of sqlite databases, CSV files, and parquet data files. All data files [are available here](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing).

1. EIA data on existing generating units are already compiled into a [single sqlite database (PUDL)](https://doi.org/10.5281/zenodo.3653158) (see instructions for using it below). This file is available at the link above or you can download it from the Zenodo repository.
2. A second sqlite database (`pg_misc_tables_efs_2023_2.sqlite`) has tables with new resource costs from NREL ATB, transmission constraints between IPM regions from EIA, and hourly demand within each IPM region derived from NREL or FERC data.
3. The hourly incremental demand for different flexible demand technologies, and stock values across a range of projection scenarios (`efs_files_utc`).

## PUDL Dependency

This project pulls data from [PUDL](https://github.com/catalyst-cooperative/pudl). As such, it requires installation of PUDL to access a normalized sqlite database and some of the convienience PUDL functions.

`catalystcoop.pudl` is included in the `environment.yml` file and will be installed automatically in the conda environment (see instructions below). Catalyst Cooperative will be creating versioned data releases of PUDL, which can be [accessed on Zenodo](https://doi.org/10.5281/zenodo.3653158). Download the zip file from Zenodo, unzip it, and find the sqlite database under `pudl_data/sqlite/pudl.sqlite`. Note that the version of `catalystcoop.pudl` software may change based on the database version you use. Look on the right-hand side of the zenodo archive to see what software version was used to compile the data. If the version in your conda environment does not match the version used to compile the data, you can change it in the `environment.yml` file or install a [different version](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#installing-packages) using `mamba install catalystcoop.pudl=<your_version>`.

![PUDL software version for database](/docs/_static/pudl_version.png)

## Installation from GitHub

1. Clone this repository to your local machine and navigate to the top level (PowerGenome) folder.

2. Create a conda environment named `powergenome` using the provided `environment.yml` file. If you don't already use conda it is easiest to download and install [Mambaforge](https://github.com/conda-forge/miniforge#mambaforge), which will install conda with mamba in the `base` environment. See [this description](https://bioconda.github.io/faqs.html#what-s-the-difference-between-miniconda-miniforge-mambaforge-micromamba) for more information on the difference between different ways to install conda and mamba. Conda usually fail to resolve dependencies in under a day so I highly recommend that you either start with Mambaforge or [install mamba in your `base` environment](https://mamba.readthedocs.io/en/latest/installation.html#existing-conda-install) and use it instead.

```sh
mamba env create -f environment.yml
```

3. Activate the `powergenome` environment.

```sh
conda activate powergenome
```

4. pip-install an editable version of this project

```sh
pip install -e .
```

5. Download the PUDL database [from Zenodo](https://doi.org/10.5281/zenodo.3653158) or the [PowerGenome data repository](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing), unzip it, and copy the `/pudl_data/sqlite/pudl.sqlite` to wherever you would like to store PowerGenome data on your computer. The zip file contains other data sets that aren't needed for PowerGenome and can be deleted.  Note that as of May 2023 the most recent version of this database (v2022.11.30) is compatible with `catalystcoop.pudl` version v2022.11.30 and may not work if an earlier software version is included in your conda environment.

6. Download the additional PowerGenome database from the [PowerGenome data repository](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing). It includes NREL ATB cost data, transmission constraints between IPM regions, and hourly demand for each IPM region. Hourly demand is based on a 2012 weather year and was constructed either directly from FERC 714 data (`load_curves_ferc`) or from NREL EFS data (`load_curves_nrel_efs`) that also sources back to FERC 714. The NREL load curves, which separate hourly demand by sector and subsector, are now the default source for load curves in PowerGenome. See [the wiki](https://github.com/PowerGenome/PowerGenome/wiki/Settings-files#demand) for more information. These files will eventually be provided through a data repository with citation information.

7. Download the appropriate renewable resource data files from the [PowerGenome data repository](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing). There is a single set of generation profiles and resource group folders specific to different regional aggregations. Read through the [included README](https://docs.google.com/document/d/1p_zDk6ng4tvgL1v1ZAXkvY9b34FmaLqdWFlJmUHP1Eo/edit?usp=share_link) for more background. This folder contains:

- `generation_profiles` can be saved in a single place and used across multiple studies.
- Each of the folders under `resource_groups` has CSV files that tell PowerGenome the metro that each potential wind/solar site will deliver power to based on a set of regional aggregations. Use the corresponding regional aggregations in your settings file. You can request new resource group files for different regional aggregations on the PowerGenome [repository discussion page](https://github.com/PowerGenome/PowerGenome/discussions)

8. Download data files derived from NREL's EFS from the [PowerGenome data repository](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing). These provide hourly demand profiles for growing electrification technologies like electric vehicles and heat pumps and are used to both build up demand profiles in the future and create flexible demand resources that can shift their load.

9. Download distributed generation profiles from the [PowerGenome data repository](https://drive.google.com/drive/folders/1K5GWF5lbe-mKSTUSuJxnFdYGCdyDJ7iE?usp=sharing) compiled from NREL Cambium 2022 scenarios.

10. Create the file `PowerGenome/powergenome/.env`. In this file, add:

- `PUDL_DB=YOUR_PATH_HERE` (your path to the PUDL database downloaded in step 5)
- `PG_DB=YOUR_PATH_HERE` (your path to the additional PowerGenome data downloaded in step 6)
- `RESOURCE_GROUP_PROFILES=YOUR_PATH_HERE` (your path to the folder with hourly wind/solar generation parquet files)
- `EFS_DATA=YOUR_PATH_HERE` (your path to the folder with EFS derived data files)
- `DISTRIBUTED_GEN_DATA=YOUR_PATH_HERE` (your path to the folder with distributed generation profiles)
- OPTIONAL: `RESOURCE_GROUPS=YOUR_PATH_HERE` (your path to the resource groups data for a project -- **this can be included in your settings file instead of the .env file**)

Quotation marks are only needed if your values contain spaces. The `.env` file is included in `.gitignore` and will not be synced with the repository.

## Installation with a packaged version (pip/conda-forge)

Installing Powergenome with pip has only been tested within a conda environment but it should work in other environment management systems. Make sure that you have an updated version of pip installed. If you hit dependency errors I suggest trying to install them using mamba or conda. PowerGenome has `catalystcoop.pudl` as a dependency, which has a large number of its own dependencies. I have not (yet) had to install `catalystcoop.pudl` using mamba but doing so may help clear up errors.

Depending on your operating system you might also have issues installing some other packages from pip. The example code below is what works for me on a Mac, where python-snappy fails to build wheels.

```sh
(base) conda create -n powergenome python=3.10 pip python-snappy=0.6.1
(base) conda activate powergenome
(powergenome) pip install powergenome
```

PowerGenome has been submitted to conda-forge but is not yet available.

If you are installing a packaged version of PowerGenome you won't be able to easily use a .env file. Instead, add the environment parameters (`PUDL_DB`, `PG_DB`, etc) to a YAML file in the same folder as the rest of your settings. It doesn't really matter which file these parameters are included in but creating a new file such as `env_params.yml` will help keep them separate from other settings parameters that might be shared with other PowerGenome users.

## Running code

### Suggested folder structure

It is best practice to set up project folders outside of the cloned repository so that git doesn't track any new/changed files within the upper-level `PowerGenome` folder. Try copying one of the example systems (settings file and extra inputs) and modifying it. Copy the `notebooks` folder into your project folder, change the path to the settings file as needed, and run code in the notebooks. This can also be a good way to learn how data are created in PowerGenome and debug problem.

Keeping project folders separate from the cloned `PowerGenome` folder will also make it easier to pull changes as they are released.

### Example systems

A few example systems are included under `PowerGenome/example_systems`. Each system has settings files in a folder (`settings`) and a folder with extra user inputs (`extra_inputs`). The different example systems are not meant to be accurate for real-world analysis, so please do not blindly use the external data files included with them in your own studies!

### Settings

Settings are controlled in a set of YAML files within a folder or combined into a single file. An example folder of settings files (`settings`) and folder with extra user inputs (`extra_inputs`) are included in each of the example systems. Scenario options across different planning years are defined in the file `test_scenario_inputs.csv`. Documentation on extra inputs is included in the folder of each example system.

### Example notebooks

A series of example notebooks are included in [`PowerGenome/notebooks`](/notebooks) describe how to access different functions within PowerGenome to create resource clusters, variable generation profiles, fuel costs, hourly demand, and transmission constraints. They include a description of how the data are compiled and the settings parameters that are required for each type of data.

### Command line interface

The outputs are all formatted for GenX we hope to make the data formatting code more module to allow users to easily switch between outputs for different power system models.

Functions from each module can be imported and used in an interactive environment (e.g. JupyterLab). Examples of how to load data in this way are included in `PowerGenome/notebooks`. To run from the command line, navigate to a project folder that contains a settings file and extra inputs (e.g. `myproject/powergenome`), activate the  `powergenome` conda environment, and use the command `run_powergenome_multiple` with flags for the settings file name and where the results should be saved. Since the `powergenome` package is installed in the `powergenome` conda environment, you can run the command line function from anywhere on your computer (not just within the cloned `PowerGenome` folder).

```sh
run_powergenome_multiple --settings_file settings --results_folder test_system
```

Or to set up files for multi-period runs in GenX

```sh
run_powergenome_multiple --settings_file settings --results_folder test_system --multi-period
```

The command line arguments `--settings_file` and `--results_folder` can be shortened to `-sf` and `-rf` respectively. For all options, run:

```sh
run_powergenome_multiple --help
```

A folder with extra user inputs is required when using the `run_powergenome_multiple` command. The name of this folder is defined in the settings YAML file with the `input_folder` parameter. Look at the files in each example system for test cases to follow.

If you have previously installed PowerGenome and the `run_powergenome_multiple` command doesn't work, try reinstalling it using `pip install -e .` as described above. If you downloaded the custom PUDL database before May of 2020, some errors may be resolved by downloading a new version.

## Licensing

PowerGenome is released under the [MIT License](https://opensource.org/licenses/MIT). Most data inputs are from US government sources (EIA, EPA, FERC, etc), which should not be [subject to copyright in the US](https://www.usa.gov/government-works). Hourly FERC demand data has been cleaned using [techniques](https://github.com/truggles/EIA_Cleaned_Hourly_Electricity_Demand_Code) developed by Tyler Ruggles and David Farnham, and allocated to IPM regions using [methods developed](https://github.com/catalyst-cooperative/electricity-demand-mapping) by Catalyst Cooperative. Hourly generation profiles for wind and solar resources were created by [Vibrant Clean Energy](https://www.vibrantcleanenergy.com/) and provided without usage restrictions. All PowerGenome data outputs are released under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode) license.

## Contributing

Contributions are welcome! There is significant work to do on this project and additional perspective on user needs will help make it better. If you see something that needs to be improved, [open an issue](https://github.com/gschivley/PowerGenome/issues). If you have questions or need assistance, join [PowerGenome on groups.io](https://groups.io/g/powergenome) and post a message there.

Pull requests are always welcome. To start modifying/adding code, make a fork of this repository, create a new branch, and [submit a pull request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork).

All code added to the project should be formatted with [black](https://black.readthedocs.io/en/stable/). After making a fork and cloning it to your own computer, run `pre-commit install` to [install the git hook scripts](https://pre-commit.com/#3-install-the-git-hook-scripts) that will run every time you make a commit. These hooks will automatically run `black` (in case you forgot), fix trailing whitespace, check yaml formatting, etc.