# Flare-Cogs - (flare#0001)

# Contact
You can contact me in the Red 3rd party server in #support_flare-cogs

---

# Installation
`[p]repo add flare-cogs https://github.com/flaree/Flare-Cogs`

`[p]cog install flare-cogs <cog_name>`

---
| Name           | Status/Version   | Description (Click to see full status)                                                                                | Authors                                  |
|----------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------|
| AntiSpam       | 0.1.0            | Block users who spam commands for a defined period of time.                                                                                | flare(flare#0001)                        |
| ApiTools       | 0.0.3            | Simple GET/POST requests via the bot.                                                                                | flare(flare#0001)                        |
| BotListsPost   | 0.0.6            | Post server count to various discord bot lists.                                                                                | flare(flare#0001)                        |
| CashDrop       | 0.2.2            | Random credits drop, where people can answer questions or pick them up!                                                                                | flare(flare#0001)                        |
| CommandStats   | 0.1.5            | Track all commands used.Track all commands used globally, guild wise and during the current session.                                                        | flare(flare#0001)                        |
| Covid          | 0.3.1            | Show Covid-19 (Novel Coronavirus) Statistics.List stats of Covid-19 (Novel Coronavirus), global or countrywise!                                             | flare(flare#0001)                        |
| Crypto         | 0.1.0            | Buy and sell cyrptocurrencies with Red economy.Buy and sell cryptocurrencies with Red economy.                                                              | flare(flare#0001), Flame, and TrustyJAID |
| DmInvites      | 0.0.4            | Respond with the bot invite link when DM'd a server invite link.Respond with the bots invite link if the bot recieves a message containing a server invite. | flare(flare#0001)                        |
| VerifyEmail    | 0.1.1            | Secure your server by making users verify their account with an email.                                                                                | flare(flare#0001)                        |
| F1             | 0.4.0            | Data related to F1.F1 data, races, drivers, constructors etc.                                                                                | flare(flare#0001)                        |
| Faceit         | 0.2.0            | List stats of faceit users, game history or specific games and even ongoing games!                                                                          | flare(flare#0001)                        |
| Forward        | 1.2.9            | Forward messages sent to the bot to the bot owner or in a specified channel.                                                                                | flare(flare#0001) and Aikaterna          |
| Giveaways      | 1.3.2            | Giveaway cog with features such duration or end timing, multipliers, role only acces, bank integration etc.                                                 | flare(flare#0001)                        |
| Highlight      | 1.11.1           | Highlight certain words to be notified of if said in a certain channel.                                                                                | flare(flare#0001)                        |
| JoinMessage    | 0.1.0            | Send a message to a guilds general chat, system channel or first available channel.                                                                         | flare (flare#0001)                       |
| Jsk            | 0.0.2            | Jishaku ported to Red.                                                                                | flare(flare#0001)                        |
| Mod            | 1.2.0            | Mod with custom messages.Core mod with the inclusion of custom messages for banning, kicking and unbanning.                                                 | flare(flare#0001)                        |
| Covid          | 0.0.3            | Grab the latest headlines, nationally or globally.Grab breaking headline around the world!                                                                  | flare(flare#0001)                        |
| Palette        | 0.0.2            | Show colour palette of images, avatars etc.Show colour palette of images, avatars, emojis etc                                                               | flare(flare#0001)                        |
| PermChecker    | 0.1.0            | This cog is a basic permission checker, used to check if a user/role/channel has a set permission.                                                          | flare(flare#0001)                        |
| R6             | 1.6.0            | Show R6 Statistics.List R6 Statistics from seasons, individual operators, all operators and more!                                                           | flare(flare#0001)                        |
| RedditPost     | 0.6.0            | Reddit Autoposting of new content.                                                                                | flare(flare#0001)                        |
| RoleHistory    | 0.1.0            | Visualize the history of a user's roles.                                                                                | flare(flare#0001)                        |
| Snipe          | 0.6.0            | Snipe the last message deleted in a channel.Snipe command converted to Red, get the last message deleted in a channel.                                      | flare(flare#0001)                        |
| StickBugged    | 0.0.1            | Get stickbugged.                                                                                | flare(flare#0001)                        |
| ThreadBumper   | 0.0.1            | Bump threads to keep them alive forever.                                                                                | flare(flare#0001)                        |
| TikTokReposter | 0.0.3            | Repost TikToks from TikTok to Discord!Repost TikToks from TikTok to Discord automatically or manually!                                                      | flare(flare#0001)                        |
| trigger        | 0.2.2            | Allow for the creation of triggers to respond to keywords in messages.                                                                                | flare(flare#0001)                        |
| Unbelievaboat  | 0.5.10           | Unbelievaboat economy commands converted for Red use.Unbelievaboat economy commands converted for Red use..                                                 | flare(flare#0001)                        |
| Userinfo       | 0.4.1            | Userinfo with user badges and economy details.Show a users normal userinfo + their badges and shared servers and bank stuff.                                | flare(flare#0001)                        |
| VoiceTracker   | 0.0.2            | Track the voice chat of yourself or those in your server.                                                                                | flare(flare#0001)                        |
---

---
## Cogs in other repos.
---
| Name | Description
| --- | --- |
| Pokecord | Pokecord - found @ [flaree/pokecord-red](https://github.com/flaree/pokecord-red) |
| LastFM | LastFM stats - port of Miso Bot - found @ [flaree/lastfm-red](https://github.com/flaree/lastfm-red) |

---


# RAiDER

Raytracing Atmospheric Delay Estimation for RADAR

[![Language](https://img.shields.io/badge/python-3.7%2B-blue.svg)](https://www.python.org/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/dbekaert/RAiDER/blob/dev/LICENSE)
[![CircleCI](https://circleci.com/gh/dbekaert/RAiDER.svg?style=svg)](https://circleci.com/gh/dbekaert/RAiDER)
[![Coverage Status](https://coveralls.io/repos/github/dbekaert/RAiDER/badge.svg?branch=dev)](https://coveralls.io/github/dbekaert/RAiDER?branch=dev)

RAiDER-tools is a package in Python which contains tools to calculate tropospheric corrections for Radar using a raytracing implementation. Its development was funded under the NASA Sea-level Change Team (NSLCT) program, the Earth Surface and Interior (ESI) program, and the NISAR Science Team (NISAR-ST) (NTR-51433). U.S. Government sponsorship acknowledged.

Copyright (c) 2019-2022, California Institute of Technology ("Caltech"). All rights reserved.

THIS IS RESEARCH CODE PROVIDED TO YOU "AS IS" WITH NO WARRANTIES OF CORRECTNESS. USE AT YOUR OWN RISK.

## Contents

1. [Getting Started](#1-getting-started)
    - [Installing With Conda](#installing-with-conda)
    - [Using the Docker Image](#using-the-docker-image)
    - [Installing from Source](#installing-from-source)
2. [Setup of third party weather model access](#2-setup-of-third-party-weather-model-access)
3. [Running RAiDER and Documentation](#3-running-raider-and-documentation)
4. [Citing](#4-citation)
5. [Development](#5-development)
    - [Contributors](#contributors)
------

## 1. Getting Started

RAiDER has been tested on the following systems:
- Ubuntu v.16 and up
- Mac OS v.10 and up

RAiDER does **not** currently run on arm64 processors on Mac. We will update this note once the build becomes available.

### Installing With Conda

RAiDER is available on [conda-forge](https://anaconda.org/conda-forge/raider). __[Conda](https://docs.conda.io/en/latest/index.html)__ is a cross-platform way to use Python that allows you to setup and use "virtual environments." These can help to keep dependencies for different sets of code separate. We recommend using [Miniforge](https://github.com/conda-forge/miniforge), a conda environment manager that uses conda-forge as its default code repo. Alternatively,see __[here](https://docs.anaconda.com/anaconda/install/)__ for help installing Anaconda and __[here](https://docs.conda.io/en/latest/miniconda.html)__ for installing Miniconda.

Installing RAiDER:
```
conda env create --name RAiDER  -c conda-forge raider
conda activate RAiDER
```

### Using the Docker image

RAiDER provides a [docker container image](https://docs.docker.com/get-started/) with all the necessary dependencies pre-installed. To get the latest released version:
```
docker pull ghcr.io/dbekaert/raider:latest
```
a specific release version (>=v0.2.0 only):
```
docker pull ghcr.io/dbekaert/raider:0.2.0
```
or the current development version:
```
docker pull ghcr.io/dbekaert/raider:test
```

To run `raider.py` inside the container:
```
docker run -it --rm ghcr.io/dbekaert/raider:latest
```
To mount your current directory inside the container so that files will be written back to your local machine:
```
docker run -it -v ${PWD}:/home/raider/work --rm ghcr.io/dbekaert/raider:latest
cd work
```
To jump into a `bash` shell inside the container:
```
docker run -it --rm --entrypoint /bin/bash ghcr.io/dbekaert/raider:latest -l
```

For more docker run options, see: <https://docs.docker.com/engine/reference/run/>.

------
## 2. Setup of third party weather model access

RAiDER has the ability to download weather models from third-parties; some of which require license agreements. See [here](https://github.com/dbekaert/RAiDER/blob/dev/docs/WeatherModels.md) for details.

------
## 3. Running RAiDER and Documentation

For detailed documentation, examples, and Jupyter notebooks see the [RAiDER-docs repository](https://github.com/dbekaert/RAiDER-docs).
We welcome contributions of other examples on how to leverage the RAiDER  (see [here](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md) for instructions).
``` raiderDelay.py -h ``` provides a help menu and list of example commands to get started.
The RAiDER scripts are highly modularized in Python and allows for building your own processing workflow.

------
## 4. Citation
TODO

------
## 5. Development

Contributions are welcome and heartily encourage! See our [contributing guide](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md).

### Development install
For development, we recommend installing directly from source.
```
git clone https://github.com/dbekaert/RAiDER.git
cd RAiDER
conda env create -f environment.yml
conda activate RAiDER
python -m pip install -e .
```
For more details on installing from source see [here](https://github.com/dbekaert/RAiDER/blob/dev/docs/Installing_from_source.md).

------
### Contributors

* David Bekaert
* Jeremy Maurer
* Raymond Hogenson
* Brett Buzzanga
* Piyush Agram _(Descartes Labs)_
* Yang Lei
* Rohan Weeden
* Simran Sangha
* [_other community members_](https://github.com/dbekaert/RAiDER/graphs/contributors)

We welcome community contributions! For instructions see [here](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md).


# pytest-splunk-addon

![PyPI](https://img.shields.io/pypi/v/pytest-splunk-addon)
![Python](https://img.shields.io/pypi/pyversions/pytest-splunk-addon.svg)

## What is PSA

A Dynamic test tool for Splunk Apps and Add-ons.

## Usage

For full usage instructions, please visit the [documentation](https://splunk.github.io/pytest-splunk-addon).


# Standardized Emission and Waste Inventories (StEWI)
[![DOI - 10.3390/app12073447](https://img.shields.io/badge/DOI-10.3390%2Fapp12073447-blue)](https://doi.org/10.3390/app12073447)
[![DOI](https://zenodo.org/badge/101654522.svg)](https://zenodo.org/doi/10.5281/zenodo.1342761)
[![DOI - 10.23719/1526441](https://img.shields.io/badge/v1.0%20DataProducts-10.23719%2F1526441-blue)](https://doi.org/10.23719/1526441)
[![build](https://github.com/USEPA/standardizedinventories/actions/workflows/python-package.yml/badge.svg)](https://github.com/USEPA/standardizedinventories/actions/workflows/python-package.yml)

StEWI is a collection of Python modules that provide processed USEPA facility-based emission and waste generation inventory data in standard tabular formats.
 The standard outputs may be further aggregated or filtered based on given criteria, and can be combined based on common facility and flows
  across the inventories.

StEWI consists of a core module, `stewi`, that digests and provides the USEPA inventory data in standard formats. Two matcher modules, the `facilitymatcher`
and `chemicalmatcher`, provide commons IDs for facilities and flows across inventories, which is used by the `stewicombo` module
to combine the data, and optionally remove overlaps and remove double counting of groups of chemicals based on user preferences.

StEWI v1 was peer-reviewed internally at USEPA and externally through _Applied Sciences_.
An article describing StEWI was published in a special issue of Applied Sciences: [Advanced Data Engineering for Life Cycle Applications](https://doi.org/10.3390/app12073447).

## USEPA Inventories Covered By Data Reporting Year (current version)

|Source|2011|2012|2013|2014|2015|2016|2017|2018|2019|2020|2021|
|---|---|---|---|---|---|---|---|---|---|---|---|
|[Discharge Monitoring Reports](https://echo.epa.gov/tools/data-downloads/icis-npdes-dmr-and-limit-data-set)* |x|x|x|x|x|x|x|x|x|x|x|
|[Greenhouse Gas Reporting Program](https://www.epa.gov/ghgreporting) |x|x|x|x|x|x|x|x|x|x|x|
|[Emissions & Generation Resource Integrated Database](https://www.epa.gov/energy/emissions-generation-resource-integrated-database-egrid) | | | |x| |x| |x|x|x|x|
|[National Emissions Inventory](https://www.epa.gov/air-emissions-inventories/national-emissions-inventory-nei)** |x|i|i|x|i|i|x|i|i|x| |
|[RCRA Biennial Report](https://www.epa.gov/hwgenerators/biennial-hazardous-waste-report)* |x| |x| |x| |x| |x| |x|
|[Toxic Release Inventory](https://www.epa.gov/toxics-release-inventory-tri-program)* |x|x|x|x|x|x|x|x|x|x|x|

*Earlier data exist and are accessible but have not been validated

**Only point sources included at this time from NEI; _i_ interim years between triennial releases, accessed through the Emissions Inventory System, are not validated

## Standard output formats

The core `stewi` module produces the following output formats:

[Flow-By-Facility](./format%20specs/FlowByFacility.md): Each row represents the total amount of release or waste of a single type in a given year from the given facility.

[Flow-By-Process](./format%20specs/FlowByProcess.md): Each row represents the total amount of release or waste of a single type in a given year from a specific process within the given facility.
Applicable only to NEI and GHGRP.

[Facility](./format%20specs/Facility.md): Each row represents a unique facility in a given inventory and given year

[Flow](./format%20specs/Flow.md): Each row represents a unique flow (substance or waste) in a given inventory and given year

The `chemicalmatcher` module produces:

[Chemical Matches](./format%20specs/ChemicalMatches.md): Each row provides a common identifier for an inventory flow chemical

The `facilitymatcher` module produces:

[Facility Matches](./format%20specs/FacilityMatches.md): Each row provides a common identifier for an inventory facility

The `stewicombo` module produces:

[Flow-By-Facility Combined](./format%20specs/FlowByFacilityCombo.md): Analagous to the flowbyfacility, with chemical and facilitymatches added

## Data Processing

The following describes details related to dataset access, processing, and validation

### DMR

Processing of the DMR uses the custom search option of the [Water Pollutant Loading Tool](https://echo.epa.gov/trends/loading-tool/get-data/custom-search/) with the following parameters:
- Parameter grouping: On - applies a parameter grouping function to avoid double-counting loads for pollutant parameters that represent the same pollutant
- Detection limit: Half - set all non-detects to ½ the detection limit
- Estimation: On - estimates loads when monitoring data are not reported for one or more monitoring periods in a reporting year
- Nutrient Aggregation: On - Nitrogen and Phosphorous flows are converted to N and P equivalents

For validation, the sum of facility releases (excluding N & P) are compared against reported state totals.
Some validation issues are expected due to differences in default parameters used by the water pollutant loading tool for calculating state totals.

### eGRID

eGRID data are sourced from EPA's [eGRID](https://www.epa.gov/egrid) site.
For validation, the sum of facility releases are compared against reported U.S. totals by flow.

### GHGRP

GHGRP data are sourced from EPA's [Envirofacts API](https://enviro.epa.gov/)
For validation, the sum of facility releases by subpart are compared against reported U.S. totals by subpart and flow.
The validation of some flows (HFC, HFE, and PFCs) are reported in carbon dioxide equivalents.
Mixed reporting of these flows in the source data in units of mass or carbon dioxide equivalents results in validation issues.

### NEI

NEI data are downloaded from the EPA Emissions Inventory System (EIS) Gateway and hosted on EPA [Data Commons](https://edap-ord-data-commons.s3.amazonaws.com/index.html?prefix=stewi/) for access by StEWI.
For validation, the sum of facility releases are compared against reported totals by flow. Validation is only available for triennial datasets.

### RCRAInfo

RCRAInfo data are sourced from the [Public Data Files](https://rcrapublic.epa.gov/rcrainfoweb/action/main-menu/view)
For validation, the sum of facility waste generation are compared against reported state totals as calculated for the National Biennial Report.

### TRI

TRI data are sourced from the [Basic Plus Data files](https://www.epa.gov/toxics-release-inventory-tri-program/tri-data-and-tools)
For validation, the sum of facility releases are compared to national totals by flow from the TRI Explorer.

## Combined Inventories

`stewicombo` module combines inventory data from within and across selected inventories by matching facilities in the [Facility Registry Service](https://www.epa.gov/frs) and
chemical flows using the [Substance Registry Service](https://sor.epa.gov/sor_internet/registry/substreg/LandingPage.do).
If the `remove_overlap` parameter is set to True (default), `stewicombo` combines records using the following default logic:
- Records that share a common compartment, SRS ID and FRS ID _within_ an inventory are summed.
- Records that share a common compartment, SRS ID and FRS ID _across_ an inventory are assessed by compartment preference (see `INVENTORY_PREFERENCE_BY_COMPARTMENT`).
- Additional steps are taken to avoid overlap of:
    - nutrient flow releases to water between the TRI and DMR
    - particulate matter releases to air reflecting PM < 10 and PM < 2.5 in the NEI
    - [Volatile Organic Compound (VOC)](https://github.com/USEPA/standardizedinventories/blob/master/stewicombo/data/VOC_SRS_IDs.csv) releases to air for individually reported VOCs and grouped VOCs

## Installation Instructions

Install a release directly from github using pip. From a command line interface, run:
> pip install git+https://github.com/USEPA/standardizedinventories.git@v1.1.0#egg=StEWI

where you can replace 'v1.1.0' with the version you wish to use under [Releases](https://github.com/USEPA/standardizedinventories/releases).

Alternatively, to install from the most current point on the repository:
```
git clone https://github.com/USEPA/standardizedinventories.git
cd standardizedinventories
pip install . # or pip install -e . for devs
```

### Secondary Context Installation Steps
In order to enable calculation and assignment of urban/rural secondary contexts, please refer to
[esupy's README.md](https://github.com/USEPA/esupy/tree/main#installation-instructions-for-optional-geospatial-packages) for installation instructions,
which may require a copy of the [`env_sec_ctxt.yaml`](https://github.com/USEPA/standardizedinventories/blob/master/env_sec_ctxt.yaml) file included here.

## Data Products
Output of StEWI can be accessed for selected releases without having to run StEWI.
See the [Data Product Links](https://github.com/USEPA/standardizedinventories/wiki/DataProductLinks) page for direct links to StEWI output files in Apache parquet format.

## Wiki
See the [Wiki](https://github.com/USEPA/standardizedinventories/wiki) for instructions on installation and use and for
citation and contact information.

## Disclaimer

The United States Environmental Protection Agency (EPA) GitHub project code is provided on an "as is" basis
and the user assumes responsibility for its use.  EPA has relinquished control of the information and no longer
has responsibility to protect the integrity , confidentiality, or availability of the information.
Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer,
or otherwise, does not constitute or imply their endorsement, recommendation or favoring by EPA.
The EPA seal and logo shall not be used in any manner to imply endorsement of any commercial product or activity
by EPA or the United States Government.


# Google Spreadsheets Python API v4

![main workflow](https://img.shields.io/github/actions/workflow/status/burnash/gspread/main.yaml?logo=github)
![GitHub licence](https://img.shields.io/pypi/l/gspread?logo=github)
![GitHub downloads](https://img.shields.io/github/downloads-pre/burnash/gspread/latest/total?logo=github)
![documentation](https://img.shields.io/readthedocs/gspread?logo=readthedocs)
![PyPi download](https://img.shields.io/pypi/dm/gspread?logo=pypi)
![PyPi version](https://img.shields.io/pypi/v/gspread?logo=pypi)
![python version](https://img.shields.io/pypi/pyversions/gspread?style=pypi)

Simple interface for working with Google Sheets.

Features:

- Open a spreadsheet by **title**, **key** or **URL**.
- Read, write, and format cell ranges.
- Sharing and access control.
- Batching updates.

## Installation

```sh
pip install gspread
```

Requirements: Python 3.8+.

## Basic Usage

1. [Create credentials in Google API Console](http://gspread.readthedocs.org/en/latest/oauth2.html)

2. Start using gspread

```python
import gspread

gc = gspread.service_account()

# Open a sheet from a spreadsheet in one go
wks = gc.open("Where is the money Lebowski?").sheet1

# Update a range of cells using the top left corner address
wks.update([[1, 2], [3, 4]], "A1")

# Or update a single cell
wks.update_acell("B42", "it's down there somewhere, let me take another look.")

# Format the header
wks.format('A1:B1', {'textFormat': {'bold': True}})
```

## v5.12 to v6.0 Migration Guide

### Upgrade from Python 3.7

Python 3.7 is [end-of-life](https://devguide.python.org/versions/). gspread v6 requires a minimum of Python 3.8.

### Change `Worksheet.update` arguments

The first two arguments (`values` & `range_name`) have swapped (to `range_name` & `values`). Either swap them (works in v6 only), or use named arguments (works in v5 & v6).

As well, `values` can no longer be a list, and must be a 2D array.

```diff
- file.sheet1.update([["new", "values"]])
+ file.sheet1.update([["new", "values"]]) # unchanged

- file.sheet1.update("B2:C2", [["54", "55"]])
+ file.sheet1.update([["54", "55"]], "B2:C2")
# or
+ file.sheet1.update(range_name="B2:C2", values=[["54", "55"]])
```

### More

<details><summary>See More Migration Guide</summary>

### Change colors from dictionary to text

v6 uses hexadecimal color representation. Change all colors to hex. You can use the compatibility function `gspread.utils.convert_colors_to_hex_value()` to convert a dictionary to a hex string.

```diff
- tab_color = {"red": 1, "green": 0.5, "blue": 1}
+ tab_color = "#FF7FFF"
file.sheet1.update_tab_color(tab_color)
```

### Switch lastUpdateTime from property to method

```diff
- age = spreadsheet.lastUpdateTime
+ age = spreadsheet.get_lastUpdateTime()
```

### Replace method `Worksheet.get_records`

In v6 you can now only get *all* sheet records, using `Worksheet.get_all_records()`. The method `Worksheet.get_records()` has been removed. You can get some records using your own fetches and combine them with `gspread.utils.to_records()`.

```diff
+ from gspread import utils
  all_records = spreadsheet.get_all_records(head=1)
- some_records = spreadsheet.get_all_records(head=1, first_index=6, last_index=9)
- some_records = spreadsheet.get_records(head=1, first_index=6, last_index=9)
+ header = spreadsheet.get("1:1")[0]
+ cells = spreadsheet.get("6:9")
+ some_records = utils.to_records(header, cells)
```

### Silence warnings

In version 5 there are many warnings to mark deprecated feature/functions/methods.
They can be silenced by setting the `GSPREAD_SILENCE_WARNINGS` environment variable to `1`

### Add more data to `gspread.Worksheet.__init__`

```diff
  gc = gspread.service_account(filename="google_credentials.json")
  spreadsheet = gc.open_by_key("{{key}}")
  properties = spreadsheet.fetch_sheet_metadata()["sheets"][0]["properties"]
- worksheet = gspread.Worksheet(spreadsheet, properties)
+ worksheet = gspread.Worksheet(spreadsheet, properties, spreadsheet.id, gc.http_client)
```

</details>

## More Examples

### Opening a Spreadsheet

```python
# You can open a spreadsheet by its title as it appears in Google Docs
sh = gc.open('My poor gym results') # <-- Look ma, no keys!

# If you want to be specific, use a key (which can be extracted from
# the spreadsheet's url)
sht1 = gc.open_by_key('0BmgG6nO_6dprdS1MN3d3MkdPa142WFRrdnRRUWl1UFE')

# Or, if you feel really lazy to extract that key, paste the entire url
sht2 = gc.open_by_url('https://docs.google.com/spreadsheet/ccc?key=0Bm...FE&hl')
```

### Creating a Spreadsheet

```python
sh = gc.create('A new spreadsheet')

# But that new spreadsheet will be visible only to your script's account.
# To be able to access newly created spreadsheet you *must* share it
# with your email. Which brings us to…
```

### Sharing a Spreadsheet

```python
sh.share('otto@example.com', perm_type='user', role='writer')
```

### Selecting a Worksheet

```python
# Select worksheet by index. Worksheet indexes start from zero
worksheet = sh.get_worksheet(0)

# By title
worksheet = sh.worksheet("January")

# Most common case: Sheet1
worksheet = sh.sheet1

# Get a list of all worksheets
worksheet_list = sh.worksheets()
```

### Creating a Worksheet

```python
worksheet = sh.add_worksheet(title="A worksheet", rows="100", cols="20")
```

### Deleting a Worksheet

```python
sh.del_worksheet(worksheet)
```

### Getting a Cell Value

```python
# With label
val = worksheet.get('B1').first()

# With coords
val = worksheet.cell(1, 2).value
```

### Getting All Values From a Row or a Column

```python
# Get all values from the first row
values_list = worksheet.row_values(1)

# Get all values from the first column
values_list = worksheet.col_values(1)
```

### Getting All Values From a Worksheet as a List of Lists

```python
from gspread.utils import GridRangeType
list_of_lists = worksheet.get(return_type=GridRangeType.ListOfLists)
```

### Getting a range of values

Receive only the cells with a value in them.

```python
>>> worksheet.get("A1:B4")
[['A1', 'B1'], ['A2']]
```

Receive a rectangular array around the cells with values in them.

```python
>>> worksheet.get("A1:B4", pad_values=True)
[['A1', 'B1'], ['A2', '']]
```

Receive an array matching the request size regardless of if values are empty or not.

```python
>>> worksheet.get("A1:B4", maintain_size=True)
[['A1', 'B1'], ['A2', ''], ['', ''], ['', '']]
```

### Finding a Cell

```python
# Find a cell with exact string value
cell = worksheet.find("Dough")

print("Found something at R%sC%s" % (cell.row, cell.col))

# Find a cell matching a regular expression
amount_re = re.compile(r'(Big|Enormous) dough')
cell = worksheet.find(amount_re)
```

### Finding All Matched Cells

```python
# Find all cells with string value
cell_list = worksheet.findall("Rug store")

# Find all cells with regexp
criteria_re = re.compile(r'(Small|Room-tiering) rug')
cell_list = worksheet.findall(criteria_re)
```

### Updating Cells

```python
# Update a single cell
worksheet.update_acell('B1', 'Bingo!')

# Update a range
worksheet.update([[1, 2], [3, 4]], 'A1:B2')

# Update multiple ranges at once
worksheet.batch_update([{
    'range': 'A1:B2',
    'values': [['A1', 'B1'], ['A2', 'B2']],
}, {
    'range': 'J42:K43',
    'values': [[1, 2], [3, 4]],
}])
```

### Get unformatted cell value or formula

```python
from gspread.utils import ValueRenderOption

# Get formatted cell value as displayed in the UI
>>> worksheet.get("A1:B2")
[['$12.00']]

# Get unformatted value from the same cell range
>>> worksheet.get("A1:B2", value_render_option=ValueRenderOption.unformatted)
[[12]]

# Get formula from a cell
>>> worksheet.get("C2:D2", value_render_option=ValueRenderOption.formula)
[['=1/1024']]
```

### Add data validation to a range

```python
import gspread
from gspread.utils import ValidationConditionType

# Restrict the input to greater than 10 in a single cell
worksheet.add_validation(
  'A1',
  ValidationConditionType.number_greater,
  [10],
  strict=True,
  inputMessage='Value must be greater than 10',
)

# Restrict the input to Yes/No for a specific range with dropdown
worksheet.add_validation(
  'C2:C7',
   ValidationConditionType.one_of_list,
   ['Yes',
   'No',]
   showCustomUi=True
)
```

## Documentation

[Documentation]\: [https://gspread.readthedocs.io/][Documentation]

[Documentation]: https://gspread.readthedocs.io/en/latest/

### Ask Questions

The best way to get an answer to a question is to ask on [Stack Overflow with a gspread tag](http://stackoverflow.com/questions/tagged/gspread?sort=votes&pageSize=50).

## Contributors

[List of contributors](https://github.com/burnash/gspread/graphs/contributors)

## How to Contribute

Please make sure to take a moment and read the [Code of Conduct](https://github.com/burnash/gspread/blob/master/.github/CODE_OF_CONDUCT.md).

### Report Issues

Please report bugs and suggest features via the [GitHub Issues](https://github.com/burnash/gspread/issues).

Before opening an issue, search the tracker for possible duplicates. If you find a duplicate, please add a comment saying that you encountered the problem as well.

### Improve Documentation

[Documentation](https://gspread.readthedocs.io/) is as important as code. If you know how to make it more consistent, readable and clear, please submit a pull request. The documentation files are in [`docs`](https://github.com/burnash/gspread/tree/master/docs) folder, use [reStructuredText](http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html#rst-index) markup and rendered by [Sphinx](http://www.sphinx-doc.org/).

### Contribute code

Please make sure to read the [Contributing Guide](https://github.com/burnash/gspread/blob/master/.github/CONTRIBUTING.md) before making a pull request.


-----------------

# Turn even the largest data into images, accurately

|    |    |
| --- | --- |
| Downloads | ![https://pypistats.org/packages/datashader](https://img.shields.io/pypi/dm/datashader?label=pypi) ![https://anaconda.org/pyviz/datashader](https://pyviz.org/_static/cache/datashader_conda_downloads_badge.svg)
| Build Status | [![Build Status](https://github.com/holoviz/datashader/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/holoviz/datashader/actions/workflows/test.yaml?query=branch%3Amain) |
| Coverage | [![codecov](https://codecov.io/gh/holoviz/datashader/branch/main/graph/badge.svg)](https://codecov.io/gh/holoviz/datashader) |
| Latest dev release | [![Github tag](https://img.shields.io/github/tag/holoviz/datashader.svg?label=tag&colorB=11ccbb)](https://github.com/holoviz/datashader/tags) [![dev-site](https://img.shields.io/website-up-down-green-red/https/holoviz-dev.github.io/datashader.svg?label=dev%20website)](https://holoviz-dev.github.io/datashader/) |
| Latest release | [![Github release](https://img.shields.io/github/release/holoviz/datashader.svg?label=tag&colorB=11ccbb)](https://github.com/holoviz/datashader/releases) [![PyPI version](https://img.shields.io/pypi/v/datashader.svg?colorB=cc77dd)](https://pypi.python.org/pypi/datashader) [![datashader version](https://img.shields.io/conda/v/pyviz/datashader.svg?colorB=4488ff&style=flat)](https://anaconda.org/pyviz/datashader) [![conda-forge version](https://img.shields.io/conda/v/conda-forge/datashader.svg?label=conda%7Cconda-forge&colorB=4488ff)](https://anaconda.org/conda-forge/datashader) [![defaults version](https://img.shields.io/conda/v/anaconda/datashader.svg?label=conda%7Cdefaults&style=flat&colorB=4488ff)](https://anaconda.org/anaconda/datashader) |
| Python | [![Python support](https://img.shields.io/pypi/pyversions/datashader.svg)](https://pypi.org/project/datashader/)
| Docs | [![DocBuildStatus](https://github.com/holoviz/datashader/workflows/docs/badge.svg?query=branch%3Amain)](https://github.com/holoviz/datashader/actions?query=workflow%3Adocs+branch%3Amain) [![site](https://img.shields.io/website-up-down-green-red/https/datashader.org.svg)](https://datashader.org) |
| Support | [![Discourse](https://img.shields.io/discourse/status?server=https%3A%2F%2Fdiscourse.holoviz.org)](https://discourse.holoviz.org/) |

-------

[![History of OS GIS Timeline](examples/assets/images/featured-badge-gh.svg)](https://makepath.com/history-of-open-source-gis/)

-------

## What is it?

Datashader is a data rasterization pipeline for automating the process of
creating meaningful representations of large amounts of data. Datashader
breaks the creation of images of data into 3 main steps:

1. Projection

   Each record is projected into zero or more bins of a nominal plotting grid
   shape, based on a specified glyph.

2. Aggregation

   Reductions are computed for each bin, compressing the potentially large
   dataset into a much smaller *aggregate* array.

3. Transformation

   These aggregates are then further processed, eventually creating an image.

Using this very general pipeline, many interesting data visualizations can be
created in a performant and scalable way. Datashader contains tools for easily
creating these pipelines in a composable manner, using only a few lines of code.
Datashader can be used on its own, but it is also designed to work as
a pre-processing stage in a plotting library, allowing that library
to work with much larger datasets than it would otherwise.

## Installation

Datashader supports Python 3.9, 3.10, 3.11, and 3.12 on Linux, Windows, or
Mac and can be installed with conda:

    conda install datashader

or with pip:

    pip install datashader

For the best performance, we recommend using conda so that you are sure
to get numerical libraries optimized for your platform. The latest
releases are avalailable on the pyviz channel `conda install -c pyviz
datashader` and the latest pre-release versions are avalailable on the
dev-labelled channel `conda install -c pyviz/label/dev datashader`.

## Fetching Examples

Once you've installed datashader as above you can fetch the examples:

    datashader examples
    cd datashader-examples

This will create a new directory called
datashader-examples with all the data
needed to run the examples.

To run all the examples you will need some extra dependencies. If you
installed datashader **within a conda environment**, with that
environment active run:

    conda env update --file environment.yml

Otherwise create a new environment:

    conda env create --name datashader --file environment.yml
    conda activate datashader

## Developer Instructions

1.  Install Python 3
    [miniconda](https://docs.conda.io/en/latest/miniconda.html) or
    [anaconda](https://www.anaconda.com/download/success), if you don't
    already have it on your system.

2.  Clone the datashader git repository if you do not already have it:

        git clone git://github.com/holoviz/datashader.git

3.  Set up a new conda environment with all of the dependencies needed
    to run the examples:

        cd datashader
        conda env create --name datashader --file ./examples/environment.yml
        conda activate datashader

4.  Put the datashader directory into the Python path in this
    environment:

        pip install --no-deps -e .

## Learning more

After working through the examples, you can find additional resources linked
from the [datashader documentation](https://datashader.org),
including API documentation and papers and talks about the approach.

## Some Examples

![USA census](examples/assets/images/usa_census.jpg)

![NYC races](examples/assets/images/nyc_races.jpg)

![NYC taxi](examples/assets/images/nyc_pickups_vs_dropoffs.jpg)


#+OPTIONS: num:nil

Quick links: [[#motivation][Motivation]] | [[file:doc/GUIDE.org#extension-features][Feature guide]] | [[#demos][*Demos*]] | [[#how-does-it-work][How does it work?]] | [[#install][*Install*]] | [[#try-it-out][Try it out & Setup]] | [[#glossary][Glossary]] | [[file:doc/GUIDE.org#FAQ][FAQ]] | [[#support][Support]] | [[file:doc/TROUBLESHOOTING.org][Troubleshooting guide]] | [[file:doc/DEVELOPMENT.org][Developer's guide]]

Promnesia is a browser extension for [[https://chrome.google.com/webstore/detail/promnesia/kdmegllpofldcpaclldkopnnjjljoiio][Chrome]]/[[https://addons.mozilla.org/en-US/firefox/addon/promnesia][Firefox]]
(including Firefox for Android!) which serves as a web surfing copilot, enhancing your browsing history and web exploration experience.

*TLDR*: it lets you explore your browsing history *in context*: where you encountered it, in chat, on Twitter, on Reddit, or just in one of the text files on your computer.
This is unlike most modern browsers, where you can only see when you visited the link.

It allows you to answer different questions about the current web page:

#+html:
- have I been here before? When? [[https://karlicoss.github.io/promnesia-demos/child-visits.webm][demo (30 s)]] [[https://karlicoss.github.io/promnesia-demos/child-visits-2.webm][demo (40s)]]
- how did I get on it? Which page has led to it? [[https://karlicoss.github.io/promnesia-demos/how_did_i_get_here.webm][demo (40s)]]
- why have I bookmarked it? who sent me this link? Can I just jump to the message? [[https://karlicoss.github.io/promnesia-demos/watch_later.webm][demo (30s)]]
- or, can I jump straight into the file where the link occurred? [[https://karlicoss.github.io/promnesia-demos/jump_to_editor.webm][demo (20s)]]
- which links on this page have I already explored? [[https://karlicoss.github.io/promnesia-demos/mark-visited.webm][demo (30s)]],
- which posts from this blog page have I already read? [[https://karlicoss.github.io/promnesia-demos/mark-visited-2.webm][demo (20s)]]
- have I annotated it? [[https://karlicoss.github.io/promnesia-demos/highlights.webm][demo (1m)]]
- how much time have I spent on it? [[https://user-images.githubusercontent.com/291333/82124084-ba040100-9794-11ea-9af9-ee250ebbb473.png][screenshot]]
- and it also works on your phone! [[https://karlicoss.github.io/promnesia-demos/mobile/panel-jbor.png][screenshot]]

See [[file:doc/GUIDE.org#extension-features][feature guide]] for more information.

You can jump straight to the [[#demos][Demos]] and [[#install][Install]] sections if you're already overexcited.

* Motivation
See a [[https://beepb00p.xyz/promnesia.html][writeup]] on history and the problems Promnesia is solving. Here's a short summary.

- Has it ever occurred to you that you were reading an old bookmark or some lengthy blog post and suddenly realized you had read it already before?
  It would be fairly easy to search in the browser history, however, it is only stored locally for three months.
  # TODO link?
- Or perhaps you even have a habit of annotating and making notes elsewhere? And you wanna know quickly if you have the current page annotated and display the annotations.
  However, most services want you to use their web apps even for viewing highlights. You don't know if you have highlights, unless you waste time checking every page.
- Or you have this feeling that someone sent you this link ages ago, but you don't remember who and where.
- Or you finally got to watch that thing in your 'Watch later' youtube playlist, that's been there for three years, and now you want to know why did you add it in the first place.

Then this tool is for you.

#+html:

* Demos
  :PROPERTIES:
  :CUSTOM_ID: demos
  :END:

See [[file:doc/GUIDE.org#extension-features][feature guide]] if you're wondering how something specific here works.

- [[https://karlicoss.github.io/promnesia-demos/screens/promnesia andy Screenshot at 2020-06-05 23-33-07.png]]

  - a green eye means that the link was visited before and has some [[#glossary][contexts]], associated with it. When you click on it, the sidebar opens with more information about the visits.
  - You can see that I've sent the link to Jane on Telegram (#1)
  - I've annotated the link on Instapaper and highlights (#3) is marked with yellow right within the page

    Normally in order to see see your Instapaper highligths you have to go to the web app first!
  - I've clipped some content to my personal notes at some point (#8), the selected text was matched and highlighted as well

    If I click =notes/ip2org.org=, it will cause my Emacs to open and jump to the file, containing this note.
    # and jump straight to the clipping within the file. -- TODO
- [[https://karlicoss.github.io/promnesia-demos/screens/promnesia universal love Screenshot at 2020-06-05 23-18-38.png]]

  - I have a note about the page in my personal notes on my filesystem (#2)
  - I chatted with a friend and sent them the link at some point (#6)

    Clicking on the link will open Telegram and jump straight to the message where the link was mentioned.
    So you can reply to it without having to search or scroll over the whole chat history.
    # Json is clearly not the most convenient way to go through conversations with friends, but that's a matter of representing chats in a plaintext form. The benefit though is that once you have any sort of grepable source it's super easy to feed it into the plugin.
  - I've tweeted about this link before (#11)

    Similarly, clicking would jump straight to my tweet.

  - I also have this link annotated via [[https://hypothes.is][Hypothesis]] (#4, #12, #13)

- Annotated demo of sidebar with direct & child visits, and 'visited' marks  (click to zoom)
#+html:

- Annotated demo of highlight and locator features (click to zoom)
#+html:

- More:
  - A [[https://www.youtube.com/watch?v=9PsOeYheIY4][video]], demonstrating
    - [[file:doc/GUIDE.org#mark-visited]['mark visited']] feature
    - highlights
    - jumping to the originating tweet; jumping straight to the org-mode heading where links occurred

  - You can find more short screencasts, demonstrating various features [[https://github.com/karlicoss/promnesia-demos][here]]
  - There are more screenshots [[https://github.com/karlicoss/promnesia/issues/5#issuecomment-619365708][here]]
  - This [[https://beepb00p.xyz/myinfra-roam.html#promnesia][post]] features a video demo of using Promnesia with a Roam Research database
  - Video demos for the mobile extension:
    - [[https://karlicoss.github.io/promnesia-demos/mobile/panel-w3c.webm][showing all references to pages on W3C website in my org-mode notes]]
    - [[https://karlicoss.github.io/promnesia-demos/mobile/mark-visited-reddit.webm][marking already visited links on the page]], to make easier to process Reddit posts

* How does it work?
:PROPERTIES:
:CUSTOM_ID: how-does-it-work
:END:
Promnesia consists of three parts:

- browser extension

  - neatly displays the history and other information in a sidebar
  - handles highlights
  - provides search interface

  However, browser addons can't read access your filesystem, so to load the data we need a helper component:

- server/backend: =promnesia serve= command

  It's called 'server', but really it's just a regular program with the only purpose to serve the data to the browser.
  It runs locally and you don't have to expose it to the outside.

- indexer: =promnesia index= command

  Indexer goes through the sources (specified in the config), processes raw data and extracts URLs along with other useful information.

  Another important thing it's doing is *normalising* URLs to establish equivalence and strip off garbage.
  I write about the motivation for it in [[https://beepb00p.xyz/promnesia.html#urls_broken]["URLs are broken"]].

You might also want to skim through the [[https://github.com/karlicoss/promnesia#glossary][glossary]] if you want to understand deeper what information Promnesia is extracting.

** Data sources
Promnesia ships with some builtin sources. It supports:

- data exports from online services: Reddit/Twitter/Hackernews/Telegram/Messenger/Hypothesis/Pocket/Instapaper, etc.

  It heavily benefits from [[https://github.com/karlicoss/HPI][HPI]] package to access the data.

- Google Takeout/Activity backups
- Markdown/org-mode/HTML or any other plaintext on your disk
  - for ex. local personal knowledge bases - Logseq ([[file:doc/config.py#L67][example config]]), Obsidian
- in general, anything that can be parsed in some way
- you can also add [[https://github.com/karlicoss/promnesia/blob/master/doc/SOURCES.org#extending][your own custom sources]], Promnesia is extensible

See [[https://github.com/karlicoss/promnesia/blob/master/doc/SOURCES.org][SOURCES]] for more information.

** Data flow

Here's a diagram, which would hopefully help to understand how data flows through Promnesia.

See HPI [[https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#data-flow][section on data flow]] for more information on HPI modules and data flow.

Also check out my [[https://beepb00p.xyz/myinfra.html#promnesia][infrastructure map]], which is more detailed!

: ┌─────────────────────────────────┐ ┌────────────────────────────┐ ┌─────────────────┐
: │ 💾       HPI sources            │ │  💾    plaintext files      │ │  other sources  │
: │ (twitter, reddit, pocket, etc.) │ │ (org-mode, markdown, etc.) │ │ (user-defined)  │
: └─────────────────────────────────┘ └────────────────────────────┘ └─────────────────┘
:                                 ⇘⇘              ⇓⇓               ⇙⇙
:                                  ⇘⇘             ⇓⇓              ⇙⇙
:                                  ┌──────────────────────────────┐
:                                  │ 🔄    promnesia indexer      │
:                                  |        (runs regularly)      │
:                                  └──────────────────────────────┘
:                                                 ⇓⇓
:                                  ┌──────────────────────────────┐
:                                  │ 💾    visits database        │
:                                  │       (promnesia.sqlite)     │
:                                  └──────────────────────────────┘
:                                                 ⇓⇓
:                                  ┌──────────────────────────────┐
:                                  │ 🔗    promnesia server       │
:                                  |       (always running)       |
:                                  └──────────────────────────────┘
:                                                 ⇣⇣
:                                  ┌─────────────────────────────────┐
:          ┌───────────────────────┤  🌐      web browser            ├────────────────────┐
:          │  💾 browser bookmarks ⇒      (promnesia extension)      ⇐  💾 browser history |
:          └───────────────────────┴─────────────────────────────────┴────────────────────┘

# https://en.wikipedia.org/wiki/List_of_Unicode_characters#Box_Drawing
# TODO would be really nice to have links here.. but not sure how without svg...

* Install
  :PROPERTIES:
  :CUSTOM_ID: install
  :END:

- extension:

  - [[https://chrome.google.com/webstore/detail/promnesia/kdmegllpofldcpaclldkopnnjjljoiio][Chrome]]: desktop version. Unfortunately mobile Chrome doesn't support web extensions.
  - [[https://addons.mozilla.org/en-US/firefox/addon/promnesia][Firefox]]: desktop and Android
    - note: web extensions on Android are mostly broken at the moment, see [[https://discourse.mozilla.org/t/add-on-support-in-new-firefox-for-android/53488][here]]
      (unless you're using [[https://blog.mozilla.org/addons/2020/09/29/expanded-extension-support-in-firefox-for-android-nightly][Firefox Nightly]])
    - unfortunately, iOS Firefox [[https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_for_iOS#Addons][doesn't support web extensions]].
  - you can also find 'unpacked' versions in [[https://github.com/karlicoss/promnesia/releases][Releases]]

    It can be useful because Chrome Web Store releases might take days to approve, but in general the store version if preferrable.

- backend

  Note that Promnesia can work without the backend, so technically this step is optional.
  But then it will only be able to use browser visits and browser bookmarks, so the benefits of the extension will be limited.

  - simplest: install from [[https://pypi.org/project/promnesia][PyPi]]: =pip3 install --user promnesia=
    - install optional dependencies with: =pip3 install --user bs4 lxml mistletoe logzero=
  - alternatively: you can clone this repository and run it as ~scripts/promnesia~

    This is mainly useful for tinkering with the code and writing new modules.

  You might also need some extra dependencies. See [[file:doc/SOURCES.org#extra-dependencies]["Extra dependencies"]] for more info.

  As for supported operating systems:

  - Linux: everything is expected to work as it's what I'm using!
  - OSX: expected to work, but there might be issues at times (I don't have any macs so working blind here). Appreciate help!

    You might want to run =brew install libmagic= for proper MIME type detection.
  - Windows: at the moment doesn't work straightaway (I don't have any Windows to test against), there is an [[https://github.com/karlicoss/promnesia/issues/91][open issue]] describing some workarounds.
  - Android: [[https://github.com/karlicoss/promnesia/issues/114#issuecomment-642757602][allegedly]], possible to run with Termux! But I haven't got to try personally.

* Try it out
You can try out Promnesia with the extension only, it uses browser history and bookmarks as data sources.
However, Promnesia's main power is using data from external sources, and for that you'll need to run the indexer and backend (more on it in the next section).

The easiest way to try this mode is to run =promnesia demo= command, it can give you a sense of what Promnesia is doing with zero configuration.

1. [[#install][Install]] the extension and the server (PIP package), in case you haven't already
2. Run ~promnesia demo https://github.com/karlicoss/exobrain~

   This clones the repository, ([[https://github.com/karlicoss/exobrain][my personal wiki]] in this case), extracts the URLs, and runs on the port =13131= (default, can be specified via =--port=)

   You can also use a path on your local filesystem (e.g. directory with text files), or a website URL.

3. After that, visit https://www.reddit.com

   If you press the extension icon, you will see the pages from my blog where I link to posts on Reddit.

* Setup
# TODO mention where they get the database
To get the most benefit from Promnesia, it's best to properly setup your own config, describing the sources you want it to use.
*If something is unclear, please feel free to open issues or reach me*, I'm working on improving the documentation.
Also check out [[file:doc/TROUBLESHOOTING.org][troubleshooting guide]] or [[https://github.com/karlicoss/promnesia/labels/documentation][open discussions on documentation]].

- create the config: =promnesia config create=

  The command will put a stub promnesia config in your user config directory, e.g. =~/.config/promnesia/config.py= on Linux. (it's possibly different on OSX and Windows, see [[https://github.com/ActiveState/appdirs/blob/3fe6a83776843a46f20c2e5587afcffe05e03b39/appdirs.py#L187-L190][this]] if you're not sure). If you wish to specify a custom location, you can set the ~PROMNESIA_CONFIG~ environment variable or pass the ~--config~ flag.

- edit the config and add some sources

  You can look at an [[file:src/promnesia/misc/config_example.py][example config]], or borrow bits from an annotated configuration example here: [[file:doc/config.py]].

  The only required setting is:

  - =SOURCES=

    SOURCES specifies the list of data sources, that will be processed and indexed by Promnesia.

    You can find the list of available sources with more documentation on each of them here: [[file:doc/SOURCES.org][SOURCES]].

    - reading example config: [[file:doc/config.py]]
    - browsing the code: [[file:src/promnesia/sources/][promnesia/sources]].

  If you want to learn about other settings, the best way at the moment (apart from reading [[file:src/promnesia/config.py][the source]])
  is, once again, [[file:doc/config.py][example config]].
  # TODO document other settings..

  - [optional] check the config

    First, you can run =promnesia doctor config=, it can be used to quickly troubleshoot typos and similar errors. Note that you may need to install [mypy](https://github.com/python/mypy) for some config checks.

    Next, you can use the demo mode: =promnesia demo --config /path/to/config.py=.

    This will index the data and launch the server immediately, so you can check that everything works as expected in your browser.

- run the indexer: =promnesia index=

  [[https://github.com/karlicoss/promnesia/issues/20][At the moment]], indexing is *periodic, not realtime*. The best is to run it via *cron/systemd* once or several times a day:

  : # run every hour in cron
  : 0 * * * *      promnesia index    >/tmp/promnesia-index.log 2>/tmp/promnesisa-index.err

  Note: you can also pass =--config /path/to/config.py=  explicitly if you prefer or want to experiment.

  - [optional] check the database

    Run =promnesia doctor database= to quickly inspect the database and check that stuff that you wanted got indexed. You might need to install =sqlitebrowser= first.

- run the server: =promnesia serve=

  You only have to start it once, it will automatically detect further changes done by =promnesia index=.

  - [optional] autostart the server with =promnesia install-server=

    This sets it up to autostart and run in the background:

    - via Systemd for Linux
    - via Launchd for OSX. I don't have a Mac nearby, so if you have any issues with it, please report them!

    I /think/ you can also use cron with =@reboot= attribute:

    : # sleep is just in case cron starts up too early. Prefer systemd script if possible!
    : @reboot     sleep 60 && promnesia serve   >/tmp/promnesia-serve.log 2>/tmp/promnesia-serve.err

    Alternatively, you can just create a manual autostart entry in your desktop environment.

  - [optional] check that the server is responding =promnesia doctor server=

- [optional] setup MIME handler to jump to files straight from the extension

  See a short [[https://karlicoss.github.io/promnesia-demos/jump_to_editor.webm][20s demo]], and if this is something you'd like,
  follow the instructions in [[https://github.com/karlicoss/open-in-editor#readme][open-in-editor]].

# TODO Frontend -- mention what settings are possible?
# TODO possibly reuse JS config stub?
* Glossary
*Visit* represents an 'occurence' of a link in your digital trace.
Obviously, visiting pages in your browser results in visits, but in Promnesia this notion also captures links that you interacted with
in other applications and services.

In code ([[file:src/promnesia/common.py][python]], [[file:extension/src/common.js][JS]]), visits are reprented as =class Visit= (and =class DbVisit=).

Visits have the following fields:

- *url*: hopefully, no explanation needed!

  The only required field.
  # TODO although already thinking about making it optional too... e.g. context but no url.
  # or jus use fake url?

- *timestamp*: when the page was visited

  Required, but in the future might be optional (sometimes you don't have a meaningful timestamp).

- *locator*: what's the origin of the visit?

  Usually it's a permalink back to the original source of the visit.

  For example:

  - locators for a link extracted from Reddit data point straight into =reddit.com= interface, for the corresponding post or comment
  - locators for a link extracted a local file point straight into these files on your disk. Clicking on the locator will open your text editor via MIME integration

  Required, but in the future might be optional. (TODO also rename to 'origin'??)
  # TODO renaming gonna be annoying because of the communication protocol..

- *context*: what was the context, in which the visit occurred?

  For example:

  - context for Telegram visits is the message body along with its sender
  - context for a link from org-mode file is the whole paragraph (outline), in which it occurred

  I usually call a visit without a context 'boring' -- it doesn't contain much information except for the mere fact of visiting the page before.
  However they are still useful to have, since they fill in the gaps and provide means of *tracing* through your history.

  Optional.

- *duration*: how much we have spent on the page

  Somewhat experimental field, at the moment it's only set for Chrome (and often not very precise).

  Optional.

*Digression*: now that you have an idea what is a Visit, you can understand few more things about Promnesia:

- source (or indexer) is any function that extract visits from raw files and generates a stream of visits (i.e. =Iterable[Visit]=).
- promnesia indexer goes through the sources, specified in config, collects the visits and puts in the database
- promnesia server reads visits form the database, and sends them to the extension

Now let's consider some *concrete* examples of different kinds of Visits:

- [[file:src/promnesia/sources/takeout.py][Google Takeout]] indexer

  Results in visits with:

  - *url*
  - *timestamp*
  - *locator*

  There isn't any context for visits from takeout, because it's basically a fancy database export.

- [[file:src/promnesia/sources/instapaper.py][Instapaper]] indexer

  Generates a visit for each highlight on the page:

  - *url*: original URL of the annotated page
  - *timestamp*: time when you created the highlight
  - *locator*: permalink to the highlight, bringing you into the Instapaper web app
  - *context*: highlight body

- [[file:src/promnesia/sources/markdown.py][Markdown]] indexer

  Extracts any links it finds in Markdown files:

  - *url*: extracted link
  - *timestamp*: Markdown doesn't have a well defined datetime format, so it's just set to the file modification time.

    However, if you do have your own format, it's possible to write your own indexer to properly take them into the account.

  - *locator*: links straight into the markdown file on your disk!
  - *context*: the markdown paragraph, containing the url

*Note*: this terminology is not set is stone, so if someone feels there are words that describe these concepts better, I'm open to suggestions!

# TODO glossary for canonical??

* FAQ
See [[file:doc/GUIDE.org#FAQ]]

* Support
The best support for me would be if you contribute to this or my other projects. Code, ideas of feedback -- everything is appreciated.

I don't need money, but I understand it's often easier to give away than time, so here are some of projects that I donate to:

- [[https://liberapay.com/org-mode][org-mode]]
- [[https://archive.org/donate][Internet Archive]]
- [[https://web.hypothes.is/donate][Hypothes.is]]
- [[https://github.com/hlissner/doom-emacs#contribute][Doom Emacs]]

* More links
- [[file:doc/TROUBLESHOOTING.org][Troubleshooting guide]]
- [[file:doc/SOURCES.org][Documentation on the sources]]
- [[file:doc/DEVELOPMENT.org][Developer's guide]]
- [[file:doc/PRIVACY.org][Privacy policy]]


I/O for mesh files.

[![PyPi Version](https://img.shields.io/pypi/v/meshio.svg?style=flat-square)](https://pypi.org/project/meshio/)
[![Anaconda Cloud](https://anaconda.org/conda-forge/meshio/badges/version.svg?=style=flat-square)](https://anaconda.org/conda-forge/meshio/)
[![Packaging status](https://repology.org/badge/tiny-repos/python:meshio.svg)](https://repology.org/project/python:meshio/versions)
[![PyPI pyversions](https://img.shields.io/pypi/pyversions/meshio.svg?style=flat-square)](https://pypi.org/project/meshio/)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1173115.svg?style=flat-square)](https://doi.org/10.5281/zenodo.1173115)
[![GitHub stars](https://img.shields.io/github/stars/nschloe/meshio.svg?style=flat-square&logo=github&label=Stars&logoColor=white)](https://github.com/nschloe/meshio)
[![Downloads](https://pepy.tech/badge/meshio/month?style=flat-square)](https://pepy.tech/project/meshio)

[![Discord](https://img.shields.io/static/v1?logo=discord&logoColor=white&label=chat&message=on%20discord&color=7289da&style=flat-square)](https://discord.gg/Z6DMsJh4Hr)

[![gh-actions](https://img.shields.io/github/workflow/status/nschloe/meshio/ci?style=flat-square)](https://github.com/nschloe/meshio/actions?query=workflow%3Aci)
[![codecov](https://img.shields.io/codecov/c/github/nschloe/meshio.svg?style=flat-square)](https://app.codecov.io/gh/nschloe/meshio)
[![LGTM](https://img.shields.io/lgtm/grade/python/github/nschloe/meshio.svg?style=flat-square)](https://lgtm.com/projects/g/nschloe/meshio)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square)](https://github.com/psf/black)

There are various mesh formats available for representing unstructured meshes.
meshio can read and write all of the following and smoothly converts between them:

> [Abaqus](http://abaqus.software.polimi.it/v6.14/index.html) (`.inp`),
> ANSYS msh (`.msh`),
> [AVS-UCD](https://lanl.github.io/LaGriT/pages/docs/read_avs.html) (`.avs`),
> [CGNS](https://cgns.github.io/) (`.cgns`),
> [DOLFIN XML](https://manpages.ubuntu.com/manpages/jammy/en/man1/dolfin-convert.1.html) (`.xml`),
> [Exodus](https://nschloe.github.io/meshio/exodus.pdf) (`.e`, `.exo`),
> [FLAC3D](https://www.itascacg.com/software/flac3d) (`.f3grid`),
> [H5M](https://www.mcs.anl.gov/~fathom/moab-docs/h5mmain.html) (`.h5m`),
> [Kratos/MDPA](https://github.com/KratosMultiphysics/Kratos/wiki/Input-data) (`.mdpa`),
> [Medit](https://people.sc.fsu.edu/~jburkardt/data/medit/medit.html) (`.mesh`, `.meshb`),
> [MED/Salome](https://docs.salome-platform.org/latest/dev/MEDCoupling/developer/med-file.html) (`.med`),
> [Nastran](https://help.autodesk.com/view/NSTRN/2019/ENU/?guid=GUID-42B54ACB-FBE3-47CA-B8FE-475E7AD91A00) (bulk data, `.bdf`, `.fem`, `.nas`),
> [Netgen](https://github.com/ngsolve/netgen) (`.vol`, `.vol.gz`),
> [Neuroglancer precomputed format](https://github.com/google/neuroglancer/tree/master/src/neuroglancer/datasource/precomputed#mesh-representation-of-segmented-object-surfaces),
> [Gmsh](https://gmsh.info/doc/texinfo/gmsh.html#File-formats) (format versions 2.2, 4.0, and 4.1, `.msh`),
> [OBJ](https://en.wikipedia.org/wiki/Wavefront_.obj_file) (`.obj`),
> [OFF](https://segeval.cs.princeton.edu/public/off_format.html) (`.off`),
> [PERMAS](https://www.intes.de) (`.post`, `.post.gz`, `.dato`, `.dato.gz`),
> [PLY](<https://en.wikipedia.org/wiki/PLY_(file_format)>) (`.ply`),
> [STL](<https://en.wikipedia.org/wiki/STL_(file_format)>) (`.stl`),
> [Tecplot .dat](http://paulbourke.net/dataformats/tp/),
> [TetGen .node/.ele](https://wias-berlin.de/software/tetgen/fformats.html),
> [SVG](https://www.w3.org/TR/SVG/) (2D output only) (`.svg`),
> [SU2](https://su2code.github.io/docs_v7/Mesh-File/) (`.su2`),
> [UGRID](https://www.simcenter.msstate.edu/software/documentation/ug_io/3d_grid_file_type_ugrid.html) (`.ugrid`),
> [VTK](https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf) (`.vtk`),
> [VTU](https://vtk.org/Wiki/VTK_XML_Formats) (`.vtu`),
> [WKT](https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry) ([TIN](https://en.wikipedia.org/wiki/Triangulated_irregular_network)) (`.wkt`),
> [XDMF](https://xdmf.org/index.php/XDMF_Model_and_Format) (`.xdmf`, `.xmf`).

([Here's a little survey](https://forms.gle/PSeNb3N3gv3wbEus8) on which formats are actually
used.)

Install with one of

```
pip install meshio[all]
conda install -c conda-forge meshio
```

(`[all]` pulls in all optional dependencies. By default, meshio only uses numpy.)
You can then use the command-line tool

```sh
meshio convert    input.msh output.vtk   # convert between two formats

meshio info       input.xdmf             # show some info about the mesh

meshio compress   input.vtu              # compress the mesh file
meshio decompress input.vtu              # decompress the mesh file

meshio binary     input.msh              # convert to binary format
meshio ascii      input.msh              # convert to ASCII format
```

with any of the supported formats.

In Python, simply do

```python
import meshio

mesh = meshio.read(
    filename,  # string, os.PathLike, or a buffer/open file
    # file_format="stl",  # optional if filename is a path; inferred from extension
    # see meshio-convert -h for all possible formats
)
# mesh.points, mesh.cells, mesh.cells_dict, ...

# mesh.vtk.read() is also possible
```

to read a mesh. To write, do

```python
import meshio

# two triangles and one quad
points = [
    [0.0, 0.0],
    [1.0, 0.0],
    [0.0, 1.0],
    [1.0, 1.0],
    [2.0, 0.0],
    [2.0, 1.0],
]
cells = [
    ("triangle", [[0, 1, 2], [1, 3, 2]]),
    ("quad", [[1, 4, 5, 3]]),
]

mesh = meshio.Mesh(
    points,
    cells,
    # Optionally provide extra data on points, cells, etc.
    point_data={"T": [0.3, -1.2, 0.5, 0.7, 0.0, -3.0]},
    # Each item in cell data must match the cells array
    cell_data={"a": [[0.1, 0.2], [0.4]]},
)
mesh.write(
    "foo.vtk",  # str, os.PathLike, or buffer/open file
    # file_format="vtk",  # optional if first argument is a path; inferred from extension
)

# Alternative with the same options
meshio.write_points_cells("foo.vtk", points, cells)
```

For both input and output, you can optionally specify the exact `file_format`
(in case you would like to enforce ASCII over binary VTK, for example).

#### Time series

The [XDMF format](https://xdmf.org/index.php/XDMF_Model_and_Format) supports
time series with a shared mesh. You can write times series data using meshio
with

```python
with meshio.xdmf.TimeSeriesWriter(filename) as writer:
    writer.write_points_cells(points, cells)
    for t in [0.0, 0.1, 0.21]:
        writer.write_data(t, point_data={"phi": data})
```

and read it with

```python
with meshio.xdmf.TimeSeriesReader(filename) as reader:
    points, cells = reader.read_points_cells()
    for k in range(reader.num_steps):
        t, point_data, cell_data = reader.read_data(k)
```

### ParaView plugin

*A Gmsh file opened with ParaView.*

If you have downloaded a binary version of ParaView, you may proceed as follows.

- Install meshio for the Python major version that ParaView uses (check `pvpython --version`)
- Open ParaView
- Find the file `paraview-meshio-plugin.py` of your meshio installation (on Linux:
  `~/.local/share/paraview-5.9/plugins/`) and load it under _Tools / Manage Plugins / Load New_
- _Optional:_ Activate _Auto Load_

You can now open all meshio-supported files in ParaView.

### Performance comparison

The comparisons here are for a triangular mesh with about 900k points and 1.8M
triangles. The red lines mark the size of the mesh in memory.

#### File sizes

#### I/O speed

#### Maximum memory usage

### Installation

meshio is [available from the Python Package Index](https://pypi.org/project/meshio/),
so simply run

```
pip install meshio
```

to install.

Additional dependencies (`netcdf4`, `h5py`) are required for some of the output formats
and can be pulled in by

```
pip install meshio[all]
```

You can also install meshio from [Anaconda](https://anaconda.org/conda-forge/meshio):

```
conda install -c conda-forge meshio
```

### Testing

To run the meshio unit tests, check out this repository and type

```
tox
```

### License

meshio is published under the [MIT license](https://en.wikipedia.org/wiki/MIT_License).


# Dash Bio
[![CircleCI](https://circleci.com/gh/plotly/dash-bio/tree/master.svg?style=svg)](https://circleci.com/gh/plotly/dash-bio)
[![PyPI version](https://badge.fury.io/py/dash-bio.svg)](https://badge.fury.io/py/dash-bio)

Dash Bio is a suite of bioinformatics components built to work with
[Dash](https://github.com/plotly/dash/).

Announcement: https://medium.com/@plotlygraphs/announcing-dash-bio-ed8835d5da0c

Demo:
[https://dash-gallery.plotly.host/Portal/?search=Bioinformatics](https://dash-gallery.plotly.host/Portal/?search=Bioinformatics)

Documentation:
[https://dash.plotly.com/dash-bio](https://dash.plotly.com/dash-bio)

## Components

The Dash Bio components each fall into one of three categories:

- Custom chart types
- Sequence analysis tools
- 3D rendering tools

### Custom chart types

- Dash Circos
- Dash Clustergram
- Dash Manhattan Plot
- Dash Needle Plot
- Dash Volcano Plot

### Sequence analysis tools

- Dash Alignment Chart
- Dash Onco Print
- Dash Forna Container
- Dash Sequence Viewer

### Visualization tools

- Dash Mol2D
- Dash Mol3D
- Dash Speck
- Dash Ngl

## Using Dash Bio

It's easy to add a fully interactive chromosomal, molecular or genomic visualization to your Dash app by simply
including the Dash Bio component into your app layout as follows:

```python
import urllib.request as urlreq
from dash import Dash, html
import dash_bio as dashbio

app = Dash(__name__)

data = urlreq.urlopen(
    'https://raw.githubusercontent.com/plotly/dash-bio-docs-files/master/alignment_viewer_p53.fasta'
).read().decode('utf-8')

app.layout = html.Div([
    dashbio.AlignmentChart(
        id='my-default-alignment-viewer',
        data=data
    )
])

if __name__ == '__main__':
    app.run_server(debug=True)
```

See the [Dash Bio documentation](https://dash.plotly.com/dash-bio) for more components and examples.

## Run Dash Bio in a JupyterLab environment

1. Create a virtual environment:

    The following steps require a virtual environment tool to be installed on your computer: `pip install virtualenv`

    a. On macOS and Linux: `python3 -m venv env`

    b. On Windows, enter: `py -m venv env`

2. Activate your new environment:

    a. On macOS and Linux, enter: `source env/bin/activate`

    b. On Windows, enter: `.\env\Scripts\activate`

3. Install required libraries (make sure you have pip installed with `pip help`):
```
pip install dash dash-bio pandas numpy Jupyterlab
```

4. To run Dash inside Jupyter lab:

    a. Install jupyter-dash:  `pip install jupyter-dash`

    b. Enter `jupyter lab build`

    (Note: This step requires Node.js and NPM installed on yourcomputer. To check if Node and NPM are installed, enter `node -v` and `npm -v` in your terminal. For install instructions see [nodejs.org](https://nodejs.org/en/).

5. To display Plotly figures in JupyterLab:
```
pip install jupyterlab "ipywidgets>=7.5”
jupyter labextension install jupyterlab-plotly@4.14.3
```

6. Start JupyterLab by typing: `jupyter lab`

    Important: JupyterLab must be run within the virtual environment that was previously activated.

For more on running a Dash app in Jupyter Lab visit [Getting Started with Jupyter Dash](https://github.com/plotly/jupyter-dash/blob/master/notebooks/getting_started.ipynb).

## Dash

Learn more about Dash at
[https://plotly.com/products/dash/](https://plotly.com/products/dash/).

## Consulting and OEM

For inquiries about Dash app development, advanced OEM integration,
and more, please [reach
out](https://plotly.typeform.com/to/mH1Cpb).

## Contributing and Local Development

If you would like to contribute to this repository, or run demo apps and tests, please refer to
the [contributing
guidelines](https://github.com/plotly/dash-bio/blob/master/CONTRIBUTING.md).


# OAuth + JupyterHub Authenticator = OAuthenticator :heart:

[![Documentation build status](https://img.shields.io/readthedocs/oauthenticator?logo=read-the-docs)](https://oauthenticator.readthedocs.org/en/latest)
[![GitHub Workflow Status - Test](https://img.shields.io/github/actions/workflow/status/jupyterhub/oauthenticator/test.yml?logo=github&label=tests)](https://github.com/jupyterhub/oauthenticator/actions)
[![Latest PyPI version](https://img.shields.io/pypi/v/oauthenticator?logo=pypi)](https://pypi.python.org/pypi/oauthenticator)
[![Latest conda-forge version](https://img.shields.io/conda/vn/conda-forge/oauthenticator?logo=conda-forge)](https://anaconda.org/conda-forge/oauthenticator)
[![GitHub](https://img.shields.io/badge/issue_tracking-github-blue?logo=github)](https://github.com/jupyterhub/oauthenticator/issues)
[![Discourse](https://img.shields.io/badge/help_forum-discourse-blue?logo=discourse)](https://discourse.jupyter.org/c/jupyterhub)
[![Gitter](https://img.shields.io/badge/social_chat-gitter-blue?logo=gitter)](https://gitter.im/jupyterhub/jupyterhub)

[OAuth](https://en.wikipedia.org/wiki/OAuth) is a token based login mechanism that doesn't rely on a username and password mapping.
In order to use this login mechanism with JupyerHub the login handlers need to be overridden.
OAuthenticator overrides these handlers for the common OAuth2 identity providers allowing them to be
plugged in and used with JupyterHub.

The following authentication services are supported through their own authenticator: [Auth0](oauthenticator/auth0.py),
[Azure AD](oauthenticator/azuread.py), [Bitbucket](oauthenticator/bitbucket.py), [CILogon](oauthenticator/cilogon.py), [FeiShu](https://github.com/tezignlab/jupyterhub_feishu_authenticator),
[GitHub](oauthenticator/github.py), [GitLab](oauthenticator/gitlab.py), [Globus](oauthenticator/globus.py),
[Google](oauthenticator/google.py), [MediaWiki](oauthenticator/mediawiki.py),
[OpenShift](oauthenticator/openshift.py).

There is also a [GenericAuthenticator](oauthenticator/generic.py)
that can be configured with any OAuth 2.0 identity provider or can be used
to create a new authenticator class when additional customization is needed.

## Installation

The installation guide can be found in the [docs](https://oauthenticator.readthedocs.io/en/latest/tutorials/install.html).

The [docs](https://oauthenticator.readthedocs.io/en/latest/tutorials/provider-specific-setup/index.html) also provide example setups for different OAuth2 identity providers.

## Running tests

To run the tests locally, first setup a development environment as described in
[CONTRIBUTING.md], and then do:

```
pytest -v ./oauthenticator/tests/
```

Or you run a specific test file with:

```
pytest -v ./oauthenticator/tests/<test-file-name>
```

[contributing.md]: https://github.com/jupyterhub/oauthenticator/blob/main/CONTRIBUTING.md


Stellar Python SDK
==================

.. image:: https://img.shields.io/github/actions/workflow/status/StellarCN/py-stellar-base/continuous-integration-workflow.yml?branch=main
    :alt: GitHub Workflow Status
    :target: https://github.com/StellarCN/py-stellar-base/actions

.. image:: https://img.shields.io/readthedocs/stellar-sdk.svg
    :alt: Read the Docs
    :target: https://stellar-sdk.readthedocs.io/en/latest/

.. image:: https://static.pepy.tech/personalized-badge/stellar-sdk?period=total&units=abbreviation&left_color=grey&right_color=brightgreen&left_text=Downloads
    :alt: PyPI - Downloads
    :target: https://pypi.python.org/pypi/stellar-sdk

.. image:: https://img.shields.io/codeclimate/maintainability/StellarCN/py-stellar-base
    :alt: Code Climate maintainability
    :target: https://codeclimate.com/github/StellarCN/py-stellar-base/maintainability

.. image:: https://img.shields.io/codecov/c/github/StellarCN/py-stellar-base/v2
    :alt: Codecov
    :target: https://codecov.io/gh/StellarCN/py-stellar-base

.. image:: https://img.shields.io/pypi/v/stellar-sdk.svg
    :alt: PyPI
    :target: https://pypi.python.org/pypi/stellar-sdk

.. image:: https://img.shields.io/badge/python-%3E%3D3.8-blue
    :alt: Python - Version
    :target: https://pypi.python.org/pypi/stellar-sdk

.. image:: https://img.shields.io/badge/implementation-cpython%20%7C%20pypy-blue
    :alt: PyPI - Implementation
    :target: https://pypi.python.org/pypi/stellar-sdk

.. image:: https://img.shields.io/badge/Stellar%20Protocol-20-blue
    :alt: Stellar Protocol
    :target: https://developers.stellar.org/docs/glossary/scp/

py-stellar-base is a Python library for communicating with
a `Stellar Horizon server`_ and `Soroban-RPC server`_. It is used for building Stellar apps on Python. It supports **Python 3.8+** as
well as PyPy 3.8+.

It provides:

- a networking layer API for Horizon endpoints.
- a networking layer API for Soroban-RPC server methods.
- facilities for building and signing transactions, for communicating with a Stellar Horizon and Soroban-RPC instance, and for submitting transactions or querying network history.

Documentation
-------------
py-stellar-base's documentation can be found at https://stellar-sdk.readthedocs.io.

Installing
----------

.. code-block:: text

    pip install --upgrade stellar-sdk

If you need to use asynchronous, please use the following command to install the required dependencies.

.. code-block:: text

    pip install --upgrade stellar-sdk[aiohttp]

We follow `Semantic Versioning 2.0.0 <https://semver.org/>`_, and I strongly
recommend that you specify its major version number in the dependency
file to avoid the unknown effects of breaking changes.

A Simple Example
----------------
You can find more examples `here <https://github.com/StellarCN/py-stellar-base/tree/main/examples>`__.

.. code-block:: python

    # Alice pay 10.25 XLM to Bob
    from stellar_sdk import Asset, Server, Keypair, TransactionBuilder, Network

    alice_keypair = Keypair.from_secret("SBFZCHU5645DOKRWYBXVOXY2ELGJKFRX6VGGPRYUWHQ7PMXXJNDZFMKD")
    bob_address = "GA7YNBW5CBTJZ3ZZOWX3ZNBKD6OE7A7IHUQVWMY62W2ZBG2SGZVOOPVH"

    server = Server("https://horizon-testnet.stellar.org")
    alice_account = server.load_account(alice_keypair.public_key)
    base_fee = 100
    transaction = (
        TransactionBuilder(
            source_account=alice_account,
            network_passphrase=Network.TESTNET_NETWORK_PASSPHRASE,
            base_fee=base_fee,
        )
        .add_text_memo("Hello, Stellar!")
        .append_payment_op(bob_address, Asset.native(), "10.25")
        .set_timeout(30)
        .build()
    )
    transaction.sign(alice_keypair)
    response = server.submit_transaction(transaction)
    print(response)

stellar-model
-------------
stellar-model allows you to parse the JSON returned by Stellar Horizon
into the Python models, click `here <https://github.com/StellarCN/stellar-model>`__ for more information.

Links
-----
* Document: https://stellar-sdk.readthedocs.io
* Code: https://github.com/StellarCN/py-stellar-base
* Examples: https://github.com/StellarCN/py-stellar-base/tree/main/examples
* Issue tracker: https://github.com/StellarCN/py-stellar-base/issues
* License: `Apache License 2.0 <https://github.com/StellarCN/py-stellar-base/blob/master/LICENSE>`_
* Releases: https://pypi.org/project/stellar-sdk/

Thank you to all the people who have already contributed to py-stellar-base!

.. _Stellar Horizon server: https://github.com/stellar/go/tree/master/services/horizon
.. _Soroban-RPC server: https://soroban.stellar.org/docs/reference/rpc


# <img src="src\dashboard-client\static\logo_compact_transparent.png" width="200">
[![Build Status](https://infraboxci.datahub.sapcloud.io/api/v1/projects/deb14c11-dcbe-41f0-ade8-9d26e87266c3/state.svg?branch=master)](https://infraboxci.datahub.sapcloud.io/dashboard/#/project/sap-infrabox)
[![REUSE status](https://api.reuse.software/badge/github.com/SAP/InfraBox)](https://api.reuse.software/info/github.com/SAP/InfraBox)

InfraBox is a continuous integration system. It's well suited for cloud native applications and has [native support for kubernetes](https://github.com/SAP/infrabox-examples/tree/master/kubernetes). Watch our short introduction video:

[![Introduction to InfraBox](https://img.youtube.com/vi/O8N2U7d404I/0.jpg)](https://www.youtube.com/watch?v=O8N2U7d404I)

Some of InfraBox's features are:

- [Static and dynamic workflows](https://github.com/SAP/infrabox-examples)
- [Set resource limits (CPU and memory) for each task](https://github.com/SAP/infrabox-examples)
- [GitHub integration](docs/install/configure/github.md)
- [Gerrit integration](docs/install/configure/gerrit.md)
- [LDAP support](docs/install/configure/ldap.md)
- [Periodically schedule builds](docs/cronjobs.md)
- [and many more, see our examples](https://github.com/SAP/infrabox-examples)

## Documentation
All our documentation can be found [here](docs/README.md). You can also look at our [example repository](https://github.com/SAP/infrabox-examples) on how to make use of the different features InfraBox provides.

## How to obtain support
If you need help please post your questions to [Stack Overflow](https://stackoverflow.com/questions/tagged/infrabox).
In case you found a bug please open a [Github Issue](https://github.com/SAP/infrabox/issues).

## Roadmap

### 1.2
- Bugfixes
- Improve Documentation (installation, job definition & API)
- Add support for SAML
- Quota management on user and project level

### 1.x
- [knative](https://cloud.google.com/knative/) integration
- Service for creating EKS K8s Clusters
- More to come, _your idea here_

## Contribute
Any contribution is highly appreciated. See our [developer's documentation](docs/dev.md) for details.

## License
Copyright (c) 2018 SAP SE or an SAP affiliate company. All rights reserved.
This file is licensed under the Apache Software License, v. 2 except as noted otherwise in the [LICENSE file](LICENSE).


===========================================================
snakePipes
===========================================================

.. image:: https://readthedocs.org/projects/snakepipes/badge/?version=latest
    :target: http://snakepipes.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

.. image:: https://travis-ci.org/maxplanck-ie/snakepipes.svg?branch=develop
    :target: https://travis-ci.org/maxplanck-ie/snakepipes
    :alt: Build Staus

.. image:: https://zenodo.org/badge/54579435.svg
    :target: https://zenodo.org/badge/latestdoi/54579435
    :alt: Citation

snakePipes are flexible and powerful workflows built using `Snakemake <https://snakemake.readthedocs.io>`__ that simplify the analysis of NGS data.

.. image:: ./docs/content/images/snakePipes_small.png
   :scale: 20 %
   :width: 100 px
   :height: 100 px
   :align: right

Workflows available
--------------------

- DNA-mapping*
- ChIP-seq*
- mRNA-seq*
- noncoding-RNA-seq*
- ATAC-seq*
- scRNA-seq
- Hi-C
- Whole Genome Bisulfite Seq/WGBS

**(*Also available in "allele-specific" mode)**

Installation
-------------

Snakepipes is a set of Snakemake workflows which use conda for installation and dependency resolution, so you will need to `install conda <https://conda.io/docs/user-guide/install/index.html>`__ first.

Afterward, simply run the following:

``conda install mamba -c conda-forge && mamba create -n snakePipes -c mpi-ie -c bioconda -c conda-forge snakePipes``

This will create a new conda environment called "snakePipes" into which snakePipes is installed. You will then need to create the conda environments needed by the various workflows. To facilitate this we provide the `snakePipes` commands:

* ``conda activate snakePipes`` to activate the appropriate conda environment.
* ``snakePipes createEnvs`` to create the various environments.

Indices and annotations needed to run the workflows could be created by a simple command :

``createIndices --genomeURL <path/url to your genome fasta> --gtfURL <path/url to genes.gtf> -o <output_dir> <name>``

where `name` refers to the name/id of your genome (specify as you wish).

A few additional steps you can then take:

1. **Modify/remove/add the organism yaml files appropriately** : these yaml files would contain location of appropriate
GTF files and genome indexes corresponding to different organisms. The location of these files after installation can be
found using ``snakePipes info`` command.

2. **Modify the cluster.yaml file appropriately** : This yaml file contains settings for your cluster scheduler (SGE/slurm).
Location revealed using ``snakePipes info`` command.

Documentation
--------------

For detailed documentation on setup and usage, please visit our `read the docs page <https://snakepipes.readthedocs.io/en/latest/>`__.

Citation
-------------

If you adopt/run snakePipes for your analysis, cite it as follows :

Bhardwaj, Vivek, Steffen Heyne, Katarzyna Sikora, Leily Rabbani, Michael Rauer, Fabian Kilpert, Andreas S. Richter, Devon P. Ryan, and Thomas Manke. 2019. “snakePipes: Facilitating Flexible, Scalable and Integrative Epigenomic Analysis.” Bioinformatics , May. `doi:10.1093/bioinformatics/btz436 <https://doi.org/10.1093/bioinformatics/btz436>`__

Note
-------------

SnakePipes are under active development. We appreciate your help in improving it further. Please use issues to the GitHub repository for feature requests or bug reports.


[![](https://github.com/PMEAL/OpenPNM/actions/workflows/nightly.yml/badge.svg)](https://github.com/PMEAL/OpenPNM/actions/workflows/nightly.yml)
[![](https://codecov.io/gh/PMEAL/OpenPNM/branch/dev/graph/badge.svg)](https://codecov.io/gh/PMEAL/OpenPNM)
[![](https://img.shields.io/badge/Documentation-Read-blue.svg)](https://pmeal.github.io/OpenPNM/)
[![](https://badge.fury.io/py/openpnm.svg)](https://pypi.python.org/pypi/openpnm)
[![](https://anaconda.org/conda-forge/openpnm/badges/installer/conda.svg)](https://anaconda.org/conda-forge/openpnm)

-----

> VERSION 3.0 of OpenPNM is now out. All the [examples on the website](https://openpnm.org/_examples/index.html) are now using the features and idioms of V3. For a description of the main changes please see our [recent blog post](http://pmeal.com/posts/2022-10-10-notebook-post/).

# Overview of OpenPNM

OpenPNM is a comprehensive framework for performing pore network simulations of porous materials.

## More Information

For more details about the package can be found in the [online documentation](https://openpnm.org)

## Installation and Requirements

### [pip](https://pypi.org/project/openpnm/)
OpenPNM can be installed using `pip` by running the following command in a terminal:

```shell
pip install openpnm
```

### [conda-forge](https://anaconda.org/conda-forge/openpnm)
OpenPNM can also be installed from the [conda-forge](https://anaconda.org/conda-forge/openpnm) repository using:

```
conda install -c conda-forge openpnm
```

> [!WARNING]
> For compatibility with ARM64 architecture, we removed `pypardiso` as a hard dependency. However, we still strongly recommend that non-macOS users (including users of older Macs with an Intel CPU) manually install `pypardiso` via `pip install pypardiso` or `conda install -c conda-forge pypardiso`, otherwise OpenPNM simulations will be very slow.

### For developers
For developers who intend to change the source code or contribute to OpenPNM, the source code can be downloaded from [Github](https://github.com/PMEAL/OpenPNM/) and installed by running:

```
pip install -e 'path/to/downloaded/files'
```

The advantage to installing from the source code is that you can edit the files and have access to your changes each time you import OpenPNM.

OpenPNM requires the Scipy Stack (Numpy, Scipy, Matplotlib, etc), which is most conveniently obtained by installing the [Anaconda Distribution](https://www.anaconda.com/download/).

## Asking Questions and Getting Help

Github now has a [Discussions](https://github.com/PMEAL/OpenPNM/discussions) function, which works similarly to [stack overflow](https://www.stackoverflow.com).  Please post your question in the [Q&A category](https://github.com/PMEAL/OpenPNM/discussions?discussions_q=category%3AQ%26A) so devs or users can provide answers, vote on accepted answers, improve on each other's answers, and generally discuss things. Most importantly, all answers are searchable so eventually, once enough questions have been posted and answered, you can find what you're looking for with a simple search.

## Contact

OpenPNM is developed by the Porous Materials Engineering and Analysis Lab [(PMEAL)](http://pmeal.com), in the [Department of Chemical Engineering](https://uwaterloo.ca/chemical-engineering/) at the [University of Waterloo](https://uwaterloo.ca/) in Waterloo, Ontario, Canada.

The lead developer for this project is Prof. Jeff Gostick (jgostick@gmail.com).

## Acknowledgements

OpenPNM is grateful to [CANARIE](https://canarie.ca) for their generous funding over the past few years.  We would also like to acknowledge the support of [NSERC of Canada](https://www.nserc-crsng.gc.ca/) for funding many of the student that have contributed to OpenPNM since it's inception in 2011.

## Citation

If you use OpenPNM in a publication, please cite the following paper:

> _Gostick et al._ "**OpenPNM: a pore network modeling package.**" Computing in Science & Engineering 18, no. 4 (2016): 60-74.
> [doi:10.1109/MCSE.2016.49](https://ieeexplore.ieee.org/document/7478437)

Also, we ask that you "star" :star: this repository so we can track the number of users who are interested in this project, which is helpful for securing future grant funding.


|logo|

Gala is a Python package for Galactic and gravitational dynamics.

|Affiliated package| |Coverage Status| |Build status|

Documentation
-------------

|Documentation Status|

The documentation for ``Gala`` is hosted on `Read the docs
<http://gala.adrian.pw>`__.

Installation and Dependencies
-----------------------------

|conda| |PyPI|

The easiest way to get Gala is to install with pip or conda.

The recommended install method is to use ``pip``::

   pip install gala

If you are on Linux or Mac, you can also install ``gala`` with conda using the
conda-forge channel::

    conda install gala --channel conda-forge

See the `installation
instructions <http://gala.adrian.pw/en/latest/install.html>`_ in the
`documentation <http://gala.adrian.pw>`__ for more information.

Attribution
-----------

|JOSS| |DOI|

If you make use of this code, please cite the `JOSS <http://joss.theoj.org>`_
paper::

    @article{gala,
      doi = {10.21105/joss.00388},
      url = {https://doi.org/10.21105%2Fjoss.00388},
      year = 2017,
      month = {oct},
      publisher = {The Open Journal},
      volume = {2},
      number = {18},
      author = {Adrian M. Price-Whelan},
      title = {Gala: A Python package for galactic dynamics},
      journal = {The Journal of Open Source Software}
    }

Please also cite the Zenodo DOI |DOI| as a software citation - see the
`documentation
<http://gala.adrian.pw/en/latest/index.html#citation-and-attribution>`_ for up
to date citation information.

License
-------

|License|

Copyright 2013-2024 Adrian Price-Whelan and contributors.

``Gala`` is free software made available under the MIT License. For details see
the `LICENSE <https://github.com/adrn/gala/blob/main/LICENSE>`_ file.

.. |Coverage Status| image:: https://codecov.io/gh/adrn/gala/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/adrn/gala
.. |Build status| image:: https://github.com/adrn/gala/actions/workflows/tests.yml/badge.svg
   :target: https://github.com/adrn/gala/actions/workflows/tests.yml
.. |License| image:: http://img.shields.io/badge/license-MIT-blue.svg?style=flat
   :target: https://github.com/adrn/gala/blob/main/LICENSE
.. |PyPI| image:: https://badge.fury.io/py/gala.svg
   :target: https://badge.fury.io/py/gala
.. |conda| image:: https://anaconda.org/conda-forge/gala/badges/version.svg
   :target: https://anaconda.org/conda-forge/gala
.. |Documentation Status| image:: https://readthedocs.org/projects/gala-astro/badge/?version=latest
   :target: http://gala-astro.readthedocs.io/en/latest/?badge=latest
.. |Affiliated package| image:: https://img.shields.io/badge/astropy-affiliated%20package-orange.svg
   :target: http://astropy.org/affiliated
.. |JOSS| image:: http://joss.theoj.org/papers/10.21105/joss.00388/status.svg
   :target: http://joss.theoj.org/papers/10.21105/joss.00388
.. |DOI| image:: https://zenodo.org/badge/17577779.svg
   :target: https://zenodo.org/badge/latestdoi/17577779
.. |ASCL| image:: https://img.shields.io/badge/ascl-1707.006-blue.svg?colorB=262255
   :target: http://ascl.net/1707.006
.. |logo| image:: https://gala.adrian.pw/en/latest/_static/Gala_Logo_RGB.png
   :target: https://github.com/adrn/gala
   :width: 400

Contributors
------------

See the `AUTHORS.rst <https://github.com/adrn/gala/blob/main/AUTHORS.rst>`_
file for a complete list of contributors to the project.