# Welcome to the new nnU-Net!

Click [here](https://github.com/MIC-DKFZ/nnUNet/tree/nnunetv1) if you were looking for the old one instead.

Coming from V1? Check out the [TLDR Migration Guide](documentation/tldr_migration_guide_from_v1.md). Reading the rest of the documentation is still strongly recommended ;-)

## **2024-04-18 UPDATE: New residual encoder UNet presets available!**
Residual encoder UNet presets substantially improve segmentation performance.
They ship for a variety of GPU memory targets. It's all awesome stuff, promised!
Read more :point_right: [here](documentation/resenc_presets.md) :point_left:

Also check out our [new paper](https://arxiv.org/pdf/2404.09556.pdf) on systematically benchmarking recent developments in medical image segmentation. You might be surprised!

# What is nnU-Net?
Image datasets are enormously diverse: image dimensionality (2D, 3D), modalities/input channels (RGB image, CT, MRI, microscopy, ...),
image sizes, voxel sizes, class ratio, target structure properties and more change substantially between datasets.
Traditionally, given a new problem, a tailored solution needs to be manually designed and optimized  - a process that
is prone to errors, not scalable and where success is overwhelmingly determined by the skill of the experimenter. Even
for experts, this process is anything but simple: there are not only many design choices and data properties that need to
be considered, but they are also tightly interconnected, rendering reliable manual pipeline optimization all but impossible!

![nnU-Net overview](documentation/assets/nnU-Net_overview.png)

**nnU-Net is a semantic segmentation method that automatically adapts to a given dataset. It will analyze the provided
training cases and automatically configure a matching U-Net-based segmentation pipeline. No expertise required on your
end! You can simply train the models and use them for your application**.

Upon release, nnU-Net was evaluated on 23 datasets belonging to competitions from the biomedical domain. Despite competing
with handcrafted solutions for each respective dataset, nnU-Net's fully automated pipeline scored several first places on
open leaderboards! Since then nnU-Net has stood the test of time: it continues to be used as a baseline and method
development framework ([9 out of 10 challenge winners at MICCAI 2020](https://arxiv.org/abs/2101.00232) and 5 out of 7
in MICCAI 2021 built their methods on top of nnU-Net,
 [we won AMOS2022 with nnU-Net](https://amos22.grand-challenge.org/final-ranking/))!

Please cite the [following paper](https://www.google.com/url?q=https://www.nature.com/articles/s41592-020-01008-z&sa=D&source=docs&ust=1677235958581755&usg=AOvVaw3dWL0SrITLhCJUBiNIHCQO) when using nnU-Net:

    Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring
    method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211.

## What can nnU-Net do for you?
If you are a **domain scientist** (biologist, radiologist, ...) looking to analyze your own images, nnU-Net provides
an out-of-the-box solution that is all but guaranteed to provide excellent results on your individual dataset. Simply
convert your dataset into the nnU-Net format and enjoy the power of AI - no expertise required!

If you are an **AI researcher** developing segmentation methods, nnU-Net:
- offers a fantastic out-of-the-box applicable baseline algorithm to compete against
- can act as a method development framework to test your contribution on a large number of datasets without having to
tune individual pipelines (for example evaluating a new loss function)
- provides a strong starting point for further dataset-specific optimizations. This is particularly used when competing
in segmentation challenges
- provides a new perspective on the design of segmentation methods: maybe you can find better connections between
dataset properties and best-fitting segmentation pipelines?

## What is the scope of nnU-Net?
nnU-Net is built for semantic segmentation. It can handle 2D and 3D images with arbitrary
input modalities/channels. It can understand voxel spacings, anisotropies and is robust even when classes are highly
imbalanced.

nnU-Net relies on supervised learning, which means that you need to provide training cases for your application. The number of
required training cases varies heavily depending on the complexity of the segmentation problem. No
one-fits-all number can be provided here! nnU-Net does not require more training cases than other solutions - maybe
even less due to our extensive use of data augmentation.

nnU-Net expects to be able to process entire images at once during preprocessing and postprocessing, so it cannot
handle enormous images. As a reference: we tested images from 40x40x40 pixels all the way up to 1500x1500x1500 in 3D
and 40x40 up to ~30000x30000 in 2D! If your RAM allows it, larger is always possible.

## How does nnU-Net work?
Given a new dataset, nnU-Net will systematically analyze the provided training cases and create a 'dataset fingerprint'.
nnU-Net then creates several U-Net configurations for each dataset:
- `2d`: a 2D U-Net (for 2D and 3D datasets)
- `3d_fullres`: a 3D U-Net that operates on a high image resolution (for 3D datasets only)
- `3d_lowres` → `3d_cascade_fullres`: a 3D U-Net cascade where first a 3D U-Net operates on low resolution images and
then a second high-resolution 3D U-Net refined the predictions of the former (for 3D datasets with large image sizes only)

**Note that not all U-Net configurations are created for all datasets. In datasets with small image sizes, the
U-Net cascade (and with it the 3d_lowres configuration) is omitted because the patch size of the full
resolution U-Net already covers a large part of the input images.**

nnU-Net configures its segmentation pipelines based on a three-step recipe:
- **Fixed parameters** are not adapted. During development of nnU-Net we identified a robust configuration (that is, certain architecture and training properties) that can
simply be used all the time. This includes, for example, nnU-Net's loss function, (most of the) data augmentation strategy and learning rate.
- **Rule-based parameters** use the dataset fingerprint to adapt certain segmentation pipeline properties by following
hard-coded heuristic rules. For example, the network topology (pooling behavior and depth of the network architecture)
are adapted to the patch size; the patch size, network topology and batch size are optimized jointly given some GPU
memory constraint.
- **Empirical parameters** are essentially trial-and-error. For example the selection of the best U-net configuration
for the given dataset (2D, 3D full resolution, 3D low resolution, 3D cascade) and the optimization of the postprocessing strategy.

## How to get started?
Read these:
- [Installation instructions](documentation/installation_instructions.md)
- [Dataset conversion](documentation/dataset_format.md)
- [Usage instructions](documentation/how_to_use_nnunet.md)

Additional information:
- [Learning from sparse annotations (scribbles, slices)](documentation/ignore_label.md)
- [Region-based training](documentation/region_based_training.md)
- [Manual data splits](documentation/manual_data_splits.md)
- [Pretraining and finetuning](documentation/pretraining_and_finetuning.md)
- [Intensity Normalization in nnU-Net](documentation/explanation_normalization.md)
- [Manually editing nnU-Net configurations](documentation/explanation_plans_files.md)
- [Extending nnU-Net](documentation/extending_nnunet.md)
- [What is different in V2?](documentation/changelog.md)

Competitions:
- [AutoPET II](documentation/competitions/AutoPETII.md)

[//]: # (- [Ignore label](documentation/ignore_label.md))

## Where does nnU-Net perform well and where does it not perform?
nnU-Net excels in segmentation problems that need to be solved by training from scratch,
for example: research applications that feature non-standard image modalities and input channels,
challenge datasets from the biomedical domain, majority of 3D segmentation problems, etc . We have yet to find a
dataset for which nnU-Net's working principle fails!

Note: On standard segmentation
problems, such as 2D RGB images in ADE20k and Cityscapes, fine-tuning a foundation model (that was pretrained on a large corpus of
similar images, e.g. Imagenet 22k, JFT-300M) will provide better performance than nnU-Net! That is simply because these
models allow much better initialization. Foundation models are not supported by nnU-Net as
they 1) are not useful for segmentation problems that deviate from the standard setting (see above mentioned
datasets), 2) would typically only support 2D architectures and 3) conflict with our core design principle of carefully adapting
the network topology for each dataset (if the topology is changed one can no longer transfer pretrained weights!)

## What happened to the old nnU-Net?
The core of the old nnU-Net was hacked together in a short time period while participating in the Medical Segmentation
Decathlon challenge in 2018. Consequently, code structure and quality were not the best. Many features
were added later on and didn't quite fit into the nnU-Net design principles. Overall quite messy, really. And annoying to work with.

nnU-Net V2 is a complete overhaul. The "delete everything and start again" kind. So everything is better
(in the author's opinion haha). While the segmentation performance [remains the same](https://docs.google.com/spreadsheets/d/13gqjIKEMPFPyMMMwA1EML57IyoBjfC3-QCTn4zRN_Mg/edit?usp=sharing), a lot of cool stuff has been added.
It is now also much easier to use it as a development framework and to manually fine-tune its configuration to new
datasets. A big driver for the reimplementation was also the emergence of [Helmholtz Imaging](http://helmholtz-imaging.de),
prompting us to extend nnU-Net to more image formats and domains. Take a look [here](documentation/changelog.md) for some highlights.

# Acknowledgements

nnU-Net is developed and maintained by the Applied Computer Vision Lab (ACVL) of [Helmholtz Imaging](http://helmholtz-imaging.de)
and the [Division of Medical Image Computing](https://www.dkfz.de/en/mic/index.php) at the
[German Cancer Research Center (DKFZ)](https://www.dkfz.de/en/index.html).


TextAttack 🐙

Generating adversarial examples for NLP models

  [TextAttack Documentation on ReadTheDocs]

  About •
  Setup •
  Usage •
  Design

## About

TextAttack is a Python framework for adversarial attacks, data augmentation, and model training in NLP.

> If you're looking for information about TextAttack's menagerie of pre-trained models, you might want the [TextAttack Model Zoo](https://textattack.readthedocs.io/en/latest/3recipes/models.html) page.

## Slack Channel

For help and realtime updates related to TextAttack, please [join the TextAttack Slack](https://join.slack.com/t/textattack/shared_invite/zt-huomtd9z-KqdHBPPu2rOP~Z8q3~urgg)!

### *Why TextAttack?*

There are lots of reasons to use TextAttack:

1. **Understand NLP models better** by running different adversarial attacks on them and examining the output
2. **Research and develop different NLP adversarial attacks** using the TextAttack framework and library of components
3. **Augment your dataset** to increase model generalization and robustness downstream
3. **Train NLP models** using just a single command (all downloads included!)

## Setup

### Installation

You should be running Python 3.6+ to use this package. A CUDA-compatible GPU is optional but will greatly improve code speed. TextAttack is available through pip:

```bash
pip install textattack
```

Once TextAttack is installed, you can run it via command-line (`textattack ...`)
or via python module (`python -m textattack ...`).

> **Tip**: TextAttack downloads files to `~/.cache/textattack/` by default. This includes pretrained models,
> dataset samples, and the configuration file `config.yaml`. To change the cache path, set the
> environment variable `TA_CACHE_DIR`. (for example: `TA_CACHE_DIR=/tmp/ textattack attack ...`).

## Usage

### Help: `textattack --help`

TextAttack's main features can all be accessed via the `textattack` command. Two very
common commands are `textattack attack <args>`, and `textattack augment <args>`. You can see more
information about all commands using
```bash
textattack --help
```
or a specific command using, for example,
```bash
textattack attack --help
```

The [`examples/`](examples/) folder includes scripts showing common TextAttack usage for training models, running attacks, and augmenting a CSV file.

The [documentation website](https://textattack.readthedocs.io/en/latest) contains walkthroughs explaining basic usage of TextAttack, including building a custom transformation and a custom constraint..

### Running Attacks: `textattack attack --help`

The easiest way to try out an attack is via the command-line interface, `textattack attack`.

> **Tip:** If your machine has multiple GPUs, you can distribute the attack across them using the `--parallel` option. For some attacks, this can really help performance. (If you want to attack Keras models in parallel, please check out `examples/attack/attack_keras_parallel.py` instead)

Here are some concrete examples:

*TextFooler on BERT trained on the MR sentiment classification dataset*:
```bash
textattack attack --recipe textfooler --model bert-base-uncased-mr --num-examples 100
```

*DeepWordBug on DistilBERT trained on the Quora Question Pairs paraphrase identification dataset*:
```bash
textattack attack --model distilbert-base-uncased-cola --recipe deepwordbug --num-examples 100
```

*Beam search with beam width 4 and word embedding transformation and untargeted goal function on an LSTM*:
```bash
textattack attack --model lstm-mr --num-examples 20 \
 --search-method beam-search^beam_width=4 --transformation word-swap-embedding \
 --constraints repeat stopword max-words-perturbed^max_num_words=2 embedding^min_cos_sim=0.8 part-of-speech \
 --goal-function untargeted-classification
```

> **Tip:** Instead of specifying a dataset and number of examples, you can pass `--interactive` to attack samples inputted by the user.

### Attacks and Papers Implemented ("Attack Recipes"): `textattack attack --recipe [recipe_name]`

We include attack recipes which implement attacks from the literature. You can list attack recipes using `textattack list attack-recipes`.

To run an attack recipe: `textattack attack --recipe [recipe_name]`

Attack Recipe Name
Goal Function
ConstraintsEnforced
Transformation
Search Method
Main Idea

Attacks on classification tasks, like sentiment classification and entailment:

a2t

Untargeted {Classification, Entailment}
Percentage of words perturbed, Word embedding distance, DistilBERT sentence encoding cosine similarity, part-of-speech consistency
Counter-fitted word embedding swap (or) BERT Masked Token Prediction
Greedy-WIR (gradient)
from (["Towards Improving Adversarial Training of NLP Models" (Yoo et al., 2021)](https://arxiv.org/abs/2109.00544))

alzantot
Untargeted {Classification, Entailment}
Percentage of words perturbed, Language Model perplexity, Word embedding distance
Counter-fitted word embedding swap
Genetic Algorithm
from (["Generating Natural Language Adversarial Examples" (Alzantot et al., 2018)](https://arxiv.org/abs/1804.07998))

bae
Untargeted Classification
USE sentence encoding cosine similarity
BERT Masked Token Prediction
Greedy-WIR
BERT masked language model transformation attack from (["BAE: BERT-based Adversarial Examples for Text Classification" (Garg & Ramakrishnan, 2019)](https://arxiv.org/abs/2004.01970)).

bert-attack
Untargeted Classification
USE sentence encoding cosine similarity, Maximum number of words perturbed
BERT Masked Token Prediction (with subword expansion)
Greedy-WIR
 (["BERT-ATTACK: Adversarial Attack Against BERT Using BERT" (Li et al., 2020)](https://arxiv.org/abs/2004.09984))

checklist
{Untargeted, Targeted} Classification
checklist distance
contract, extend, and substitutes name entities
Greedy-WIR
Invariance testing implemented in CheckList . (["Beyond Accuracy: Behavioral Testing of NLP models with CheckList" (Ribeiro et al., 2020)](https://arxiv.org/abs/2005.04118))

 clare
Untargeted {Classification, Entailment}
USE sentence encoding cosine similarity
RoBERTa Masked Prediction for token swap, insert and merge
Greedy
["Contextualized Perturbation for Textual Adversarial Attack" (Li et al., 2020)](https://arxiv.org/abs/2009.07502))

deepwordbug
{Untargeted, Targeted} Classification
Levenshtein edit distance
{Character Insertion, Character Deletion, Neighboring Character Swap, Character Substitution}
Greedy-WIR
Greedy replace-1 scoring and multi-transformation character-swap attack (["Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers" (Gao et al., 2018)](https://arxiv.org/abs/1801.04354)

 faster-alzantot
Untargeted {Classification, Entailment}
Percentage of words perturbed, Language Model perplexity, Word embedding distance
Counter-fitted word embedding swap
Genetic Algorithm
Modified, faster version of the Alzantot et al. genetic algorithm, from (["Certified Robustness to Adversarial Word Substitutions" (Jia et al., 2019)](https://arxiv.org/abs/1909.00986))

hotflip (word swap)
Untargeted Classification
Word Embedding Cosine Similarity, Part-of-speech match, Number of words perturbed
Gradient-Based Word Swap
Beam search
 (["HotFlip: White-Box Adversarial Examples for Text Classification" (Ebrahimi et al., 2017)](https://arxiv.org/abs/1712.06751))

iga
Untargeted {Classification, Entailment}
Percentage of words perturbed, Word embedding distance
Counter-fitted word embedding swap
Genetic Algorithm
Improved genetic algorithm -based word substitution from (["Natural Language Adversarial Attacks and Defenses in Word Level (Wang et al., 2019)"](https://arxiv.org/abs/1909.06723)

input-reduction
Input Reduction

Word deletion
Greedy-WIR
Greedy attack with word importance ranking , Reducing the input while maintaining the prediction through word importance ranking (["Pathologies of Neural Models Make Interpretation Difficult" (Feng et al., 2018)](https://arxiv.org/pdf/1804.07781.pdf))

kuleshov
Untargeted Classification
Thought vector encoding cosine similarity, Language model similarity probability
Counter-fitted word embedding swap
Greedy word swap
(["Adversarial Examples for Natural Language Classification Problems" (Kuleshov et al., 2018)](https://openreview.net/pdf?id=r1QZ3zbAZ))

pruthi
Untargeted Classification
Minimum word length, Maximum number of words perturbed
{Neighboring Character Swap, Character Deletion, Character Insertion, Keyboard-Based Character Swap}
Greedy search
simulates common typos (["Combating Adversarial Misspellings with Robust Word Recognition" (Pruthi et al., 2019)](https://arxiv.org/abs/1905.11268)

pso
Untargeted Classification

HowNet Word Swap
Particle Swarm Optimization
(["Word-level Textual Adversarial Attacking as Combinatorial Optimization" (Zang et al., 2020)](https://www.aclweb.org/anthology/2020.acl-main.540/))

pwws
Untargeted Classification

WordNet-based synonym swap
Greedy-WIR (saliency)
Greedy attack with word importance ranking based on word saliency and synonym swap scores (["Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency" (Ren et al., 2019)](https://www.aclweb.org/anthology/P19-1103/))

textbugger : (black-box)
Untargeted Classification
USE sentence encoding cosine similarity
{Character Insertion, Character Deletion, Neighboring Character Swap, Character Substitution}
Greedy-WIR
([(["TextBugger: Generating Adversarial Text Against Real-world Applications" (Li et al., 2018)](https://arxiv.org/abs/1812.05271)).

textfooler
Untargeted {Classification, Entailment}
Word Embedding Distance, Part-of-speech match, USE sentence encoding cosine similarity
Counter-fitted word embedding swap
Greedy-WIR
Greedy attack with word importance ranking  (["Is Bert Really Robust?" (Jin et al., 2019)](https://arxiv.org/abs/1907.11932))

Attacks on sequence-to-sequence models:

morpheus
Minimum BLEU Score

Inflection Word Swap
Greedy search
Greedy to replace words with their inflections with the goal of minimizing BLEU score (["It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations"](https://www.aclweb.org/anthology/2020.acl-main.263.pdf)

seq2sick :(black-box)
Non-overlapping output

Counter-fitted word embedding swap
Greedy-WIR
Greedy attack with goal of changing every word in the output translation. Currently implemented as black-box with plans to change to white-box as done in paper (["Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples" (Cheng et al., 2018)](https://arxiv.org/abs/1803.01128))

#### Recipe Usage Examples

Here are some examples of testing attacks from the literature from the command-line:

*TextFooler against BERT fine-tuned on SST-2:*
```bash
textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examples 10
```

*seq2sick (black-box) against T5 fine-tuned for English-German translation:*
```bash
 textattack attack --model t5-en-de --recipe seq2sick --num-examples 100
```

### Augmenting Text: `textattack augment`

Many of the components of TextAttack are useful for data augmentation. The `textattack.Augmenter` class
uses a transformation and a list of constraints to augment data. We also offer  built-in recipes
for data augmentation:
- `wordnet` augments text by replacing words with WordNet synonyms
- `embedding` augments text by replacing words with neighbors in the counter-fitted embedding space, with a constraint to ensure their cosine similarity is at least 0.8
- `charswap` augments text by substituting, deleting, inserting, and swapping adjacent characters
- `eda` augments text with a combination of word insertions, substitutions and deletions.
- `checklist` augments text by contraction/extension and by substituting names, locations, numbers.
- `clare` augments text by replacing, inserting, and merging with a pre-trained masked language model.
- `back_trans` augments text by backtranslation approach.
- `back_transcription` augments text by back transcription approach.

#### Augmentation Command-Line Interface
The easiest way to use our data augmentation tools is with `textattack augment <args>`. `textattack augment`
takes an input CSV file and text column to augment, along with the number of words to change per augmentation
and the number of augmentations per input example. It outputs a CSV in the same format with all the augmentation
examples corresponding to the proper columns.

For example, given the following as `examples.csv`:

```csv
"text",label
"the rock is destined to be the 21st century's new conan and that he's going to make a splash even greater than arnold schwarzenegger , jean- claud van damme or steven segal.", 1
"the gorgeously elaborate continuation of 'the lord of the rings' trilogy is so huge that a column of words cannot adequately describe co-writer/director peter jackson's expanded vision of j . r . r . tolkien's middle-earth .", 1
"take care of my cat offers a refreshingly different slice of asian cinema .", 1
"a technically well-made suspenser . . . but its abrupt drop in iq points as it races to the finish line proves simply too discouraging to let slide .", 0
"it's a mystery how the movie could be released in this condition .", 0
```

The command
```bash
textattack augment --input-csv examples.csv --output-csv output.csv  --input-column text --recipe embedding --pct-words-to-swap .1 --transformations-per-example 2 --exclude-original
```
will augment the `text` column by altering 10% of each example's words, generating twice as many augmentations as original inputs, and exclude the original inputs from the
output CSV. (All of this will be saved to `augment.csv` by default.)

> **Tip:** Just as running attacks interactively, you can also pass `--interactive` to augment samples inputted by the user to quickly try out different augmentation recipes!

After augmentation, here are the contents of `augment.csv`:
```csv
text,label
"the rock is destined to be the 21st century's newest conan and that he's gonna to make a splashing even stronger than arnold schwarzenegger , jean- claud van damme or steven segal.",1
"the rock is destined to be the 21tk century's novel conan and that he's going to make a splat even greater than arnold schwarzenegger , jean- claud van damme or stevens segal.",1
the gorgeously elaborate continuation of 'the lord of the rings' trilogy is so huge that a column of expression significant adequately describe co-writer/director pedro jackson's expanded vision of j . rs . r . tolkien's middle-earth .,1
the gorgeously elaborate continuation of 'the lordy of the piercings' trilogy is so huge that a column of mots cannot adequately describe co-novelist/director peter jackson's expanded vision of j . r . r . tolkien's middle-earth .,1
take care of my cat offerings a pleasantly several slice of asia cinema .,1
taking care of my cat offers a pleasantly different slice of asiatic kino .,1
a technically good-made suspenser . . . but its abrupt drop in iq points as it races to the finish bloodline proves straightforward too disheartening to let slide .,0
a technically well-made suspenser . . . but its abrupt drop in iq dot as it races to the finish line demonstrates simply too disheartening to leave slide .,0
it's a enigma how the film wo be releases in this condition .,0
it's a enigma how the filmmaking wo be publicized in this condition .,0
```

The 'embedding' augmentation recipe uses counterfitted embedding nearest-neighbors to augment data.

#### Augmentation Python Interface
In addition to the command-line interface, you can augment text dynamically by importing the
`Augmenter` in your own code. All `Augmenter` objects implement `augment` and `augment_many` to generate augmentations
of a string or a list of strings. Here's an example of how to use the `EmbeddingAugmenter` in a python script:

```python
>>> from textattack.augmentation import EmbeddingAugmenter
>>> augmenter = EmbeddingAugmenter()
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I notable create, I do not understand.', 'What I significant create, I do not understand.', 'What I cannot engender, I do not understand.', 'What I cannot creating, I do not understand.', 'What I cannot creations, I do not understand.', 'What I cannot create, I do not comprehend.', 'What I cannot create, I do not fathom.', 'What I cannot create, I do not understanding.', 'What I cannot create, I do not understands.', 'What I cannot create, I do not understood.', 'What I cannot create, I do not realise.']
```
You can also create your own augmenter from scratch by importing transformations/constraints from `textattack.transformations` and `textattack.constraints`. Here's an example that generates augmentations of a string using `WordSwapRandomCharacterDeletion`:

```python
>>> from textattack.transformations import WordSwapRandomCharacterDeletion
>>> from textattack.transformations import CompositeTransformation
>>> from textattack.augmentation import Augmenter
>>> transformation = CompositeTransformation([WordSwapRandomCharacterDeletion()])
>>> augmenter = Augmenter(transformation=transformation, transformations_per_example=5)
>>> s = 'What I cannot create, I do not understand.'
>>> augmenter.augment(s)
['What I cannot creae, I do not understand.', 'What I cannot creat, I do not understand.', 'What I cannot create, I do not nderstand.', 'What I cannot create, I do nt understand.', 'Wht I cannot create, I do not understand.']
```

#### Prompt Augmentation
In additional to augmentation of regular text, you can augment prompts and then generate responses to
the augmented prompts using a large language model (LLMs). The augmentation is performed using the same
`Augmenter` as above. To generate responses, you can use your own LLM, a HuggingFace LLM, or an OpenAI LLM.
Here's an example using a pretrained HuggingFace LLM:

```python
>>> from textattack.augmentation import EmbeddingAugmenter
>>> from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
>>> from textattack.llms import HuggingFaceLLMWrapper
>>> from textattack.prompt_augmentation import PromptAugmentationPipeline
>>> augmenter = EmbeddingAugmenter(transformations_per_example=3)
>>> model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-small")
>>> tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-small")
>>> model_wrapper = HuggingFaceLLMWrapper(model, tokenizer)
>>> pipeline = PromptAugmentationPipeline(augmenter, model_wrapper)
>>> pipeline("Classify the following piece of text as `positive` or `negative`: This movie is great!")
[('Classify the following piece of text as `positive` or `negative`: This film is great!', ['positive']), ('Classify the following piece of text as `positive` or `negative`: This movie is fabulous!', ['positive']), ('Classify the following piece of text as `positive` or `negative`: This movie is wonderful!', ['positive'])]
```

### Training Models:  `textattack train`

Our model training code is available via `textattack train` to help you train LSTMs,
CNNs, and `transformers` models using TextAttack out-of-the-box. Datasets are
automatically loaded using the `datasets` package.

#### Training Examples
*Train our default LSTM for 50 epochs on the Yelp Polarity dataset:*
```bash
textattack train --model-name-or-path lstm --dataset yelp_polarity  --epochs 50 --learning-rate 1e-5
```

*Fine-Tune `bert-base` on the `CoLA` dataset for 5 epochs**:
```bash
textattack train --model-name-or-path bert-base-uncased --dataset glue^cola --per-device-train-batch-size 8 --epochs 5
```

### To check datasets: `textattack peek-dataset`

To take a closer look at a dataset, use `textattack peek-dataset`. TextAttack will print some cursory statistics about the inputs and outputs from the dataset. For example,
```bash
textattack peek-dataset --dataset-from-huggingface snli
```
will show information about the SNLI dataset from the NLP package.

### To list functional components: `textattack list`

There are lots of pieces in TextAttack, and it can be difficult to keep track of all of them. You can use `textattack list` to list components, for example, pretrained models (`textattack list models`) or available search methods (`textattack list search-methods`).

## Design

### Models

TextAttack is model-agnostic!  You can use `TextAttack` to analyze any model that outputs IDs, tensors, or strings. To help users, TextAttack includes pre-trained models for different common NLP tasks. This makes it easier for
users to get started with TextAttack. It also enables a more fair comparison of attacks from
the literature.

#### Built-in Models and Datasets

TextAttack also comes built-in with models and datasets. Our command-line interface will automatically match the correct
dataset to the correct model. We include 82 different (Oct 2020) pre-trained models for each of the nine [GLUE](https://gluebenchmark.com/)
tasks, as well as some common datasets for classification, translation, and summarization.

A list of available pretrained models and their validation accuracies is available at
[textattack/models/README.md](textattack/models/README.md). You can also view a full list of provided models
& datasets via `textattack attack --help`.

Here's an example of using one of the built-in models (the SST-2 dataset is automatically loaded):

```bash
textattack attack --model roberta-base-sst2 --recipe textfooler --num-examples 10
```

#### HuggingFace support: `transformers` models and `datasets` datasets

We also provide built-in support for [`transformers` pretrained models](https://huggingface.co/models)
and datasets from the [`datasets` package](https://github.com/huggingface/datasets)! Here's an example of loading
and attacking a pre-trained model and dataset:

```bash
textattack attack --model-from-huggingface distilbert-base-uncased-finetuned-sst-2-english --dataset-from-huggingface glue^sst2 --recipe deepwordbug --num-examples 10
```

You can explore other pre-trained models using the `--model-from-huggingface` argument, or other datasets by changing
`--dataset-from-huggingface`.

#### Loading a model or dataset from a file

You can easily try out an attack on a local model or dataset sample. To attack a pre-trained model,
create a short file that loads them as variables `model` and `tokenizer`.  The `tokenizer` must
be able to transform string inputs to lists or tensors of IDs using a method called `encode()`. The
model must take inputs via the `__call__` method.

##### Custom Model from a file
To experiment with a model you've trained, you could create the following file
and name it `my_model.py`:

```python
model = load_your_model_with_custom_code() # replace this line with your model loading code
tokenizer = load_your_tokenizer_with_custom_code() # replace this line with your tokenizer loading code
```

Then, run an attack with the argument `--model-from-file my_model.py`. The model and tokenizer will be loaded automatically.

### Custom Datasets

#### Dataset from a file

Loading a dataset from a file is very similar to loading a model from a file. A 'dataset' is any iterable of `(input, output)` pairs.
The following example would load a sentiment classification dataset from file `my_dataset.py`:

```python
dataset = [('Today was....', 1), ('This movie is...', 0), ...]
```

You can then run attacks on samples from this dataset by adding the argument `--dataset-from-file my_dataset.py`.

#### Dataset loading via other mechanism, see: [more details at here](https://textattack.readthedocs.io/en/latest/api/datasets.html)

```python
import textattack
my_dataset = [("text",label),....]
new_dataset = textattack.datasets.Dataset(my_dataset)
```

#### Dataset via AttackedText class

To allow for word replacement after a sequence has been tokenized, we include an `AttackedText` object
which maintains both a list of tokens and the original text, with punctuation. We use this object in favor of a list of words or just raw text.

### Attacks and how to design a new attack

We formulate an attack as consisting of four components: a **goal function** which determines if the attack has succeeded, **constraints** defining which perturbations are valid, a **transformation** that generates potential modifications given an input, and a **search method** which traverses through the search space of possible perturbations. The attack attempts to perturb an input text such that the model output fulfills the goal function (i.e., indicating whether the attack is successful) and the perturbation adheres to the set of constraints (e.g., grammar constraint, semantic similarity constraint). A search method is used to find a sequence of transformations that produce a successful adversarial example.

This modular design unifies adversarial attack methods into one system, enables us to easily assemble attacks from the literature while re-using components that are shared across attacks. We provides clean, readable implementations of 16 adversarial attack recipes from the literature (see above table). For the first time, these attacks can be benchmarked, compared, and analyzed in a standardized setting.

TextAttack is model-agnostic - meaning it can run attacks on models implemented in any deep learning framework. Model objects must be able to take a string (or list of strings) and return an output that can be processed by the goal function. For example, machine translation models take a list of strings as input and produce a list of strings as output. Classification and entailment models return an array of scores. As long as the user's model meets this specification, the model is fit to use with TextAttack.

#### Goal Functions

A `GoalFunction` takes as input an `AttackedText` object, scores it, and determines whether the attack has succeeded, returning a `GoalFunctionResult`.

#### Constraints

A `Constraint` takes as input a current `AttackedText`, and a list of transformed `AttackedText`s. For each transformed option, it returns a boolean representing whether the constraint is met.

#### Transformations

A `Transformation` takes as input an `AttackedText` and returns a list of possible transformed `AttackedText`s. For example, a transformation might return all possible synonym replacements.

#### Search Methods

A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a final `GoalFunctionResult` The search is given access to the `get_transformations` function, which takes as input an `AttackedText` object and outputs a list of possible transformations filtered by meeting all of the attack’s constraints. A search consists of successive calls to `get_transformations` until the search succeeds (determined using `get_goal_results`) or is exhausted.

## On Benchmarking Attacks

- See our analysis paper: Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples at [EMNLP BlackBoxNLP](https://arxiv.org/abs/2009.06368).

- As we emphasized in the above paper, we don't recommend to directly compare Attack Recipes out of the box.

- This comment is due to that attack recipes in the recent literature used different ways or thresholds in setting up their constraints. Without the constraint space held constant, an increase in attack success rate could come from an improved search or transformation method or a less restrictive search space.

- Our Github on benchmarking scripts and results:  [TextAttack-Search-Benchmark Github](https://github.com/QData/TextAttack-Search-Benchmark)

## On Quality of Generated Adversarial Examples in Natural Language

- Our analysis Paper in [EMNLP Findings](https://arxiv.org/abs/2004.14174)
- We analyze the generated adversarial examples of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.
- Our Github on Reevaluation results: [Reevaluating-NLP-Adversarial-Examples Github](https://github.com/QData/Reevaluating-NLP-Adversarial-Examples)
- As we have emphasized in this analysis paper, we recommend researchers and users to be EXTREMELY mindful on the quality of generated adversarial examples in natural language
- We recommend the field to use human-evaluation derived thresholds for setting up constraints

## Multi-lingual Support

- see example code: [https://github.com/QData/TextAttack/blob/master/examples/attack/attack_camembert.py](https://github.com/QData/TextAttack/blob/master/examples/attack/attack_camembert.py) for using our framework to attack French-BERT.

- see tutorial notebook: [https://textattack.readthedocs.io/en/latest/2notebook/Example_4_CamemBERT.html](https://textattack.readthedocs.io/en/latest/2notebook/Example_4_CamemBERT.html) for using our framework to attack French-BERT.

- See [README_ZH.md](https://github.com/QData/TextAttack/blob/master/README_ZH.md) for our README in Chinese

## Contributing to TextAttack

We welcome suggestions and contributions! Submit an issue or pull request and we will do our best to respond in a timely manner. TextAttack is currently in an "alpha" stage in which we are working to improve its capabilities and design.

See [CONTRIBUTING.md](https://github.com/QData/TextAttack/blob/master/CONTRIBUTING.md) for detailed information on contributing.

## Citing TextAttack

If you use TextAttack for your research, please cite [TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP](https://arxiv.org/abs/2005.05909).

```bibtex
@inproceedings{morris2020textattack,
  title={TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP},
  author={Morris, John and Lifland, Eli and Yoo, Jin Yong and Grigsby, Jake and Jin, Di and Qi, Yanjun},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
  pages={119--126},
  year={2020}
}
```


  CodeChecker

**CodeChecker** is a static analysis infrastructure built on the [LLVM/Clang
Static Analyzer](http://clang-analyzer.llvm.org) toolchain, replacing
[`scan-build`](http://clang-analyzer.llvm.org/scan-build.html) in a Linux or
macOS (OS X) development environment.

![Web interface showing list of analysed projects and bugs](images/demo.gif)

**:bulb: Check out our [DEMO](https://codechecker-demo.eastus.cloudapp.azure.com) showing some analysis results of open-source projects!**

# Main features
## Command line C/C++ Analysis
  * Executes [_Clang-Tidy_](http://clang.llvm.org/extra/clang-tidy/), [_Clang Static Analyzer_](http://clang-analyzer.llvm.org/) with Cross-Translation Unit analysis, Statistical Analysis (when checkers are available), [_Cppcheck_](https://cppcheck.sourceforge.io/), and the [_GCC Static Analyzer_](https://gcc.gnu.org/wiki/StaticAnalyzer).
  * Creates the JSON compilation database by wiretapping any build process (e.g., `CodeChecker log -b "make"`).
  * Automatically analyzes GCC cross-compiled projects: detecting GCC or Clang compiler configuration and forming the corresponding clang analyzer invocations.
  * Incremental analysis: Only the changed files and its dependencies need to be reanalyzed.
  * False positive suppression with a possibility to add review comments.
  * Result visualization in command line or in static HTML.

## Web-based report storage
  * **You can store & visualize thousands of analysis reports** of many analyzers like
    Clang Static Analyzer (C/C++), Clang Tidy (C/C++), Facebook Infer (C/C++, Java), Clang Sanitizers (C/C++), Spotbugs (Java), Pylint (Python), Eslint (Javascript) ...
    For a complete list see [Supported Analyzers](supported_code_analyzers.md)
  * **Web application** for viewing discovered code defects with a streamlined,
    easy experience (with PostgreSQL, or SQLite backend).
  * **Gerrit and GitLab integration** Shows analysis results as [GitLab](gitlab_integration.md) or [Gerrit](jenkins_gerrit_integration.md) reviews.
  * **Filterable** (defect checker name, severity, source paths, ...) and
    **comparable** (calculates difference between two analyses of the project,
    showing which bugs have been fixed and which are newly introduced) result
    viewing.
  * **Diff mode:** This shows the list of bugs that have been introduced since your last analyzer
    execution.
  * Results can be shared with fellow developers, the **comments** and
    **review** system helps communication of code defects.
  * Easily implementable [Thrift](http://thrift.apache.org)-based
    server-client communication used for storing and querying of discovered
    defects.
  * Support for multiple bug visualization frontends, such as the web
    application, a [command-line tool](usage.md) and an
    [Eclipse plugin](http://github.com/Ericsson/CodeCheckerEclipsePlugin).

## Command line features
`CodeChecker` command has many subcommands which can be used for example to
log and analyze your projects, print the results or start a web server. For
full list see the following table or check the help message of this command
(`CodeChecker --help`):

| `CodeChecker` subcommand | Description                                                                                |
|--------------------------|-----------------------------------------------------------------------------------------------|
| `analyze`                | Execute the supported code analyzers for the files recorded in a JSON Compilation Database.   |
| `analyzer-version`       | Print the version of CodeChecker analyzer package that is being used.                         |
| `analyzers`              | List supported and available analyzers.                                                       |
| `check`                  | Perform analysis on a project and print results to standard output.                           |
| `checkers`               | List the checkers available for code analysis.                                                |
| `cmd`                    | View analysis results on a running server from the command line.                              |
| `fixit`                  | Apply automatic fixes based on the suggestions of the analyzers.                              |
| `log`                    | Run a build command, collect the executed compilation commands and store them in a JSON file. |
| `parse`                  | Print analysis summary and results in a human-readable format.                                |
| `server`                 | Start and manage the CodeChecker Web server.                                                  |
| `store`                  | Save analysis results to a database.                                                          |
| `version`                | Print the version of CodeChecker package that is being used.                                  |
| `web-version`            | Print the version of CodeChecker server package that is being used.                           |

`CodeChecker cmd` subcommand also has many other subcommands which can be used
to get data (products, runs, results, statistics) from a running CodeChecker
server. For full list see the following table or check the help message of this
subcommand (`CodeChecker cmd --help`):

| `CodeChecker cmd` subcommand | Description                                                                                |
|------------------------------|---------------------------------------------------------------------------------------------------|
| `runs`                       | List the available analysis runs.                                                                 |
| `history`                    | Show run history of multiple runs.                                                                |
| `results`                    | List analysis result (finding) summary for a given run.                                           |
| `diff`                       | Compare two analysis runs and show the difference.                                                |
| `sum`                        | Show statistics of checkers.                                                                      |
| `token`                      | Access subcommands related to configuring personal access tokens managed by a CodeChecker server. |
| `del`                        | Delete analysis runs.                                                                             |
| `update`                     | Update an analysis run.                                                                           |
| `suppress`                   | Manage and import suppressions of reports on a CodeChecker server.                                |
| `products`                   | Access subcommands related to configuring the products managed by a CodeChecker server.           |
| `components`                 | Access subcommands related to configuring the source components managed by a CodeChecker server.  |
| `login`                      | Authenticate into CodeChecker servers that require privileges.                                    |
| `export`                     | Export comments and review statuses from CodeChecker.                                             |
| `import`                     | Import comments and review statuses into CodeChecker.                                             |

# Usage flow
![Usage diagram](images/usage_flow.png)

- *Step 1*: `CodeChecker log` runs the given build command and records the
executed compilation steps. These steps are written to an output file
(Compilation Database) in a JSON format.
- *Step 2*: `CodeChecker analyze` uses the previously created JSON Compilation
Database to perform an analysis on the project, outputting analysis results in
a machine-readable (plist) format.
- *Step 3*: In this step, you can do multiple things:
    - Parse and pretty-print the summary and results from analysis result files
    (`CodeChecker parse`).
    - Store the results to a running CodeChecker server (`CodeChecker store`).
    - Compare two analysis results/runs to show the results that differ between
    the two (`CodeChecker cmd diff`).
    - etc.

For more information how to use CodeChecker see our [user guide](usage.md).

# User documentation

* [Getting started (How-To with examples)](usage.md)

## C/C++ Analysis
* [Analyzer User guide](analyzer/user_guide.md)
* [Avoiding or suppressing false positives](analyzer/false_positives.md)
* [Checker and Static Analyzer configuration](analyzer/checker_and_analyzer_configuration.md)
* [GCC incompatibilities](analyzer/gcc_incompatibilities.md)
* [Suppressing false positives](analyzer/user_guide.md#source-code-comments)

## Web based report management
* [Webserver User Guide](web/user_guide.md)
* [WEB GUI User Guide](/web/server/vue-cli/src/assets/userguide/userguide.md)
* [Command line and WEB UI Feature overview](feature_comparison.md)
* Security configuration
  * [Configuring Authentication](web/authentication.md)
  * [Configuring Authorization](web/permissions.md)
* Deployment
  * [Deploy server using docker](web/docker.md#deployment)
* Server Configuration
  * [Configuring Server Logging](logging.md)
  * [Setting up multiple CodeChecker repositories in one server](web/products.md)
* Continuous Integration (CI)
  * [CodeChecker as a GitHub Action](http://github.com/marketplace/actions/codechecker-static-analysis)
  * [Setting up CI gating with Gerrit and Jenkins](jenkins_gerrit_integration.md)
* Database Configuration
  * [PostgreSQL database backend setup guide](web/postgresql_setup.md)
  * [CodeChecker server and database schema upgrade guide](web/db_schema_guide.md)

### Storage of reports from analyzer tools
CodeChecker can be used as a generic tool for visualizing analyzer results.

The following tools are supported:

| Language       | Analyzer                                                                     |
|----------------|------------------------------------------------------------------------------|
| **C/C++**      | [Clang Static Analyzer](https://clang-analyzer.llvm.org/)                    |
|                | [Clang Tidy](https://clang.llvm.org/extra/clang-tidy/)                       |
|                | [Clang Sanitizers](supported_code_analyzers.md#clang-sanitizers)             |
|                | [Cppcheck](/docs/tools/report-converter.md#cppcheck)                         |
|                | [Facebook Infer](/docs/tools/report-converter.md#facebook-infer)             |
|                | [Coccinelle](/docs/tools/report-converter.md#coccinelle)                     |
|                | [Smatch](/docs/tools/report-converter.md#smatch)                             |
|                | [Kernel-Doc](/docs/tools/report-converter.md#kernel-doc)                     |
|                | [Sparse](/docs/tools/report-converter.md#sparse)                             |
|                | [cpplint](/docs/tools/report-converter.md#cpplint)                           |
| **C#**         | [Roslynator.DotNet.Cli](/docs/tools/report-converter.md#roslynatordotnetcli) |
| **Java**       | [SpotBugs](/docs/tools/report-converter.md#spotbugs)                         |
|                | [Facebook Infer](/docs/tools/report-converter.md#facebook-infer)                    |
| **Python**     | [Pylint](/docs/tools/report-converter.md#pylint)                             |
|                | [Pyflakes](/docs/tools/report-converter.md#pyflakes)                         |
| **JavaScript** | [ESLint](/docs/tools/report-converter.md#eslint)                             |
| **TypeScript** | [TSLint](/docs/tools/report-converter.md#tslint)                             |
| **Go**         | [Golint](/docs/tools/report-converter.md#golint)                             |
| **Markdown**   | [Markdownlint](/docs/tools/report-converter.md#markdownlint)                 |
|                | [Sphinx](/docs/tools/report-converter.md#sphinx)                             |

For details see
[supported code analyzers](supported_code_analyzers.md) documentation and the
[Report Converter Tool](/docs/tools/report-converter.md).

## Common Tools
Useful tools that can also be used outside CodeChecker.

* [Build Logger (to generate JSON Compilation Database from your builds)](/analyzer/tools/build-logger/README.md)
* [Plist/Sarif to HTML converter (to generate HTML files from the given plist or sarif files)](/docs/tools/report-converter.md#plist-to-html-tool)
* [Report Converter Tool (to convert analysis results from other analyzers to the codechecker report directory format))](/docs/tools/report-converter.md)
* [Translation Unit Collector (to collect source files of a translation unit or to get source files which depend on the given header files)](/docs/tools/tu_collector.md)
* [Report Hash generator (to generate unique hash identifiers for reports)](/docs/tools/report-converter.md#report-hash-generation-module)

## Helper Scripts
* [Helper Scripts for daily analysis](script_daily.md)

# Install guide

## Install CodeChecker via `pip`
CodeChecker is available on the [pypi](https://pypi.org/project/codechecker/)
and can be installed with the following command:
```sh
pip3 install codechecker
```

**Note:** this package can be installed on `Linux`, `OSX` and `Windows`
systems where `pip3` command is available. On `OSX`, `intercept-build` must be
installed for logging (`CodeChecker log`). On `Windows`, logging is not
available.

## Installing CodeChecker via the `snap` package manager
CodeChecker is available on the [Snap Store](https://snapcraft.io/codechecker)
and can be installed with the following command:
```sh
sudo snap install codechecker --classic
```

**Note:** Unfortunately, the snap package supports only lower-case command names.
For this reason, you need to use `codechecker` command instead of `CodeChecker`
everywhere. For a full list of available commands in the _codechecker_ snap
package, run `snap info codechecker`.

## Linux

For a detailed dependency list, and for instructions on how to install newer
Clang and Clang-Tidy versions, please see [Requirements](deps.md).
The following commands are used to bootstrap CodeChecker on Ubuntu 20.04 LTS:

```sh
# Install mandatory dependencies for a development and analysis environment.
# NOTE: clang or clang-tidy can be any sufficiently fresh version, and need not
#       come from package manager!
#       In case of Cppcheck, the minimal supported version is 1.80.
#       In case of gcc, the minimal supported version is 13.0.0.
sudo apt-get install clang clang-tidy cppcheck g++ build-essential curl
      gcc-multilib git python3-dev python3-venv python3-setuptools

# Install nodejs dependency for web. In case of Debian/Ubuntu you can use the
# following commands. For more information see the official docs:
# https://nodejs.org/en/download/package-manager/
curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs

# Check out CodeChecker source code.
git clone https://github.com/Ericsson/CodeChecker.git --depth 1 ~/codechecker
cd ~/codechecker

# Create a Python virtualenv and set it as your environment.
# NOTE: if you want to develop CodeChecker, use the `venv_dev` target instead
# of `venv`.
make venv
source $PWD/venv/bin/activate

# [Optional] If you want to use external authentication methods (LDAP / PAM)
# follow the instructions in
# docs/web/authentication.md#external-authentication-methods

# Build and install a CodeChecker package.
make package

# For ease of access, add the build directory to PATH.
export PATH="$PWD/build/CodeChecker/bin:$PATH"

cd ..
```

**Notes**:
- By default, `make package` will build ldlogger shared objects for
`32bit` and `64bit` too. If you would like to build and package `64 bit only`
shared objects and ldlogger binary you can set `BUILD_LOGGER_64_BIT_ONLY`
environment variable to `YES` before the package build:
`BUILD_LOGGER_64_BIT_ONLY=YES make package`.
- By default, the `make package` will build the UI code if it's not built yet
or the UI code is changed. If you wouldn't like to build the UI code you can
set the `BUILD_UI_DIST` environment variable to `NO` before the package build:
`BUILD_UI_DIST=NO make package`.
- Use `make standalone_package` instead of `make package` to avoid
having to manually activate the environment before running CodeChecker.

### Upgrading environment after system or Python upgrade

If you have upgraded your system's Python to a newer version (e.g., from
`3.8` to `3.11` – this is the case when upgrading Ubuntu from
20.04 LTS to 22.04 LTS), the installed environment will not work
out-of-the-box. To fix this issue, run the following command to upgrade your
`checker_env` too:

```sh
cd ~/codechecker/venv
python3 -m venv .
```

## Mac OS X
For installation instructions for Mac OS X see [Mac OS X Installation Guide](install_macosx.md) documentation.

## Docker
To run the CodeChecker server in Docker see the [Docker](web/docker.md) documentation.
You can find the CodeChecker web-server container at the [Docker Hub](https://hub.docker.com/r/codechecker/codechecker-web).

## Visual Studio Code plugin

You can install and use CodeChecker VSCode extension from the
[Visual Studio Marketplace](http://marketplace.visualstudio.com/items?itemName=codechecker.vscode-codechecker)
or from [Open VSX](http://open-vsx.org/extension/codechecker/codechecker).

Main features:
- Run CodeChecker analysis from the editor and see the results automatically.
- Re-analyze the current file when saved.
- Commands and build tasks for running CodeChecker as part of a build system.
- Browse through the found reports and show the reproduction steps directly in the code.
- Navigate between the reproduction steps.

![VSCode plugin](images/vscode.gif)

For more information how to install and use this plugin see the
[repository](https://github.com/Ericsson/codecheckervsCodePlugin/) of this
extension.

## GitHub Actions CI

![CodeChecker executed in GitHub Actions](images/github-actions.png)

CodeChecker can be executed via a reusable GitHub action for your project!
You need only specify the build command, as if you would run the analysis
locally.

For more information, check out the
[CodeChecker Static Analysis](http://github.com/marketplace/actions/codechecker-static-analysis)
action on the GitHub Actions Marketplace.

# Analyze your first project

## Setting up the environment in your Terminal

These steps must always be taken in a new command prompt you wish to execute
analysis in.

```sh
source ~/codechecker/venv/bin/activate

# Path of CodeChecker package
# NOTE: SKIP this line if you want to always specify CodeChecker's full path.
export PATH=~/codechecker/build/CodeChecker/bin:$PATH

# Path of the built LLVM/Clang
# NOTE: SKIP this line if clang is available in your PATH as an installed Linux package.
export PATH=~/<user path>/build/bin:$PATH
```
## Execute analysis

Analyze your project with the `check` command:

    CodeChecker check -b "cd ~/your-project && make clean && make" -o ./results

`check` will print an overview of the issues found in your project by the
analyzers. The reports will be stored in the `./results` directory in `plist`
XML format.

## Export the reports as static HTML files
You can visualize the results as static HTML by executing

`CodeChecker parse -e html ./results -o ./reports_html`

An index page will be generated with a list of all repors in
`./reports_html/index.html`

## Optionally store the results in Web server & view the results

If you have hundreds of results, you may want to store them on the web
server with a database backend.

Start a CodeChecker web and storage server in another terminal or as a
background process. By default, it will listen on `localhost:8001`.

The SQLite database containing the reports will be placed in your workspace
directory (`~/.codechecker` by default), which can be provided via the `-w`
flag.

    CodeChecker server

Store your analysis reports onto the server to be able to use the Web Viewer.

    CodeChecker store ./results -n my-project

Open the [CodeChecker Web Viewer](http://localhost:8001) in your browser, and
you should be greeted with a web application showing you the analysis results.

## Developer documentations
* [Architecture](architecture.md)
* [Package layout](package_layout.md)
* [Dependencies](deps.md)
* [Thrift interface](web/api/README.md)
* [Package and integration tests](tests.md)
* [Checker documentation mapping file](web/checker_docs.md)

## Conference papers, presentations
* An overview about the CodeChecker infrastructure was given at [PLDI 2020](http://pldi20.sigplan.org).

  **Márton, Gábor and Krupp, Dániel**:

  [_Tool Talk: CodeChecker_](http://youtube.com/watch?v=bVqrhaoxHlc)
* A high-level overview about the infrastructure is available amongst the
  [2015 Euro LLVM Conference](http://llvm.org/devmtg/2015-04) presentations.

  **Krupp, Dániel and Orbán, György and Horváth, Gábor and Babati, Bence**:

  [_Industrial Experiences with the Clang Static Analysis Toolset_](http://llvm.org/devmtg/2015-04/slides/Clang_static_analysis_toolset_final.pdf)


# Indico ![CI Status](https://github.com/indico/indico/workflows/CI/badge.svg) [![License](https://img.shields.io/github/license/indico/indico.svg)](https://github.com/indico/indico/blob/master/LICENSE) [![Available on PyPI](https://img.shields.io/pypi/v/indico.svg)](https://pypi.python.org/pypi/indico/) [![Made at CERN!](https://img.shields.io/badge/CERN-Open%20Source-%232980b9.svg)](https://home.cern)

**Indico** is:
 * 🗓 a general-purpose **event management** tool;
 * 🌍 fully **web-based**;
 * 🧩 **feature-rich** but also **extensible** through the use of [plugins](https://docs.getindico.io/en/stable/plugins/);
 * ⚖ **Open-Source** Software under the MIT License;
 *  **made at CERN**, [the place where the web was born](https://home.cern/science/computing/birth-web)!

![A sneak peek of Indico](https://raw.githubusercontent.com/indico/indico/master/sneakpeek.gif)

## What does it do?
Indico's **main features** are:
 * a powerful and flexible **hierarchical content management** system for events;
 * a full-blown **conference organization workflow** with:
   - 📢 **Call for Abstracts** and **abstract reviewing** modules;
   - 📝 flexible **registration form** creation and configuration;
   - 💰 integration with existing **payment systems**;
   - ✅ a **paper reviewing** workflow;
   - 🗓 a **drag and drop timetable management interface**;
   - 🎫 a simple **badge editor** with the possibility to print badges and tickets for participants;
 * tools for **meeting management** and archival of presentation materials;
 * a powerful **room booking** interface;
 * integration with existing **video conferencing solutions**;

A more detailed list [can be found here](https://getindico.io/features/). There is also a [video](https://www.youtube.com/watch?v=yo8rgg9dOcc)!

## I just want to try it out!

We've got a [Sandbox](https://sandbox.getindico.io)🧪!

## Browser support

These are the minimum versions of major browsers currently supported by Indico. We try to target all modern browsers as
much as possible, but only issues detected on those will be considered at critical level.

|
Firefox |
Chrome |
Safari |
Edge |
|:---------:|:---------:|:---------:|:---------:|
| 78+ | 83+ | 13+ | 17+ |

However, if you have an issue with a browser [on this list](https://browserl.ist/?q=and_chr%2084%2C%20and_ff%2068%2C%20chrome%2084%2C%20chrome%2083%2C%20chrome%2081%2C%20edge%2084%2C%20edge%2083%2C%20edge%2018%2C%20edge%2017%2C%20firefox%2079%2C%20firefox%2078%2C%20firefox%2077%2C%20firefox%2068%2C%20ios_saf%2013.4-13.5%2C%20ios_saf%2013.3%2C%20safari%2013.1%2C%20safari%2013%2C%20samsung%2011.1-11.2), please feel free to open a bug report.

## Getting Indico

Information on how to get the latest release can be found [at the project's web site](https://getindico.io/getting-started/).
There are installation guides for different systems available in [the project's documentation](https://docs.getindico.io/en/stable/installation/).

## Contributing

Indico is the result of the collective work of more than 100 different developers, translators and usability
specialists of many nationalities. You can be the next one - read our
[Contribution Guide](https://github.com/indico/indico/blob/master/CONTRIBUTING.md) if you'd like to help out.

**You don't need to know how to write code in order to help!**

## Roadmap

The full roadmap is [available on the project site](https://getindico.io/roadmap/).

## Community

The main meeting points for the community are:
 * the [Project Forum](https://talk.getindico.io);
 * the Chat Room
([#indico on Libera.Chat](https://web.libera.chat/gamja/?channels=#indico) or on [Matrix](https://app.element.io/#/room/#indico:matrix.org)).

We follow [CERN's Values](https://hr.web.cern.ch/cerns-values) and the principles established by
[CERN's Code of Conduct](https://hr.web.cern.ch/codeofconduct).

## History

This software project was initially funded by the European Union's FP5 programme🇪🇺, in what was called the [**In**tegrated **Di**gital **Co**nferencing Project](https://cordis.europa.eu/project/rcn/61849/factsheet/en), or just **InDiCo**. CERN was responsible for the development of the "Make-a-Confererence" workpackage (inspired by an already existing system called **CDS Agenda**, also developed at the Organization) which would then become what we nowadays know as **Indico**.

We have since stopped using the ~~*InDiCo*~~ acronym, as it no longer reflects accurately the nature of the project. The word *Indico* now has no particular meaning other than the product's name.

---

|||
|-|-|
||Made at [CERN](https://home.cern)
[Take part!](https://careers.cern/)|
|||

> ### Note
>
> *In applying the MIT license, CERN does not waive the privileges and immunities
> granted to it by virtue of its status as an Intergovernmental Organization
> or submit itself to any jurisdiction.*


========
llvmlite
========

.. image:: https://dev.azure.com/numba/numba/_apis/build/status/numba.llvmlite?branchName=main
   :target: https://dev.azure.com/numba/numba/_build/latest?definitionId=2&branchName=main
   :alt: Azure Pipelines
.. image:: https://codeclimate.com/github/numba/llvmlite/badges/gpa.svg
   :target: https://codeclimate.com/github/numba/llvmlite
   :alt: Code Climate
.. image:: https://coveralls.io/repos/github/numba/llvmlite/badge.svg
   :target: https://coveralls.io/github/numba/llvmlite
   :alt: Coveralls.io
.. image:: https://readthedocs.org/projects/llvmlite/badge/
   :target: https://llvmlite.readthedocs.io
   :alt: Readthedocs.io

A Lightweight LLVM Python Binding for Writing JIT Compilers
-----------------------------------------------------------

.. _llvmpy: https://github.com/llvmpy/llvmpy

llvmlite is a project originally tailored for Numba_'s needs, using the
following approach:

* A small C wrapper around the parts of the LLVM C++ API we need that are
  not already exposed by the LLVM C API.
* A ctypes Python wrapper around the C API.
* A pure Python implementation of the subset of the LLVM IR builder that we
  need for Numba.

Why llvmlite
============

The old llvmpy_  binding exposes a lot of LLVM APIs but the mapping of
C++-style memory management to Python is error prone. Numba_ and many JIT
compilers do not need a full LLVM API.  Only the IR builder, optimizer,
and JIT compiler APIs are necessary.

Key Benefits
============

* The IR builder is pure Python code and decoupled from LLVM's
  frequently-changing C++ APIs.
* Materializing a LLVM module calls LLVM's IR parser which provides
  better error messages than step-by-step IR building through the C++
  API (no more segfaults or process aborts).
* Most of llvmlite uses the LLVM C API which is small but very stable
  (low maintenance when changing LLVM version).
* The binding is not a Python C-extension, but a plain DLL accessed using
  ctypes (no need to wrestle with Python's compiler requirements and C++ 11
  compatibility).
* The Python binding layer has sane memory management.
* llvmlite is faster than llvmpy thanks to a much simpler architecture
  (the Numba_ test suite is twice faster than it was).

Compatibility
=============

llvmlite has been tested with Python 3.9 -- 3.12 and is likely to work with
greater versions.

As of version 0.41.0, llvmlite requires LLVM 14.x.x on all architectures

Historical compatibility table:

=================  ========================
llvmlite versions  compatible LLVM versions
=================  ========================
0.41.0 - ...       14.x.x
0.40.0 - 0.40.1    11.x.x and 14.x.x (12.x.x and 13.x.x untested but may work)
0.37.0 - 0.39.1    11.x.x
0.34.0 - 0.36.0    10.0.x (9.0.x for  ``aarch64`` only)
0.33.0             9.0.x
0.29.0 - 0.32.0    7.0.x, 7.1.x, 8.0.x
0.27.0 - 0.28.0    7.0.x
0.23.0 - 0.26.0    6.0.x
0.21.0 - 0.22.0    5.0.x
0.17.0 - 0.20.0    4.0.x
0.16.0 - 0.17.0    3.9.x
0.13.0 - 0.15.0    3.8.x
0.9.0 - 0.12.1     3.7.x
0.6.0 - 0.8.0      3.6.x
0.1.0 - 0.5.1      3.5.x
=================  ========================

Documentation
=============

You'll find the documentation at http://llvmlite.pydata.org

Pre-built binaries
==================

We recommend you use the binaries provided by the Numba_ team for
the Conda_ package manager.  You can find them in Numba's `anaconda.org
channel <https://anaconda.org/numba>`_.  For example::

   $ conda install --channel=numba llvmlite

(or, simply, the official llvmlite package provided in the Anaconda_
distribution)

.. _Numba: http://numba.pydata.org/
.. _Conda: http://conda.pydata.org/
.. _Anaconda: http://docs.continuum.io/anaconda/index.html

Other build methods
===================

If you don't want to use our pre-built packages, you can compile
and install llvmlite yourself.  The documentation will teach you how:
http://llvmlite.pydata.org/en/latest/install/index.html


LSP

Like an IDE, except it's the good parts. Learn more.

## Installation

### Stable Version

Open the command palette and run `Package Control: Install Package`, then select `LSP`.

### Development Version

Clone this repository into your Packages directory. Open the command palette and run `Package Control: Satisfy Dependencies`.

## Getting started

Follow the installation steps for a [specific language server](https://lsp.sublimetext.io/language_servers).

Open a document supported by the language server. LSP should report the language server starting in the status bar.

See more information in the [documentation](https://lsp.sublimetext.io) :open_book:.

## Getting help

If you have any problems, see the [troubleshooting](https://sublimelsp.github.io/LSP/troubleshooting/) guide for tips and known limitations. If the documentation cannot solve your problem, you can look for help in:

* The [#lsp](https://discordapp.com/channels/280102180189634562/645268178397560865) channel (join the [SublimeHQ Discord](https://discord.gg/TZ5WN8t) first!)
* By [searching or creating a new issue](https://github.com/sublimelsp/LSP/issues)


Papermerge DMS

Papermerge DMS or simply Papermerge is a open source document management
system designed to work with scanned documents (also called digital
archives). It extracts text from your scans using OCR, indexes them, and
prepares them for full text search. Papermerge provides look and feel of
modern desktop file browsers. It has features like dual panel document
browser, drag and drop, tags, hierarchical folders and full text search so
that you can efficiently store and organize your documents.

It supports PDF, TIFF, JPEG and PNG document file formats. Papermerge is
perfect tool for long term storage of your documents.

Papermerge's main use case is **long term storage of digital archives**.

This is web-based software. This means there is no executable file (aka no
.exe files), and it must be run on a web server and accessed through a web
browser.

![Papermerge](./img/papermerge3.png)

## Repositories

**This is meta-repository** - which means that source code of the
application is not here. **This repository is used to track project's existence,
status and its issues.**

As the application grew it was necessary to split it
into multiple repositories and in same time move new repositories under
[Papermerge Github Organization](https://github.com/papermerge).

| Repository      | Description |
| :---------------|-------------|
| [ciur/papermerge](https://github.com/ciur/papermerge)| Meta-repository which keeps track the project existence, status, and its issues.|
| [papermerge/papermerge-core](https://github.com/papermerge/papermerge-core)| Source code for REST API Backend server. The heart of the project.|
| [papermerge/documentation](https://github.com/papermerge/documentation)| Source code for the documentation.|
| [papermerge/ansible](https://github.com/papermerge/ansible)| Ansible playbook for deployment on remote server/VM|

## Other Resources

| Resource        | Description |
|-----------------|-------------|
|[https://docs.papermerge.io](https://docs.papermerge.io/)| Online Documentation |
|[https://papermerge.com](https://papermerge.com) | Homepage |
|[https://papermerge.blog](https://papermerge.blog) | Blog |
|[YouTube Channel](https://www.youtube.com/channel/UC8KjEsDexEERBw_-VyDbWDg) | YouTube channel |
|[X/Former Twitter](https://twitter.com/papermerge) | X/Former Twitter |
|[Reddit](https://www.reddit.com/r/Papermerge/) | Reddit |

## Features Highlight

* Documents of PDF, JPG, PNG, TIFF formats are supported
* Desktop like user interface
* **OCR** - used to extract text for documents indexing
* Full text search
* Document Versioning (all operations on the documents are non destructive)
* Tags - assign colored tags to documents or folders
* Documents and Folders - users can organize documents in folders
* Multi-User
* REST API

See last section for details on feature set in Papermerge 2.0, 2.1, 3.0

## Donations, Fundraising, Your Support

For donations, you can use PayPal and GitHub Sponsorship:

* [Donate via Paypal](https://www.paypal.com/paypalme/eugenciur)
* [Sponsor via Github](https://github.com/sponsors/ciur)

## Contributing

We welcome contributions! In general, if change is very small, like fixing a
documentation typo, remove unused variable or minor adjustments of docker
related files - you can create a pull request right away. If your change is
small and reasonable it will be (very likely) almost immediately accepted.

For bigger changes, like a new feature or even change/add/remove of
whole paragraph in documentation - please **first discuss the
change** you wish to make via GitHub issue, pull request or [email](mailto:eugen@papermerge.com).

For more information, see the
[contributing](https://github.com/ciur/papermerge/blob/master/CONTRIBUTING.md)
file.

## Versions 2.0 / 2.1 / 3.0

Papermerge version 2.0, version 2.1 and version 3.0 are entirely different and
incompatible applications.

Here is a table of features comparisons of all three major versions:

| Feature      | 2.0 | 2.1 | 3.0 |
|-----------------|-------------|----------------|----------------------|
|PDF format| :heavy_check_mark: |:heavy_check_mark:|:heavy_check_mark: |
|png format| :heavy_check_mark: |:x:| :heavy_check_mark:|
|jpeg format| :heavy_check_mark: |:x:| :heavy_check_mark:|
|tiff format| :heavy_check_mark: |:x:| :heavy_check_mark:|
|Desktop like UI|:heavy_check_mark:|:heavy_check_mark:| :heavy_check_mark:|
|Dual Panel|:x:|:heavy_check_mark:|:heavy_check_mark:|
|Realtime OCR feedback|:x:|:heavy_check_mark:|:heavy_check_mark:|
|OCR|:heavy_check_mark:|:heavy_check_mark:| :heavy_check_mark:|
|Download document with OCRed text layer|:x:|:heavy_check_mark:| :heavy_check_mark:|
|Full Text Search|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|Document Versioning|:x:|:heavy_check_mark:|:heavy_check_mark:|
|User defined metadata per folder/document/page|:heavy_check_mark:|:x:|:x:|
|Automates|:heavy_check_mark:|:x:|:x:|
|Tags - assign colored tags to documents or folders|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|Documents and Folders|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|Multi-User|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|User permissions management|:heavy_check_mark:|:heavy_check_mark:|:x:|
|Document permissions management|:heavy_check_mark:|:x:|:x:|
|REST API|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|Page Delete/Reorder/Move|:heavy_check_mark:|:heavy_check_mark:|:heavy_check_mark:|
|Page Rotation|:x:|:heavy_check_mark:|:heavy_check_mark:|
|Documents Merging|:x:|:heavy_check_mark:|:heavy_check_mark:|
|Cloud Native/K8s Support|:x:|:heavy_check_mark:|:heavy_check_mark:|


terraform-compliance

`terraform-compliance` is a lightweight, security and compliance focused test framework against terraform to enable negative testing capability for your infrastructure-as-code.

- __compliance:__ Ensure the implemented code is following security standards, your own custom standards
- __behaviour driven development:__ We have BDD for nearly everything, why not for IaC ?
- __portable:__ just install it from `pip` or run it via `docker`. See [Installation](https://terraform-compliance.com/pages/installation/)
- __pre-deploy:__ it validates your code before it is deployed
- __easy to integrate:__ it can run in your pipeline (or in git hooks) to ensure all deployments are validated.
- __segregation of duty:__ you can keep your tests in a different repository where a separate team is responsible.
- __why ?:__ why not ?

## Performance

If terraform-compliance is not running quickly enough make sure to check the
optional faster\_parsing pip install flag in the [Installation Guide](https://terraform-compliance.com/pages/installation/)

## Idea

`terraform-compliance` mainly focuses on [negative testing](https://en.wikipedia.org/wiki/Negative_testing) instead
of having fully-fledged [functional tests](https://en.wikipedia.org/wiki/Functional_testing) that are mostly used for
proving a component of code is performing properly.

Fortunately, `terraform` is a marvellous abstraction layer for any API
that __creates__/__updates__/__destroys__ entities. `terraform` also provides the
[capability](https://www.terraform.io/docs/commands/plan.html#detailed-exitcode)
to ensure everything is up-to-date between the local configuration and the remote API(s) responses.

Given the fact, `terraform` is used mostly against Cloud APIs, what was missing is to ensure
your code against your infrastructure must follow specific policies. Currently HashiCorp provides
[Sentinel](https://www.hashicorp.com/sentinel/) for Enterprise Products. `terraform-compliance` is providing a
similar functionality only for `terraform` while it is free-to-use and it is Open Source.

E.g. a sample policy could be, if you are working with `AWS`, you should not create an `S3 bucket`,
without having any `encryption`. Of course, this is just an example which may or not be applicable
for your case.

`terraform-compliance` provides a test framework to create these policies that will be executed against
your [terraform plan](https://www.terraform.io/docs/commands/plan.html) in a context where both
developers and security teams can understand easily while reading it, by applying [Behaviour Driven
Development](https://en.wikipedia.org/wiki/Behavior-driven_development) Principles.

As returning back to the example, our example defined above will be translated into a BDD Feature
and Scenario, as also seen in below ;

```
if you are working with AWS, you should not create an S3 bucket, without having any encryption
```

translates into ;

```gherkin
Given I have AWS S3 Bucket defined
Then it must contain server_side_encryption_configuration
```

`server_side_encryption_configuration` is coming from the terraform code, as shown below ;

```
resource "aws_s3_bucket" "b" {
  bucket = "my-bucket"
  acl    = "private"

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        kms_master_key_id = "${aws_kms_key.mykey.arn}"
        sse_algorithm     = "aws:kms"
      }
    }
  }
}
```

This policy ( Scenario ) will allow all S3 buckets newly created or updated must have encryption configuration set within the code. In an ideal way, this Scenario (among with all other Scenarios) will run on a CI/CD pipeline that will ensure that nothing is deployed by violating your policies.

See [Examples](https://terraform-compliance.com/pages/Examples/) for more sample use cases.

Regarding the feature file format - `radish` is used to parse files with extension `.feature` - https://radish.readthedocs.io/en/stable/tutorial.html

![Example Run](https://github.com/eerkunt/terraform-compliance/blob/master/terraform-compliance-demo.gif?raw=true)

## Sponsors

* [resmo.com](resmo.com): Discover unmatched insights for Cloud and SaaS assets. Use SQL to ask questions and get real-time notifications for security and compliance violations.

## License
[MIT](https://tldrlegal.com/license/mit-license)


![bcbio banner](https://raw.githubusercontent.com/bcbio/bcbio-nextgen/master/docs/contents/images/banner.png)

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis. You write a high level configuration file specifying your inputs and analysis parameters. This input drives a parallel run that handles distributed execution, idempotent processing restarts and safe transactional steps. bcbio provides a shared community resource that handles the data processing component of sequencing analysis, providing researchers with more time to focus on the downstream biology.

[![Build Status](https://travis-ci.org/bcbio/bcbio-nextgen.svg?branch=master)](https://travis-ci.org/bcbio/bcbio-nextgen)
[![Documentation status](https://readthedocs.org/projects/bcbio-nextgen/badge/?version=latest)](https://bcbio-nextgen.readthedocs.io/en/latest/?badge=latest)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3564938.svg)](https://doi.org/10.5281/zenodo.3564938)

## Features

* Community developed: We welcome contributors with the goal of overcoming the biological, algorithmic and computational challenges that face individual developers working on complex pipelines in quickly changing research areas. See our [users page](https://bcbio-nextgen.readthedocs.io/en/latest/contents/users.html) for examples of bcbio-nextgen deployments, and the [developer documentation](https://bcbio-nextgen.readthedocs.io/en/latest/contents/development.html) for tips on contributing.
* Installation: [A single installer script](https://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#automated) prepares all third party software, data libraries and system configuration files.
* [Automated validation](https://bcb.io/2014/05/12/wgs-trio-variant-evaluation/): Compare variant calls against common reference materials or sample specific SNP arrays to ensure call correctness. Incorporation of multiple approaches for alignment, preparation and variant calling enable unbiased comparisons of algorithms.
* Distributed: Focus on [parallel analysis and scaling](https://bcb.io/2013/05/22/scaling-variant-detection-pipelines-for-whole-genome-sequencing-analysis/) to handle large population studies and whole genome analysis. Runs on single multicore computers, in compute clusters using [IPython parallel](https://ipyparallel.readthedocs.io/en/latest/), or on the Amazon cloud. See the [parallel documentation](https://bcbio-nextgen.readthedocs.org/en/latest/contents/parallel.html) for full details.
* Multiple analysis algorithms: bcbio-nextgen provides configurable [variant calling (small and copy number), RNA-seq, ATAC-seq, , BS-Seq, SC RNA-seq, and small RNA pipelines](https://bcbio-nextgen.readthedocs.io/en/latest/).

## Quick start

1. [Install](https://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#automated) `bcbio-nextgen` with all tool dependencies and data files:
    ```shell script
    wget https://raw.githubusercontent.com/bcbio/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py
    python bcbio_nextgen_install.py /usr/local/share/bcbio --tooldir=/usr/local \
          --genomes hg38 --aligners bwa --aligners bowtie2
    ```
   producing an editable [system configuration file](https://github.com/bcbio/bcbio-nextgen/blob/master/config/bcbio_system.yaml) referencing the installed software, data and system information.

1. [Automatically create a processing description](https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#automated-sample-configuration) of sample FASTQ and BAM files from your project, and a CSV file of sample metadata:
    ```shell script
    bcbio_nextgen.py -w template freebayes-variant project1.csv sample1.bam sample2_1.fq sample2_2.fq
    ```
    This produces a [sample description file](https://github.com/bcbio/bcbio-nextgen/blob/master/config/bcbio_sample.yaml) containing pipeline [configuration options](https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html).

1.  Run analysis, distributed across 8 local cores:
    ```shell script
    cd project1/work
    bcbio_nextgen.py ../config/project1.yaml -n 8
    ```

## Documentation

See the [full documentation](https://bcbio-nextgen.readthedocs.io/en/latest/) and [longer analysis-based articles](https://bcb.io). We welcome enhancements or problem reports using [GitHub](https://github.com/bcbio/bcbio-nextgen/issues) and discussion on the [biovalidation mailing list](https://groups.google.com/d/forum/biovalidation).

## Contributors

* [Miika Ahdesmaki](https://github.com/mjafin), AstraZeneca
* [Luca Beltrame](https://github.com/lbeltrame), IRCCS "Mario Negri" Institute for Pharmacological Research, Milan, Italy
* [Christian Brueffer](https://github.com/cbrueffer), Lund University, Lund, Sweden
* [Alla Bushoy](https://github.com/abushoy), AstraZeneca
* [Guillermo Carrasco](https://github.com/guillermo-carrasco), Science for Life Laboratory, Stockholm
* [Nick Carriero](https://www.simonsfoundation.org/team/nick-carriero/), Simons Foundation
* [Brad Chapman](https://github.com/chapmanb), Harvard Chan Bioinformatics Core
* [Saket Choudhary](https://github.com/saketkc), University Of Southern California
* [Peter Cock](https://github.com/peterjc), The James Hutton Institute
* [Matthias De Smet](https://github.com/matthdsm), Center for Medical Genetics, Ghent University Hospital, Belgium
* [Matt Edwards](https://github.com/matted), MIT
* [Mario Giovacchini](https://github.com/mariogiov), Science for Life Laboratory, Stockholm
* [Karl Gutwin](https://twitter.com/kgutwin), Biogen
* [Jeff Hammerbacher](https://github.com/hammer), Icahn School of Medicine at Mount Sinai
* [Oliver Hofmann](https://umccr.github.io/), University of Melbourne Centre for Cancer Research
* [John Kern](https://github.com/kern3020)
* [Rory Kirchner](https://github.com/roryk), Harvard Chan Bioinformatics Core
* [Tetiana Khotiainsteva](https://github.com/tetianakh), Ardigen
* [Kerrin Mendler](https://github.com/kmendler), AstraZeneca
* [Sergey Naumenko](https://github.com/naumenko-sa), Harvard Chan Bioinformatics Core
* [Jakub Nowacki](https://github.com/jsnowacki), AstraZeneca
* [John Morrissey](https://github.com/jwm), Harvard Chan Bioinformatics Core
* [Lorena Pantano](https://github.com/lpantano), Harvard Chan Bioinformatics Core
* [Brent Pedersen](https://github.com/brentp), University of Colorado Denver
* [James Porter](https://github.com/porterjamesj), The University of Chicago
* [Vlad Saveliev](https://github.com/vladsaveliev), Center for Algorithmic Biotechnology, St. Petersburg University
* [Valentine Svensson](https://github.com/vals), Science for Life Laboratory, Stockholm
* [Paul Tang](https://github.com/tanglingfung), UCSF
* [Stephen Turner](https://github.com/stephenturner), University of Virginia
* [Roman Valls](https://github.com/brainstorm), Science for Life Laboratory, Stockholm
* [Kevin Ying](https://github.com/kevyin), Garvan Institute of Medical Research, Sydney, Australia
* [Steffen Möller](https://github.com/smoe), University of Rostock, Germany
* [WimSpee](https://github.com/wimspee)

## License

The code is freely available under the [MIT license](https://opensource.org/licenses/mit-license.html).


# Kitsune

![Status Sustain](https://img.shields.io/badge/Status-Sustain-green)

Kitsune is the platform that powers [SuMo (support.mozilla.org)](https://support.mozilla.org)

## Usage

It is a [Django](http://www.djangoproject.com/) application. There is [documentation](https://mozilla.github.io/kitsune/) online.

## Releasing a new version

-   Create a [signed tag](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) for the new version.

    We are using [semantic versioning](https://semver.org/).

    Given a version number MAJOR.MINOR.PATCH, increment the:

        MAJOR version when you make incompatible API changes
        MINOR version when you add functionality in a backward compatible manner
        PATCH version when you make backward compatible bug fixes

    Example:

    `git tag -s 1.0.1 -m "Bump version: 1.0.0 to 1.0.1`

-   Draft a new release in GitHub for the new tag. Document the **highlights** of the release and also use the option to automatically document the release through the commit history.

-   Trigger the release for the specified tag in the deploy repository.

> [!TIP]
> You can access the staging site at <https://support.allizom.org/>.

## Code of Conduct

By participating in this project, you're agreeing to uphold the [Mozilla Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/). If you need to report a problem, please see our [Code of Conduct](./CODE_OF_CONDUCT.md) guide.

## Contribute

See our [contribution guide](https://mozilla.github.io/kitsune/contributors), or dive into [setting up your development environment](https://mozilla.github.io/kitsune/hacking_howto/).

## Issues

We use [Bugzilla](https://bugzilla.mozilla.org/enter_bug.cgi?product=support.mozilla.org) for submitting and prioritizing issues.

## Thanks to all of our contributors ❤