# `fabric`

![Static Badge](https://img.shields.io/badge/mission-human_flourishing_via_AI_augmentation-purple)

![GitHub top language](https://img.shields.io/github/languages/top/danielmiessler/fabric)
![GitHub last commit](https://img.shields.io/github/last-commit/danielmiessler/fabric)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)

fabric is an open-source framework for augmenting humans using AI.

[Introduction Video](#introduction-video-by-network-chuck) •
[What and Why](#what-and-why) •
[Philosophy](#philosophy) •
[Quickstart](#quickstart) •
[Structure](#structure) •
[Examples](#examples) •
[Custom Patterns](#custom-patterns) •
[Helper Apps](#helper-apps) •
[Examples](#examples) •
[Meta](#meta)

## Navigation

- [Introduction Videos](#introduction-video-by-network-chuck)
- [What and Why](#what-and-why)
- [Philosophy](#philosophy)
  - [Breaking problems into components](#breaking-problems-into-components)
  - [Too many prompts](#too-many-prompts)
  - [The Fabric approach to prompting](#our-approach-to-prompting)
- [Quickstart](#quickstart)
  - [Setting up the fabric commands](#setting-up-the-fabric-commands)
  - [Using the fabric client](#using-the-fabric-client)
  - [Just use the Patterns](#just-use-the-patterns)
  - [Create your own Fabric Mill](#create-your-own-fabric-mill)
- [Structure](#structure)
  - [Components](#components)
  - [CLI-native](#cli-native)
  - [Directly calling Patterns](#directly-calling-patterns)
- [Examples](#examples)
- [Custom Patterns](#custom-patterns)
- [Helper Apps](#helper-apps)
- [Meta](#meta)
  - [Primary contributors](#primary-contributors)

> [!NOTE]
> May 23, 2024 — We will be switching Fabric to Go in a few weeks to avoid all the installation issues with Python. The Go version will be dead-simple to install and will be even faster. Plus easier to update. We already have it working thanks to the heroic efforts of @xssdoctor, and we're just working on testing now! Stay tuned for more info on the release date!

## Introduction video by Network Chuck!

This is a **brilliant** video by Network Chuck that goes over why he's started using Fabric for all things AI. He talks about the spirit of the project, how to install it, and how he uses it, and he just generally articulates the spirit of what we're doing here SO WELL. Thanks to Chuck for this!

## What and why

Since the start of 2023 and GenAI we've seen a massive number of AI applications for accomplishing tasks. It's powerful, but _it's not easy to integrate this functionality into our lives._

In other words, AI doesn't have a capabilities problem—it has an integration problem.

Fabric was created to address this by enabling everyone to granularly apply AI to everyday challenges.

## Philosophy

> AI isn't a thing; it's a _magnifier_ of a thing. And that thing is **human creativity**.

We believe the purpose of technology is to help humans flourish, so when we talk about AI we start with the **human** problems we want to solve.

### Breaking problems into components

Our approach is to break problems into individual pieces (see below) and then apply AI to them one at a time. See below for some examples.

### Too many prompts

Prompts are good for this, but the biggest challenge I faced in 2023——which still exists today—is **the sheer number of AI prompts out there**. We all have prompts that are useful, but it's hard to discover new ones, know if they are good or not, _and manage different versions of the ones we like_.

One of fabric's primary features is helping people collect and integrate prompts, which we call _Patterns_, into various parts of their lives.

Fabric has Patterns for all sorts of life and work activities, including:

- Extracting the most interesting parts of YouTube videos and podcasts.
- Writing an essay in your own voice with just an idea as an input.
- Summarizing opaque academic papers.
- Creating perfectly matched AI art prompts for a piece of writing.
- Rating the quality of content to see if you want to read/watch the whole thing.
- Getting summaries of long, boring content.
- Explaining code to you.
- Turning bad documentation into usable documentation.
- Creating social media posts from any content input.
- And a million more…

### Our approach to prompting

Fabric _Patterns_ are different than most prompts you'll see.

- **First, we use `Markdown` to help ensure maximum readability and editability**. This not only helps the creator make a good one, but also anyone who wants to deeply understand what it does. _Importantly, this also includes the AI you're sending it to!_

Here's an example of a Fabric Pattern

```bash
https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/system.md
```

- **Next, we are extremely clear in our instructions**, and we use the Markdown structure to emphasize what we want the AI to do, and in what order.

- **And finally, we tend to use the System section of the prompt almost exclusively**. In over a year of being heads-down with this stuff, we've just seen more efficacy from doing that. If that changes, or we're shown data that says otherwise, we will adjust.

## Quickstart

The most feature-rich way to use Fabric is to use the `fabric` client, which can be found under `/client` directory in this repository.

### Required Python Version
Ensure you have at least python3.10 installed on your operating system. Otherwise, when you attempt to run the pip install commands, the project will fail to build due to certain dependencies.

### Setting up the fabric commands

Follow these steps to get all fabric-related apps installed and configured.

1. Navigate to where you want the Fabric project to live on your system in a semi-permanent place on your computer.

```bash
# Find a home for Fabric
cd /where/you/keep/code
```

2. Clone the project to your computer.

```bash
# Clone Fabric to your computer
git clone https://github.com/danielmiessler/fabric.git
```

3. Enter Fabric's main directory.

```bash
# Enter the project folder (where you cloned it)
cd fabric
```

4. Install pipx:

macOS:

```bash
brew install pipx
```

Linux:

```bash
sudo apt install pipx
```

Windows:

Use WSL and follow the Linux instructions.

5. Install fabric:

```bash
pipx install .
```

6. Run setup:

```bash
fabric --setup
```

7. Restart your shell to reload everything.

8. Now you are up and running! You can test by running the help.

```bash
# Making sure the paths are set up correctly
fabric --help
```

> [!NOTE]
> If you're using the `server` functions, `fabric-api` and `fabric-webui` need to be run in distinct terminal windows.

### Using the `fabric` client

If you want to use it with OpenAI API-compatible inference servers, such as [FastChat](https://github.com/lm-sys/FastChat), [Helmholtz Blablador](http://helmholtz-blablador.fz-juelich.de), [LM Studio](https://lmstudio.ai) and others, simply export the following environment variables:

- `export OPENAI_BASE_URL=https://YOUR-SERVER:8000/v1/`
- `export DEFAULT_MODEL="YOUR_MODEL"`

And if your server needs authentication tokens, as Blablador does, you export the token the same way you would with OpenAI:

- `export OPENAI_API_KEY="YOUR TOKEN"`

Once you have it all set up, here's how to use it:

1. Check out the options
   `fabric -h`

```bash
usage: fabric -h
usage: fabric [-h] [--text TEXT] [--copy] [--agents] [--output [OUTPUT]] [--session [SESSION]] [--gui] [--stream] [--list] [--temp TEMP] [--top_p TOP_P] [--frequency_penalty FREQUENCY_PENALTY]
              [--presence_penalty PRESENCE_PENALTY] [--update] [--pattern PATTERN] [--setup] [--changeDefaultModel CHANGEDEFAULTMODEL] [--model MODEL] [--listmodels]
              [--remoteOllamaServer REMOTEOLLAMASERVER] [--context]

An open-source framework for augmenting humans using AI.

options:
  -h, --help            show this help message and exit
  --text TEXT, -t TEXT  Text to extract summary from
  --copy, -C            Copy the response to the clipboard
  --agents, -a          Use praisonAI to create an AI agent and then use it. ex: 'write me a movie script'
  --output [OUTPUT], -o [OUTPUT]
                        Save the response to a file
  --session [SESSION], -S [SESSION]
                        Continue your previous conversation. Default is your previous conversation
  --gui                 Use the GUI (Node and npm need to be installed)
  --stream, -s          Use this option if you want to see the results in realtime. NOTE: You will not be able to pipe the output into another command.
  --list, -l            List available patterns
  --temp TEMP           sets the temperature for the model. Default is 0
  --top_p TOP_P         set the top_p for the model. Default is 1
  --frequency_penalty FREQUENCY_PENALTY
                        sets the frequency penalty for the model. Default is 0.1
  --presence_penalty PRESENCE_PENALTY
                        sets the presence penalty for the model. Default is 0.1
  --update, -u          Update patterns. NOTE: This will revert the default model to gpt4-turbo. please run --changeDefaultModel to once again set the default model
  --pattern PATTERN, -p PATTERN
                        The pattern (prompt) to use
  --setup               Set up your fabric instance
  --changeDefaultModel CHANGEDEFAULTMODEL
                        Change the default model. For a list of available models, use the --listmodels flag.
  --model MODEL, -m MODEL
                        Select the model to use
  --listmodels          List all available models
  --remoteOllamaServer REMOTEOLLAMASERVER
                        The URL of the remote ollamaserver to use. ONLY USE THIS if you are using a local ollama server in a non-default location or port
  --context, -c         Use Context file (context.md) to add context to your pattern
```

#### Example commands

The client, by default, runs Fabric patterns without needing a server (the Patterns were downloaded during setup). This means the client connects directly to OpenAI using the input given and the Fabric pattern used.

1. Run the `summarize` Pattern based on input from `stdin`. In this case, the body of an article.

```bash
pbpaste | fabric --pattern summarize
```

2. Run the `analyze_claims` Pattern with the `--stream` option to get immediate and streaming results.

```bash
pbpaste | fabric --stream --pattern analyze_claims
```

3. Run the `extract_wisdom` Pattern with the `--stream` option to get immediate and streaming results from any Youtube video (much like in the original introduction video).

```bash
yt --transcript https://youtube.com/watch?v=uXs-zPc63kM | fabric --stream --pattern extract_wisdom
```

4. **new** All of the patterns have been added as aliases to your bash (or zsh) config file

```bash
pbpaste | analyze_claims --stream
```

> [!NOTE]
> More examples coming in the next few days, including a demo video!

### Just use the Patterns

If you're not looking to do anything fancy, and you just want a lot of great prompts, you can navigate to the [`/patterns`](https://github.com/danielmiessler/fabric/tree/main/patterns) directory and start exploring!

We hope that if you used nothing else from Fabric, the Patterns by themselves will make the project useful.

You can use any of the Patterns you see there in any AI application that you have, whether that's ChatGPT or some other app or website. Our plan and prediction is that people will soon be sharing many more than those we've published, and they will be way better than ours.

The wisdom of crowds for the win.

### Create your own Fabric Mill

But we go beyond just providing Patterns. We provide code for you to build your very own Fabric server and personal AI infrastructure!

## Structure

Fabric is themed off of, well… _fabric_—as in…woven materials. So, think blankets, quilts, patterns, etc. Here's the concept and structure:

### Components

The Fabric ecosystem has three primary components, all named within this textile theme.

- The **Mill** is the (optional) server that makes **Patterns** available.
- **Patterns** are the actual granular AI use cases (prompts).
- **Stitches** are chained together _Patterns_ that create advanced functionality (see below).
- **Looms** are the client-side apps that call a specific **Pattern** hosted by a **Mill**.

### CLI-native

One of the coolest parts of the project is that it's **command-line native**!

Each Pattern you see in the `/patterns` directory can be used in any AI application you use, but you can also set up your own server using the `/server` code and then call APIs directly!

Once you're set-up, you can do things like:

```bash
# Take any idea from `stdin` and send it to the `/write_essay` API!
echo "An idea that coding is like speaking with rules." | write_essay
```

### Directly calling Patterns

One key feature of `fabric` and its Markdown-based format is the ability to _directly reference_ (and edit) individual [Patterns](#components) directly—on their own—without any surrounding code.

As an example, here's how to call _the direct location_ of the `extract_wisdom` pattern.

```bash
https://github.com/danielmiessler/fabric/blob/main/patterns/extract_wisdom/system.md
```

This means you can cleanly, and directly reference any pattern for use in a web-based AI app, your own code, or wherever!

Even better, you can also have your [Mill](#components) functionality directly call _system_ and _user_ prompts from `fabric`, meaning you can have your personal AI ecosystem automatically kept up to date with the latest version of your favorite [Patterns](#components).

Here's what that looks like in code:

```bash
https://github.com/danielmiessler/fabric/blob/main/server/fabric_api_server.py
```

```python
# /extwis
@app.route("/extwis", methods=["POST"])
@auth_required  # Require authentication
def extwis():
    data = request.get_json()

    # Warn if there's no input
    if "input" not in data:
        return jsonify({"error": "Missing input parameter"}), 400

    # Get data from client
    input_data = data["input"]

    # Set the system and user URLs
    system_url = "https://raw.githubusercontent.com/danielmiessler/fabric/main/patterns/extract_wisdom/system.md"
    user_url = "https://raw.githubusercontent.com/danielmiessler/fabric/main/patterns/extract_wisdom/user.md"

    # Fetch the prompt content
    system_content = fetch_content_from_url(system_url)
    user_file_content = fetch_content_from_url(user_url)

    # Build the API call
    system_message = {"role": "system", "content": system_content}
    user_message = {"role": "user", "content": user_file_content + "\n" + input_data}
    messages = [system_message, user_message]
    try:
        response = openai.chat.completions.create(
            model="gpt-4-1106-preview",
            messages=messages,
            temperature=0.0,
            top_p=1,
            frequency_penalty=0.1,
            presence_penalty=0.1,
        )
        assistant_message = response.choices[0].message.content
        return jsonify({"response": assistant_message})
    except Exception as e:
        return jsonify({"error": str(e)}), 500
```

## Examples

Here's an abridged output example from the `extract_wisdom` pattern (limited to only 10 items per section).

```bash
# Paste in the transcript of a YouTube video of Riva Tez on David Perrel's podcast
pbpaste | extract_wisdom
```

```markdown
## SUMMARY:

The content features a conversation between two individuals discussing various topics, including the decline of Western culture, the importance of beauty and subtlety in life, the impact of technology and AI, the resonance of Rilke's poetry, the value of deep reading and revisiting texts, the captivating nature of Ayn Rand's writing, the role of philosophy in understanding the world, and the influence of drugs on society. They also touch upon creativity, attention spans, and the importance of introspection.

## IDEAS:

1. Western culture is perceived to be declining due to a loss of values and an embrace of mediocrity.
2. Mass media and technology have contributed to shorter attention spans and a need for constant stimulation.
3. Rilke's poetry resonates due to its focus on beauty and ecstasy in everyday objects.
4. Subtlety is often overlooked in modern society due to sensory overload.
5. The role of technology in shaping music and performance art is significant.
6. Reading habits have shifted from deep, repetitive reading to consuming large quantities of new material.
7. Revisiting influential books as one ages can lead to new insights based on accumulated wisdom and experiences.
8. Fiction can vividly illustrate philosophical concepts through characters and narratives.
9. Many influential thinkers have backgrounds in philosophy, highlighting its importance in shaping reasoning skills.
10. Philosophy is seen as a bridge between theology and science, asking questions that both fields seek to answer.

## QUOTES:

1. "You can't necessarily think yourself into the answers. You have to create space for the answers to come to you."
2. "The West is dying and we are killing her."
3. "The American Dream has been replaced by mass-packaged mediocrity porn, encouraging us to revel like happy pigs in our own meekness."
4. "There's just not that many people who have the courage to reach beyond consensus and go explore new ideas."
5. "I'll start watching Netflix when I've read the whole of human history."
6. "Rilke saw beauty in everything... He sees it's in one little thing, a representation of all things that are beautiful."
7. "Vanilla is a very subtle flavor... it speaks to sort of the sensory overload of the modern age."
8. "When you memorize chapters [of the Bible], it takes a few months, but you really understand how things are structured."
9. "As you get older, if there's books that moved you when you were younger, it's worth going back and rereading them."
10. "She [Ayn Rand] took complicated philosophy and embodied it in a way that anybody could resonate with."

## HABITS:

1. Avoiding mainstream media consumption for deeper engagement with historical texts and personal research.
2. Regularly revisiting influential books from youth to gain new insights with age.
3. Engaging in deep reading practices rather than skimming or speed-reading material.
4. Memorizing entire chapters or passages from significant texts for better understanding.
5. Disengaging from social media and fast-paced news cycles for more focused thought processes.
6. Walking long distances as a form of meditation and reflection.
7. Creating space for thoughts to solidify through introspection and stillness.
8. Embracing emotions such as grief or anger fully rather than suppressing them.
9. Seeking out varied experiences across different careers and lifestyles.
10. Prioritizing curiosity-driven research without specific goals or constraints.

## FACTS:

1. The West is perceived as declining due to cultural shifts away from traditional values.
2. Attention spans have shortened due to technological advancements and media consumption habits.
3. Rilke's poetry emphasizes finding beauty in everyday objects through detailed observation.
4. Modern society often overlooks subtlety due to sensory overload from various stimuli.
5. Reading habits have evolved from deep engagement with texts to consuming large quantities quickly.
6. Revisiting influential books can lead to new insights based on accumulated life experiences.
7. Fiction can effectively illustrate philosophical concepts through character development and narrative arcs.
8. Philosophy plays a significant role in shaping reasoning skills and understanding complex ideas.
9. Creativity may be stifled by cultural nihilism and protectionist attitudes within society.
10. Short-term thinking undermines efforts to create lasting works of beauty or significance.

## REFERENCES:

1. Rainer Maria Rilke's poetry
2. Netflix
3. Underworld concert
4. Katy Perry's theatrical performances
5. Taylor Swift's performances
6. Bible study
7. Atlas Shrugged by Ayn Rand
8. Robert Pirsig's writings
9. Bertrand Russell's definition of philosophy
10. Nietzsche's walks
```

## Custom Patterns

You can also use Custom Patterns with Fabric, meaning Patterns you keep locally and don't upload to Fabric.

One possible place to store them is `~/.config/custom-fabric-patterns`.

Then when you want to use them, simply copy them into `~/.config/fabric/patterns`.

```bash
cp -a ~/.config/custom-fabric-patterns/* ~/.config/fabric/patterns/
```

Now you can run them with:

```bash
pbpaste | fabric -p your_custom_pattern
```

## Agents

NEW FEATURE! We have incorporated [PraisonAI](https://github.com/MervinPraison/PraisonAI) into Fabric. This feature creates AI agents and then uses them to perform a task.

```bash
echo "Search for recent articles about the future of AI and write me a 500-word essay on the findings" | fabric --agents
```

This feature works with all OpenAI and Ollama models but does NOT work with Claude. You can specify your model with the -m flag.

For more information about this amazing project, please visit https://github.com/MervinPraison/PraisonAI.

## Helper Apps

These are helper tools to work with Fabric. Examples include things like getting transcripts from media files, getting metadata about media, etc.

## yt (YouTube)

`yt` is a command that uses the YouTube API to pull transcripts, pull user comments, get video duration, and other functions. It's primary function is to get a transcript from a video that can then be stitched (piped) into other Fabric Patterns.

```bash
usage: yt [-h] [--duration] [--transcript] [url]

vm (video meta) extracts metadata about a video, such as the transcript and the video's duration. By Daniel Miessler.

positional arguments:
  url           YouTube video URL

options:
  -h, --help    Show this help message and exit
  --duration    Output only the duration
  --transcript  Output only the transcript
  --comments    Output only the user comments
```

## ts (Audio transcriptions)

'ts' is a command that uses the OpenAI Whisper API to transcribe audio files. Due to the context window, this tool uses pydub to split the files into 10 minute segments. for more information on pydub, please refer https://github.com/jiaaro/pydub

### Installation

```bash

mac:
brew install ffmpeg

linux:
apt install ffmpeg

windows:
download instructions https://www.ffmpeg.org/download.html
```

```bash
ts -h
usage: ts [-h] audio_file

Transcribe an audio file.

positional arguments:
  audio_file  The path to the audio file to be transcribed.

options:
  -h, --help  show this help message and exit
```

## Save

`save` is a "tee-like" utility to pipeline saving of content, while keeping the output stream intact. Can optionally generate "frontmatter" for PKM utilities like Obsidian via the
"FABRIC_FRONTMATTER" environment variable

If you'd like to default variables, set them in `~/.config/fabric/.env`. `FABRIC_OUTPUT_PATH` needs to be set so `save` where to write. `FABRIC_FRONTMATTER_TAGS` is optional, but useful for tracking how tags have entered your PKM, if that's important to you.

### usage

```bash
usage: save [-h] [-t, TAG] [-n] [-s] [stub]

save: a "tee-like" utility to pipeline saving of content, while keeping the output stream intact. Can optionally generate "frontmatter" for PKM utilities like Obsidian via the
"FABRIC_FRONTMATTER" environment variable

positional arguments:
  stub                stub to describe your content. Use quotes if you have spaces. Resulting format is YYYY-MM-DD-stub.md by default

options:
  -h, --help          show this help message and exit
  -t, TAG, --tag TAG  add an additional frontmatter tag. Use this argument multiple timesfor multiple tags
  -n, --nofabric      don't use the fabric tags, only use tags from --tag
  -s, --silent        don't use STDOUT for output, only save to the file
```

### Example

```bash
echo test | save --tag extra-tag stub-for-name
test

$ cat ~/obsidian/Fabric/2024-03-02-stub-for-name.md
---
generation_date: 2024-03-02 10:43
tags: fabric-extraction stub-for-name extra-tag
---
test
```

## Meta

> [!NOTE]
> Special thanks to the following people for their inspiration and contributions!

- _Caleb Sima_ for pushing me over the edge of whether to make this a public project or not.
- _Joel Parish_ for super useful input on the project's Github directory structure.
- _Jonathan Dunn_ for spectacular work on the soon-to-be-released universal client.
- _Joseph Thacker_ for the idea of a `-c` context flag that adds pre-created context in the `./config/fabric/` directory to all Pattern queries.
- _Jason Haddix_ for the idea of a stitch (chained Pattern) to filter content using a local model before sending on to a cloud model, i.e., cleaning customer data using `llama2` before sending on to `gpt-4` for analysis.
- _Dani Goland_ for enhancing the Fabric Server (Mill) infrastructure by migrating to FastAPI, breaking the server into discrete pieces, and Dockerizing the entire thing.
- _Andre Guerra_ for simplifying installation by getting us onto Poetry for virtual environment and dependency management.

### Primary contributors

`fabric` was created by Daniel Miessler in January of 2024.

![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/danielmiessler)


|logo|

``eventyay-talk`` is a comprehensive conference planning tool designed to deliver an exceptional experience for organizers, speakers, reviewers, and attendees. It efficiently manages the submission process with a customizable Call for Participation, facilitates the review and selection of submissions, and handles scheduling and release logistics. Post-event, ``eventyay-talk`` enables speakers to receive feedback, upload their slides, and allows organizers to embed recordings.

For a detailed overview of its capabilities, check out our feature list on our main site. ``eventyay-talk`` typically covers all the essential tasks required to curate submissions and content for a multi-track, multi-day conference.

You can host eventyay-talk yourself, or use our public instance at `eventyay.com`.

|screenshots|

``eventyay-talk`` is highly customizable, allowing you to modify its appearance and behavior to suit your event's needs. If the default settings don't meet your requirements, you can also create your own plugins for additional functionality.

Project status
-----------------

`eventyay-talk` is under active development and used by many events. It supports everything required for talk submission, speaker communication, and scheduling. The project is based on pretalx.

Contributing
---------------

Please check out our issues and start contributing.

License
------------
We publish `eventyay-talk` under the terms of the Apache 2 License. See the LICENSE file for further information and the complete license text

.. |logo| image:: assets/logo.svg
   :alt: eventyay-talk logo
   :target: https://eventyay.com
.. |screenshots| image:: assets/screenshots.png
   :target: https://eventyay.com
   :alt: Screenshots of eventyay-talk
.. _issues: https://github.com/fossasia/eventyay-talk/issues/


# Arista Validated Designs

![Arista AVD](https://img.shields.io/badge/Arista-AVD%20Automation-blue) ![collection version](https://img.shields.io/github/v/release/aristanetworks/avd) ![License](https://img.shields.io/github/license/aristanetworks/avd)

Arista Validated Designs (AVD) is an extensible data model that defines Arista's Unified Cloud Network architecture as "code".

AVD Documentation:

- [Stable version](https://avd.arista.com/stable/)
- [Development version](https://avd.arista.com/devel/)

## Features

- **Flexibility with Open Data Models:** Extensible fabric-wide network models, simplifying configuration, delivering consistency, and reducing errors
- **Simplification through Multi-Domain Automation:** A framework that can automate the data center, campus or wide area network, enabled by a consistent EOS software image and management platform
- **Comprehensive Workflows:** Automating the full life cycle of network provisioning from config generation to pre- and post-deployment validation, and self-documentation of the network

## Reference designs

- [L3LS VXLAN-EVPN, L2LS, and MPLS](https://avd.arista.com/stable/roles/eos_designs/index.html)

## AVD Ansible Collection

[Arista Networks](https://www.arista.com/) supports Ansible for managing devices running Arista's **Extensible Operating System (EOS)** natively through it's **EOS API (eAPI)** or [**CloudVision Portal (CVP)**](https://www.arista.com/en/products/eos/eos-cloudvision). The collection includes a set of Ansible roles and modules to help kick-start your automation with Arista. The various roles and templates provided are designed to be customized and extended to your needs.

### Examples

- [Getting started](https://avd.arista.com/stable/docs/getting-started/intro-to-ansible-and-avd.html)
- [Examples](https://avd.arista.com/stable/examples/single-dc-l3ls/index.html)

## Additional resources

- Ansible [EOS modules](https://docs.ansible.com/ansible/latest/collections/arista/eos/index.html) on Ansible documentation
- Ansible [CloudVision modules](https://cvp.avd.sh/en/stable/)
- [CloudVision Portal](https://www.arista.com/en/products/eos/eos-cloudvision)
- [Arista Design and Deployment Guides](https://www.arista.com/en/solutions/design-guides)

## Support

- AVD version 4.x releases with full support from Arista TAC. If your organization has the [A-Care subscription](https://www.arista.com/assets/data/pdf/AVD-A-Care-TAC-Support-Overview.pdf) please don't hesitate to contact TAC with any questions or issues.
- Community support is provided via [Github discussions board](https://github.com/aristanetworks/avd/discussions).

## Contributing

Contributing pull requests are gladly welcomed for this repository. If you are planning a big change, please start a discussion first to make sure we'll be able to merge it. Please see the [contribution guide](ansible_collections/arista/avd/docs/contribution/overview.md) for additional details.

You can also open an [issue](https://github.com/aristanetworks/avd/issues) to report any problems or submit enhancements.

## License

Copyright (c) 2019-2024 Arista Networks, Inc.

The project is published under [Apache 2.0 License](LICENSE)


    <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://github.com/atopile/atopile/assets/9785003/00f19584-18a2-4b5f-9ce4-1248798974dd">
    <source media="(prefers-color-scheme: light)" src="https://github.com/atopile/atopile/assets/9785003/d38941c1-d7c1-42e6-9b94-a62a0996bc19">

    </picture>

## 📖 What Is `atopile`?
`atopile` is a tool to build electronic circuit boards with code.

## 🗣 Join Us On Discord
What's your story in electronics? What would you like us to build? Come talk on discord.

[![Discord Banner 3](https://discordapp.com/api/guilds/1022538123915300865/widget.png?style=banner2)](https://discord.gg/nr5V3QRUd3)

## ⚡`ato` Code Examples

### A simple voltage divider
```python
from "generics/resistors.ato" import Resistor
from "generics/interfaces.ato" import Power, Pair

module VDiv: #this name needs to match the name in the ato.yaml config file
    power = new Power
    output = new Pair

    r_top = new Resistor
    r_top.package = "0402"

    r_bottom = new Resistor
    r_bottom.package = "0402"

    power.vcc ~ r_top.p1; r_top.p2 ~ output.io
    output.io ~ r_bottom.p1; r_bottom.p2 ~ power.gnd; power.gnd ~ output.gnd

    v_in: voltage
    v_out: voltage
    i_q: current

    assert v_in * r_bottom.value / (r_top.value + r_bottom.value) within v_out
    assert v_in / (r_bottom.value + r_top.value) within i_q

    v_in = 3.3V +/- 2%
    v_out = 1.8V +/- 5%
    i_q = 1mA +/- 10%
```

### The classic "Blinky" circuit

Define your design with **ato code**
```python
import RP2040Kit from "rp2040/RP2040Kit.ato" # run `ato install rp2040` to install
import LEDIndicatorRed from "generics/leds.ato"
import LV2842Kit from "lv2842xlvddcr/lv2842kit.ato" # run `ato install lv2842xlvddcr` to install
import USBCConn from "usb-connectors/usb-connectors.ato" # run `ato install usb-connectors` to install

module Blinky:
    micro_controller = new RP2040Kit
    led_indicator = new LEDIndicatorRed
    voltage_regulator = new LV2842Kit
    usb_c_connector = new USBCConn

    usb_c_connector.power ~ voltage_regulator.power_in
    voltage_regulator.power_out ~ micro_controller.power
    micro_controller.gpio13 ~ led_indicator.input
    micro_controller.power.gnd ~ led_indicator.gnd

    led_indicator.v_in = 3.3volt +/-10%
```
Generate a **block diagram** from code

    <picture>

    </picture>

Produce **schematics** for documentation

    <picture>

    </picture>

### Discover Full Projects

Checkout out the [servo drive project](https://github.com/atopile/spin-servo-drive) or the [swoop motion controller](https://github.com/atopile/swoop).

## 🔨 Getting Started

Find our [documentation](https://atopile.io/getting-started/), [installation video](https://www.youtube.com/watch?v=XqFhFs-FhQ0) and getting started [video](https://www.youtube.com/watch?v=7aeZLlA_VYA).

`atopile` is on pypi.org: https://pypi.org/project/atopile/

### Most Basic Installation

`atopile` requires *python3.11* or later, which you can install using your package manager or from [python.org](https://www.python.org/downloads/).

Then just `pipx install atopile` and you're good to go!

## ❓ Why Atopile?

The objective of atopile is to help push forward these paradigms from the software world to hardware, mainly these points:

* **Intelligent Design Capture**: Define hardware specifications like ratios and tolerances in code, enabling precise control and easy reuse of designs.
* **Version Control Integration**: Use git to manage design changes, facilitating collaboration and ensuring each iteration is thoroughly reviewed and validated.
* **Continuous Integration (CI)**: Implement CI to guarantee high-quality, compliant designs with every commit, represented by a green checkmark for assurance.

Describing hardware with code might seem odd at first glance. But once you realize it introduces software development paradigms and toolchains to hardware design, you'll be hooked, just like we've become.

Code can **capture the intelligence** you put into your work. Imagine configuring not the resistance values of a voltage divider, but its ratio and total resistance, all using **physical units** and **tolerances**. You can do this because someone before you described precisely what this module is and described the relationships between the values of the components and the function you care about. Now instead imagine what you can gain from **reusing** a buck design you can merely **configure** the target voltage and ripple of. Now imagine **installing** a [servo drive](https://github.com/atopile/spin-servo-drive) the same way you might numpy.

Version controlling your designs using **git** means you can deeply **validate** and **review** changes a feature at a time, **isolated** from impacting others' work. It means you can detangle your organisation and **collaborate** on an unprecedented scale. We can forgo half-baked "releases" in favor of stamping a simple git-hash on our prototypes, providing an anchor off which to **associate test data** and expectations.

Implementing CI to **test** our work ensures both **high-quality** and **compliance**, all summarised in a green check mark, emboldening teams to target excellence.

## 🔍 Discover what people build

Browse and submit your modules at [packages.atopile.io](https://packages.atopile.io)


eventyay-tickets
======

.. image:: https://codecov.io/gh/fossasia/eventyay-tickets/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/pretix/pretix

Project status & release cycle
------------------------------

Welcome to the eventyay project. The ticketing component of the system provides options for ticket sales and event related items such a Tshirts. Eventyay is developed since 2014. The ticketing component is based on a fork of pretix.

Support
-------

This project is free and open source software. Professional support is available to customers of the hosted eventyay service or eventyay enterprise offerings. If you are interested in commercial support, hosting services or supporting this project financially, please go to `eventyay.com`.

Contributing
------------
Please look through our issues and start contributing.

License
-------
The code in this repository is published under the terms of the Apache 2 License.
See the LICENSE file for the complete license text.

This project is maintained by FOSSASIA. See the AUTHORS file for a list of all the awesome contributors of this project.

.. _installation guide: https://docs.pretix.eu/en/latest/admin/installation/index.html
.. _eventyay.com: https://eventyay.com
.. _blog: https://blog.eventyay.com


eventyay-video
=========
eventyay-video allows you to run events fully online. You can make use of different modules including live video streams, interactive video calls, group chats and more.

License
-------

eventyay-video is released under the terms of the Apache 2 License See the ``LICENSE`` file for the full license text as well as the license parameters.

Credits
^^^^^^^

`Twemoji <https://twemoji.twitter.com/>`_ emoji graphics are licensed under `Creative Commons Attribution 4.0 International <https://creativecommons.org/licenses/by/4.0/>`_ (CC-BY-4.0) by Twitter, Inc. and other contributors.


## Envoy toolshed

Python libraries, runners and checkers for Envoy proxy's CI

[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/envoyproxy/toolshed/badge)](https://securityscorecards.dev/viewer/?uri=github.com/envoyproxy/toolshed)

### Packages

#### [abstracts](abstracts)

version: 0.0.13.dev0

pypi: https://pypi.org/project/abstracts

---

#### [aio.api.bazel](aio.api.bazel)

version: 0.0.3.dev0

pypi: https://pypi.org/project/aio.api.bazel

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3

---

#### [aio.api.github](aio.api.github)

version: 0.2.6.dev0

pypi: https://pypi.org/project/aio.api.github

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aiohttp](https://pypi.org/project/aiohttp) >=3.8.1
- [gidgethub](https://pypi.org/project/gidgethub)
- [multidict](https://pypi.org/project/multidict) >=6.0.2
- [packaging](https://pypi.org/project/packaging) >=23.0
- [yarl](https://pypi.org/project/yarl) >=1.7.2

---

#### [aio.api.nist](aio.api.nist)

version: 0.0.5.dev0

pypi: https://pypi.org/project/aio.api.nist

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aiohttp](https://pypi.org/project/aiohttp) >=3.8.1
- [multidict](https://pypi.org/project/multidict) >=6.0.2
- [packaging](https://pypi.org/project/packaging)
- [yarl](https://pypi.org/project/yarl) >=1.7.2

---

#### [aio.core](aio.core)

version: 0.10.3.dev0

pypi: https://pypi.org/project/aio.core

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [pytz](https://pypi.org/project/pytz)
- [pyyaml](https://pypi.org/project/pyyaml)
- [trycast](https://pypi.org/project/trycast) >=0.7.3

---

#### [aio.run.checker](aio.run.checker)

version: 0.5.8.dev0

pypi: https://pypi.org/project/aio.run.checker

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3

---

#### [aio.run.runner](aio.run.runner)

version: 0.3.4.dev0

pypi: https://pypi.org/project/aio.run.runner

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [coloredlogs](https://pypi.org/project/coloredlogs)
- [frozendict](https://pypi.org/project/frozendict)
- [uvloop](https://pypi.org/project/uvloop)
- [verboselogs](https://pypi.org/project/verboselogs)

---

#### [dependatool](dependatool)

version: 0.2.4.dev0

pypi: https://pypi.org/project/dependatool

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.checker](https://pypi.org/project/aio.run.checker) >=0.5.7

---

#### [envoy.base.utils](envoy.base.utils)

version: 0.5.2.dev0

pypi: https://pypi.org/project/envoy.base.utils

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.api.github](https://pypi.org/project/aio.api.github) >=0.2.5
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [aiohttp](https://pypi.org/project/aiohttp) >=3.8.1
- [frozendict](https://pypi.org/project/frozendict)
- [jinja2](https://pypi.org/project/jinja2)
- [multidict](https://pypi.org/project/multidict) >=6.0.2
- [orjson](https://pypi.org/project/orjson)
- [packaging](https://pypi.org/project/packaging) >=23.0
- [protobuf](https://pypi.org/project/protobuf)
- [pytz](https://pypi.org/project/pytz)
- [pyyaml](https://pypi.org/project/pyyaml)
- [trycast](https://pypi.org/project/trycast) >=0.7.3
- [yarl](https://pypi.org/project/yarl) >=1.7.2
- [zstandard](https://pypi.org/project/zstandard)

---

#### [envoy.code.check](envoy.code.check)

version: 0.5.13

pypi: https://pypi.org/project/envoy.code.check

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.checker](https://pypi.org/project/aio.run.checker) >=0.5.7
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.4.2
- [flake8](https://pypi.org/project/flake8) >=6
- [packaging](https://pypi.org/project/packaging) >=23.0
- [pep8-naming](https://pypi.org/project/pep8-naming)
- [yamllint](https://pypi.org/project/yamllint)
- [yapf](https://pypi.org/project/yapf)

---

#### [envoy.dependency.check](envoy.dependency.check)

version: 0.1.13.dev0

pypi: https://pypi.org/project/envoy.dependency.check

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.api.github](https://pypi.org/project/aio.api.github) >=0.1.6
- [aio.api.nist](https://pypi.org/project/aio.api.nist) >=0.0.4
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.checker](https://pypi.org/project/aio.run.checker) >=0.5.7
- [aiohttp](https://pypi.org/project/aiohttp) >=3.8.1
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.10
- [gidgethub](https://pypi.org/project/gidgethub)
- [jinja2](https://pypi.org/project/jinja2)
- [multidict](https://pypi.org/project/multidict) >=6.0.2
- [packaging](https://pypi.org/project/packaging)
- [yarl](https://pypi.org/project/yarl) >=1.7.2

---

#### [envoy.distribution.distrotest](envoy.distribution.distrotest)

version: 0.0.11.dev0

pypi: https://pypi.org/project/envoy.distribution.distrotest

##### requirements:

- [aio.run.checker](https://pypi.org/project/aio.run.checker) >=0.5.7
- [aiodocker](https://pypi.org/project/aiodocker)
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.9
- [envoy.docker.utils](https://pypi.org/project/envoy.docker.utils) >=0.0.2

---

#### [envoy.distribution.release](envoy.distribution.release)

version: 0.0.10.dev0

pypi: https://pypi.org/project/envoy.distribution.release

##### requirements:

- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [envoy.github.abstract](https://pypi.org/project/envoy.github.abstract) >=0.0.22
- [envoy.github.release](https://pypi.org/project/envoy.github.release) >=0.0.14

---

#### [envoy.distribution.repo](envoy.distribution.repo)

version: 0.0.9.dev0

pypi: https://pypi.org/project/envoy.distribution.repo

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.9
- [envoy.github.abstract](https://pypi.org/project/envoy.github.abstract) >=0.0.22
- [envoy.github.release](https://pypi.org/project/envoy.github.release) >=0.0.14

---

#### [envoy.distribution.verify](envoy.distribution.verify)

version: 0.0.12.dev0

pypi: https://pypi.org/project/envoy.distribution.verify

##### requirements:

- [aio.run.checker](https://pypi.org/project/aio.run.checker) >=0.5.7
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.9
- [envoy.distribution.distrotest](https://pypi.org/project/envoy.distribution.distrotest) >=0.0.9

---

#### [envoy.docker.utils](envoy.docker.utils)

version: 0.0.3.dev0

pypi: https://pypi.org/project/envoy.docker.utils

##### requirements:

- [aiodocker](https://pypi.org/project/aiodocker)

---

#### [envoy.docs.sphinx_runner](envoy.docs.sphinx_runner)

version: 0.2.12.dev0

pypi: https://pypi.org/project/envoy.docs.sphinx_runner

##### requirements:

- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [colorama](https://pypi.org/project/colorama)
- [docutils](https://pypi.org/project/docutils) ~=0.19.0
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.10
- [envoy.code.check](https://pypi.org/project/envoy.code.check) >=0.3.6
- [packaging](https://pypi.org/project/packaging) >=23.0
- [pygments](https://pypi.org/project/pygments) >=2.11.1
- [pyyaml](https://pypi.org/project/pyyaml)
- [sphinx-copybutton](https://pypi.org/project/sphinx-copybutton)
- [sphinx-rtd-theme](https://pypi.org/project/sphinx-rtd-theme) >=2.0.0rc2
- [sphinx](https://pypi.org/project/sphinx) >=7.2.2
- [sphinxcontrib-applehelp](https://pypi.org/project/sphinxcontrib-applehelp)
- [sphinxcontrib-devhelp](https://pypi.org/project/sphinxcontrib-devhelp)
- [sphinxcontrib-htmlhelp](https://pypi.org/project/sphinxcontrib-htmlhelp)
- [sphinxcontrib-httpdomain](https://pypi.org/project/sphinxcontrib-httpdomain)
- [sphinxcontrib-jquery](https://pypi.org/project/sphinxcontrib-jquery) >=3.0.0
- [sphinxcontrib-qthelp](https://pypi.org/project/sphinxcontrib-qthelp)
- [sphinxcontrib-serializinghtml](https://pypi.org/project/sphinxcontrib-serializinghtml)
- [sphinxext-rediraffe](https://pypi.org/project/sphinxext-rediraffe)

---

#### [envoy.github.abstract](envoy.github.abstract)

version: 0.0.23.dev0

pypi: https://pypi.org/project/envoy.github.abstract

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.2.1
- [aiohttp](https://pypi.org/project/aiohttp)
- [gidgethub](https://pypi.org/project/gidgethub)
- [verboselogs](https://pypi.org/project/verboselogs)

---

#### [envoy.github.release](envoy.github.release)

version: 0.0.16.dev0

pypi: https://pypi.org/project/envoy.github.release

##### requirements:

- [abstracts](https://pypi.org/project/abstracts) >=0.0.12
- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [aiofiles](https://pypi.org/project/aiofiles)
- [aiohttp](https://pypi.org/project/aiohttp)
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.3.9
- [envoy.github.abstract](https://pypi.org/project/envoy.github.abstract) >=0.0.22
- [gidgethub](https://pypi.org/project/gidgethub)
- [packaging](https://pypi.org/project/packaging) >=23.0
- [verboselogs](https://pypi.org/project/verboselogs)

---

#### [envoy.gpg.identity](envoy.gpg.identity)

version: 0.1.2.dev0

pypi: https://pypi.org/project/envoy.gpg.identity

##### requirements:

- [aio.core](https://pypi.org/project/aio.core) >=0.10.1
- [python-gnupg](https://pypi.org/project/python-gnupg)

---

#### [envoy.gpg.sign](envoy.gpg.sign)

version: 0.2.1.dev0

pypi: https://pypi.org/project/envoy.gpg.sign

##### requirements:

- [aio.run.runner](https://pypi.org/project/aio.run.runner) >=0.3.3
- [envoy.base.utils](https://pypi.org/project/envoy.base.utils) >=0.4.7
- [envoy.gpg.identity](https://pypi.org/project/envoy.gpg.identity) >=0.1.1

---

#### [mypy-abstracts](mypy-abstracts)

version: 0.0.7.dev0

pypi: https://pypi.org/project/mypy-abstracts

##### requirements:

- [mypy](https://pypi.org/project/mypy)

---

#### [pytest-abstracts](pytest-abstracts)

version: 0.0.5.dev0

pypi: https://pypi.org/project/pytest-abstracts

##### requirements:

- [abstracts](https://pypi.org/project/abstracts)
- [pytest](https://pypi.org/project/pytest) >=3.5.0

---

#### [pytest-iters](pytest-iters)

version: 0.0.4.dev0

pypi: https://pypi.org/project/pytest-iters

##### requirements:

- [pytest](https://pypi.org/project/pytest) >=3.5.0

---

#### [pytest-patches](pytest-patches)

version: 0.0.4.dev0

pypi: https://pypi.org/project/pytest-patches

##### requirements:

- [pytest](https://pypi.org/project/pytest) >=3.5.0

---


[![Documentation Status](https://readthedocs.org/projects/varfish-server/badge/?version=latest)](https://varfish-server.readthedocs.io/en/latest/?badge=latest)
[![Code Coverage](https://codecov.io/gh/varfish-org/varfish-server/branch/main/graph/badge.svg?token=5ZACSH5MZZ)](https://codecov.io/gh/varfish-org/varfish-server)
[![image](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)

# VarFish

**Comprehensive DNA variant analysis for diagnostics and research.**

This is the repository for the web server component.

Holtgrewe, M.; Stolpe, O.; Nieminen, M.; Mundlos, S.; Knaus, A.; Kornak, U.; Seelow, D.; Segebrecht, L.; Spielmann, M.; Fischer-Zirnsak, B.; Boschann, F.; Scholl, U.; Ehmke, N.; Beule, D. *VarFish: Comprehensive DNA Variant Analysis for Diagnostics and Research*. Nucleic Acids Research 2020, gkaa241. <https://doi.org/10.1093/nar/gkaa241>.

## Getting Started

- [VarFish Homepage](https://www.cubi.bihealth.org/software/varfish/)
- [Manual](https://varfish-server.readthedocs.io/en/latest/)
  - [Installation Instructions](https://varfish-server.readthedocs.io/en/latest/admin_install.html).
- [Docker Compose Installer](https://github.com/varfish-org/varfish-docker-compose#run-varfish-server-using-docker-compose).

## VarFish Repositories

- [varfish-server](https://github.com/varfish-org/varfish-server) The VarFish Server is the web frontend used by the end users / data analysts.
- [varfish-annotator](https://github.com/varfish-org/varfish-annotator) The VarFish Annotator is a command line utility used for annotating VCF files and converting them to files that can be imported into VarFish Server.
- [varfish-cli](https://github.com/varfish-org/varfish-cli) The VarFish Command Line Interface allows to import data through the VarFish REST API.
- [varfish-db-downloader](https://github.com/varfish-org/varfish-db-downloader) The VarFish DB Downloader is a command line tool for downloading the background database.
- [varfish-docker-compose](https://github.com/varfish-org/varfish-docker-compose) Quickly get started running a VarFish server by using Docker Compose. We provide a prebuilt data set with some already imported data.

## At a Glance

- License: MIT
- Dependencies / Tech Stack
  - Python \>=3.8
  - Django 3
  - PostgreSQL \>=12

GitHub is used for public issue tracking. Currently, development happens
on internal infrastructure.

## VarFish Component Compatibility Table

The following combinations have been validated / are supported to work.

| VarFish Server | VarFish CLI | VarFish Annotator |
| -------------- | ----------- | ----------------- |
| v1.2.2         | v0.3.0      | v0.21             |
| v1.2.1         | v0.3.0      | v0.21             |
| v1.2.0         | v0.3.0      | v0.21             |

## VarFish Data Release Compatibility Table

The following combinations have been validated / are supported to work.

| VarFish Server | Data Release | VarFish DB Downloader |
| -------------- | ------------ | --------------------- |
| v1.2.2         | 20210728c    | v0.3.\*               |
| v1.2.1         | 20210728     | v0.3.\*               |
| v1.2.1         | 20210728b    | v0.3.\*               |
| v1.2.0         | 20210728     | v0.3.\*               |
| v1.2.0         | 20210728b    | v0.3.\*               |


# WA Cyber Security Unit (Office of Digital Government)

[![OpenSSF Scorecard](https://img.shields.io/ossf-scorecard/github.com/wagov/wasocshared.svg?label=openssf%20scorecard)](https://securityscorecards.dev/viewer/?uri=github.com/wagov/wasocshared)

This site contains technical information to support WA Government Cyber Security activities. Please propose updates directly via the edit link on each page or email [cybersecurity@dpc.wa.gov.au](mailto:cybersecurity@dpc.wa.gov.au) with any feedback. The site is built with [Material for MkDocs (reference)](https://squidfunk.github.io/mkdocs-material/reference/) which includes several [extensions to markdown](https://squidfunk.github.io/mkdocs-material/setup/extensions/) for enhanced technical writing.

!!! tip "RSS Feeds"
    If you would like to subscribe to updates for this site please use the [RSS](/rss.xml) or [ATOM](/atom.xml) feeds.

## WA Security Operations Centre (WA SOC)

- [Connecting to the WA SOC](onboarding.md) ([Sentinel Guidance](onboarding/sentinel-guidance.md))
- [Advisories (TLP:CLEAR)](advisories.md)
- [Incident Reporting User Guide (Jira)](guidelines/incident-reporting.md)
- [Threat Hunting (MITRE ATT&CK Tactics and Techniques)](guidelines/TTP_Hunt/ttp-detection-guidelines.md)
- [ACSC Essential Eight Assessment Process Guide](https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/essential-eight/essential-eight-assessment-process-guide)

## Baselines & Guidelines

Baselines are for use as self-assessment checklists, and guidelines are for general implementation guidance.

!!! abstract "Baselines"
    - [Security Operations Baseline](baselines/security-operations.md) - aligned with [MITRE 11 Strategies of a World-Class Cybersecurity Operations Center](pdfs/11-strategies-of-a-world-class-cybersecurity-operations-center.pdf) and [ACSC's Cyber Incident Response Plan Resource](https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/publications/cyber-incident-response-plan).
    - [Detection Coverage Baseline](baselines/data-sources.md) - *[telemetry collection](https://attack.mitre.org/datasources/)* and *[detection analytics](https://attack.mitre.org)* aligned to the [MITRE ATT&CK Framework](https://attack.mitre.org).
    - [Vulnerability Management Baseline](baselines/vulnerability-management.md) - focused on undertaking operational **Identify** and **Protect** capabilities.

!!! danger "Critical Infrastructure Entities and Operational Technology"
    The [CISA Cross-Sector Cybersecurity Performance Goals](https://www.cisa.gov/cross-sector-cybersecurity-performance-goals) are clear targeted recommendations focusing on most common and impactful threats, including cost, complexity and impact ratings against each recommendation. These are highly relevant targets for entities in scope of [SOCI regulatory obligations](https://www.cisc.gov.au/legislative-information-and-reforms/critical-infrastructure/regulatory-obligations).

    <iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/podtgMpjVp4?si=rmJ_tqjca9iQX_in" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

!!! tip "Guidelines"
    - [Supply Chain Risk Management Guideline](guidelines/supply-chain-risk-mgmt.md) - Implementation guidance for [ACSC Cyber Supply Chain Risk Management](https://www.cyber.gov.au/resources-business-and-government/maintaining-devices-and-systems/outsourcing-and-procurement/cyber-supply-chains/cyber-supply-chain-risk-management).
    - [Guide to Securing Remote Access Software (CISA)](https://www.cisa.gov/resources-tools/resources/guide-securing-remote-access-software) - remote access software overview, including the malicious use of remote access software, detection methods, and recommendations for all organizations.
    - [#StopRansomware Guide (CISA)](https://www.cisa.gov/resources-tools/resources/stopransomware-guide) - one-stop resource to help organizations reduce the risk of ransomware incidents through best practices to detect, prevent, respond, and recover, including step-by-step approaches to address potential attacks.
    - [Microsoft Sentinel Guidance](onboarding/sentinel-guidance.md) - Implementation guidance for using Sentinel for [ACSC Guidelines for System Monitoring](https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/ism/cyber-security-guidelines/guidelines-system-monitoring)
    - [Network Management Guideline](guidelines/network-management.md) - Implementation guidance for [ACSC Network gateway hardening](https://www.cyber.gov.au/resources-business-and-government/maintaining-devices-and-systems/system-hardening-and-administration/gateway-hardening/gateway-security-guidance-package-executive-guidance).
    - [Patch Management Guideline](guidelines/patch-management.md) - Implementation guidance for [ACSC Assessing Security Vulnerabilities and Applying Patches](https://www.cyber.gov.au/resources-business-and-government/maintaining-devices-and-systems/system-hardening-and-administration/system-administration/assessing-security-vulnerabilities-and-applying-patches).

## Additional documentation

The below documents are for general use.

!!! note "Technical Documentation"
    - [SOC Analyst Induction](training/analyst-induction.md)
    - [Collecting Digital Forensic Evidence](guidelines/collecting-evidence.md)
    - [Cyber Security Playbooks](guidelines/playbooks.md)

{% include 'threat-activity.md' %}

<script>
    if (window.location.hash && window.location.hash[1] === "/") {
        var location_parts = window.location.hash.slice(1).split("?id=");
        window.location.hash = '';
        if (location_parts[1]) {
            window.location.hash = location_parts[1];
        }
        window.location.pathname = window.location.pathname + location_parts[0].replace(".md", "");
    }
</script>


# Bit-Bots Software Stack

[![Build & Test](https://github.com/bit-bots/bitbots_main/actions/workflows/ci.yml/badge.svg)](https://github.com/bit-bots/bitbots_main/actions/workflows/ci.yml)
[![Code style checks](https://github.com/bit-bots/bitbots_main/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/bit-bots/bitbots_main/actions/workflows/pre-commit.yml)
[![ROS Version Iron](https://img.shields.io/badge/ROS%20Version-Iron-ab8c71)](https://docs.ros.org/en/iron/index.html)

This git repository contains all RoboCup-related code and documentation from the Hamburg Bit-Bots team.
All code is written as individual ROS 2 packages targeting Ubuntu.

## Installation

Full step-by-step instructions for installing the Bit-Bots software stack and ROS 2 can be found in our documentation [here](https://doku.bit-bots.de/meta/manual/tutorials/install_software_ros2.html).

## Update the codebase

If you want to update this repo, all third party source files as well as the supplementing files, run

``` shell
make pull-all
```

If you encounter any problems consider cleaning the third party source files (the `lib` folder) first:

**THIS DELETES ALL CHANGES YOU MADE TO THE THIRD PARTY SOURCE FILES!**

``` shell
make fresh-libs
```

## Run auto formatting

To format all code in the repository, run

``` shell
make format
```

## More documentation

Our documentation is hosted on [docs.bit-bots.de](https://docs.bit-bots.de/).


![](https://user-images.githubusercontent.com/47793918/233812617-beab2e71-57b9-479e-8bff-c3931347ca40.png)

Table of Contents
=======================

* [Join our Discord](#-join-our-discord)
* [What is sunnypilot?](#-what-is-sunnypilot)
* [Running in a car](#-running-on-a-dedicated-device-in-a-car)
* [Read Before Installing](#-read-before-installing)
* [Prohibited Safety Modifications](#-prohibited-safety-modifications)
* [Installation](#-installation)
* [Highlight Features](#-highlight-features)
* [Driving Enhancements](#-driving-enhancements)
* [Branch Definitions](#-branch-definitions)
* [Recommended Branches](#-recommended-branches)
* [How-To's](#-How-Tos)
* [Pull Requests](#-Pull-Requests)
* [Special Thanks](#-special-thanks)
* [User Data](#-user-data)
* [Licensing](#licensing)
* [Donate](#-support-sunnypilot)

---

💭 Join our Discord

---

Join the official sunnypilot Discord server to stay up to date with all the latest features and be a part of shaping the future of sunnypilot!
* https://discord.gg/sunnypilot

  ![](https://dcbadge.vercel.app/api/server/wRW3meAgtx?style=flat) ![Discord Shield](https://discordapp.com/api/guilds/880416502577266699/widget.png?style=shield)

🌞 What is sunnypilot?

---

[sunnypilot](https://github.com/sunnyhaibin/sunnypilot) is a fork of comma.ai's openpilot, an open source driver assistance system. sunnypilot offers the user a unique driving experience for over 250+ supported car makes and models with modified behaviors of driving assist engagements. sunnypilot complies with comma.ai's safety rules as accurately as possible.

🚘 Running on a dedicated device in a car

---

To use sunnypilot in a car, you need the following:
* A supported device to run this software
    * a [comma three](https://comma.ai/shop/products/three), or
    * a comma two (only with older versions below 0.8.13)
* This software
* One of [the 250+ supported cars](https://github.com/commaai/openpilot/blob/master/docs/CARS.md). We support Honda, Toyota, Hyundai, Nissan, Kia, Chrysler, Lexus, Acura, Audi, VW, Ford and more. If your car is not supported but has adaptive cruise control and lane-keeping assist, it's likely able to run sunnypilot.
* A [car harness](https://comma.ai/shop/products/car-harness) to connect to your car

Detailed instructions for [how to mount the device in a car](https://comma.ai/setup).

🚨 Read Before Installing

---

It is recommended to read this **entire page** before proceeding. This will ensure that you fully understand each added feature on sunnypilot, and you are selecting the right branch for your car to have the best driving experience.

This is a fork of [comma.ai's openpilot](https://github.com/commaai/openpilot). By installing this software, you accept all responsibility for anything that might occur while you use it. All contributors to sunnypilot are not liable. ❗**Use at your own risk.**❗

⛔ Prohibited Safety Modifications

---

All [official sunnypilot branches](https://github.com/sunnyhaibin/sunnypilot/branches) strictly adhere to [comma.ai's safety policy](https://github.com/commaai/openpilot/blob/master/docs/SAFETY.md). Any changes that go against this policy will result in your fork and your device being banned from both comma.ai and sunnypilot channels.

The following changes are a **VIOLATION** of this policy and **ARE NOT** included in any sunnypilot branches:
* Driver Monitoring:
    * ❌ "Nerfing" or reducing monitoring parameters.
* Panda safety:
    * ❌ No preventing disengaging of **LONGITUDINAL CONTROL** (acceleration/brake) on brake pedal press.
    * ❌ No auto re-engaging of **LONGITUDINAL CONTROL** (acceleration/brake) on brake pedal release.
    * ❌ No disengaging on ACC MAIN in OFF state.

⚒ Installation

---

  URL (Easy)

comma three
------

Please refer to [Recommended Branches](#-recommended-branches) to find your preferred/supported branch. This guide will assume you want to install the latest `release-c3` branch.

* sunnypilot not installed or you installed a version before 0.8.17?
  1. [Factory reset/uninstall](https://github.com/commaai/openpilot/wiki/FAQ#how-can-i-reset-the-device) the previous software if you have another software/fork installed.
  2. After factory reset/uninstall and upon reboot, select `Custom Software` when given the option.
  3. Input the installation URL per [Recommended Branches](#-recommended-branches). Example: ```release-c3.sunnypilot.ai``` [^4] (note: `https://` is not requirement on the comma three)
  4. Complete the rest of the installation following the onscreen instructions.

* sunnypilot already installed and you installed a version after 0.8.17?
  1. On the comma three, go to `Settings` ▶ `Software`.
  2. At the `Download` option, press `CHECK`. This will fetch the list of latest branches from sunnypilot.
  3. At the `Target Branch` option, press `SELECT` to open the Target Branch selector.
  4. Scroll to select the desired branch per [Recommended Branches](#-recommended-branches). Example: `release-c3`

|    Branch    |         Installation URL         |
|:------------:|:--------------------------------:|
| `release-c3` | https://release-c3.sunnypilot.ai |
| `staging-c3` | https://staging-c3.sunnypilot.ai |
|   `dev-c3`   | https://dev-c3.sunnypilot.ai     |

Requires further assistance with software installation? Join the [sunnypilot Discord server](https://discord.sunnypilot.com) and message us in the `#installation-help` channel.

comma two
------

1. [Factory reset/uninstall](https://github.com/commaai/openpilot/wiki/FAQ#how-can-i-reset-the-device) the previous software if you have another software/fork installed.
2. After factory reset/uninstall and upon reboot, select `Custom Software` when given the option.
3. Input the installation URL per [Recommended Branches](#-recommended-branches). Example: ```https://smiskol.com/fork/sunnyhaibin/0.8.12-4-prod```
4. Complete the rest of the installation following the onscreen instructions.

Requires further assistance with software installation? Join the [sunnypilot Discord server](https://discord.sunnypilot.com) and message us in the `#installation-help` channel.

  SSH (More Versatile)

Prerequisites: [How to SSH](https://github.com/commaai/openpilot/wiki/SSH)

If you are looking to install sunnypilot via SSH, run the following command in an SSH terminal after connecting to your device:

comma three:
------
* [`release-c3`](https://github.com/sunnyhaibin/openpilot/tree/release-c3):

  ```
  cd /data; rm -rf ./openpilot; git clone -b release-c3 --recurse-submodules https://github.com/sunnyhaibin/sunnypilot.git openpilot; cd openpilot; sudo reboot
  ```

comma two:
------
* [`0.8.12-prod-personal-hkg`](https://github.com/sunnyhaibin/openpilot/tree/0.8.12-prod-personal-hkg):

  ```
  cd /data; rm -rf ./openpilot; git clone -b 0.8.12-prod-personal-hkg --recurse-submodules https://github.com/sunnyhaibin/sunnypilot.git openpilot; cd openpilot; sudo reboot
  ```

After running the command to install the desired branch, your comma device should reboot.

🚗 Highlight Features

---

### Quality of Life Enhancements
- [**Modified Assistive Driving Safety (MADS)**](#modified-assistive-driving-safety-mads) - Automatic Lane Centering (ALC) / Lane Keep Assist System (LKAS) and Adaptive Cruise Control (ACC) / Smart Cruise Control (SCC) can be engaged independently of each other
- [**Dynamic Lane Profile (DLP)**](#dynamic-lane-profile-dlp) - Dynamically switch lane profile (between Laneful and Laneless) based on lane recognition confidence
- [**Enhanced Speed Control**](#enhanced-speed-control) - Automatically adjust cruise control speed using vision model, OpenStreetMap (OSM) data, and/or Speed Limit control (SLC) without user interaction
    * Vision-based Turn Speed Control (V-TSC) - lower speed when going around corners using vision model
    * Map-Data-based Turn Speed Control (M-TSC) - lower speed when going around corners using OSM data[^1]
    * Speed Limit Control (SLC) - Set speed limit based on map data or car interface (if applicable)
    * HKG only: Highway Driving Assist (HDA) status integration - Use cars native speed sign detection to set desired speed (on applicable HKG cars only)
- [**Gap Adjust Cruise (GAC)**](#gap-adjust-cruise) - Allow `GAP`/`INTERVAL`/`DISTANCE` button on the steering wheel or on-screen button to adjust the follow distance from the lead car. See table below for options
    - [**Quiet Drive 🤫**](#-quiet-drive) - Toggle to mute all notification sounds (excluding driver safety warnings)
    - [**Auto Lane Change Timer**](#Auto-Lane-Change-Timer) - Set a timer to delay the auto lane change operation when the blinker is used. No nudge on the steering wheel is required to auto lane change if a timer is set
    - [**Force Car Recognition (FCR)**](#Force-Car-Recognition-) - Use a selector to force your car to be recognized by sunnypilot
    - [**Fix sunnypilot No Offroad**](#Fix-sunnypilot-No-Offroad) - Enforce sunnypilot to go offroad and turns off after shutting down the car. This feature fixes non-official devices running sunnypilot without comma power
    - [**Enable ACC+MADS with RES+/SET-**](#Enable-ACC+MADS-with-RES+/SET-) - Engage both ACC and MADS with a single press of RES+ or SET- button
    - [**Offline OSM Maps**](#Offline-OSM-Maps) - OSM database can now be downloaded locally for offline use[^2]. This enables offline SLC, V-TSC and M-TSC. Currently available for US South, US West, US Northeast, Florida, Taiwan, South Africa and New Zealand
    - [**Various Live Tuning**](#Various-Live-Tuning) - Ability to tailor your driving experience on the fly:
        * Enforce Torque Lateral Control - Use the newest [torque controller](https://blog.comma.ai/0815release/#torque-controller) for all vehicles.
        * Torque Lateral Control Live Tune - Ability to adjust the torque controller’s `FRICTION` and `LAT_ACCEL_FACTOR` values to suit your vehicle.
        * Torque Lateral Controller Self-Tune - Enable automatic turning for the Torque controller.

### Visual Enhancements
* **M.A.D.S Status Icon** - Dedicated icon to display M.A.D.S. engagement status
    * Green🟢: M.A.D.S. engaged
    * White⚪: M.A.D.S. suspended or disengaged
* **Lane Path Color** - Various lane path colors to display real-time Lane Model and M.A.D.S. engagement status
    * 0.8.14 and later:
        * Blue🔵: Laneful mode & M.A.D.S. engaged
        * Green🟢: Laneless mode & M.A.D.S. engaged
        * Yellow🟡: Experimental e2e & M.A.D.S. engaged
    * Pre 0.8.14:
        * Green🟢: Laneful mode & M.A.D.S. engaged
        * Red🔴: Laneless mode & M.A.D.S. engaged
    * White⚪: M.A.D.S. suspended or disengaged
    * Black⚫: M.A.D.S. engaged, steering is being manually overridden by user
* **Developer (Dev) UI** - Display various real-time metrics on screen while driving
* **Stand Still Timer** - Display time spent at a stop with M.A.D.S engaged (i.e., at traffic lights, stop signs, traffic congestions)
* **Braking Status** - Current car speed text turns red when the car is braking by the driver or ACC/SCC

### Operational Enhancements
* **Fast Boot** - sunnypilot will fast boot by creating a Prebuilt file
* **Disable Onroad Uploads** - Disable uploads completely when onroad. Necessary to avoid high data usage when connected to Wi-Fi hotspot
* **Brightness Control (Global)** - Manually adjusts the global brightness of the screen
* **Driving Screen Off Timer** - Turns off the device screen or reduces brightness to protect the screen after car starts
* **Driving Screen Off Brightness (%)** - When using the Driving Screen Off feature, the brightness is reduced according to the automatic brightness ratio
* **Max Time Offroad** - Device is automatically turned off after a set time when the engine is turned off (off-road) after driving (on-road)

🚗 Driving Enhancements

---

### Modified Assistive Driving Safety (MADS)
The goal of Modified Assistive Driving Safety (MADS) is to enhance the user driving experience with modified behaviors of driving assist engagements. This feature complies with comma.ai's safety rules as accurately as possible with the following changes:
* sunnypilot Automatic Lane Centering (ALC) and ACC/SCC can be engaged independently of each other
* Dedicated button to toggle sunnypilot ALC:
    * `CRUISE (MAIN)` button: All supported cars on sunnypilot
        * `LFA` button: Newer HKG cars with `LFA` button
        * `LKAS` button: Honda, Toyota, Global Subaru
* `SET-` button enables ACC/SCC
* `CANCEL` button only disables ACC/SCC
* `CRUISE (MAIN)` must be `ON` to use ACC/SCC
* `CRUISE (MAIN)` button disables sunnypilot completely when `OFF` **(strictly enforced in panda safety code)**

### Disengage Lateral ALC on Brake Press Mode toggle
Dedicated toggle to handle Lateral state on brake pedal press and release:
1. `ON`: `BRAKE pedal` press will pause Automatic Lane Centering; `BRAKE pedal` release will resume Automatic Lane Centering. Note: `BRAKE pedal` release will NOT resume ACC/SCC/Long control without explicit user engagement **(strictly enforced in panda safety code)**
2. `OFF`: `BRAKE pedal` press will NOT pause Automatic Lane Centering; `BRAKE pedal` release will NOT resume ACC/SCC/Long control without explicit user engagement **(strictly enforced in panda safety code)**

### Miscellaneous
* `TURN SIGNALS` (`Left` or `Right`) will pause Automatic Lane Centering if the vehicle speed is below the [threshold](https://github.com/commaai/openpilot/blob/master/selfdrive/controls/lib/desire_helper.py#L8) for Automatic Lane Change
* Event audible alerts are more relaxed to match manufacturer's stock behavior
* Critical events trigger disengagement of Automatic Lane Centering completely. The disengagement is enforced in sunnypilot and panda safety

### Dynamic Lane Profile (DLP)

Dynamic Lane Profile (DLP) aims to provide the best driving experience at staying within a lane confidently. Dynamic Lane Profile allows sunnypilot to dynamically switch between lane profiles based on lane recognition confidence level on road.

There are 3 modes to select on the onroad camera screen:
* **Auto Lane**: sunnypilot dynamically chooses between `Laneline` or `Laneless` model
* **Laneline**: sunnypilot uses Laneline model only.
* **Laneless**: sunnypilot uses Laneless model only.

To use Dynamic Lane Profile, do the following:
```
1. sunnypilot Settings -> `SP - Controls` -> Enable Dynamic Lane Profile -> ON toggle
2. Reboot.
3. Before driving, on the onroad camera screen, toggle between the 3 modes by pressing on the button.
4. Drive.
```

### Enhanced Speed Control
This fork now allows supported cars to dynamically adjust the longitudinal plan based on the fetched map data. Big thanks to the Move Fast team for the amazing implementation!

**Supported cars:**
* sunnypilot Longitudinal Control capable
* Stock Longitudinal Control
    * Hyundai/Kia/Genesis (non CAN-FD)
    * Honda Bosch
    * Volkswagen MQB

Certain features are only available with an active data connection, via:
* [comma Prime](https://comma.ai/prime) - Intuitive service provided directly by comma, or
* Personal Hotspot - From your mobile device, or a dedicated hotspot from a cellular carrier.

**Features:**
* Vision-based Turn Speed Control (VTSC) - Use vision path predictions to estimate the appropriate speed to drive through turns ahead - i.e. slowing down for curves
* Map-Data-based Turn Speed Control (MTSC) - Use curvature information from map data to define speed limits to take turns ahead - i.e. slowing down for curves[^1]
* Speed Limit Control (SLC) - Use speed limit signs information from map data and car interface to automatically adapt cruise speed to road limits
    * HKG only: Highway Driving Assist (HDA) status integration - on applicable HKG cars only[^1]
    * Speed Limit Offset - When Speed Limit Control is enabled, set speed limit slightly higher than the actual speed limit for a more natural drive[^1]
* Toggle Hands on Wheel Monitoring - Monitors and alerts the driver when their hands have not been on the steering wheel for an extended time

### Custom Stock Longitudinal Control
While using stock Adaptive/Smart Cruise Control, Custom Stock Longitudinal Control in sunnypilot allows sunnypilot to manipulate and take over the set speed on the car's dashboard.

**Supported Cars:**
* Hyundai/Kia/Genesis
    * CAN platform
    * CAN-FD platform with 0x1CF broadcasted in CAN traffic
* Honda Bosch
* Volkswagen MQB

**Instruction**

**📗 How to use Custom Longitudinal Control on sunnypilot **

When using Speed Limit, Vision, or Map based Turn control, you will be setting the "MAX" ACC speed on the sunnypilot display instead of the one in the dashboard. The car will then set the ACC setting in the dashboard to the targeted speed, but will never exceed the max speed set on the sunnypilot display. A quick press of the RES+ or SET- buttons will change this speed by 5 MPH or KM/H on the sunnypilot display, while a long deliberate press (about a 1/2 second press) changes it by 1 MPH or KM/H. DO NOT hold the RES+ or SET- buttons for longer that a 1 second. Either make quick or long deliberate presses only.

**‼ Where to look when setting ACC speed ‼**

Do not look at the dashboard when setting your ACC max speed. Instead, only look at the one on the sunnypilot display, "MAX". The reason you need to look at the sunnypilot display is because sunnypilot will be changing the one in the dashboard. It will be adjusting it as needed, never raising it above the one set on the sunnypilot display. ONLY look at the MAX speed on the sunnypilot display when setting the ACC speed instead of the dashboard!

(Courtesy instructions from John, author of jvePilot)

### Gap Adjust Cruise
This fork now allows supported openpilot longitudinal cars to adjust the cruise gap between the car and the lead car.

**Supported cars:**
* sunnypilot Longitudinal Control capable

🚨**PROCEED WITH EXTREME CAUTION AND BE READY TO MANUALLY TAKE OVER AT ALL TIMES**

There are 4 modes to select on the steering wheel and/or the onroad camera screen:
* **Stock Gap**: Stock sunnypilot distance - 1.45 second profile
* **Mild Gap**: Semi-aggressive distance - 1.25 second profile
* 🚨**Aggro Gap**🚨: Aggressive distance - 1.0 second profile

**Availability**

|      Car Make       | Stock Gap | Mild Gap | Aggro Gap |
|:-------------------:|:---------:|:--------:|:---------:|
|     Honda/Acura     |     ✅     |    ✅     |     ✅     |
| Hyundai/Kia/Genesis |     ✅     |    ✅     |     ✅     |
|    Toyota/Lexus     |     ✅     |    ✅     |     ✅     |
|  Volkswagen MQB/PQ  |     ✅     |    ✅     |     ✅     |

⚒ Branch Definitions

---

|    Tag    | Definition           | Description                                                                                |
|:---------:|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `release` | Release branches     | Include features that are **verified** by trusted testers and the community. Ready to use. ✅                                                                                |
| `staging` | Staging branches     | Include new features that are **tested** by trusted testers and the community. Stability may vary. ⚠                                                                                |
|   `dev`   | Development branches | All features are gathered in respective versions. Reviewed and merged features will be committed to `dev`. Stability may vary. ⚠                                                            |
| `master`  | Main branch          | Syncs with [commaai's openpilot `master`](https://github.com/commaai/openpilot) upstream branch. Accepts all pull requests. Does not include all sunnypilot features. Stability may vary. ⚠ |

Example:
* [`release-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/release-c3): Latest release branch for comma three that are verified by trusted testers and the community. Ready to use.
* [`staging-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/staging-c3): Latest staging branch for comma three that are tested by trusted testers and the community. Verification required.
* [`dev-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/dev-c3): Latest development branch for comma three that include all sunnypilot features. Testing required.

✅ Recommended Branches

---

| Branch                                                                              | Definition                                              | Compatible Device | Changelogs                                                                                |
|:------------------------------------------------------------------------------------|---------------------------------------------------------|-------------------|--------------------------------------------------------------------------------------------|
| [`release-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/release-c3)           | • Latest release/stable branch                          | comma three       | [`CHANGELOGS.md`](https://github.com/sunnyhaibin/sunnypilot/blob/release-c3/CHANGELOGS.md) |
| [`staging-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/staging-c3)           | • Latest staging branch                                 | comma three       | [`CHANGELOGS.md`](https://github.com/sunnyhaibin/sunnypilot/blob/staging-c3/CHANGELOGS.md) |
| [`dev-c3`](https://github.com/sunnyhaibin/sunnypilot/tree/dev-c3)                   | • Latest development branch with experimental features  | comma three       | [`CHANGELOGS.md`](https://github.com/sunnyhaibin/sunnypilot/blob/dev-c3/CHANGELOGS.md)     |

📗 How To's

---

How-To instructions can be found in [HOW-TOS.md](https://github.com/sunnyhaibin/openpilot/blob/(!)README/HOW-TOS.md).

🎆 Pull Requests

---

We welcome both pull requests and issues on GitHub. Bug fixes are encouraged.

Pull requests should be against the most current `master` branch.

🏆 Special Thanks

---

* [spektor56](https://github.com/spektor56/openpilot)
* [rav4kumar](https://github.com/rav4kumar/openpilot)
* [mob9221](https://github.com/mob9221/opendbc)
* [briantran33](https://github.com/briantran33/openpilot)
* [Aragon7777](https://github.com/aragon7777/openpilot)
* [sshane](https://github.com/sshane/openpilot-installer-generator)
* [jung](https://github.com/chanhojung/openpilot)
* [dri94](https://github.com/dri94/openpilot)
* [FrogAi](https://github.com/frogAi/FrogPilot/)
* [twilsonco](https://github.com/twilsonco/openpilot)
* [martinl](https://github.com/martinl/openpilot)
* [multikyd](https://github.com/openpilotkr)
* [Move Fast GmbH](https://github.com/move-fast/openpilot)
* [dragonpilot](https://github.com/dragonpilot-community/dragonpilot)
* [neokii](https://github.com/neokii/openpilot)
* [AlexandreSato](https://github.com/AlexandreSato/openpilot)
* [Moodkiller](https://github.com/moodkiller)

📊 User Data

---

By default, sunnypilot uploads the driving data to comma servers. You can also access your data through [comma connect](https://connect.comma.ai/).

sunnypilot is open source software. The user is free to disable data collection if they wish to do so.

sunnypilot logs the road-facing camera, CAN, GPS, IMU, magnetometer, thermal sensors, crashes, and operating system logs.
The driver-facing camera is only logged if you explicitly opt-in in settings. The microphone is not recorded.

By using this software, you understand that use of this software or its related services will generate certain types of user data, which may be logged and stored at the sole discretion of comma. By accepting this agreement, you grant an irrevocable, perpetual, worldwide right to comma for the use of this data.

Licensing

openpilot is released under the MIT license. Some parts of the software are released under other licenses as specified.

Any user of this software shall indemnify and hold harmless comma.ai, Inc. and its directors, officers, employees, agents, stockholders, affiliates, subcontractors and customers from and against all allegations, claims, actions, suits, demands, damages, liabilities, obligations, losses, settlements, judgments, costs and expenses (including without limitation attorneys’ fees and costs) which arise out of, relate to or result from any use of this software by user.

**THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT.
YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS.
NO WARRANTY EXPRESSED OR IMPLIED.**

💰 Support sunnypilot

---

If you find any of the features useful, consider becoming a [patron on Patreon](https://www.patreon.com/sunnyhaibin) or a [sponsor on GitHub](https://github.com/sponsors/sunnyhaibin) to support future feature development and improvements.

By becoming a patron/sponsor, you will gain access to exclusive content, early access to new features, and the opportunity to directly influence the project's development.

Patreon

GitHub Sponsor

PayPal

Your continuous love and support are greatly appreciated! Enjoy 🥰

- Jason, Founder of sunnypilot

[^1]:Requires data connection if not using Offline Maps data
[^2]:At least 50 GB of storage space is required. If you have the 32 GB version of comma three, upgrading with a compatible 250 GB or 1 TB SSD is strongly recommended
[^4]:Shortened URL for convenience. Full URL is ```smiskol.com/fork/sunnyhaibin/release-c3```


# IOb-SoC

IOb-SoC is a System-on-Chip (SoC) template comprising an open-source RISC-V
processor (picorv32), an internal SRAM memory subsystem, a UART, and
an optional interface to external memory. If the external memory interface is
selected, an instruction L1 cache, a data L1 cache, and a shared L2 cache are
added to the system. The L2 cache communicates with a 3rd party memory
controller IP (typically a DDR controller) using an AXI4 master bus.

## Nix environment

You can use
[nix-shell](https://nixos.org/download.html#nix-install-linux) to run
IOb-SoC in a [Nix](https://nixos.org/) environment with all dependencies
available except for Vivado and Quartus for FPGA compilation and running.

After installing `nix-shell,` it can be initialized by calling any Makefile target in the IOb-SoC root directory, for example
```Bash
make setup
```

The first time it runs, `nix-shell` will automatically install all the required dependencies. This can take a couple of hours, but after that, you can enjoy IOb-SoC and not worry about installing software tools.

## Dependencies

If you prefer, you may install all the dependencies manually and run IOb-SoC without nix-shell. The following tools should be installed:
- GNU Bash >=5.1.16
- GNU Make >=4.3
- RISC-V GNU Compiler Toolchain =2022.06.10  (Instructions at the end of this README)
- Python3 >=3.10.6
- Python3-Parse >=1.19.0

Optional tools, depending on the desired run strategy:
- Icarus Verilog >=10.3
- Verilator >=5.002
- gtkwave >=3.3.113
- Vivado >=2020.2
- Quartus >=20.1

Older versions of the dependencies above may work but still need to be tested.

## Operating Systems

IOb-SoC can be used in Linux Operating Systems. The following instructions work
for CentOS 7 and Ubuntu 18.04, 20.04, and 22.04 LTS.

## Clone the repository

The first step is to clone this repository. IOb-SoC uses git sub-module trees, and
GitHub will ask for your password for each downloaded module if you clone it by *https*. To avoid this,
setup GitHub access with *ssh* and type:

```Bash
git clone --recursive git@github.com:IObundle/iob-soc.git
cd iob-soc
```

Alternatively, you can still clone this repository using *https* if you cache
your credentials before cloning the repository, using: ``git config --global
credential.helper 'cache --timeout=<time_in_seconds>'``

## Configure your SoC

To configure your system, edit the `iob_soc.py` file, which can be found at the
repository root. This file has the system configuration variables;
hopefully, each variable is explained by a comment.

## Set environment variables for local or remote building and running

The various simulators, FPGA compilers, and FPGA boards may run locally or
remotely. For running a tool remotely, you need to set two environmental
variables: the server logical name and the server user name. Consider placing
these settings in your `.bashrc` file so that they apply to every session.

### Set up the remote simulator server

Using the open-source simulator Icarus Verilog (`iverilog`) as an example, note that in
`submodules/hardware/simulation/icarus.mk,` the variable for the server logical name,
`SIM_SERVER,` is set to `IVSIM_SERVER,` and the variable for the user name,
`SIM_USER` is set to `IVSIM_USER`.

To run the simulator on the server *mysimserver.myorg.com* as user *ivsimuser*, set the following environmental
variables beforehand, or place them in your `.bashrc` file:

```Bash
export IVSIM_SERVER=ivsimserver.myorg.com
export IVSIM_USER=ivsimuser
```

When you start the simulation, IOb-SoC's simulation Makefile will log you on to the server using `ssh,` then `rsync` the files to a remote build directory and run the simulation there.  If you do not set these variables, the simulator will run locally if installed.

### Set up the remote FPGA toolchain and board servers

Using the CYCLONEV-GT-DK board as an example, note that in
`hardware/fpga/quartus/CYCLONEV-GT-DK/Makefile,` the variable for the FPGA tool
server logical name, `FPGA_SERVER,` is set to `QUARTUS_SERVER,` and the
variable for the user name, `FPGA_USER`, is set to `QUARTUS_USER`; the
variable for the board server, `BOARD_SERVER,` is set to `CYC5_SERVER`, and
the variable for the board user, `BOARD_USER,` is set to `CYC5_USER`. As in the
previous example, set these variables as follows:

```Bash
export QUARTUS_SERVER=quartusserver.myorg.com
export QUARTUS_USER=quartususer
export CYC5_SERVER=cyc5server.myorg.com
export CYC5_USER=cyc5username
```

In each remote server, the environment variable for the license server used must be defined as in the following example:

```Bash
export LM_LICENSE_FILE=port@licenseserver.myorg.com;lic_or_dat_file
```

## Create the build directory

IOb-SoC uses intricate Python scripting to create a build directory with all the necessary files and makefiles to run the different tools. The build directory is placed in the folder above at ../iob_soc_Vx.y by running the following command from the root directory.
```Bash
make setup
```

If you want to avoid getting into the complications of our Python scripts, use the ../iob_soc_Vx.y directory to build your SoC. It only has code files and a few Makefiles. Enter this directory and call the available Makefile targets. Alternatively, using another Makefile in the IOb-SoC root directory, the same targets can be called. For example, to run the simulation, the IOb-SoC's top Makefile has the following target:

```Bash
sim-run:
	nix-shell --run 'make clean setup INIT_MEM=$(INIT_MEM) USE_EXTMEM=$(USE_EXTMEM) && make -C ../$(CORE)_V*/ sim-run SIMULATOR=$(SIMULATOR)'
```
The above target invokes the `nix-shell` environment to call the local targets `clean` and `setup` and the target `sim-run` in the build directory. Below, the targets available in IOb-SoC's top Makefile are explained.

## Emulate the system on PC

You can *emulate* IOb-SoC's on a PC to develop and debug your embedded system. There is also a model to emulate the UART, which communicates with a run-time Python script server. If you develop peripherals, you can build embedded software models to run them using PC emulation. To emulate IOb-SoC's embedded software on a PC, type:

```Bash
make pc-emul-run
```

The Makefile compiles and runs the software in the `../iob_soc_Vx.y/software/` directory. The Makefile includes the `sw_build.mk` segment supplied initially in the `./software/` directory in the IOb-SoC root. Please feel free to change this file for your specific project. To run an emulation test comparing the result to the expected result, run
```Bash
make pc-emul-test
```

## Simulate the system

To simulate IOb-SoC's RTL using a Verilog simulator, run
```Bash
make sim-run [SIMULATOR=icarus!verilator|xcelium|vcs|questa] [INIT_MEM=0|1] [USE_EXTMEM=0|1]
```

The INIT_MEM variable specifies whether the firmware is initially loaded in the memory, skipping the boot process, and the USE_EXTMEM variable indicates whether an external memory such as DRAM is used, in which case the cache system described above is instantiated.

The Makefile compiles and runs the software in the `../iob_soc_Vx.y/hardware/simulation` directory. The Makefile includes the `./hardware/simulation/sim_build.mk`, which you can change for your project. To run a simulation test comprising several simulations with different parameters, run
```Bash
make sim-test
```
The simulation test contents can be edited in IOb-SoC's top Makefile.

Each simulator must be described in the `./submodules/LIB/hardware/simulation/<simulator>.mk` file. For example, the file `vcs.mk` describes the VCS simulator.

The host machine must run an access server, a Python program in `./submodules/LIB/scripts/board_server.py,` set up to run as a service. The client connects to the host using the SSH protocol and runs the board client program `/submodules/LIB/scripts/board_client.py.` Note that the term *board* is used instead of *simulator* because the same server/client programs control the access to the board and FPGA compilers. The client requests the simulator for GRAB_TIMEOUT seconds, which is 300 seconds by default. Its value can be specified in the `./hardware/fpga/fpga_build.mk` Makefile segment, for example, as
```Bash
GRAB_TIMEOUT ?= 3600
```

## Build and run on FPGA board

To build and run IOb-SoC on an FPGA board, the FPGA design tools must be
installed locally or remotely. The FPGA board must also be attached to the local
or remote host, not necessarily the same host where the design tools are installed.

Each board must be described under the `/submodules/LIB/hardware/fpga/<tool>/<board_dir>` directory. For example, the `hardware/fpga/vivado/BASYS3`
directory contents describe the board BASYS3, which has an FPGA device that can be programmed by the Xilinx/AMD Vivado design tool. The access to the board is controlled by the same server/client programs described above for the simulators.
To build an FPGA design of an IOb-SoC system and run it on the board located in the `board_dir` directory, type
```Bash
make fpga-run [BOARD=<board_dir>] [INIT_MEM=0|1] [USE_EXTMEM=0|1]
```

To run an FPGA test comparing the result to the expected result, run
```Bash
make fpga-test
```
The FPGA test contents can be edited in IOb-SoC's top Makefile.

The remote machines that have an FPGA board attached to it must run our board
access control service script, which can be found in
`submodules/LIB/scripts/board_server.py`. When IOb-SoC needs to access a remote
FPGA server, it runs the board access script located in
`submodules/LIB/scripts/board_server.py`.

To install `board_server.py` as a service, run the following command on the remote FPGA server:
```
sudo make board_server_install
```

To uninstall the service, run

```
sudo make board_server_uninstall
```

Finally, to query the board status, run
```
sudo make board_server_uninstall
```

## Compile the documentation

To compile documents, the LaTeX software must be installed. Three document types are generated: the Product Brief (pb), the User Guide (ug), and a presentation. To build a given document type DOC, run
```Bash
make doc-build [DOC=pb|ug|presentation]
```

To generate the three documents as a test, run
```Bash
make doc-test
```

## Total test

To run all simulation, FPGA board, and documentation tests, type:

```Bash
make test-all
```

## Running more Makefile Targets

The examples above are the Makefile targets at IOb-SoC's root directory that call the targets in the top Makefile in the build directory. Please explore the available targets in the build directory's top Makefile to add more targets to the root directory Makefile.

## Cleaning the build directory
To clean the build directory, run
```Bash
make clean
```

## Instructions for Installing the RISC-V GNU Compiler Toolchain

### Get sources and check out the supported stable version

```Bash
git clone https://github.com/riscv/riscv-gnu-toolchain
cd riscv-gnu-toolchain
git checkout 2022.06.10
```

### Prerequisites

For the Ubuntu OS and its variants:

```Bash
sudo apt install autoconf automake autotools-dev curl python3 python2 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
```

For CentOS and its variants:

```Bash
sudo yum install autoconf automake python3 python2 libmpc-devel mpfr-devel gmp-devel gawk  bison flex texinfo patchutils gcc gcc-c++ zlib-devel expat-devel
```

### Installation

```Bash
./configure --prefix=/path/to/riscv --enable-multilib
sudo make -j$(nproc)
```

This will take a while. After it is done, type:

```Bash
export PATH=$PATH:/path/to/riscv/bin
```

The above command should be added to your `~/.bashrc` file so you do not have to type it on every session.

## Ethernet

To setup the system with ethernet capability, set the `USE_ETHERNET` macro value to `True`.

When running the system with ethernet, please set the `RMAC_ADDR` and `IOB_CONSOLE_PYTHON_ENV` environment variables.
These values will select which network interface and which python environment to use for the console.

For example, you can add the following to your `~/.bashrc`:

```Bash
# IOb-SoC console network interface (loopback interfacce)
export RMAC_ADDR=000000000000
# Custom IOb-SoC console python interperter with `CAP_NET_RAW` capability.
export IOB_CONSOLE_PYTHON_ENV=/opt/pyeth3/bin/python
```

You could also set those variables in the build directory's `config_build.mk` file.

# Acknowledgements

First of all, we acknowledge all the volunteer contributors for all their valuable pull requests, issues, and discussions.

The work has been partially performed in the scope of the A-IQ Ready project, which receives funding within Chips Joint Undertaking (Chips JU) - the Public-Private Partnership for research, development, and innovation under Horizon Europe – and National Authorities under grant agreement No. 101096658.

The A-IQ Ready project is supported by the Chips Joint Undertaking (Chips JU) - the Public-Private Partnership for research, development, and innovation under Horizon Europe – and National Authorities under Grant Agreement No. 101096658.

![image](https://github.com/IObundle/iob-soc/assets/5718971/78f2a3ee-d10b-4989-b221-71154fe6e409) ![image](https://github.com/IObundle/iob-soc/assets/5718971/d57e0430-bb60-42e3-82a3-c5b6b0417322)

This project provides the basic infrastructure to other projects funded through the NGI Assure Fund, a fund established by NLnet
with financial support from the European Commission's Next Generation Internet program under the aegis of DG Communications Networks, Content, and Technology.