[![slack](https://img.shields.io/badge/slack-aerie-brightgreen?logo=slack)](https://join.slack.com/t/nasa-ammos/shared_invite/zt-1mlgmk5c2-MgqVSyKzVRUWrXy87FNqPw)

Aerie is a software framework for modeling spacecraft. Its main features include:

- A Java-based mission modeling library
- A discrete-event simulator
- An embedded TypeScript DSL for defining and executing scheduling goals
- An embedded TypeScript DSL for defining and executing constraints
- An embedded TypeScript DSL for defining and executing activity command expansions
- An embedded TypeScript DSL for defining sequences
- A [GraphQL API](https://nasa-ammos.github.io/aerie-docs/api/introduction)
- A web-based [client application][ui-repo]

## Getting Started

To get started using Aerie for the first time please do our [fast track tutorial][fast-track] on our documentation website.

## Need Help?

- Join us on the [NASA-AMMOS Slack](https://join.slack.com/t/nasa-ammos/shared_invite/zt-1mlgmk5c2-MgqVSyKzVRUWrXy87FNqPw) (#aerie-users)
- Contact aerie-support@googlegroups.com

## News and Updates

- Join the [Aerie Users](https://groups.google.com/u/3/g/aerie-users) group to stay up to date on news, releases, and our project roadmap.

## Directory Structure

```sh
.
├── .github                     # GitHub metadata
├── constraints                 # Java library for constraint checking
├── contrib                     # Java convenience classes for mission models
├── db-tests                    # Database unit tests
├── deployment                  # Deployment artifacts and documentation
├── docker                      # Additional Dockerfiles for Aerie-specific images
├── docs                        # Documentation
├── e2e-tests                   # End-to-end tests
├── examples                    # Example mission models
├── gradle                      # Gradle Wrapper
├── load-tests                  # Load testing code and configuration
├── merlin-driver               # Java library for discrete-event simulation
├── merlin-framework            # Java library for mission modeling
├── merlin-framework-junit      # Extension of JUnit to unit test mission models
├── merlin-framework-processor  # Java annotation processor for mission models
├── merlin-sdk                  # Java interface between mission models and the merlin-driver
├── merlin-server               # Service for planning and simulation
├── merlin-worker               # Worker for executing simulations
├── parsing-utilities           # Java classes for JSON serialization and deserialization
├── permissions                 # Java library for authorizing endpoint requests
├── scheduler-driver            # Java library for goal-oriented scheduling
├── scheduler-server            # Service for scheduling
├── scheduler-worker            # Worker for executing scheduling goals
├── sequencing-server           # Service for sequence generation and management
└── third-party                 # External Java dependencies that are not obtained from Maven
```

## Want to help?

Want to file a bug, contribute some code, or improve documentation? Excellent! Read up on our guidelines for [contributing][contributing]. If you are a developer you can get started quickly by reading the [developer documentation][dev].

## License

The scripts and documentation in this project are released under the [MIT License](LICENSE).

[contributing]: ./docs/CONTRIBUTING.md
[deployment]: ./deployment
[dev]: ./docs/DEVELOPER.md
[fast-track]: https://nasa-ammos.github.io/aerie-docs/introduction/#fast-track
[ui-repo]: https://github.com/NASA-AMMOS/aerie-ui


# Adobe Analytics 2.0 API Documentation

This is the source repository for the [Adobe Analytics 2.0 API documentation](https://developer.adobe.com/analytics-apis/docs/2.0/). See the [Contributing](.github/CONTRIBUTING.md) page to learn how to make edits or improvements to this repository so they are reflected in the published documentation.

Most help content is located under `src/pages/guides/endpoints/`.

This site was built with [Adobe Developer Gatsby Theme](https://github.com/adobe/gatsby-theme-aio).

<!-- View the [demo](https://adobedocs.github.io/dev-site-documentation-template/) running on Github Pages.

Follow the [instructions](https://github.com/adobe/gatsby-theme-aio#getting-started) to get started. -->


---
copyright: Copyright (c) Runtime Verification, Inc. All Rights Reserved.
permalink: README.html
---

[Join the chat on Matrix](https://matrix.to/#/#k:matrix.org)

# Introduction

The K Framework is a tool for designing and modeling programming languages
and software/hardware systems.
At the core of the K Framework is a programming, modeling, and specification
language called K.
The K Framework includes tools for compiling K specifications to build
interpreters, model checkers, verifiers, associated documentation, and more.

## Quick Start

If you are not a K developer, but just want to get started using K, we provide a
streamlined installation process for any system that supports
[Nix](https://nixos.org/download.html):
```shell
bash <(curl https://kframework.org/install)
kup install k
```

For more information on the `kup` tool and other packaged releases of K, please
refer to our [installation notes](k-distribution/INSTALL.md).

## Preface

This is a readme file for _K developers_. Users should feel comfortable using
the command line, as we do not provide GUI tools at this time.

_K-based tool users_ should:

1.  Consult their tool documentation for build/installation instructions.
2.  If needed, download a [packaged release](https://github.com/runtimeverification/k/releases/)
    of the K Framework as part of their tool setup process.

If you are interested in quickly trying out the K Framework without building
from source, please see our
[packaged release installation guide](https://github.com/runtimeverification/k/blob/master/k-distribution/INSTALL.md).

The rest of this file assumes you intend to build and install the K Framework
from source.

Note that the K Framework can only be built on (x86-64) Linux-like systems,
e.g., this also includes macOS/brew (x86-64) as well as the Windows Subsystem
for Linux.
All 32-bit systems are **not supported**.
See the
[installation notes](https://github.com/runtimeverification/k/blob/master/k-distribution/INSTALL.md)
for details about supported configurations and system setup.

## Contents

1.  [Prerequisite Install Guide](#prerequisite-install-guide)
2.  [Build and Install Guide](#build-and-install-guide)
3.  [IDE Setup](#ide-setup)
4.  [Running the Test Suite](#running-the-test-suite)
5.  [Changing the KORE Data Structures](#changing-the-kore-data-structures)
6.  [Building the Final Release Directory/Archives](#building-the-final-release-directoryarchives)
7.  [Compiling Definitions and Running Programs](#compiling-definitions-and-running-programs)
8.  [Installing Python Support](#installing-python-support)
9.  [Troubleshooting](#troubleshooting)

# Prerequisite Install Guide

Before building and installing the K Framework, the following prerequisites
must first be installed.

## The Short Version

On Ubuntu Linux 22.04 (Jammy):

```shell
git submodule update --init --recursive
sudo apt-get install    \
    bison               \
    build-essential     \
    clang-15            \
    cmake               \
    curl                \
    flex                \
    g++                 \
    gcc                 \
    libboost-test-dev   \
    libfmt-dev          \
    libgmp-dev          \
    libjemalloc-dev     \
    libmpfr-dev         \
    libsecp256k1-dev    \
    libyaml-dev         \
    libz3-dev           \
    lld-15              \
    llvm-15-tools       \
    m4                  \
    maven               \
    openjdk-17-jdk      \
    pkg-config          \
    python3             \
    python3-dev         \
    z3                  \
    zlib1g-dev
curl -sSL https://get.haskellstack.org/ | sh
```

If you install this list of dependencies, continue directly to the [Build and Install Guide](#build-and-install-guide).

On macOS using [Homebrew](https://brew.sh/):

```shell
git submodule update --init --recursive
brew install    \
    bison       \
    boost       \
    cmake       \
    flex        \
    fmt         \
    gcc         \
    gmp         \
    openjdk     \
    jemalloc    \
    libyaml     \
    llvm        \
    make        \
    maven       \
    mpfr        \
    pkg-config  \
    python      \
    secp256k1   \
    stack       \
    zlib        \
    z3
```

## The Long Version

The following dependencies are needed either at build time or runtime:

*   [bison](https://www.gnu.org/software/bison/)
*   [boost](https://www.boost.org/)
*   [cmake](https://cmake.org/)
*   [flex](https://github.com/westes/flex)
*   [fmt](https://fmt.dev/)
*   [gcc](https://gcc.gnu.org/)
*   [gmp](https://gmplib.org/)
*   [jdk](https://openjdk.java.net/) (version 17 or greater)
*   [libjemalloc](https://github.com/jemalloc/jemalloc)
*   [libsecp256k1](https://github.com/bitcoin-core/secp256k1)
*   [libyaml](https://pyyaml.org/wiki/LibYAML)
*   [llvm](https://llvm.org/) (We require version 15 or greater for clang, lld, and llvm-tools. On some distributions, the utilities below are also needed and packaged separately.)
    * [clang](http://clang.llvm.org/)
    * [lld](https://lld.llvm.org/)
*   [make](https://www.gnu.org/software/make/)
*   [maven](https://maven.apache.org/)
*   [mpfr](http://www.mpfr.org/)
*   [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/)
*   [python](https://www.python.org)
*   [stack](https://docs.haskellstack.org/en/stable/README/)
*   [zlib](https://www.zlib.net/)
*   [z3](https://github.com/Z3Prover/z3) (on some distributions libz3 is also
    needed and packaged separately) Note that you need version 4.12.1 of Z3,
    which may require you to build and install from source if your package
    manager does not supply it. Other versions are known to have bugs and
    performance regressions likely to cause issues in the K test suite.

Typically, these can all be installed from your package manager.
On some system configurations, special installation steps or post-installation
configuration steps are required.
See the notes below.

### Installation Notes

1.  Java Development Kit (required JDK 17 or higher)

    *   Linux: Download from package manager
        (e.g. `sudo apt-get install openjdk-17-jdk`).

    *   macOS/brew: Download from package manager
        (e.g. `brew install java`).

    To make sure that everything works you should be able to call
    `java -version` and `javac -version` from a terminal.

2.  LLVM

    *   macOS/brew: Since LLVM is distributed as a keg-only package, we must
        explicitly make it available for command line usage. See the results
        of the `brew info llvm` command for more information on how to do this.

3.  Flex / Bison

    *   macOS/brew: The versions of these packages supplied by the OS are too
        old, and are not compatible with the K build. You must ensure that the
        Homebrew-installed versions are first on your `PATH` when building K
        (i.e. `which flex` is **not** `/usr/bin/flex`).

4.  Apache Maven

    *   Linux: Download from package manager
        (e.g. `sudo apt-get install maven`).

    *   macOS/brew: Download it from a package manager or from
        http://maven.apache.org/download.cgi and follow the instructions on
        the webpage.

    Maven usually requires setting an environment variable `JAVA_HOME` pointing
    to the installation directory of the JDK (not to be mistaken with JRE).

    You can test if it works by calling `mvn -version` in a terminal.
    This will provide the information about the JDK Maven is using, in case
    it is the wrong one.

5.   Haskell Stack

     To install, go to <https://docs.haskellstack.org/en/stable/README/> and
     follow the instructions.
     You may need to do `stack upgrade` to ensure the latest version of Haskell
     Stack.

# Build and Install Guide

## Building with Maven

Checkout the project source at your desired location and call `mvn package`
from the main directory to build the distribution. For convenient usage, you
can update your `$PATH` with
`<checkout-dir>/k-distribution/target/release/k/bin`
(strongly recommended, but optional).

You are also encouraged to set the environment variable `MAVEN_OPTS` to
`-XX:+TieredCompilation`, which will significantly speed up the incremental
build process.

### Apple Silicon Support

K is fully tested and supported on ARM (M1/M2) family macOS machines. However,
to work around an [upstream
bug](https://github.com/commercialhaskell/stack/issues/4373) in the Haskell /
Stack ecosystem, care needs to be taken when initially building K from source.
Before running any Maven commands, the Haskell Stack build needs to be
configured without Homebrew's LLVM appearing on your `$PATH`:

First, run the following command from the K source root:
```shell
cd haskell-backend/src/main/native/haskell-backend && stack setup && cd -
```

Then, ensure that Homebrew-installed versions of `llvm-config`, `flex` and
`bison` are on your `PATH` ahead of any macOS-supplied versions.
[`direnv`](https://direnv.net/) offers a convenient way to automate this. To do
so:
```shell
brew install direnv
# Follow the instructions at https://direnv.net/docs/hook.html
# ...for example, if your shell is bash, run:
#   echo 'eval "$(direnv hook bash)"' >> ~/.bashrc
# then restart your shell.
cp macos-envrc .envrc
direnv allow
# You should see a message like:
#   direnv: loading .../k/.envrc
#   direnv: export ~PATH
# The llvm-config binary should also be on your PATH; check with:
which llvm-config
```

If you subsequently encounter errors when building the Haskell components of K
(the Haskell backend and booster), try removing the entire Stack cache
(`~/.stack`) and retrying the instructions above.

## Building with Nix flakes (Recommended)

We now support building K using [nix flakes](https://nixos.wiki/wiki/Flakes).
To set up nix flakes you will need to be on `nix` 2.4 or higher and follow the instructions [here](https://nixos.wiki/wiki/Flakes).

For example, if you are on a standard Linux distribution, such as Ubuntu, first [install nix](https://nixos.org/download.html#download-nix)
and then enable flakes by editing either `~/.config/nix/nix.conf` or `/etc/nix/nix.conf` and adding:

```
experimental-features = nix-command flakes
```

This is needed to expose the Nix 2.0 CLI and flakes support that are hidden behind feature-flags.

By default, Nix will build the project and its transitive dependencies from
source, which can take up to an hour. We recommend setting up
[the binary cache](https://app.cachix.org/cache/kore) to speed up the build
process significantly. You will also need to add the following sections to `/etc/nix/nix.conf` or, if you are a trusted user, `~/.config/nix/nix.conf` (if you don't know what a "trusted user" is, you probably want to do the former):

```
trusted-public-keys = ... hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=
substituters = ... https://cache.iog.io
```

i.e. if the file was originally

```
substituters = https://cache.nixos.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
```

it will now read

```
substituters = https://cache.nixos.org https://cache.iog.io
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=
```

To build the K Framework itself, run:

```bash
nix build .
```

This will build all of K and put a link to the resulting binaries in the `result/` folder.

_Note: Mac users, especially those running M1/M2 Macs may find nix segfaulting on occasion. If this happens, try running the nix command like this: `GC_DONT_GC=1 nix build .`_

If you want to temporarily add the K binaries (such as `kompile` or `kast`) to the current shell, run

```bash
nix shell .
```

To run the integration tests:

```
nix build .#test
```

If you change any `pom.xml`, you must run

```
nix run .#update-maven
```

and commit the updated `nix/mavenix.lock` file.

# IDE Setup

## General

You should run K from the k-distribution project, because it is the only project to have the complete
classpath and therefore all backends.

## IntelliJ IDEA

IntelliJ IDEA comes with built-in maven integration. For more information, refer to
the [IntelliJ IDEA wiki](http://wiki.jetbrains.net/intellij/Creating_and_importing_Maven_projects)

# Running the Test Suite

To completely test the current version of the K framework, run `mvn verify`.
This normally takes roughly 30 minutes on a fast machine. If you are interested only
in running the unit tests and checkstyle goals, run `mvn verify -DskipKTest` to
skip the lengthy `ktest` execution.

# Building the Final Release Directory/Archives
Call `mvn install` in the base directory. This will attach an artifact to the local
maven repository containing a zip and tar.gz of the distribution.

The functionality to create a tagged release is currently incomplete.

# Compiling Definitions and Running Programs
Assuming k-distribution/target/release/k/bin is in your path, you can compile definitions using
the `kompile` command.  To execute a program you can use `krun`.

For running either program in the debugger, use the main class `org.kframework.main.Main` with an additional argument `-kompile` or `-krun` added before other command line arguments, and use the classpath from the `k-distribution` module.

# Installing Python Support

Python tools for K can be found under [runtimeverification/pyk](https://github.com/runtimeverification/pyk).

# Troubleshooting
Common build-time error messages:

-   `Error: JAVA_HOME not found in your environment.
     Please set the JAVA_HOME variable in your environment to match the
     location of your Java installation.`
    + Make sure `JAVA_HOME` points to the JDK and not the JRE directory.

-   `[WARNING] Cannot get the branch information from the git repository:
     Detecting the current branch failed: 'git' is not recognized as an internal or external command,
     operable program or batch file.`
    + `git` might not be installed on your system. Make sure that you can execute
      `git` from the command line.

-   `1) Error injecting constructor, java.lang.Error: Unresolved compilation problems:
          The import org.kframework.parser.outer.Outer cannot be resolved
          Outer cannot be resolved`
    + You may run into this issue if target/generated-sources/javacc is not added to the
      build path of your IDE. Generally this is solved by regenerating your project /
      re-syncing it with the pom.xml.

-   `[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run
     (build-haskell) on project haskell-backend: An Ant BuildException has occured: exec returned: 1`

    and scrolling up, you see an error message similar to:

    `[exec] Installing GHC ...
     [exec] ghc-pkg: Couldn't open database $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.1/lib/ghc-8.10.1/package.conf.d for modification:
     {handle: $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.1/lib/ghc-8.10.1/package.conf.d/package.cache.lock}:
     hLock: invalid argument (Invalid argument)`
    + If you are using a [WSL version 1 environment](https://docs.microsoft.com/en-us/windows/wsl/compare-versions),
      then you have encountered a known issue with the latest versions of GHC. In this
      case, please either:
      -   upgrade to [WSL version 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10),
      -   install a [packaged release for your WSL version 1 distribution](https://github.com/runtimeverification/k/releases/),
      -   switch to a supported system configuration (e.g. Linux on a virtual machine), or
      -   if you do not need the symbolic execution capabilities of the K Framework, disable them
          at build time (and remove the GHC dependency) by doing: `mvn package -Dhaskell.backend.skip`.

-   When building with nix and getting:
    -   ```bash
        error: hash mismatch in fixed-output derivation '/nix/store/wjz7gjqs3cch9lgdjhs1fnb8wfl352vd-k-6.1.0-dirty-maven-deps.drv':
                specified: sha256-kLpjMj05uC94/5vGMwMlFzLKNFOKey◊Nvq/vmB6pHTAo=
                    got:    sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=
        error: 1 dependencies of derivation '/nix/store/79hazjbxp8829wpjvhh9c7kzc1m0ii22-k-6.1.0-dirty.drv' failed to build
        ```

        copy the `got:` hash (`sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=`) and replace it in `flake.nix`:

        ```nix
        k-framework = { haskell-backend-bins, llvm-kompile-libs }:
            prev.callPackage ./nix/k.nix {
                mvnHash = "sha256-fFlRqlLDZnVuoJniPvXjqdYEjnKxmFCEniavau/1gcQ=";
                ...
        ```
    -   `[ERROR] Failed to execute goal ... org.apache.maven.artifact.resolver.ArtifactNotFoundException: The following artifacts could not be resolved: org.scala-lang:scala-compiler:jar:2.12.18 ...`
        Add `"org.scala-lang:scala-compiler:2.12.18"` (without the `jar:`) to `manualMvnArtifacts` in `flake.nix`

If something unexpected happens and the project fails to build, try `mvn clean` and
rebuild the entire project. Generally speaking, however, the project should build incrementally
without needing to be cleaned first.

If you are doing work with snapshot dependencies, you can update them to the latest version by
running maven with the `-U` flag.

If you are configuring artifacts in a repository and need to purge the local repository's cache
of artifacts, you can run `mvn dependency:purge-local-repository`.

If tests fail but you want to run the build anyway to see what happens, you can use `mvn package -DskipTests`.

If you still cannot build, please contact a K developer.


// SPDX-License-Identifier: MIT

:toc:
:toclevels: 4
:toc-placement!:

[link=https://github.com/mercedes-benz/sechub/actions?workflow=Java+%26+Go+CI]
image::https://github.com/mercedes-benz/sechub/workflows/Java%20&%20Go%20CI/badge.svg[Build status]

toc::[]

== SecHub

image::sechub-doc/src/docs/asciidoc/images/sechub-logo.png["Eugen" - the SecHub mascot]

The free and open-source security platform SecHub, provides a central API to test software with different security tools. Many free and open-source as well proprietary security tools are supported by SecHub.

SecHub features:

* <<user-perspective, Easy to use>>
* <<user-perspective, Scan using one API/client>>
* <<report, Single human readable report>>
* <<mark-false-positives, Mark findings as false-positive>>
* <<modules, Supports many security tools>>
* <<sechub-plugins, Provides IDE and text editor plugins>>

Supported security tools:

* <<codescan, Code scanners>>
* <<secretscan, Secrets scanners>>
* <<webscan, Web scanners>>
* <<infrascan, Infrastructure scanners>>
* <<licensescan, License scanners>>

=== Getting Started

* https://mercedes-benz.github.io/sechub/latest/sechub-getting-started.html[SecHub Getting Started Guide]

=== Installation

Please visit https://github.com/mercedes-benz/sechub/wiki/ for detailed information.

=== Documentation

* https://mercedes-benz.github.io/sechub/[Documentation]

=== Introduction

SecHub orchestrates various security and vulnerability scanners which can find potential vulnerabilities in sourcecode, binaries or web applications.
This enables security, development and operation teams to review and fix security issues. As a result, SecHub improves application security.

.SecHub basic architecture overview
[[figure-architecture-overview]]
[ditaa]
....
                                                   +--------------+
                                              +--> | PDS + Tool A |
                                              |    +--------------+
+--------+                     +---------+    |
| SecHub | ---- scan data ---> | SecHub  | <--+
| Client | <---- report ------ |   API   | <--+
+--------+                     +---------+    |
                                              |    +--------------+
                                              +--> | PDS + Tool B |
                                                   +--------------+
....

The objective of SecHub is to help secure the software development lifecyle (SDLC) phases: development, deployment and maintenance. From the first written line of code to the application being in production. SecHub can be used to scan the software continuously.

The security tools are categorized into modules which are named after the security testing method they perform: `codeScan`, `licenseScan`, `secretScan`, `webScan` etc.

[NOTE]
--
The terms SAST (Static Application Security Testing) and DAST (Dynamic Application Security Testing) are intentionally not used for the module names, because the designers feel those terms are vague and difficult to understand for non-security experts. On the other hand, security experts can easily map: `codeScan` to SAST and `webScan` to DAST.
--

Because of the modules, the user only needs to understand what security testing a module performs, rather than to know what specific security tools are used for the actual scan. The user describes in a configuration file what module(s) to use. Or in other words, what security testing methods should be used to test the software with.

To start a scan, the user sends the configuration file to the SecHub server via API.
For ease of use SecHub offers a https://github.com/mercedes-benz/sechub/releases?q=%22client+version%22&expanded=true[client] which calls the REST API for the user.
The SecHub client can be used manually or can be integrated into a continuous-integration build pipeline (see below figure).

.CI/CD systems or users can scan with SecHub
[[figure-cicd-user-sechub]]
[ditaa]
....
/------\
| User | ----+
\------/     |            +--------+
             +-- Scan --> |        |
                          | SecHub |
             +-- Scan --> |        |
/-------\    |            +--------+
| CI/CD | ---+
\-------/
....

On receiving the user request, SecHub creates a job, delegates it to one or more security tools, collects the results and converts all into a report.
Next, the user can download the <<report,report>> in a JSON or HTML format.

[[user-perspective]]
=== User Perspective

SecHub is designed to provide an efficient user workflow. The basic idea of SecHub is, that the user has to do as little as possible to execute a security scan.
The user has two options: +
a) to use the https://mercedes-benz.github.io/sechub/latest/sechub-restapi.html[REST API] directly +
b) to use the https://mercedes-benz.github.io/sechub/latest/sechub-client.html[SecHub client]. +
Both can be integrated into a CI/CD pipeline.

Using the REST API requires several steps, which is fine if SecHub needs to be integrated into another software or platform.

However, it is recommended to use the SecHub client. +
The SecHub client reduces the workflow to three steps:

.SecHub three steps to scan
[[figure-three-steps-to-scan]]
[ditaa]
....

1. Create a configuration file  // (1)

/------+
| JSON |
+------/

2. Set Credentials // (2)

export SECHUB_USERID=myUserName…
…

3. Scan // (3)

+--------+                 +--------+
|        | ---- scan ----> |        |
| Client |                 | SecHub |
|        | <-- report ---- |        |
+--------+                 +--------+
....

(1) Create a https://mercedes-benz.github.io/sechub/latest/sechub-client.html#section-client-configuration-file[SecHub configuration] file. This step only needs to be done the first time.

(2) Provide the SecHub credentials. +
Example: +
`export SECHUB_USERID=myUserName` +
`export SECHUB_APITOKEN=NTg5YSMkGRkM2Uy00NDJjLTkYTY4NjEXAMPLE` +
`export SECHUB_SERVER=https://sechub.example.com:8443`

(3) Scan using `sechub scan`

Once the scan is finished, the client returns a <<report,report>>.

If the client is used to scan asynchronously it will return a `jobUUID` which can be used to get the report:

.Scan asynchronously
[[figure-scan-asynchronously]]
[ditaa]
....

1. Scan asynchronously // (1)

+--------+                     +--------+
|        | --- scanAsync ----> |        |
| Client |                     | SecHub |
|        | <--- jobUUID ------ |        |
+--------+                     +--------+

2. GetReport // (2)

+--------+                     +--------+
|        | --- getReport ----> |        |
| Client |                     | SecHub |
|        | <--- report ------- |        |
+--------+                     +--------+
....
(1) Scan asynchronously using `sechub scanAsync`. +
(2) Get report `sechub -jobUUID <jobUUID> getReport`.

In general, the `jobUUID` can be used to download the report again and again by different users and in different formats.

[[report]]
=== Report

SecHub collects the scan results from various security tools and converts them into a unified reporting format called: SecHub Report. The advantage is that the user needs to learn only one report format. The json-report below shows how a report can look like:

.JSON report example based on a scan of the https://securego.io/docs/rules/g101.html[G101], https://securego.io/docs/rules/g103.html[G103], https://securego.io/docs/rules/g304.html[G304] examples from GoSec.
[[json-report]]
[json]
----
{
   "result": {
      "count": 4,
      "findings": [
         {
            "id": 1,
            "description": "Potential hardcoded credentials",
            "name": "Potential hardcoded credentials",
            "severity": "HIGH",
            "code": {
               "location": "examples/g101.go",
               "line": 7,
               "column": 9,
               "source": "var password = \"f62e5bcda4fae4f82370da0c6f20697b8f8447ef\""
            },
            "type": "codeScan",
            "cweId": 798
         },
         {
            "id": 2,
            "description": "Use of unsafe calls should be audited",
            "name": "Use of unsafe calls should be audited",
            "severity": "MEDIUM",
            "code": {
               "location": "examples/g103.go",
               "line": 16,
               "column": 21,
               "source": "intPtr = (*int)(unsafe.Pointer(addressHolder))"
            },
            "type": "codeScan",
            "cweId": 242
         },
         {
            "id": 3,
            "description": "Use of unsafe calls should be audited",
            "name": "Use of unsafe calls should be audited",
            "severity": "MEDIUM",
            "code": {
               "location": "examples/g103.go",
               "line": 15,
               "column": 30,
               "source": "addressHolder := uintptr(unsafe.Pointer(intPtr)) + unsafe.Sizeof(intArray[0])"
            },
            "type": "codeScan",
            "cweId": 242
         },
         {
            "id": 4,
            "description": "Use of unsafe calls should be audited",
            "name": "Use of unsafe calls should be audited",
            "severity": "MEDIUM",
            "code": {
               "location": "examples/g103.go",
               "line": 15,
               "column": 56,
               "source": "addressHolder := uintptr(unsafe.Pointer(intPtr)) + unsafe.Sizeof(intArray[0])"
            },
            "type": "codeScan",
            "cweId": 242
         }
      ]
   },
   "messages": [],
   "reportVersion": "1.0",
   "trafficLight": "RED",
   "status": "SUCCESS",
   "jobUUID": "15a96c07-dcf3-4cbc-8d82-0acc9facd3a6"
}
----

The report can be downloaded in two flavors: JSON and HTML. Both are human readable. The HTML report is self-contained and can be read in any browser.
The JSON format is machine readable and can be read by the <<sechub-plugins, SecHub plugins>>.

[[mark-false-positives]]
==== Mark Findings as False-Positives

There are two major reasons for marking a security finding as false-positive: +
- It is an actual false-positive. +
- The finding is a false-positive in the context of the application. For example, the application is never deployed to be reachable from the internet.

Regardless the reason, https://mercedes-benz.github.io/sechub/latest/sechub-client.html#section-client-false-positives-mark[SecHub supports marking findings as false-positives]. The marking of false-positives is a SecHub feature and is independent of the security tools used to scan.

[[sechub-plugins]]
==== SecHub Plugins

The SecHub Plugins improve the user experience by enabling the user to work directly with the SecHub report in the IDE or text editor.

SecHub plugins exist for the following text editors and IDEs:

* Plugin for https://marketplace.eclipse.org/content/sechub[Eclipse IDE] (https://github.com/mercedes-benz/sechub-plugin-eclipse[source code])
* Plugin for https://github.com/mercedes-benz/sechub-plugin-intellij[IntelliJ platttform] (https://github.com/mercedes-benz/sechub-plugin-intellij[source code])
* Plugin for https://open-vsx.org/extension/mercedes-benz/sechub[VSCode, VSCodium, Eclipse Theia] (https://github.com/mercedes-benz/sechub-plugin-vscode[source code])

All plugins are free and open-source software (FOSS) and can be installed directly from within the IDE or text editors.

[[modules]]
=== Modules

Security tools are categorized into modules.

Each module performs a different security testing method:

* <<codescan, `codeScan`>> - scans code or binaries for potential vulnerabilities (weaknesses). +
  This includes SAST (static application security testing) and IaC (infrastructure as code).
* <<infrascan, `infraScan`>> - scans infrastructure for vulnerabilities.
* <<licensescan, `licenseScan`>> - scans code or artifacts for license information.
* <<secretscan, `secretScan`>> - scans code or artifacts for secrets (API tokens, certificates, passwords).
* <<webscan, `webScan`>> - scans a deployed web application for vulnerabilities. Also knows as DAST.

[[codescan]]
==== codeScan

__Alias: Static application security testing (SAST), static code analysis, infrastructure-as-code (IaC) scan__

**Status: Productive**

The `codeScan` module scans source code or binary artifacts for potential vulnerabilities (weaknesses). To scan the user uploads the code or binary to SecHub. Once the files are uploaded, SecHub delegates the scan to one of many security tools.

More details: https://mercedes-benz.github.io/sechub/latest/sechub-client.html#sechub-config-code-scan

PDS-Solutions: +
- GoSec +
- PMD +
- FindSecurityBugs +
- Bandit +
- Checkmarx SAST (wrapper only) +
- … and more

[[secretscan]]
==== secretScan

**Status: Productive**

Scans code or artifacts for secrets (API tokens, certificates, passwords).

PDS-Solutions: +
- Gitleaks

[[webscan]]
==== webScan

__Alias: Dynamic application security testing (DAST)__

**Status: Productive**

The `webScan` module scans running web applications for vulnerabilities. The only requirement is that the web application can be reached by SecHub via network.

More details: https://mercedes-benz.github.io/sechub/latest/sechub-client.html#web-scan

PDS-Solutions: +
- OWASP ZAP

[[infrascan]]
==== infraScan

**Status: Experimental**

The `infraScan` scans systems in a network.

More details: https://mercedes-benz.github.io/sechub/latest/sechub-client.html#infrastructure-scan

[[licensescan]]
==== licenseScan

**Status: Experimental**

The `licenseScan` module scans code or artifacts for license information.

PDS-Solutions: +
- Scancode +
- Tern

=== Architecture

SecHub is designed to execute hundreds of scans. It can scale horizontally and vertically. It can run on bare-metal, virtual machines, kubernetes or in the cloud.

The smallest useful setup is: a single SecHub server and a single product delegation server (PDS). Those two components are enough to start scanning.

For a larger setup, the number of SecHub server instances can be increased. More PDS instances can be added. +
The only requirements to scale SecHub are: a PostgreSQL database and an object store or file share.
SecHub and PDS instances use the PostgreSQL database to share information between instances.
For example, the job queue is kept in PostgreSQL. In addition,an object store or file share is necessary, so that all SecHub or PDS instances can store/read files.

Regardless of the backend complexity, whether one SecHub server or many are used the <<user-perspective, workflow for the user>> stays the same.

For more details about the architecture have a look at the architecture documentation: https://mercedes-benz.github.io/sechub/latest/sechub-architecture.html.

=== Operations Perspective

One needs to configure the scan tools as well as manage users and projects. +
For details please check the https://mercedes-benz.github.io/sechub/latest/sechub-operations.html[operations guide].

=== REST API

All user and administrative tasks can be done via https://mercedes-benz.github.io/sechub/latest/sechub-restapi.html[REST API]. SecHub is designed as a RESTful server.

=== Contributing

We welcome any contributions.
If you want to contribute to this project, please read the link:CONTRIBUTING.md[contributing guide].

=== Code of Conduct

Please read our https://github.com/mercedes-benz/foss/blob/master/CODE_OF_CONDUCT.md[Code of Conduct] as it is our base for interaction.

=== License

This project is licensed under the link:LICENSE[MIT LICENSE].

=== Provider Information

Please visit https://www.mercedes-benz-techinnovation.com/en/imprint/ for information on the provider.

Notice: Before you use the program in productive use, please take all necessary precautions,
e.g. testing and verifying the program with regard to your specific use.
The program was tested solely for our own use cases, which might differ from yours.


[![FINOS - Active](https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-active.svg)](https://community.finos.org/docs/governance/Software-Projects/stages/active)
[![codecov](https://codecov.io/gh/finos/spring-bot/branch/spring-bot-develop/graph/badge.svg?token=CSa0aaqYD0)](https://codecov.io/gh/finos/spring-bot)
![License](https://img.shields.io/github/license/finos/spring-bot)
![build](https://github.com/finos/spring-bot/actions/workflows/build.yml/badge.svg)
[![javadoc](https://javadoc.io/badge2/org.finos.springbot/chat-workflow/javadoc.svg)](https://javadoc.io/doc/org.finos.springbot/chat-workflow)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5948/badge)](https://bestpractices.coreinfrastructure.org/projects/5948)

|Version          |Number                 |Status                  |
|-----------------|-----------------------|------------------------|
|Spring Bot       |[![Maven Central](https://img.shields.io/maven-central/v/org.finos.springbot/spring-bot)](https://search.maven.org/search?q=org.finos.springbot)      |![build](https://github.com/finos/spring-bot/actions/workflows/build.yml/badge.svg)
|Symphony Java Toolkit |[![Maven Central](https://img.shields.io/maven-central/v/org.finos.symphony.toolkit/symphony-java-toolkit)](https://search.maven.org/search?q=org.finos.symphony.toolkit)|![build](https://github.com/finos/spring-bot/actions/workflows/build.yml/badge.svg?branch=symphony-java-toolkit-master)|

![Spring Bot Logo](logo.png)

# Spring Bot

Spring Bot is a set of [Spring Boot](https://spring.io/projects/spring-boot) starters designed for building chat bots in Java & Kotlin.

## What Is It?

 - Spring Bot provides a simple annotation-based way to create chat bots.
 - It currently supports Symphony and Microsoft Teams.
 - It allows you to build your chatbot using annotations, binding messages to methods in a similar way to [Spring Web MVC](https://docs.spring.io/spring-framework/docs/6.0.x/reference/html/web.html#spring-web)

## Why Is This Important?

🧑🏽🤝🧑🏽 **Multiple Platforms**

Like it or not, most of us find ourselves working in environments with _multiple chat platforms_.  As bot developers, we need to be able to reach users irrespective of which platform they are on.

⛅ **Work at a higher level**

Developers of chat-bots are not well-supported with Open-Source tooling.  Too much effort gets spent on low-level message-handling, which could easily be done by a library.

🔒 **Avoid Lock-In**

Most bot development tools are commercial.  If you build bots with them you are locked into paying license fees forever.  For example, the Microsoft Bot Framework support multiple chat platforms at the expense of locking you into the Azure ecosystem.

## How Does It Work?

This is best described at [springbot.finos.org](https://springbot.finos.org).  This takes you through the process of building several example bots from scratch.

## What Are The Benefits?

### ✔ Build Bots that run on multiple platforms

> By using Spring Bot's abstractions, you can target both Symphony and Microsoft Teams.  Depending on demand (or contributions) further platforms may be supported in the future.  In the tutorials we show how to build a [claim processing bot](https://springbot.finos.org/getting-started.html) which runs a workflow concurrently on Teams and Symphony.

### ✔Focus On Business Logic

> Spring Bot handles marshalling from/to the underlying message formats on your platform.

### ✔ Advanced Templating

> Templates are automatically generated for any Java objects (POJOs) that you want to send to the chat platform.  This allows users to read/edit objects and send them back to your bot.  Additionally, templates are fully customizable on a per-platform basis, currently using Freemarker and Thymeleaf.

### ✔Annotations Make It Super-Easy

> As the [tutorials](https://springbot.finos.org) show, you just drop a few annotations into your application to enable it to read and send chat messages.

### ✔ A Mature Ecosystem

> By building bots in a mature ecosystem like Java, you avoid all of the usual issues with no-code and low-code environments.  Java is a mature language with every conceivable library, a wealth of Stack-Overflow QA's and a convincing DevOps/Deployment story (Java runs anywhere).

### ✔ Avoid The Lock-In Trap

> Most low-code / no-code platforms are a trap:  they are looking to lock-in customers and revenue streams by persuading you to build your business with them as a dependency.  Spring Bot is open-source so this is literally a non-issue.

## Watch It In Action

[![Short 3 minute intro](https://img.youtube.com/vi/TyaGdGmqeGo/hqdefault.jpg)](https://youtu.be/TyaGdGmqeGo)

## Plenty Of Example Bots To Learn From

This project contains various example bots that were (originally) written at Deutsche Bank in response to real use-cases.  However, we felt they were general enough to warrant Open-Sourcing and including as code to work from.

👍 All of these bots work on Microsoft Teams and Symphony, and are deployed into maven central as executable jars.

### 🤖 Poll Bot

A bot for running polls in a Symphony/Teams chat room.  👍 _Production Ready_

 - [View the README](tools/poll-bot/README.md)

### 🤖 News (RSS) Bot

A bot for feeding news into a Symphony/Teams chat room.  👍 _Production Ready_

 - [View the README](tools/rss-bot/README.md)

### 🤖 Reminder Bot

Reads the contents of a room and creates reminders based on future events mentioned therein.  👍 _Production Ready_

 - [View the README](tools/reminder-bot/README.md)

## Plenty of Tutorials / Demo Code

Inside the `/demos` directory you can see three example bots:

- `demo-bot` : The "Hello World" of Spring Bot
- `todo-bot` : Runs a simple todo-list inside a chat room on Teams or Symphony.
- `room-bot` : A room broadcasting bot, that shows how to use the chat dropdown, and address messages across Teams and Symphony.
- `claim-bot`: An example of an expense-claim workflow running across Teams and Symphony concurrently.
- `custom-help-bot`: Shows you how you can override built-in beans (like the `HelpController`) to change the functionality of Spring Bot.

Tutorials for building these and the reference guide are found here:  [Spring Bot](https://springbot.finos.org)

# License

Copyright 2022 FINOS

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0

# Vulnerabilities / Security

Please see our [Security Policy](SECURITY.md).

# How To Contribute

We are always looking for helpful contributors!  You can check out the [Good First Issues](https://github.com/finos/spring-bot/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or our [Contributing Guidelines](CONTRIBUTING.md) page for more details.

We have a [weekly Spring Bot project meeting](https://calendar.google.com/calendar/u/0/embed?src=finos.org_fac8mo1rfc6ehscg0d80fi8jig@group.calendar.google.com&ctz=America/New_York) which you can attend.  Minutes are [held as issues](https://github.com/finos/spring-bot/issues).

# Roadmap

The spring bot roadmap is a number of tagged issues that we are working on _next_.

You can view the roadmap issues [here](https://github.com/finos/spring-bot/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap).

# For Maintainers

## Branches

There are two main branches:

- `spring-bot-master`:  new code for the multi-platform Symphony / Microsoft Teams bot builder
- `symphony-java-toolkit-master`: legacy code for when this project only supported symphony.

## Releasing This Project

In order to do a release:

1.  Changes.md

- Update this file to include details of the release.

2. On Releaser's machine:

```
# on spring-bot-master branch
mvn clean test -P symphony-ci
mvn versions:set -DnewVersion=<our breaking change no>.<minor version no>.<our-patch-version-no> -DartifactId=\*  -DgroupId=\*
mvn -DskipTests clean test source:jar javadoc:javadoc deploy -P symphony-release
mvn versions:set -DnewVersion=<our breaking change no>.<symphony-api-version>.<our-release-no+1>-SNAPSHOT -DartifactId=\*  -DgroupId=\*
# then PR spring-bot-master branch with the new version number
```

3. SECURITY.md

- Update this with new version numbers if needed.

4.  On `oss.sonatype.org`

- Close the Staging Repository
- Release it.

5.  Perform release on github with same number


# Stratego

This repository contains the Spoofax implementations of the Stratego language.

1. The `org.metaborg.meta.lang.stratego*` directories contain the old Stratego editor (_Stratego 1_ or _Stratego/XT 0.17_), which uses pre-compiled libraries from the _Stratego/XT_ project that can be found in the `strategoxt` repository of this organisation.
2. The `stratego.lang*` and `stratego.test`/`stratego.example` directories contain the current _Stratego 2_ effort.
3. The `stratego.build*` directories contain the Java code that glues together code in `stratego.lang` into the _Stratego 2_ incremental compiler.
4. The `_cellar` directory contains two (abandoned) experiments that were not clearly older copies of the above projects:
    1. The `stratego.typed*` directories contain an experiment to add more static analysis to _Stratego 1_ with the _NaBL2_ and _FlowSpec_ meta-languages of Spoofax.
    2. The `stratego`/`stratego-test` directories contain an experiment from the days of Spoofax 1, using an old version _NaBL_ to add static analysis. Plus some tests apparently.


Hilla

The modern web framework
for Java

![Latest Stable Version](https://img.shields.io/npm/v/@hilla/frontend.svg)
[![Releases](https://img.shields.io/maven-metadata/v?metadataUrl=https%3A%2F%2Frepo1.maven.org%2Fmaven2%2Fdev%2Fhilla%2Fhilla%2Fmaven-metadata.xml)](https://github.com/vaadin/hilla/releases)

[hilla.dev](https://hilla.dev) · [Docs](https://hilla.dev/docs) · [Forum](https://vaadin.com/forum/c/hilla/18)

---

Hilla integrates a Spring Boot Java backend with a reactive TypeScript front end. It helps you build apps faster with type-safe server communication, included UI components, and integrated tooling.

## Simple type-safe server communication

Hilla helps you access the backend easily with type-safe endpoints.

`index.ts`

```ts
// Type info is automatically generated based on Java
import Person from 'Frontend/generated/com/vaadin/hilla/demo/entity/Person';
import { PersonEndpoint } from 'Frontend/generated/endpoints';

async function getPeopleWithPhoneNumber() {
  const people: Person[] = await PersonEndpoint.findAll();

  // Compile error: The property is 'phone', not 'phoneNumber'
  return people.filter((person) => !!person.phoneNumber);
}

console.log('People with phone numbers: ', getPeopleWithPhoneNumber());
```

`PersonEndpoint.java`

```java
@Endpoint
@AnonymousAllowed
public class PersonEndpoint {

    private PersonRepository repository;

    public PersonEndpoint(PersonRepository repository) {
        this.repository = repository;
    }

    public @Nonnull List<@Nonnull Person> findAll() {
        return repository.findAll();
    }
}
```

`Person.java`

```java
@Entity
public class Person {

    @Id
    @GeneratedValue
    private Integer id;

    @Nonnull private String firstName;
    @Nonnull private String lastName;
    @Email @Nonnull private String email;
    @Nonnull private String phone;

    // getters, setters
}
```

Learn more at [hilla.dev](https://hilla.dev)

## Get started

Follow the tutorials at https://hilla.dev/docs

## Contributing

The best way to contribute is to try out Hilla and provide feedback to the development team in our [Forum](https://vaadin.com/forum/c/hilla/18) or with [GitHub issues](https://github.com/vaadin/hilla/issues).

### Development

If you want to develop Hilla, you can clone the repo and run tests using the following commands:

```sh
git clone https://github.com/vaadin/hilla.git
npm install
npm run build
npm test
```

You need the following versions of Node.js and npm:

- **Node.js**: `>= 18 LTS` (native support for ES Modules and NodeJS execution of the newest hooks),
- **npm**: `^9` (`package-lock.json` is of version 3)

---

![Frontend CI](https://github.com/vaadin/hilla/actions/workflows/ts.yml/badge.svg)
![Java CI](https://github.com/vaadin/hilla/actions/workflows/java.yml/badge.svg)
[![codecov](https://codecov.io/gh/vaadin/hilla/branch/main/graph/badge.svg?token=PQMTMS8ECC)](https://codecov.io/gh/vaadin/hilla)


# Welcome to WSO2 HTTP Transport
This repository holds HTTP protocol handling implementations for C5 based products.

# Introduction
WSO2 middleware products deals with many different transport protocols. HTTP, JMS, AMQP, MQTT are few of the highly used protocols.
This repository contains the HTTP protocol handling implementations which are used by various WSO2 products built on top of C5 (Carbon 5) based products.

# Features
- Provides the implementation for handling transport layer complexity of the message interaction
- Handles the concurrent connections from the source systems and connect with multiple target systems


json-io
=======

[![Maven Central](https://badgen.net/maven/v/maven-central/com.cedarsoftware/json-io)](https://central.sonatype.com/search?q=json-io&namespace=com.cedarsoftware)
[![Javadoc](https://javadoc.io/badge/com.cedarsoftware/json-io.svg)](http://www.javadoc.io/doc/com.cedarsoftware/json-io)

Useful tool for Java serialization to and from JSON format.
Available on [Maven Central](https://central.sonatype.com/search?q=json-io&namespace=com.cedarsoftware).
This library has no dependencies on other libraries for runtime other than our own `java-util.`
The `json-io.jar`file is only`185K` and `java-util` is `260K.` Compatible with JDK1.8 through JDK 22.
## Compatibility

### JPMS (Java Platform Module System)

This library is fully compatible with JPMS, commonly known as Java Modules. It includes a `module-info.class` file that
specifies module dependencies and exports.

### OSGi

This library also supports OSGi environments. It comes with pre-configured OSGi metadata in the `MANIFEST.MF` file, ensuring easy integration into any OSGi-based application.

Both of these features ensure that our library can be seamlessly integrated into modular Java applications, providing robust dependency management and encapsulation.

___
To include in your project:
##### Gradle
```groovy
implementation 'com.cedarsoftware:json-io:4.26.0'
```

##### Maven
```xml
 <dependency>
   <groupId>com.cedarsoftware</groupId>
   <artifactId>json-io</artifactId>
   <version>4.26.0</version>
 </dependency>
```
___

## User Guide
>#### [Usage](/user-guide.md)
>#### [WriteOptions reference](/user-guide-writeOptions.md)
>#### [ReadOptions reference](/user-guide-readOptions.md)
>#### [Revision History](/changelog.md)

## Releases
>### 5.0.0 (future)
>- [ ] **Bundling**: Both JPMS (Java Platform Module System) and OSGi (Open Service Gateway initiative)
>- [ ] **Maintained**: Fully
>- [ ] **Java Package**: com.cedarsoftware.io
>- [ ] **Java**: JDK17+ (Class file 52 format, includes module-info.class - multi-release JAR)
>- [ ] **API**
   >  - Static methods on [JsonIo](https://www.javadoc.io/doc/com.cedarsoftware/json-io/latest/com/cedarsoftware/io/JsonIo.html): [toJson()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#toJson(java.lang.Object,com.cedarsoftware.io.WriteOptions)), [toObjects()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#toObjects(java.lang.String,com.cedarsoftware.io.ReadOptions,java.lang.Class)), [formatJson()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#formatJson(java.lang.String,com.cedarsoftware.io.ReadOptions,com.cedarsoftware.io.WriteOptions)), [deepCopy()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#deepCopy(java.lang.Object,com.cedarsoftware.io.ReadOptions,com.cedarsoftware.io.WriteOptions))
>  - Use [ReadOptionsBuilder](/user-guide-readOptions.md) and [WriteOptionsBuilder](/user-guide-writeOptions.md) to configure `JsonIo`
>  - Use [JsonReader.ClassFactory](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonReader.ClassFactory.html) for difficult classes (hard to instantiate & fill)
>  - Use [JsonWriter.JsonClassWriter](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonWriter.JsonClassWriter.html) to customize the output JSON for a particular class
>- [ ] Updates will be 5.1.0, 5.2.0, ...
>### 4.26.0 (current)
>- [ ] **Version**: [4.26.0](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.26.0/index.html)
>- [ ] **Bundling**: Both JPMS (Java Platform Module System) and OSGi (Open Service Gateway initiative)
>- [ ] **Maintained**: Fully
>- [ ] **Java Package**: com.cedarsoftware.io
>- [ ] **Java**: JDK1.8+ (Class file 52 format, includes module-info.class - multi-release JAR)
>- [ ] **API**
>  - Static methods on [JsonIo](https://www.javadoc.io/doc/com.cedarsoftware/json-io/latest/com/cedarsoftware/io/JsonIo.html): [toJson()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#toJson(java.lang.Object,com.cedarsoftware.io.WriteOptions)), [toObjects()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#toObjects(java.lang.String,com.cedarsoftware.io.ReadOptions,java.lang.Class)), [formatJson()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#formatJson(java.lang.String,com.cedarsoftware.io.ReadOptions,com.cedarsoftware.io.WriteOptions)), [deepCopy()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonIo.html#deepCopy(java.lang.Object,com.cedarsoftware.io.ReadOptions,com.cedarsoftware.io.WriteOptions))
>  - Use [ReadOptionsBuilder](/user-guide-readOptions.md) and [WriteOptionsBuilder](/user-guide-writeOptions.md) to configure `JsonIo`
>  - Use [JsonReader.ClassFactory](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonReader.ClassFactory.html) for difficult classes (hard to instantiate & fill)
>  - Use [JsonWriter.JsonClassWriter](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.26.0/com/cedarsoftware/io/JsonWriter.JsonClassWriter.html) to customize the output JSON for a particular class
>- [ ] Updates will be 4.27.0, 4.28.0, ...
>### 4.14.x (supported)
>- [ ] **Version**: [4.14.3](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/index.html)
>- [ ] **Bundling**: Both JPMS (Java Platform Module System) and OSGi (Open Service Gateway initiative)
>- [ ] **Maintained**: Bug fixes, CVE's
>- [ ] **Java Package**: com.cedarsoftware.util.io
>- [ ] **Java**: JDK1.8+ (Class file 52 format, includes module-info.class - multi-release JAR)
>- [ ] **API**
>  - Static methods on [JsonReader](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.html): [jsonToJava()](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.html#jsonToJava-java.lang.String-java.util.Map-), [jsonToMaps()](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.html#jsonToMaps-java.lang.String-java.util.Map-), special: [jsonObjectsToJava()](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.html#jsonObjectsToJava-com.cedarsoftware.util.io.JsonObject-)
>  - Static methods on [JsonWriter](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonWriter.html): [objectToJson()](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonWriter.html#objectToJson-java.lang.Object-java.util.Map-)), [formatJson()](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonWriter.html#formatJson-java.lang.String-java.util.Map-java.util.Map-)
>  - Configuration via `Map` with constants defined in [JsonReader](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.14.3/constant-values.html#com.cedarsoftware.util.io.JsonReader.CLASSLOADER)/[JsonWriter](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.14.3/constant-values.html#com.cedarsoftware.util.io.JsonWriter.CLASSLOADER).
>  - Use [JsonReader.JsonClassReader](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.JsonClassReader.html) or [JsonReader.JsonClassReaderEx](https://www.javadoc.io/doc/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.JsonClassReaderEx.html) for customer reader
>  - Use [JsonReader.assignInstantiator()](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonReader.html#assignInstantiator-java.lang.Class-com.cedarsoftware.util.io.JsonReader.Factory-) when `json-io` cannot construct a particular class
>  - Use [JsonWriter.JsonClassWriter](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonWriter.JsonClassWriter.html) or [JsonWriter.JsonClassWriterEx](https://www.javadoc.io/static/com.cedarsoftware/json-io/4.14.3/com/cedarsoftware/util/io/JsonWriter.JsonClassWriterEx.html) to customize particular class output JSON
>- [ ] Updates will be 4.14.4, 4.14.5, ...

Featured on [json.org](http://json.org).

For useful Java utilities, check out [java-util](http://github.com/jdereg/java-util)
___
### Sponsors
[![Alt text](https://www.yourkit.com/images/yklogo.png "YourKit")](https://www.yourkit.com/.net/profiler/index.jsp)

YourKit supports open source projects with its full-featured Java Profiler.
YourKit, LLC is the creator of YourKit Java Profiler
and YourKit .NET Profiler,
innovative and intelligent tools for profiling Java and .NET applications.

**Intellij IDEA**
___
### License
```
Copyright (c) 2007 Cedar Software LLC.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    <a href="http://www.apache.org/licenses/LICENSE-2.0">License</a>

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```

by John DeRegnaucourt and Kenny Partlow


# Micronaut OpenAPI #

[![Maven Central](https://img.shields.io/maven-central/v/io.micronaut.openapi/micronaut-openapi.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:%22io.micronaut.openapi%22%20AND%20a:%22micronaut-openapi%22)
[![Build Status](https://github.com/micronaut-projects/micronaut-openapi/workflows/Java%20CI/badge.svg)](https://github.com/micronaut-projects/micronaut-openapi/actions)
[![Revved up by Develocity](https://img.shields.io/badge/Revved%20up%20by-Develocity-06A0CE?logo=Gradle&labelColor=02303A)](https://ge.micronaut.io/scans)

This project allows generating OpenAPI 3.x (Swagger) specifications for a Micronaut application at compilation time. By moving the generation of OpenAPI specs into the compiler this project allows Micronaut applications to save on memory and avoid the need for costly computation at runtime unlike most implementations of OpenAPI which rely on processing to occur on the server.

## Documentation ##

See the [Documentation](https://micronaut-projects.github.io/micronaut-openapi/latest/guide/index.html) for more information.

See the [Snapshot Documentation](https://micronaut-projects.github.io/micronaut-openapi/snapshot/guide/index.html) for the latest development version documentation.

## Examples

You can generate example applications at https://launch.micronaut.io by selecting the `Add Feature` button and the `openapi` feature then generate!

## Snapshots and Releases

Snapshots are automatically published to [Sonatype Snapshots](https://s01.oss.sonatype.org/content/repositories/snapshots/io/micronaut/) using [GitHub Actions](https://github.com/micronaut-projects/micronaut-openapi/actions).

See the documentation in the [Micronaut Docs](https://docs.micronaut.io/latest/guide/index.html#usingsnapshots) for how to configure your build to use snapshots.

Releases are published to Maven Central via [GitHub Actions](https://github.com/micronaut-projects/micronaut-openapi/actions).

A release is performed with the following steps:

* [Publish the draft release](https://github.com/micronaut-projects/micronaut-openapi/releases). There should be already a draft release created, edit and publish it. The Git Tag should start with `v`. For example `v1.0.0`.
* [Monitor the Workflow](https://github.com/micronaut-projects/micronaut-openapi/actions?query=workflow%3ARelease) to check it passed successfully.
* Celebrate!


# Micronaut Maven Plugin

[![Maven Central](https://img.shields.io/maven-central/v/io.micronaut.maven/micronaut-maven-plugin.svg?label=Maven%20Central)](https://search.maven.org/artifact/io.micronaut.maven/micronaut-maven-plugin)
[![Build Status](https://github.com/micronaut-projects/micronaut-maven-plugin/actions/workflows/snapshot.yml/badge.svg)](https://github.com/micronaut-projects/micronaut-maven-plugin/actions)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=io.micronaut.maven%3Amicronaut-maven-plugin-parent&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=io.micronaut.maven%3Amicronaut-maven-plugin-parent)
[![Revved up by Develocity](https://img.shields.io/badge/Revved%20up%20by-Develocity-06A0CE?logo=Gradle&labelColor=02303A)](https://ge.micronaut.io/scans)

Maven plugin to execute Micronaut applications.

## Documentation

See the [Documentation](https://micronaut-projects.github.io/micronaut-maven-plugin/latest) for more information.

See the [Snapshot Documentation](https://micronaut-projects.github.io/micronaut-maven-plugin/snapshot) for the current development docs.

To see the documentation of a particular version, replace `latest` or `snapshot` in the above URLs with the version string.

## Snapshots and Releases

Snaphots and releases are automatically published to Maven Central using [GitHub Actions](https://github.com/micronaut-projects/micronaut-maven-plugin/actions).

A release is performed with the following steps:

* [Publish the draft release](https://github.com/micronaut-projects/micronaut-maven-plugin/releases). There should be already a draft release created, edit and publish it. The Git Tag should start with `v`. For example `v1.0.0`.
* [Monitor the Workflow](https://github.com/micronaut-projects/micronaut-maven-plugin/actions?query=workflow%3ARelease) to check it passed successfully.
* Celebrate!

## Contributing tips and tricks

### Running integration tests

You can run integration tests by executing `mvn verify`

If you want to run individual tests, you can execute `mvn verify "-Dinvoker.test=dockerfile*"`. In this case,
`dockerfile*` will match all test projects under `src/it` folder with a name that starts with "dockerfile".

### Debugging

To debug the plugin, you first need to publish a snapshot to your Maven local:

```shell
$ mvn install
```

You can skip execution of integration tests by adding `-Dinvoker.skip=true` to the command line.

Then you need a sample application. The one at `examples/java` is the most up-to-date, but you can in principle generate
a new one from Micronaut Starter. Then, change its `pom.xml` to set the following property:

```xml
<micronaut-maven-plugin.version>1.1.5-SNAPSHOT</micronaut-maven-plugin.version>
```

Pointing to whatever snapshot version you published before.

Finally, from the sample application folder, run the Maven goal you are interested into with `mvnDebug` instead of `mvn`:

```shell
mvnDebug package -Dpackaging=docker
```

Then in your IDE, attach a remote debugger to port 8000.

### Preparing for a new minor/major version

```shell
mvn release:update-versions -DautoVersionSubmodules=true -DdevelopmentVersion=4.6.0-SNAPSHOT
```


# ArrayV

![Discord](https://img.shields.io/discord/592082838791127075?color=%237289DA&label=Discord&logo=discord&logoColor=white)
![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/Gaming32/ArrayV/maven.yml?branch=main)
![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/Gaming32/ArrayV/checkstyle.yml?branch=main&label=checkstyle)

Over 200 sorting algorithms animated with 15 unique graphic designs

[Join the Discord server!](https://discord.gg/thestudio)

To compile use:
```shell
./mvnw clean verify
```

To run use:
```shell
./run
```
Alternatively, you can double click on "run" or "run.bat" on Windows.

You can always download the most up-to-date pre-built JAR from [GitHub Actions](https://nightly.link/Gaming32/ArrayV/workflows/maven/main/standalone-jar.zip) (link via [nightly.link](https://nightly.link)).

## Legacy changelog

### 6/8/2020 - Version 3.5
- NEW VISUALS: Sine Wave and Wave Dots!!
- New sort: Bogobogosort
- The bogo shuffle method is now unbiased
- MultipleSortThreads further refactored
- Visuals, VisualStyles enum, and Renderer significantly refactored (more to come!)

### 6/4/2020 - Version 3.2
- New sort: Optimized Cocktail Shaker Sort
- Significant refactoring for MultipleSortThreads and RunAllSorts
- "Run All" button approx. time simplified
- Modified delays for Binary Gnomesort
- Documentation of GCC's median-of-three pivot selection in Introsort

### 6/3/2020 - Version 3.12
- Counting Sort fixed
- Optimized Bubblesort now optimized for already sorted inputs
- Speeds for Quicksorts and Weave Merge during "Run All Sorts" improved

### 6/2/2020 - Version 3.11
- Minor update to MIT license
- Fixed typo in Flipped Min Heapsort
- Improved highlights on Heapsorts (Already sorted heaps now display redundant comparisons)
- Bug fix for Patiencesort on reversed arrays
- Quicksorts exhibiting worst-case behavior during "Run All Sorts" run much faster
- Same tweak as above to Weave Merge Sort

### 5/30/2020 - Version 3.1
- Error messages with detailed information will now appear within the program!
- Sound effects are now consistent on all platforms
- New sort: "Flipped Min Heap Sort" by 'landfillbaby'!
- Minor changes to code organization
- New webhook to my Discord server! Check it out here: https://discord.com/invite/2xGkKC2

### 5/22/2020 - Version 3.01
- Quick bug fix to the "Linked Dots" visual;
  The first line is no longer horizontal.

### 5/21/2020 - Version 3.0 is now released!
- Sound effects are much more pleasant at slower speeds
- Revamped "Run All Sorts" (It is now easier to create your own sequence of sorts!)
- More accurate delay algorithm
- Improved random shuffle algorithm (now with 0% bias!)
- Cleaner statistics
- Sort an array up to 16,384 (2^14) numbers!
- The "green sweep" animation also verifies an array is properly sorted after watching a sort.
  If a sort fails, a warning message pops up, highlighting where the first out-of-order item is.
- Minor tweak to the sort time method. It should be a slight bit more accurate now.
- Slowsort and Sillysort's comparisons are now shown.
- Gravity Sort has a more detailed visual now
- Pancake Sorting is fixed
- Counting Sort is fixed
- Holy Grail Sort is enabled, but just note that it's a mock algorithm; not finished yet.
- "Auxillary" typo fixed; program now says 'Writes to Auxiliary Array(s)'
- Bug fixes and minor tweaks
  - Minor fixes to "Skip Sort" button
  - Weird static line bug with linked dots squashed
  - Other miscellaneous fixes and changes here and there

### 10/19/2019 - Version 2.1
- Both Odd-Even Mergesorts now display comparisons
- PDQSort's Insertion Sorts have been slowed down
- New sorts, courtesy of Piotr Grochowski (https://github.com/PiotrGrochowski/ArrayVisualizer):
  - Iterative Pairwise Sorting Network
  - Recursive Pairwise Sorting Network
  - Recursive Combsort

### 10/13/2019 - Version 2.0 is now released!
- Now includes 73 sorting algorithms, with 2 more that will be finished in the future
  - NEW SORTS:
    - Unoptimized Bubble Sort
    - Rotation-based In-Place Merge Sort
    - "Lazy Stable Sort" from Andrey Astrelin's GrailSort
    - Grail sorting with a static buffer
    - Grail sorting with a dynamic buffer
    - Andrey Astrelin's "SqrtSort"
    - CircleSort
    - Introspective CircleSort
    - Orson Peters' "Pattern-Defeating Quicksort" (PDQSort)
    - Branchless PDQSort
    - Morwenn's implementation of "Poplar Heapsort"
    - Recursive Binary Quicksort
    - Iterative Binary Quicksort
    - Iterative Bitonic Sort
    - Iterative Odd-Even Mergesort
    - "Bubble Bogosort"
    - "Exchange Bogosort"
    - Treesort
    - Optimized Gnomesort with Binary Search
    - "Cocktail Mergesort" (https://www.youtube.com/watch?v=fWubJgIWyxQ)
    - NOTE: "Quick Shell Sort" has been removed.
- Significantly refactored code, more object-oriented
- Optimized visuals -- the program runs smoother than ever!
- Plug-and-play functionality -- using classgraph, you can now easily add your own sorting algorithms to the program! Documentation on that will be available in the future.
- Sort delay system redesigned -- you can now change the speed of the program in the middle of a delayed compare or swap
- Speed dialogue is now disabled while other windows are open
- WikiSort no longer gets stuck on sorting its internal buffer
- Tweaks to TimSort, mostly reimplementing its binary insertion sort
- Binary Insertion Sort is now stable
- The write/swap counts for inputs already sorted are fixed
- The main/auxillary array write counts for Bottom-up Merge are fixed
- Shuffling the array now clears the statistics
- The highest pitches of the program's sound effects are fixed
- Speeds for the "green sweep" and shuffling animations have been tweaked
- Shatter Sort's highlights slightly tweaked
- GrailSort's highlights slightly tweaked

**KNOWN BUGS:**
- Certain sorts (comb sort, radix sorts) cause the program to forget the current speed
- Certain sorts do not work with the "Skip Sort" button
- Missing soundfont
- SkaSort and HolyGrailSort produce errors -- this is normal, they aren't finished yet
- No circular pointer -- will be fixed soon

**PLANS FOR FUTURE RELEASES:**
- Javadocs!!
- SkaSort
- "Holy Grail Sort"
- Options to:
  - Enter in your own set of numbers
  - Select CombSort gap sequence
  - Select ShellSort gap sequence
  - Change TimSort "minrun" value
  - Change IntroSort threshold for insertion/heap sort
  - Change Simple Shatter Sort rate(?)
  - Stop Run All Sorts(?)
  - Stop TimeSort(?)
- Pre-shuffled arrays
- Organize list of sorts into more categories
- Run All Sorts in specific category
- Subheadings for customizable sorts (e.g. display the number of buckets during a bucket sort)
- "Pipe organ" distribution (half ascending, half descending)
- Fixed circular pointer with much cleaner math
- Toogle between pointer and black bar with circular visuals
- Refactor/reorganize prompts and frames
- Cleaner:
  - Tree Sort
  - getters/setters
  - method parameters
- Small organizational changes


# ChunJun

[![EN doc](https://img.shields.io/badge/document-English-blue.svg)](README.md)
[![CN doc](https://img.shields.io/badge/文档-中文版-blue.svg)](README_CH.md)

## Introduce

ChunJun is a distributed integration framework, and currently is based on Apache Flink. It was initially known as FlinkX and renamed ChunJun on February 22, 2022. It can realize data synchronization and calculation between various heterogeneous data sources. ChunJun has been deployed and running stably in thousands of companies so far.

Official website of ChunJun: https://dtstack.github.io/chunjun/

## Features of ChunJun

ChunJun abstracts different databases into reader/source plugins, writer/sink plugins and lookup plugins, and it has the following features:

- Based on the real-time computing engine--Flink, and supports JSON template and SQL script configuration tasks. The SQL script is compatible with Flink SQL syntax;
- Supports distributed operation, support flink-standalone, yarn-session, yarn-per job and other submission methods;
- Supports Docker one-click deployment, support deploy and run on k8s;
- Supports a variety of heterogeneous data sources, and supports synchronization and calculation of more than 20 data sources such as MySQL, Oracle, SQLServer, Hive, Kudu, etc.
- Easy to expand, highly flexible, newly expanded data source plugins can integrate with existing data source plugins instantly, plugin developers do not need to care about the code logic of other plugins;
- Not only supports full synchronization, but also supports incremental synchronization and interval training;
- Not only supports offline synchronization and calculation, but also compatible with real-time scenarios;
- Supports dirty data storage, and provide indicator monitoring, etc.;
- Cooperate with the flink checkpoint mechanism to achieve breakpoint resuming, task disaster recovery;
- Not only supports synchronizing DML data, but also supports DDL synchronization, like 'CREATE TABLE', 'ALTER COLUMN', etc.;

## Build And Compilation

### Get the code

Use the git to clone the code of ChunJun

```shell
git clone https://github.com/DTStack/chunjun.git
```

### build

Execute the command in the project directory.

```shell
./mvnw clean package
```

Or execute

```shell
sh build/build.sh
```

### Common problem

#### Compiling module 'ChunJun-core' then throws 'Failed to read artifact descriptor for com.google.errorprone:javac-shaded'

Error message：

```java
[ERROR]Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.4.2:check(spotless-check)on project chunjun-core:
        Execution spotless-check of goal com.diffplug.spotless:spotless-maven-plugin:2.4.2:check failed:Unable to resolve dependencies:
        Failed to collect dependencies at com.google.googlejavaformat:google-java-format:jar:1.7->com.google.errorprone:javac-shaded:jar:9+181-r4173-1:
        Failed to read artifact descriptor for com.google.errorprone:javac-shaded:jar:9+181-r4173-1:Could not transfer artifact
        com.google.errorprone:javac-shaded:pom:9+181-r4173-1 from/to aliyunmaven(https://maven.aliyun.com/repository/public):
        Access denied to:https://maven.aliyun.com/repository/public/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.pom -> [Help 1]
```

Solution：
Download the 'javac-shaded-9+181-r4173-1.jar' from url 'https://repo1.maven.org/maven2/com/google/errorprone/javac-shaded/9+181-r4173-1/javac-shaded-9+181-r4173-1.jar', and then install locally by using command below:

```shell
mvn install:install-file -DgroupId=com.google.errorprone -DartifactId=javac-shaded -Dversion=9+181-r4173-1 -Dpackaging=jar -Dfile=./jars/javac-shaded-9+181-r4173-1.jar
```

## Quick Start

The following table shows the correspondence between the branches of ChunJun and the version of flink. If the versions are not aligned, problems such as 'Serialization Exceptions', 'NoSuchMethod Exception', etc. mysql occur in tasks.

| Branches     | Flink version |
|--------------|---------------|
| master       | 1.16.1        |
| 1.12_release | 1.12.7        |
| 1.10_release | 1.10.1        |
| 1.8_release  | 1.8.3         |

ChunJun supports running tasks in multiple modes. Different modes depend on different environments and steps. The following are

### Local

Local mode does not depend on the Flink environment and Hadoop environment, and starts a JVM process in the local environment to perform tasks.

#### Steps

Go to the directory of 'chunjun-dist' and execute the command below:

```shell
sh bin/chunjun-local.sh  -job $SCRIPT_PATH
```

The parameter of "$SCRIPT_PATH" means 'the path where the task script is located'.
After execute, you can perform a task locally.

note:
```
when you package in windows and run sh in linux , you need to execute command  sed -i "s/\r//g" bin/*.sh to fix the '\r' problems.
```

[Reference video](https://www.bilibili.com/video/BV1mT411g7fJ?spm_id_from=333.999.0.0)

### Standalone

Standalone mode depend on the Flink Standalone environment and does not depend on the Hadoop environment.

#### Steps
##### 1. add jars of chunjun
1) Find directory of jars:
   if you build this project using maven, the directory name is 'chunjun-dist' ;
   if you download tar.gz file from release page, after decompression, the directory name would be like 'chunjun-assembly-${revision}-chunjun-dist'.

2) Copy jars to directory of Flink lib, command example:
```shell
cp -r chunjun-dist $FLINK_HOME/lib
```
Notice: this operation should be executed in all machines of Flink cluster, otherwise some jobs will fail because of ClassNotFoundException.

##### 2. Start Flink Standalone Cluster

```shell
sh $FLINK_HOME/bin/start-cluster.sh
```

After the startup is successful, the default port of Flink Web is 8081, which you can configure in the file of 'flink-conf.yaml'. We can access the 8081 port of the current machine to enter the flink web of standalone cluster.

##### 3. Submit task

Go to the directory of 'chunjun-dist' and execute the command below:

```shell
sh bin/chunjun-standalone.sh -job chunjun-examples/json/stream/stream.json
```

After the command execute successfully, you can observe the task staus on the flink web.

[Reference video](https://www.bilibili.com/video/BV1TT41137UV?spm_id_from=333.999.0.0)

### Yarn Session

YarnSession mode depends on the Flink jars and Hadoop environments, and the yarn-session needs to be started before the task is submitted.

#### Steps

##### 1. Start yarn-session environment

Yarn-session mode depend on Flink and Hadoop environment. You need to set $HADOOP_HOME and $FLINK_HOME in advance, and we need to upload 'chunjun-dist' with yarn-session '-t' parameter.

```shell
cd $FLINK_HOME/bin
./yarn-session -t $CHUNJUN_HOME -d
```

##### 2. Submit task

Get the application id $SESSION_APPLICATION_ID corresponding to the yarn-session through yarn web, then enter the directory 'chunjun-dist' and execute the command below:

```shell
sh ./bin/chunjun-yarn-session.sh -job chunjun-examples/json/stream/stream.json -confProp {\"yarn.application.id\":\"SESSION_APPLICATION_ID\"}
```

'yarn.application.id' can also be set in 'flink-conf.yaml'.
After the submission is successful, the task status can be observed on the yarn web.

[Reference video](https://www.bilibili.com/video/BV1oU4y1D7e7?spm_id_from=333.999.0.0)

### Yarn Per-Job

Yarn Per-Job mode depend on Flink and Hadoop environment. You need to set $HADOOP_HOME and $FLINK_HOME in advance.

#### Steps

The yarn per-job task can be submitted after the configuration is correct. Then enter the directory 'chunjun-dist' and execute the command below:

```shell
sh ./bin/chunjun-yarn-perjob.sh -job chunjun-examples/json/stream/stream.json
```

After the submission is successful, the task status can be observed on the yarn web.

## Docs of Connectors

For details, please visit：https://dtstack.github.io/chunjun/documents/

## Contributors

Thanks to all contributors! We are very happy that you can contribute Chunjun.

## Contributor Over Time

[![Stargazers Over Time](https://contributor-overtime-api.git-contributor.com/contributors-svg?chart=contributorOverTime&repo=DTStack/chunjun)](https://git-contributor.com?chart=contributorOverTime&repo=DTStack/chunjun)

## License

ChunJun is under the Apache 2.0 license. Please visit [LICENSE](http://www.apache.org/licenses/LICENSE-2.0) for details.

## Contact Us

Join ChunJun Slack.
https://join.slack.com/t/chunjun/shared_invite/zt-1hzmvh0o3-qZ726NXmhClmLFRMpEDHYw


= Spring Data Relational image:https://jenkins.spring.io/buildStatus/icon?job=spring-data-jdbc%2Fmain&subject=Build[link=https://jenkins.spring.io/view/SpringData/job/spring-data-jdbc/] https://gitter.im/spring-projects/spring-data[image:https://badges.gitter.im/spring-projects/spring-data.svg[Gitter]] image:https://img.shields.io/badge/Revved%20up%20by-Develocity-06A0CE?logo=Gradle&labelColor=02303A["Revved up by Develocity", link="https://ge.spring.io/scans?search.rootProjectNames=Spring Data Relational Parent"]

The primary goal of the https://projects.spring.io/spring-data[Spring Data] project is to make it easier to build Spring-powered applications that use new data access technologies such as non-relational databases, map-reduce frameworks, and cloud based data services.

Spring Data Relational, part of the larger Spring Data family, makes it easy to implement repositories for SQL databases.
This module deals with enhanced support for JDBC and R2DBC based data access layers.
It makes it easier to build Spring powered applications that use data access technologies.

It aims at being conceptually easy.
In order to achieve this it does NOT offer caching, lazy loading, write behind or many other features of JPA.
This makes Spring Data JDBC and Spring Data R2DBC a simple, limited, opinionated ORM.

== Features

* Implementation of CRUD methods for Aggregates.
* `@Query` annotation
* Support for transparent auditing (created, last changed)
* Events for persistence events
* Possibility to integrate custom repository code
* JavaConfig based repository configuration through `@EnableJdbcRepositories` respective `@EnableR2dbcRepositories`
* JDBC-only: Integration with MyBatis

== Code of Conduct

This project is governed by the https://github.com/spring-projects/.github/blob/e3cc2ff230d8f1dca06535aa6b5a4a23815861d4/CODE_OF_CONDUCT.md[Spring Code of Conduct]. By participating, you are expected to uphold this code of conduct. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.

== Getting Started with JDBC

Here is a quick teaser of an application using Spring Data JDBC Repositories in Java:

[source,java]
----
interface PersonRepository extends CrudRepository<Person, Long> {

  @Query("SELECT * FROM person WHERE lastname = :lastname")
  List<Person> findByLastname(String lastname);

  @Query("SELECT * FROM person WHERE firstname LIKE :firstname")
  List<Person> findByFirstnameLike(String firstname);
}

@Service
class MyService {

  private final PersonRepository repository;

  public MyService(PersonRepository repository) {
    this.repository = repository;
  }

  public void doWork() {

    repository.deleteAll();

    Person person = new Person();
    person.setFirstname("Jens");
    person.setLastname("Schauder");
    repository.save(person);

    List<Person> lastNameResults = repository.findByLastname("Schauder");
    List<Person> firstNameResults = repository.findByFirstnameLike("Je%");
 }
}

@Configuration
@EnableJdbcRepositories
class ApplicationConfig extends AbstractJdbcConfiguration {

  @Bean
  public DataSource dataSource() {
    return …;
  }

  @Bean
  public NamedParameterJdbcTemplate namedParameterJdbcTemplate(DataSource dataSource) {
    return new NamedParameterJdbcTemplate(dataSource);
  }
}
----

=== Maven configuration

Add the Maven dependency:

[source,xml]
----
<dependency>
  <groupId>org.springframework.data</groupId>
  <artifactId>spring-data-jdbc</artifactId>
  <version>${version}</version>
</dependency>
----

If you'd rather like the latest snapshots of the upcoming major version, use our Maven snapshot repository and declare the appropriate dependency version.

[source,xml]
----
<dependency>
  <groupId>org.springframework.data</groupId>
  <artifactId>spring-data-jdbc</artifactId>
  <version>${version}-SNAPSHOT</version>
</dependency>

<repository>
  <id>spring-snapshot</id>
  <name>Spring Snapshot Repository</name>
  <url>https://repo.spring.io/snapshot</url>
</repository>
----

== Getting Started with R2DBC

Here is a quick teaser of an application using Spring Data R2DBC Repositories in Java:

[source,java]
----
interface PersonRepository extends ReactiveCrudRepository<Person, Long> {

  @Query("SELECT * FROM person WHERE lastname = :lastname")
  Flux<Person> findByLastname(String lastname);

  @Query("SELECT * FROM person WHERE firstname LIKE :firstname")
  Flux<Person> findByFirstnameLike(String firstname);
}

@Service
class MyService {

  private final PersonRepository repository;

  public MyService(PersonRepository repository) {
    this.repository = repository;
  }

  public Flux<Person> doWork() {

    Person person = new Person();
    person.setFirstname("Jens");
    person.setLastname("Schauder");
    repository.save(person);

    Mono<Void> deleteAll = repository.deleteAll();

    Flux<Person> lastNameResults = repository.findByLastname("Schauder");
    Flux<Person> firstNameResults = repository.findByFirstnameLike("Je%");

    return deleteAll.thenMany(lastNameResults.concatWith(firstNameResults));
 }
}

@Configuration
@EnableR2dbcRepositories
class ApplicationConfig extends AbstractR2dbcConfiguration {

  @Bean
  public ConnectionFactory connectionFactory() {
    return ConnectionFactories.get("r2dbc:<driver>://<host>:<port>/<database>");
  }

}
----

=== Maven configuration

Add the Maven dependency:

[source,xml]
----
<dependency>
  <groupId>org.springframework.data</groupId>
  <artifactId>spring-data-r2dbc</artifactId>
  <version>${version}</version>
</dependency>
----

If you'd rather like the latest snapshots of the upcoming major version, use our Maven snapshot repository and declare the appropriate dependency version.

[source,xml]
----
<dependency>
  <groupId>org.springframework.data</groupId>
  <artifactId>spring-data-r2dbc</artifactId>
  <version>${version}-SNAPSHOT</version>
</dependency>

<repository>
  <id>spring-libs-snapshot</id>
  <name>Spring Snapshot Repository</name>
  <url>https://repo.spring.io/snapshot</url>
</repository>
----

== Getting Help

Having trouble with Spring Data?
We’d love to help!

* If you are new to Spring Data JDBC read the following two articles https://spring.io/blog/2018/09/17/introducing-spring-data-jdbc["Introducing Spring Data JDBC"] and https://spring.io/blog/2018/09/24/spring-data-jdbc-references-and-aggregates["Spring Data JDBC, References, and Aggregates"].
* Check the
https://docs.spring.io/spring-data/relational/reference/[reference documentation], and https://docs.spring.io/spring-data/jdbc/docs/current/api/[Javadocs].
* Learn the Spring basics – Spring Data builds on Spring Framework, check the https://spring.io[spring.io] web-site for a wealth of reference documentation.
If you are just starting out with Spring, try one of the https://spring.io/guides[guides].
* If you are upgrading, check out the https://github.com/spring-projects/spring-data-relational/releases[changelog] for "`new and noteworthy`" features.
* Ask a question - we monitor https://stackoverflow.com[stackoverflow.com] for questions tagged with https://stackoverflow.com/tags/spring-data[`spring-data`].
You can also chat with the community on https://gitter.im/spring-projects/spring-data[Gitter].

== Reporting Issues

Spring Data uses GitHub as issue tracking system to record bugs and feature requests.If you want to raise an issue, please follow the recommendations below:

* Before you log a bug, please search the Spring Data JDBCs https://github.com/spring-projects/spring-data-relational/issues[issue tracker] to see if someone has already reported the problem.
* If the issue doesn’t already exist, https://github.com/spring-projects/spring-data-relational/issues/new[create a new issue].
* Please provide as much information as possible with the issue report, we like to know the version of Spring Data that you are using and JVM version.
Please include full stack traces when applicable.
* If you need to paste code, or include a stack trace use triple backticks before and after your text.
* If possible try to create a test-case or project that replicates the issue.
Attach a link to your code or a compressed file containing your code.
Use an in-memory database when possible.
If you need a different database include the setup using https://github.com/testcontainers[Testcontainers] in your test.

== Building from Source

You don’t need to build from source to use Spring Data (binaries in https://repo.spring.io[repo.spring.io]), but if you want to try out the latest and greatest, Spring Data can be easily built with the https://github.com/takari/maven-wrapper[maven wrapper].
You also need JDK 17.

[source,bash]
----
 $ ./mvnw clean install
----

If you want to build with the regular `mvn` command, you will need https://maven.apache.org/run-maven/index.html[Maven v3.8.0 or above].

_Also see link:CONTRIBUTING.adoc[CONTRIBUTING.adoc] if you wish to submit pull requests, and in particular please sign the https://cla.pivotal.io/sign/spring[Contributor’s Agreement] before your first non-trivial change._

=== Running Integration Tests

[source,bash]
----
 $ ./mvnw clean install
----

Runs integration test against a single in memory database.

To run integration tests against all supported databases specify the Maven Profile `all-dbs`.

[source,bash]
----
./mvnw clean install -Pall-dbs
----

This requires an appropriate `container-license-acceptance.txt` to be on the classpath, signaling that you accept the license of the databases used.

If you don't want to accept these licences you may add the Maven Profile `ignore-missing-license`.
This will ignore the tests that require an explicit license acceptance.

[source,bash]
----
./mvnw clean install -Pall-dbs,ignore-missing-license
----

If you want to run an integration tests against a different database you can do so by activating an apropriate Spring Profile.
Available are the following Spring Profiles:

`db2`, `h2`, `hsql` (default), `mariadb`, `mssql`, `mysql`, `oracle`, `postgres`

=== Building reference documentation

Building the documentation builds also the project without running tests.

[source,bash]
----
 $ ./mvnw clean install -Pantora
----

The generated documentation is available from `spring-data-jdbc-distribution/target/antora/site/index.html`.

== Modules

There are a number of modules in this project, here is a quick overview:

* Spring Data Relational: Common infrastructure abstracting general aspects of relational database access.
* link:spring-data-jdbc[Spring Data JDBC]: Repository support for JDBC-based datasources.
* link:spring-data-r2dbc[Spring Data R2DBC]: Repository support for R2DBC-based datasources.

== Examples

* https://github.com/spring-projects/spring-data-examples/[Spring Data Examples] contains example projects that explain specific features in more detail.

== License

Spring Data Relational is Open Source software released under the https://www.apache.org/licenses/LICENSE-2.0.html[Apache 2.0 license].