## [Nakadi Event Broker](https://zalando.github.io/nakadi/)

**Nakadi is no longer developed in the open as we've not seen usage outside of Zalando for many years. **
If you are a Zalando employee, please use Zalando's internal repository for contributing.

## ARCHIVED

[![Build Status](https://travis-ci.org/zalando/nakadi.svg?branch=master)](https://travis-ci.org/zalando/nakadi)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/785ccd4ab5e34867b760a8b07c3b62f1)](https://www.codacy.com/app/aruha/nakadi?utm_source=www.github.com&utm_medium=referral&utm_content=zalando/nakadi&utm_campaign=Badge_Grade)

Nakadi is a distributed event bus broker that implements a RESTful API abstraction on top of Kafka-like queues, which can be used to send, receive, and analyze streaming data in real time, in a reliable and highly available manner.

One of the most prominent use cases of Nakadi is to decouple micro-services by building data streams between producers and consumers.

Main users of nakadi are developers and analysts. Nakadi provides features like REST based integration, multi consumer, ordered delivery, interactive UI, fully managed, security, ensuring data quality, abstraction of big data technology, and push model based consumption.

Nakadi is in active developement and is currently in production inside Zalando as the backbone of our microservices sending millions of events daily with a throughput of more than hundreds gigabytes per second. In one line, Nakadi is a **high-scalability data-stream for enterprise engineering teams**.

![Nakadi Deployment Diagram](docs/img/NakadiDeploymentDiagram.png)

More detailed information can be found on our [website](http://zalando.github.io/nakadi/).

### Project goal

The goal of Nakadi (**ნაკადი** means *stream* in Georgian) is to provide an event broker infrastructure to:

- Abstract event delivery via a secured [RESTful API](https://zalando.github.io/nakadi/manual.html#nakadi-event-bus-api).

    This allows microservices teams to maintain service boundaries, and not directly depend on any specific message broker technology.
    Access can be managed individually for every queue and secured using *OAuth* and custom authorization plugins.

- Enable convenient development of event-driven applications and asynchronous microservices.

    Event types can be defined with [Event type schemas](https://zalando.github.io/nakadi/manual.html#using_event-types)
    and managed via a registry. All events will be validated against the schema before publishing.
    This guarantees data quality and consistency for consumers.

- Efficient low latency event delivery.

    Once a publisher sends an event using a simple [HTTP POST](https://zalando.github.io/nakadi/manual.html#using_producing-events),
    consumers can be pushed to via a [streaming](https://zalando.github.io/nakadi/manual.html#using_consuming-events-lola)
    HTTP connection, allowing near real-time event processing.
    The consumer connection has keepalive controls and support for managing stream offsets using
    [subscriptions](https://zalando.github.io/nakadi/manual.html#using_consuming-events-hila).

### Development status

- Nakadi is high-load production ready.
- Zalando uses Nakadi as its central Event Bus Service.
- Nakadi reliably handles the traffic from thousands event types with the throughput of more than hundreds gigabytes per second.
- The project is in active development.

### Presentations

- [Watch the talk on Nakadi at Fosdem](https://archive.fosdem.org/2018/schedule/event/nakadi/)
- [Background on the Zalando Microservices platform](https://www.youtube.com/watch?v=gEeHZwjwehs)

#### Features

* Stream:
    * REST abstraction over Kafka-like queues.
    * CRUD for event types.
    * Event batch publishing.
    * Low-level interface (**deprecated**).
        * manual client side partition management is needed
        * no support of commits
    * High-level interface (Subscription API).
        * automatic redistribution of partitions between consuming clients
        * commits should be issued to move server-side cursors
* Schema:
    * Schema registry.
    * Several event type categories (Undefined, Business, Data Change).
    * Several partitioning strategies (Random, Hash, User defined).
    * Event enrichment strategies.
    * Schema evolution.
    * Events validation using an event type schema.
* Security:
    * OAuth2 authentication.
    * Per-event type authorization.
    * Blacklist of users and applications.
* Operations:
    * [STUPS](https://stups.io/) platform compatible.
    * [ZMON](https://zmon.io/) monitoring compatible.
    * SLO monitoring.
    * Timelines:
        * this allows transparently switch production and consumption to different cluster (tier, region, AZ) without
        moving actual data and any service degradation.
        * opens the possibility for implementation of other streaming technologies and engines besides Kafka
        (like Amazon Kinesis or Google Cloud Pub/Sub)

Read more about latest development on the [releases page](https://github.com/zalando/nakadi/releases).

#### Additional features that we plan to cover in the future are:

* Support for different streaming technologies and engines. Nakadi currently uses [Apache Kafka](http://kafka.apache.org/)
    as its broker, but other providers (such as Kinesis) will be possible.
* Filtering of events for subscribing consumers.
* Store old published events forever using transparent fall back backup shortages like AWS S3.
* Separate the internal schema register to standalone service.
* Use additional schema formats and protocols like Avro, protobuf and [others](https://en.wikipedia.org/wiki/Comparison_of_data_serialization_formats).

#### Related projects

The [zalando-nakadi](https://github.com/zalando-nakadi/) organisation contains many useful related projects
like

* [Use Nakadi with a client in your language of choice](https://nakadi.io/manual.html#using_clients),
* SDK,
* [Web UI](https://github.com/zalando-incubator/nakadi-ui),
* DevOps tools, and more.

## How to contribute to Nakadi

Read our [contribution guidelines](CONTRIBUTING.md) on how to submit issues and pull requests, then get Nakadi up and running locally using Docker:

### Dependencies

The Nakadi server is a Java 8 [Spring Boot](https://projects.spring.io/spring-boot/) application.
It uses [Kafka 1.1.1](https://kafka.apache.org/11/documentation.html) as its broker and
 [PostgreSQL 9.5](https://www.postgresql.org/docs/9.5/static/release-9-5.html) as its supporting database.

Nakadi requires recent versions of docker and docker-compose. In
particular, docker-compose >= v1.7.0 is required. See [Install Docker
Compose](https://docs.docker.com/compose/install/) for information on
installing the most recent docker-compose version.

The project is built with [Gradle](http://gradle.org).
The `./gradlew` [wrapper script](http://www.gradle.org/docs/current/userguide/gradle_wrapper.html) will bootstrap
the right Gradle version if it's not already installed.

### Install

To get the source, clone the git repository.
```sh
git clone https://github.com/zalando/nakadi.git
```
### Building

The gradle setup is fairly standard, the main tasks are:

- `./gradlew build`: run a build and test
- `./gradlew clean`: clean down the build

Some other useful tasks are:

- `./gradlew startNakadi`: build Nakadi and start docker-compose services: nakadi, postgresql, zookeeper and kafka
- `./gradlew stopNakadi`: shutdown docker-compose services
- `./gradlew startStorages`: start docker-compose services: postgres, zookeeper and kafka (useful for development purposes)
- `./gradlew fullAcceptanceTest`: start Nakadi configured for acceptance tests and run acceptance tests

For working with an IDE, the `eclipse` IDE task is available and you'll be able to import the `build.gradle` into Intellij IDEA directly.

### Running a Server

**Note**: Nakadi Docker for ARM processors is available at [here](./docker-arm)

From the project's home directory you can start Nakadi via Gradle:

```sh
./gradlew startNakadi
```

This will build the project and run docker compose with 4 services:

- Nakadi (8080)
- PostgreSQL (5432)
- Kafka (9092)
- Zookeeper (2181)

To stop the running Nakadi server:

```sh
./gradlew stopNakadi
```

## Using Nakadi and its API

Please read the [manual](https://zalando.github.io/nakadi/manual.html) for the full API usage details.

### Creating Event Types

The Nakadi API allows the publishing and consuming of _events_ over HTTP.
To do this the producer must register an _event type_ with the Nakadi schema
registry.

This example shows a minimalistic `undefined` category event type with a wildcard schema:

```sh
curl -v -XPOST http://localhost:8080/event-types -H "Content-type: application/json" -d '{
  "name": "order.ORDER_RECEIVED",
  "owning_application": "order-service",
  "category": "undefined",
  "schema": {
    "type": "json_schema",
    "schema": "{ \"additionalProperties\": true }"
  }
}'
```
**Note:** This is not a recommended category and schema. It should be used only for testing.

You can read more about this in the [manual](https://zalando.github.io/nakadi/manual.html#using_event-types).

### Consuming Events

You can open a stream for an event type via the `events` sub-resource:

```sh
curl -v http://localhost:8080/event-types/order.ORDER_RECEIVED/events

HTTP/1.1 200 OK

{"cursor":{"partition":"0","offset":"82376-000087231"},"events":[{"order_number": "ORDER_001"}]}
{"cursor":{"partition":"0","offset":"82376-000087232"}}
{"cursor":{"partition":"0","offset":"82376-000087232"},"events":[{"order_number": "ORDER_002"}]}
{"cursor":{"partition":"0","offset":"82376-000087233"},"events":[{"order_number": "ORDER_003"}]}
```
You will see the events when you publish them from another console for example.
The records without `events` field are `Keep Alive` messages.

**Note:** the [low-level API](https://zalando.github.io/nakadi/manual.html#using_consuming-events-lola) should be
used only for debugging. It is not recommended for production systems.
For production systems, please use the [Subscriptions API](https://zalando.github.io/nakadi/manual.html#using_consuming-events-hila).

### Publishing Events

Events for an event type can be published by posting to its "events" collection:

```sh
curl -v -XPOST http://localhost:8080/event-types/order.ORDER_RECEIVED/events \
 -H "Content-type: application/json" \
 -d '[{
    "order_number": "24873243241"
  }, {
    "order_number": "24873243242"
  }]'

HTTP/1.1 200 OK
```

Read more in the [manual](https://zalando.github.io/nakadi/manual.html#using_producing-events).

## Contributing

Nakadi accepts contributions from the open-source community.

Please read [`CONTRIBUTING.md`](CONTRIBUTING.md).

Please also note our [`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md).

## Contact

This [email address](MAINTAINERS) serves as the main contact address for this project.

Bug reports and feature requests are more likely to be addressed
if posted as [issues](https://github.com/zalando/nakadi/issues) here on GitHub.

## License

Please read the full [`LICENSE`](LICENSE)

The MIT License (MIT) Copyright © 2015 Zalando SE, https://tech.zalando.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the “Software”), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions
of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


# terracotta-platform

## For Developers

### License Headers

Check for missing licenses:

```bash
./mvnw license:check
```

Format files:

```bash
./mvnw license:format
```

### Fast build

This project is big and development and testing times are impacted by Maven build time since there is no gradle cache.
So we need to be able several times per day to build the project *fast*.

```bash
./mvnw clean install -DskipTests -Dfast -Djava.build.vendor=openjdk
```

The command should only do the minimal required steps:

- compilation steps
- felix
- packaging (jars and source jars - to use snapshots in downstream projects)

*No toolchain, verification, license check, findbugs, etc.*

The command needs to be fast: *less than 30 seconds.*

### Unit tests vs IT tests

We are able to run the unit tests only and all tests:

Skip all the tests:

```bash
./mvnw clean verify -DskipTests -Dfast
```

Skip Galvan and Angela IT tests but run unit tests (*last ~2-3min*)

```bash
./mvnw clean verify -DskipITs -Dfast
```

Run everything, like on Azure. *Lasts ~1h20min*

```bash
./mvnw clean verify -Dtest.parallel.forks=1C -Djava.test.vendor=openjdk -Djava.build.vendor=openjdk
```

### Concurrent Maven executions

It is possible to further speed the execution of the build by using `-TxC`.

Example, running the unit tests with `-T4C` leads to a build lasting only 1 min 20 sec.

```bash
./mvnw clean install -DskipTests -Dfast -T4C
```

### Plugin config

- Use `<trimStackTrace>false</trimStackTrace>`

### Choose the JVM used for testing

We can run the tests with one of the JVM on the toolchain but compilation is done in 1.8.

*Examples*

Run a test with Java 11 (will look at your toolchains.xml file to fine the JVM):

```bash
./mvnw verify -f management/testing/integration-tests/pom.xml -Dit.test=DiagnosticIT -Djava.test.version=1.11 -Dtest.parallel.forks=1C -Djava.test.vendor=openjdk -Djava.build.vendor=openjdk
```

Run a test with Java 8 (will look at your toolchains.xml file to fine the JVM):

```bash
./mvnw verify -f management/testing/integration-tests/pom.xml -Dit.test=DiagnosticIT -Djava.test.version=1.8 -Dtest.parallel.forks=1C -Djava.test.vendor=openjdk -Djava.build.vendor=openjdk
```

Run a test with Java 8 (will look at your toolchains.xml file to fine the JVM):

```bash
./mvnw verify -f management/testing/integration-tests/pom.xml -Dit.test=DiagnosticIT
```

Run a test by using the default JAVA_HOME found from your shell. -Dfast has no enforcement at all and does not use toolchain. -Dfast goal is to compile and package fast.

```bash
./mvnw verify -f management/testing/integration-tests/pom.xml -Dit.test=DiagnosticIT -Dfast
```

The same applies for Angela tests in dynamic-config/testing/system-tests.

*Verification*

Look inside the servers logs of the nodes started by Angela and Galvan.
The server logs contain the list of system properties and JVM details used to start the servers.
The version should match the one requested.

- For Angela tests: `target/angela/work/XYZ/logs/stripe1/node-1-1/terracotta.server.log`
- For Galvan tests: `target/galvan/XYZ/stripe1/testServer0/logs/terracotta.server.log`

Look for something like that in the logs:

```
java.runtime.name             : OpenJDK Runtime Environment
java.runtime.version          : 11.0.6+10
java.specification.name       : Java Platform API Specification
java.specification.vendor     : Oracle Corporation
java.specification.version    : 11
```

### IDE support

After having done a `./mvnw clean verify -DskipTests -Dfast` to build the project and kit, you should be able to go in your IDE and right-click run the test DiagnosticIT for example.


# Interlok

[![GitHub tag](https://img.shields.io/github/tag/adaptris/interlok.svg)](https://github.com/adaptris/interlok/tags)
[![license](https://img.shields.io/github/license/adaptris/interlok.svg)](https://github.com/adaptris/interlok/blob/develop/LICENSE)
[![Actions Status](https://github.com/adaptris/interlok/actions/workflows/gradle-publish.yml/badge.svg)](https://github.com/adaptris/interlok/actions)
[![codecov](https://codecov.io/gh/adaptris/interlok/branch/develop/graph/badge.svg)](https://codecov.io/gh/adaptris/interlok)
[![CodeQL](https://github.com/adaptris/interlok/workflows/CodeQL/badge.svg)](https://github.com/adaptris/interlok/security/code-scanning)
[![Known Vulnerabilities](https://snyk.io/test/github/adaptris/interlok/badge.svg?targetFile=build.gradle)](https://snyk.io/test/github/adaptris/interlok?targetFile=build.gradle)
[![Closed PRs](https://img.shields.io/github/issues-pr-closed/adaptris/interlok)](https://github.com/adaptris/interlok/pulls?q=is%3Apr+is%3Aclosed)
[![Appveyor](https://ci.appveyor.com/api/projects/status/efwdjhhdh5ug3rkk/branch/develop?svg=true)](https://ci.appveyor.com/project/adaptris/interlok/branch/develop)

This is the base repository for Interlok. It contains the base adapter pared of all 3rd party dependencies other than open source ones. There is a single dependency (commented out) on Microsoft SQL Server JDBC provider which is not required unless you explicitly want to test JDBC against SQL Server.

## Setting up your Development Environment ##

### Pre-requisites ###

* JAVA 17

### Optional pre-reqs ###

* MySQL (for tests)
* SFTP Server (for tests)

## Doing your first build ##

```
#!shell

./gradlew compileJava
# You will see if you have setup your environment properly.
./gradlew test
# This will take about 10 minutes, and at the end there will be a directory that contains the test output according to the usual gradle conventions (build/reports/tests/test)
```
### Fixing your build ###

Your first build may well have errors!

* Have you installed the (JCE) Unlimited Strength Jurisdiction Policy Files files? (some of the unit tests rely on strong crypto...); If you're using an OpenJDK build, then I think it's automatically installed unless you have explicitly disabled it.
* Windows User? then you will have to *at least once* encrypt a file so that an EFS certificate is created (for com.adaptris.security.password)

### JDK work-arounds ###

* Windows User? With a 32bit JVM you might get some stupid __Could not reserve enough space for object heap/OutOfMemory__; just stop it, and use a 64 bit one.

## Modifying your build ##

Everything relies on build.properties; in the previous section you didn't make any changes, so it skipped a bunch of tests that required the optional components. You might want to enable those tests, possibly because you're working on fixing those components or well whatever. So what you will need to do is to create your own copy of build.properties which is simply a standard "properties" file with key=value entries.

### Standard properties ###

Define your properties either on the gradle commandline `gradle -PpropertyName=value test` etc. or in gradle.properties

Property Key | Default Value | Description | Notes
------------ | ------------- | ----------- | -----
junit.test.classes|```**/*Test*```|Standard filter so that when you run _gradle test_ it only tests what you want to test ||
verboseTests|false|If your console mode is "plain" (org.gradle.console=plain), then this prints out each test suite before it's executed|WinGit : it is probably plain|

So, `gradle -PverboseTests=true test` will print out test suite names; gradle `-Pjunit.test.classes=**/metadata/**/*Test* test` will just test metadata classes.

### Custom Tests

Create a file `src/test/resources/default-test.properties.template.<machinename>`. This will be imported as part of the `processTestResources` step and will allow you control behaviour within individual tests such as whether or not stored procedure tests are run (or actual SFTP against a real server).

Check default-test.properties.template for specific values.

## How to publish your changes ##

If you've made some changes, and you want to check that the downstream projects (like interlok-optional) are OK with your changes, then the easiest solution is to build the snapshot release from Jenkins. This will trigger all the downstream builds once successful (it might take a while) and you'll be able to see any impact your changes may have had.

And that's it.


# High-Performance Java Persistence

The [High-Performance Java Persistence](https://vladmihalcea.com/books/high-performance-java-persistence?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp) book and video course code examples. I wrote [this article](https://vladmihalcea.com/high-performance-java-persistence-github-repository/) about this repository since it's one of the best way to test JDBC, JPA, Hibernate or even jOOQ code. Or, if you prefer videos, you can watch [this presentation on YouTube](https://www.youtube.com/watch?v=U8MoOe8uMYA).

### Are you struggling with application performance issues?

Imagine having a tool that can automatically detect if you are using JPA and Hibernate properly. No more performance issues, no more having to spend countless hours trying to figure out why your application is barely crawling.

Imagine discovering early during the development cycle that you are using suboptimal mappings and entity relationships or that you are missing performance-related settings.

More, with Hypersistence Optimizer, you can detect all such issues during testing and make sure you don't deploy to production a change that will affect data access layer performance.

[Hypersistence Optimizer](https://vladmihalcea.com/hypersistence-optimizer/?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp) is the tool you've been long waiting for!

#### Training

If you are interested in on-site training, I can offer you my [High-Performance Java Persistence training](https://vladmihalcea.com/trainings/?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp)
which can be adapted to one, two or three days of sessions. For more details, check out [my website](https://vladmihalcea.com/trainings/?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp).

#### Consulting

If you want me to review your application and provide insight into how you can optimize it to run faster,
then check out my [consulting page](https://vladmihalcea.com/consulting/?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp).

#### High-Performance Java Persistence Video Courses

If you want the fastest way to learn how to speed up a Java database application, then you should definitely enroll in [my High-Performance Java Persistence video courses](https://vladmihalcea.com/courses/?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp).

#### High-Performance Java Persistence Book

Or, if you prefer reading books, you are going to love my [High-Performance Java Persistence book](https://vladmihalcea.com/books/high-performance-java-persistence?utm_source=GitHub&utm_medium=banner&utm_campaign=hpjp) as well.

## Java

All examples require at least Java 17 because of the awesome [Text Blocks](https://openjdk.java.net/jeps/355) feature, which makes JPQL and SQL queries so much readable.

## Maven

You need to use Maven 3.6.2 or newer to build the project.

## IntelliJ IDEA

On IntelliJ IDEA, the project runs just fine. You will have to make sure to select Java 17 or newer.

## Database setup

The project uses various database systems for integration testing, and you can configure the JDBC connection settings using the
`DatasourceProvider` instances (e.g., `PostgreSQLDataSourceProvider`).

By default, without configuring any database explicitly, HSQLDB is used for testing.

However, since some integration tests are designed to work on specific relational databases, we will need to have those databases started prior to running those tests.

Therefore, when running a DB-specific test, this GitHub repository will execute the following steps:

1. First, the test will try to find whether there's a local RDBMS it can use to run the test.
2. If no local database is found, the integration tests will use Testcontainers to bootstrap a Docker container
with the required *Oracle*, *SQL Server*, *PostgreSQL*, *MySQL*, *MariaDB*, *YugabyteDB*, or *CockroachDB* instance on demand.

> While you don't need to install any database manually on your local OS, this is recommended since your tests will run much faster than if they used Testcontainers.

### Manual Database configuration

- PostgreSQL

    You can install PostgreSQL, and the password for the `postgres` user should be `admin`.

    Now you need to create a `high_performance_java_persistence` database.

- Oracle

    You need to download and install Oracle XE

    Set the `sys` password to `admin`

    Connect to Oracle using the "sys as sysdba" user and create a new user:

        alter session set "_ORACLE_SCRIPT"=true;

        create user oracle identified by admin default tablespace users;

        grant dba to oracle;

        alter system set processes=1000 scope=spfile;

        alter system set sessions=1000 scope=spfile;

        ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

    Open the `C:\app\${user.name}\product\21c\homes\OraDB21Home1\network\admin` folder where `${user.name}` is your current Windows username.

    Locate the `tnsnames.ora` and `listener.ora` files and change the port from `1522` to `1521` if that's the case. If you made these modifications,
    you need to restart the `OracleOraDB21Home1TNSListener` and `OracleServiceXE` Windows services.

- MySQL

    You should install MySQL 8, and the password for the `mysql` user should be `admin`.

    Now, you need to create a `high_performance_java_persistence` schema

    Besides having all privileges on this schema, the `mysql` user also requires select permission on `mysql.PROC`.

    If you don't have a `mysql` user created at database installation time, you can create one as follows:

    ````
    CREATE USER 'mysql'@'localhost';

    SET PASSWORD for 'mysql'@'localhost'='admin';

    GRANT ALL PRIVILEGES ON high_performance_java_persistence.* TO 'mysql'@'localhost';

    GRANT SELECT ON mysql.* TO 'mysql'@'localhost';

    FLUSH PRIVILEGES;
    ````

- SQL Server

    You can install SQL Server Express Edition with Tools. Choose mixed mode authentication and set the `sa` user password to `adm1n`.

    Open SQL Server Configuration Manager -> SQL Server Network Configuration and enable Named Pipes and TCP

    In the right pane of the TCP/IP option, choose Properties, then IP Addresses and make sure you Enable all listed IP addresses.
    You need to blank the dynamic TCP port value and configure the static TCP port 1433 for all IPs.

    Open SQL Server Management Studio and create the `high_performance_java_persistence` database

## Maven

> To build the project, don't use *install* or *package*. Instead, just compile test classes like this:
>
>    mvnw clean test-compile

Or you can just run the `build.bat` or `build.sh` scripts which run the above Maven command.

Afterward, just pick one test from the IDE and run it individually.

> Don't you run all tests at once (e.g. `mvn clean test`) because the test suite will take a very long time to complete.
>
> So, run the test you are interested in individually.

Enjoy learning more about Java Persistence, Hibernate, and database systems!


[![Twitter URL](https://img.shields.io/twitter/url/https/twitter.com/dbeaver_news.svg?style=social&label=Follow%20%40dbeaver_news)](https://twitter.com/dbeaver_news)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/fa0bb9cf5a904c7d87424f8f6351ba92)](https://app.codacy.com/gh/dbeaver/dbeaver/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
[![Apache 2.0](https://img.shields.io/github/license/cronn-de/jira-sync.svg)](http://www.apache.org/licenses/LICENSE-2.0)

# DBeaver

Free multi-platform database tool for developers, SQL programmers, database administrators and analysts.
Supports any database which has JDBC driver (which basically means - ANY database).
Commercial versions also support non-JDBC datasources such as
MongoDB, Cassandra, Couchbase, Redis, BigTable, ScyllaDB, DynamoDB, etc.
You can find the list of all databases supported in commercial versions
here.

* Has a lot of features including metadata editor, SQL editor, rich data editor, ERD, data export/import/migration, SQL execution plans, etc.
* Based on Eclipse platform.
* Uses plugins architecture and provides additional functionality for the following databases: MySQL/MariaDB, PostgreSQL, Greenplum, Oracle, IBM Db2, Exasol, SQL Server, Sybase/SAP ASE, SQLite, Firebird, H2, HSQLDB, Derby, Teradata, Vertica, Netezza, Informix, etc.

## Download

You can download prebuilt binaries from official website or directly from GitHub releases.
You can also download Early Access version. We publish daily.

## Running

Just run an installer (or unzip an archive) and run `dbeaver`.

Note: DBeaver needs Java to run. Open JDK 17 is included in all DBeaver distributions.
You can change default JDK version by replacing directory `jre` in dbeaver installation folder.

## Documentation

* WIKI
* Issue tracker

## Build from sources

See this article.

## Feedback

- For bug reports and feature requests - please create a ticket.
- If you have any questions, ideas, etc - please start a discussion.
- Pull requests are welcome.
- Visit https://dbeaver.io or https://dbeaver.com for more information.
- Follow us on [Twitter](https://twitter.com/dbeaver_news/) and [Facebook](https://www.facebook.com/DBeaverCorporation)
- Thanks for using DBeaver! Star if you like it.

## Contribution: help the Beaver!

Hooray, we have reached 35k+ stars on GitHub and continue to grow!
That's really cool, and we are glad that you like DBeaver.

- We are actively looking for new source code contributors. We have added labels “Good first issue” and “Help wanted” to some tickets. If you want to be a part of our development team, just be brave and take a ticket.
- You can buy one of our commercial versions. They include NoSQL databases support, additional extensions, and official online support. Also, licensed users have priorities in bug fixes and the development of new features.

Thank you!

- DBeaver Team (contributors)

---------

DBeaver is a desktop client.
If you are looking for a web-based database management tool - check our new product: CloudBeaver.
It is based on DBeaver platform and thus supports any database and most of DBeaver features.


# Conveyal R5 Routing Engine

## R5: Rapid Realistic Routing on Real-world and Reimagined networks
R5 is the routing engine for [Conveyal](https://www.conveyal.com/learn), a web-based system that allows users to create transportation scenarios and evaluate them in terms of cumulative opportunities accessibility indicators. See the [Conveyal user manual](https://docs.conveyal.com/) for more information.

We refer to the routing method as "realistic" because it works by planning door-to-door trips at many different departure times in a time window, which better reflects how people use transportation systems than planning a single trip at an exact departure time. R5 handles both scheduled public transit and headway-based lines, using novel methods to characterize variation and uncertainty in travel times. It is designed for one-to-many and many-to-many travel-time calculations used in access indicators, offering substantially better performance than repeated calls to older tools that provide one-to-one routing results. For a comparison with OpenTripPlanner, see [this background](http://docs.opentripplanner.org/en/latest/Version-Comparison/#commentary-on-otp1-features-removed-from-otp2).

We say "Real-world and Reimagined" networks because R5's networks are built from widely available open OSM and GTFS data describing baseline transportation systems, but R5 includes a system for applying light-weight patches to those networks for immediate, interactive scenario comparison.

**Please note** that the Conveyal team does not provide technical support for third-party deployments. R5 is a component of a specialized commercial system, and we align development efforts with our roadmap and the needs of subscribers to our hosted service. This service is designed to facilitate secure online collaboration, user-friendly data management and scenario editing through a web interface, and complex calculations performed hundreds of times faster using a compute cluster. These design goals may not align well with other use cases. This project is open source primarily to ensure transparency and reproducibility in public planning and decision making processes, and in hopes that it may help researchers, students, and potential collaborators to understand and build upon our methodology.

While the Conveyal team provides ongoing support and compatibility to subscribers, third-party projects using R5 as a library may not work with future releases. R5 does not currently expose a stable programming interface ("API" or "SDK"). As we release new features, previous functions and data types may change. The practical effect is that third-party wrappers or language bindings (e.g., for R or Python) may need to continue using an older release of R5 for feature compatibility (though not necessarily result compatibility, as the methods used in R5 are now relatively mature).

## Methodology

For details on the core methods implemented in Conveyal Analysis and R5, see:

* [Conway, Byrd, and van der Linden (2017)](https://keep.lib.asu.edu/items/127809)
* [Conway, Byrd, and van Eggermond (2018)](https://www.jtlu.org/index.php/jtlu/article/view/1074)
* [Conway and Stewart (2019)](https://files.indicatrix.org/Conway-Stewart-2019-Charlie-Fare-Constraints.pdf)

### Citations

The Conveyal team is always eager to see cutting-edge uses of our software, so feel free to send us a copy of any thesis, report, or paper produced using this software. We also ask that any academic or research publications using this software cite the papers above, where relevant and appropriate.

## Configuration

It is possible to run a Conveyal Analysis UI and backend locally (e.g. on your laptop), which should produce results identical to those from our hosted platform. However, the computations for more complex analyses may take quite a long time. Extension points in the source code allow the system to be tailored to cloud computing environments to enable faster parallel computation.

### Running Locally

To get started, copy the template configuration (`analysis.properties.tmp`) to `analysis.properties`.
To run locally, use the default values in the template configuration file. `offline=true` will create a local instance that avoids cloud-based storage, database, or authentication services.
By default, analysis-backend will use the `analysis` database in a local MongoDB instance, so you'll also need to install and start a MongoDB instance.

Database configuration variables include:

- `database-uri`: URI to your MongoDB cluster
- `database-name`: name of the database to use in your MongoDB cluster

## Building and running

Once you have configured `analysis.properties` and started MongoDB locally, you can build and run the analysis backend with `gradle runBackend`. If you have checked out a commit (such as a release tag) where you are sure all tests will pass, you can skip the tests with `gradle -x test runBackend`.

You can build a single self-contained JAR file containing all the dependencies with `gradle shadowJar` and start it with `java -Xmx2g -cp build/libs/r5-vX.Y.Z-all.jar com.conveyal.analysis.BackendMain`.

Once you have this backend running, follow the instructions to start the [analysis-ui frontend](https://github.com/conveyal/analysis-ui). Once that the UI is running, you should be able to log in without authentication (using the frontend URL, e.g. http://localhost:3000).

## Creating a development environment

In order to do development on the frontend or backend, you'll need to set up a local development environment. We use [IntelliJ IDEA](https://www.jetbrains.com/idea/). The free/community edition is sufficient for working on R5. Import R5 into IntelliJ as a new project from existing sources. You can then create a run configuration for `com.conveyal.analysis.BackendMain`, which is the main class. You will need to configure the JVM options and properties file mentioned above.

By default, IntelliJ will follow common Gradle practice and build R5 using the "Gradle wrapper" approach, in which operating-system specific scripts are run that download and install a specific version of Gradle in the projet directory. We have encountered problems with this approach where IntelliJ seems to have insufficient control over the build/run/debug cycle. IntelliJ has its own internal implementation of the Gradle build process, and in our experience this works quite smoothly and is better integrated with the debug cycle. To switch to this appraoch, in the Gradle section of the IntelliJ settings, choose "Build and run using IntelliJ IDEA" and "Run tests using IntelliJ IDEA". Below that you may also want to choose "Use Gradle from specified location" to use your local system-wide copy.

## Structured Commit Messages

We use structured commit messages to help generate changelogs.

The first line of these messages is in the following format: `<type>(<scope>): <summary>`

The `(<scope>)` is optional and is often a class name. The `<summary>` should be in the present tense. The type should be one of the following:

- feat: A new feature from the user point of view, not a new feature for the build.
- fix: A bug fix from the user point of view, not a fix to the build.
- docs: Changes to the user documentation, or to code comments.
- style: Formatting, semicolons, brackets, indentation, line breaks. No change to program logic.
- refactor: Changes to code which do not change behavior, e.g. renaming a variable.
- test: Adding tests, refactoring tests. No changes to user code.
- build: Updating build process, scripts, etc. No changes to user code.
- devops: Changes to code that only affect deployment, logging, etc. No changes to user code.
- chore: Any other changes causing no changes to user code.

The body of the commit message (if any) should begin after one blank line.

From 2018 to 2020, we used major/minor/patch release numbering as suggested by https://www.conventionalcommits.org. Starting in 2021, we switched to major/minor release numbering, incrementing the minor version with regular feature releases and the major version only when there are substantial changes to the cluster computing components of our system. Because there is no public API at this time, the conventional definition of breaking changes under semantic versioning does not apply.