[![Apache 2.0](https://img.shields.io/github/license/pgcodekeeper/pgcodekeeper.svg)](http://www.apache.org/licenses/LICENSE-2.0)

# pgCodeKeeper

A tool for easier PostgreSQL development.

* Comparing database code is very easy now. Compare live DB instances, pg_dump output, as well as pgCodeKeeper projects.
* Generate migration scripts via a user-friendly interface. You can use both live DB instances and DB dumps as initial data. You can also compare pgCodeKeeper projects — useful when working with versions control systems.
* pgCodeKeeper is an [Eclipse IDE](http://www.eclipse.org) plug-in. The DB code is saved as an Eclipse project, and the project can be tracked under any of the versions control systems supported by Eclipse — Git, SVN, Mercurial, CVS and many others.

## Download

pgCodeKeeper requires Java (JRE) 11+ to run.

You can download prebuilt packages and CLI version from [GitHub releases](https://github.com/pgcodekeeper/pgcodekeeper/releases).

If you already have Eclipse IDE installed you can [install](https://pgcodekeeper.readthedocs.io/en/latest/installation.html) pgCodeKeeper plugin using Marketplace or `https://pgcodekeeper.org/update/` update site.

## Documentation

* [User manual](https://pgcodekeeper.readthedocs.io/en/latest/)
* [Issue tracker](https://github.com/pgcodekeeper/pgcodekeeper/issues)

## Build

Build requires Java (JDK) 11+ and Apache Maven 3.6+.

```sh
git clone https://github.com/pgcodekeeper/pgcodekeeper.git
cd pgcodekeeper
mvn verify
```

Binaries will be created in `ru.taximaxim.codekeeper.mainapp/product/rcp/target/products`
CLI package will be created in `ru.taximaxim.codekeeper.mainapp/product/standalone/target/products`

## Notes

- If you have any questions, suggestions, ideas, etc - contact us in our [Telegram chat](https://t.me/pgcodekeeper) or create an issue.
- Pull requests are welcome.
- Visit https://pgcodekeeper.org for more information.
- Thanks for using pgCodeKeeper!

## Contributing

### Project Structure

The project consists of several modules that implement Eclipse RCP plugins (and, by extension, OSGi bundles).

#### Main Modules

- `pom.xml` - first, root pom.xml defines Maven build order and environment. Not required for application development.
- `ru.taximaxim.codekeeper.core` - the core module containing database schema classes and loaders, comparison logic, SQL code generators and parsers.
- `ru.taximaxim.codekeeper.cli` - CLI implementation module.
- `ru.taximaxim.codekeeper.ui` - GUI module, Eclipse integration.

#### Test Modules

- `ru.taximaxim.codekeeper.core.tests` - majority of unit and functional tests.
- `ru.taximaxim.codekeeper.cli.tests` - tests for CLI.
- `ru.taximaxim.codekeeper.ui.tests` - tests for pieces of logic that were implemented in UI module instead of the core module.

#### Misc Modules

- `ru.taximaxim.codekeeper.mainapp` - "branding plugin", provides Eclipse with feature information, images, etc. Also contains development Target platform files.
- `ru.taximaxim.codekeeper.mainapp/report` - tests coverage report module.
- `ru.taximaxim.codekeeper.mainapp/feature` - Eclipse feature module.
- `ru.taximaxim.codekeeper.mainapp/updatesite` - Eclipse p2 repository/update site module.
- `ru.taximaxim.codekeeper.mainapp/product/rcp` - Eclipse product description, used to build pgCodeKeeper packages.
- `ru.taximaxim.codekeeper.mainapp/product/standalone` - Eclipse product description, used to build CLI version packages.
- `ru.taximaxim.codekeeper.mainapp/deploy` - GitHub deploy module.

### Module Contents

#### ru.taximaxim.codekeeper.core

This module was derived from [apgdiff](https://github.com/fordfrog/apgdiff) project. At this point it is almost fully rewritten according to our needs.
Primary packages of this module are:

- `ru.taximaxim.codekeeper.core.schema` - contains classes that describe SQL objects. Each class contains object properties, creation SQL code generator, object comparison logic and ALTER code generator. Notable classes:
  - `PgStatement` - all object classes must be derived from this one. Defines main API for working with objects.
  - `PgDatabase` - root class for every loaded database schema.
  - `GenericColumn` - generic reference to any object (used to be a column reference).
- `ru.taximaxim.codekeeper.core.schema.meta` - simplified SQL object classes for database structure serialization.
- `ru.taximaxim.codekeeper.core.schema.meta` (resources) - serialized descriptions of PostgreSQL's system objects (built using JdbcSystemLoader).
- `ru.taximaxim.codekeeper.core.loader` - database schema loaders: project, SQL file, JDBC, library.
  - `ProjectLoader` - loads a database from project structure. **(!)** Keep in sync with `UIProjectLoader`.
  - `PgDumpLoader` - loads objects into a database structure from a file. **(!)** Keep in sync with `PgUIDumpLoader`.
- `ru.taximaxim.codekeeper.core.loader` (resources) - SQL queries for JdbcLoaders, used by Java code via `JdbcQueries`.
- `ru.taximaxim.codekeeper.core.loader.jdbc` - classes that create schema objects based on JDBC ResultSets.
- `ru.taximaxim.codekeeper.core.parsers.antlr` - generated ANTLR4 parser code, plus utility parser classes. Notable classes:
  - `AntlrParser` - factory methods for parser creation and parallel parser operations.
  - `QNameParser` - utility class for parsing and splitting qualified names.
  - `ScriptParser` - simple parser that splits statements for later execution.
  - `Custom(T)SQLParserListener` - main entry point for parsing SQLs into schema objects.
- `ru.taximaxim.codekeeper.core.parsers.antlr.statements` - processor classes that create schema objects based on parser-read data. Notable classes:
  - `ParserAbstract` - common base class for all processors.
- `ru.taximaxim.codekeeper.core.parsers.antlr.expr` - expression analysis classes: find dependencies in expressions and infer expression types for overloaded function call resolution.
- `ru.taximaxim.codekeeper.core.parsers.antlr.expr.launcher` - support for parallel expression analysis.
- `ru.taximaxim.codekeeper.core.formatter` - SQL and pl/pgsql code formatters.
- `ru.taximaxim.codekeeper.core` - util and general-purpose classes. Notable classes:
  - `PgDiff` - entry point for database schema diff operations.
- `ru.taximaxim.codekeeper.core.model.difftree` - classes representing and creating an object diff tree. Notable classes:
  - `DbObjType` - enum of all SQL object types supported by the program.
  - `TreeElement` - a node in an object diff tree.
  - `DiffTree` - object diff tree factory.
  - `TreeFlattener` - filters and flattens the tree into a list of nodes according to requested parameters.
- `ru.taximaxim.codekeeper.core.model.graph` - object dependency graph classes, built using JGraphT library. Notable classes:
  - `DepcyGraph` - the dependency graph and its builder.
  - `DepcyResolver` - main script building algorithm. Creates a list of `StatementAction`s according to requested object changes and object dependencies.
  - `DepcyTreeExtender` - extends an object diff tree according to object dependencies.
  - `DepcyWriter` - prints a dependency tree for an object.
- `ru.taximaxim.codekeeper.core.model.exporter` - classes that save and update the project files on disk. Notable classes:
  - `ModelExporter`, `MsModelExporter`, `AbstractModelExporter` - exporters for PG and MS projects and their common abstract part.
  - `OverridesModelExporter` - exports project structure containing only permission overrides.
- `ru.taximaxim.codekeeper.core.sql` - a categorized list of all PostgreSQL keywords. Generated from PostgreSQL source.
- `ru.taximaxim.codekeeper.core.ignoreparser` - builder for `IgnoreList`s.
- `ru.taximaxim.codekeeper.core` - main package containing general stuff: e.g. string constants, utils.
- `antlr-src` - sources for ANTLR4 parsers. We maintain parsers for PostgreSQL, pl/pgsql, T-SQL, and also our custom Ignore Lists and PostgreSQL ACLs syntax.
These need to be built using your preferred ANTLR4 builder into `ru.taximaxim.codekeeper.core.parsers.antlr` package.

#### ru.taximaxim.codekeeper.ui

Majority of code in this module implements Eclipse integration, and also wraps core logic into a more UI-suitable forms.

- `plugin.xml` - main Eclipse integration file. All integration code is referenced from here.
- `ru.taximaxim.codekeeper.ui.editors` - main project editor and its supporting classes. Notable classes:
  - `ProjectEditorDiffer` - main project editor. Loads and compares databases, allows user to select changes and saves the into project or generates and diff script for the database.
- `ru.taximaxim.codekeeper.ui.differ` - classes for diff creation. Notable classes:
  - `DbSource` - an abstraction, factory, and implementations for various database loaders.
  - `TreeDiffer`, `Differ` - first creates a diff tree, second creates a diff script based on the tree and user selection.
  - `DiffTableViewer` - GUI element responsible for showing tree diff elements. Part of `ProjectEditorDiffer` separated for reuse in other UIs.
- `ru.taximaxim.codekeeper.ui.pgdbproject` - project classes and various project wizards (New, Import, etc). Notable classes:
  - `PgDbProject` - utility wrapper for Eclipse's `IProject`.
- `ru.taximaxim.codekeeper.ui.pgdbproject.parser` - UI-specific parts of database loaders. Notable classes:
  - `PgDbParser` - stores, serializes and updates project indices (object metadata and references).
  - `UIProjectLoader` - loads a database from project structure in Eclipse Workspace. **(!)** Keep in sync with `ProjectLoader`.
  - `PgUIDumpLoader` - loads objects into a database structure from an Eclipse's `IFile`. **(!)** Keep in sync with `PgDumpLoader`.
- `ru.taximaxim.codekeeper.ui.builders` - Project Builders launch project index updates (metadata, errors etc) when an IDE build is triggered.
- `ru.taximaxim.codekeeper.ui.sqledit` - SQL code editor. Notable classes:
  - `SQLEditorSourceViewerConfiguration` - main editor configuration class, ties in most other editor classes.
- `ru.taximaxim.codekeeper.ui.dbstore` - database connection storage and GUI selection classes. Notable classes:
  - `DbInfo` - database connection parameters.
- `ru.taximaxim.codekeeper.ui.handlers` - handlers for commands (IDE actions) registered with Eclipse in `plugin.xml`.
- `ru.taximaxim.codekeeper.ui.job` - singleton wrapper for Eclipse `Job`s.
- `ru.taximaxim.codekeeper.ui.dialogs` - various dialogs. Notable classes:
  - `ExceptionNotifier` - reports and logs errors using Eclipse's `StatusManager`.
- `ru.taximaxim.codekeeper.ui.externalcalls` - logic for calling external utilities (pg_dump etc).
- `ru.taximaxim.codekeeper.ui.generators` - value generator for dummy data generation wizard (`MockDataWizard`).
- `ru.taximaxim.codekeeper.ui.prefs` - Eclipse Preferences integration - global program settings.
- `ru.taximaxim.codekeeper.ui.properties` - per-project settings.
- `ru.taximaxim.codekeeper.ui.reports` - Google Analytics reports.
- `ru.taximaxim.codekeeper.ui.views` - views showing misc info. Notable classes:
  - `DepcyGraphView` - shows dependency graph for objects currently selected in Project Editor.
  - `ProjectOverrideView` - shows library objects overridden by the project in current Project Editor.
  - `ResultSetView` - shows rows returned by SELECT or other commands executed in a script in SQL Editor.
- `ru.taximaxim.codekeeper.ui.views.navigator` - Eclipse Project Explorer integration, see also `plugin.xml`
- `ru.taximaxim.codekeeper.ui` - main package containing general stuff: e.g. string constants, utils. Notable classes:
  - `UiSync` - exception safety wrapper around SWT's `Display.asyncExec()`. **(!)** Use this instead of calling `getDisplay().asyncExec()`.
  - `UIConsts` - majority of string IDs and other constants commonly used in UI.

#### ru.taximaxim.codekeeper.core.tests

Majority of tests are here.

- `ru.taximaxim.codekeeper.core` - `PgDiffTest`, `MsDiffTest` - these test cases load old and new database schemas, generate a migration script and compare it to the expected diff file.
- `ru.taximaxim.codekeeper.core.depcies` - tests here work similarly to simple diff tests above, except the concept of "user selection" is added. Migration script is built only with objects in `usr` files as starting points, other objects must be added to the script via dependency mechanism.
- `ru.taximaxim.codekeeper.core.loader` - `PgAntlrLoaderTest`, `MsAntlrLoaderTest` - these tests load simple schemas from files and compare loaded PgDatabase objects with predefined ones.
- `ru.taximaxim.codekeeper.core.parsers` - these tests simply parse test pieces of code (taken from PostgreSQL source) to verify parser validity.
- `ru.taximaxim.codekeeper.core.parsers.antlr.expr` - these tests verify expression analysis and type inference mechanism.
- `ru.taximaxim.codekeeper.core.model.exporter` - these test update exported project directories and verify which files have actually been changed.

#### ru.taximaxim.codekeeper.ui.tests

Simple tests for core logic wrappers: `DbSource`, `Differ`, `ProjectUpdater`.

### Program Lifecycle

General program lifecycle goes as follows:
1. `PgDiffArguments` object is filled with operation parameters.
2. `PgDatabase`s are loaded from requested sources, including their libraries and privilege overrides. Ignored schemas are skipped at this step.
   1. During the load dependencies of each object are found and recorded. Expressions are also analyzed to extract their dependencies including overloaded function calls.
   2. All parser and expression analysis operations are run in parallel using `AntlrParser.ANTLR_POOL` thread pool to speed up the process. Parallel operations are serialized by calling `finishLoaders` at the end of each loading process.
3. The diff tree (represented by root `TreeElement`) is created by comparing two `PgDatabase`s. In GUI this is then shown to the user to assess the situation and choose objects to work with.
4. The diff tree, now containing "user selection", is used to selectively update project files on disk, or to generate a migration script.
5. In latter case, each "selected" TreeElement is passed to `DepcyResolver` to generate script actions fulfilling the requested change, including actions on dependent objects. To do this, JGraphT object dependency graphs are built using dependency information found at the loading stage.
6. Generated actions are now converted into SQL code with some last-moment post-processing and filtering.
7. Generated SQL script is written to a file/stdout or shown in the GUI for user to review and run on their database.


# Fabric Loom

A [Gradle](https://gradle.org/) plugin to setup a deobfuscated development environment for Minecraft mods. Primarily used in the Fabric toolchain.

* Has built in support for tiny mappings (Used by [Yarn](https://github.com/FabricMC/yarn))
* Utilises the Fernflower and CFR decompilers to generate source code with comments.
* Designed to support modern versions of Minecraft (Tested with 1.14.4 and upwards)
* Built in support for IntelliJ IDEA, Eclipse and Visual Studio Code to generate run configurations for Minecraft.
* Loom targets the latest version of Gradle 7 or newer
* Supports Java 16 upwards

## Use Loom to develop mods

To get started developing your own mods please follow the guide on [Setting up a mod development environment](https://fabricmc.net/wiki/tutorial:setup).

## Debugging Loom (Only needed if you want to work on Loom itself)

_This guide assumes you are using IntelliJ IDEA, other IDE's have not been tested; your experience may vary._

1. Import as a Gradle project by opening the build.gradle
2. Create a Gradle run configuration to run the following tasks `build publishToMavenLocal -x test`. This will build Loom and publish to a local maven repo without running the test suite. You can run it now.
3. Prepare a project for using the local version of Loom:
   * A good starting point is to clone the [fabric-example-mod](https://github.com/FabricMC/fabric-example-mod) into your working directory
   * Add `mavenLocal()` to the repositories:
     * If you're using `id 'fabric-loom'` inside `plugins`, the correct `repositories` block is inside `pluginManagement` in settings.gradle
     * If you're using `apply plugin:` for Loom, the correct `repositories` block is inside `buildscript` in build.gradle
   * Change the loom version to `0.6.local`. For example `id 'fabric-loom' version '0.6.local'`
4. Create a Gradle run configuration:
   * Set the Gradle project path to the project you have just configured above
   * Set some tasks to run, such as `clean build` you can change these to suit your needs.
   * Add the run configuration you created earlier to the "Before Launch" section to rebuild loom each time you debug
5. You should now be able to run the configuration in debug mode, with working breakpoints.


# com.palantir.consistent-versions [ ![Download](https://api.bintray.com/packages/palantir/releases/gradle-consistent-versions/images/download.svg) ](https://plugins.gradle.org/plugin/com.palantir.consistent-versions)

_A gradle plugin to ensure your dependency versions are *consistent* across all subprojects, without requiring you to hunt down and force every single conflicting transitive dependency._

Direct dependencies are specified in a top level `versions.props` file and then the plugin relies on [Gradle constraints][dependency constraints] to figure out sensible versions for all transitive dependencies - finally the whole transitive graph is captured in a compact `versions.lock` file.

1. Apply the plugin (root project only):
    ```gradle
    plugins {
        id "com.palantir.consistent-versions" version "<current version>"
    }
    ```
    You can find the current version under [releases](https://github.com/palantir/gradle-consistent-versions/releases).

2. In one of your build.gradle files, define a _versionless_ dependency on some jar:

    ```gradle
    apply plugin: 'java'

    dependencies {
        implementation 'com.squareup.okhttp3:okhttp'
    }
    ```

3. Create a `versions.props` file and provide a version number for the jar you just added:

    ```
    com.squareup.okhttp3:okhttp = 3.12.0
    ```

4. Run **`./gradlew writeVersionsLocks`** and see your versions.lock file be automatically created. This file should be checked into your repo:

    ```bash
    # Run ./gradlew writeVersionsLocks to regenerate this file
    com.squareup.okhttp3:okhttp:3.12.0 (1 constraints: 38053b3b)
    com.squareup.okio:okio:1.15.0 (1 constraints: 810cbb09)
    ```

## Contents
1. [Motivation](#motivation)
    1. An evolution of `nebula.dependency-recommender`
1. [Concepts](#concepts)
    1. versions.props: lower bounds for dependencies
    1. versions.lock: compact representation of your prod classpath
    1. ./gradlew why
    1. ./gradlew checkUnusedConstraints
    1. getVersion
    1. BOMs
    1. Specifying exact versions
    1. Downgrading things
    1. Common workflow: SLF4J
    1. Common workflow: dependencySubstitution
    1. Common workflow: internal test utility projects
    1. Resolving dependencies at configuration time is banned
    1. Known limitation: root project must have a unique name
    1. Scala
1. [Migration](#migration)
    1. How to make this work with Baseline
    1. `dependencyRecommendations.getRecommendedVersion` -> `getVersion`
1. [Technical explanation](#technical-explanation)
    1. Are these vanilla Gradle lockfiles?
    1. Conflict-safe lock files

## Motivation
Many languages have arrived at a similar workflow for dependency management: direct dependencies are specified in some top level file and then the build tool figures out a sensible version for all the transitive dependencies.  The whole dependency graph is then written out to some lock file. We asked ourselves, why doesn't this exist for Java?

- JavaScript repos using [yarn](https://yarnpkg.com/) define dependencies in a `package.json` and then yarn auto-generates a `yarn.lock` file.
- Rust repos using [cargo](https://doc.rust-lang.org/stable/cargo/) define direct dependencies in a `cargo.toml` file and have `cargo.lock` auto-generated.
- Go repos using [dep](https://golang.github.io/dep/) define dependencies in a `Gopkg.toml` file and have `Gopkg.lock` auto-generated.
- Ruby repos using [bundler](https://bundler.io/) define dependencies in a `Gemfile` file and have `Gemfile.lock` auto-generated.

Specifically this plugin delivers:

1. **one version per library** - When you have many Gradle subprojects, it can be frustrating to have lots of different versions of the same library floating around on your classpath. You usually just want one version of Jackson, one version of Guava etc. (`failOnVersionConflict()` does the job, but it comes with some significant downsides - see below). With gradle-consistent-versions, dependencies across all your subprojects have a sensible version provided.
1. **better visibility into dependency changes** - Small changes to your requested dependencies can have cascading effects on your transitive graph.  For example, you might find that a minor bump of some library brings in 30 new jars, or affects the versions of your other dependencies. With gradle-consistent-versions, it's easy to spot these changes in your `versions.lock`.

### An evolution of `nebula.dependency-recommender`
[nebula.dependency-recommender][] pioneered the idea of 'versionless dependencies', where gradle files just declare dependencies using `compile "group:name"` and then versions are declared separately (e.g. in a `versions.props` file). Using `failOnVersionConflict()` and nebula.dependency-recommender's `OverrideTransitives` ensures there's only one version of each jar on the classpath.

nebula.dependency-recommender *forces* all your dependencies, which overrides all version information that libraries themselves provide.

Unfortunately, failOnVersionConflict means developers often pick conflict resolution versions out of thin air, without knowledge of the actual requested ranges. This is dangerous because users may unwittingly pick versions that actually violate dependency constraints and may break at runtime, resulting in runtime errors suchs as `ClassNotFoundException`, `NoSuchMethodException` etc

[nebula.dependency-recommender]: https://github.com/nebula-plugins/nebula-dependency-recommender-plugin
[dependency constraints]: https://docs.gradle.org/current/userguide/dependency_constraints.html
[gradle BOM import]: https://docs.gradle.org/5.1/userguide/managing_transitive_dependencies.html#sec:bom_import

## Concepts

### versions.props: lower bounds for dependencies
Specify versions for your direct dependencies in a single root-level `versions.props` file. Think of these versions as the _minimum_ versions your project requires.

```
com.fasterxml.jackson.*:jackson-* = 2.9.6
com.google.guava:guava = 21.0
com.squareup.okhttp3:okhttp = 3.12.0
junit:junit = 4.12
org.assertj:* = 3.10.0
```

The * notation ensures that every matching jar will have the same version - they will be aligned[1](#rule-specificity) using a [virtual platform][].

Note that this does not force okhttp to exactly 3.12.0, it just declares that your project requires at least 3.12.0.  If something else in your transitive graph needs a newer version, Gradle will happily select this.  See below for how to downgrade something if you really know what you're doing.

[1]: If multiple lines from `versions.props` match a particular jar, **the most specific one** will be chosen (the one with the most characters being different from `*`).
This has the side effect that a line referring specifically to a jar is independent, and that jar's version never gets aligned to the versions of other jars, even if there are other lines containing `*` which would otherwise match that jar.

[virtual platform]: https://docs.gradle.org/current/userguide/dependency_version_alignment.html

### versions.lock: compact representation of your prod classpath
When you run `./gradlew --write-locks`, the plugin will automatically write a new file: `versions.lock` which contains a version for every single one of your transitive dependencies.

Notably, this lockfile is a _compact_ representation of your dependency graph as it just has one line per dependency (unlike nebula lock files which spanned thousands of lines).

```
javax.annotation:javax.annotation-api:1.4.0 (3 constraints: 1a310f90)
javax.inject:javax.inject:1 (2 constraints: d614a0ab)
javax.servlet:javax.servlet-api:3.1.0 (1 constraints: 830dcc28)
javax.validation:validation-api:1.1.0.Final (3 constraints: dc393f20)
javax.ws.rs:javax.ws.rs-api:2.0.1 (8 constraints: 7e9ce067)

[Test dependencies]
cglib:cglib-nodep:3.1 (1 constraints: 2a0e1330)
com.github.zafarkhaja:java-semver:0.9.0 (1 constraints: c315c0d2)
com.jayway.awaitility:awaitility:1.6.5 (1 constraints: c615c1d2)
```

The lockfile sources production dependencies from the _compileClasspath_ and _runtimeClasspath_ configurations, and
test dependencies from the compile/runtime classpaths of any source set named `jmh` or whose name ends in "test"
(e.g. `test`, `integrationTest`, `eteTest`).

There is a `verifyLocks` task (automatically run as part of `check`) that will ensure `versions.lock` is still consistent
with the current dependencies.

### ./gradlew why
To understand why a particular version in your lockfile has been chosen, run `./gradlew why --dependency <dependency>` to expand the constraints:
```
> Task :why
com.fasterxml.jackson.core:jackson-databind:2.9.8
        com.fasterxml.jackson.module:jackson-module-jaxb-annotations -> 2.9.8
        com.netflix.feign:feign-jackson -> 2.6.4
        com.palantir.config.crypto:encrypted-config-value -> 2.6.1
        com.palantir.config.crypto:encrypted-config-value-module -> 2.6.1
```

This is effectively just a more concise version of `dependencyInsight`:

```
./gradlew  dependencyInsight --configuration unifiedClasspath --dependency jackson-databind
```

## ./gradlew checkUnusedConstraints
`checkUnusedConstraints` prevents unnecessary constraints from accruing in your `versions.props` file. Run
`./gradlew checkUnusedConstraints --fix` to automatically remove any unused constraints from your props file.

### getVersion
If you want to use the resolved version of some dependency elsewhere in your Gradle files, gradle-consistent-versions offers the `getVersion(group, name, [configuration])` convenience function. For example:

```gradle
task demo {
    doLast {
        println "We chose guava " + getVersion('com.google.guava', 'guava')
    }
}
```

This function may not be invoked at [Gradle Configuration time](https://docs.gradle.org/current/userguide/build_lifecycle.html) as it involves resolving dependencies.  Put it inside a closure or provider to ensure it is only invoked at Execution time.

By default, this function resolve the `unifiedClasspath` configuration to supply a version, but you can always supply a different configuration if you want to:

```gradle
task printSparkVersion {
    doLast {
        println "Using spark version: " + getVersion('org.apache.spark', 'spark-sql_2.11', configurations.spark)
    }
}
```

### BOMs
Gradle has [first-class support][bom] for sourcing version constraints from published BOMs so they work fine with gradle-consistent-versions:

```gradle
allprojects {
    apply plugin: 'java-base'
    dependencies {
        rootConfiguration platform('com.foo.bar:your-bom')
    }
}
```
[bom]: https://docs.gradle.org/4.6/release-notes.html#bom-import

Make sure you apply BOMs within an `allprojects` closure, as gradle-consistent-versions must be able to unify constraints from all subprojects.

Note: **`java-base` is necessary**, even on projects that don't have java source code, otherwise gradle will silently interpret the `platform(...)` dependency as if it was a normal library dependency, and will not import the constraints from that BOM.

### Specifying exact versions
The preferred way to control your dependency graph is using [dependency constraints][] on gradle-consistent-versions' `rootConfiguration`. For example:

```gradle
dependencies {
    constraints {
        rootConfiguration 'org.conscrypt:conscrypt-openjdk-uber', {
            version { strictly '1.4.1' }
            because '1.4.2 requires newer glibc than available on Centos6'
        }

        rootConfiguration 'io.dropwizard.metrics:metrics-core', {
            version { strictly '[3, 4[' }
            because "Spark still uses 3.X, which can't co-exist with 4.X"
        }
    }
}
```

Gradle will fail if something in your dependency graph is unable to satisfy these [strictly][] constraints. This is desirable because nothing is _forced_ in your transitive graph.

[strictly]: https://docs.gradle.org/current/userguide/declaring_dependencies.html#sub:declaring_dependency_rich_version

### Downgrading things
If you discover a bug in some library on your classpath, the recommended approach is to use dependencyInsight to figure out why that version is on your classpath in the first place and then downgrade things until that library is no longer brought in.  Once the dependency is gone, you can specify a rootConfiguration constraint to make sure it doesn't come back (see above).

```
./gradlew dependencyInsight --configuration unifiedClasspath --dependency retrofit
```

Occasionally however, downgrading things like this is not feasible and you just want to force a particular transitive dependency.  This is dangerous because something in your transitive graph clearly compiled against this library and might be relying on methods only present in the newer version, so forcing down may result in NoSuchMethodErrors at runtime on certain codepaths.

```gradle
allprojects {
    configurations.all {
        resolutionStrategy {
            force 'com.squareup.retrofit2:retrofit:2.4.0'
        }
    }
}
```

### Common workflow: SLF4J
Developers usually want just one SLF4J implementation on the classpath. If some of your dependencies rely on their own logging implementations (e.g. commons-logging or log4j), you can use the following snippet to ensure that all logging will go through SLF4J.

```gradle
allprojects {
    dependencies {
        modules {
            module('commons-logging:commons-logging') {
                replacedBy('org.slf4j:jcl-over-slf4j', 'slf4j allows us supply our own implementation')
            }
            module('log4j:log4j') {
                replacedBy('org.slf4j:log4j-over-slf4j', 'slf4j allows us supply our own implementation')
            }
        }
    }
}
```

### Common workflow: dependencySubstitution
We've seen the following error when using dependencySubstitution:
```
> Could not find method module() for arguments [org.glassfish.hk2.external:javax.inject] on configuration ':my-project:subprojectUnifiedClasspathCopy' of type org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.
```
Adding explicit `it` calls works around this error:
```diff
 configurations.configureEach {
     resolutionStrategy.dependencySubstitution {
-        substitute module('org.glassfish.hk2.external:javax.inject') with module('javax.inject:javax.inject:1')
+        it.substitute it.module('org.glassfish.hk2.external:javax.inject') with it.module('javax.inject:javax.inject:1')
     }
 }
```

### Common workflow: internal test utility projects
Sometimes, devs have multiple test projects (unit tests, integration tests) that use a subset of common test classes.
```
* :foo
   \--- source set 'test'
         \--- :foo-test-common
* :foo-integration
   \--- source set 'integrationTest'
         \--- :foo-test-common
* :foo-test-common
   \--- source set 'main'
```

In this case, we'd like to prevent GCV from locking `:foo-test-common`'s `main` source set to production dependencies,
and instead treat the entire project as test dependencies. We can do this via the following snippet:

```gradle
# foo-test-common/build.gradle
apply plugin: 'java'

versionsLock {
    testProject()
}
```

### Resolving dependencies at configuration time is banned
In order for this plugin to function, we must be able to guarantee that no dependencies are resolved at configuration time.  Gradle already [recommends this](https://guides.gradle.org/performance/#don_t_resolve_dependencies_at_configuration_time) but gradle-consistent-versions enforces it.

In many cases, it's just a matter of using a closure or a provider, for example:

```diff
 task copySomething(type: Copy) {
-    from configurations.spark.singleFile // 🌶🌶🌶 this downloads spark at configuration time, slowing down every `./gradlew` invocation!
+    from { configurations.spark.singleFile }
     into "$buildDir/foo/bar"
 }
```

### Known limitation: root project must have a unique name
Due to an implementation detail of this plugin, we require settings.gradle to declare a `rootProject.name` which is unique.

```diff
+rootProject.name = 'tracing-root'
-rootProject.name = 'tracing'

 include 'tracing'
 include 'tracing-api'
 include 'tracing-jaxrs'
 include 'tracing-okhttp3'
 include 'tracing-jersey'
 include 'tracing-servlet'
 include 'tracing-undertow'
```

### Scala
By default, this plugin will apply the constraints from `versions.props` to _all_ configurations.
To exclude a configuration from receiving the constraints, you can add it to `excludeConfigurations`, configurable through the `versionRecommendations` extension (in the root project):

    versionRecommendations {
        excludeConfigurations 'zinc'
    }

## Migration
Using a combination of automation and some elbow grease, we've migrated ~150 projects from `nebula.dependency-recommender` to `com.palantir.consistent-version`:

```diff
 plugins {
+    id 'com.palantir.consistent-versions' version '<current version>'
 }

 allprojects {
     dependencies {
+        rootConfiguration platform('com.palantir.witchcraft:witchcraft-core-bom')
     }

-    dependencyRecommendations {
-        mavenBom module: 'com.palantir.witchcraft:witchcraft-core-bom'
-    }

-    configurations.all {
-        resolutionStrategy {
-            failOnVersionConflict()
-        }
-    }
 }
```

_You can also likely delete the 'conflict resolution' section of your versions.props._

### How to make this work with Baseline
GCV will just work out of the box with Baseline 3.0.0 and newer.

_If still using Baseline 2.x, then you'll need to disable its own versions plugin which conflicts with GCV:_

Add the following to your `gradle.properties` fully turn off nebula.dependency-recommender (only necessary if you use [`com.palantir.baseline`](https://github.com/palantir/gradle-baseline/#usage)):

```diff
 org.gradle.parallel=true
+com.palantir.baseline-versions.disable=true
```

### `dependencyRecommendations.getRecommendedVersion` -> `getVersion`
If you rely on this Nebula function, then gradle-consistent-versions has a similar alternative:

```diff
-println dependencyRecommendations.getRecommendedVersion('com.google.guava:guava')
+println getVersion('com.google.guava:guava')
```

Note that you can't invoke this function at configuration time (e.g. in the body of a task declaration), because the plugin needs to resolve dependencies to return the answer and Gradle [strongly discourages](https://guides.gradle.org/performance/#don_t_resolve_dependencies_at_configuration_time) resolving dependencies at configuration time.

Alternatives:

- if you rely on this function for sls-packaging `productDependencies`, use `detectConstraints = true` or upgrade to 3.X
- if you rely on this function to configure the `from` or `to` parameters of a `Copy` task, use a closure or move the whole thing into a doLast block.

```diff
 task copySomething(type: Copy) {
-    from "$buildDir/foo/bar-${dependencyRecommendations.getRecommendedVersion('group', 'bar')}"
+    from { "$buildDir/foo/bar-${getVersion('group:bar')}" }
     ...
```

## Technical explanation

### Are these vanilla Gradle lockfiles?
No.  We tried Gradle 4.8's [first-class lockfiles](https://docs.gradle.org/current/userguide/dependency_locking.html), but found a critical usability problem: it allowed semantic merge conflicts.  For example:

- Two PRs might change apparently unrelated dependencies - both changing different lines of gradle's `gradle/dependency-locks/x.lockfile`
- One PR merges first, so develop has a few lockfile lines updated
- Second PR has no git conflicts with develop, so it merges
- Develop is now broken due to a semantic merge conflict in the gradle lockfile!

Concrete example:

- PR1
  - before PR: com.palantir.spark.files:file-relation required jaxrs-clients 3.45.0
  - after PR: com.palantir.spark.files:file-relation doesn't pull in jaxrs-clients at all
  - unchanged: com.palantir.witchcraft:witchcraft-core required jaxrs-clients 3.43.0
  - result is: gradle's lockfile therefore picked jaxrs-clients 3.43.0
- PR2
  - before PR: com.palantir.witchcraft:witchcraft-core required remoting 3.43.0
  - after PR: com.palantir.witchcraft:witchcraft-core required remoting 3.45.0
  - result is: gradle's lockfile line for jaxrs-clients was _unchanged_ (still 3.45.0)

The badness arises when:

1. user merges PR1 onto develop
1. PR2 has no conflicts because it didn't change the jaxrs-clients 3.45.0 line that PR1 downgraded to 3.43.0.
1. user merges PR2 onto develop
1. develop is now broken, because PR2 required jaxrs-clients 3.45.0 but PR1 had downgraded it on develop to 3.43.0 and git didn't detect this conflict! (This is a semantic merge conflict)

Our format avoids this problem by adding some redundant information about the dependency graph so that changes like these will result in pre-merge _git conflicts_ instead of post-merge _semantic conflicts_.

### Conflict-safe lock files
Our format extends gradle's lockfiles format by writing down who wanted each component (`com.google.guava:guava`) and which version they requestedm, e.g.

```
com.google.guava:guava:18.0 (2 constraints 4a440103)
```

The hash, `4a440103`, is derived from Guava's dependents:

```
com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
com.github.rholder:guava-retrying -> [10.+,)
```

This ensures that if two PRs affect the set of constraints on guava they will result in a _git conflict_, which prevents the semantic merge conflicts described above and ensures develop won't break:

```
com.google.guava:guava:10.0 (1 constraints: 50295fc1)
```

The hash from one side of the merge confict (`f59715c4`) came from adding a tracing dependent:
```diff
 com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
+com.palantir.tracing:tracing -> 16.0
 com.github.rholder:guava-retrying -> [10.+,)
```

While the other hash `50295fc1`, is derived by deleting jackson, leaving just a single dependent:
```diff
-com.fasterxml.jackson.datatype:jackson-datatype-guava -> 18.0
 com.github.rholder:guava-retrying -> [10.+,)
```

With vanilla Gradle 4.8 lockfiles, this scenario would have merged but then failed on develop due to an inconsistent lockfile.

### Comparison to nebula dependency lock plugin
Internally, a few projects started using Nebula's [Gradle Dependency Lock Plugin](https://github.com/nebula-plugins/gradle-dependency-lock-plugin) but eventually abandoned it because it introduced too much dev friction.  Key problems:

- PR diffs could be thousands and thousands of lines for small changes
- Contributors often updated versions.props but forgot to update the lockfiles, leading them to think they'd picked up a new version when they actually hadn't!

Both of these problems are solved by this plugin because the new gradle lock files are extremely compact (one line per dependency) and they can never get out of date because gradle validates their correctness every time you resolve the unifiedClasspath configuration.


# Nalu

[![License](https://img.shields.io/:license-apache-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)
![GWT3/J2CL compatible](https://img.shields.io/badge/GWT3/J2CL-compatible-brightgreen.svg)
[![Join the chat at https://gitter.im/Nalukit42/Lobby](https://badges.gitter.im/Nalukit42/Lobby.svg)](https://gitter.im/Nalukit42/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Maven Central](https://img.shields.io/maven-central/v/com.github.nalukit/nalu.svg?colorB=44cc11)](https://central.sonatype.com/artifact/com.github.nalukit/nalu)
[![Build & Deploy](https://github.com/NaluKit/nalu/actions/workflows/build.yaml/badge.svg?branch=dev)](https://github.com/NaluKit/nalu/actions/workflows/build.yaml)

Nalu is a tiny framework that helps you to create GWT based applications quite easily. Using the HTML 5 history for routing and navigation, Nalu supports the browser's back-, forward-, and reload-button by default and without any need to implement anything.

Nalu supports:

* Route based navigation

* HTML links and programmatically routing thanks to a router

* Browser's back-, forward- and reload-button (In case the hasHistory-attribute of the `Application`-annotation is set to **true**)

* An optional loader that will be executed at application start to load data from the server

* Client side context, router and event bus which will be automatically injected in every controller, filter, loader and handler (Handlers have only access to the context and the event bus)

* Filters to intercept routing.

* Separation of views into controllers and components with framework sided injection - similar to GWT Activities.

* A controller life-cycle using `start`-, `mayStop`- and `stop`- similar to GWT Activities.

* Popups

* Controller based handler manager, that will remove all handlers from the event bus in case the controller is stopped to prevent memory leaks (handler registrations must be added to the manager).

* UiBinder (nalu-plugin-gwt)

* Composites to support smaller units of work

* Controller & composite caching

* Component creation inside a controller to support GWT replacement rules and static factory methods

* Multi Shell Support

* (nearly) Every widget lib (tested with: GWT, GXT, Domino-UI v1 & v2, Elemento, Elemental2, GWT-Material)

* Maven multi module projects to separate an application in smaller parts (module feature)

* Tracking user routing (in case a tracker is added)

## Basic Concept
Nalu uses a hash to navigate. **Starting with version 1.1.0 Nalu supports the use of hash less URLs.** Everything explained here will also work without hash. In case of working with a hash less URL, you need to implement something on the server to handle a reload.

Example hash:
```
#[shell]/[route]/:[parameter_1]/:[parameter_2]
```

where
* shell: the shell to be used to display the route
* route: navigation end point (a controller)
* parameter_x: parameters of the route

The following flow shows the steps, once a routing is initiated. The flow will end with appending the new component to the DOM.

![Route Flow](https://github.com/NaluKit/nalu/blob/main/etc/images/routeFlow.png)

To connect a component to a route, just create a controller class which extend
`AbstractComponentController` and add the controller annotation `@Controller`.
```JAVA
@Controller(route = "/shell/route/{parameter_01}/{parameter_02}",
            selector = "content",
            component = MyComponent.class,
            componentInterface = IMyComponent.class)
public class MyController
    extends AbstractComponentController<MyApplicationContext, IMyComponent, HTMLElement>
    implements ISearchComponent.Controller {

  @AcceptParameter("parameter_01")
  public void setParaemter01(String p01)
      throws RoutingInterceptionException {
      ...
  }

  @AcceptParameter("parameter_02")
  public void setParaemter02(String p02)
      throws RoutingInterceptionException {
      ...
  }

}
```
To navigate to a new route use inside a controller:
```JAVA
    this.router.route("/shell/route/{parameter_01}/{parameter_02}",
                      parameter_1,
                      parameter_2);
```
The router is injected in the controller. To route to a new component call the route method and add at least the new route. In case the route has parameters, just add them as additional parameters. (**Important note:** parameters are always Strings!)

## Using

### Dependencies
To use Nalu add the following dependencies to your pom:

* **GWT 2.8.2**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu</artifactId>
    <version>2.10.1-gwt-2.8.2</version>
</dependency>
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-processor</artifactId>
    <version>2.10.1-gwt-2.8.2</version>
    <scope>provided</scope>
</dependency>
```
* **GWT 2.9.0 (and newer) - SNAPSHOT**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu</artifactId>
    <version>HEAD-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-processor</artifactId>
    <version>HEAD-SNAPSHOT</version>
    <scope>provided</scope>
</dependency>
```
* **GWT 2.9.0 (and newer)  - Release**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu</artifactId>
    <version>2.20.0</version>
</dependency>
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-processor</artifactId>
    <version>2.20.0</version>
    <scope>provided</scope>
</dependency>
```

Depending on the widget set your project is using, add one of the following plugins:

If the project uses a widget set based on **Elemental2**, **Elemento** or **Domino-UI**, use the **Nalu-Plugin-Elemental2** by adding the following lines to your pom:

* **GWT 2.8.2**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemental2</artifactId>
    <version>2.10.1-gwt-2.8.2</version>
```

* **GWT 2.9.0 (and newer)  - SNAPSHOT**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemental2</artifactId>
    <version>HEAD-SNAPSHOT</version>
```

* **GWT 2.9.0 (and newer)  - Release**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemental2</artifactId>
    <version>2.20.0</version>
</dependency>
```

For Elemento there's a dedicated plugin which supports `org.jboss.gwt.elemento.core.IsElement<E>` as widget type:

* **GWT 2.8.2**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemento</artifactId>
    <version>2.10.1-gwt-2.8.2</version>
</dependency>
```

* **GWT 2.9.0 (and newer)  - SNAPSHOT**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemento</artifactId>
    <version>HEAD-SNAPSHOT</version>
</dependency>
```

* **GWT 2.9.0 (and newer)  - Release**
```XML
<dependency>
    <groupId>com.github.nalukit</groupId>
    <artifactId>nalu-plugin-elemento</artifactId>
    <version>2.20.0</version>
</dependency>
```

The **nalu-plugin-elemento** can also be used with Domino-ui.

**(These plugins are ready to use with J2CL / GWT 3)**

If your project uses a widget set based on **GWT** 2.8.2 or newer, use the **Nalu-Plugin-GWT** by adding the following lines to your pom:

* **GWT 2.8.2**
```XML
   <dependency>
     <groupId>com.github.nalukit</groupId>
     <artifactId>nalu-plugin-gwt</artifactId>
     <version>2.10.1-gwt-2.8.2</version>
   </dependency>
   <dependency>
      <groupId>com.github.nalukit</groupId>
      <artifactId>nalu-plugin-gwt-processor</artifactId>
      <version>2.10.1-gwt-2.8.2</version>
      <scope>provided</scope>
   </dependency>
```

* **GWT 2.9.0 (and newer)  - SNAPSHOT**
```XML
   <dependency>
      <groupId>com.github.nalukit</groupId>
      <artifactId>nalu-plugin-gwt</artifactId>
      <version>HEAD-SNAPSHOT</version>
   </dependency>
   <dependency>
      <groupId>com.github.nalukit</groupId>
      <artifactId>nalu-plugin-gwt-processor</artifactId>
      <version>HEAD-SNAPSHOT</version>
      <scope>provided</scope>
   </dependency>
```

* **GWT 2.9.0 (and newer)  - Release**
```XML
   <dependency>
      <groupId>com.github.nalukit</groupId>
      <artifactId>nalu-plugin-gwt</artifactId>
      <version>2.20.0</version>
   </dependency>
   <dependency>
      <groupId>com.github.nalukit</groupId>
      <artifactId>nalu-plugin-gwt-processor</artifactId>
      <version>2.20.0</version>
      <scope>provided</scope>
   </dependency>
```

**(This plugin will not work with J2CL / GWT 3)**

See the wiki for more information about Nalu and how to use it.

### Configuration

Due to a bug in older version of the maven-compiler-plugin, the output of a annotation processor - during a Maven build - will not be logged.
To ensure, that the processor output is logged, use the latest maven-compiler-plugin.

## Wiki
More useful information about Nalu and how to use it, can be found inside the [Wiki](https://github.com/nalukit/nalu/wiki).

## J2CL / GWT3
With the next version of GWT (GWT 3) and the new J2CL transpiler, there will be major changes in the GWT development. For example: JSNI and generators, besides other things, will be gone. To be prepared for the future things like JSNI, generators or any other dependency to GWT has to be removed and must be avoided.

Nalu uses only the already migrated `gwt-events` from `org.gwtproject`.

Nalu has **no** dependency to gwt-user nor Nalu's dependencies! Nalu does not use JSNI, generators or anything else from GWT (except the nalu-plugin-gwt) which will block moving to J2CL.

Nalu is ready to use with J2CL / GWT 3 as long as you do not use the nalu-plugin-gwt!

## To get in touch with the developer
Please visit use us at [Nalu Gitter room](https://gitter.im/Nalukit42/Lobby).

## Examples
Here you will find many examples that show Nalu in action: [https://github.com/nalukit/nalu-examples](https://github.com/nalukit/nalu-examples).

## YouTube
You will find a session about Nalu at the GWT Community Channel [here](https://www.youtube.com/watch?v=GsHY4f3cvRU).

## Notes
In case you find a bug, please open an issue or post it inside the [Nalu Gitter room](https://gitter.im/Nalukit42/Lobby).

## Comparsion between GWT Activities & Places versus Nalu
Anyone, who is familiar with GWT Activities & Places, will find [here](https://github.com/NaluKit/nalu/wiki/01.-Motivation-&-Concepts#comparison-gwt-activities--places-versus-nalu) a comparison between GWT Activities & Places and Nalu.


# FAIR Data Point

[![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/FAIRDataTeam/FAIRDataPoint?sort=semver)](https://github.com/FAIRDataTeam/FAIRDataPoint/releases)
[![Documentation Status](https://readthedocs.org/projects/fairdatapoint/badge/?version=latest)](https://fairdatapoint.readthedocs.io/en/latest/?badge=latest)
[![License](https://img.shields.io/github/license/FAIRDataTeam/FAIRDataPoint)](https://github.com/FAIRDataTeam/FAIRDataPoint/blob/develop/LICENSE)
[![Docker Pulls](https://img.shields.io/docker/pulls/fairdata/fairdatapoint)](https://hub.docker.com/r/fairdata/fairdatapoint)
[![Libraries.io](https://img.shields.io/librariesio/github/FAIRDataTeam/FAIRDataPoint)](https://libraries.io/github/FAIRDataTeam/FAIRDataPoint)

[**FAIR Data Point** (FDP)](https://www.fairdatapoint.org) is a REST API for creating, storing, and serving **FAIR
metadata**. This FDP implementation also presents a Web-based graphical user interface (GUI). The metadata contents are
generated **semi-automatically** according to
the [FAIR Data Point software specification](https://specs.fairdatapoint.org) document.

## Usage

More information about FDP, how to deploy it and use it can be found in
the [FDP Deployment and REST API usage Documentation](https://fairdatapoint.readthedocs.io/).

## Related GitHub Projects

- [FAIR Data Point Client](https://github.com/FAIRDataTeam/FAIRDataPoint-client)
- [FAIR Data Point E2E Tests](https://github.com/FAIRDataTeam/FAIRDataPoint-E2E-Tests)
- [FAIR Data Point Documentation](https://github.com/FAIRDataTeam/FAIRDataPoint-Docs)
- [OpenRefine Metadata Extension](https://github.com/FAIRDataTeam/OpenRefine-metadata-extension)

### API Documentation

FAIR Data Point API comes with an embedded [OpenAPI documentation using Swagger](https://swagger.io/specification/). The
details of API calls can be found there. It also allows trying out API calls directly. To access the FDP swagger
document please visit the following url via web
browser [localhost:8080/swagger-ui.html](http://localhost:8080/swagger-ui.html) (for local deployment)
or `https://your.domain.tld/swagger-ui.html` for your deployment (
e.g. [app.fairdatapoint.org/swagger-ui.html](https://app.fairdatapoint.org/swagger-ui.html)).  More detailed descriptions and examples of these API calls is available in the [Deployment and Usage instructions](https://fairdatapoint.readthedocs.io/)

## Development

### Technology Stack

- **Java** (JDK 21)
- **PostgreSQL** (15.0 or higher)
- **Maven** (3.2.5 or higher)
- **Docker** (19.03.0-ce or higher) - *for building Docker image only*

### Build & Run

To run the application, a PostgreSQL DB instance is required to be running. To configure the database with standard
connection (`postgresql://localhost:54321/fdp`, credentials `fdp:fdp`), simply instruct Spring Boot to use the
`development` profile. Then run:

```bash
$ mvn spring-boot:run -Dspring-boot.run.profiles=development
```

Alternatively, create an `application.yml` file in the project root
and [configure the database](https://fairdatapoint.readthedocs.io/en/latest/deployment/advanced-configuration.html)
or use environment variables (`FDP_POSTGRES_HOST`, `FDP_POSTGRES_PORT`, `FDP_POSTGRES_DB`, `FDP_POSTGRES_USERNAME`,
and `FDP_POSTGRES_PASSWORD`; and then run:

```bash
$ mvn spring-boot:run
```

### Run tests

Run from the root of the project:

```bash
$ mvn test
```

### Package the application

Run from the root of the project:

```bash
$ mvn package
```

### Create a Docker image

Run from the root of the project (requires building `jar` file using `mvn package` as shown above):

```bash
$ docker build -t fairdatapoint:local .
```

### Build using Docker

If you do not have Java and Maven locally, you can build the Docker image using Docker (instead of using locally
built `jar` file):

```bash
$ docker build -f Dockerfile.build -t fairdatapoint:local .
```

## Security

Most of the `GET` requests are publicly accessible. In contrast, `POST`, `PUT`, `DELETE`, and `PATCH` requests are
mainly secured. We use [JWT Tokens](https://jwt.io/)
and [Bearer Token Authentication](https://swagger.io/docs/specification/authentication/bearer-authentication/). The
token can be retrieved using `/tokens` endpoint where you send username and password. For details, visit the OpenAPI
documentation.

**Default users**

- **ADMIN:**
    - Username: `albert.einstein@example.com`
    - Password: `password`
- **USER:**
    - Username: `nikola.tesla@example.com`
    - Password: `password`

## Contributing

We maintain a [CHANGELOG](CHANGELOG.md), you should also take a look at our [Contributing guidelines](CONTRIBUTING.md)
and [Code of Conduct](CODE_OF_CONDUCT.md).

## Citation

The following paper can be cite as a reference paper for the FAIR Data Point:

    @article{10.1162/dint_a_00160,
    author = {Bonino da Silva Santos, Luiz Olavo and Burger, Kees and Kaliyaperumal, Rajaram and Wilkinson, Mark D.},
    title = "{FAIR Data Point: A FAIR-oriented approach for metadata publication}",
    journal = {Data Intelligence},
    pages = {1-21},
    year = {2022},
    month = {08},
    issn = {2641-435X},
    doi = {10.1162/dint_a_00160},
    url = {https://doi.org/10.1162/dint\_a\_00160},
    eprint = {https://direct.mit.edu/dint/article-pdf/doi/10.1162/dint\_a\_00160/2038268/dint\_a\_00160.pdf}}

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for more details.


# CWL Viewer

This is a [Spring Boot](http://projects.spring.io/spring-boot/) MVC application which fetches [Common Workflow Language](http://www.commonwl.org/) files from a Github repository and creates a page for it detailing the main workflow and its inputs, outputs and steps.

[![Build Status](https://github.com/common-workflow-language/cwlviewer/workflows/CWL%20Viewer%20Build/badge.svg?branch=main)](https://github.com/common-workflow-language/cwlviewer/actions?query=workflow%3A%22CWL%20Viewer%20Build%22) [![Coverage Status](https://coveralls.io/repos/github/common-workflow-language/cwlviewer/badge.svg)](https://coveralls.io/github/common-workflow-language/cwlviewer) [![Gitter](https://img.shields.io/gitter/room/gitterHQ/gitter.svg)](https://gitter.im/common-workflow-language/cwlviewer?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![](https://images.microbadger.com/badges/image/commonworkflowlanguage/cwlviewer.svg)](https://microbadger.com/images/commonworkflowlanguage/cwlviewer "MicroBadger commonworkflowlanguage/cwlviewer") [![Docker image commonworkflowlanguage/cwlviewer](https://images.microbadger.com/badges/version/commonworkflowlanguage/cwlviewer.svg)](https://hub.docker.com/r/commonworkflowlanguage/cwlviewer/ "Docker Hub commonworkflowlanguage/cwlviewer")
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.823534.svg)](https://doi.org/10.5281/zenodo.823534)

# Using CWL Viewer

You are recommended to use the **production instance** of CWL Viewer at https://view.commonwl.org/ which runs the latest [release](https://github.com/common-workflow-language/cwlviewer/releases). Any downtime should be reported on the [gitter chat for cwlviewer](https://gitter.im/common-workflow-language/cwlviewer).

<!--- I don't think this is a thing any more.
The **dev instance** at http://view.commonwl.org:8082/ corresponds to the current `master` branch, and is updated every 6 minutes to run the latest [commonworkflowlanguage/cwlviewer docker image](https://hub.docker.com/r/commonworkflowlanguage/cwlviewer/builds/). Note that this instance is NOT secured and might break at any time.
-->

# Running

If you are a developer, or you want to use the CWL Viewer in a closed environment, then you can run your own instance.

## Recommended - Running with Docker

This application can be started with [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/install/) and Docker Compose is the recommended method of running or developing this codebase.

Then run the following commands to clone the project in your local system.

   ```
    git clone https://github.com/common-workflow-language/cwlviewer.git
    cd cwlviewer
  ```

In the project directory, to start CWLViewer exposed on port `8080`, run:

    docker-compose up

The web server will connect to a local host, you'll see the message saying "Tomcat started on port(s):8080".

To see the locally running CWL Viewer app, visit http://localhost:8080/ in your web browser.

To stop and remove:

    docker-compose down

If you change the source code, then use this `docker-compose.override.yml` and
re-build with `docker-compose build`:

```yaml
version: '3.9'
services:
  spring:
    build: .
```

See the [docker-compose.yml](docker-compose.yml) file for details.

If you have modified the source code, then you may want to build the docker image locally first:

    docker build -t commonworkflowlanguage/cwlviewer .

## Running Spring Boot locally for development, with PostgreSQL and Jena Fuseki in Docker

Create `docker-compose.override.yml`:

```
version: '3.9'
services:
  postgres:
    ports:
     - "5432:5432"
  sparql:
    ports:
     - "3030:3030"
```

Then start the containers:

```
docker-compose up
```

Then start Spring Boot locally:

```
mvn spring-boot:run -Dserver.port=7999
```

Now you can connect to http://localhost:7999 in your browser.

## Deleting the data volumes to reset state

To completely reset the state, you must delete the data volumes:

```
docker-compose down
docker volume rm  cwlviewer_bundle cwlviewer_git cwlviewer_graphviz cwlviewer_postgres cwlviewer_sparql
```

## Running without Docker

### Requirements

#### PostgreSQL

You will need to have [PostgreSQL](https://www.postgresql.org/) running,
by default on `localhost:5432`

If you are running from the command line, you can override this by supplying
system properties like `-Dspring.datasource.url=jdbc:postgresql://localhost:5432/cwlviewer` and
`-Dspring.datasource.password=sa`

#### Apache Jena Fuseki (or alternative SPARQL server)

You will also need to have a SPARQL server such as [Apache Jena Fuseki](https://jena.apache.org/documentation/fuseki2/) running,
by default on `localhost:3030`

#### Ruby and Licensee

To retrieve license information, CWL Viewer uses the [Licensee](https://github.com/licensee/licensee) Ruby Gem. To install it,
[configure Ruby](https://www.ruby-lang.org/en/documentation/installation/) on your environment and then run

```bash
gem install licensee
```

You may use a dependency from your operating system package manager if you
prefer too, e.g. `ruby-licensee` for Ubuntu LTS 22.04.1.

Before running Maven, try running `licensee` in the command-line to verify
it was installed successfully.

## Compiling and Running

To compile you will need [Java 17](https://www.oracle.com/java/technologies/downloads/) or a compatible distribution
(e.g. [Eclipse Adoptium](https://projects.eclipse.org/projects/adoptium)) and version, as well as
[Apache Maven 3](https://maven.apache.org/download.cgi) (`apt install maven`).

Spring Boot uses an embedded HTTP server. The Spring Boot Maven plugin includes a run goal which can be used to quickly compile and run it:

```
$ mvn spring-boot:run
```

Alternatively, you can run the application from your IDE as a simple Java application by importing the Maven project.

You need to install [Graphviz](http://www.graphviz.org/) for all unit tests to pass.

You can create an executable JAR file by using:

    mvn clean install

Afterwards, run:

    java -jar target/cwlviewer*.jar

(The exact filename will vary per version)

Once CWL Viewer is running, you should see log output somewhat like:

```
()..)
s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
org.researchobject.CwlViewerApplication  : Started CwlViewerApplication in 28.604 seconds
```

Now check out http://localhost:8080/ to access CWL Viewer.

## Configuration

There are a variety of configuration options detailed in the [application configuration file](https://github.com/common-workflow-language/cwlviewer/blob/master/src/main/resources/application.properties) which can be adjusted.

When deploying with docker, these can be overridden externally by creating/modifying `docker-compose.override.yml` as follows:

```yaml
version: '3.9'
services:
  spring:
    environment:
            applicationName: Common Workflow Language Viewer
            applicationURL: https://view.commonwl.org
            cacheDays: 1
```

The properties can alternatively be provided as system properties on the
command line, e.g. `-DcacheDays=1` or via a [variety of other methods supported by Spring Boot](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html)

# Dump/restore

While you can perform backup of the Docker volumes,
for larger upgrades of CWL Viewer it is recommended instead to do a JSON dump
and re-load, which will force CWL Viewer to fetch and parse again.

The script `dump.py` can be used for regular backups, it will store the full
output of /workflows as one or multiple timestamped JSON files (you can use
`gzip` to compress them):

    $ python dump.py --viewer https://view.commonwl.org/ --output /var/backups --page 0 --size 100
      INFO:Viewer URL: https://view.commonwl.org/
      INFO:Output: /var/backups
      INFO:Dumping workflows from https://view.commonwl.org/, page 0, size 100 to /var/backups

    $ python dump.py -o /var/backups -a
      INFO:Viewer URL: https://view.commonwl.org/
      INFO:Output: /var/backups
      INFO:Dumping all the workflows from https://view.commonwl.org/ to /var/backups
      100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [04:39<00:00, 17.49s/it]

The script `load.py` (requires Python 3) can be used to restore from such JSON dumps:

    ./load.py /var/backups/cwl/2018-06-06T135133+0000.json.gz https://view.commonwl.org/

The optional parameter `--no-commits` can be added to skip those entries that
look like a commit ID. Note that this might break previous permalinks.

# Documentation

2017 Poster <https://doi.org/10.7490/f1000research.1114375.1?>

2017 Video overview <https://youtu.be/_yjhVTmvxLU>

2017 Technical Report <https://doi.org/10.5281/zenodo.823295>

## License

Distributed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
See the file [LICENSE.md](LICENSE.md) for details, and
[NOTICE.md](NOTICE.md) for required attribution notices.

## Contribute

Feel free to contribute! You may [raise an issue](https://github.com/common-workflow-language/cwlviewer/issues),
provide a [pull request](https://github.com/common-workflow-language/cwlviewer/pulls)
or join the [gitter chat for cwlviewer](https://gitter.im/common-workflow-language/cwlviewer)!

## Changelog
See [CHANGELOG](https://github.com/common-workflow-language/cwlviewer/blob/main/CHANGELOG.md)

## Making a development snapshot container image
(and optionally publishing that image to DockerHub)

```shell
# confirm the build arguments
# if these don't look correct, troubleshoot before continuing.
BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') VCS_REF=$(git rev-parse HEAD) VERSION=$(git describe)
echo BUILD_DATE=${BUILD_DATE} VCS_REF=${VCS_REF} VERSION=${VERSION}
# build the container image
docker build --build-arg BUILD_DATE=${BUILD_DATE} --build-arg VCS_REF=${VCS_REF} \
  --build-arg VERSION=${VERSION} \
  -t cwlviewer:${VERSION} .
# the rest is optional
docker tag cwlviewer:${VERSION} docker.io/commonworkflowlanguage/cwlviewer:${VERSION}
docker tag cwlviewer:${VERSION} quay.io/commonwl/cwlviewer:${VERSION}
docker push docker.io/commonworkflowlanguage/cwlviewer:${VERSION}
docker push quay.io/commonwl/cwlviewer:${VERSION}
```

## Making a release and publishing to GitHub, DockerHub, and Quay.io

After CHANGELOG.md has been updated and the `-SNAPSHOT` suffix removed from `pom.xml`, run the following:

```shell
git checkout main
git pull
new_version=1.4.3  # CHANGEME
# create an annotated git tag
git tag -a -m "release version ${new_version}" v${new_version}
# confirm the build arguments
# if these don't look correct, troubleshoot before continuing.
# for example, was your tag an annotated (-a) tag?
BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') VCS_REF=$(git rev-parse HEAD) VERSION=$(git describe)
echo BUILD_DATE=${BUILD_DATE} VCS_REF=${VCS_REF} VERSION=${VERSION}
# build the container image
docker build --build-arg BUILD_DATE=${BUILD_DATE} --build-arg VCS_REF=${VCS_REF} \
  --build-arg VERSION=${VERSION} \
  -t cwlviewer:${VERSION} .
# tag this container image in preparation for pushing to Docker Hub and Quay.io
docker tag cwlviewer:${VERSION} docker.io/commonworkflowlanguage/cwlviewer:${VERSION}
docker tag cwlviewer:${VERSION} docker.io/commonworkflowlanguage/cwlviewer:latest
docker tag cwlviewer:${VERSION} quay.io/commonwl/cwlviewer:${VERSION}
docker tag cwlviewer:${VERSION} quay.io/commonwl/cwlviewer:latest
# push the container image to Docker Hub and Quay.io
docker push docker.io/commonworkflowlanguage/cwlviewer:${VERSION}
docker push docker.io/commonworkflowlanguage/cwlviewer:latest
docker push quay.io/commonwl/cwlviewer:${VERSION}
docker push quay.io/commonwl/cwlviewer:latest
# upload the annotated tag to GitHub
git push --tags
git push
```

Then copy the changelog into https://github.com/common-workflow-language/cwlviewer/releases/new
using the tag you just pushed.

Finally, make a new PR to bump the version and restore the `-SNAPSHOT` suffix in `pom.xml`.

# Thanks

Developers and [contributors](https://github.com/common-workflow-language/cwlviewer/graphs/contributors) include:

* **Mark Robinson** http://orcid.org/0000-0002-8184-7507
* Stian Soiland-Reyes http://orcid.org/0000-0001-9842-9718
* Michael Crusoe http://orcid.org/0000-0002-2961-9670
* Carole Goble http://orcid.org/0000-0003-1219-2137
* Charles Overbeck https://github.com/coverbeck
* Finn Bacall http://orcid.org/0000-0002-0048-3300
* Osakpolor Obaseki https://github.com/obasekiosa
* Bruno P. Kinoshita https://orcid.org/0000-0001-8250-4074

Thanks to:

* [eScience Lab](http://www.esciencelab.org.uk/) at [The University of Manchester](http://www.cs.manchester.ac.uk/)
* [BioExcel Center of Excellence for Computational Biomolecular Research](http://bioexcel.eu/)
* * European Commission's [H2020 grant 675728](http://cordis.europa.eu/projects/675728)
* [Common Workflow Language](http://www.commonwl.org/) community and the [CWL Gitter room](https://gitter.im/common-workflow-language/common-workflow-language)


![commercetools-java-sync-logos 002](https://user-images.githubusercontent.com/9512131/31182587-90d47f0a-a924-11e7-9716-66e6bec7f79b.png)
# commercetools sync java
[![CI](https://github.com/commercetools/commercetools-sync-java/workflows/CI/badge.svg)](https://github.com/commercetools/commercetools-sync-java/actions?query=workflow%3ACI)
[![codecov](https://codecov.io/gh/commercetools/commercetools-sync-java/branch/master/graph/badge.svg)](https://codecov.io/gh/commercetools/commercetools-sync-java)
[![Javadoc](https://javadoc.io/badge2/com.commercetools/commercetools-sync-java/javadoc.svg?label=Javadoc)](https://commercetools.github.io/commercetools-sync-java/v/10.0.5/)
[![Known Vulnerabilities](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646/badge.svg)](https://snyk.io/test/github/commercetools/commercetools-sync-java/4b2e26113d591bda158217c5dc1cf80a88665646)

> Note: The current version of this library uses [JVM-SDK-V2](http://commercetools.github.io/commercetools-sdk-java-v2). This doc already contains updated information. Please migrate to this version, using our [Migration Guide](/docs/MIGRATION_GUIDE.md). The support for JVM SDK v1 has been discontinued with [commercetools-sync-java v9.2.3](https://github.com/commercetools/commercetools-sync-java/tree/9.2.3)
> If migration isn't an option for you, you can still use deprecated versions of this library available at [Maven central](https://central.sonatype.com/artifact/com.commercetools/commercetools-sync-java/9.2.3).

More at https://commercetools.github.io/commercetools-sync-java

Java library which allows to import/synchronise (import changes) the data from any arbitrary source to commercetools project.

Supported resources: [Categories](/docs/usage/CATEGORY_SYNC.md), [Products](/docs/usage/PRODUCT_SYNC.md), [InventoryEntries](/docs/usage/INVENTORY_SYNC.md), [ProductTypes](/docs/usage/PRODUCT_TYPE_SYNC.md), [Types](/docs/usage/TYPE_SYNC.md), [CartDiscounts](/docs/usage/CART_DISCOUNT_SYNC.md), [States](/docs/usage/STATE_SYNC.md), [TaxCategories](/docs/usage/TAX_CATEGORY_SYNC.md), [CustomObjects](/docs/usage/CUSTOM_OBJECT_SYNC.md), [Customers](/docs/usage/CUSTOMER_SYNC.md), [ShoppingLists](/docs/usage/SHOPPING_LIST_SYNC.md)

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Usage](#usage)
  - [Quick Start](/docs/usage/QUICK_START.md)
  - [Prerequisites](#prerequisites)
  - [Installation](#installation)
    - [Maven](#maven)
    - [Gradle](#gradle)
    - [SBT](#sbt)
    - [Ivy](#ivy)
- [Release Notes](/docs/RELEASE_NOTES.md)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
## Usage

Create your own event or cronjob based application and use the library to transform any external data (JSON, CSV, XML, REST API, DB, ...) into [JVM-SDK-V2](http://commercetools.github.io/commercetools-sdk-java-v2) resource draft objects (e.g. [CategoryDraft](https://github.com/commercetools/commercetools-sdk-java-v2/blob/main/commercetools/commercetools-sdk-java-api/src/main/java-generated/com/commercetools/api/models/category/CategoryDraft.java)) and import those into the commercetools project.

Notes:

- It is often more efficient if you can setup your external data source to provide you only the changes (deltas) instead of the full data set on every import iteration.
- There is dockerized ready-to-use CLI application [commercetools-project-sync](https://github.com/commercetools/commercetools-project-sync) which based on this library can synchronize entire data catalogue between the 2 commercetools projects.
- During a synchronisation, resources are either created or updated, but **not** deleted.

⚡ See the [Quick Start Guide](/docs/usage/QUICK_START.md) for more information on building a product importer!

![commercetools-java-sync-final 001](https://user-images.githubusercontent.com/3469524/126317637-a946a81c-2948-4751-86bb-02bcecfeca95.png)

### Prerequisites

 - Library requires the min JDK version `>= 11`.
   > The library tested with each major JDK version (i.e: 11, 12, 13...) as well as some specific updates of LTS versions (i.e: 11.0.3 and above).
 - A target commercetools project for syncing your source data to.

### Installation

There are multiple ways to add the commercetools sync dependency to your project, based on your dependency manager.
Here are the most popular ones:

#### Maven

````xml
<dependency>
  <groupId>com.commercetools</groupId>
  <artifactId>commercetools-sync-java</artifactId>
  <version>10.0.5</version>
</dependency>
````

#### Gradle

````groovy
implementation 'com.commercetools:commercetools-sync-java:10.0.5'
````

#### SBT

````
libraryDependencies += "com.commercetools" % "commercetools-sync-java" % "10.0.5"
````

#### Ivy

````xml
<dependency org="com.commercetools" name="commercetools-sync-java" rev="10.0.5"/>
````

**Note**: To avoid `commercetools JVM SDK` libraries version mismatch between projects.
 It is better not to add `commercetools JVM SDK` dependencies explicitly into your project and use them from `commercetools-Sync-Java` dependencies instead.
 Please remove them if you have already added the below dependencies in your project.

For Gradle users, remove:
```groovy
implementation "com.commercetools.sdk:commercetools-http-client:${version}"
implementation "com.commercetools.sdk:commercetools-sdk-java-api:${version}"
```

For Maven users, remove:

````xml
<dependency>
  <groupId>com.commercetools.sdk</groupId>
  <artifactId>commercetools-http-client</artifactId>
  <version>version</version>
</dependency>
<dependency>
  <groupId>com.commercetools.sdk</groupId>
  <artifactId>commercetools-sdk-java-v2</artifactId>
  <version>version</version>
</dependency>
````

If you want to use a different `commercetools JVM SDK` version than the version used in this project.
, below you will find examples on how to exclude `commercetools JVM SDK` from commercetools-sync-java library. Beware that library might not work with the older `commercetools JVM SDK` versions.

For Gradle:
````groovy
implementation('com.commercetools:commercetools-sync-java') {
    exclude group: 'com.commercetools.sdk', module: 'commercetools-http-client'
    exclude group: 'com.commercetools.sdk', module: 'commercetools-sdk-java-api'
}
````

For Maven:
````xml
<dependency>
  <groupId>com.commercetools</groupId>
  <artifactId>commercetools-sync-java</artifactId>
  <version>version</version>
  <exclusions>
    <exclusion>
        <groupId>com.commercetools.sdk</groupId>
        <artifactId>commercetools-http-client</artifactId>
    </exclusion>
    <exclusion>
        <groupId>com.commercetools.sdk</groupId>
        <artifactId>commercetools-sdk-java-v2</artifactId>
    </exclusion>
  </exclusions>
</dependency>
````


![Undergarden Logo](https://i.imgur.com/jpKtqf5.png)
# Deep underground, a forgotten land awaits.
https://www.curseforge.com/minecraft/mc-mods/the-undergarden


![](https://img.shields.io/travis/javaito/HolandaCatalinaFw.svg)
![](https://img.shields.io/github/license/javaito/HolandaCatalinaFw.svg)
![](https://img.shields.io/github/release/javaito/HolandaCatalinaFw.svg)
![Maven Central](https://img.shields.io/maven-central/v/com.github.javaito/hcjf.svg)
![](https://img.shields.io/github/forks/javaito/HolandaCatalinaFw.svg?style=social&label=Fork)
![](https://img.shields.io/github/stars/javaito/HolandaCatalinaFw.svg?style=social&label=Star)
![](https://img.shields.io/github/watchers/javaito/HolandaCatalinaFw.svg?style=social&label=Watch)

# Holanda Catalina

Holanda Catalina is a powerful framework to create your own solution based on a micro-services architecture, without realizing that you are doing it. ;)

## Table of Contents
 - [Introduction](introduction.md)
 - [Getting started](getting-started.md)
 - [My first service](my-first-service.md)
 - [Very good things!](very-good-things.md)

## Acknowledgements

 - [Microservice Architecture](http://microservices.io/index.html)


# Mantis Documentation

[![Build Status](https://img.shields.io/travis/com/Netflix/mantis.svg)](https://travis-ci.com/Netflix/mantis)
[![OSS Lifecycle](https://img.shields.io/osslifecycle/Netflix/mantis.svg)](https://github.com/Netflix/mantis)
[![License](https://img.shields.io/github/license/Netflix/mantis.svg)](https://www.apache.org/licenses/LICENSE-2.0)

[Official Website](https://netflix.github.io/mantis/)

---

## Development

### Setting up Intellij
Run `./gradlew idea` to (re-) generate IntelliJ project and module files from the templates in `.baseline`. The generated project is pre-configured with Baseline code style settings and support for the CheckStyle-IDEA plugin.

### Install Docker
Install and run Docker to support local containers.

### Building

```sh
$ ./gradlew clean build
```

### Testing

```sh
$ ./gradlew clean test
```

### Formatting the code
Run `./gradlew format` task which autoformats all Java files using [Spotless](https://github.com/diffplug/spotless).

### Building deployment into local Maven cache

```sh
$ ./gradlew clean publishNebulaPublicationToMavenLocal
```

### Releasing

We release by tagging which kicks off a CI build. The CI build will run tests, integration tests,
static analysis, checkstyle, build, and then publish to the public Bintray repo to be synced into Maven Central.

Tag format:

```
vMajor.Minor.Patch
```

You can tag via git or through Github's Release UI.

## Contributing

Mantis is interested in building the community. We welcome any forms of contributions through discussions on any
of our [mailing lists](https://netflix.github.io/mantis/community/#mailing-lists) or through patches.

For more information on contribution, check out the contributions file [here](https://github.com/Netflix/mantis/blob/master/CONTRIBUTING.md).

### Module Structure
This excludes all connectors and examples as they are mostly leaf nodes in the dependency graph.

Module                      | Purpose                                                                                | Example Classes                                                                                | Package Prefixes
----------------------------|-------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------
mantis-common-serde         | Support for serializing and deserializing POJOs using Json                                                              |
JsonSerializer
                                                                                |
io.mantisrx.common.*

mantis-discovery-proto      | todo: need to fill this in                                                                                |
JobDiscoveryProto

AppJobClustersMap

StageWorkers
                                   |
com.netflix.mantis.discovery.proto.*

mantis-common               | Mantis common code shared across modules                                                                                |
MantisJobDefinition

MachineDefinition

MantisJobState                                    |
io.mantisrx.common.\*

 io.mantisrx.runtime.\*
io.mantisrx.server.code.\*

io.reactivx.mantis.operators.\*

com.mantisrx.common.utils.\*
mantis-remote-observable    | connection to other job,stage related code                                                                              |
ConnectToObservable

ConnectToGroupedObservable

RemoteObservable                         |
io.reactivex.mantis.remote.observable.\*

io.reactivex.netty.codec.\*

mantis-control-plane-core   | common code between mantis-control-plane-server, mantis-control-plane-client, mantis-server-worker, mantis-server-agent |
TaskExecutorID

ClusterID

ExecuteStageRequest

JobAssignmentResult

Status |
io.mantisrx.server.core.\*

io.mantisrx.server.master.resourcecluster.\*

io.mantisrx.server.worker.\*
mantis-control-plane-client | API to talk to the mantis control plane server                                                                          |
MasterClientWrapper

MantisMasterGateway

MantisMasterClientApi                           |
io.mantisrx.server.master.client.\*
 io.mantisrx.server.master.resourcecluster.\*
mantis-network              | todo: need to fill this in                                                                                |                                                                                |
mantis-publish-core         | todo: need to fill this in                                                                                |                                                                                |
mantis-server-worker-client | API to interact with workers                                                                                |
MetricsClient

 MetricsClientImpl

WorkerConnection

WorkerMetricsClient           |
io.mantisrx.server.worker.client.\*
mantis-runtime              | Runtime that the jobs need to depend upon. Job DSL should go in here along with how to talk to other jobs               |
KeyToKey

GroupToGroup

Source

Sink                                               |
io.mantisrx.runtime.\*
mantis-publish-netty        | todo: need to fill this in                                                                                |                                                                                |
mantis-client               | client to interact with mantis control plane                                                                            |
MantisClient::submitJob

 MantisClient::killJob                                                  |
io.mantisrx.client.*

mantis-publish-netty-guice  | todo: need to fill this in                                                                                |                                                                                |
mantis-control-plane-server | Actual server that runs the mantis master code                                                                          |
MasterMain

SchedulingService

ServiceLifecycle                                           |
io.mantisrx.master.\*
io.mantisrx.server.master.\*
mantis-server-agent         | Contains mantis-runtime agnostic code to start the task executor that runs on the agent                                 |
TaskExecutor

TaskExecutorStarter

BlobStore

BlobStoreAwareClassLoaderHandle      |
io.mantisrx.server.agent.\*
mantis-server-worker        | One implementation of Mantis Worker that depends on the master runtime                                                  |
MantisWorker

Task

ExecuteStageRequestService
JobAutoScaler                           |
io.mantisrx.server.worker.config.\*
io.mantisrx.server.worker.jobmaster.\*

### Dependency Graph

```mermaid
graph TD;
    A[mantis-common-serde]-->B[mantis-common];
    B-->C[mantis-control-plane-core];
    B-->D[mantis-runtime];
    C-->E[mantis-control-plane-client];
    E-->F[mantis-server-worker-client];
    F-->G[mantis-client];
    E-->G;
    C-->H[mantis-control-plane-server];
    C-->I[mantis-server-worker];
    F-->I;
    D-->I;
    F-->J[mantis-server-agent];
    E-->J;
```


# MCA Selector
[![total downloads](https://img.shields.io/github/downloads/Querz/mcaselector/total.svg)](https://github.com/Querz/mcaselector/releases) [![paypal](https://img.shields.io/badge/donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3PV2GDWZL8HCA) [![chat](https://img.shields.io/discord/722924391805223113?logo=discord)](https://discord.gg/h942U8U)

#### An external tool to export or delete selected chunks and regions from a world save of Minecraft Java Edition.
---

---

**MCA Selector modifies and deletes chunks in your Minecraft world. Please make backups of your world before using.**

[**Download Version 2.4.1 (Windows Installer)**](https://github.com/Querz/mcaselector/releases/download/2.4.1/MCA_Selector_Setup.exe)

[**Download Version 2.4.1 (Universal)**](https://github.com/Querz/mcaselector/releases/download/2.4.1/mcaselector-2.4.1.jar)

---

## Wiki

The wiki can be found [here](https://github.com/Querz/mcaselector/wiki).

## Video Tutorials
For people who prefer watching a video to understand how the MCA Selector works, there some very good tutorials on
YouTube explaining the basics:

* [How To Reset Chunks In Your Minecraft World!](https://www.youtube.com/watch?v=1xx8fwynlRs) by [Farzy](https://www.youtube.com/channel/UCVtz3s3FUxVxBgPl2OWtIJQ)
* [How To Clear Unwanted Chunks In Minecraft 1.16 | MCASelector Tutorial](https://www.youtube.com/watch?v=ADDTXGRJo20) by [Muriako](https://www.youtube.com/channel/UCpt-MjKkc5X4W7bUFV3Dwrw)
* [Preparing Your World for the Nether Update! - The Minecraft Survival Guide](https://www.youtube.com/watch?v=1fiyVvoD9jQ) starting at [2:52](https://www.youtube.com/watch?v=1fiyVvoD9jQ&t=2m52s) by [Pixlriffs](https://www.youtube.com/channel/UCgGjBqZZtAjxfpGSba7d6ww)
* [How To Reset The End Dimension! - The Minecraft Survival Guide](https://www.youtube.com/watch?v=p-2gFkJl_Lo) starting at [8:45](https://www.youtube.com/watch?v=p-2gFkJl_Lo&t=8m45s) by [Pixlriffs](https://www.youtube.com/channel/UCgGjBqZZtAjxfpGSba7d6ww)

---
## Supported Versions
MCA Selector currently supports the following Minecraft versions:

| Minecraft Version | DataVersion |
|-------------------|-------------|
| 1.2.1 - 1.12.2    | None - 1343 |
| 1.13 - 1.13.2     | 1444 - 1631 |
| 1.14 - 1.14.4     | 1901 - 1976 |
| 1.15 - 1.15.2     | 2200 - 2230 |
| 1.16 - 1.16.5     | 2566 - 2586 |
| 1.17 - 1.17.1     | 2724 - 2730 |
| 1.18 - 1.18.2     | 2825 - 2975 |
| 1.19 - 1.19.4     | 3105 - 3337 |
| 1.20 - 1.20.6     | 3463 - 3839 |
| 1.21 - ?          | 3953 - ?    |

There is no guarantee for worlds generated in a Snapshot version to work, even if it is specified in the table above.
This only represents the current development status towards the next Minecraft release. Old Snapshots of past
Minecraft releases are not officially supported.


<img alt="logo" src="https://www.objectionary.com/cactus.svg" height="100px" />

[![EO principles respected here](https://www.elegantobjects.org/badge.svg)](https://www.elegantobjects.org)
[![DevOps By Rultor.com](http://www.rultor.com/b/yegor256/cactoos)](http://www.rultor.com/p/yegor256/cactoos)
[![We recommend IntelliJ IDEA](https://www.elegantobjects.org/intellij-idea.svg)](https://www.jetbrains.com/idea/)

[![mvn](https://github.com/yegor256/cactoos/actions/workflows/mvn.yml/badge.svg)](https://github.com/yegor256/cactoos/actions/workflows/mvn.yml)
[![Javadoc](http://www.javadoc.io/badge/org.cactoos/cactoos.svg)](http://www.javadoc.io/doc/org.cactoos/cactoos)
[![PDD status](http://www.0pdd.com/svg?name=yegor256/cactoos)](http://www.0pdd.com/p?name=yegor256/cactoos)
[![Maven Central](https://img.shields.io/maven-central/v/org.cactoos/cactoos.svg)](https://maven-badges.herokuapp.com/maven-central/org.cactoos/cactoos)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/yegor256/cactoos/blob/master/LICENSE.txt)
[![Test Coverage](https://img.shields.io/codecov/c/github/yegor256/cactoos.svg)](https://codecov.io/github/yegor256/cactoos?branch=master)
[![SonarQube](https://img.shields.io/badge/sonar-ok-green.svg)](https://sonarcloud.io/dashboard?id=org.cactoos%3Acactoos)
[![Hits-of-Code](https://hitsofcode.com/github/yegor256/cactoos)](https://hitsofcode.com/view/github/yegor256/cactoos)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/dc71110fe0ac43b8a535eec8b8fec389)](https://www.codacy.com/gh/yegor256/cactoos/dashboard)

Project architect: [@victornoel](https://github.com/victornoel)

**ATTENTION**: We're still in a very early alpha version, the API
may and _will_ change frequently. Please, use it at your own risk,
until we release version 1.0. You can view our progress towards
this release [here](https://github.com/yegor256/cactoos/milestone/1).

Cactoos is a collection of object-oriented Java primitives.

**Motivation**.
We are not happy with
[JDK](https://en.wikipedia.org/wiki/Java_Development_Kit),
[Guava](https://github.com/google/guava), and
[Apache Commons](https://commons.apache.org/) because
they are procedural and not object-oriented. They do their job,
but mostly through static methods. Cactoos is suggesting
to do almost exactly the same, but through objects.

**Principles**.
These are the [design principles](https://www.elegantobjects.org#principles) behind Cactoos.

**How to use**.
The library has no dependencies. All you need is this
(get the latest version [here](https://github.com/yegor256/cactoos/releases)):

Maven:

```xml
<dependency>
  <groupId>org.cactoos</groupId>
  <artifactId>cactoos</artifactId>
  <version>0.55.0</version>
</dependency>
```

Gradle:

```groovy
dependencies {
  compile 'org.cactoos:cactoos::0.55.0
}
```

Java version required: 1.8+.

StackOverflow tag is [cactoos](https://stackoverflow.com/questions/tagged/cactoos).

## Input/Output

More about it here:
[Object-Oriented Declarative Input/Output in Cactoos](http://www.yegor256.com/2017/06/22/object-oriented-input-output-in-cactoos.html).

To read a text file in UTF-8:

```java
String text = new TextOf(
  new File("/code/a.txt")
).asString();
```

To write a text into a file:

```java
new LengthOf(
  new TeeInput(
    "Hello, world!",
    new File("/code/a.txt")
  )
).value();
```

To read a binary file from classpath:

```java
byte[] data = new BytesOf(
  new ResourceOf("foo/img.jpg")
).asBytes();
```

## Text/Strings

To format a text:

```java
String text = new FormattedText(
  "How are you, %s?",
  name
).asString();
```

To manipulate with a text:

```java
// To lower case
new Lowered(
	new TextOf("Hello")
);
// To upper case
new Upper(
	new TextOf("Hello")
);
```

## Iterables/Collections/Lists/Sets

More about it here: [Lazy Loading and Caching via Sticky Cactoos Primitives](http://www.yegor256.com/2017/10/17/lazy-loading-caching-sticky-cactoos.html).

To filter a collection:

```java
Collection<String> filtered = new ListOf<>(
  new Filtered<>(
    s -> s.length() > 4,
    new IterableOf<>("hello", "world", "dude")
  )
);
```

To flatten one iterable:
```java
new Joined<>(
  new Mapped<IterableOf>(
    iter -> new IterableOf<>(
      new ListOf<>(iter).toArray(new Integer[]{})
    ),
    new IterableOf<>(1, 2, 3, 4, 5, 6)
  )
);    // Iterable<Integer>
```

To flatten and join several iterables:
```java
new Joined<>(
  new Mapped<IterableOf>(
    iter -> new IterableOf<>(
      new Joined<>(iter)
    ),
    new Joined<>(
      new IterableOf<>(new IterableOf<>(1, 2, 3)),
      new IterableOf<>(new IterableOf<>(4, 5, 6))
    )
  )
);    // Iterable<Integer>
```

To iterate a collection:

```java
new And(
  new Mapped<>(
    new FuncOf<>(
      input -> {
        System.out.printf("Item: %s\n", input);
      },
      new True()
    ),
    new IterableOf<>("how", "are", "you", "?")
  )
).value();
```

Or even more compact:

```java
new ForEach<String>(
    input -> System.out.printf(
        "Item: %s\n", input
    )
).exec(new IterableOf("how", "are", "you", "?"));
```

To sort a list of words in the file:

```java
List<Text> sorted = new ListOf<>(
  new Sorted<>(
    new Mapped<>(
      text -> new ComparableText(text),
      new Split(
        new TextOf(
          new File("/tmp/names.txt")
        ),
        new TextOf("\\s+")
      )
    )
  )
);
```

To count elements in an iterable:

```java
int total = new LengthOf(
  new IterableOf<>("how", "are", "you")
).value().intValue();
```

To create a set of elements by providing variable arguments:

```java
final Set<String> unique = new SetOf<String>(
    "one",
    "two",
    "one",
    "three"
);
```

To create a set of elements from existing iterable:
```java
final Set<String> words = new SetOf<>(
    new IterableOf<>("abc", "bcd", "abc", "ccc")
);
```

To create a sorted iterable with unique elements from existing iterable:
```java
final Iterable<String> sorted = new Sorted<>(
    new SetOf<>(
        new IterableOf<>("abc", "bcd", "abc", "ccc")
    )
);
```

To create a sorted set from existing vararg elements using comparator:
```java
final Set<String> sorted = new org.cactoos.set.Sorted<>(
    (first, second) -> first.compareTo(second),
    "abc", "bcd", "abc", "ccc", "acd"
);
```

To create a sorted set from existing iterable using comparator:
```java
final Set<String> sorted = new org.cactoos.set.Sorted<>(
    (first, second) -> first.compareTo(second),
    new IterableOf<>("abc", "bcd", "abc", "ccc", "acd")
);
```

## Funcs and Procs

This is a traditional `foreach` loop:

```java
for (String name : names) {
  System.out.printf("Hello, %s!\n", name);
}
```

This is its object-oriented alternative (no streams!):

```java
new And(
  n -> {
    System.out.printf("Hello, %s!\n", n);
    return new True().value();
  },
  names
).value();
```

This is an endless `while/do` loop:

```java
while (!ready) {
  System.out.println("Still waiting...");
}
```

Here is its object-oriented alternative:

```java
new And(
  ready -> {
    System.out.println("Still waiting...");
    return !ready;
  },
  new Endless<>(booleanParameter)
).value();
```

## Dates and Times
From our `org.cactoos.time` package.

Our classes are divided in two groups: those that parse strings into date/time objects, and those that format those objects into strings.

For example, this is the traditional way of parsing a string into an [OffsetDateTime](https://docs.oracle.com/javase/8/docs/api/java/time/OffsetDateTime.html):

```java
final OffsetDateTime date = OffsetDateTime.parse("2007-12-03T10:15:30+01:00");
```

Here is its object-oriented alternative (no static method calls!) using `OffsetDateTimeOf`, which is a `Scalar`:

```java
final OffsetDateTime date = new OffsetDateTimeOf("2007-12-03T10:15:30+01:00").value();
```

To format an `OffsetDateTime` into a `Text`:

```java
final OffsetDateTime date = ...;
final String text = new TextOfDateTime(date).asString();
```

## Our objects vs. their static methods

Cactoos | Guava | Apache Commons | JDK 8
------ | ------ | ------ | ------
`And` | `Iterables.all()` | - | -
`Filtered` | `Iterables.filter()` | ? | -
`FormattedText` | - | - | `String.format()`
`IsBlank` | - | `StringUtils.isBlank()`| -
`Joined` | - | - | `String.join()`
`LengthOf` | - | - | `String#length()`
`Lowered` | - | - | `String#toLowerCase()`
`Normalized` | - | `StringUtils.normalize()` | -
`Or` | `Iterables.any()` | - | -
`Repeated` | - | `StringUtils.repeat()` | -
`Replaced` | - | - | `String#replace()`
`Reversed` | - | - | `StringBuilder#reverse()`
`Rotated` | - | `StringUtils.rotate()`| -
`Split` | - | - | `String#split()`
`StickyList` | `Lists.newArrayList()` | ? | `Arrays.asList()`
`Sub` | - | - | `String#substring()`
`SwappedCase` | - | `StringUtils.swapCase()` | -
`TextOf` | ? | `IOUtils.toString()` | -
`TrimmedLeft` | - | `StringUtils.stripStart()` | -
`TrimmedRight` | - | `StringUtils.stripEnd()` | -
`Trimmed` | - | `StringUtils.stripAll()` | `String#trim()`
`Upper` | - | - | `String#toUpperCase()`

## Questions

Ask your questions related to cactoos library on [Stackoverflow](https://stackoverflow.com/questions/ask) with [cactoos](https://stackoverflow.com/tags/cactoos/info) tag.

## How to contribute?

Just fork the repo and send us a pull request.

Make sure your branch builds without any warnings/issues:
```
mvn clean verify -Pqulice
```

To run a build similar to the CI with Docker only, use:
```
docker run \
	--tty \
	--interactive \
	--workdir=/main \
	--volume=${PWD}:/main \
	--volume=cactoos-mvn-cache:/root/.m2 \
	--rm \
	maven:3-jdk-8 \
	bash -c "mvn clean install site -Pqulice -Psite --errors; chown -R $(id -u):$(id -g) target/"
```

To remove the cache used by Docker-based build:
```
docker volume rm cactoos-mvn-cache
```

Note: [Checkstyle](https://en.wikipedia.org/wiki/Checkstyle) is used as a static code analyze tool with
[checks list](http://checkstyle.sourceforge.net/checks.html) in GitHub precommits.

## Contributors

  - [@yegor256](https://github.com/yegor256) as Yegor Bugayenko ([Blog](http://www.yegor256.com))
  - [@g4s8](https://github.com/g4s8) as Kirill Che. (g4s8.public@gmail.com)
  - [@fabriciofx](https://github.com/fabriciofx) as Fabrício Cabral
  - [@englishman](https://github.com/englishman) as Andriy Kryvtsun
  - [@VsSekorin](https://github.com/VsSekorin) as Vseslav Sekorin
  - [@DronMDF](https://github.com/DronMDF) as Andrey Valyaev
  - [@dusan-rychnovsky](https://github.com/dusan-rychnovsky) as Dušan Rychnovský ([Blog](http://blog.dusanrychnovsky.cz/))
  - [@timmeey](https://github.com/timmeey) as Tim Hinkes ([Blog](https://blog.timmeey.de))
  - [@alex-semenyuk](https://github.com/alex-semenyuk) as Alexey Semenyuk
  - [@smallcreep](https://github.com/smallcreep) as Ilia Rogozhin
  - [@memoyil](https://github.com/memoyil) as Mehmet Yildirim
  - [@llorllale](https://github.com/llorllale) as George Aristy
  - [@driver733](https://github.com/driver733) as Mikhail Yakushin
  - [@izrik](https://github.com/izrik) as Richard Sartor
  - [@Vatavuk](https://github.com/Vatavuk) as Vedran Grgo Vatavuk
  - [@dgroup](https://github.com/dgroup) as Yurii Dubinka
  - [@iakunin](https://github.com/iakunin) as Maksim Iakunin
  - [@fanifieiev](https://github.com/fanifieiev) as Fevzi Anifieiev
  - [@victornoel](https://github.com/victornoel) as Victor Noël
  - [@paulodamaso](https://github.com/paulodamaso) as Paulo Lobo