WAFFLE - Windows Authentication Framework
=========================================

[![Java CI](https://github.com/Waffle/waffle/workflows/Java%20CI/badge.svg)](https://github.com/Waffle/waffle/actions?query=workflow%3A%22Java+CI%22)
[![DotNET CI](https://github.com/Waffle/waffle/workflows/DotNET/badge.svg)](https://github.com/Waffle/waffle/workflows/DotNET)
[![Coverity Scan Status](https://scan.coverity.com/projects/22153/badge.svg)](https://scan.coverity.com/projects/22153)
[![Coverage Status Sonar](https://sonarcloud.io/api/project_badges/measure?project=Waffle_waffle&metric=coverage)](https://sonarcloud.io/component_measures?id=Waffle_waffle&metric=coverage)
[![Coverage Status Coveralls](https://coveralls.io/repos/Waffle/waffle/badge.svg?branch=master&service=github)](https://coveralls.io/github/Waffle/waffle?branch=master)
[![Maven central](https://maven-badges.herokuapp.com/maven-central/com.github.waffle/waffle-jna/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.github.waffle/waffle-jna)
[![releases](https://img.shields.io/github/v/release/Waffle/waffle)](https://github.com/Waffle/waffle/releases/tag/waffle-parent-3.4.0)
[![MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Project Stats](https://www.openhub.net/p/waffle/widgets/project_thin_badge.gif)](https://www.openhub.net/p/waffle)
[![Github All Releases](https://img.shields.io/github/downloads/Waffle/waffle/total.svg)](https://github.com/Waffle/waffle/)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=Waffle_waffle&metric=alert_status)](https://sonarcloud.io/dashboard?id=Waffle_waffle)

![waffle](https://github.com/Waffle/waffle/raw/master/waffle.jpg)

WAFFLE is a native Windows Authentication Framework consisting of two C# and Java libraries that perform functions related to Windows authentication, supporting Negotiate, NTLM and Kerberos. Waffle also includes libraries that enable drop-in Windows Single Sign On for popular Java web servers, when running on Windows. While Waffle makes it ridiculously easy to do Windows Authentication in Java, on Windows, Waffle does not work on *nix(UNIX-like).

Unlike many other implementations Waffle on Windows does not require any server-side Kerberos keytab setup, it's a drop-in solution. You can see it in action in [this slightly blurry video](https://www.youtube.com/watch?v=LmTwbOh0hBU) produced for [TeamShatter.com](http://www.teamshatter.com/topics/general/team-shatter-exclusive/securing-java-applications-with-smart-cards-and-single-sign-on/).

Sites
-----

* [Site Page](https://waffle.github.io/waffle/)
* [sonarqube-java](https://sonarcloud.io/dashboard?id=Waffle_waffle)
* [sonarqube-.net](https://sonarqube.com/dashboard/index?id=waffle)

Essentials
----------

* [Download Version 3.4.0](https://search.maven.org/remotecontent?filepath=com/github/waffle/waffle-distro/3.4.0/waffle-distro-3.4.0-distro.zip)
* [Waffle in Maven Central](https://search.maven.org/search?q=waffle)
* [Waffle Snapshots](https://oss.sonatype.org/content/repositories/snapshots/com/github/waffle/)
* [Get Waffle To Work in Tomcat, Jetty, WebSphere, etc.](Docs/ServletSingleSignOnSecurityFilter.md)
* [Need Help? Google Group](https://groups.google.com/group/waffle-users)
* [Troubleshooting](Docs/Troubleshooting.md)
* [Frequently Asked Questions](Docs/FAQ.md)
* [Older Versions on CodePlex](https://waffle.codeplex.com/).
* [PlatformSDK Security Group](https://groups.google.com/group/microsoft.public.platformsdk.security)

Documentation
-------------

There're several semi-independent parts to Waffle. Choose the appropriate HowTo.

* Simple native interfaces in C# and Java to do all things Windows authentication. Useful if you're building a custom client that requires Windows authentication. See [Getting Started with WAFFLE API](https://github.com/Waffle/waffle/blob/master/Docs/GettingStartedWithWaffleAPI.md)
* A generic Servlet Negotiate (NTLM and Kerberos) Security Filter that can be used with many web servers, including Tomcat, Jetty and WebSphere. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/ServletSingleSignOnSecurityFilter.md).
* A Tomcat Negotiate (NTLM and Kerberos) Authenticator Valve, built for the Tomcat Web Container. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/tomcat/TomcatSingleSignOnValve.md).
* A Tomcat Single Sign-On + Form Authentication Mixed Valve, built for the Tomcat Web Container and allowing users to choose whether to do form authentication (a username and password sent to the server from a form) or Windows SSO (NTLM or Kerberos). See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/tomcat/TomcatMixedSingleSignOnAndFormAuthenticatorValve.md).
* A Spring-Security Negotiate (NTLM and Kerberos) Filter. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/spring/SpringSecuritySingleSignOnFilter.md).
* A Spring-Security Windows Authentication Manager. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/spring/SpringSecurityAuthenticationProvider.md).
* A JAAS Login Module, useful when extending a custom Java client that already implements JAAS to support Windows SSO. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/tomcat/TomcatWindowsLoginJAASAuthenticator.md).
* A WildFly Security Domain implementation, offering support for local Windows and Active Directory users authentication when deploying web apps on WildFly servers. See [HowTo](https://github.com/Waffle/waffle/blob/master/Docs/wildfly/WildFlySecurityDomain.md).

Waffle was created and is sponsored by [Application Security Inc.](https://www.trustwave.com/Company/AppSecInc-is-now-Trustwave/). For a long story, read the [Project History](https://github.com/Waffle/waffle/blob/master/HISTORY.md). Also, feel free to use [this PowerPoint presentation](http://www.slideshare.net/dblockdotorg/waffle-at-nycjavasig) from NYJavaSIG to talk about Waffle.

Features
--------

* Account lookup locally and in Active Directory via Win32 API with zero configuration.
* Enumerating Active Directory domains and domain information.
* Returns computer domain / workgroup join information.
* Supports logon for local and domain users returning consistent fully qualified names, identity (SIDs), local and domain groups, including nested.
* Supports all functions required for implementing server-side single-signon with Negotiate and NTLM and various implementations for Java web servers.
* Supports Windows Identity impersonation.
* Includes a Windows Installer Merge Module for distribution of C# binaries.

How do I resolve JNA `NoClassDefFound` errors?
----------------------------------------------
WAFFLE uses the latest version of JNA, which may conflict with other dependencies your project (or its parent) includes. If you experience issues with `NoClassDefFound` errors for JNA artifacts, consider one or more of the following steps to resolve the conflict:
* Listing WAFFLE earlier (or first) in your dependency list
* Specifying the most recent version of JNA as a dependency
* If you are using a parent (e.g., Spring Boot) that includes JNA as a dependency, override the `jna.version` property

Demos
-----
WAFFLE packages serveral demos.  When building from the source, you can use ```mvn cargo:run``` to bring any of the demos up.  See further README documentation in each demo (and what doesn't quite work).  Additionally demos are documented in various states in documentation.  This area does need some user help so feel free to contribute.

Related and Similar Products
----------------------------

* [Cross-Platform SPNEGO](http://spnego.sourceforge.net/)
* [Tomcat SPNEGO](https://tomcatspnego.codeplex.com/)
* [Quest Vintela Single-Sign-On](http://www.quest.com/single-sign-on-for-java/) (Commercial)
* [IOPlex Jespa](https://www.ioplex.com/) (Commercial)
* [Josso](http://www.josso.org/) (Commercial)

Contributing
------------

* Fork the project.
* [Set Up a Development Environment](Docs/SettingUpDevelopmentEnvironment.md).
* Make your code changes. Don't forget tests.
* Update [CHANGELOG](CHANGELOG.md).
* Make pull requests. Bonus points for topic branches.

License and Copyright
---------------------

Copyright (c) [Application Security Inc.](https://www.trustwave.com/Company/AppSecInc-is-now-Trustwave/), 2010-2020 and Contributors.

This project is licensed under the [MIT License](https://github.com/Waffle/waffle/blob/master/LICENSE).

Project maintained by [Daniel Doubrovkine](https://github.com/dblock) & [Jeremy Landis](https://github.com/hazendaz).


# DITA Open Toolkit [![DITA-OT Discussions][discussions]](https://github.com/orgs/dita-ot/discussions)

_DITA Open Toolkit_, or _DITA-OT_ for short, is an open-source publishing engine for content authored in the _Darwin Information Typing Architecture_.

Visit the project website at [dita-ot.org][site] for documentation, information about releases, and [download packages][dist].

For information on additional DITA and DITA-OT resources, see [SUPPORT]. To report a bug or suggest a feature, [create an issue][issue]. For more information on how you can help contribute to the project, see [CONTRIBUTING].

- [Prerequisites: Java 17](#prerequisites-java-17)
- [Installing](#installing)
- [Building output](#building-output)
- [Development](#development)
    - [Running tests](#running-tests)
    - [Formatting code](#formatting-code)
    - [Distribution builds](#distribution-builds)
- [License](#license)

## Prerequisites: Java 17

To build and run DITA-OT, you’ll need Java Development Kit (JDK), version 17 or newer.

You can download the OpenJDK from [AdoptOpenJDK][adoptopenjdk].

## Installing

1.  Download the distribution package from [dita-ot.org/download][dist].
2.  Extract the contents of the package to the directory where you want to install DITA-OT.

<details>
<summary>Installing via Homebrew</summary>

On macOS and Linux, you can also install DITA-OT using the [Homebrew] package manager:

```shell
brew install dita-ot
```

Homebrew will automatically download the latest version of the toolkit, install it in a subfolder of the local package Cellar and symlink the `dita` command to the `bin` subfolder of the Homebrew installation directory.

> **Note**
>
> Homebrew’s default installation location depends on the operating system architecture:
>
> - `/usr/local` on macOS Intel
> - `/opt/homebrew` on macOS ARM
> - `/home/linuxbrew/.linuxbrew` on Linux

</details>

## Building output

You can generate output using the `dita` command-line tool included with DITA Open Toolkit.

1.  On the command line, change to the `bin` folder of the DITA-OT installation directory:
    ```shell
    cd path/to/dita-ot-dir/bin
    ```
2.  Run the `dita` command to generate output:

    ```shell
    dita --input=input-file --format=format [options]
    ```

    where:

    - _`input-file`_ is the DITA map or DITA file that you want to process
    - _`format`_ is the output format (or “transformation type”)

See the [documentation][docs] for arguments and [options].

## Development

Building the toolkit from source code and compiling the distribution package

1.  Clone the DITA-OT Git repository, including submodules:
    ```shell
    git clone --recurse-submodules git://github.com/dita-ot/dita-ot.git
    ```
2.  Change to the DITA-OT directory:
    ```shell
    cd dita-ot
    ```
3.  In the root directory, run Gradle to compile the Java code and install plugins:
    ```shell
    ./gradlew
    ```

### Running tests

```shell
./gradlew check
```

All tests are run by GitHub Actions [test workflow] on each push and
for every pull request.

### Formatting code

Requirements:

- Node.js

Prettier is used retain consistent Java formatting.

1.  Run Prettier:
    ```shell
    npm run fmt
    ```

### Distribution builds

1.  In the root directory, set up the build environment:
    ```shell
    ./gradlew
    ```
2.  Build the distribution packages:

    ```shell
    ./gradlew dist
    ```

    Distribution packages are built in the `build/distributions` directory.

    If Gradle throws an error like `java.lang.OutOfMemoryError: Java heap space`, you probably need to increase the maximum Java heap size. One way to do this is to set the `GRADLE_OPTS` environment variable to a value like `-Xmx1024m`.

    For more information on the `-Xmx` option, see the [Java SE Documentation][javadoc].

## License

DITA Open Toolkit is licensed for use under the [Apache License 2.0][apache].

[discussions]: https://img.shields.io/github/discussions/dita-ot/dita-ot?label=DITA-OT%20Discussions
[site]: https://www.dita-ot.org/
[dist]: https://www.dita-ot.org/download
[support]: https://github.com/dita-ot/.github/blob/master/SUPPORT.md
[adoptopenjdk]: https://adoptopenjdk.net/
[homebrew]: https://brew.sh
[docs]: https://www.dita-ot.org/dev/
[options]: https://www.dita-ot.org/dev/topics/build-using-dita-command.html
[javadoc]: http://docs.oracle.com/javase/8/docs/technotes/tools/windows/java.html#BABHDABI
[apache]: http://www.apache.org/licenses/LICENSE-2.0
[issue]: https://github.com/dita-ot/dita-ot/issues/new/choose
[contributing]: https://github.com/dita-ot/.github/blob/master/CONTRIBUTING.md
[test workflow]: https://github.com/dita-ot/dita-ot/actions/workflows/test.yml


# [Traccar](https://www.traccar.org)

## Overview

Traccar is an open source GPS tracking system. This repository contains Java-based back-end service. It supports more than 200 GPS protocols and more than 2000 models of GPS tracking devices. Traccar can be used with any major SQL database system. It also provides easy to use [REST API](https://www.traccar.org/traccar-api/).

Other parts of Traccar solution include:

- [Traccar web app](https://github.com/traccar/traccar-web)
- [Traccar Manager Android app](https://github.com/traccar/traccar-manager-android)
- [Traccar Manager iOS app](https://github.com/traccar/traccar-manager-ios)

There is also a set of mobile apps that you can use for tracking mobile devices:

- [Traccar Client Android app](https://github.com/traccar/traccar-client-android)
- [Traccar Client iOS app](https://github.com/traccar/traccar-client-ios)

## Features

Some of the available features include:

- Real-time GPS tracking
- Driver behaviour monitoring
- Detailed and summary reports
- Geofencing functionality
- Alarms and notifications
- Account and device management
- Email and SMS support

## Build

Please read [build from source documentation](https://www.traccar.org/build/) on the official website.

## Team

- Anton Tananaev ([anton@traccar.org](mailto:anton@traccar.org))
- Andrey Kunitsyn ([andrey@traccar.org](mailto:andrey@traccar.org))

## License

    Apache License, Version 2.0

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


AnySoftKeyboard [![Latest release](https://img.shields.io/github/release/AnySoftKeyboard/AnySoftKeyboard.svg)](https://github.com/AnySoftKeyboard/AnySoftKeyboard/releases)
====================
Discussions can be found on [💬 GitHub](https://github.com/AnySoftKeyboard/AnySoftKeyboard/discussions)

Follow us on Mastodon

Support chat-channels can be found on [![Gitter chat](https://badges.gitter.im/AnySoftKeyboard/gitter.png)](https://gitter.im/AnySoftKeyboard)

`main` latest build ![](https://github.com/AnySoftKeyboard/AnySoftKeyboard/workflows/checks/badge.svg?event=push&branch=main)

`main` coverage [![codecov](https://codecov.io/gh/AnySoftKeyboard/AnySoftKeyboard/branch/main/graph/badge.svg)](https://codecov.io/gh/AnySoftKeyboard/AnySoftKeyboard)

Android (f/w 4.0.3+, API level 15+) on screen keyboard for multiple languages.

Sign up to beta-channel [here](https://play.google.com/apps/testing/com.menny.android.anysoftkeyboard). Beta versions are pushed via Google Play Store.

## Features
 * All kinds of keyboards:
     * Supporting lots of languages via external packages. E.g., English (QWERTY, Dvorak, AZERTY, Colemak, and Workman), Hebrew, Russian, Arabic, Lao, Bulgarian, Swiss, German, Swedish, Spanish, Catalan, Belarusian, Portuguese, Ukrainian and [many more](addons/languages/PACKS.md).
     * Special keyboard for text fields which require only numbers.
     * Special keyboard for text fields which require email or URI addresses.
 * Physical keyboard is supported as-well.
 * Auto-capitalization.
 * Word suggestions, and Next-Word suggestions.
     * Automatic correction can be customized, or turned off entirely.
     * External packages include word lists that can be freely mixed. You can use a French layout and get suggestions for German and Russian!
 * Gesture typing.
 * Dark mode, automatic (based on system) and manual.
 * Power saving mode, disables various features to save battery.
 * Per-app tint, the keyboard changes color depending on the current app.
 * Special key-press effects:
     * Sound on key press (if phone is not muted).
     * Vibrate on key press.
 * Voice input.
 * Incognito Mode - will not learn new words, will not keep history of what was typed (including emoji history).
 * Plenty of emojis - long-press the smiley key. You customize those by clicking the Settings icon in emojis window.
 * More on AnySoftKeyboard can be found [here](https://anysoftkeyboard.github.io/).

## Releases

### from _main_ branch

#### IME - AnySoftKeyboard
* Every commit to _main_ branch will [deploy](.github/workflows/checks.yml) a new release to the _ALPHA_ channel in Google Play-Store. You can subscribe to this release channel by joining the [Google Groups](https://groups.google.com/d/forum/anysoftkeyboard-alpha-testers) group, and opt-in by visiting [this link](https://play.google.com/apps/testing/com.menny.android.anysoftkeyboard).
* Every Wednesday the latest _ALPHA_ will be [promoted](.github/workflows/deployment_promote.yml) to _BETA_. You can opt-in to this channel by visiting [this link](https://play.google.com/apps/testing/com.menny.android.anysoftkeyboard).
* Once all requirements for a release were finished, a _STABLE_ release branch (in the format of `release-branch-ime-vX.X-rX`) will be cut. Every commit to this branch will be automatically published to Google Play Store (_STABLE_ channel) and will roll-out users gradually.

#### AddOns
* Every commit to _main_ branch will [deploy](.github/workflows/checks.yml) all addons to closed-testing _ALPHA_ channel in Google Play-Store. If you want to subscribe to this channel, open a [ticket](https://github.com/AnySoftKeyboard/AnySoftKeyboard/discussions/3391) and asking to join.
* Every Wednesday the latest _ALPHA_ will be promoted to _BETA_. This is only for supported packs. The _BETA_ channel is an open-testing group, you can subscribe to this release channel by joining the _BETA_ channel for each [addon](addons/languages/PACKS.md).
* Once all requirements for a release were finished, a _STABLE_ release branch (in the format of `release-branch-addons-vX.X-rX`) will be cut. Every commit to this branch will be automatically published to Google Play Store (_STABLE_ channel) and will roll-out users gradually.

### from _release-branch_

#### IME - AnySoftKeyboard
* Each new commit to the release-branch will be published to 10% of the users.
* Each day - if no new commit was pushed to the release-branch - we will increase the roll-out.
* When roll-out reaches 100%, an fdroid release will be made.

#### AddOns
* Each new commit to the release-branch will be published to 10% of the users.
* Each day - if no new commit was pushed to the release-branch - we will increase the roll-out.

## Read more
* Our fancy [web-site](https://anysoftkeyboard.github.io/)
* [Language-Pack](addons/languages/PACKS.md) add-ons in this repo.
* [Theme](addons/themes/PACKS.md) add-ons in this repo.
* [Quick-Text](addons/quicktexts/PACKS.md) add-ons in this repo.
* [Crowdin](https://crowdin.com/project/anysoftkeyboard) to translate the app to your language. [![Crowdin](https://badges.crowdin.net/anysoftkeyboard/localized.svg)](https://crowdin.com/project/anysoftkeyboard)
* Via [Bountysource](https://www.bountysource.com/teams/anysoftkeyboard/issues), users can post bounties and developers can claim them when fixing bugs. This is an external community project and not directly sponsored by the ASK developers!

# Development/Contributing
Want to develop a new feature, fix a bug, or add new language-pack? Read more [here](CONTRIBUTING.md).
Contributors should adhere to the [Code of Conduct](CODE_OF_CONDUCT.md) document.

## Copyright requirement
*Remember:* the components in this repository are released under the Apache2 license. By contributing to this repository you give all copyright and distribution rights to the [AnySoftKeyboard maintainer](https://github.com/menny).

# License

    Copyright 2009 Menny Even-Danan

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.


<picture>
    <source srcset="./media/logo-dark.svg" media="(prefers-color-scheme: dark)" alt="realm by MongoDB">

</picture>

[![Maven Central](https://img.shields.io/maven-central/v/io.realm/realm-gradle-plugin?colorB=4dc427&label=Maven%20Central)](https://search.maven.org/artifact/io.realm/realm-gradle-plugin)
[![License](https://img.shields.io/badge/License-Apache-blue.svg)](https://github.com/realm/realm-java/blob/master/LICENSE)

Realm is a mobile database that runs directly inside phones, tablets or wearables.
This repository holds the source code for the Java version of Realm, which currently runs only on Android.

## Realm Kotlin

The [Realm Kotlin SDK](https://github.com/realm/realm-kotlin) is now GA and can be used for both Android and Kotlin Multiplatform. While we are still adding features, please consider using Realm Kotlin for any new project, and let us know if you miss anything there!

## Features

* **Mobile-first:** Realm is the first database built from the ground up to run directly inside phones, tablets, and wearables.
* **Simple:** Data is directly exposed as objects and queryable by code, removing the need for ORM's riddled with performance & maintenance issues. Plus, we've worked hard to [keep our API down to very few classes](https://www.mongodb.com/docs/atlas/device-sdks/sdk/java/): most of our users pick it up intuitively, getting simple apps up & running in minutes.
* **Modern:** Realm supports easy thread-safety, relationships & encryption.
* **Fast:** Realm is faster than even raw SQLite on common operations while maintaining an extremely rich feature set.
* **[Device Sync](https://www.mongodb.com/atlas/app-services/device-sync)**: Makes it simple to keep data in sync across users, devices, and your backend in real-time. Get started for free with [a template application](https://github.com/mongodb/template-app-react-native-todo) and [create the cloud backend](http://mongodb.com/realm/register?utm_medium=github_atlas_CTA&utm_source=realm_js_github).

## Getting Started

Please see the [detailed instructions in our docs](https://www.mongodb.com/docs/atlas/device-sdks/sdk/java/install/) to add Realm to your project.

## Documentation

Documentation for Realm can be found at [mongodb.com/docs/atlas/device-sdks/sdk/java/](https://www.mongodb.com/docs/atlas/device-sdks/sdk/java/).
The API reference is located at [mongodb.com/docs/atlas/device-sdks/sdk/java/api/](https://www.mongodb.com/docs/atlas/device-sdks/sdk/java/api/).

## Getting Help

- **Got a question?**: Look for previous questions on the [#realm tag](https://stackoverflow.com/questions/tagged/realm?sort=newest) — or [ask a new question](http://stackoverflow.com/questions/ask?tags=realm). We actively monitor & answer questions on StackOverflow! You can also check out our [Community Forum](https://developer.mongodb.com/community/forums/tags/c/realm/9/realm-sdk) where general questions about how to do something can be discussed.
- **Think you found a bug?** [Open an issue](https://github.com/realm/realm-java/issues/new?template=bug_report.md). If possible, include the version of Realm, a full log, the Realm file, and a project that shows the issue.
- **Have a feature request?** [Open an issue](https://github.com/realm/realm-java/issues/new?template=feature_request.md). Tell us what the feature should do, and why you want the feature.

## Using Snapshots

If you want to test recent bugfixes or features that have not been packaged in an official release yet, you can use a **-SNAPSHOT** release of the current development version of Realm via Gradle, available on [Sonatype OSS](https://oss.sonatype.org/#nexus-search;quick~realm-gradle-plugin)

```
buildscript {
    repositories {
        mavenCentral()
        google()
        maven {
            url 'https://oss.sonatype.org/content/repositories/snapshots/'
        }
        jcenter()
    }
    dependencies {
        classpath "io.realm:realm-gradle-plugin:<version>-SNAPSHOT"
    }
}

allprojects {
    repositories {
        mavenCentral()
        google()
        maven {
            url 'https://oss.sonatype.org/content/repositories/snapshots/'
        }
        jcenter()
    }
}
```

See [version.txt](version.txt) for the latest version number.

## Building Realm

In case you don't want to use the precompiled version, you can build Realm yourself from source.

### Prerequisites

 * Download the [**JDK 8**](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) from Oracle and install it.
 * The latest stable version of Android Studio. Currently [4.1.1](https://developer.android.com/studio/).
 * Download & install the Android SDK **Build-Tools 29.0.3**, **Android Pie (API 29)** (for example through Android Studio’s **Android SDK Manager**).
 * Install CMake version 3.18.4 and build Ninja.
 * Install the NDK (Side-by-side) **21.0.6113669** from the SDK Manager in Android Studio. Remember to check `☑  Show package details` in the manager to display all available versions.

 * Add the Android home environment variable to your profile:

    ```
    export ANDROID_HOME=~/Library/Android/sdk
    ```

 * If you are launching Android Studio from the macOS Finder, you should also run the following command:

    ```
    launchctl setenv ANDROID_HOME "$ANDROID_HOME"
    ```

 * If you'd like to specify the location in which to store the archives of Realm Core, define the `REALM_CORE_DOWNLOAD_DIR` environment variable. It enables caching core release artifacts.

   ```
   export REALM_CORE_DOWNLOAD_DIR=~/.realmCore
   ```

   macOS users must also run the following command for Android Studio to see this environment variable.

   ```
   launchctl setenv REALM_CORE_DOWNLOAD_DIR "$REALM_CORE_DOWNLOAD_DIR"
   ```

It would be a good idea to add all of the symbol definitions (and their accompanying `launchctl` commands, if you are using macOS) to your `~/.profile` (or `~/.zprofile` if the login shell is `zsh`)

 * If you develop Realm Java with Android Studio, we recommend you to exclude some directories from indexing target by executing following steps on Android Studio. It really speeds up indexing phase after the build.

    - Under `/realm/realm-library/`, select `build`, `.cxx` and `distribution` folders in `Project` view.
    - Press `Command + Shift + A` to open `Find action` dialog. If you are not using default keymap nor using macOS, you can find your shortcut key in `Keymap` preference by searching `Find action`.
    - Search `Excluded` (not `Exclude`) action and select it. Selected folder icons should become orange (in default theme).
    - Restart Android Studio.

### Download sources

You can download the source code of Realm Java by using git. Since realm-java has git submodules, use `--recursive` when cloning the repository.

```
git clone git@github.com:realm/realm-java.git --recursive
```

or

```
git clone https://github.com/realm/realm-java.git --recursive
```

### Build

Once you have completed all the pre-requisites building Realm is done with a simple command.

```
./gradlew assemble
```

That command will generate:

 * a jar file for the Realm Gradle plugin
 * an aar file for the Realm library
 * a jar file for the annotations
 * a jar file for the annotations processor

The full build may take an hour or more, to complete.

### Building from source

It is possible to build Realm Java with the submodule version of Realm Core. This is done by providing the following parameter when building: `-PbuildCore=true`.

```
./gradlew assembleBase -PbuildCore=true
```

You can turn off interprocedural optimizations with the following parameter: `-PenableLTO=false`.

```
./gradlew assembleBase -PenableLTO=false`
```

Note: Building the `Base` variant would always build realm-core.

Note: Interprocedural optimizations are enabled by default.

Note: If you want to build from source inside Android Studio, you need to update the Gradle parameters by going into the Realm projects settings `Settings > Build, Execution, Deployment > Compiler > Command-line options` and add `-PbuildCore=true` or `-PenableLTO=false` to it. Alternatively you can add it into your `gradle.properties`:

```
buildCore=true
enableLTO=false
```

Note: If building on OSX you might like to prevent Gatekeeper to block all NDK executables by disabling it: `sudo spctl --master-disable`. Remember to enable it afterwards: `sudo spctl --master-enable`

### Other Commands

 * `./gradlew tasks` will show all the available tasks
 * `./gradlew javadoc` will generate the Javadocs
 * `./gradlew monkeyExamples` will run the monkey tests on all the examples
 * `./gradlew installRealmJava` will install the Realm library and plugin to mavenLocal()
 * `./gradlew clean -PdontCleanJniFiles` will remove all generated files except for JNI related files. This reduces recompilation time a lot.
 * `./gradlew connectedUnitTests -PbuildTargetABIs=$(adb shell getprop ro.product.cpu.abi)` will build JNI files only for the ABI which corresponds to the connected device.  These tests require a running Object Server (see below)

Generating the Javadoc using the command above may generate warnings. The Javadoc is generated despite the warnings.

### Upgrading Gradle Wrappers

 All gradle projects in this repository have `wrapper` task to generate Gradle Wrappers. Those tasks refer to `gradle` property defined in `/dependencies.list` to determine Gradle Version of generating wrappers.
We have a script `./tools/update_gradle_wrapper.sh` to automate these steps. When you update Gradle Wrappers, please obey the following steps.

 1. Edit `gradle` property in defined in `/dependencies.list` to new Gradle Wrapper version.
 2. Execute `/tools/update_gradle_wrapper.sh`.

### Gotchas

The repository is organized into six Gradle projects:

 * `realm`: it contains the actual library (including the JNI layer) and the annotations processor.
 * `realm-annotations`: it contains the annotations defined by Realm.
 * `realm-transformer`: it contains the bytecode transformer.
 * `gradle-plugin`: it contains the Gradle plugin.
 * `examples`: it contains the example projects. This project directly depends on `gradle-plugin` which adds a dependency to the artifacts produced by `realm`.
 * The root folder is another Gradle project.  All it does is orchestrate the other jobs.

This means that `./gradlew clean` and `./gradlew cleanExamples` will fail if `assembleExamples` has not been executed first.
Note that IntelliJ [does not support multiple projects in the same window](https://youtrack.jetbrains.com/issue/IDEABKL-6118#)
so each of the six Gradle projects must be imported as a separate IntelliJ project.

Since the repository contains several completely independent Gradle projects, several independent builds are run to assemble it.
Seeing a line like: `:realm:realm-library:compileBaseDebugAndroidTestSources UP-TO-DATE` in the build log does *not* imply
that you can run `./gradlew :realm:realm-library:compileBaseDebugAndroidTestSources`.

## Examples

The `./examples` folder contains many example projects showing how Realm can be used. If this is the first time you checkout or pull a new version of this repository to try the examples, you must call `./gradlew installRealmJava` from the top-level directory first. Otherwise, the examples will not compile as they depend on all Realm artifacts being installed in `mavenLocal()`.

Standalone examples can be [downloaded from website](https://www.mongodb.com/docs/realm/sdk/java/quick-starts/quick-start-local/#complete-example).

## Running Tests on a Device

To run these tests, you must have a device connected to the build computer, and the `adb` command must be in your `PATH`

1. Connect an Android device and verify that the command `adb devices` shows a connected device:

    ```sh
    adb devices
    List of devices attached
    004c03eb5615429f device
    ```

2. Run instrumentation tests:

    ```sh
    cd realm
    ./gradlew connectedBaseDebugAndroidTest
    ```

These tests may take as much as half an hour to complete.

## Running Tests Using The Realm Object Server

Tests in `realm/realm-library/src/syncIntegrationTest` require a running testing server to work.
A docker image can be built from `tools/sync_test_server/Dockerfile` to run the test server.
`tools/sync_test_server/start_server.sh` will build the docker image automatically.

To run a testing server locally:

1. Install [docker](https://www.docker.com/products/overview) and run it.

2. Run `tools/sync_test_server/start_server.sh`:

    ```sh
    cd tools/sync_test_server
    ./start_server.sh
    ```

    This command will not complete until the server has stopped.

3. Run instrumentation tests

    In a new terminal window, run:

    ```sh
    cd realm
    ./gradlew connectedObjectServerDebugAndroidTest
    ```

Note that if using VirtualBox (Genymotion), the network needs to be bridged for the tests to work.
This is done in `VirtualBox > Network`. Set "Adapter 2" to "Bridged Adapter".

These tests may take as much as half an hour to complete.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for more details!

This project adheres to the [MongoDB Code of Conduct](https://www.mongodb.com/community-code-of-conduct).
By participating, you are expected to uphold this code. Please report
unacceptable behavior to [community-conduct@mongodb.com](mailto:community-conduct@mongodb.com).

The directory `realm/config/studio` contains lint and style files recommended for project code.
Import them from Android Studio with Android Studio > Preferences... > Code Style > Manage... > Import,
or Android Studio > Preferences... > Inspections > Manage... > Import.  Once imported select the
style/lint in the drop-down to the left of the Manage... button.

## License

Realm Java is published under the Apache 2.0 license.

Realm Core is also published under the Apache 2.0 license and is available
[here](https://github.com/realm/realm-core).

## Feedback

**_If you use Realm and are happy with it, all we ask is that you, please consider sending out a tweet mentioning [@realm](http://twitter.com/realm) to share your thoughts!_**

**_And if you don't like it, please let us know what you would like improved, so we can fix it!_**


# XMage — Magic, Another Game Engine

[![Build Status](https://app.travis-ci.com/magefree/mage.svg?branch=master)](https://app.travis-ci.com/magefree/mage)
[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=magefree_mage&metric=ncloc)](https://sonarcloud.io/summary/new_code?id=magefree_mage)
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=magefree_mage&metric=coverage)](https://sonarcloud.io/summary/new_code?id=magefree_mage)
[![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=magefree_mage&metric=sqale_index)](https://sonarcloud.io/summary/new_code?id=magefree_mage)
[![latest release](https://img.shields.io/github/v/release/magefree/mage)](https://github.com/magefree/mage/releases/)
[![commints since latest release](https://img.shields.io/github/commits-since/magefree/mage/latest)](https://github.com/magefree/mage/commits/)
[![Join the chat at https://gitter.im/magefree/mage](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/magefree/mage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

XMage allows you to play Magic against one or more online players or computer opponents.
It includes full rules enforcement for over **25 000** unique cards and more than 65 000 reprints from different editions.
You can also find custom sets like Star Wars. All regular sets have nearly all implemented cards.

It supports single matches and tournaments with dozens of game modes like duel, multiplayer, standard, modern, commander,
pauper, oathbreaker, historic, freeform, richman and much more.

Local server supports a [special test mode](https://github.com/magefree/mage/wiki/Development-Testing-Tools) for
testing combos and other game situations with pre-defined conditions.

There are [public servers](http://xmage.today/servers/) where you can play XMage against other players.
You can also host your own server to play against the AI and/or your friends.

XMage community and resources:
* [Official XMage support and feature request on github](https://github.com/magefree/mage/issues);
* [Official XMage forum](http://www.slightlymagic.net/forum/viewforum.php?f=70) (outdated);
* [Reddit XMage group](https://www.reddit.com/r/XMage/);
* [Reddit XMage discord channel](https://discord.gg/Pqf42yn);
* [Latest changes](https://github.com/magefree/mage/commits/master);
* [Latest news](https://jaydi85.github.io/xmage-web-news/news.html);
* [Latest releases](https://github.com/magefree/mage/releases);
* [Project documentation](https://github.com/magefree/mage/wiki).

Servers status:
* http://xmage.today/servers/ (temporarily out of service)

Beta server with un-released or under development features:
* http://xmage.today/ (release version)

## Features

* Multiplatform app: Windows, Linux, MacOS;
* Deck editor (support multiple deck formats and deck sources);
* Two player duel or a multiplayer free-for-all game with up to 10 players;
* Computer AI opponents;
* Players rating system (Glicko);
* Supports special formats like Commander (up to 10 players), Oathbreaker, Cube, Tiny Leaders, Super Standard, Historic Standard and more;
* Single matches or tournaments supported (elimination or swiss type handling), which can be played with up to 16 players:
    * Booster (also Cube) draft tournaments (4-16)
    * Sealed (also from Cube) tournaments (2-16)

## Bug report / feature request

The best way to report bug or feature request is [github's issues page](https://github.com/magefree/mage/issues).

## Installation / running

* Download [latest XMage launcher and app files](http://xmage.today/) and un-pack it to any folder like `D:\games\xmage`;
* You need to have [Java version 8 or later](http://java.com/) to run launcher.

[Wiki page](https://github.com/magefree/mage/wiki) contains detail information about private or public server setup.

### Server options

The XMage server locates by default the configuration file from the current working directory to the relative path `config/config.xml`
(`config\config.xml` in Windows). To change this location, start the server with the property `xmage.config.path` set
to the desired location, for example `-Dxmage.config.path=config/otherconfig.xml`. The option can be set from the
XMageLauncher in `Settings > Java > Server java options`.

## Troubleshooting / FAQ

Github issues page contain [popular problems and fixes](https://github.com/magefree/mage/issues?q=is%3Aissue+label%3AFAQ+):
* [Any: program freezes on startup (white/blue/black screen)](https://github.com/magefree/mage/issues/4461#issuecomment-361108597);
* [Any: can't download images or it stops after some time](https://www.reddit.com/r/XMage/comments/agmcjf/new_xmage_release_with_ravnica_allegiance_rna/);
* [Any: can't run client, could not open ...jvm.cfg](https://github.com/magefree/mage/issues/1272#issuecomment-529789018);
* [Any: no texts or small buttons in launcher](https://github.com/magefree/mage/issues/4126);
* [Windows: ugly cards, buttons or other GUI drawing artifacts](https://github.com/magefree/mage/issues/4626#issuecomment-374640070);
* [MacOS: can't run on M1/M2](https://github.com/magefree/mage/issues/8406#issuecomment-1011720728);
* [MacOS: can't open launcher](https://www.reddit.com/r/XMage/comments/kf8l34/updated_java_on_osx_xmage_not_working/ggej8cq/);
* [MacOS: client freezes in GUI (on connect dialog, on new match)](https://github.com/magefree/mage/issues/4920#issuecomment-517944308);
* [Linux: run on non-standard OS or hardware like Raspberry Pi](https://github.com/magefree/mage/issues/11611#issuecomment-1879385151);
* [Linux: ugly GUI and drawing artifacts](https://github.com/magefree/mage/issues/11611#issuecomment-1879396921);

## Performance tweaks

If you have a good GPU, it's very likely you'll be able to increase performance by **a lot** through extra Java flags.

XMage runs on JRE 8, so [this link](https://docs.oracle.com/javase/8/docs/technotes/guides/2d/flags.html) should give you the available flags.

### How to enable the extra flags

1. Launch XMage
2. In the menu bar from the launcher, click on "Settings", which will open up the Settings Window
3. Go to the "Java" tab
4. You can pass extra flags by editing the "Client java options" text field

### Linux tweaks

#### Enable OpenGL

More info about [OpenGL](https://docs.oracle.com/javase/8/docs/technotes/guides/2d/flags.html#opengl):
* `-Dsun.java2d.opengl=true`

Possible problems:
* _**Caveat**_: [There's a bug](https://bugs.openjdk.java.net/browse/JDK-6545140) with the file chooser when OpenGL is enabled (you use the file chooser when you, for instance, try to load a deck from disk). The [suggested workaround](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6439320) will make the game crash, so it's not an option.
* _**Workaround**_: When using [i3](https://github.com/i3/i3), you're able to work around the bug by toggling the "floating" capabilities of the window and forcing it to re-render.

#### Enable XRender

More info about [XRender](https://docs.oracle.com/javase/8/docs/technotes/guides/2d/flags.html#xrender):
* `-Dsun.java2d.xrender=true`

#### Increase memory usage

This is not guaranteed to yield improvements, but it depends on your use-case.
If you have a lot of RAM to spare, you can increase the initial heap size for good measure.
More details about [memory settings](https://stackoverflow.com/a/57839720/8401696):
* `-Xms1G -Xmx2G`

## Developer

Full project documentation and instructions for developers can be found in [wiki pages](http://github.com/magefree/mage/wiki/).

First steps for Xmage's developers:
* [Setting up your Development Environment](https://github.com/magefree/mage/wiki/Setting-up-your-Development-Environment)
* [Development Testing Tools](https://github.com/magefree/mage/wiki/Development-Testing-Tools)
* [Development Workflow](https://github.com/magefree/mage/wiki/Development-Workflow)
* [Development HOWTO Guides](https://github.com/magefree/mage/wiki/Development-HOWTO-Guides)

[Torch icons created by Freepik - Flaticon](https://www.flaticon.com/free-icons/torch)


image:https://spring.io/badges/spring-hateoas/ga.svg[link=https://spring.io/projects/spring-hateoas]
image:https://spring.io/badges/spring-hateoas/snapshot.svg[link=https://spring.io/projects/spring-hateoas]
image:https://badges.gitter.im/spring-projects/spring-hateoas.png[link=https://gitter.im/spring-projects/spring-hateoas]

image:https://jenkins.spring.io/buildStatus/icon?job=spring-hateoas%2Fmain&subject=main[link=https://jenkins.spring.io/view/SpringHATEOAS/job/spring-hateoas/]
image:https://jenkins.spring.io/buildStatus/icon?job=spring-hateoas%2F1.2.x&subject=1.2.x[link=https://jenkins.spring.io/view/SpringHATEOAS/job/spring-hateoas/]
image:https://jenkins.spring.io/buildStatus/icon?job=spring-hateoas%2F1.1.x&subject=1.1.x[link=https://jenkins.spring.io/view/SpringHATEOAS/job/spring-hateoas/]
image:https://jenkins.spring.io/buildStatus/icon?job=spring-hateoas%2F1.0.x&subject=1.0.x[link=https://jenkins.spring.io/view/SpringHATEOAS/job/spring-hateoas/]
image:https://jenkins.spring.io/buildStatus/icon?job=spring-hateoas%2F0.25.x&subject=0.25.x[link=https://jenkins.spring.io/view/SpringHATEOAS/job/spring-hateoas/]

= Spring HATEOAS

This project provides some APIs to ease creating REST representations that follow the https://en.wikipedia.org/wiki/HATEOAS[HATEOAS] principle when working with Spring and especially Spring MVC. The core problem it tries to address is link creation and representation assembly.

== Working with Spring HATEOAS

Since all commits are headlined with its github issue, git will treat it as a comment. To get around this, apply the following configuration to your clone:

[source]
----
git config core.commentchar "/"
----

== Making a release

1. Create a new release (on the main branch).
+
----
% ci/create-release.sh <release ticket without hash> <release version> <next snapshot version>
----
+
2. With the release tagged, push the tagged version to the release branch.
+
----
% git checkout -b release
% git reset --hard <tag>
% git push -f origin release
----

NOTE: You can chain the previous set of commands together using `&&`.

The pipeline will build and release the "release" branch. It will also build a new snapshot and stage it on artifactory.
And if it's a `.RELEASE`, the pipeline will push it out to Maven Central. To complete a release on Maven Central, you must login
to the server shown in `pom.xml`, close, and release. This is a stop gap to guard against bad releases accidentally getting pushed
out to Maven Central.

=== Running CI tasks locally

Since the pipeline uses Docker, it's easy to:

* Debug what went wrong on your local machine.
* Test out a a tweak to your `test.sh` script before sending it out.
* Experiment against a new image before submitting your pull request.

All of these use cases are great reasons to essentially run what Jenkins does on your local machine.

IMPORTANT: To do this you must have Docker installed on your machine.

1. `docker run -it --mount type=bind,source="$(pwd)",target=/spring-hateoas-github adoptopenjdk/openjdk8:latest /bin/bash`
+
This will launch the Docker image and mount your source code at `spring-hateoas-github`.
+
2. `cd spring-hateoas-github`
+
Next, run the `test.sh` script from inside the container:
+
2. `PROFILE=none ci/test.sh`

Since the container is binding to your source, you can make edits from your IDE and continue to run build jobs.

If you need to test the `build.sh` script, then do this:

1. `docker run -it --mount type=bind,source="$(pwd)",target=/spring-hateoas-github --mount type=bind,source="/tmp/spring-hateoas-artifactory",target=/spring-hateoas-artifactory adoptopenjdk/openjdk8:latest /bin/bash`
+
This will launch the Docker image and mount your source code at `spring-hateoas-github` and the temporary
artifactory output directory at `spring-hateoas-artifactory`.
+
Next, run the `build.sh` script from inside the container:
+
2. `ci/build.sh`

IMPORTANT: `build.sh` will attempt to push to Artifactory. If you don't supply credentials, it will fail.

NOTE: Docker containers can eat up disk space fast! From time to time, run `docker system prune` to clean out old images.

== Resources

* https://spring.io/projects/spring-hateoas#learn[Reference documentation]
* https://spring.io/guides/gs/rest-hateoas/[Getting started guide]


# zipkin

[![Gitter chat](http://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/openzipkin/zipkin)
[![Build Status](https://github.com/openzipkin/zipkin/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/openzipkin/zipkin/actions?query=workflow%3Atest+branch%3Amaster)
[![Maven Central](https://img.shields.io/maven-central/v/io.zipkin/zipkin-server.svg)](https://search.maven.org/search?q=g:io.zipkin%20AND%20a:zipkin-server)

[Zipkin](https://zipkin.io) is a distributed tracing system. It helps gather
timing data needed to troubleshoot latency problems in service architectures.
Features include both the collection and lookup of this data.

If you have a trace ID in a log file, you can jump directly to it. Otherwise,
you can query based on attributes such as service, operation name, tags and
duration. Some interesting data will be summarized for you, such as the
percentage of time spent in a service, and whether operations failed.

The Zipkin UI also presents a dependency diagram showing how many traced
requests went through each application. This can be helpful for identifying
aggregate behavior including error paths or calls to deprecated services.

Application’s need to be “instrumented” to report trace data to Zipkin. This
usually means configuration of a [tracer or instrumentation library](https://zipkin.io/pages/tracers_instrumentation.html). The most
popular ways to report data to Zipkin are via http or Kafka, though many other
options exist, such as Apache ActiveMQ, gRPC and RabbitMQ. The data served to
the UI is stored in-memory, or persistently with a supported backend such as
Apache Cassandra or Elasticsearch.

## Quick-start

The quickest way to get started is to fetch the [latest released server](https://search.maven.org/remote_content?g=io.zipkin&a=zipkin-server&v=LATEST&c=exec) as a self-contained
executable jar. Note that the Zipkin server requires minimum JRE 17+. For example:

```bash
curl -sSL https://zipkin.io/quickstart.sh | bash -s
java -jar zipkin.jar
```

You can also start Zipkin via Docker.
```bash
# Note: this is mirrored as ghcr.io/openzipkin/zipkin
docker run -d -p 9411:9411 openzipkin/zipkin
```

Once the server is running, you can view traces with the Zipkin UI at http://localhost:9411/zipkin.

If your applications aren't sending traces, yet, configure them with [Zipkin instrumentation](https://zipkin.io/pages/tracers_instrumentation) or try one of our [examples](https://github.com/openzipkin?utf8=%E2%9C%93&q=example).

Check out the [`zipkin-server`](zipkin-server/README.md) documentation for configuration details, or [Docker examples](docker/examples) for how to use docker-compose.

### Zipkin Slim

The slim build of Zipkin is smaller and starts faster. It supports in-memory and Elasticsearch storage, but doesn't support messaging transports like Kafka or RabbitMQ. If these constraints match your needs, you can try slim like below:

Running via Java:
```bash
curl -sSL https://zipkin.io/quickstart.sh | bash -s io.zipkin:zipkin-server:LATEST:slim zipkin.jar
java -jar zipkin.jar
```

Running via Docker:
```bash
# Note: this is mirrored as ghcr.io/openzipkin/zipkin-slim
docker run -d -p 9411:9411 openzipkin/zipkin-slim
```

Running via [Homebrew](https://formulae.brew.sh/formula/zipkin):
```bash
brew install zipkin
# to run in foreground
zipkin
# to run in background
brew services start zipkin
```

## Core Library
The [core library](zipkin/src/main/java/zipkin2) is used by both Zipkin instrumentation and the Zipkin server.

This includes built-in codec for Zipkin's v1 and v2 json formats. A direct dependency on gson
(json library) is avoided by minifying and repackaging classes used. The result is a 155k jar which
won't conflict with any library you use.

Ex.
```java
// All data are recorded against the same endpoint, associated with your service graph
localEndpoint = Endpoint.newBuilder().serviceName("tweetie").ip("192.168.0.1").build()
span = Span.newBuilder()
    .traceId("d3d200866a77cc59")
    .id("d3d200866a77cc59")
    .name("targz")
    .localEndpoint(localEndpoint)
    .timestamp(epochMicros())
    .duration(durationInMicros)
    .putTag("compression.level", "9");

// Now, you can encode it as json
bytes = SpanBytesEncoder.JSON_V2.encode(span);
```

Note: The above is just an example, most likely you'll want to use an existing tracing library like [Brave](https://github.com/openzipkin/brave)

### Core Library Requires Java 8+

The minimum Java language level of the core library is 8. This helps support those writing agent
instrumentation. Version 2.x was the last to support Java 6.

*Note*: [zipkin-reporter-brave](https://github.com/openzipkin/zipkin-reporter-java/blob/master/brave/README.md)
does not use this library. So, [brave](https://github.com/openzipkin/brave) still supports Java 6.

## Storage Component
Zipkin includes a [StorageComponent](zipkin/src/main/java/zipkin2/storage/StorageComponent.java), used to store and query spans and
dependency links. This is used by the server and those making collectors, or span reporters.
For this reason, storage components have minimal dependencies, though require Java 17+.

Ex.
```java
// this won't create network connections
storage = ElasticsearchStorage.newBuilder()
                              .hosts(asList("http://myelastic:9200")).build();

// prepare a call
traceCall = storage.spanStore().getTrace("d3d200866a77cc59");

// execute it synchronously or asynchronously
trace = traceCall.execute();

// clean up any sessions, etc
storage.close();
```

### In-Memory
The [InMemoryStorage](zipkin-server#in-memory-storage) component is packaged in zipkin's core library. It
is neither persistent, nor viable for realistic work loads. Its purpose
is for testing, for example starting a server on your laptop without any
database needed.

### Cassandra
The [Cassandra](zipkin-server#cassandra-storage) component uses Cassandra
3.11.3+ features, but is tested against the latest patch of Cassandra 4.1.

This is the second generation of our Cassandra schema. It stores spans
using UDTs, such that they appear like Zipkin v2 json in cqlsh. It is
designed for scale, and uses a combination of SASI and manually
implemented indexes to make querying larger data more performant.

Note: This store requires a [job to aggregate](https://github.com/openzipkin/zipkin-dependencies) dependency links.

### Elasticsearch
The [Elasticsearch](zipkin-server#elasticsearch-storage) component uses
Elasticsearch 5+ features, but is tested against Elasticsearch 7-8.x and
OpenSearch 2.x.

It stores spans as Zipkin v2 json so that integration with other tools is
straightforward. To help with scale, this uses a combination of custom
and manually implemented indexing.

Note: This store requires a [spark job](https://github.com/openzipkin/zipkin-dependencies) to aggregate dependency links.

### Disabling search
The following API endpoints provide search features, and are enabled by
default. Search primarily allows the trace list screen of the UI operate.
* `GET /services` - Distinct Span.localServiceName
* `GET /remoteServices?serviceName=X` - Distinct Span.remoteServiceName by Span.localServiceName
* `GET /spans?serviceName=X` - Distinct Span.name by Span.localServiceName
* `GET /autocompleteKeys` - Distinct keys of Span.tags subject to configurable whitelist
* `GET /autocompleteValues?key=X` - Distinct values of Span.tags by key
* `GET /traces` - Traces matching a query possibly including the above criteria

When search is disabled, traces can only be retrieved by ID
(`GET /trace/{traceId}`). Disabling search is only viable when there is
an alternative way to find trace IDs, such as logs. Disabling search can
reduce storage costs or increase write throughput.

`StorageComponent.Builder.searchEnabled(false)` is implied when a zipkin
is run with the env variable `SEARCH_ENABLED=false`.

### Legacy (v1) components
The following components are no longer encouraged, but exist to help aid
transition to supported ones. These are indicated as "v1" as they use
data layouts based on Zipkin's V1 Thrift model, as opposed to the
simpler v2 data model currently used.

#### MySQL
The [MySQL v1](zipkin-storage/mysql-v1) component uses MySQL 5.6+
features, but is tested against MariaDB 10.11.

The schema was designed to be easy to understand and get started with;
it was not designed for performance. Ex spans fields are columns, so
you can perform ad-hoc queries using SQL. However, this component has
[known performance issues](https://github.com/openzipkin/zipkin/issues/1233): queries will eventually take seconds to return
if you put a lot of data into it.

This store does not require a [job to aggregate](https://github.com/openzipkin/zipkin-dependencies) dependency links.
However, running the job will improve performance of dependencies
queries.

## Running the server from source
The [Zipkin server](zipkin-server) receives spans via HTTP POST and respond to queries
from its UI. It can also run collectors, such as RabbitMQ or Kafka.

To run the server from the currently checked out source, enter the
following. JDK 17+ is required to compile the source.
```bash
# Build the server and also make its dependencies
$ ./mvnw -q --batch-mode -DskipTests --also-make -pl zipkin-server clean install
# Run the server
$ java -jar ./zipkin-server/target/zipkin-server-*exec.jar
```

## Artifacts
Server artifacts are under the maven group id `io.zipkin`
Library artifacts are under the maven group id `io.zipkin.zipkin2`

### Library Releases
Releases are at [Sonatype](https://oss.sonatype.org/content/repositories/releases) and [Maven Central](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22io.zipkin%22)

### Library Snapshots
Snapshots are uploaded to [Sonatype](https://oss.sonatype.org/content/repositories/snapshots) after
commits to master.

### Docker Images
Released versions of zipkin-server are published to Docker Hub as `openzipkin/zipkin` and GitHub
Container Registry as `ghcr.io/openzipkin/zipkin`. See [docker](docker) for details.

### Helm Charts
Helm charts are available via `helm repo add zipkin https://zipkin.io/zipkin-helm`.
See [zipkin-helm](https://github.com/openzipkin/zipkin-helm) for details.

### Javadocs
https://zipkin.io/zipkin contains versioned folders with JavaDocs published on each (non-PR) build, as well
as releases.


# File System Crawler for Elasticsearch

Welcome to the FS Crawler for [Elasticsearch](https://elastic.co/)

This crawler helps to index binary documents such as PDF, Open Office, MS Office.

**Main features**:

* Local file system (or a mounted drive) crawling and index new files, update existing ones and removes old ones.
* Remote file system over SSH/FTP crawling.
* REST interface to let you "upload" your binary documents to elasticsearch.

## Latest versions

Current "most stable" versions are:

| Elasticsearch | FS Crawler    | Released   | Docs                                                                          |
|---------------|---------------|------------|-------------------------------------------------------------------------------|
| 6.x, 7.x, 8.x | 2.10-SNAPSHOT |            | [2.10-SNAPSHOT](https://fscrawler.readthedocs.io/en/latest/)                  |

[![Maven Central](https://img.shields.io/maven-central/v/fr.pilato.elasticsearch.crawler/fscrawler-distribution)](https://repo1.maven.org/maven2/fr/pilato/elasticsearch/crawler/fscrawler-distribution/)
![GitHub Release Date](https://img.shields.io/github/release-date/dadoonet/fscrawler)
[![Maven metadata URL](https://img.shields.io/maven-metadata/v?metadataUrl=https%3A%2F%2Fs01.oss.sonatype.org%2Fcontent%2Frepositories%2Fsnapshots%2Ffr%2Fpilato%2Felasticsearch%2Fcrawler%2Ffscrawler-distribution%2Fmaven-metadata.xml&label=Latest%20SNAPSHOT&link=https%3A%2F%2Fs01.oss.sonatype.org%2Fcontent%2Frepositories%2Fsnapshots%2Ffr%2Fpilato%2Felasticsearch%2Fcrawler%2Ffscrawler-distribution%2F)](https://s01.oss.sonatype.org/content/repositories/snapshots/fr/pilato/elasticsearch/crawler/fscrawler-distribution/)
![GitHub last commit](https://img.shields.io/github/last-commit/dadoonet/fscrawler)

![Docker Pulls](https://img.shields.io/docker/pulls/dadoonet/fscrawler)
![Docker Image Size (tag)](https://img.shields.io/docker/image-size/dadoonet/fscrawler/2.10-SNAPSHOT?label=Docker%20image%20size)
![Docker Image Version (latest semver)](https://img.shields.io/docker/v/dadoonet/fscrawler)

## Build and Quality Status

[![Build](https://github.com/dadoonet/fscrawler/actions/workflows/maven.yml/badge.svg)](https://github.com/dadoonet/fscrawler/actions/workflows/maven.yml)
[![Documentation Status](https://readthedocs.org/projects/fscrawler/badge/?version=latest)](https://fscrawler.readthedocs.io/en/latest/?badge=latest)

[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=ncloc)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=duplicated_lines_density)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=sqale_index)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=reliability_rating)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)

[![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=vulnerabilities)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Bugs](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=bugs)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Code Smells](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=code_smells)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)
[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=dadoonet_fscrawler&metric=security_rating)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)

## GitHub stats

![GitHub commits since latest release (by SemVer including pre-releases)](https://img.shields.io/github/commits-since/dadoonet/fscrawler/latest/master)
![GitHub commit activity (branch)](https://img.shields.io/github/commit-activity/t/dadoonet/fscrawler)
![GitHub contributors](https://img.shields.io/github/contributors/dadoonet/fscrawler)

![GitHub issues](https://img.shields.io/github/issues/dadoonet/fscrawler)
![GitHub pull requests](https://img.shields.io/github/issues-pr/dadoonet/fscrawler)

## Documentation

The guide has been moved to [ReadTheDocs](https://fscrawler.readthedocs.io/en/latest/).

![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/dadoonet)

## Contribute

Works on my machine - and yours ! Spin up pre-configured, standardized dev environments of this repository, by clicking on the button below.

[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#/https://github.com/dadoonet/fscrawler)

# License

![GitHub](https://img.shields.io/github/license/dadoonet/fscrawler)

Read more about the [Apache2 License](https://fscrawler.readthedocs.io/en/latest/index.html#license).

# Thanks

Thanks to [JetBrains](https://www.jetbrains.com/?from=FSCrawler) for the IntelliJ IDEA License!

Thanks to SonarCloud for the free analysis!

[![SonarCloud](https://sonarcloud.io/images/project_badges/sonarcloud-white.svg)](https://sonarcloud.io/summary/new_code?id=dadoonet_fscrawler)


An Embeddable, Bitemporal, Append-Only Database System and Event Store

Stores small-sized, immutable snapshots of your data in an append-only manner. It facilitates querying and reconstructing the entire history as well as easy audits.

[![Tweet](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/intent/tweet?text=SirixDB+-+a+storage+system%2C+which+creates+%28very+small-sized%29+snapshots+of+your+data+on+every+transaction-commit+through+the+implementation+of+a+novel+sliding+snapshot+algorithm.&url=http://sirix.io&via=sirix&hashtags=versioning,diffing,xml,kotlin,coroutines,vertx)

[![Follow](https://img.shields.io/twitter/follow/sirixdb.svg?style=social)](https://twitter.com/sirixdb)

**[Download ZIP](https://github.com/sirixdb/sirix/archive/master.zip)** | **[Join us on Discord](https://discord.gg/yC33wVpv7t)** | **[Community Forum](https://sirix.discourse.group/)** | **[Documentation](https://sirix.io/docs/index.html)** | **[Architecture & Concepts](https://sirix.io/docs/concepts.html)**

**Working on your first Pull Request?** You can learn how from this *free* series [How to Contribute to an Open Source Project on GitHub](https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github) and another tutorial: [How YOU can contribute to OSS, a beginners guide](https://dev.to/itnext/how-you-can-contribute-to-oss-36id)

>"Remember that you're lucky, even if you don't think you are because there's always something that you can be thankful for." - Esther Grace Earl (http://tswgo.org)

**We want to build the database system together with you. Help us and become a maintainer yourself. Why? You may like the software and want to help us. Furthermore, you'll learn a lot. You may want to fix a bug or add a feature. Do you want to add an awesome project to your portfolio? Do you want to grow your network?... All of this are valid reasons besides probably many more**: [Collaborating on Open Source Software](https://youtu.be/TQN8BIxUmRo)

SirixDB appends data to an indexed log file without the need of a WAL. It can be embedded and used as a library from your favorite language on the JVM to store and query data locally or by using a simple CLI. An asynchronous HTTP server, which adds the core and query modules as dependencies, can interact with SirixDB over the network using Keycloak for authentication/authorization. One file stores the data with all revisions and possibly secondary indexes. A second file stores offsets into the file to quickly search for a revision by a given timestamp using an in-memory binary search. Furthermore, a few maintenance files exist, which store the configuration of a resource and the definitions of secondary indexes (if any are configured). Other JSON files keep track of changes in delta files if enabled.

It currently supports the storage and (time travel) querying of XML and JSON data in its binary encoding, tailored to support versioning. The index structures and the whole storage engine has been written from scratch to support versioning natively. We might also implement storing and querying other data formats as relational data.

SirixDB uses a huge persistent (in the functional sense) tree of tries, wherein the committed snapshots share unchanged pages and even common records in changed pages. The system only stores page fragments during a copy-on-write out-of-place operation instead of full pages during a commit to reduce write-amplification. During read operations, the system reads the page fragments in parallel to reconstruct an in-memory page (thus, a fast, random access storage device as a PCIe SSD is best suited or even byte-addressable storage as Intel DC optane memory shortly -- as SirixDB stores fine granular cache-size (not page) aligned modifications in a single file.

**Please consider [sponsoring](https://opencollective.com/sirixdb/) our Open Source work if you like the project.**

**Note: Let us know if you'd like to build a brand-new frontend with, for instance [Svelte](https://svelte.dev), [D3.js](https://d3js), and Typescript.**

**Discuss it in the [Community Forum](https://sirix.discourse.group).**

## Table of contents
-   [Keeping All Versions of Your Data By Sharing Structure](#keeping-all-versions-of-your-data-by-sharing-structure)
-   [JSONiq examples](#jsoniq-examples)
-   [SirixDB Features](#sirixdb-features)
    -   [Design Goals](#design-goals)
    -   [Revision Histories](#revision-histories)
-   [Getting Started](#getting-started)
    -   [Download ZIP or Git Clone](#download-zip-or-git-clone)
    -   [Maven Artifacts](#maven-artifacts)
    -   [Setup of the SirixDB HTTP-Server and Keycloak to use the REST-API](#setup-of-the-sirixdb-http-server-and-keycloak-to-use-the-rest-api)
    -   [Command line tool](#command-line-tool)
    -   [Documentation](#documentation)
-   [Getting Help](#getting-help)
    -   [Community Forum](#community-forum)
    -   [Join us on Discord](#join-us-on-discord)
-   [Contributors](#contributors-)
-   [License](#license)

## Keeping All Versions of Your Data By Sharing Structure
We could write a lot about why keeping all states of your data in a storage system is of great value. In a nutshell, it's all about looking at the evolution of your data, finding trends, doing audits, and implementing efficient undo-/redo-operations. The [Wikipedia page](https://en.wikipedia.org/wiki/Temporal_database) has a bunch of examples. We recently also added use cases over [here](https://sirix.io/docs/index.html).

Our firm belief is that a temporal storage system must address the issues which arise from keeping past states way better than traditional approaches. Usually, storing time-varying, temporal data in database systems that do not support the storage thereof natively results in many unwanted hurdles. They waste storage space, query performance to retrieve past states of your data is not most ideal, and usually, temporal operations are missing altogether.

The DBS must store data so that storage space is used as effectively as possible while supporting the reconstruction of each revision, as the database saw it during the commits. All this should be handled in linear time, whether it's the first revision or the most recent revision. Ideally, the query time of old/past revisions and the most recent revision should be in the same runtime complexity (logarithmic when querying for specific records).

SirixDB not only supports snapshot-based versioning on a record granular level through a novel versioning algorithm called sliding snapshot, but also time travel queries, efficient diffing between revisions, and storing semi-structured data.

Executing the following time-travel query on our binary JSON representation of [Twitter sample data](https://raw.githubusercontent.com/sirixdb/sirix/master/bundles/sirix-core/src/test/resources/json/twitter.json) gives an initial impression of the possibilities:

```xquery
let $statuses := jn:open('mycol.jn','mydoc.jn', xs:dateTime('2019-04-13T16:24:27Z')).statuses
let $foundStatus := for $status in $statuses
  let $dateTimeCreated := xs:dateTime($status.created_at)
  where $dateTimeCreated > xs:dateTime("2018-02-01T00:00:00") and not(exists(jn:previous($status)))
  order by $dateTimeCreated
  return $status
return {"revision": sdb:revision($foundStatus), $foundStatus{text}}
```
The query opens a database/resource in a specific revision based on a timestamp (`2019–04–13T16:24:27Z`) and searches for all statuses, which have a `created_at` timestamp, which has to be greater than the 1st of February in 2018 and did not exist in the previous revision. `.` is a dereferencing operator used to dereference keys in JSON objects, array values can be accessed as shown looping over the values or through specifying an index, starting with zero: `array[0]` for instance, specifies the first value of the array. [Brackit](https://github.com/sirixdb/brackit), our query processor, also supports Python-like array slices to simplify tasks.

## JSONiq examples

To verify changes in a node or its subtree, first, select the node in the revision and then
query for changes using our stored Merkle hash tree, which builds and updates hashes for each node and its subtree and checks the hashes with `sdb:hash($item)`. The function `jn:all-times` delivers the node in all revisions in which it exists. `jn:previous` delivers
the node in the previous revision or an empty sequence if there's none.

```xquery
let $node := jn:doc('mycol.jn','mydoc.jn').fieldName[1]
let $result := for $node-in-rev in jn:all-times($node)
               let $nodeInPreviousRevision := jn:previous($node-in-rev)
               return
                 if ((not(exists($nodeInPreviousRevision)))
                      or (sdb:hash($node-in-rev) ne sdb:hash($nodeInPreviousRevision))) then
                   $node-in-rev
                 else
                   ()
return [
  for $jsonItem in $result
  return { "node": $jsonItem, "revision": sdb:revision($jsonItem) }
]
```

Emit all diffs between the revisions in a JSON format:

```xquery
let $maxRevision := sdb:revision(jn:doc('mycol.jn','mydoc.jn'))
let $result := for $i in (1 to $maxRevision)
               return
                 if ($i > 1) then
                   jn:diff('mycol.jn','mydoc.jn',$i - 1, $i)
                 else
                   ()
return [
  for $diff at $pos in $result
  return {"diffRev" || $pos || "toRev" || $pos + 1: jn:parse($diff).diffs}
]
```

We support easy updates as in

```xquery
let $array := jn:doc('mycol.jn','mydoc.jn')
return insert json {"bla":true} into $array at position 0
```

to insert a JSON object into a resource, whereas the root node is an array at the first position (0). The transaction is implicitly committed. Thus, a new revision is created, and the specific revision can be queried using a single third argument, either a simple integer ID or a timestamp. The following query issues a query on the first revision (thus without the changes).
```xquery
jn:doc('mycol.jn','mydoc.jn',1)
```

Omitting the third argument opens the resource in the most recent revision, but you could, in this case, also specify revision number 2. You can also use a timestamp as in:

```xquery
jn:open('mycol.jn','mydoc.jn',xs:dateTime('2022-03-01T00:00:00Z'))
```

A simple join (whereas joins are optimized in our query processor called Brackit):

```xquery
(* first: store stores in a stores resource *)
sdb:store('mycol.jn','stores','
[
  { "store number" : 1, "state" : "MA" },
  { "store number" : 2, "state" : "MA" },
  { "store number" : 3, "state" : "CA" },
  { "store number" : 4, "state" : "CA" }
]')

(* second: store sales in a sales resource *)
sdb:store('mycol.jn','sales','
[
  { "product" : "broiler", "store number" : 1, "quantity" : 20  },
  { "product" : "toaster", "store number" : 2, "quantity" : 100 },
  { "product" : "toaster", "store number" : 2, "quantity" : 50 },
  { "product" : "toaster", "store number" : 3, "quantity" : 50 },
  { "product" : "blender", "store number" : 3, "quantity" : 100 },
  { "product" : "blender", "store number" : 3, "quantity" : 150 },
  { "product" : "socks", "store number" : 1, "quantity" : 500 },
  { "product" : "socks", "store number" : 2, "quantity" : 10 },
  { "product" : "shirt", "store number" : 3, "quantity" : 10 }
]')

let $stores := jn:doc('mycol.jn','stores')
let $sales := jn:doc('mycol.jn','sales')
let $join :=
  for $store in $stores, $sale in $sales
  where $store."store number" = $sale."store number"
  return {
    "nb" : $store."store number",
    "state" : $store.state,
    "sold" : $sale.product
  }
return [$join]
```

SirixDB through Brackit also supports array slices. The start index is 0, the step is 1, and end index is 1 (exclusive) in the next query:

```xquery
let $array := [{"foo": 0}, "bar", {"baz": true}]
return $array[0:1:1]
```

The query returns the first object `{"foo":0}`.

With the function `sdb:nodekey` you can find out the internal unique node key of a node, which will never change. You for instance might be interested in which revision it has been removed. The following query uses the function `sdb:select-item` which, as the first argument needs a context node and, as the second argument the key of the item or node to select. `jn:last-existing` finds the most recent version and `sdb:revision` retrieves the revision number.

```xquery
sdb:revision(jn:last-existing(sdb:select-item(jn:doc('mycol.jn','mydoc.jn',1), 26)))
```

### Index types

SirixDB has three types of indexes along with a path summary tree, which is basically a tree of all distinct paths:

- name indexes, to index a set of object fields
- path indexes, to index a set of paths (or all paths in a resource)
- CAS indexes, so-called content-and-structure indexes, which index paths and typed values (for instance, all xs:integers). In this case, on the paths specified, only integer values are indexed on the path, but no other types

We base the indexes on the following serialization of three revisions of a very small SirixDB resource.

```json
{
  "sirix": [
    {
      "revisionNumber": 1,
      "revision": {
        "foo": [
          "bar",
          null,
          2.33
        ],
        "bar": {
          "hello": "world",
          "helloo": true
        },
        "baz": "hello",
        "tada": [
          {
            "foo": "bar"
          },
          {
            "baz": false
          },
          "boo",
          {},
          []
        ]
      }
    },
    {
      "revisionNumber": 2,
      "revision": {
        "tadaaa": "todooo",
        "foo": [
          "bar",
          null,
          103
        ],
        "bar": {
          "hello": "world",
          "helloo": true
        },
        "baz": "hello",
        "tada": [
          {
            "foo": "bar"
          },
          {
            "baz": false
          },
          "boo",
          {},
          []
        ]
      }
    },
    {
      "revisionNumber": 3,
      "revision": {
        "tadaaa": "todooo",
        "foo": [
          "bar",
          null,
          23.76
        ],
        "bar": {
          "hello": "world",
          "helloo": true
        },
        "baz": "hello",
        "tada": [
          {
            "foo": "bar"
          },
          {
            "baz": false
          },
          "boo",
          {},
          [
            {
              "foo": "bar"
            }
          ]
        ]
      }
    }
  ]
}
```

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $stats := jn:create-name-index($doc, ('foo','bar'))
return {"revision": sdb:commit($doc)}
```
The index is created for "foo" and "bar" object fields. You can query for "foo" fields as for instance:

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $nameIndexNumber := jn:find-name-index($doc, 'foo')
for $node in jn:scan-name-index($doc, $nameIndexNumber, 'foo')
order by sdb:revision($node), sdb:nodekey($node)
return {"nodeKey": sdb:nodekey($node), "path": sdb:path($node), "revision": sdb:revision($node)}
```

Second, whole paths are indexable.

Thus, the following path index is applicable to both queries: `.sirix[].revision.tada[].foo` and `.sirix[].revision.tada[][4].foo`. Thus, essentially both foo nodes are indexed and the first child has to be fetched afterwards. For the second query also the array index 4 has to be checked if the indexed node is really on index 4.

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $stats := jn:create-path-index($doc, '/sirix/[]/revision/tada//[]/foo')
return {"revision": sdb:commit($doc)}
```

The index might be scanned as follows:

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $pathIndexNumber := jn:find-path-index($doc, '/sirix/[]/revision/tada//[]/foo')
for $node in jn:scan-path-index($doc, $pathIndexNumber, '/sirix/[]/revision/tada//[]/foo')
order by sdb:revision($node), sdb:nodekey($node)
return {"nodeKey": sdb:nodekey($node), "path": sdb:path($node)}
```

CAS indexes index a path plus the value. The value itself must be typed (so in this case we index only decimals on a path).

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $stats := jn:create-cas-index($doc, 'xs:decimal', '/sirix/[]/revision/foo/[]')
return {"revision": sdb:commit($doc)}
```

We can do an index range-scan as for instance via the next query (2.33 and 100 are the min and max, the next two arguments are two booleans which denote if the min and max should be retrieved or if it's >min and <max). The last argument is usually a path if we index more paths in the same index (in this case we only index `/sirix/[]/revision/foo/[]`).

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $casIndexNumber := jn:find-cas-index($doc, 'xs:decimal', '/sirix/[]/revision/foo/[]')
for $node in jn:scan-cas-index-range($doc, $casIndexNumber, 2.33, 100, false(), true(), ())
order by sdb:revision($node), sdb:nodekey($node)
return {"nodeKey": sdb:nodekey($node), "node": $node}
```

You can also create a CAS index on all string values on all paths (all object fields: `//*`; all arrays: `//[]`):

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $stats := jn:create-cas-index($doc,'xs:string',('//*','//[]'))
return {"revision": sdb:commit($doc)}
```

To query for string values with a certain name (`bar`) on all paths (empty sequence `()`):

```xquery
let $doc := jn:doc('mycol.jn','mydoc.jn')
let $casIndexNumber := jn:find-cas-index($doc, 'xs:string', '//*')
for $node in jn:scan-cas-index($doc, $casIndexNumber, 'bar', '==', ())
order by sdb:revision($node), sdb:nodekey($node)
return {"nodeKey": sdb:nodekey($node), "node": $node, "path": sdb:path(sdb:select-parent($node))}
```

The argument `==` means check for equality of the string. Other values that make more sense for integers, and decimals... are `<`, `<=`, `>=` and `>`.

## SirixDB Features
SirixDB is a log-structured, temporal JSON and XML database system, which stores evolutionary data. It never overwrites any data on disk. Thus, we're able to restore and query the full revision history of a resource in the database.

### Design Goals
Some of the most important core principles and design goals are:

Embeddable

Similar to SQLite and DucksDB, SirixDB is embeddable at its core. Other APIs as the non-blocking REST API are built on top.

Minimize Storage Overhead

SirixDB shares unchanged data pages as well as records between revisions, depending on a chosen versioning algorithm during the initial bootstrapping of a resource. SirixDB aims to balance read and writer performance in its default configuration.

Concurrent

SirixDB contains very few locks and aims to be as suitable for multithreaded systems as possible.

Asynchronous

Operations can happen independently; each transaction is bound to a specific revision and only one read/write transaction on a resource is permitted concurrently with N read-only transactions.

Versioning/Revision history

SirixDB stores a revision history of every resource in the database without imposing extra overhead. It uses a huge persistent, durable page tree for indexing revisions and data.

Data integrity

SirixDB, like ZFS, stores full checksums of the pages in the parent pages. That means that almost all data corruption can be detected upon reading in the future; we aim to partition and replicate databases in the future.

Copy-on-write semantics

Similarly to the file systems Btrfs and ZFS, SirixDB uses CoW semantics, meaning that SirixDB never overwrites data. Instead, database-page fragments are copied/written to a new location. SirixDB does not simply copy whole pages. Instead, it only copies changed records plus records, which fall out of a sliding window.

Per revision and page versioning

SirixDB does not only version on a per revision but also on a per page base. Thus, whenever we change a potentially small fraction
of records in a data page, it does not have to copy the whole page and write it to a new location on a disk or flash drive. Instead, we can specify one of several versioning strategies known from backup systems or a novel sliding snapshot algorithm during the creation of a database resource. The versioning type we specify is used by SirixDB to version data pages.

Guaranteed atomicity and consistency (without a WAL)

The system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won't ever damage the system. This is accomplished without the overhead of a write-ahead log.
(WAL)

Log-structured and SSD friendly

SirixDB batches writes and syncs everything sequentially to a flash drive
during commits. It never overwrites committed data.

### Revision Histories
**Keeping the revision history is one of the main features in
SirixDB.** You can revert any revision to an earlier version or back up the system automatically without the overhead of copying. SirixDB only ever copies changed database pages and, depending on the versioning algorithm you chose during the creation of a database/resource, only page fragments, and ancestor index pages to create a new revision.

You can reconstruct every revision in O(n), where n
denotes the number of nodes in the revision. Binary search is used on
an in-memory (linked) map to load the revision, thus finding the
revision root page has an asymptotic runtime complexity of O(log
n), where n, in this case, is the number of stored
revisions.

Currently, SirixDB offers two built-in native data models, namely a
binary XML store and a JSON store.

Articles published on Medium:
- [Asynchronous, Temporal  REST With Vert.x, Keycloak and Kotlin Coroutines](https://medium.com/hackernoon/asynchronous-temporal-rest-with-vert-x-keycloak-and-kotlin-coroutines-217b25756314)
- [Pushing Database Versioning to Its Limits by Means of a Novel Sliding Snapshot Algorithm and Efficient Time Travel Queries](https://medium.com/sirixdb-sirix-io-how-we-built-a-novel-temporal/why-and-how-we-built-a-temporal-database-system-called-sirixdb-open-source-from-scratch-a7446f56f201)
- [How we built an asynchronous, temporal RESTful API based on Vert.x, Keycloak and Kotlin/Coroutines for Sirix.io (Open Source)](https://medium.com/sirixdb-sirix-io-how-we-built-a-novel-temporal/how-we-built-an-asynchronous-temporal-restful-api-based-on-vert-x-4570f681a3)
- [Why Copy-on-Write Semantics and Node-Level-Versioning are Key to Efficient Snapshots](https://hackernoon.com/sirix-io-why-copy-on-write-semantics-and-node-level-versioning-are-key-to-efficient-snapshots-754ba834d3bb)

## Status
SirixDB as of now has not been tested in production. It is recommended for experiments, testing, benchmarking, etc., but is not recommended for production usage. Let us know if you'd like to use SirixDB in production and get in touch. We'd like to test real-world datasets and fix issues we encounter along the way.

Please also get in touch if you like our vision, and you want to sponsor us or help with man-power or if you want to use SirixDB as a research system. We'd be glad to get input from the database and scientific community.

## Getting started

### [Download ZIP](https://github.com/sirixdb/sirix/archive/master.zip) or Git Clone

```
git clone https://github.com/sirixdb/sirix.git
```

or use the following dependencies in your Maven or Gradle project.

**SirixDB uses Java 22, thus you need an up-to-date Gradle (if you want to work on SirixDB) or simply use the gradle wrapper and an IDE (for instance IntelliJ or Eclipse). Also make sure to use the provided Gradle wrapper.**

### Maven artifacts
At this stage of development, you should use the latest SNAPSHOT artifacts from [the OSS snapshot repository](https://oss.sonatype.org/content/repositories/snapshots/io/sirix/) to get the most recent changes.

Just add the following repository section to your POM or build.gradle file:
```xml
<repository>
  <id>sonatype-nexus-snapshots</id>
  <name>Sonatype Nexus Snapshots</name>
  <url>https://oss.sonatype.org/content/repositories/snapshots</url>
  <releases>
    <enabled>false</enabled>
  </releases>
  <snapshots>
    <enabled>true</enabled>
  </snapshots>
</repository>
```
```groovy
repository {
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots/"
        mavenContent {
            snapshotsOnly()
        }
    }
}
```

Note that we changed the groupId from `com.github.sirixdb.sirix` to `io.sirix`.

Maven artifacts are deployed to the central maven repository (however please use the SNAPSHOT-variants as of now). Currently, the following artifacts are available:

Core project:
```xml
<dependency>
  <groupId>io.sirix</groupId>
  <artifactId>sirix-core</artifactId>
  <version>0.x.y-SNAPSHOT</version>
</dependency>
```
```groovy
compile group:'io.sirix', name:'sirix-core', version:'0.x.y-SNAPSHOT'
```

Brackit binding:
```xml
<dependency>
  <groupId>io.sirix</groupId>
  <artifactId>sirix-query</artifactId>
  <version>0.x.y-SNAPSHOT</version>
</dependency>
```
```groovy
compile group:'io.sirix', name:'sirix-query', version:'0.x.y-SNAPSHOT'
```

Asynchronous, RESTful API with Vert.x, Kotlin and Keycloak (the latter for authentication via OAuth2/OpenID-Connect):
```xml
<dependency>
  <groupId>io.sirix</groupId>
  <artifactId>sirix-rest-api</artifactId>
  <version>0.x.y-SNAPSHOT</version>
</dependency>
```

```groovy
compile group: 'io.sirix', name: 'sirix-rest-api', version: '0.x.y-SNAPSHOT'
```

Other modules are currently not available (namely the GUI, the distributed package as well as an outdated Saxon binding).

You have to add the following JVM parameters currently:

```
-ea
--enable-preview
--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED
--add-exports=java.base/sun.nio.ch=ALL-UNNAMED
--add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-opens=jdk.compiler/com.sun.tools.javac=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
```

Plus we recommend using the Shenandoah GC or ZGC (if possible in the future the generational versions):

```
-XX:+UseZGC
-Xlog:gc
-XX:+AlwaysPreTouch
-XX:+UseLargePages
-XX:-UseBiasedLocking
-XX:+DisableExplicitGC
```

We've also had perfect results using GraalVM, possibly due to its JIT compiler and the improved escape analysis.

### Setup of the SirixDB HTTP-Server and Keycloak to use the REST-API

The REST-API is asynchronous at its very core. We use Vert.x, which is a toolkit built on top of Netty. It is heavily inspired by Node.js but for the JVM. As such, it uses event loop(s), which is thread(s), which never should by blocked by long-running CPU tasks or disk-bound I/O. We are using Kotlin with coroutines to keep the code simple. SirixDB uses OAuth2 (Password Credentials/Resource Owner Flow) using a Keycloak authorization server instance.

### Keycloak setup (Standalone Setup / Docker Hub Image)

You can set up Keycloak as described in this excellent [tutorial](
https://piotrminkowski.wordpress.com/2017/09/15/building-secure-apis-with-vert-x-and-oauth2/). Our [docker-compose](https://raw.githubusercontent.com/sirixdb/sirix/master/docker-compose.yml) file imports a sirix realm with a default admin user with all available roles assigned. You can skip steps 3 - 7 and 10, 11, and simply recreate a `client-secret` and change `oAuthFlowType` to "PASSWORD". If you want to run or modify the integration tests, the client secret must not be changed. Make sure to delete the line "build: ." in the `docker-compse.yml` file for the server image if you want to use the Docker Hub image.

1. Open your browser. URL: http://localhost:8080
2. Login with username "admin" and password "admin"
3. Create a new **realm** with the name **"sirixdb"**
4. Go to `Clients` => `account`
5. Change client-id to "sirix"
6. Make sure `access-type` is set to `confidential`
7. Go to the `Credentials` tab
8. Put the `client secret` into the SirixDB HTTP-Server [configuration file]( https://raw.githubusercontent.com/sirixdb/sirix/master/bundles/sirix-rest-api/src/main/resources/sirix-conf.json). Change the value of "client.secret" to whatever Keycloak set up.
9. If "oAuthFlowType" is specified in the same configuration file change the value to "PASSWORD" (if not default is "PASSWORD").
10. Regarding Keycloak the `direct access` grant on the settings tab must be `enabled`.
11. Our (user-/group-)roles are "create" to allow creating databases/resources, "view" to allow to query database resources, "modify" to modify a database resource and "delete" to allow deletion thereof. You can also assign `${databaseName}-` prefixed roles.

### Start Docker Keycloak-Container using docker-compose
For setting up the SirixDB HTTP-Server and a basic Keycloak-instance with a test realm:

1. `git clone https://github.com/sirixdb/sirix.git`
2. `sudo docker-compose up keycloak`

### Start the SirixDB HTTP-Server and the Keycloak-Container using docker-compose

_This section describes setting up the Keycloak using `docker compose`. If you are looking for configuring keycloak from scratch, instructions for that is described in the previous section_

For setting up the SirixDB HTTP-Server and a basic Keycloak-instance with a test `sirixdb` realm:

1. At first clone this repository with the following command (Or download .zip)

       git clone https://github.com/sirixdb/sirix.git

2. `cd` into the sirix folder that was just cloned.

       cd sirix/

3. Run the Keycloak container using `docker compose`

       sudo docker compose up keycloak

4. Visit `http://localhost:8080` and login to the admin console using username: `admin` and password: `admin`
5. From the navigation panel on the left, select `Realm Settings` and verify that the `Name` field is set to `sirixdb`
6. Select the client with Client ID `sirix`
   1. Verify direct access grant is enabled.
   2. Verify that `Access Type` is set to `confidential`
   3. In the `credentials` tab
      1. Verify that the Client Authentication is set to `Client Id and Secret`
      2. Click on Regenerate Secret to generate a new secret. Set the value of the field named `client.secret` of the [configuration file](https://raw.githubusercontent.com/sirixdb/sirix/master/bundles/sirix-rest-api/src/main/resources/sirix-conf.json) to this secret.

7. Finally run the SirixDB-HTTP Server and Keycloak container with docker compose

        docker compose up

### SirixDB HTTP-Server Setup Without Docker/docker-compose

To created a fat-JAR. Download our ZIP-file for instance, then

1. `cd bundles/sirix-rest-api`
2. `./gradlew build -x test`

And a fat-JAR with all required dependencies should have been created in your target folder.

Furthermore, a `key.pem` and a `cert.pem` file are needed. These two files have to be in your user home directory in a directory called "sirix-data", where Sirix stores the databases. For demo purposes, they can be copied from our [resources directory](https://github.com/sirixdb/sirix/tree/master/bundles/sirix-rest-api/src/main/resources).

Once also Keycloak is set up we can start the server via:

`java -jar -Duser.home=/opt/sirix sirix-rest-api-*-SNAPSHOT-fat.jar -conf sirix-conf.json -cp /opt/sirix/*`

If you like to change your user home directory to `/opt/sirix` for instance.

The fat-JAR in the future will be downloadable from the [maven repository](https://oss.sonatype.org/content/repositories/snapshots/io/sirix/sirix-rest-api/0.9.0-SNAPSHOT/).

### Run the Integration Tests
In order to run the integration tests under `bundles/sirix-rest-api/src/test/kotlin` make sure that you assign your admin user all the user roles you have created in the Keycloak setup (last step). Make sure that Keycloak is running first and execute the tests in your favorite IDE for instance.

Note that the following VM parameters currently are needed: `-ea --add-modules=jdk.incubator.foreign --enable-preview`

### Command-line tool
We ship a (very) simple command-line tool for the sirix-query bundle:

Get the [latest sirix-xquery JAR](https://oss.sonatype.org/content/repositories/snapshots/io/sirix/sirix-query/) with dependencies.

### Documentation
We are currently working on the documentation. You may find first drafts and snippets in the [documentation](https://sirix.io/docs/) and in this README. Furthermore, you are kindly invited to ask any question you might have (and you likely have many questions) in the community forum (preferred) or in the Discord channel.
Please also have a look at and play with our sirix-example bundle which is available via Maven or our new asynchronous RESTful API (shown next).

## Getting Help

### Community Forum
If you have any questions or are considering contributing or using Sirix, please use the [Community Forum](https://sirix.discourse.group) to ask questions. Any kind of question, may it be an API question or enhancement proposal, questions regarding use cases are welcome... Don't hesitate to ask questions or make suggestions for improvements. At the moment also, API-related suggestions and critics are of utmost importance.

### Join us on Discord
You may find us on [Discord](https://discord.gg/yC33wVpv7t) for quick questions.

## Contributors ✨

SirixDB is maintained by

* Johannes Lichtenberger

And the Open Source Community.

As the project was forked from a university project called Treetank, my deepest gratitude to Marc Kramis, who came up with the idea of building a versioned, secure, and energy-efficient data store, which retains the history of resources of his Ph.D. Furthermore, Sebastian Graf came up with a lot of ideas and greatly improved the implementation of his Ph.D. Besides, a lot of students worked and improved the project considerably.

Thanks go to these wonderful people, who greatly improved SirixDB lately. SirixDB couldn't exist without the help of the Open Source community:

Ilias YAHIA
💻

BirokratskaZila
📖

Andrei Buiza
💻

Bondar Dmytro
💻

santoshkumarkannur
📖

Lars Eckart
💻

Jayadeep K M
📆

Keith Kim
🎨

Theofanis Despoudis
📖

Mario Iglesias Alarcón
🎨

Antonio Nuno Monteiro
📆

Fulton Browne
📖

Felix Rabe
📖

Ethan Willis
📖

Erik Axelsson
💻

Sérgio Batista
📖

chaensel
📖

Balaji Vijayakumar
💻

Fernanda Campos
💻

Joel Lau
💻

add09
💻

Emil Gedda
💻

Andreas Rohlén
💻

Marcin Bielecki
💻

Manfred Nentwig
💻

Raj
💻

Moshe Uminer
💻

Contributions of any kind are highly welcome!

## License

This work is released under the [BSD 3-clause license](LICENSE).