ShedLock
========
[![Apache License 2](https://img.shields.io/badge/license-ASF2-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0.txt) [![Build Status](https://github.com/lukas-krecan/ShedLock/workflows/CI/badge.svg)](https://github.com/lukas-krecan/ShedLock/actions) [![Maven Central](https://maven-badges.herokuapp.com/maven-central/net.javacrumbs.shedlock/shedlock-parent/badge.svg)](https://maven-badges.herokuapp.com/maven-central/net.javacrumbs.shedlock/shedlock-parent)

ShedLock makes sure that your scheduled tasks are executed at most once at the same time.
If a task is being executed on one node, it acquires a lock which prevents execution of the same task from another node (or thread).
Please note, that **if one task is already being executed on one node, execution on other nodes does not wait, it is simply skipped**.

ShedLock uses an external store like Mongo, JDBC database, Redis, Hazelcast, ZooKeeper or others for coordination.

Feedback and pull-requests welcome!

#### ShedLock is not a distributed scheduler
Please note that ShedLock is not and will never be full-fledged scheduler, it's just a lock. If you need a distributed
scheduler, please use another project ([db-scheduler](https://github.com/kagkarlsson/db-scheduler), [JobRunr](https://www.jobrunr.io/en/)).
ShedLock is designed to be used in situations where you have scheduled tasks that are not ready to be executed in parallel, but can be safely
executed repeatedly. Moreover, the locks are time-based and ShedLock assumes that clocks on the nodes are synchronized.

+ [Versions](#versions)
+ [Components](#components)
+ [Usage](#usage)
+ [Lock Providers](#configure-lockprovider)
  - [JdbcTemplate](#jdbctemplate)
  - [R2DBC](#r2dbc)
  - [jOOQ](#jooq-lock-provider)
  - [Micronaut Data Jdbc](#micronaut-data-jdbc)
  - [Mongo](#mongo)
  - [DynamoDB](#dynamodb)
  - [DynamoDB 2](#dynamodb-2)
  - [ZooKeeper (using Curator)](#zookeeper-using-curator)
  - [Redis (using Spring RedisConnectionFactory)](#redis-using-spring-redisconnectionfactory)
  - [Redis (using Spring ReactiveRedisConnectionFactory)](#redis-using-spring-reactiveredisconnectionfactory)
  - [Redis (using Jedis)](#redis-using-jedis)
  - [Hazelcast](#hazelcast)
  - [Couchbase](#couchbase)
  - [ElasticSearch](#elasticsearch)
  - [OpenSearch](#opensearch)
  - [CosmosDB](#cosmosdb)
  - [Cassandra](#cassandra)
  - [Consul](#consul)
  - [ArangoDB](#arangodb)
  - [Neo4j](#neo4j)
  - [Etcd](#etcd)
  - [Apache Ignite](#apache-ignite)
  - [In-Memory](#in-memory)
  - [Memcached](#memcached-using-spymemcached)
  - [Datastore](#datastore)
+ [Multi-tenancy](#multi-tenancy)
+ [Customization](#customization)
+ [Duration specification](#duration-specification)
+ [Extending the lock](#extending-the-lock)
+ [Micronaut integration](#micronaut-integration)
+ [CDI integration](#cdi-integration)
+ [Locking without a framework](#locking-without-a-framework)
+ [Troubleshooting](#troubleshooting)
+ [Modes of Spring integration](#modes-of-spring-integration)
  - [Scheduled method proxy](#scheduled-method-proxy)
  - [TaskScheduler proxy](#taskscheduler-proxy)
+ [Release notes](#release-notes)

## Versions
If you are using JDK >17 and up-to-date libraries like Spring 6, use version **5.1.0** ([Release Notes](#500-2022-12-10)). If you
are on older JDK or library, use version **4.44.0** ([documentation](https://github.com/lukas-krecan/ShedLock/tree/version4)).

## Components
Shedlock consists of three parts
* Core - The locking mechanism
* Integration - integration with your application, using Spring AOP, Micronaut AOP or manual code
* Lock provider - provides the lock using an external process like SQL database, Mongo, Redis and others

## Usage
To use ShedLock, you do the following
1) Enable and configure Scheduled locking
2) Annotate your scheduled tasks
3) Configure a Lock Provider

### Enable and configure Scheduled locking (Spring)
First of all, we have to import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-spring</artifactId>
    <version>5.13.0</version>
</dependency>
```

Now we need to integrate the library with Spring. In order to enable schedule locking use `@EnableSchedulerLock` annotation

```java
@Configuration
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "10m")
class MySpringConfiguration {
    ...
}
```

### Annotate your scheduled tasks

```java
import net.javacrumbs.shedlock.spring.annotation.SchedulerLock;

...

@Scheduled(...)
@SchedulerLock(name = "scheduledTaskName")
public void scheduledTask() {
    // To assert that the lock is held (prevents misconfiguration errors)
    LockAssert.assertLocked();
    // do something
}
```

The `@SchedulerLock` annotation has several purposes. First of all, only annotated methods are locked, the library ignores
all other scheduled tasks. You also have to specify the name for the lock. Only one task with the same name can be executed
at the same time.

You can also set `lockAtMostFor` attribute which specifies how long the lock should be kept in case the
executing node dies. This is just a fallback, under normal circumstances the lock is released as soon the tasks finishes
(unless `lockAtLeastFor` is specified, see below)
**You have to set `lockAtMostFor` to a value which is much longer than normal execution time.** If the task takes longer than
`lockAtMostFor` the resulting behavior may be unpredictable (more than one process will effectively hold the lock).

If you do not specify `lockAtMostFor` in `@SchedulerLock` default value from `@EnableSchedulerLock` will be used.

Lastly, you can set `lockAtLeastFor` attribute which specifies minimum amount of time for which the lock should be kept.
Its main purpose is to prevent execution from multiple nodes in case of really short tasks and clock difference between the nodes.

All the annotations support Spring Expression Language (SpEL).

#### Example
Let's say you have a task which you execute every 15 minutes and which usually takes few minutes to run.
Moreover, you want to execute it at most once per 15 minutes. In that case, you can configure it like this:

```java
import net.javacrumbs.shedlock.core.SchedulerLock;

@Scheduled(cron = "0 */15 * * * *")
@SchedulerLock(name = "scheduledTaskName", lockAtMostFor = "14m", lockAtLeastFor = "14m")
public void scheduledTask() {
    // do something
}

```
By setting `lockAtMostFor` we make sure that the lock is released even if the node dies. By setting `lockAtLeastFor`
we make sure it's not executed more than once in fifteen minutes.
Please note that **`lockAtMostFor` is just a safety net in case that the node executing the task dies, so set it to
a time that is significantly larger than maximum estimated execution time.**  If the task takes longer than `lockAtMostFor`,
it may be executed again and the results will be unpredictable (more processes will hold the lock).

### Configure LockProvider
There are several implementations of LockProvider.

#### JdbcTemplate
First, create lock table (**please note that `name` has to be primary key**)

```sql
# MySQL, MariaDB
CREATE TABLE shedlock(name VARCHAR(64) NOT NULL, lock_until TIMESTAMP(3) NOT NULL,
    locked_at TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), locked_by VARCHAR(255) NOT NULL, PRIMARY KEY (name));

# Postgres
CREATE TABLE shedlock(name VARCHAR(64) NOT NULL, lock_until TIMESTAMP NOT NULL,
    locked_at TIMESTAMP NOT NULL, locked_by VARCHAR(255) NOT NULL, PRIMARY KEY (name));

# Oracle
CREATE TABLE shedlock(name VARCHAR(64) NOT NULL, lock_until TIMESTAMP(3) NOT NULL,
    locked_at TIMESTAMP(3) NOT NULL, locked_by VARCHAR(255) NOT NULL, PRIMARY KEY (name));

# MS SQL
CREATE TABLE shedlock(name VARCHAR(64) NOT NULL, lock_until datetime2 NOT NULL,
    locked_at datetime2 NOT NULL, locked_by VARCHAR(255) NOT NULL, PRIMARY KEY (name));

# DB2
CREATE TABLE shedlock(name VARCHAR(64) NOT NULL PRIMARY KEY, lock_until TIMESTAMP NOT NULL,
    locked_at TIMESTAMP NOT NULL, locked_by VARCHAR(255) NOT NULL);
```

Or use [this](micronaut/test/micronaut-jdbc/src/main/resources/db/liquibase-changelog.xml) liquibase change-set.

Add dependency

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-jdbc-template</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider;

...
@Bean
public LockProvider lockProvider(DataSource dataSource) {
            return new JdbcTemplateLockProvider(
                JdbcTemplateLockProvider.Configuration.builder()
                .withJdbcTemplate(new JdbcTemplate(dataSource))
                .usingDbTime() // Works on Postgres, MySQL, MariaDb, MS SQL, Oracle, DB2, HSQL and H2
                .build()
            );
}
```
By specifying `usingDbTime()` the lock provider will use UTC time based on the DB server clock.
If you do not specify this option, clock from the app server will be used (the clocks on app servers may not be
synchronized thus leading to various locking issues).

It's strongly recommended to use `usingDbTime()` option as it uses DB engine specific SQL that prevents INSERT conflicts.
See more details [here](https://stackoverflow.com/a/76774461/277042).

For more fine-grained configuration use other options of the `Configuration` object

```java
new JdbcTemplateLockProvider(builder()
    .withTableName("shdlck")
    .withColumnNames(new ColumnNames("n", "lck_untl", "lckd_at", "lckd_by"))
    .withJdbcTemplate(new JdbcTemplate(getDatasource()))
    .withLockedByValue("my-value")
    .withDbUpperCase(true)
    .build())
```

If you need to specify a schema, you can set it in the table name using the usual dot notation
`new JdbcTemplateLockProvider(datasource, "my_schema.shedlock")`

To use a database with case-sensitive table and column names, the `.withDbUpperCase(true)` flag can be used.
Default is `false` (lowercase).

#### Warning
**Do not manually delete lock row from the DB table.** ShedLock has an in-memory cache of existing lock rows
so the row will NOT be automatically recreated until application restart. If you need to, you can edit the row/document, risking only
that multiple locks will be held.

#### R2DBC
If you are really brave, you can try experimental R2DBC support. Please keep in mind that the
capabilities of this lock provider are really limited and that the whole ecosystem around R2DBC
is in flux and may easily break.

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-r2dbc</artifactId>
    <version>5.13.0</version>
</dependency>
```

and use it.

```java
@Override
protected LockProvider getLockProvider() {
    return new R2dbcLockProvider(connectionFactory);
}
```
I recommend using [R2DBC connection pool](https://github.com/r2dbc/r2dbc-pool).

#### jOOQ lock provider
First, create lock table as described in the [JdbcTemplate](#jdbctemplate) section above.

Add dependency

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-jooq</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.jooq;

...
@Bean
public LockProvider getLockProvider(DSLContext dslContext) {
    return new JooqLockProvider(dslContext);
}
```

jOOQ provider has a bit different transactional behavior. While the other JDBC lock providers
create new transaction (with REQUIRES_NEW), jOOQ [does not support setting it](https://github.com/jOOQ/jOOQ/issues/4836).
ShedLock tries to create a new transaction, but depending on your set-up, ShedLock DB operations may
end-up being part of the enclosing transaction.

If you need to configure the table name, schema or column names, you can use jOOQ render mapping as
described [here](https://github.com/lukas-krecan/ShedLock/issues/1830#issuecomment-2015820509).

#### Micronaut Data Jdbc
If you are using Micronaut data and you do not want to add dependency on Spring JDBC, you can use
Micronaut JDBC support. Just be aware that it has just a basic functionality when compared to
the JdbcTemplate provider.

First, create lock table as described in the [JdbcTemplate](#jdbctemplate) section above.

Add dependency

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-jdbc-micronaut</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.jdbc.micronaut.MicronautJdbcLockProvider;

...
@Singleton
public LockProvider lockProvider(TransactionOperations<Connection> transactionManager) {
    return new MicronautJdbcLockProvider(transactionManager);
}
```

#### Mongo
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-mongo</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.mongo.MongoLockProvider;

...

@Bean
public LockProvider lockProvider(MongoClient mongo) {
    return new MongoLockProvider(mongo.getDatabase(databaseName))
}
```

Please note that MongoDB integration requires Mongo >= 2.4 and mongo-java-driver >= 3.7.0

#### Reactive Mongo
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-mongo-reactivestreams</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.mongo.reactivestreams.ReactiveStreamsMongoLockProvider;

...

@Bean
public LockProvider lockProvider(MongoClient mongo) {
    return new ReactiveStreamsMongoLockProvider(mongo.getDatabase(databaseName))
}
```

Please note that MongoDB integration requires Mongo >= 4.x and mongodb-driver-reactivestreams 1.x

#### DynamoDB 2
Depends on AWS SDK v2.

Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-dynamodb2</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.dynamodb2.DynamoDBLockProvider;

...

@Bean
public LockProvider lockProvider(software.amazon.awssdk.services.dynamodb.DynamoDbClient dynamoDB) {
    return new DynamoDBLockProvider(dynamoDB, "Shedlock");
}
```

> Please note that the lock table must be created externally with `_id` as a partition key.
> `DynamoDBUtils#createLockTable` may be used for creating it programmatically.
> A table definition is available from `DynamoDBLockProvider`'s Javadoc.

#### ZooKeeper (using Curator)
Import
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-zookeeper-curator</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure

```java
import net.javacrumbs.shedlock.provider.zookeeper.curator.ZookeeperCuratorLockProvider;

...

@Bean
public LockProvider lockProvider(org.apache.curator.framework.CuratorFramework client) {
    return new ZookeeperCuratorLockProvider(client);
}
```
By default, nodes for locks will be created under `/shedlock` node.

#### Redis (using Spring RedisConnectionFactory)
Import
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-redis-spring</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure

```java
import net.javacrumbs.shedlock.provider.redis.spring.RedisLockProvider;
import org.springframework.data.redis.connection.RedisConnectionFactory;

...

@Bean
public LockProvider lockProvider(RedisConnectionFactory connectionFactory) {
    return new RedisLockProvider(connectionFactory, ENV);
}
```

#### Redis (using Spring ReactiveRedisConnectionFactory)
Import
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-redis-spring</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure

```java
import net.javacrumbs.shedlock.provider.redis.spring.ReactiveRedisLockProvider;
import org.springframework.data.redis.connection.ReactiveRedisConnectionFactory;

...

@Bean
public LockProvider lockProvider(ReactiveRedisConnectionFactory connectionFactory) {
    return new ReactiveRedisLockProvider.Builder(connectionFactory)
        .environment(ENV)
        .build();
}
```

Redis lock provider uses classical lock mechanism as described [here](https://redis.io/commands/setnx#design-pattern-locking-with-codesetnxcode)
which may not be reliable in case of Redis master failure.

#### Redis (using Jedis)
Import
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-redis-jedis4</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure

```java
import net.javacrumbs.shedlock.provider.redis.jedis.JedisLockProvider;

...

@Bean
public LockProvider lockProvider(JedisPool jedisPool) {
    return new JedisLockProvider(jedisPool, ENV);
}
```

#### Hazelcast
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-hazelcast4</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.hazelcast4.HazelcastLockProvider;

...

@Bean
public HazelcastLockProvider lockProvider(HazelcastInstance hazelcastInstance) {
    return new HazelcastLockProvider(hazelcastInstance);
}
```

#### Couchbase
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-couchbase-javaclient3</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.couchbase.javaclient.CouchbaseLockProvider;

...

@Bean
public CouchbaseLockProvider lockProvider(Bucket bucket) {
    return new CouchbaseLockProvider(bucket);
}
```

For Couchbase 3 use `shedlock-provider-couchbase-javaclient3` module and `net.javacrumbs.shedlock.provider.couchbase3` package.

#### Elasticsearch
I am really not sure it's a good idea to use Elasticsearch as a lock provider. But if you have no other choice, you can. Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-elasticsearch8</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import static net.javacrumbs.shedlock.provider.elasticsearch8.ElasticsearchLockProvider;

...

@Bean
public ElasticsearchLockProvider lockProvider(ElasticsearchClient client) {
    return new ElasticsearchLockProvider(client);
}
```

#### OpenSearch
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-opensearch</artifactId>
    <version>4.36.1</version>
</dependency>
```

Configure:

```java
import static net.javacrumbs.shedlock.provider.opensearch.OpenSearchLockProvider;

...

@Bean
public OpenSearchLockProvider lockProvider(RestHighLevelClient highLevelClient) {
    return new OpenSearchLockProvider(highLevelClient);
}
```

#### CosmosDB
CosmosDB support is provided by a third-party module available [here](https://github.com/jesty/shedlock-provider-cosmosdb)

#### Cassandra
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-cassandra</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.cassandra.CassandraLockProvider;
import net.javacrumbs.shedlock.provider.cassandra.CassandraLockProvider.Configuration;

...

@Bean
public CassandraLockProvider lockProvider(CqlSession cqlSession) {
    return new CassandraLockProvider(Configuration.builder().withCqlSession(cqlSession).withTableName("lock").build());
}
```

Example for creating default keyspace and table in local Cassandra instance:
```sql
CREATE KEYSPACE shedlock with replication={'class':'SimpleStrategy', 'replication_factor':1} and durable_writes=true;
CREATE TABLE shedlock.lock (name text PRIMARY KEY, lockUntil timestamp, lockedAt timestamp, lockedBy text);
```

Please, note that CassandraLockProvider uses Cassandra driver v4, which is part of Spring Boot since 2.3.

#### Consul
ConsulLockProvider has one limitation: lockAtMostFor setting will have a minimum value of 10 seconds. It is dictated by consul's session limitations.

Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-consul</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.consul.ConsulLockProvider;

...

@Bean // for micronaut please define preDestroy property @Bean(preDestroy="close")
public ConsulLockProvider lockProvider(com.ecwid.consul.v1.ConsulClient consulClient) {
    return new ConsulLockProvider(consulClient);
}
```

Please, note that Consul lock provider uses [ecwid consul-api client](https://github.com/Ecwid/consul-api), which is part of spring cloud consul integration (the `spring-cloud-starter-consul-discovery` package).

#### ArangoDB
Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-arangodb</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.arangodb.ArangoLockProvider;

...

@Bean
public ArangoLockProvider lockProvider(final ArangoOperations arangoTemplate) {
    return new ArangoLockProvider(arangoTemplate.driver().db(DB_NAME));
}
```

Please, note that ArangoDB lock provider uses ArangoDB driver v6.7, which is part of [arango-spring-data](https://github.com/arangodb/spring-data) in version 3.3.0.

#### Neo4j
Import the project

```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-neo4j</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:
```java
import net.javacrumbs.shedlock.core.LockConfiguration;

...

@Bean
Neo4jLockProvider lockProvider(org.neo4j.driver.Driver driver) {
    return new Neo4jLockProvider(driver);
}
```

Please make sure that ```neo4j-java-driver``` version used by ```shedlock-provider-neo4j``` matches the driver version used in your
project (if you use `spring-boot-starter-data-neo4j`, it is probably provided transitively).

#### Etcd
Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-etcd-jetcd</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.etcd.jetcd.EtcdLockProvider;

...

@Bean
public LockProvider lockProvider(Client client) {
    return new EtcdLockProvider(client);
}
```

#### Apache Ignite
Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-ignite</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure:

```java
import net.javacrumbs.shedlock.provider.ignite.IgniteLockProvider;

...

@Bean
public LockProvider lockProvider(Ignite ignite) {
    return new IgniteLockProvider(ignite);
}
```

#### In-Memory
If you want to use a lock provider in tests there is an in-Memory implementation.

Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-inmemory</artifactId>
    <version>5.13.0</version>
    <scope>test</scope>
</dependency>
```

```java
import net.javacrumbs.shedlock.provider.inmemory.InMemoryLockProvider;

...

@Bean
public LockProvider lockProvider() {
    return new InMemoryLockProvider();
}
```

#### Memcached (using spymemcached)
Please, be aware that memcached is not a database but a cache. It means that if the cache is full,
[the lock may be released prematurely](https://stackoverflow.com/questions/6868256/memcached-eviction-prior-to-key-expiry/10456364#10456364)
**Use only if you know what you are doing.**

Import
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-memcached-spy</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure

```java
import net.javacrumbs.shedlock.provider.memcached.spy.MemcachedLockProvider;

...

@Bean
public LockProvider lockProvider(net.spy.memcached.MemcachedClient client) {
    return new MemcachedLockProvider(client, ENV);
}
```

P.S.:

Memcached Standard Protocol:
- A key (arbitrary string up to 250 bytes in length. No space or newlines for ASCII mode)
- An expiration time, in `seconds`. '0' means never expire. Can be up to 30 days. After 30 days, is treated as a unix timestamp of an exact date. (support `seconds`、`minutes`、`days`, and less than `30` days)

#### Datastore

Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-datastore</artifactId>
    <version>5.13.0</version>
</dependency>
```

and configure
```java
import net.javacrumbs.shedlock.provider.datastore.DatastoreLockProvider;

...

@Bean
public LockProvider lockProvider(com.google.cloud.datastore.Datastore datastore) {
    return new DatastoreLockProvider(datastore);
}

```
#### Spanner
Import the project
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <artifactId>shedlock-provider-spanner</artifactId>
    <version>5.13.0</version>
</dependency>
```
Configure
```java
import net.javacrumbs.shedlock.provider.spanner.SpannerLockProvider;

...

// Basic
@Bean
public LockProvider lockProvider(DatabaseClient databaseClient) {
    return new SpannerLockProvider(databaseClientSupplier);
}

// Custom host, table and column names
@Bean
public LockProvider lockProvider(DatabaseClient databaseClient) {
    var config = SpannerLockProvider.Configuration.builder()
        .withDatabaseClient(databaseClientSupplier)
        .withTableConfiguration(SpannerLockProvider.TableConfiguration.builder()
            ...
            // Custom table and column names
            .build())
        .withHostName("customHostName")
        .build();

    return new SpannerLockProvider(config);
}
```

## Multi-tenancy
If you have multi-tenancy use-case you can use a lock provider similar to this one
(see the full [example](https://github.com/lukas-krecan/ShedLock/blob/master/providers/jdbc/shedlock-provider-jdbc-template/src/test/java/net/javacrumbs/shedlock/provider/jdbctemplate/MultiTenancyLockProviderIntegrationTest.java#L87))
```java
private static abstract class MultiTenancyLockProvider implements LockProvider {
    private final ConcurrentHashMap<String, LockProvider> providers = new ConcurrentHashMap<>();

    @Override
    public @NonNull Optional<SimpleLock> lock(@NonNull LockConfiguration lockConfiguration) {
        String tenantName = getTenantName(lockConfiguration);
        return providers.computeIfAbsent(tenantName, this::createLockProvider).lock(lockConfiguration);
    }

    protected abstract LockProvider createLockProvider(String tenantName) ;

    protected abstract String getTenantName(LockConfiguration lockConfiguration);
}
```

## Customization
You can customize the behavior of the library by implementing `LockProvider` interface. Let's say you want to implement
a special behavior after a lock is obtained. You can do it like this:

```java
public class MyLockProvider implements LockProvider {
    private final LockProvider delegate;

    public MyLockProvider(LockProvider delegate) {
        this.delegate = delegate;
    }

    @Override
    public Optional<SimpleLock> lock(LockConfiguration lockConfiguration) {
        Optional<SimpleLock> lock = delegate.lock(lockConfiguration);
        if (lock.isPresent()) {
            // do something
        }
        return lock;
    }
}
```

## Duration specification
All the annotations where you need to specify a duration support the following formats

* duration+unit - `1s`, `5ms`, `5m`, `1d` (Since 4.0.0)
* duration in ms - `100` (only Spring integration)
* ISO-8601 - `PT15M` (see [Duration.parse()](https://docs.oracle.com/javase/8/docs/api/java/time/Duration.html#parse-java.lang.CharSequence-) documentation)

## Extending the lock
There are some use-cases which require to extend currently held lock. You can use LockExtender in the
following way:

```java
LockExtender.extendActiveLock(Duration.ofMinutes(5), ZERO);
```

Please note that not all lock provider implementations support lock extension.

## KeepAliveLockProvider
There is also KeepAliveLockProvider that is able to keep the lock alive by periodically extending it. It can be
used by wrapping the original lock provider. My personal opinion is that it should be used only in special cases,
it adds more complexity to the library and the flow is harder to reason about so please use moderately.

```java
@Bean
public LockProvider lockProvider(...) {
    return new  KeepAliveLockProvider(new XyzProvider(...), scheduler);
}
```
KeepAliveLockProvider extends the lock in the middle of the lockAtMostFor interval. For example, if the lockAtMostFor
is 10 minutes the lock is extended every 5 minutes for 10 minutes until the lock is released. Please note that the minimal
lockAtMostFor time supported by this provider is 30s. The scheduler is used only for the lock extension, single thread
should be enough.

## Micronaut integration
Since version 4.0.0, it's possible to use Micronaut framework for integration

Import the project:
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <!-- Micronaut 3 -->
    <artifactId>shedlock-micronaut</artifactId>
    <!-- For Micronaut 4 use -->
    <!-- <artifactId>shedlock-micronaut4</artifactId> -->
    <version>5.13.0</version>
</dependency>
```

Configure default lockAtMostFor value (application.yml):
```yaml
shedlock:
  defaults:
    lock-at-most-for: 1m
```

Configure lock provider:
```java
@Singleton
public LockProvider lockProvider() {
    ... select and configure your lock provider
}
```

Configure the scheduled task:
```java
@Scheduled(fixedDelay = "1s")
@SchedulerLock(name = "myTask")
public void myTask() {
    assertLocked();
    ...
}
```

## CDI integration
Since version 5.0.0, it's possible to use CDI for integration (tested only with Quarkus)

Import the project:
```xml
<dependency>
    <groupId>net.javacrumbs.shedlock</groupId>
    <!-- use shedlock-cdi-vintage for quarkus 2.x -->
    <artifactId>shedlock-cdi</artifactId>
    <version>5.13.0</version>
</dependency>
```

Configure default lockAtMostFor value (application.properties):
```properties
shedlock.defaults.lock-at-most-for=PT30S
```

Configure lock provider:
```java
@Produces
@Singleton
public LockProvider lockProvider() {
        ...
}
```

Configure the scheduled task:
```java
@Scheduled(every = "1s")
@SchedulerLock(name = "myTask")
public void myTask() {
    assertLocked();
    ...
}
```

The implementation only depends on `jakarta.enterprise.cdi-api` and `microprofile-config-api` so it should be
usable in other CDI compatible frameworks, but it has not been tested with anything else than Quarkus. It's
built on top of javax annotation as Quarkus has not moved to Jakarta EE namespace yet.

The support is minimalistic, for example there is no support for expressions in the annotation parameters yet,
if you need it, feel free to send a PR.

## Locking without a framework
It is possible to use ShedLock without a framework

```java
LockingTaskExecutor executor = new DefaultLockingTaskExecutor(lockProvider);

...

Instant lockAtMostUntil = Instant.now().plusSeconds(600);
executor.executeWithLock(runnable, new LockConfiguration("lockName", lockAtMostUntil));

```

## Extending the lock
Some lock providers support extension of the lock. For the time being, it requires manual lock manipulation,
directly using `LockProvider` and calling `extend` method on the `SimpleLock`.

## Modes of Spring integration
ShedLock supports two modes of Spring integration. One that uses an AOP proxy around scheduled method (PROXY_METHOD)
and one that proxies TaskScheduler (PROXY_SCHEDULER)

#### Scheduled Method proxy
Since version 4.0.0, the default mode of Spring integration is an AOP proxy around the annotated method.

The main advantage of this mode is that it plays well with other frameworks that want to somehow alter the default Spring scheduling mechanism.
The disadvantage is that the lock is applied even if you call the method directly. If the method returns a value and the lock is held
by another process, null or an empty Optional will be returned (primitive return types are not supported).

Final and non-public methods are not proxied so either you have to make your scheduled methods public and non-final or use TaskScheduler proxy.

![Method proxy sequenceDiagram](https://github.com/lukas-krecan/ShedLock/raw/master/documentation/method_proxy.png)

#### TaskScheduler proxy
This mode wraps Spring `TaskScheduler` in an AOP proxy. **This mode does not play well with instrumentation libraries**
like opentelementry that also wrap TaskScheduler. Please only use it if you know what you are doing.
It can be switched-on like this (PROXY_SCHEDULER was the default method before 4.0.0):

```java
@EnableSchedulerLock(interceptMode = PROXY_SCHEDULER)
```

If you do not specify your task scheduler, a default one is created for you. If you have special needs, just create a bean implementing `TaskScheduler`
interface and it will get wrapped into the AOP proxy automatically.

```java
@Bean
public TaskScheduler taskScheduler() {
    return new MySpecialTaskScheduler();
}
```

Alternatively, you can define a bean of type `ScheduledExecutorService` and it will automatically get used by the tasks
scheduling mechanism.

![TaskScheduler proxy sequence diagram](https://github.com/lukas-krecan/ShedLock/raw/master/documentation/scheduler_proxy.png)

### Spring XML configuration
Spring XML configuration is not supported as of version 3.0.0. If you need it, please use version 2.6.0 or file an issue explaining why it is needed.

## Lock assert
To prevent misconfiguration errors, like AOP misconfiguration, missing annotation etc., you can assert that the lock
works by using LockAssert:

```java
@Scheduled(...)
@SchedulerLock(..)
public void scheduledTask() {
    // To assert that the lock is held (prevents misconfiguration errors)
    LockAssert.assertLocked();
    // do something
}
```

In unit tests you can switch-off the assertion by calling `LockAssert.TestHelper.makeAllAssertsPass(true)` on given thread (as in this [example](https://github.com/lukas-krecan/ShedLock/commit/e8d63b7c56644c4189e0a8b420d8581d6eae1443)).

## Kotlin gotchas
The library is tested with Kotlin and works fine. The only issue is Spring AOP which does not work on final method. If you use `@SchedulerLock` with `@Component`
annotation, everything should work since Kotlin Spring compiler plugin will automatically 'open' the method for you. If `@Component` annotation is not present, you
have to open the method by yourself. (see [this issue](https://github.com/lukas-krecan/ShedLock/issues/1268) for more details)

## Caveats
Locks in ShedLock have an expiration time which leads to the following possible issues.
1. If the task runs longer than `lockAtMostFor`, the task can be executed more than once
2. If the clock difference between two nodes is more than `lockAtLeastFor` or minimal execution time the task can be
executed more than once.

## Troubleshooting
Help, ShedLock does not do what it's supposed to do!

1. Upgrade to the newest version
2. Use [LockAssert](https://github.com/lukas-krecan/ShedLock#lock-assert) to ensure that AOP is correctly configured.
   - If it does not work, please read about Spring AOP internals (for example [here](https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#aop-proxying))
3. Check the storage. If you are using JDBC, check the ShedLock table. If it's empty, ShedLock is not properly configured.
If there is more than one record with the same name, you are missing a primary key.
4. Use ShedLock debug log. ShedLock logs interesting information on DEBUG level with logger name `net.javacrumbs.shedlock`.
It should help you to see what's going on.
5. For short-running tasks consider using `lockAtLeastFor`. If the tasks are short-running, they could be executed one
after another, `lockAtLeastFor` can prevent it.

# Release notes
## 5.13.0 (2024-04-05)
* #1779 Ability to rethrow unexpected exception in JdbcTemplateStorageAccessor
* Dependency updates

## 5.12.0 (2024-02-29)
* #1800 Enable lower case for database type when using usingDbTime()
* #1804 Startup error with Neo4j 5.17.0
* Dependency updates

## 4.47.0 (2024-03-01)
* #1800 Enable lower case for database type when using usingDbTime() (thanks @yuagu1)

## 5.11.0 (2024-02-13)
* #1753 Fix SpEL for methods with parameters
* Dependency updates

## 5.10.2 (2023-12-07)
* #1635 fix makeAllAssertsPass locks only once
* Dependency updates

## 5.10.1 (2023-12-06)
* #1635 fix makeAllAssertsPass(false) throws NoSuchElementException
* Dependency updates

## 5.10.0 (2023-11-07)
* SpannerLockProvider added (thanks @pXius)
* Dependency updates

## 5.9.1 (2023-10-19)
* QuarkusRedisLockProvider supports Redis 6.2 (thanks  @ricardojlrufino)

## 5.9.0 (2023-10-15)
* Support Quarkus 2 Redis client (thanks  @ricardojlrufino)
* Better handling of timeouts in ReactiveStreamsMongoLockProvider
* Dependency updates

## 5.8.0 (2023-09-15)
* Support for Micronaut 4
* Use Merge instead of Insert for Oracle #1528 (thanks @xmojsic)
* Dependency updates

## 5.7.0 (2023-08-25)
* JedisLockProvider supports extending (thanks @shotmk)
* Better behavior when locks are nested #1493

## 4.46.0 (2023-09-05)
* JedisLockProvider (version 3) supports extending (thanks @shotmk)

## 4.45.0 (2023-09-04)
* JedisLockProvider supports extending (thanks @shotmk)

## 5.6.0
* Ability to explicitly set database product in JdbTemplateLockProvider (thanks @metron2)
* Removed forgotten versions from BOM
* Dependency updates

## 5.5.0 (2023-06-19)
* Datastore support (thanks @mmastika)
* Dependency updates

## 5.4.0 (2023-06-06)
* Handle [uncategorized SQL exceptions](https://github.com/lukas-krecan/ShedLock/pull/1442) (thanks @jaam)
* Dependency updates

## 5.3.0 (2023-05-13)
* Added shedlock-cdi module (supports newest CDI version)
* Dependency updates

## 5.2.0 (2023-03-06)
* Uppercase in JdbcTemplateProvider (thanks @Ragin-LundF)
* Dependency updates

## 5.1.0 (2023-01-07)
* Added SpEL support to @SchedulerLock name attribute (thanks @ipalbeniz)
* Dependency updates

## 5.0.1 (2022-12-10)
* Work around broken Spring 6 exception translation https://github.com/lukas-krecan/ShedLock/issues/1272

## 4.44.0 (2022-12-29)
* Insert ignore for MySQL https://github.com/lukas-krecan/ShedLock/commit/8a4ae7ad8103bb47f55d43bccf043ca261c24d7a

## 5.0.0 (2022-12-10)
* Requires JDK 17
* Tested with Spring 6 (Spring Boot 3)
* Micronaut updated to 3.x.x
* R2DBC 1.x.x (still sucks)
* Spring Data 3.x.x
* Rudimentary support for CDI (tested with quarkus)
* New jOOQ lock provider
* SLF4j 2
* Deleted all deprecated code and support for old versions of libraries

## 4.43.0 (2022-12-04)
* Better logging in JdbcTemplateProvider
* Dependency updates

## 4.42.0 (2022-09-16)
* Deprecate old Couchbase lock provider
* Dependency updates

## 4.41.0 (2022-08-17)
* Couchbase collection support (thanks @mesuutt)
* Dependency updates

## 4.40.0 (2022-08-11)
* Fixed caching issues when the app is started by the DB does not exist yet (#1129)
* Dependency updates

## 4.39.0 (2022-07-26)
* Introduced elasticsearch8 LockProvider and deperecated the orignal one (thanks @MarAra)
* Dependency updates

## 4.38.0 (2022-07-02)
* ReactiveRedisLockProvider added (thanks @ericwcc)
* Dependency updates

## 4.37.0 (2022-06-14)
* OpenSearch provider (thanks @Pinny3)
* Fix wrong reference to reactive Mongo in BOM #1048
* Dependency updates

## 4.36.0 (2022-05-28)
* shedlock-bom module added
* Dependency updates

## 4.35.0 (2022-05-16)
* Neo4j allows to specify database thanks @SergeyPlatonov
* Dependency updates

## 4.34.0 (2022-04-09)
* Dropped support for Hazelcast <= 3 as it has unfixed vulnerability
* Dropped support for Spring Data Redis 1 as it is not supported
* Dependency updates

## 4.33.0
* memcached provider added (thanks @pinkhello)
* Dependency updates

## 4.32.0
* JDBC provider does not change autocommit attribute
* Dependency updates

## 4.31.0
* Jedis 4 lock provider
* Dependency updates

## 4.30.0
* In-memory lock provider added (thanks @kkocel)
* Dependency updates

## 4.29.0
* R2DBC support added (thanks @sokomishalov)
* Library upgrades

## 4.28.0
* Neo4j lock provider added (thanks @thimmwork)
* Library upgrades

## 4.27.0
* Ability to set transaction isolation in JdbcTemplateLockProvider
* Library upgrades

## 4.26.0
* KeepAliveLockProvider introduced
* Library upgrades

## 4.25.0
* LockExtender added

## 4.24.0
* Support for Apache Ignite (thanks @wirtsleg)
* Library upgrades

## 4.23.0
* Ability to set serialConsistencyLevel in Cassandra (thanks @DebajitKumarPhukan)
* Introduced shedlock-provider-jdbc-micronaut module (thanks @drmaas)

## 4.22.1
* Catching and logging Cassandra exception

## 4.22.0
* Support for custom keyspace in Cassandra provider

## 4.21.0
* Elastic unlock using IMMEDIATE refresh policy #422
* DB2 JDBC lock provider uses microseconds in DB time
* Various library upgrades

## 4.20.1
* Fixed DB JDBC server time #378

## 4.20.0
* Support for etcd (thanks grofoli)

## 4.19.1
* Fixed devtools compatibility #368

## 4.19.0
* Support for enhanced configuration in Cassandra provider (thanks DebajitKumarPhukan)
* LockConfigurationExtractor exposed as a Spring bean #359
* Handle CannotSerializeTransactionException #364

## 4.18.0
* Fixed Consul support for tokens and added enhanced Consul configuration (thanks DrWifey)

## 4.17.0
* Consul support for tokens

## 4.16.0
* Spring - EnableSchedulerLock.order param added to specify AOP proxy order
* JDBC - Log unexpected exceptions at ERROR level
* Hazelcast upgraded to 4.1

## 4.15.1
* Fix session leak in Consul provider #340 (thanks @haraldpusch)

## 4.15.0
* ArangoDB lock provider added (thanks @patrick-birkle)

## 4.14.0
* Support for Couchbase 3 driver (thanks @blitzenzzz)
* Removed forgotten configuration files form micronaut package (thanks @drmaas)
* Shutdown hook for Consul (thanks @kaliy)

## 4.13.0
* Support for Consul (thanks @kaliy)
* Various dependencies updated
* Deprecated default LockConfiguration constructor

## 4.12.0
* Lazy initialization of SqlStatementsSource #258

## 4.11.1
* MongoLockProvider uses mongodb-driver-sync
* Removed deprecated constructors from MongoLockProvider

## 4.10.1
* New Mongo reactive streams driver (thanks @codependent)

## 4.9.3
* Fixed JdbcTemplateLockProvider useDbTime() locking #244 thanks @gjorgievskivlatko

## 4.9.2
* Do not fail on DB type determining code if DB connection is not available

## 4.9.1
* Support for server time in DB2
* removed shedlock-provider-jdbc-internal module

## 4.9.0
* Support for server time in JdbcTemplateLockProvider
* Using custom non-null annotations
* Trimming time precision to milliseconds
* Micronaut upgraded to 1.3.4
* Add automatic DB tests for Oracle, MariaDB and MS SQL.

## 4.8.0
* DynamoDB 2 module introduced (thanks Mark Egan)
* JDBC template code refactored to not log error on failed insert in Postgres
    * INSERT .. ON CONFLICT UPDATE is used for Postgres

## 4.7.1
* Make LockAssert.TestHelper public

## 4.7.0
* New module for Hazelcasts 4
* Ability to switch-off LockAssert in unit tests

## 4.6.0
* Support for Meta annotations and annotation inheritance in Spring

## 4.5.2
* Made compatible with PostgreSQL JDBC Driver 42.2.11

## 4.5.1
* Inject redis template

## 4.5.0
* ClockProvider introduced
* MongoLockProvider(MongoDatabase) introduced

## 4.4.0
* Support for non-void returning methods when PROXY_METHOD interception is used

## 4.3.1
* Introduced shedlock-provider-redis-spring-1 to make it work around Spring Data Redis 1 issue #105 (thanks @rygh4775)

## 4.3.0
* Jedis dependency upgraded to 3.2.0
* Support for JedisCluster
* Tests upgraded to JUnit 5

## 4.2.0
* Cassandra provider (thanks @mitjag)

## 4.1.0
* More configuration option for JdbcTemplateProvider

## 4.0.4
* Allow configuration of key prefix in RedisLockProvider #181 (thanks @krm1312)

## 4.0.3
* Fixed junit dependency scope #179

## 4.0.2
* Fix NPE caused by Redisson #178
## 4.0.1
* DefaultLockingTaskExecutor made reentrant #175
## 4.0.0
Version 4.0.0 is a major release changing quite a lot of stuff
* `net.javacrumbs.shedlock.core.SchedulerLock` has been replaced by `net.javacrumbs.shedlock.spring.annotation.SchedulerLock`. The original annotation has been in wrong module and
was too complex. Please use the new annotation, the old one still works, but in few years it will be removed.
* Default intercept mode changed from `PROXY_SCHEDULER` to `PROXY_METHOD`. The reason is that there were a lot of issues with  `PROXY_SCHEDULER` (for example #168). You can still
use `PROXY_SCHEDULER` mode if you specify it manually.
* Support for more readable [duration strings](#duration-specification)
* Support for lock assertion `LockAssert.assertLocked()`
* [Support for Micronaut](#micronaut-integration) added

## 3.0.1
* Fixed bean definition configuration #171

## 3.0.0
* `EnableSchedulerLock.mode` renamed to `interceptMode`
* Use standard Spring AOP configuration to honor Spring Boot config (supports `proxyTargetClass` flag)
* Removed deprecated SpringLockableTaskSchedulerFactoryBean and related classes
* Removed support for XML configuration

## 2.6.0
* Updated dependency to Spring 2.1.9
* Support for lock extensions (beta)

## 2.5.0
* Zookeeper supports *lockAtMostFor* and *lockAtLeastFor* params
* Better debug logging

## 2.4.0
* Fixed potential deadlock in Hazelcast (thanks @HubertTatar)
* Finding class level annotation in proxy method mode (thanks @volkovs)
* ScheduledLockConfigurationBuilder deprecated

## 2.3.0
* LockProvides is initialized lazilly so it does not change DataSource initialization order

## 2.2.1
* MongoLockProvider accepts MongoCollection as a constructor param

## 2.2.0
* DynamoDBLockProvider added

## 2.1.0
* MongoLockProvider rewritten to use upsert
* ElasticsearchLockProvider added

## 2.0.1
* AOP proxy and annotation configuration support

## 1.3.0
* Can set Timezone to JdbcTemplateLock provider

## 1.2.0
* Support for Couchbase (thanks to @MoranVaisberg)

## 1.1.1
* Spring RedisLockProvider refactored to use RedisTemplate

## 1.1.0
* Support for transaction manager in JdbcTemplateLockProvider (thanks to @grmblfrz)

## 1.0.0
* Upgraded dependencies to Spring 5 and Spring Data 2
* Removed deprecated net.javacrumbs.shedlock.provider.jedis.JedisLockProvider (use  net.javacrumbs.shedlock.provider.redis.jedis.JedisLockProvide instead)
* Removed deprecated SpringLockableTaskSchedulerFactory (use ScheduledLockConfigurationBuilder instead)

## 0.18.2
* ablility to clean lock cache

## 0.18.1
* shedlock-provider-redis-spring made compatible with spring-data-redis 1.x.x

## 0.18.0
* Added shedlock-provider-redis-spring (thanks to @siposr)
* shedlock-provider-jedis moved to shedlock-provider-redis-jedis

## 0.17.0
* Support for SPEL in lock name annotation

## 0.16.1
* Automatically closing TaskExecutor on Spring shutdown

## 0.16.0
* Removed spring-test from shedlock-spring compile time dependencies
* Added Automatic-Module-Names

## 0.15.1
* Hazelcast works with remote cluster

## 0.15.0
* Fixed ScheduledLockConfigurationBuilder interfaces #32
* Hazelcast code refactoring

## 0.14.0
* Support for Hazelcast (thanks to @peyo)

## 0.13.0
* Jedis constructor made more generic (thanks to @mgrzeszczak)

## 0.12.0
* Support for property placeholders in annotation lockAtMostForString/lockAtLeastForString
* Support for composed annotations
* ScheduledLockConfigurationBuilder introduced (deprecating SpringLockableTaskSchedulerFactory)

## 0.11.0
* Support for Redis (thanks to @clamey)
* Checking that lockAtMostFor is in the future
* Checking that lockAtMostFor is larger than lockAtLeastFor

## 0.10.0
* jdbc-template-provider does not participate in task transaction

## 0.9.0
* Support for @SchedulerLock annotations on proxied classes

## 0.8.0
* LockableTaskScheduler made AutoClosable so it's closed upon Spring shutdown

## 0.7.0
* Support for lockAtLeastFor

## 0.6.0
* Possible to configure defaultLockFor time so it does not have to be repeated in every annotation

## 0.5.0
* ZooKeeper nodes created under /shedlock by default

## 0.4.1
* JdbcLockProvider insert does not fail on DataIntegrityViolationException

## 0.4.0
* Extracted LockingTaskExecutor
* LockManager.executeIfNotLocked renamed to executeWithLock
* Default table name in JDBC lock providers

## 0.3.0
* `@ShedlulerLock.name` made obligatory
* `@ShedlulerLock.lockForMillis` renamed to lockAtMostFor
* Adding plain JDBC LockProvider
* Adding ZooKeepr LockProvider


<!--
Copyright Pravega Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Pravega [![Build Status](https://github.com/pravega/pravega/workflows/build/badge.svg?branch=master)](https://github.com/pravega/pravega/actions?query=branch%3Amaster) [![codecov](https://codecov.io/gh/pravega/pravega/branch/master/graph/badge.svg?token=6xOvaR0sIa)](https://codecov.io/gh/pravega/pravega) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0) [![Version](https://img.shields.io/github/release/pravega/pravega.svg)](https://github.com/pravega/pravega/releases) [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4178/badge)](https://bestpractices.coreinfrastructure.org/projects/4178)

Pravega is an open source distributed storage service implementing **Streams**. It offers Stream as the main primitive for the foundation of reliable storage systems: a *high-performance, durable, elastic, and unlimited append-only byte stream with strict ordering and consistency*.

To learn more about Pravega, visit https://pravega.io

## Prerequisites

- Java 11+

 In spite of the requirements of using JDK 11+ to build this project, `client` artifacts (and its dependencies) must be compatible with a *Java 8* runtime. All other components are built and ran using JDK11+.

The `clientJavaVersion` project property determines the version used to build the client (defaults to *8*).
## Building Pravega

Checkout the source code:

```
git clone https://github.com/pravega/pravega.git
cd pravega
```

Build the pravega distribution:

```
./gradlew distribution
```

Install pravega jar files into the local maven repository. This is handy for running the `pravega-samples` locally against a custom version of pravega.

```
./gradlew install
```

Running unit tests:

```
./gradlew test
```

## Setting up your IDE

Pravega uses [Project Lombok](https://projectlombok.org/) so you should ensure you have your IDE setup with the required plugins. Using IntelliJ is recommended.

To import the source into IntelliJ:

1. Import the project directory into IntelliJ IDE. It will automatically detect the gradle project and import things correctly.
2. Enable `Annotation Processing` by going to `Build, Execution, Deployment` -> `Compiler` > `Annotation Processors` and checking 'Enable annotation processing'.
3. Install the `Lombok Plugin`. This can be found in `Preferences` -> `Plugins`. Restart your IDE.
4. Pravega should now compile properly.

For eclipse, you can generate eclipse project files by running `./gradlew eclipse`.

<sup> Note: Some unit tests will create (and delete) a significant amount of files. For improved performance on Windows machines, be sure to add the appropriate 'Microsoft Defender' exclusion.</sup>

## Releases

The latest pravega releases can be found on the [Github Release](https://github.com/pravega/pravega/releases) project page.

## Snapshot artifacts

All snapshot artifacts from `master` and `release` branches are available in GitHub Packages Registry

Add the following to your repositories list and import dependencies as usual.

```
maven {
    url "https://maven.pkg.github.com/pravega/pravega"
    credentials {
        username = "pravega-public"
        password = "\u0067\u0068\u0070\u005F\u0048\u0034\u0046\u0079\u0047\u005A\u0031\u006B\u0056\u0030\u0051\u0070\u006B\u0079\u0058\u006D\u0035\u0063\u0034\u0055\u0033\u006E\u0032\u0065\u0078\u0039\u0032\u0046\u006E\u0071\u0033\u0053\u0046\u0076\u005A\u0049"
    }
}
```
Note GitHub Packages requires authentication to download packages thus credentials above are required. Use the provided password as is, please do not decode it.

If you need a dedicated token to use in your repository (and GitHub Actions) please reach out to us.

As alternative option you can use JitPack (https://jitpack.io/#pravega/pravega) to get pre-release artifacts.

## Quick Start

Read [Getting Started](documentation/src/docs/getting-started/quick-start.md) page for more information, and also visit [sample-apps](https://github.com/pravega/pravega-samples) repo for more applications.

## Running Pravega

Pravega can be installed locally or in a distributed environment. The installation and deployment of pravega is covered in the [Running Pravega](documentation/src/docs/deployment/deployment.md) guide.

## Support

Don’t hesitate to ask! Contact the developers and community on [slack](https://pravega-io.slack.com/)
([signup](https://join.slack.com/t/pravega-io/shared_invite/zt-245sgpw47-vbLBLiLfBdW9TlKemXkUnw)) if you need any help.
Open an issue if you found a bug on [Github Issues](https://github.com/pravega/pravega/issues).

## Documentation

The Pravega documentation is hosted on the website:
<https://pravega.io/docs/latest> or in the [documentation](documentation/src/docs) directory of the source code.

## Contributing

Become one of the contributors! We thrive to build a welcoming and open
community for anyone who wants to use the system or contribute to it.
[Here](documentation/src/docs/contributing.md) we describe how to contribute to Pravega!
You can see the roadmap document [here](documentation/src/docs/roadmap.md).

## About

Pravega is 100% open source and community-driven. All components are available
under [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0.html) on
GitHub.


# Reactor Netty

[![Join the chat at https://gitter.im/reactor/reactor-netty](https://img.shields.io/gitter/room/nwjs/nw.js.svg)](https://gitter.im/reactor/reactor-netty)

[![Reactor Netty](https://img.shields.io/maven-central/v/io.projectreactor.netty/reactor-netty.svg?colorB=brightgreen)](https://mvnrepository.com/artifact/io.projectreactor.netty/reactor-netty)

[![publish](https://github.com/reactor/reactor-netty/actions/workflows/publish.yml/badge.svg)](https://github.com/reactor/reactor-netty/actions/workflows/publish.yml) [![CodeQL](https://github.com/reactor/reactor-netty/workflows/CodeQL/badge.svg?event=push)](https://github.com/reactor/reactor-netty/actions?query=workflow%3ACodeQL)

`Reactor Netty` offers non-blocking and backpressure-ready `TCP`/`HTTP`/`UDP`/`QUIC`
clients & servers based on `Netty` framework.

## Getting it
`Reactor Netty` requires Java 8 or + to run.

With `Gradle` from [repo.spring.io](https://repo.spring.io) or `Maven Central` repositories (stable releases only):

```groovy
    repositories {
      //maven { url 'https://repo.spring.io/snapshot' }
      maven { url 'https://repo.spring.io/milestone' }
      mavenCentral()
    }

    dependencies {
      //compile "io.projectreactor.netty:reactor-netty-core:1.2.0-SNAPSHOT"
      compile "io.projectreactor.netty:reactor-netty-core:1.2.0-M3"
      //compile "io.projectreactor.netty:reactor-netty-http:1.2.0-SNAPSHOT"
      compile "io.projectreactor.netty:reactor-netty-http:1.2.0-M3"
    }
```

See the [Reference documentation](https://projectreactor.io/docs/netty/release/reference/index.html#getting)
for more information on getting it (eg. using `Maven`, or on how to get milestones and snapshots).

## Getting Started
New to `Reactor Netty`? Check this [Reactor Netty Workshop](https://violetagg.github.io/reactor-netty-workshop/)
and the [Reference documentation](https://projectreactor.io/docs/netty/release/reference/index.html)

Here is a very simple `HTTP` server and the corresponding `HTTP` client example

```java
HttpServer.create()   // Prepares an HTTP server ready for configuration
          .port(0)    // Configures the port number as zero, this will let the system pick up
                      // an ephemeral port when binding the server
          .route(routes ->
                      // The server will respond only on POST requests
                      // where the path starts with /test and then there is path parameter
                  routes.post("/test/{param}", (request, response) ->
                          response.sendString(request.receive()
                                                     .asString()
                                                     .map(s -> s + ' ' + request.param("param") + '!')
                                                     .log("http-server"))))
          .bindNow(); // Starts the server in a blocking fashion, and waits for it to finish its initialization
```

```java
HttpClient.create()             // Prepares an HTTP client ready for configuration
          .port(server.port())  // Obtains the server's port and provides it as a port to which this
                                // client should connect
          .post()               // Specifies that POST method will be used
          .uri("/test/World")   // Specifies the path
          .send(ByteBufFlux.fromString(Flux.just("Hello")))  // Sends the request body
          .responseContent()    // Receives the response body
          .aggregate()
          .asString()
          .log("http-client")
          .block();

```

## Getting help
Having trouble with `Reactor Netty`? We'd like to help!
* If you are upgrading, read the [release notes](https://github.com/reactor/reactor-netty/releases)
  for upgrade instructions and *new and noteworthy* features.
* Ask a question - we monitor [stackoverflow.com](https://stackoverflow.com) for questions
  tagged with [`reactor-netty`](https://stackoverflow.com/questions/tagged/reactor-netty). You can also chat
  with the community on [Gitter](https://gitter.im/reactor/reactor-netty).
* Report bugs with `Reactor Netty` at [github.com/reactor/reactor-netty/issues](https://github.com/reactor/reactor-netty/issues).
* More about [Support and Deprecation policies](https://github.com/reactor/.github/blob/main/SUPPORT.adoc)

## Reporting Issues
`Reactor Netty` uses `GitHub’s` integrated issue tracking system to record bugs and feature requests.
If you want to raise an issue, please follow the recommendations below:
* Before you log a bug, please [search the issue tracker](https://github.com/reactor/reactor-netty/search?type=Issues)
  to see if someone has already reported the problem.
* If the issue doesn't already exist, [create a new issue](https://github.com/reactor/reactor-netty/issues/new/choose).
* Please provide as much information as possible with the issue report, we like to know
  the version of `Reactor Netty` that you are using, as well as your `Operating System` and
  `JVM` version.
* If you want to raise a security vulnerability, please review our [Security Policy](https://github.com/reactor/reactor-netty/security/policy) for more details.

## Contributing
See our [Contributing Guide](https://github.com/reactor/.github/blob/main/CONTRIBUTING.md) for information about contributing to `Reactor Netty`.

## Building from Source
You don't need to build from source to use `Reactor Netty` (binaries in
[repo.spring.io](https://repo.spring.io)), but if you want to try out the latest and
greatest, `Reactor Netty` can be easily built with the
[gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html). You also need JDK 1.8.

```shell
$ git clone https://github.com/reactor/reactor-netty.git
$ cd reactor-netty
$ ./gradlew build
```

If you want to publish the artifacts to your local `Maven` repository use:

```shell
$ ./gradlew publishToMavenLocal
```

## Javadoc
https://projectreactor.io/docs/netty/release/api/

## Guides

* https://projectreactor.io/docs/netty/release/reference/index.html
* https://violetagg.github.io/reactor-netty-workshop/

## License
`Reactor Netty` is Open Source Software released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)


[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.jooby/jooby/badge.svg)](https://maven-badges.herokuapp.com/maven-central/io.jooby/jooby)
[![Javadoc](https://javadoc.io/badge/io.jooby/jooby.svg)](https://javadoc.io/doc/io.jooby/jooby/latest)
[![Github](https://github.com/jooby-project/jooby/workflows/Full%20Build/badge.svg)](https://github.com/jooby-project/jooby/actions)
[![Slack](https://img.shields.io/badge/slack-chat-brightgreen.svg?logo=slack)](https://joobyio.slack.com)
[![Discord](https://img.shields.io/discord/1225457509909922015?label=discord)](https://discord.gg/nmfJmmrq)
[![Become a Patreon](https://img.shields.io/badge/patreon-donate-orange.svg)](https://patreon.com/edgarespina)
[![Donate](https://img.shields.io/badge/Donate-PayPal-blue.svg?logo=paypal&style=flat-square)](https://paypal.me/edgarespina)

# ∞ do more, more easily

[Jooby](https://jooby.io) is a modern, performant and easy to use web framework for Java and Kotlin built on top of your
favorite web server.

Java:

```java
import static io.jooby.Jooby.runApp;

public class App {

  public static void main(final String[] args) {
    runApp(args, app -> {
      app.get("/", ctx -> "Welcome to Jooby!");
    });
  }
}

```

Kotlin:

```kotlin
import io.jooby.runApp

fun main(args: Array<String>) {
  runApp(args) {
    get ("/") {
      "Welcome to Jooby!"
    }
  }
}

```

documentation
=====

Documentation is available at [https://jooby.io](https://jooby.io)

help
=====

[Slack](https://joobyio.slack.com)

donate & support
=====
- [Sponsor](https://github.com/sponsors/jknack)
- [Paypal](https://www.paypal.com/paypalme2/edgarespina)
- [Patreon](https://www.patreon.com/edgarespina)
- [support@jooby.io](mailto:support@jooby.io?Subject=Jooby%20Support)

1.x version
=====

Documentation for 1.x is available at [https://jooby.io/v1](https://jooby.io/v1)

Source code for 1.x is available at the [1.x branch](https://github.com/jooby-project/jooby/tree/1.x)

author
=====

 [Edgar Espina](https://twitter.com/edgarespina)

license
=====

[Apache License 2](http://www.apache.org/licenses/LICENSE-2.0.html)


# ModelDB: An open-source system for Machine Learning model versioning, metadata, and experiment management.
----

  Quickstart
   ·
  Workflow
   ·
  Examples
   ·
  Contribute
   ·
  Support (Slack)

----
ModelDB is an open-source system to version machine learning models including their ingredients code, data, config, and environment and to track ML metadata across the model lifecycle.

Use ModelDB in order to:
* Make your ML models reproducible
* Manage your ML experiments, build performance dashboards, and share reports
* Track models across their lifecycle including development, deployment, and live monitoring

Features:
* Works on Docker, Kubernetes
* Clients in Python and Scala
* Beautiful dashboards for model performance and reporting
* Git-like operations on any model
* Flexible metadata logging including metrics, artifacts, tags and user information
* Pluggable storage systems
* Integration into state-of-the-art frameworks like Tensorflow and PyTorch
* Battle-tested in production environments

If you are looking for a hosted version of ModelDB, please reach out at modeldb@verta.ai.

This version of ModelDB is built upon its predecessor from [CSAIL, MIT](https://www.csail.mit.edu/). The previous version can be found on Github [here](https://github.com/VertaAI/modeldb/releases/tag/v1.0.0). The ModelDB project is now maintained by [Verta.ai](https://verta.ai).

----

## What’s In This Document

- [Up and Running in 5 minutes](#up-and-running-in-5-minutes)
- [Documentation](#documentation)
- [Community](#community)
- [Architecture](#architecture)
- [How to Contribute](#how-to-contribute)
- [License](#license)
- [Thanks to Our Contributors](#thanks)

----

## Up and Running in 5 minutes

0. Install [Docker](https://docs.docker.com/compose/install/) (and Docker Compose)

1. Setup ModelDB via Docker Compose

```bash
docker-compose -f docker-compose-all.yaml up
```
*Note: modeldb-backend service needs backend/config/config.yaml to run, either clone the repo before running docker-compose or create the file manually.*

2. Install the ModelDB pip package. *Note it comes packaged in the verta package.*
```bash
pip install verta
```

3. Version a model or log a workflow. *Alternatively, run any of the detailed [examples](https://github.com/VertaAI/examples) in our repository.*

```python
from verta import Client
client = Client("http://localhost:3000")

proj = client.set_project("My first ModelDB project")
expt = client.set_experiment("Default Experiment")

# log the first run
run = client.set_experiment_run("First Run")
run.log_hyperparameters({"regularization" : 0.5})
# ... model training code goes here
run.log_metric('accuracy', 0.72)

# log the second run
run = client.set_experiment_run("Second Run")
run.log_hyperparameters({"regularization" : 0.8})
# ... model training code goes here
run.log_metric('accuracy', 0.83)
```

**That's it!** Navigate to **<http://localhost:3000>** to find the ModelDB Web UI and check out the models you just logged.

----

For information on debugging the Docker-based ModelDB installation, check [here](DEPLOY.md#Deploy-pre-published-images).

**Other ways to install ModelDB are:**

1. [Building the source code and deploying](DEPLOY.md#build-images-from-source-and-deploy)
1. [Deploy on kubernetes via helm](DEPLOY.md#kubernetes-setUp)
1. [Using a ModelDB ami](DEPLOY.md#AWS)
1. If you are looking for a hosted version of ModelDB, please reach out at modeldb@verta.ai.

----

## Documentation

Official documentation for ModelDB can be found [here](https://docs.verta.ai/).

----

## Community

For Getting Started guides, Tutorials, and API reference check out our [docs](https://docs.verta.ai/).

To report a bug, file a documentation issue, or submit a feature request, please open a GitHub issue.

For help, questions, contribution discussions and release announcements, please join us on [Slack](http://bit.ly/modeldb-mlops).

----

## Architecture

At a high level the architecture of ModelDB in a Kubernetes cluster or a Docker application looks as below:

![image](doc-resources/images/modeldb-architecture.png)

- **ModelDB Client** available in Python and Scala which can instantiated in the user's model building code and exposes functions to store information to ModelDB.
- **ModelDB Frontend**  developed in JavaScript and typescript is the visual reporting module of ModelDB. It also acts as an entry point for the ModelDB cluster.
  - It receives the request from client (1) and the browser and route them to the appropriate container.
  - The gRPC calls (2) for creating, reading,updating or deleting Projects, Experiments, ExperimentRuns, Dataset, DatasetVersions or their metadata are routed to ModelDB Proxy.
  - The HTTP calls (3) for storing and retrieving binary artifacts are forwarded directly to backend.
- **ModelDB Backend Proxy** developed in golang is a light weight gRPC to Http convertor.
  - It receives the gRPC request from the front end (2) and sends them to backend (4). In the other direction it converts the response from backend and sends it to the frontend.
- **ModelDB Backend** developed in java is module which stores, retrieves or deletes information as triggered by user via the client or the front end.
  - It exposes gRPC endpoints (4) for most of the operations which is used by the proxy.
  - It has http endpoints (3) for storing, retrieving and deleting artifacts used directly by the frontend.
- **Database** ModelDB Backend stores (5) the information from the requests it receive into a Relational database.
  - Out of the box ModelDB is configured and verified to work against PostgreSQL, but since it uses Hibernate as a ORM and liquibase for change management, it should be easy to configure ModelDB to run on another SQL Database supported by the the tools.

*Volumes : The relational database and the artifact store in backend need volumes attached to enable persistent storage.*

### Repo Structure

Each module in the architecture diagram has a designated folder in this repository, and has their own README covering in depth documentation and contribution guidelines.

1. **protos** has the protobuf definitions of the objects and endpoint used across ModelDB. More details [here](protos/README.md).
1. **backend** has the source code and tests for ModelDB Backend. It also holds the proxy at **backend/proxy**. More details [here](backend/README.md).
1. **client** has the source code and tests for ModelDB client. More details [here](client/README.md).
1. **webapp** has the source and tests for ModelDB frontend. More details [here](webapp/README.md).

Other supporting material for deployment and documentation is at:

1. **chart** has the helm chart to deploy ModelDB onto your Kubernetes cluster. More details [here](chart/modeldb/README.md).
1. **doc-resources** has images for documentation.

----

## Contributions

As seen from the [Architecture](#architecture) ModelDB provides a full stack solution to tracking, versioning and auditing  machine learning models.
We are open to contributions to any of the modules in form of Pull Requests.

The main skill sets for each module are as below:

1. backend: If you are interested in `Java` development or are interested in database design using technologies like `Hibernate` and `Liquibase` please take a look at [backend README](backend/README.md) for setup and development instructions.
1. client: If you are interested in `Python` or `Scala` development or are interested in building examples notebooks on various ML frameworks logging data to Modeldb please take a look at [client README](client/README.md).
1. frontend: If you are interested  in `Node`,`React` or `Redux` based development please take a look at [webapp README](webapp/README.md)

Please reach out to us in [slack](http://bit.ly/modeldb-mlops) for any assistance in getting started with the development set up or any other feedback.

----

## License

ModelDB is licensed under Apache 2.0.

----

## Thanks

Thanks to our many contributors and users.


<!--

    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

-->
[English](./README.md) | [中文](./README_ZH.md)

# IoTDB
[![Unit-Test](https://github.com/apache/iotdb/actions/workflows/unit-test.yml/badge.svg)](https://github.com/apache/iotdb/actions/workflows/unit-test.yml)
[![codecov](https://codecov.io/github/apache/iotdb/graph/badge.svg?token=ejF3UGk0Nv)](https://codecov.io/github/apache/iotdb)
[![GitHub release](https://img.shields.io/github/release/apache/iotdb.svg)](https://github.com/apache/iotdb/releases)
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
![](https://github-size-badge.herokuapp.com/apache/iotdb.svg)
![](https://img.shields.io/github/downloads/apache/iotdb/total.svg)
![](https://img.shields.io/badge/platform-win%20%7C%20macos%20%7C%20linux-yellow.svg)
![](https://img.shields.io/badge/java--language-1.8%20%7C%2011%20%7C%2017-blue.svg)
[![IoTDB Website](https://img.shields.io/website-up-down-green-red/https/shields.io.svg?label=iotdb-website)](https://iotdb.apache.org/)
[![Maven Version](https://maven-badges.herokuapp.com/maven-central/org.apache.iotdb/iotdb-parent/badge.svg)](http://search.maven.org/#search|gav|1|g:"org.apache.iotdb")
[![Gitpod Ready-to-Code](https://img.shields.io/badge/Gitpod-Ready--to--Code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/apache/iotdb)
[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://join.slack.com/t/apacheiotdb/shared_invite/zt-qvso1nj8-7715TpySZtZqmyG5qXQwpg)

# Overview

IoTDB (Internet of Things Database) is a data management system for time series data, which provides users with specific services, including data collection, storage and analysis. Due to its lightweight structure, high performance and usable features, together with its seamless integration with the Hadoop and Spark ecosystem, IoTDB meets the requirements of massive dataset storage, high throughput data input, and complex data analysis in the industrial IoT field.

[Click for More Information](https://www.timecho.com/archives/shi-xu-shu-ju-ku-iotdb-gong-neng-xiang-jie-yu-xing-ye-ying-yong)

IoTDB depends on [TsFile](https://github.com/apache/tsfile) which is a columnar storage file format designed for time series data. The branch `iotdb` of TsFile project is used to deploy SNAPSHOT version for IoTDB project.

# Main Features

The main features of IoTDB are as follows:

1. Flexible deployment strategy. IoTDB provides users with a one-click installation tool on either the cloud platform or the terminal devices, and a data synchronization tool bridging the data on cloud platform and terminals.
2. Low cost on hardware. IoTDB can reach a high compression ratio of disk storage.
3. Efficient directory structure. IoTDB supports efficient organization for complex time series data structures from intelligent networking devices, organization for time series data from devices of the same type, and fuzzy searching strategy for massive and complex directory of time series data.
4. High-throughput read and write. IoTDB supports millions of low-power devices' strong connection data access, high-speed data read and write for intelligent networking devices and mixed devices mentioned above.
5. Rich query semantics. IoTDB supports time alignment for time series data across devices and measurements, computation in time series field (frequency domain transformation) and rich aggregation function support in time dimension.
6. Easy to get started. IoTDB supports SQL-like language, JDBC standard API and import/export tools which are easy to use.
7. Seamless integration with state-of-the-practice Open Source Ecosystem. IoTDB supports analysis ecosystems, such as Hadoop and Spark, as well as visualization tools, such as Grafana.

For the latest information about IoTDB, please visit [IoTDB official website](https://iotdb.apache.org/). If you encounter any problems or identify any bugs while using IoTDB, please report an issue in [Jira](https://issues.apache.org/jira/projects/IOTDB/issues).

<!-- TOC -->

## Outline

- [IoTDB](#iotdb)
- [Overview](#overview)
- [Main Features](#main-features)
  - [Outline](#outline)
- [Quick Start](#quick-start)
  - [Prerequisites](#prerequisites)
  - [Installation](#installation)
    - [Build from source](#build-from-source)
    - [Configurations](#configurations)
  - [Start](#start)
    - [Start IoTDB](#start-iotdb)
    - [Use IoTDB](#use-iotdb)
      - [Use Cli](#use-cli)
      - [Basic commands for IoTDB](#basic-commands-for-iotdb)
    - [Stop IoTDB](#stop-iotdb)
  - [Only build server](#only-build-server)
  - [Only build cli](#only-build-cli)
  - [Usage of CSV Import and Export Tool](#usage-of-csv-import-and-export-tool)

<!-- /TOC -->

# Quick Start

This short guide will walk you through the basic process of using IoTDB. For a more detailed introduction, please visit our website's [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).

## Prerequisites

To use IoTDB, you need to have:

1. Java >= 1.8 (1.8, 11 to 17 are verified. Please make sure the environment path has been set accordingly).
2. Maven >= 3.6 (If you want to compile and install IoTDB from source code).
3. Set the max open files num as 65535 to avoid the "too many open files" error.
4. (Optional) Set the somaxconn as 65535 to avoid "connection reset" error when the system is under high load.
    ```
    # Linux
    > sudo sysctl -w net.core.somaxconn=65535

    # FreeBSD or Darwin
    > sudo sysctl -w kern.ipc.somaxconn=65535
    ```
### Linux

(This guide is based on an installation of Ubuntu 22.04.)

#### Git

Make sure `Git` is installed, if it's missing, simply install it via:

    sudo apt install git

#### Java

Make sure `Java` is installed, if it's missing, simply install it via:

    sudo apt install default-jdk

#### Flex

    sudo apt install flex

#### Bison

    sudo apt install bison

#### Boost

    sudo apt install libboost-all-dev

#### OpenSSL header files

Usually OpenSSL is already installed, however it's missing the header files we need to compile.
So ensure these are installed:

    sudo apt install libssl-dev

### Mac OS

#### Git

First ensure `git` works.

Usually on a new Mac, as soon as you simply type `git` in a `Terminal` window, a popup will come up and ask if you want to finish installing the Mac developer tools.
Just say yes.
As soon as this is finished, you are free to use `git`.

#### Homebrew

Then install `Homebrew` - If this hasn't been installed yet, as we are going to be installing everything using `Homebrew`.

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

#### Java

As soon as that's done install `Java`, if this hasn't been installed yet:

    brew install java

Depending on your version of Homebrew, it will tell you to do one of the following (depending on the type of processor in your device).

Mainly on the Intel-based models:

    sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

Mainly on the ARM-based models:

    sudo ln -sfn /opt/homebrew/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk

#### CPP Prerequisites

Building `Thrift` requires us to add two more dependencies to the picture.

This however is only needed when enabling the `compile-cpp` profile:

    brew install boost
    brew install bison
    brew install openssl

### Windows

#### Chocolatey

Then install `Chocolatey` - If this hasn't been installed yet, as we are going to be installing everything using `Chocolatey`.

https://chocolatey.org/install

#### Git

    choco install git.install

#### Java

    choco install openjdk

#### Visual Studio 19 2022

    choco install visualstudio2022community
    choco install visualstudio2022buildtools
    choco install visualstudio2022-workload-nativedesktop

#### Flex / Bison

    choco install winflexbison

#### Boost

    choco install boost-msvc-14.2

#### OpenSSL

    choco install openssl

## Installation

IoTDB provides three installation methods, you can refer to the following suggestions, choose the one fits you best:

* Installation from source code. If you need to modify the code yourself, you can use this method.
* Installation from binary files. Download the binary files from the official website. This is the recommended method, in which you will get a binary released package which is out-of-the-box.
* Using Docker：The path to the dockerfile is [here](https://github.com/apache/iotdb/tree/master/docker/src/main).

Here in the Quick Start, we give a brief introduction of using source code to install IoTDB. For further information, please refer to [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).

## Build from source

### Prepare Thrift compiler

Skip this chapter if you are using Windows.

As we use Thrift for our RPC module (communication and
protocol definition), we involve Thrift during the compilation, so Thrift compiler 0.13.0 (or
higher) is required to generate Thrift Java code. Thrift officially provides binary compiler for
Windows, but unfortunately, they do not provide that for Unix OSs.

If you have permission to install new software, use `apt install` or `yum install` or `brew install`
to install the Thrift compiler. (If you already have installed the thrift compiler, skip this step.)
Then, you may add the following parameter
when running Maven: `-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt -Dthrift.exec.absolute.path=<YOUR LOCAL THRIFT BINARY FILE>`.

If not, then you have to compile the thrift compiler, and it requires you install a boost library first.
Therefore, we compiled a Unix compiler ourselves and put it onto GitHub, and with the help of a
maven plugin, it will be downloaded automatically during compilation.
This compiler works fine with gcc8 or later, Ubuntu MacOS, and CentOS, but previous versions
and other OSs are not guaranteed.

If you can not download the thrift compiler automatically because of a network problem, you can download
it by yourself, and then either:
rename your thrift file to `{project_root}\thrift\target\tools\thrift_0.12.0_0.13.0_linux.exe`;
or, add Maven commands:
`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt -Dthrift.exec.absolute.path=<YOUR LOCAL THRIFT BINARY FILE>`.

### Compile IoTDB

You can download the source code from:

```
git clone https://github.com/apache/iotdb.git
```

The default dev branch is the master branch, if you want to use a released version x.x.x:

```
git checkout vx.x.x
```

Or checkout to the branch of a big version, e.g., the branch of 1.0 is rel/1.0.

```
git checkout rel/x.x
```

### Build IoTDB from source

Under the root path of iotdb:

```
> mvn clean package -pl distribution -am -DskipTests
```

After being built, the IoTDB distribution is located at the folder: "distribution/target".

### Only build cli

Under the iotdb/iotdb-client path:

```
> mvn clean package -pl cli -am -DskipTests
```

After being built, the IoTDB cli is located at the folder "cli/target".

### Build Others

Use `-P compile-cpp` for compiling the cpp client. (For more details, read client-cpp's Readme file.)

**NOTE: Directories "`thrift/target/generated-sources/thrift`", "`thrift-sync/target/generated-sources/thrift`",
"`thrift-cluster/target/generated-sources/thrift`", "`thrift-influxdb/target/generated-sources/thrift`"
and "`antlr/target/generated-sources/antlr4`" need to be added to sources roots to avoid compilation errors in the IDE.**

**In IDEA, you just need to right click on the root project name and choose "`Maven->Reload Project`" after
you run `mvn package` successfully.**

### Configurations

Configuration files are under the "conf" folder.

  * environment config module (`datanode-env.bat`, `datanode-env.sh`),
  * system config module (`iotdb-datanode.properties`)
  * log config module (`logback.xml`).

For more information, please see [Config Manual](https://iotdb.apache.org/UserGuide/Master/Reference/DataNode-Config-Manual.html).

## Start

You can go through the following steps to test the installation. If there is no error returned after execution, the installation is completed.

### Start IoTDB

Users can start 1C1D IoTDB by the start-standalone script under the sbin folder.

```
# Unix/OS X
> sbin/start-standalone.sh

# Windows
> sbin\start-standalone.bat
```

### Use IoTDB

#### Use Cli

IoTDB offers different ways to interact with server, here we introduce the basic steps of using Cli tool to insert and query data.

After installing IoTDB, there is a default user 'root', its default password is also 'root'. Users can use this
default user to login Cli to use IoTDB. The start-up script of Cli is the start-cli script in the folder sbin. When executing the script, user should assign
IP, PORT, USER_NAME and PASSWORD. The default parameters are "-h 127.0.0.1 -p 6667 -u root -pw -root".

Here is the command for starting the Cli:

```
# Unix/OS X
> sbin/start-cli.sh -h 127.0.0.1 -p 6667 -u root -pw root

# Windows
> sbin\start-cli.bat -h 127.0.0.1 -p 6667 -u root -pw root
```

The command line cli is interactive, so you should see the welcome logo and statements if everything is ready:

```
 _____       _________  ______   ______
|_   _|     |  _   _  ||_   _ `.|_   _ \
  | |   .--.|_/ | | \_|  | | `. \ | |_) |
  | | / .'`\ \  | |      | |  | | |  __'.
 _| |_| \__. | _| |_    _| |_.' /_| |__) |
|_____|'.__.' |_____|  |______.'|_______/  version x.x.x

IoTDB> login successfully
IoTDB>
```

#### Basic commands for IoTDB

Now, let us introduce the way of creating timeseries, inserting data and querying data.

The data in IoTDB is organized as timeseries. Each timeseries includes multiple data–time pairs, and is owned by a database. Before defining a timeseries, we should define a database using CREATE DATABASE first, and here is an example:

```
IoTDB> CREATE DATABASE root.ln
```

We can also use SHOW DATABASES to check the database being created:

```
IoTDB> SHOW DATABASES
+-------------+
|     Database|
+-------------+
|      root.ln|
+-------------+
Total line number = 1
```

After the database is set, we can use CREATE TIMESERIES to create a new timeseries. When creating a timeseries, we should define its data type and the encoding scheme. Here we create two timeseries:

```
IoTDB> CREATE TIMESERIES root.ln.wf01.wt01.status WITH DATATYPE=BOOLEAN, ENCODING=PLAIN
IoTDB> CREATE TIMESERIES root.ln.wf01.wt01.temperature WITH DATATYPE=FLOAT, ENCODING=RLE
```

In order to query the specific timeseries, we can use SHOW TIMESERIES <Path>. <Path> represent the location of the timeseries. The default value is "null", which queries all the timeseries in the system (the same as using "SHOW TIMESERIES root"). Here are some examples:

1. Querying all timeseries in the system:

```
IoTDB> SHOW TIMESERIES
+-----------------------------+-----+-------------+--------+--------+-----------+----+----------+
|                   Timeseries|Alias|Database|DataType|Encoding|Compression|Tags|Attributes|
+-----------------------------+-----+-------------+--------+--------+-----------+----+----------+
|root.ln.wf01.wt01.temperature| null|      root.ln|   FLOAT|     RLE|     SNAPPY|null|      null|
|     root.ln.wf01.wt01.status| null|      root.ln| BOOLEAN|   PLAIN|     SNAPPY|null|      null|
+-----------------------------+-----+-------------+--------+--------+-----------+----+----------+
Total line number = 2
```

2. Querying a specific timeseries (root.ln.wf01.wt01.status):

```
IoTDB> SHOW TIMESERIES root.ln.wf01.wt01.status
+------------------------+-----+-------------+--------+--------+-----------+----+----------+
|              timeseries|alias|database|dataType|encoding|compression|tags|attributes|
+------------------------+-----+-------------+--------+--------+-----------+----+----------+
|root.ln.wf01.wt01.status| null|      root.ln| BOOLEAN|   PLAIN|     SNAPPY|null|      null|
+------------------------+-----+-------------+--------+--------+-----------+----+----------+
Total line number = 1
```

Inserting timeseries data is a basic operation of IoTDB, you can use the ‘INSERT’ command to finish this. Before insertion, you should assign the timestamp and the suffix path name:

```
IoTDB> INSERT INTO root.ln.wf01.wt01(timestamp,status) values(100,true);
IoTDB> INSERT INTO root.ln.wf01.wt01(timestamp,status,temperature) values(200,false,20.71)
```

The data that you have just inserted will be displayed as follows:

```
IoTDB> SELECT status FROM root.ln.wf01.wt01
+------------------------+------------------------+
|                    Time|root.ln.wf01.wt01.status|
+------------------------+------------------------+
|1970-01-01T00:00:00.100Z|                    true|
|1970-01-01T00:00:00.200Z|                   false|
+------------------------+------------------------+
Total line number = 2
```

You can also query several timeseries data using one SQL statement:

```
IoTDB> SELECT * FROM root.ln.wf01.wt01
+------------------------+-----------------------------+------------------------+
|                    Time|root.ln.wf01.wt01.temperature|root.ln.wf01.wt01.status|
+------------------------+-----------------------------+------------------------+
|1970-01-01T00:00:00.100Z|                         null|                    true|
|1970-01-01T00:00:00.200Z|                        20.71|                   false|
+------------------------+-----------------------------+------------------------+
Total line number = 2
```

To change the time zone in Cli, you can use the following SQL:

```
IoTDB> SET time_zone=+08:00
Time zone has set to +08:00
IoTDB> SHOW time_zone
Current time zone: Asia/Shanghai
```

Add then the query result will show using the new time zone.

```
IoTDB> SELECT * FROM root.ln.wf01.wt01
+-----------------------------+-----------------------------+------------------------+
|                         Time|root.ln.wf01.wt01.temperature|root.ln.wf01.wt01.status|
+-----------------------------+-----------------------------+------------------------+
|1970-01-01T08:00:00.100+08:00|                         null|                    true|
|1970-01-01T08:00:00.200+08:00|                        20.71|                   false|
+-----------------------------+-----------------------------+------------------------+
Total line number = 2
```

The commands to exit the Cli are:

```
IoTDB> quit
or
IoTDB> exit
```

For more information about the commands supported by IoTDB SQL, please see [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).

### Stop IoTDB

The server can be stopped with "ctrl-C" or the following script:

```
# Unix/OS X
> sbin/stop-standalone.sh

# Windows
> sbin\stop-standalone.bat
```

# The use of CSV Import and Export Tool

see [The use of CSV Import and Export Tool](https://iotdb.apache.org/UserGuide/latest/Tools-System/Import-Export-Tool.html)

# Frequent Questions for Compiling
see [Frequent Questions when Compiling the Source Code](https://iotdb.apache.org/Development/ContributeGuide.html#_Frequent-Questions-when-Compiling-the-Source-Code)

# Contact Us

### QQ Group

* Apache IoTDB User Group: 659990460

### Wechat Group

* Add friend: `tietouqiao` or `liutaohua001`, and then we'll invite you to the group.

### Slack

* [Slack channel](https://join.slack.com/t/apacheiotdb/shared_invite/zt-qvso1nj8-7715TpySZtZqmyG5qXQwpg)

see [Join the community](https://github.com/apache/iotdb/issues/1995) for more!


= start.spring.io image:https://badges.gitter.im/spring-io/initializr.svg[link="https://gitter.im/spring-io/initializr?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge"]
:library: https://github.com/spring-io/initializr

This repository configures a {library}[Spring Initializr] instance with a custom UI
running at https://start.spring.io. The following modules are available:

* `start-client`: client-side assets
* `start-site`: server infrastructure and metadata configuration
* `start-site-verification`: tests to verify the validity of the metadata

[[using]]
== Using
There is a link:USING.adoc[dedicated page] that describes how you can interact with the
service.

[[build]]
== Building from Source

You need Java 17 and a bash-like shell.

[[building]]
=== Building

Invoke the build at the root of the project

[indent=0]
----
    $ ./mvnw clean install
----

The project also includes a number of longer-running verification tests. They
can be built and run using the `verification` profile:

[indent=0]
----
    $ ./mvnw -Pverification clean install
----

The project's other tests are not included in the `verification` profile. All of
the project's tests can be run using the `full` profile:

[indent=0]
----
    $ ./mvnw -Pfull clean install
----

If building `start-client` fails, you may have an outdated cache that can be deleted as
follows:

[indent=0]
----
    $ cd start-client
    $ rm -rf .cache node_modules
----

[[run-app]]
=== Running the app locally
As long as you've built the project beforehand (in particular `start-client`), you can
easily start the app as any other Spring Boot app:

[indent=0]
----
    $ cd start-site
    $ ../mvnw spring-boot:run
----

[[run-ide]]
=== Running the app in an IDE
You should be able to import the project into your IDE with no problems. Once there you
can run the `StartApplication` from its main method and debug it. If you also need to
work on the library, adding the `initializr` project in your workspace would make sure
to reload the app whenever you make any change.

This is the recommended way to operate while you are developing the application,
especially the UI.

== Reusing the Web UI
This instance has a thin layer with our opinions about getting started with Spring Boot.
You can reuse them but please keep in mind that this is not supported with the same
level as the Spring Initializr library. The Web UI, in particular, is for our own
purpose and is not particularly engineered to be easily extended.

== Deploying to Cloud Foundry

If you are on a Mac and using https://brew.sh/[homebrew], install the Cloud Foundry
CLI:

[indent=0]
----
    $ brew install cloudfoundry-cli
----

Alternatively, download a suitable binary for your platform from
https://console.run.pivotal.io/tools[Pivotal Web Services].

You should ensure that the application name and URL (name and host values) are
suitable for your environment before running `cf push`.

First, make sure that you have <<building, built the application>>, then make sure first
that the jar has been created:

[indent=0]
----
    $ cd start-site
    $ ../mvnw package
----

The project creates a regular library jar and a repackaged archive that can be used to
start the application with the `exec` classifier. Once the build as completed, you can
push the application:

[indent=0]
----
    $ cf push your-start -p target/start-site-exec.jar
----

== License
The start.spring.io website is Open Source software released under the
https://www.apache.org/licenses/LICENSE-2.0.html[Apache 2.0 license].