Disclaimer: This is not an official Google product.

# Mug
A small Java 8 utilities library ([javadoc](http://google.github.io/mug/apidocs/index.html)), with **0 deps** (Proto, BigQuery, Guava addons are in separate artifacts). ![](https://travis-ci.org/google/mug.svg?branch=master)

Offers:
* **Intuitive to read**, powerful string manipulation ([StringFormat](https://github.com/google/mug/wiki/StringFormat-Explained), [Substring](https://github.com/google/mug/wiki/Substring-Explained))
   * `new StringFormat("/{user}-home/{yyyy}/{mm}/{dd}").parse(path, (user, yyyy, mm, dd) -> ...)`
   * `String user = before(first('@')).from(email).orElseThrow();`
* Streaming pairs ([BiStream](https://github.com/google/mug/wiki/BiStream-Explained))
   * `Map<Instant, Long> histogram = zip(times, counts).toMap();`
   * `Map<K, V> combined = concat(map1, map2).toMap();`
   * `Map<Principal, V> keyedByPrincipal = BiStream.from(keyedByUserId).mapKeys(UserId::principal).toMap();`
* More ([MoreStreams](#morestreams), [Optionals](#optionals), [DateTimeFormats](https://github.com/google/mug/wiki/Parsing-Date-Time-Should-Be-10x-Easier), [...](https://github.com/google/mug/wiki))
  * Create `Optional` with a guard condition:
    * `return optionally(count > 0, () -> total / count);`
  * Parse any legit date/time string (without a pattern string):
    * `Instant timestamp = DateTimeFormats.parseToInstant("2024-01-30 15:30:00-08")`
    * `DateTimeFormats.formatOf("Tue, 10 Jan 2023 10:00:00.123 America/Los_Angeles")`

## Installation
### Maven

Add the following to pom.xml:
```
  <dependency>
    <groupId>com.google.mug</groupId>
    <artifactId>mug</artifactId>
    <version>8.0</version>
  </dependency>
```

Add `mug-errorprone` to your annotationProcessorPaths:

```
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <configuration>
            <annotationProcessorPaths>
              <path>
                <groupId>com.google.errorprone</groupId>
                <artifactId>error_prone_core</artifactId>
                <version>2.23.0</version>
              </path>
              <path>
                <groupId>com.google.mug</groupId>
                <artifactId>mug-errorprone</artifactId>
                <version>8.0</version>
              </path>
            </annotationProcessorPaths>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
```

Protobuf utils ([javadoc](https://google.github.io/mug/apidocs/com/google/mu/protobuf/util/package-summary.html)):
```
  <dependency>
    <groupId>com.google.mug</groupId>
    <artifactId>mug-protobuf</artifactId>
    <version>8.0</version>
  </dependency>
```

Guava add-ons (with [`SafeQuery`](https://google.github.io/mug/apidocs/com/google/mu/safesql/SafeQuery.html) and [`GoogleSql`](https://google.github.io/mug/apidocs/com/google/mu/safesql/GoogleSql.html)):
```
  <dependency>
    <groupId>com.google.mug</groupId>
    <artifactId>mug-guava</artifactId>
    <version>8.0</version>
  </dependency>
```

### Gradle

Add to build.gradle:
```
  implementation 'com.google.mug:mug:8.0'
  implementation 'com.google.mug:mug-guava:8.0'
  implementation 'com.google.mug:mug-protobuf:8.0'
```

## StringFormat

Extracts structured data from string:

```java
new StringFormat("/users/{user}/.{hidden_file_name}")
    .parse(filePath, (user, fileName) -> ...);
```

```java
new StringFormat("{hour}:{minute}:{second}.{millis}")
    .parse(“10:26:30.748”, (hour, minute, second, millis) -> ...);
```

An ErrorProne check is in place to check that the number of lambda parameters and
the parameter names match the format string.

This allows you to define `StringFormat` objects as private class constant, and safely use them
many lines away.

## Substring

**Example 1: strip off a prefix if existent:**
```java
String httpStripped = Substring.prefix("http://").removeFrom(uri);
```

**Example 2: strip off any scheme prefix from a uri:**
```java
String schemeStripped = Substring.upToIncluding(first("://")).removeFrom(uri);
```

**Example 3: split a string in the format of "name=value" into `name` and `value`:**
```java
Substring.first('=').split("name=value").map((name, value) -> ...);
```

**Example 4: replace trailing "//" with "/" :**
```java
Substring.suffix("//").replaceFrom(path, "/");
```

**Example 5: strip off the suffix starting with a dash (-) character :**
```java
last('-').toEnd().removeFrom(str);
```

**Example 6: extract a substring using regex :**
```java
String quoted = Substring.first(Pattern.compile("'(.*?)'"), 1)
    .from(str)
    .orElseThrow(...);
```

**Example 7: find the substring between the first and last curly braces ({) :**
```java
String body = Substring.between(first('{'), last('}'))
    .from(source)
    .orElseThrow(...);
```

## Stream

#### [BiStream](https://google.github.io/mug/apidocs/com/google/mu/util/stream/BiStream.html) streams pairs of objects.

This class closely mirrors JDK `Stream` API (the few extra methods of "its own" are very straight-forward). If you are familiar with Jdk stream, learning curve is minimal.

**Example 1: to concatenate `Map`s:**

```java
import static com.google.mu.util.stream.BiStream.concat;

Map<AccountId, Account> allAccounts = concat(primaryAccouunts, secondaryAccounts).toMap();
```

**Example 2: to combine two streams:**

```java
BiStream.zip(requests, responses)
    .mapToObj(RequestAndResponseLog::new);
```

**Example 3: to build a Map fluently:**

```java
Map<DoctorId, Patient> patientsByDoctorId = BiStream.zip(doctors, patients)
    .filter((doctor, patient) -> patient.likes(doctor))
    .mapKeys(Doctor::getId)
    .collect(toMap());
```

**Example 4: to build Guava ImmutableListMultimap fluently:**

```java
ImmutableListMultimap<ZipCode, Address> addressesByZipCode = BiStream.from(addresses)
    .mapKeys(Address::getZipCode)
    .collect(ImmutableListMultimap::toImmutableListMultimap);
```

**Example 5: to
a `Map` into sub-maps:**

```java
import static com.google.mu.util.stream.BiCollectors.groupingBy;

Map<Address, PhoneNumber> phonebooks = ...;
Map<State, Map<Address, PhoneNumber>> statePhonebooks = BiStream.from(phonebooks)
    .collect(groupingBy(Address::state, Collectors::toMap))
    .toMap();
```

**Example 6: to merge `Map` entries:**

```java
import static com.google.mu.util.stream.BiCollectors.toMap;
import static com.google.mu.util.stream.MoreCollectors.flatteningMaps;

Map<Account, Money> totalPayouts = projects.stream()
    .map(Project::payments)  // Stream<Map<Account, Money>>
    .collect(flatteningMaps(toMap(Money::add)));
```

**Example 7: to apply grouping over `Map` entries:**

```java
import static com.google.mu.util.stream.BiCollectors.toMap;
import static com.google.mu.util.stream.MoreCollectors.flatteningMaps;
import static java.util.stream.Collectors.summingInt;

Map<EmployeeId, Integer> workerHours = projects.stream()
    .map(Project::getTaskAssignments)  // Stream<Map<Employee, Task>>
    .collect(flatteningMaps(toMap(summingInt(Task::hours))));
```

**Example 8: to turn a `Collection<Pair<K, V>>` to `BiStream<K, V>`:**

```java
BiStream<K, V> stream = RiStream.from(pairs, Pair::getKey, Pair::getValue);
```

**Q: Why not `Map<Foo, Bar>` or `Multimap<Foo, Bar>`?**

A: Sometimes Foo and Bar are just an arbitrary pair of objects, with no key-value relationship. Or you may not trust `Foo#equals()` and `hashCode()`. Instead, drop-in replace your `Stream<Pair<Foo, Bar>>`/`List<Pair<Foo, Bar>>` with `BiStream<Foo, Bar>`/`BiCollection<Foo, Bar>` to get better readability.

**Q: Why not `Stream<FooAndBar>`?**

A: When you already have a proper domain object, sure. But you might find it cumbersome to define a bunch of FooAndBar, PatioChairAndKitchenSink one-off classes especially if the relationship between the two types is only relevant in the local code context.

**Q: Why not `Stream<Pair<Foo, Bar>>`?**

A: It's distracting to read code littered with opaque method names like `getFirst()` and `getSecond()`.

#### [MoreStreams](https://google.github.io/mug/apidocs/com/google/mu/util/stream/MoreStreams.html)

**Example 1: to group consecutive elements in a stream:**

```java
List<StockPrice> pricesOrderedByTime = ...;

List<List<StockPrice>> priceSequences =
    MoreStreams.groupConsecutive(
            pricesOrderedByTime.stream(), (p1, p2) -> closeEnough(p1, p2), toList())
        .collect(toList());
```

**Example 2: to iterate over `Stream`s in the presence of checked exceptions or control flow:**

The `Stream` API provides `forEach()` to iterate over a stream, if you don't have to throw checked exceptions.

When checked exception is in the way, or if you need control flow (`continue`, `return` etc.), `iterateThrough()` and `iterateOnce()` can help. The following code uses `iterateThrough()` to write objects into an `ObjectOutputStream`, with  `IOException` propagated:

```java
Stream<?> stream = ...;
ObjectOutput out = ...;
iterateThrough(stream, out::writeObject);
```

with control flow:
```java
for (Object obj : iterateOnce(stream)) {
  if (...) continue;
  else if (...) return;
  out.writeObject(obj);
}
```

**Example 3: to merge maps:**
```java
interface Page {
  Map<Day, Long> getTrafficHistogram();
}

List<Page> pages = ...;

// Merge traffic histogram across all pages of the web site
Map<Day, Long> siteTrafficHistogram = pages.stream()
    .map(Page::getTrafficHistogram)
    .collect(flatteningMaps(groupingBy(day -> day, Long::sum)))
    .toMap();
```

## Optionals

**Example 1: to combine two Optional instances into a single one:**
```java
Optional<Couple> couple = Optionals.both(optionalHusband, optionalWife).map(Couple::new);
```

**Example 2: to run code when two Optional instances are both present:**
```java
Optionals.both(findTeacher(), findStudent()).ifPresent(Teacher::teach);
```

**Example 3: or else run a fallback code block:**
```java
static import com.google.mu.util.Optionals.ifPresent;

Optional<Teacher> teacher = findTeacher(...);
Optional<Student> student = findStudent(...);
ifPresent(teacher, student, Teacher::teach)             // teach if both present
    .or(() -> ifPresent(teacher, Teacher::workOut))     // teacher work out if present
    .or(() -> ifPresent(student, Student::doHomework))  // student do homework if present
    .orElse(() -> log("no teacher. no student"));       // or else log
```

**Example 4: wrap a value in Optional if it exists:**
```java
static import com.google.mu.util.Optionals.optionally;

Optional<String> id = optionally(request.hasId(), request::getId);
```

**Example 5: add an optional element to a list if present:**
```java
static import com.google.mu.util.Optionals.asSet;

names.addAll(asSet(optionalName));
```

All Optionals utilites propagate checked exception from the the lambda/method references.


[![Maven Central](https://img.shields.io/maven-central/v/org.rm3l/maoni)](https://search.maven.org/artifact/org.rm3l/maoni)
[![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat)](https://github.com/rm3l/maoni/blob/master/LICENSE)

[![Android Arsenal](https://img.shields.io/badge/Android%20Arsenal-Maoni-blue.svg?style=flat)](http://android-arsenal.com/details/1/3925)
[![Website](https://img.shields.io/website-up-down-green-red/http/maoni.rm3l.org.svg)](http://maoni.rm3l.org)

[![CI](https://github.com/rm3l/maoni/actions/workflows/build.yml/badge.svg)](https://github.com/rm3l/maoni/actions/workflows/build.yml)
[![Translations](https://github.com/rm3l/maoni/actions/workflows/translations.yml/badge.svg)](https://github.com/rm3l/maoni/actions/workflows/translations.yml)
[![Website](https://github.com/rm3l/maoni/actions/workflows/website.yml/badge.svg)](https://github.com/rm3l/maoni/actions/workflows/website.yml)

[![Maintainability](https://api.codeclimate.com/v1/badges/6deec1732c359d492b3e/maintainability)](https://codeclimate.com/github/rm3l/maoni/maintainability)
[![Issue Count](https://codeclimate.com/github/rm3l/maoni/badges/issue_count.svg)](https://codeclimate.com/github/rm3l/maoni)

[![All Contributors](https://img.shields.io/badge/all_contributors-12-orange.svg?style=flat-square)](#contributors-)

[![GitHub watchers](https://img.shields.io/github/watchers/rm3l/maoni.svg?style=social&label=Watch)](https://github.com/rm3l/maoni)
[![GitHub stars](https://img.shields.io/github/stars/rm3l/maoni.svg?style=social&label=Star)](https://github.com/rm3l/maoni)
[![GitHub forks](https://img.shields.io/github/forks/rm3l/maoni.svg?style=social&label=Fork)](https://github.com/rm3l/maoni)

**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [Motivations](#motivations)
- [Sample App](#sample-app)
- [Preview](#preview)
- [Getting started](#getting-started)
  - [Putting it together](#putting-it-together)
    - [Available callbacks](#available-callbacks)
      - [maoni-email](#maoni-email)
      - [maoni-slack](#maoni-slack)
      - [maoni-github](#maoni-github)
      - [maoni-jira](#maoni-jira)
      - [maoni-doorbell](#maoni-doorbell)
    - [Sharing the files captured with other apps](#sharing-the-files-captured-with-other-apps)
- [Contribute and Improve Maoni!](#contribute-and-improve-maoni)
  - [Building from source](#building-from-source)
  - [Translations](#translations)
  - [Contributing callbacks for Maoni](#contributing-callbacks-for-maoni)
  - [Publishing a new release](#publishing-a-new-release)
- [In use in the following apps](#in-use-in-the-following-apps)
- [Credits](#credits)
- [Developed by](#developed-by)
- [Contributors](#contributors)
- [Sponsors](#sponsors)
- [License](#license)

**Maoni** is a lightweight open-source library for integrating
a way to collect user feedback from within Android applications.

Built from the ground up with the [Material Design](https://www.google.com/design/spec/material-design)
principles in mind, it allows to capture a screenshot of the activity the user is currently viewing
and attach it to the feedback.

Just provide callbacks implementations and you're good to go.
Maoni will take care of collecting feedback and call those implementations.

Below is a quick overview of the features included:
- **Contextual information**. Device and application information, if available.
    - Device screen resolution, mobile data and GPS states, ...
- **Screenshot and logs capture**.
    - Because receiving a feedback with contextual information is much much
    better for analysis, Maoni allows to take a screen capture of the calling activity, along with the application logs.
    Note that the inclusion of such screenshot and logs in the feedback object is opt-out, at the user's discretion.
    - Touch to preview screenshot
    - Ability for users of your app to highlight or blackout items in the screen capture. They may choose to
    highlight relevant items or blackout any sensitive information.
- **Customization**.
    - Besides the default form fields, you are free to include an extra layout
    with additional views. And you always have access to the underlying view elements.
    - Theme elements and styles can be adjusted.
- **Callbacks**.
    - Form validation. You can provide your own if needed for example for your extra fields.
    - Listeners. Upon validation, Maoni calls the callbacks implementations you provided earlier.
    So you just have limitless possibilities for an integration with any remote feedback services. For reference, the following implementations are provided as external dependencies:
        - [maoni-email](callbacks/maoni-email/), so your users can send their feedback via email
        - [maoni-slack](callbacks/maoni-slack/), so your users can send their feedback to Slack
        - [maoni-jira](callbacks/maoni-jira/), to send user feedback as JIRA issues (to the JIRA host of your choice)
        - [maoni-github](callbacks/maoni-github/), to send user feedback as Github issues (to the Github repository of your choice)
        - [maoni-doorbell](callbacks/maoni-doorbell), to send user feedback to [Doorbell.io](https://doorbell.io/home)

Take a look at the [sample application](https://play.google.com/store/apps/details?id=org.rm3l.maoni.sample)
for a quick overview.

## Motivations

While working on a new version of [DD-WRT Companion](https://ddwrt-companion.app/),
one of my Android apps, I needed a simple yet pleasant way to collect user feedback,
along with some contextual information.
I experimented with a simple dialog, then tried a bunch of other libraries,
but could not find one with screenshot capturing capabilities, not vendor lock-in,
and which is almost a no-brainer as to integrating with any remote services.
I was also looking for screen capture highlight / blackout capabilities, as in use for issue reporting in
several apps from Google.

So as a way to give back to the Open Source community,
I decided to create Maoni as a separate library project.

By the way, as a side note, Maoni is a Swahili word for comments or opinions.

## Sample App

## Preview

## Getting started

This library is published on [Maven Central](https://search.maven.org/artifact/org.rm3l/maoni). So importing it should be straightforward.
Declare the [Maven Central repository](https://repo.maven.apache.org/maven2/) (if not done yet) and import this project:

```gradle
  repositories {
      //...
      mavenCentral()
  }

  dependencies {
    // ...
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

### Putting it together

Integrating with Maoni is intended to be seamless and straightforward for most existing
Android applications.

Just leverage the fluent Maoni Builder to construct and start an Maoni instance at the right place
within your application workflow (for example a button click listener, or a touch of a menu item).

For example, to start with just the defaults:
```java
    //The optional file provider authority allows you to
    //share the screenshot capture file to other apps (depending on your callback implementation)
    new Maoni.Builder(<myContextObject>, MY_FILE_PROVIDER_AUTHORITY)
        .withDefaultToEmailAddress("feedback@my.company.com")
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling activity
```

To customize every aspect of your Maoni activity, call the fluent methods of `Maoni.Builder`, e.g.:
```java
    // MyHandlerForMaoni is a custom implementation of Maoni.Handler,
    // which is a shortcut interface for defining both a validator and listeners for Maoni
    final MyHandlerForMaoni myHandlerForMaoni = new MyHandlerForMaoni(MaoniSampleMainActivity.this);

    //The optional file provider authority allows you to
    //share the screenshot capture file to other apps (depending on your callback implementation)
    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withWindowTitle("Send Feedback") //Set to an empty string to clear it
        .withMessage("Hey! Love this app? We would love to hear from you.")
        .withExtraLayout(R.layout.my_feedback_activity_extra_content)
        .withHandler(myHandlerForMaoni) //Custom Callback for Maoni
        .withFeedbackContentHint("[Custom hint] Write your feedback here")
        .withIncludeScreenshotText("[Custom text] Include screenshot")
        .withTouchToPreviewScreenshotText("Touch To Preview and Edit")
        .withContentErrorMessage("Custom error message")
        .withScreenshotHint("Custom test: Lorem Ipsum Dolor Sit Amet...")
        //... there are other aspects you can customize
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling activity
```

**You're good to go!** Maoni will take care of validating / collecting user feedback
and call your callbacks implementations.

#### Available callbacks
Some common callbacks for Maoni are available as external dependencies to include in your application.

##### [maoni-email](callbacks/maoni-email/README.md)

This callback opens up an Intent for sending an email with the feedback collected.
This is the default fallback listener used in case no other listener has been set explicitly.
In other words, you need not import `maoni-email` as an extra dependency.
Just import `maoni` as depicted above, and you're good to go.

Add this additional line to your `build.gradle`:

```gradle
  dependencies {
    // ...
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

And set it as the listener for your Maoni instance:
```java
    final org.rm3l.maoni.email.MaoniEmailListener emailListenerForMaoni =
            new org.rm3l.maoni.email.MaoniEmailListener(...);

    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withListener(emailListenerForMaoni) //Callback from maoni-email
        //...
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling activity
```

Visit the dedicated [README](callbacks/maoni-email/README.md) for further details.

##### [maoni-slack](callbacks/maoni-slack/README.md)

This callback sends feedback collected to Slack via an [an incoming WebHook integration](https://my.slack.com/services/new/incoming-webhook).

 To use it, you must first [set up an incoming WebHook integration](https://my.slack.com/services/new/incoming-webhook), and grab the Webhook URL.

Add this additional line to your `build.gradle`:

```gradle
  dependencies {
    // ...
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
    implementation('org.rm3l:maoni-slack:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

And set it as the listener for your Maoni instance
```java
    final org.rm3l.maoni.slack.MaoniSlackListener slackListenerForMaoni =
            new org.rm3l.maoni.slack.MaoniSlackListener(...); //Pass the Slack WebHook URL, channel, ...

    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withListener(slackListenerForMaoni) //Callback from maoni-slack
        //...
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling activity
```

Visit the dedicated [README](callbacks/maoni-slack/README.md) for further details.

##### [maoni-github](callbacks/maoni-github/README.md)

This callback sends feedback collected as a Github issue to a specified Github repository.

To use it, you will need to have an account there, and grab your Personal Access Token.
You may want to create a dedicated `reporter` user for that purpose.

Add this additional line to your `build.gradle`:

```gradle
  dependencies {
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
    implementation('org.rm3l:maoni-github:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

And set it as the listener for your Maoni instance:
```java
    //Customize the maoni-github listener, with things like your user personal Access Token on Github
    final org.rm3l.maoni.github.MaoniGithubListener listenerForMaoni =
            new org.rm3l.maoni.github.MaoniGithubListener(...);

    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withListener(listenerForMaoni) //Callback from maoni-github
        //...
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling context
```

Visit the dedicated [README](callbacks/maoni-github/README.md) for further details.

##### [maoni-jira](callbacks/maoni-jira/README.md)

This callback sends feedback collected as a JIRA issue to a specified JIRA project.

You may want to create a dedicated `reporter` user on your JIRA Host for that purpose.

Add this additional line to your `build.gradle`:

```gradle
  dependencies {
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
    implementation('org.rm3l:maoni-jira:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

And set it as the listener for your Maoni instance:
```java
    //Customize the maoni-jira listener, with things like your REST URL, username, password
    final org.rm3l.maoni.jira.MaoniJiraListener listenerForMaoni =
            new org.rm3l.maoni.jira.MaoniJiraListener(...);

    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withListener(listenerForMaoni) //Callback from maoni-jira
        //...
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling context
```

Visit the dedicated [README](callbacks/maoni-jira/README.md) for further details.

##### [maoni-doorbell](callbacks/maoni-doorbell/README.md)

This callback sends feedback collected to [Doorbell](https://www.doorbell.io).

To use it, you will need to have an account there, and grab your application ID and secret key.

Add this additional line to your `build.gradle`:

```gradle
  dependencies {
    // ...
    implementation('org.rm3l:maoni:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
    implementation('org.rm3l:maoni-doorbell:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```

And set it as the listener for your Maoni instance:
```java
    final org.rm3l.maoni.doorbell.MaoniDoorbellListener doorbellListenerForMaoni =
            new org.rm3l.maoni.doorbell.MaoniDoorbellListener(...);

    new Maoni.Builder(MY_FILE_PROVIDER_AUTHORITY)
        .withListener(doorbellListenerForMaoni) //Callback from maoni-doorbell
        //...
        .build()
        .start(MaoniSampleMainActivity.this); //The screenshot captured is relative to this calling activity
```

Visit the dedicated [README](callbacks/maoni-doorbell/README.md) for further details.

#### Sharing the files captured with other apps

The file provider authority specified in the `Maoni.Builder` constructor allows you to
share the screenshot capture and logs files to other apps (depending on your callback implementation).
By default, Maoni stores the files captured in your application cache directory,
but this is (again) entirely customizable.

You must declare a file content provider in your `AndroidManifest.xml` file with an explicit list
of sharable directories for other apps to be able to read the screenshot file.
For example:

- If you use AndroidX:

```xml
<application>
    <!-- ... -->
    <!-- If not defined yet, declare a file provider to be able to share screenshots captured by Maoni -->
    <provider
        android:name="androidx.core.content.FileProvider"
        android:authorities="com.mydomain.fileprovider"
        android:grantUriPermissions="true"
        android:exported="false">
        <meta-data
            android:name="android.support.FILE_PROVIDER_PATHS"
            android:resource="@xml/filepaths" />
    </provider>
</application>
```

- Otherwise:

```xml
<application>
    <!-- ... -->
    <!-- If not defined yet, declare a file provider to be able to share screenshots captured by Maoni -->
    <provider
        android:name="android.support.v4.content.FileProvider"
        android:authorities="com.mydomain.fileprovider"
        android:grantUriPermissions="true"
        android:exported="false">
        <meta-data
            android:name="android.support.FILE_PROVIDER_PATHS"
            android:resource="@xml/filepaths" />
    </provider>
</application>
```

Along with the XML file that specifies the sharable directories (under `res/xml/filepaths.xml` as specified above):
```xml
<paths>
    <!-- By default, Maoni stores files captured (screenshots and logs) in the application cache directory.
    So you must declare the path '.' as shareable. Specify something else if you are using a different path -->
    <cache-path name="maoni-shares" path="." />
    <!-- <files-path path="maoni-working-dir/" name="myCustomWorkingDirForMaoni" /> -->
</paths>
```

See [https://goo.gl/31nStZ](https://goo.gl/31nStZ) for further instructions on how to setup file sharing.

## Contribute and Improve Maoni!

Contributions and issue reporting are more than welcome.
So to help out, do feel free to fork this repo and open up a pull request.
I'll review and merge your changes as quickly as possible.

You can use [GitHub issues](https://github.com/rm3l/maoni/issues) to report bugs.
However, please make sure your description is clear enough and has sufficient instructions
to be able to reproduce the issue.

You can also use the [sample app](https://play.google.com/store/apps/details?id=org.rm3l.maoni.sample)
to send your feedback with Maoni. ;-)

### Building from source

Make sure you have the Android SDK installed.

Also make sure you have the appropriate Build Tools installed. You can install them via the Android's `sdkmanager`:
```bash
sdkmanager "build-tools;28.0.3"
```

Now you can build the project with the `Gradle Wrapper`:
```bash
./gradlew lintDebug testDebug assembleDebug
```

You will then find the artifacts under the following folders:
- `maoni/build/outputs/aar/`
- `maoni-common/build/libs/`
- `maoni-sample/build/outputs/apk/`

### Translations

I use Crowdin as the translation system. All related resources for localization are automatically generated from files got with Crowdin.

To help out with any translation, please head to [Crowdin](http://crowdin.net/project/maoni)
and request to join the translation team.
If your language is not listed there, just drop me an e-mail at <apps+maoni@rm3l.org>.

Please do **not** submit GitHub pull requests with translation fixes as any changes will be overwritten
with the next update from Crowdin.

### Contributing callbacks for Maoni

You can create separate Android Library Projects that implement any of the Maoni callbacks interfaces
(`Validator`, `Listener`, `UiListener`, `Handler` or any combination),
so users can use them in their projects.

You just have to include `maoni-common` as a dependency in your project, e.g., with Gradle:

```gradle
  dependencies {
    // ...
    api('org.rm3l:maoni-common:10.0.0@aar') {
        transitive = true
        //Needed because of https://github.com/rm3l/maoni/issues/294
        exclude module: 'unspecified'
    }
  }
```
You can write your project in any JVM language of your choice (e.g., [Kotlin](https://kotlinlang.org/), as with [maoni-slack](https://github.com/rm3l/maoni-slack) and [maoni-github](https://github.com/rm3l/maoni-github)), as long as the callback implementation can be called from Maoni.

### Publishing a new release

All releases (Git tags) are published to [Maven Central](https://search.maven.org/search?q=g:org.rm3l) via [Sonatype](https://oss.sonatype.org/#welcome).

The `.github/workflows/build.yml` Workflow file contains a Job responsible for publishing libraries to Sonatype whenever a new tag is pushed.

Alternatively, this operation may be performed manually.
To do so, you can update or create a `local.properties` file (local only, **not** under version control) file at the root of this project.
This file should contain at least the following properties:

- `signing.keyId` : the GPG Signing Key ID
- `signing.secretKeyRingFile` : the path to the GPG signing key file, to use for signing files uploaded to Maven Central
- `signing.password` : the GPG signing key password
- `ossrhUsername` : the Sonatype Nexus Repository username
- `ossrhPassword` : the Sonatype Nexus Repository user password
- `sonatypeStagingProfileId`: the Staging Repo Profile ID (ask one maintainer to provide such information)

The following command can then be run to publish a new version:

```bash
./gradlew javadoc publishToSonatype closeAndReleaseStagingRepository
```

## In use in the following apps

(If you use Maoni, please drop me a line at <apps+maoni@rm3l.org>
(or again, fork, modify this file and submit a pull request), so I can list your app(s) here)

* [DD-WRT Companion](https://ddwrt-companion.app)
* [DD-WRT Companion Tasker Plugin](https://play.google.com/store/apps/details?id=org.rm3l.ddwrt.tasker)
* [Androcker](https://play.google.com/store/apps/details?id=org.rm3l.container_companion)

## Credits

* [DrawableView](https://github.com/PaNaVTEC/DrawableView), by [Christian Panadero Martinez](https://github.com/PaNaVTEC)

## Developed by

* Armel Soro
  * [keybase.io/rm3l](https://keybase.io/rm3l)
  * [rm3l.org](https://rm3l.org) - <apps+maoni@rm3l.org> - [@rm3l](https://twitter.com/rm3l)
  * [paypal.me/rm3l](https://paypal.me/rm3l)
  * [coinbase.com/rm3l](https://www.coinbase.com/rm3l)

## Contributors

Thanks to the following people who help(ed) improve Maoni,
either by suggesting translations or by reporting an issue and/or submitting pull requests.

In no particular order:

Marius Volkhart
💻 🐛

Lean Rada
💻

Isamar Castillo
🌍

omersurer
🌍

ihtiht
🌍

naofum
🌍

fuzeh
🌍

Apsimati
🌍

Dimitar Dinchev
💻 🐛

vlad-roid
💻 🐛

Gabriel Miklós
💻

Dennis Deng
💻

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind are welcome!

## Sponsors

Special thanks to [JetBrains](https://www.jetbrains.com/) for supporting this project by providing us with a free license of [IntelliJ IDEA](https://www.jetbrains.com/idea/).

![JetBrains Logo (Main) logo](https://resources.jetbrains.com/storage/products/company/brand/logos/jb_beam.svg)

## License

    The MIT License (MIT)

    Copyright (c) 2016-2022 Armel Soro

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE.


# light-example-4j
Example APIs to demo all feature of the light-4j and frameworks built on top of light-4j.

[Stack Overflow](https://stackoverflow.com/questions/tagged/light-4j) |
[Google Group](https://groups.google.com/forum/#!forum/light-4j) |
[Gitter Chat](https://gitter.im/networknt/light-4j) |
[Subreddit](https://www.reddit.com/r/lightapi/) |
[Youtube Channel](https://www.youtube.com/channel/UCHCRMWJVXw8iB7zKxF55Byw) |
[Documentation](https://doc.networknt.com) |
[Contribution Guide](https://doc.networknt.com/contribute/) |


- [Tutorials](#tutorials)
  * [Setup](#setup)
  * [Add code for a new tutorial](#add-code-for-a-new-tutorial)
  * [Using tutorial author tools](#using-tutorial-author-tools)
  * [Description of tutorial parts](#description-of-tutorial-parts)
  * [Add a narrative and test for the tutorial](#add-a-narrative-and-test-for-the-tutorial)
  * [Testing Locally](#testing-locally)
  * [Updating dependency versions](#updating-dependency-versions)
  * [Updating Tutorials](#updating-tutorials)

# Tutorials

This GitHub repo has the source code for Kafka [Tutorials](https://developer.confluent.io/tutorials/). Read about it [in our blog post](https://www.confluent.io/blog/announcing-apache-kafka-tutorials).

## Setup

If you want to hack on this site to add a new tutorial or make a change, follow these instructions.

### Prerequisites

The following prerequisites are **_only_** required if you are going to run the microsite locally.
If you are interested in testing tutorials locally see the [Testing Locally](#testing-locally) section of the README.

- ruby 2.3 or later
- [bundler](https://bundler.io/)
- npm
- python3 / pip3
- gradle
- [Docker Compose](https://docs.docker.com/compose/)

On a Mac, you can get the dependencies like this:

```
brew install ruby node
gem install bundler -v 2.4.22
```

You'll now have an executable called harness-runner on your path. (Note that if you use Python, you likely already have the `pyyaml` package installed.)

### Installing

#### 1. Clone this repository

```
git clone git@github.com:confluentinc/kafka-tutorials.git
```

Then `cd` into the directory.

#### 2. Install the node packages

```
npm install
```

This will bring in some external JavaScript and CSS packages that we're using.

#### 3. Install the gems

```
bundle install
```

This will install Jekyll itself and any other gems that we use.

#### 4. Run the development server

```
bundle exec jekyll serve --livereload --baseurl /tutorials
```

This will launch a web server so that you can work on the site locally. Check it out on `http://localhost:4000/tutorials/`.

#### 5. Install the Pip package

This repository uses a Python package to facilitate testing the tutorials. To keep things simple, we bundled it into this repository. You can get everything you need by running the following:

```
pip3 install pyyaml
cd harness_runner
pip3 install -e .
```

#### 6. Running the tests locally

You can run any of the automated tests locally using a command like:

```
cd ..
make -C _includes/tutorials/filtering/ksql/code tutorial
```

Substituting in the appropriate tutorial path.

Note: if you are testing unreleased ksqlDB Docker images, you will need to be logged in to AWS ECR.

## Add code for a new tutorial

A tutorial is a short procedure, targeted at developers, for getting a certain thing done using Confluent Platform.

In many cases, you can get that thing done using one of several _stacks_. For example, you might be able to perform data filtering by writing a KSQL query, by writing a Kafka Streams application, or by directly using the Kafka Consumer API. These comprise the three stacks this site supports: `ksql`, `kstreams`, and `kafka`.

These Tutorials are a bit unique in that each tutorial is self-testing. That is, we have built a light-weight harness system that's able to instrument the code that belongs to each tutorial to make sure that it actually works. This is really useful as we expect to have a lot of tutorials.

With in each stack, these tutorials contain a few pieces. These are described below.

## Using tutorial author tools

There are several pieces that you will put together for a tutorial.  Aside from the original content you need to provide (tutorial description, actual tutorial content, code, etc.), all of the steps described below can be automated.  You accomplish tutorial automation via one of two scripts in the `tools` directory

  1. `gen_project.sh`
  2. `clone_tutorial.sh`

Let's describe each script.

### The `gen_project.sh` script

As the name implies, the `gen_project.sh` script generates the minimal structure for a viable tutorial.  The code and text contained in the tutorial are place holders, and you'll update those with your code and writing to complete the tutorial.

The script expects you to supply a properties file for setting the name and other parts of the tutorial.  Here's an example properties file you can use.
```sh
#!/bin/sh

# space-separated list of types could be one of ksql or kstreams
CARDS="kstreams ksql"

# all lower-case with no spaces
TUTORIAL_SHORT_NAME=my-tutorial-name

# the MAIN_CLASS variable will generate a test named MAIN_CLASSTest
MAIN_CLASS=FilteringCode
AK_VERSION=3.1.0
CP_VERSION=7.3.0
KSQLDB_VERSION=0.28.2
SEMAPHORE_TEST_NAME="Test name for Semaphore run"
PERMALINK="seo-friendly-link-to-my-tutorial"
```

To run this script:

1. Create a branch
2. Make sure you are in the `kafka-tutorials` base directory
3. Execute `./tools/gen_project.sh my_tutorial_props.sh`

You'll see a lot of information scroll across the screen, describing each step of the tutorial generation process. The last part of the information presented is a checklist of what you'll need to do to complete your tutorial, aside from adding your code and tutorial text.

```
Your tutorial, my-tutorial-name, has been generated successfully!
The script adds a copy of this checklist to the my-tutorial-name directory.
```

There are some additional steps you'll need to take to complete the tutorial:

1. Update the following entries in _data/tutorials.yaml file
    a. title
    b. meta-description
    c. canonical (optional: preferred version of a page)
    d. question
    e. introduction

You can find these fields by searching for my-tutorial-name in the _data/tutorials.yaml file.

2. Add the new tutorial to the tutorials/recipes landing page in Contentful.

3. For completeness, update the link text then add the following link(s) to the [index.html](index.html) file in the appropriate section:

```
   <li><a href="seo-friendly-link-to-my-tutorial/kstreams.html">MEANINGFUL LINK TEXT HERE</a></li>
   <li><a href="seo-friendly-link-to-my-tutorial/ksql.html">MEANINGFUL LINK TEXT HERE</a></li>
```

If you only specified one tutorial type (`ksql` or `kstreams`) then you'd only have one link in the output.  Also, a copy of this output is copied in your tutorial directory `/Tutorials base dir/_includes/tutorials/you_tutorial_short_name`

### The `clone-tutorial.sh` script

As the name implies, this script creates a clone of an existing tutorial.  The clone script changes the name of the tutorial throughout the content.  To clone a tutorial, you also need to provide a properties file

```sh
#!/bin/sh

#SAMPLE-PROPERTIES-FILE-CONTENT

# By using the basename of a tutorial everything is cloned possibly ksql, kstreams, and kafka
ORIG_TUTORIAL=connect-add-key-to-source

# to only clone a kafka tutorial
ORIG_TUTORIAL=console-consumer-producer-basic/kafka

# to clone just the ksql part
ORIG_TUTORIAL=connect-add-key-to-source/ksql

# to clone just the kstreams portion
ORIG_TUTORIAL=connect-add-key-to-source/kstreams

!!!! IMPORTANT YOU MUST ONLY HAVE ONE ORIG_TUTORIAL VARIABLE !!!!

# Add a new tutorial name
NEW_TUTORIAL=junkA
# Add a new sempahore test name
SEMAPHORE_TEST_NAME="My New Junk Tutorial"
# Add a new PERMALINK
PERMALINK=a-junk-tutorial
```

To clone a tutorial:

1. Create a branch
2. Make sure you are in the `kafka-tutorials` base directory
3. Execute `./tools/clone_tutorial.sh my-clone-tutorial-props.sh`

You'll see a similar output scroll across the screen, including the checklist for items you'll need to do for a completed tutorial.

### Which script to use?

How do you decide which script to run?  If you are creating a new tutorial that does not resemble an existing tutorial, then the `gen_project` script is probably the better way to go.  If you are creating a tutorial that is closely related to a current tutorial, a hopping-windows tutorial, when there is already a tumbling-windows tutorial, for example, then the clone approach is probably better.

It's still valuable to read through the next section to learn how all the tutorial pieces fit together.

## Description of tutorial parts

_If you used the clone script then many of these will already exist and will just need customising for your particular tutorial._

#### 1. Describe the question your tutorial answers

The first thing to do is articulate what question your tutorial is meant to answer. Every tutorial contains a question and an example scenario. Edit `_data/tutorials.yml` and add your entry. The top item in this file represents the _short name_ for your tutorial. For example, the tutorial for transforming events of a stream is _transforming_. You'll also notice a `status` attribute. You can `enable` as many stacks as you'd like to author for this tutorial, but we recommend starting with just one.

#### 2. Make the directory structure

Next, make a few directories to set up for the project:

```
mkdir _includes/tutorials/<your tutorial short name>/<stack>/code
mkdir _includes/tutorials/<your tutorial short name>/<stack>/markup
```

#### 3. Write the code for the tutorial

Add your code for the tutorial under the `code/` directory you created. This should be entirely self-contained and executable with a `docker-compose.yml` file and a platform-appropriate build. Follow the conventions of existing tutorials as closely as possible.

At this point, you should feel free to submit a PR! A member of Confluent will take care of writing the markup and test files to integrate your code into the site. You can, of course, proceed to the next section and do it yourself, if you'd like.

## Add a narrative and test for the tutorial

This section is generally for those who work at Confluent and will be integrating new tutorials into the site. We need to do a little more work than just authoring the code. We also need to write the markup to describe the tutorial in narrative form, and also write the tests that we described to make sure it all works. This section describes how to do that.

#### 1. Create a harness for the tutorial

The harness is the main data structure that allows us to both test and render a tutorial from one form. Make a new directory under `_data/harnesses/` for your tutorial slug name and stack, like `_data/harnesses/<your tutorial short name>/ksql.yml`. Follow the existing harnesses to get a feel for what this looks like. The main thing to notice is that each step has a `render` attribute that points to a file. Create the markup for this in the next section.

#### 2. Create markup for the tutorial

Under the `markup/` directory that was created earlier, create 3 subdirectories: `dev`, `test`, and `prod`. Write the tutorial prose content here, following the conventions of existing tutorials. These files should be authored in Asciidoc.

#### 3. Tie it all together

Make a file named `/tutorials/<your tutorial short name>/<stack>.html`, specifying all the variables of interest. Note: the directory structure for these files is distinct from `/_includes/tutorials`.

For example, to display the tutorial with the ksqlDB stack:

```yml
# /tutorials/filter-a-stream-of-events/ksql.html
---
layout: tutorial
permalink: /tutorials/filter-a-stream-of-events/ksql
stack: ksql
static_data: filtering
---
```

You can do the same for Kafka Streams and Kafka, by using the `kstreams` and `kafka` stacks, respectively.

#### 4. Add your tutorial into build system

Lastly, create a Makefile in the `code` directory to invoke the harness runner and check any outputs that it produces. Then modify the `.semaphore/semaphore.yml` file to invoke that Makefile. This will make sure your tutorial gets checked by the CI system.

## Testing Locally

Each tutorial shows how to manually execute each tutorial step-by-step.
However, there are some scenarios when a user may want to run and test a tutorial in a more automated fashion:

- End-to-end: user makes a small change to the code and wants to validate that the tutorial still works end-to-end
- Run-and-play: user runs a tutorial and wants to leave it running to play with the environment

This section describes how you can do either of these scenarios using the `harness-runner` to programmatically run a single tutorial.

### Prerequisites

The following prerequisites are required if you are going to run a tutorial programmatically.

- python3 / pip3
- gradle
- [Docker Compose](https://docs.docker.com/compose/)

### Environment Setup

1. Check out the kafka-tutorials GitHub repo:

```bash
git clone https://github.com/confluentinc/kafka-tutorials.git
cd kafka-tutorials
```

2. Install the packages for the harness runner.

If you have `pip3` installed locally:

```bash
(cd harness_runner/ && pip3 install -e .)
```

If you don't have `pip3` installed locally, create a `Dockerfile` with the following content:

```text
FROM python:3.7-slim
RUN pip3 install pyyaml
```

and then run the following command to build and execute the Docker image:

```
docker build -t runner . ; docker run -v ${PWD}/harness_runner:/harness_runner/ -it --rm runner bash -c 'cd /harness_runner/ && pip3 install -e .'
```

3. Install [gradle](https://gradle.org/install/) for tutorials that compile any code.

4. Install Docker Compose

### Run a tutorial

1. (optional) If you want to augment or override a tutorial's Docker environment, set the Docker Compose CLI environment variable `COMPOSE_FILE` to include `docker-compose.yml` and **_the absolute path_** to a `docker-compose.override.yml` file.  For example, to use Confluent Control Center with any tutorial, set `COMPOSE_FILE` to `docker-compose.yml` and the absolute path to [this docker-compose.override.yml](tools/docker-compose.override.yml).

```
export COMPOSE_FILE=docker-compose.yml:<path to tutorials>/tools/docker-compose.override.yml
```

2. End-to-end: execute the harness runner for a single tutorial by calling `make`, across all `dev`, `test`, and `prod` stages, to validate it works end-to-end. Identify the tutorial you want and then run `make`. Note that this destroys all the resources and Docker containers it created, so it cleans up after itself.  Format: `(cd _includes/tutorials/<tutorial name>/<type>/code && make)` where type is one of `ksql | kstreams | kafka`. Example:

```
(cd _includes/tutorials/transforming/kstreams/code/ && make)
```

3. Run-and-play: execute the harness runner for a single tutorial by calling `make SEQUENCE='"dev, test"'`, just across `dev` and `test` stages, which leaves all resources and Docker containers running so you can then play with it.  Format: `(cd _includes/tutorials/<tutorial name>/<type>/code && make SEQUENCE='"dev, test"')` where type is one of `ksql | kstreams | kafka`. Example:

```
(cd _includes/tutorials/transforming/kstreams/code/ && make SEQUENCE='"dev, test"')
```

Now you can play with the environment, some sample commands shown below.

```
docker exec -t broker kafka-topics --list --bootstrap-server localhost:9092
docker exec -it ksqldb-cli ksql http://ksqldb-server:8088
```

Because the Docker containers are left running, don't forget to clean up when you are done.

```
docker container ls
docker container rm -f <container id>
```

### Makefile Details

The `Makefile` will delete and re-create the `outputs` directory used to contain files with output from various steps used to verify the tutorial steps.

Here is [the contents of an actual `Makefile`](_includes/tutorials/fk-joins/kstreams/code/Makefile) :

```yml
STEPS_DIR := tutorial-steps
DEV_OUTPUTS_DIR := $(STEPS_DIR)/dev/outputs
TEMP_DIR := $(shell mktemp -d)

tutorial:
  rm -r $(DEV_OUTPUTS_DIR) || true
  mkdir $(DEV_OUTPUTS_DIR)
  harness-runner ../../../../../_data/harnesses/fk-joins/kstreams.yml $(TEMP_DIR)
  diff --strip-trailing-cr $(STEPS_DIR)/dev/expected-output-events.json $(DEV_OUTPUTS_DIR)/music-interest.json
```

The last line uses the `diff` command to validate expected output against the tutorial steps' actual output.  The `Makefile` may have more than one validation action to have separate `diff` commands for each verification.

For example, here's the `Makefile` from the [dynamic output topics tutorial](_includes/tutorials/dynamic-output-topic/kstreams/code/Makefile)

```yml
STEPS_DIR := tutorial-steps
DEV_OUTPUTS_DIR := $(STEPS_DIR)/dev/outputs
TEMP_DIR := $(shell mktemp -d)

tutorial:
  rm -r $(DEV_OUTPUTS_DIR) || true
  mkdir $(DEV_OUTPUTS_DIR)
  harness-runner ../../../../../_data/harnesses/dynamic-output-topic/kstreams.yml $(TEMP_DIR)
  diff --strip-trailing-cr $(STEPS_DIR)/dev/expected-output.json $(DEV_OUTPUTS_DIR)/actual-output.json
  diff --strip-trailing-cr $(STEPS_DIR)/dev/expected-special-output.json $(DEV_OUTPUTS_DIR)/actual-special-order-output.json
```

### Harness Details

Given that the test harness is the `heart` of a tutorial, it will be helpful to describe in detail how to work with a `kafka|ksql|kstreams.yml` file.  You should note the harness file is in the [YAML file format](https://en.wikipedia.org/wiki/YAML), so formatting properly is essential.  The harness files generate the rendered tutorial structure and validate any output of tutorial steps against expected values.

New tutorial authors should not need to create a harness file from scratch, using either the `tools/gen_project.sh` or  `tools/clone.sh` script will provide a usable harness file.  This section should provide enough guidance to add, update, or remove parts as needed.

#### 1. Structure

Three top-level sections make up the harness file:

* dev - the setup and teaching portion of the tutorial (required)
* test - test setup and execution of tests, if any (optional)
* prod - steps to build and deploy a docker image of the tutorial code (optional)

In some cases, having a test and/or prod section doesn't make sense, so you can omit those portions of the harness file.  The Apache Kafka [console producer and consumer basic operations](_data/harnesses/console-consumer-producer-basic/kafka.yml) and the Apache Kafka [console consumer with primitive keys and values](_data/harnesses/console-consumer-primitive-keys-values/kafka.yml) tutorials are an excellent example of tutorials that don't need a test or prod section.

The `dev`, `test`, and `prod` sections contain a top-level element `steps`.  The `steps` contains any number of well, steps for the user to walk through.  Additionally, the `harness_runner` script follows the same steps for executing the tutorial automatically during builds.  All sections contain the same step structure, so we'll only discuss the make-up of a single section.

For reference here is an example section of the harness file from the [console consumer primitive keys and values tutorial](_data/harnesses/console-consumer-primitive-keys-values/kafka.yml)

```yml
 - title: Get Confluent Platform
      content:
        - action: make_file
          file: docker-compose.yml
          render:
            file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/make-docker-compose.adoc

        - action: execute_async
          file: tutorial-steps/dev/docker-compose-up.sh
          render:
            file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/start-compose.adoc

        - action: execute
          file: tutorial-steps/dev/wait-for-containers.sh
          render:
            skip: true

        - name: wait for ksqldb-server and connectors
          action: sleep
          ms: 30000
          render:
            skip: true
```

* Title - each section starts with a `title` element, and as the label suggests, the text provided here is the text used the label the step for the tutorial user and the output to the console by the harness runner.  The `title` section contains one element - `content`

* content - the `content` section (a YAML `dictionary`) contains an arbitrary sized list of YAML dictionaries named `action`. An `action` key creates an anonymous step, i.e., not specified in the test runner's output.  For output in the test runner, you can provide a `-name` key, followed by some text for console output.
* action - `action` keys drive the behavior of the harness. An action key can be one of these values
    * `action: make_file` - Prompts the user to create a file for the tutorial.
    * `action: execute` - a synchronous action step
    * `action: execute_async` - an asynchronous step, this indicates a step the user will keep running for some portion of the tutorial.
    * `action: sleep` - pause the test runner for an amount of time specified by the `ms` key. You use `sleep` key to allow some async action to complete
        * `ms: NNN` - the time in milliseconds you want the test harness to pause execution.  You only use `ms` after an `action: sleep` entry.
    * `docker_ksql_cli_session` - an action starting a ksqlDB CLI session for working through a tutorial

In the next sections, you'll see how to use `action` keys to organize your harness files.

#### 2. Action type descriptions and examples

* `make_file`
    The `make_file` instructs the user to create a file an existing file required to run the tutorial.  Some examples are the `docker-compose.yml` file, a `statements.sql` (ksqlDB), and Java files.

    A `make_file` with a file to render look like this:
    ```yml
    - action: make_file
          file: docker-compose.yml
          render:
            file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/make-docker-compose.adoc
    ```

    * `file:` the path and name to the file.  The path is relative to `<tutorial-name>/<type>/code>` .The harness runner will use the file during the automated tutorial execution.
    * `render:` the file to render.  The `render` key has one of the two possible keys:
        * `file:` the path and name of the file to render to the user.

* `execute` The `execute` step is a synchronous execution step during the tutorial for the user and the harness runner

    ```yml
     - action: execute
          file: tutorial-steps/dev/init.sh
          render:
            file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/init.adoc
    ```

    Sometimes the `execute` step is an _internal_ step only for the harness runner. Internal steps ignore the `render:` using a `skip: true` entry
    ```yml
      - name: wait for ksqldb-server and connectors
          action: sleep
          ms: 30000
          render:
            skip: true
    ```

    The `execute` steps may also capture output from stdout for tests executed by the harness runner
    ```yml
    - title: Invoke the tests
      content:
        - action: execute
          file: tutorial-steps/test/run-tests.sh
          stdout: tutorial-steps/test/outputs/test-results.log
          render:
            file: tutorials/aggregating-sum/ksql/markup/test/run-tests.adoc
    ```
    Here you can see the `stdout:`  key specifying the file used to capture the output of an `execute` step.

* `execute_async` The `execute_async` step is for steps needing to execute in the background while the user continues going through the tutorial

     Here's an example of running `docker compose` to start docker containers to run the duration of the tutorial

     ```yml
      - action: execute_async
            file: tutorial-steps/dev/docker-compose-up.sh
            render:
              file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/start-compose.adoc
     ```

    Here's another example of using a console consumer. Note the use of `stdout:` to capture the output from the consumer for testing.
    ```yml
     - title: Start an initial console consumer
        content:
          - action: execute_async
            file: tutorial-steps/dev/harness-console-consumer-keys.sh
            stdout: tutorial-steps/dev/outputs/actual-output-step-one.txt
            render:
              file: tutorials/console-consumer-primitive-keys-values/kafka/markup/dev/consume-topic-no-deserializers.adoc
    ```

* `docker_ksql_cli_session` The `docker_ksql_cli_session` sets up ksqlDB tutorial users to start a ksqlDB CLI session so the user can execute various `SQL` files to complete the tutorial.
    ```yml
      - action: docker_ksql_cli_session
          container: ksqldb-cli
          docker_bootup_file: tutorial-steps/dev/start-cli.sh
          column_width: 20
          render:
            file: tutorials/aggregating-sum/ksql/markup/dev/start-cli.adoc
    ```
    The `docker_ksql_cli_session` contains the following keys:
    * `container:` - The name of the [ksqldb-cli docker image in the docker-compose.yml](_includes/tutorials/aggregating-sum/ksql/code/docker-compose.yml#L70-L82) file.
    * `docker_bootup_file:` - Tutorial users and the harness runner [execute this file](_includes/tutorials/aggregating-sum/ksql/code/tutorial-steps/dev/start-cli.sh) to start the dockerized CLI session.
    * `column_width:` - Formats the ksqlDB query output
    * `render:` - The harness renders the file  corresponding to the `file:` key to tutorial users with the command to start the CLI session.
    * `stdin:` - stdin key [contains one or more `file`](_data/harnesses/aggregating-sum/ksql.yml#L41-L68) keys specifying the different `SQL` file to execute for the tutorial.
    ```yml
        stdin:
            - file: tutorial-steps/dev/create-movie-ticket-sales.sql
              render:
                file: tutorials/aggregating-sum/ksql/markup/dev/create-movie-ticket-sales.adoc

            - file: tutorial-steps/dev/populate-movie-ticket-sales.sql
              render:
                file: tutorials/aggregating-sum/ksql/markup/dev/populate-movie-ticket-sales.adoc
    ```
    * `stdout:` - Contains `directory:` specifying the directory to capture all query output.

## Updating dependency versions

The following regular expressions may be useful to group-update all dependencies within the repo:

* `confluentinc\/cp-([^:]*):\d+.\d+.\d+` will match all Confluent Platform components, _except_ ksqlDB.
   Capture group 1 can be used to build the component name.
* `confluentinc\/ksqldb-([^:]*):\d+.\d+.\d+` will match all ksqlDB components.
   Capture group 1 can be used to build the component name.

## Updating Tutorials

Confluent manages the release process and the process described here must be done by a Confluent employee.

The `release` branch tracks the content and code comprising the live site.

The `ksqldb-latest` branch builds against the latest `master` branch of ksqlDB, and should be used for updates that are only in the master branch of ksqlDB.

### Create a release PR

1. Open a pull request from `master` to `release`. A pull request into the `release` branch denotes a request to update the live site.

- Title the PR “KT release [DD-MM-YYYY]”
- In the description add links to PRs that resulted in new tutorials, content changes, or any other noteworthy addition, e.g.:
```
This release contains:

New Tutorial! A fun new tutorial #1287
New recipe group for financial services use cases #1264
plus many other small fixes
```
- Add a link to the staging site with the updates (snag the staging site name from Semaphore's "Deploy to staging site" step)
- Tag reviewers
- Create PR

2. The semaphore tests for the PR will automatically create the staging site. You can manually click to deploy to staging site if you trust that failing tests work (ksqlDB tests can be flaky sometimes).

![image](assets/img/manual-deploy-staging.png)

3. Look at the staging site from different browsers. Ensure that you get approval from required reviewers. Submit PRs with fixes, if needed.

4. Once the PR is approved, merge the PR via `Merge commit`.

### Deploy to live site

To deploy artifacts from the `release` branch to the live site, it should be done from the `release` branch, not the above PR.  So do not `deploy to live site` from the PR, because that would deploy from `master`.  Instead, go to the `release` branch, let the tests pass, and then deploy

![image](assets/img/release-promote.png)

### Landing page

The landing page is at https://developer.confluent.io/tutorials/ (it is not [index.html](index.html)), and the source for it is in Contentful.  Any changes to the landing page should be worked through Confluent.


# Industrial Foregoing
[![Curse Forge](http://cf.way2muchnoise.eu/full_industrial-foregoing_downloads.svg?badge_style=for_the_badge)](https://minecraft.curseforge.com/projects/industrial-foregoing)
[![Curse Forge](http://cf.way2muchnoise.eu/versions/industrial-foregoing.svg?badge_style=for_the_badge)](https://minecraft.curseforge.com/projects/industrial-foregoing)
[![Discord](https://img.shields.io/discord/102860784329052160.svg?style=for-the-badge&logo=discord)](https://discord.gg/4tPfwjn)
[![](https://img.shields.io/github/contributors/Buuz135/Industrial-Foregoing.svg?style=for-the-badge&logo=github)](https://github.com/Buuz135/Industrial-Foregoing/graphs/contributors)
[![](https://img.shields.io/github/issues/Buuz135/Industrial-Foregoing.svg?style=for-the-badge&logo=github)](https://github.com/Buuz135/Industrial-Foregoing/issues)
[![](https://img.shields.io/github/issues-pr/Buuz135/Industrial-Foregoing.svg?style=for-the-badge&logo=github)](https://github.com/Buuz135/Industrial-Foregoing/pulls)
[![](https://img.shields.io/github/forks/Buuz135/Industrial-Foregoing.svg?style=for-the-badge&logo=github)](https://github.com/Buuz135/Industrial-Foregoing/network/members)
[![](https://img.shields.io/github/stars/Buuz135/Industrial-Foregoing.svg?style=for-the-badge&logo=github)](https://github.com/Buuz135/Industrial-Foregoing/stargazers)
[![](https://img.shields.io/github/license/Buuz135/Industrial-Foregoing.svg?logo=github&style=for-the-badge)](https://github.com/Buuz135/Industrial-Foregoing/blob/master/LICENSE)

Automating the unautomatable

## Contact [![Discord](https://img.shields.io/discord/102860784329052160.svg?style=for-the-badge&logo=discord)](https://discord.gg/4tPfwjn)
Join my discord or follow me on twitter `@Buuz135mods`

## About
Textures by CyanideX and the Unity team: https://mods.curse.com/texture-packs/minecraft/222097-unity


# RepoSense

[![Build Status](https://github.com/reposense/RepoSense/actions/workflows/integration.yml/badge.svg)](https://github.com/reposense/RepoSense/actions/workflows/integration.yml)
[![codecov.io](https://codecov.io/gh/reposense/RepoSense/branch/master/graphs/badge.svg?branch=master)](http://codecov.io/github/reposense/RepoSense?branch=master)
[![Netlify Status](https://api.netlify.com/api/v1/badges/260983b3-589e-4619-a2e8-0bfb7a2b4422/deploy-status)](https://app.netlify.com/sites/reposense/deploys)

RepoSense is a contribution analysis tool for Git repositories. It is particularly useful for those who want a chronological visualisation of the code contributions and commits made to a group of repositories. Educators can use RepoSense to track the work done in class projects, while team leads may use it to oversee their project teams.

- **User documentation** i.e., if you are interested in using RepoSense
  - [**Product website**](https://reposense.org)
  - [**User guide**](https://reposense.org/ug/index.html)
  - [**About us**](https://reposense.org/about.html)
- **Developer documentation** i.e., if you are interested in contributing code to RepoSense
  - [Developer guide](https://reposense.github.io/RepoSense/dg/index.html)
  - [User Guide for the latest `master` (not yet released to users)](https://reposense.github.io/RepoSense)

---
### Our Contributors :

**Acknowledgements**: The web previews of RepoSense is powered by Netlify and Surge.