remixerdec/Inflect-Nano-v1-GGUF

Inflect-Nano GGUF Quantized Weights

This repository contains quantized GGUF weights of Inflect-Nano-v1 model for the InflectNanoTTS C++/GGML runtime. The original model was published by its author, owensong under Apache-2.0 license. Inflect-Nano-v1 aims to deliver ultra-lightweight english text to speech to edge devices.

The goal of this port was to run TTS on an ESP32 device and it has been achieved. With current implementation the model is too compute-heavy for practical use-cases on ESP32: with an alternative vocoder, processing a single word took 10 seconds, with default vocoder it is 7 times slower. There is still some room for runtime optimizations left: migrating ggml ops to esp_nn or esp_dsp.

Repository Contents

acoustic/
  inflect_acoustic_f16.gguf
  inflect_acoustic_q2_k.gguf
  inflect_acoustic_q3_k.gguf
  inflect_acoustic_q3_k_dynamic.gguf
  inflect_acoustic_q4_0.gguf
  inflect_acoustic_q4_0_E.gguf
  inflect_acoustic_q4_k.gguf
  inflect_acoustic_q5_0.gguf
  inflect_acoustic_q5_k.gguf
  inflect_acoustic_q6_k.gguf
  inflect_acoustic_q8_0.gguf
vocoder/
  inflect_vocoder_f16.gguf
  inflect_vocoder_q4_0.gguf
  inflect_vocoder_q4_0_E.gguf
  inflect_vocoder_q5_k.gguf
  inflect_vocoder_q8_0.gguf
cmudict.bin
cmudict.idx

Quality drop becomes significant lower than q3_k, others work relatively well.

The inflect_acoustic_q4_0_E.gguf and inflect_vocoder_q4_0_E.gguf files are experimental low-memory quants. They keep biases in fp16 and the last layer is also quantized to q4_0, making the pair small enough to fit in an ESP32's 8MB PSRAM.

Usage

Build the runtime from the InflectNanoTTS.cpp project:

./tools/build.sh

Run inference:

build/linux-arm64/inflect-nano \
  -a acoustic/inflect_acoustic_q4_0.gguf \
  -v vocoder/inflect_vocoder_q4_0.gguf \
  -d cmudict.bin \
  -t "Hello, this is a test." \
  -o output.wav

Adjust the binary path for your platform, for example build/macos-arm64/inflect-nano on Apple Silicon.

cmudict.bin is a compact binary pronunciation dictionary used by the text frontend.

cmudict.idx is an index file for cmudict.bin that makes dictionary initialization faster.

It is derived from CMU Pronouncing Dictionary: English words mapped to ARPAbet phoneme sequences. The project converts that text dictionary into a faster binary lookup format so the runtime can avoid parsing the full text dictionary at startup.

Its purpose is to turn English words into phoneme IDs before acoustic inference. In practice: input text -> normalize/tokenize -> dictionary lookup for pronunciations -> phoneme/tone/lang IDs -> acoustic model -> mel -> vocoder audio.

cmudict.bin was obtained by converting cmudict.rep from the original repo with this script:

python tools/compile_cmudict.py \
    third_party/tiny_tts_frontend/tiny_tts/text/cmudict.rep \
    cmudict.bin