File size: 2,697 Bytes
75c1b53 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
license: cc-by-4.0
language:
- en
- de
- es
- fr
library_name: coreml
tags:
- coreml
- apple-silicon
- ane
- automatic-speech-recognition
- canary
- attention-encoder-decoder
- aed
- fluidaudio
base_model: nvidia/canary-1b-v2
base_model_relation: quantized
pipeline_tag: automatic-speech-recognition
---
# Canary-1B-v2 — CoreML (ANE)
CoreML conversion of [`nvidia/canary-1b-v2`](https://huggingface.co/nvidia/canary-1b-v2)
for Apple Silicon / Neural Engine, packaged for [FluidAudio](https://github.com/FluidInference/FluidAudio).
Canary is a **FastConformer encoder + Transformer attention encoder-decoder (AED)**
ASR model (25 European languages, 16384-token SentencePiece BPE). It is decoded
autoregressively: the transformer decoder cross-attends to the encoder output and
emits tokens greedily until EOS (id 3), with a 1024→16384 projection head.
## Files
| File | Role | Precision |
|------|------|-----------|
| `Preprocessor.mlmodelc` | waveform `[1,240000]` → mel `[1,128,1501]` | fp32 (CPU) |
| `EncoderInt4.mlmodelc` | mel → encoder `[1,1024,188]` | int4 (ANE, iOS18) |
| `DecoderInt4.mlmodelc` | autoregressive transformer → hidden `[1,256,1024]` | int4 (ANE, iOS18) |
| `Projection.mlmodelc` | hidden `[1,1024]` → logits `[1,16384]` | fp16 (ANE) |
| `vocab.json` | 16384 SentencePiece pieces (`id → piece`) | — |
| `projection_weights.npz` | raw projection weights (for Python reference pipelines) | fp32 |
| `metadata.json` | shapes, sample rate, special token ids | — |
**Contract:** 15 s window (240000 samples @ 16 kHz), 256 decoder steps,
`eos=3`, `pad=2`, `bos=4`. int4 weight payloads require **iOS 18 / macOS 15**.
## Variants
- **int4** (this default): ANE-runnable, ~573 MB, fastest. Per-block-32 symmetric.
- **fp16**: exact parity with PyTorch, iOS 17, ~1.8 GB (not included here by default).
- int8 per-channel decodes correctly only on CPU (crashes the GPU/ANE MPSGraph
backend), so it is not recommended; use int4 for an ANE-resident small build.
## Accuracy / speed (LibriSpeech test-clean, ≤15 s, int4, M-series ANE)
| Metric | Value |
|--------|-------|
| WER | ~2.1% |
| RTFx | ~7x |
fp16 CoreML output is byte-identical to the NeMo PyTorch greedy decode.
## Usage (FluidAudio)
```swift
let manager = try await CanaryManager.load(precision: .int4)
let text = try await manager.transcribe(audioURL: url)
```
## Conversion
See the [mobius](https://github.com/FluidInference/mobius) conversion pipeline
(`models/stt/canary-1b-v2/coreml/`): `convert-coreml.py` (NeMo→CoreML),
`quantize_int4.py`, `build_projection.py`, `validate.py`, `stage_hf.py`.
## License
Inherits `cc-by-4.0` from the base model `nvidia/canary-1b-v2`.
|