You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
These weights are DERIVED from Meta's SAM3 (SAM License) and are provided for research/evaluation. Redistribution requires SAM License pass-through. Commercial/ production use requires a separate license from General Instinct, Inc.
Log in or Sign Up to review the conditions and access this model content.
InstinctSAM — Compressed SAM3 (commercial-clean components)
Compressed components for SAM3 open-vocabulary ("concept", text-prompted) instance
segmentation. This repo ships only our own distilled weights — a compressed ViT-B vision
trunk and commercial-clean GIText text towers — which you graft onto the (separately
licensed, Meta-gated) SAM3 heads. No Meta vision/head weights and no Apple/MobileCLIP
(apple-amlr) weights are redistributed here.
Metric: cgF1 on SA-Co/Gold = pmF1 × IL_MCC (official SAM3 leaderboard metric). Teacher (SAM3, 840M): cgF1 0.521 on our 6544-pair SA-1B/Gold eval.
The compression↔accuracy frontier (6544-pair SA-Co/Gold cgF1)
| variant | what's compressed | params | vs teacher | cgF1 | % teacher | commercial |
|---|---|---|---|---|---|---|
| teacher SAM3 | — | 840M | 1.0× | 0.521 | 100% | — |
| LiteText (GIText-large) | text 354M→63M | 551M | 1.5× | 0.493 | 94.6% | ✅ clean |
| LiteText (GIText-base) | text 354M→44M | 530M | 1.6× | 0.489 | 93.9% | ✅ clean |
| Hiera-L (vision-compressed) | vision 454M→150M (SAM2-Hiera-L) | 537M | 1.6× | 0.431 | 82.7% | ✅ clean |
| vision-compressed (ViT-B, earlier) | vision 454M→107M | 493M | 1.7× | 0.353 | 68% | ✅ clean |
| dual-compressed | vision + text | 182M | 4.6× | 0.246 | 47% | ✅ clean |
Key findings. (1) The text encoder is the "free" thing to compress — distilling it costs
~10% accuracy for a similar param saving, while vision compression costs ~32%. (2) Our
commercial-clean GIText text tower (a from-scratch CLIP-BPE transformer, our code, no Apple
dependency) matches and beats the apple-amlr MobileCLIP LiteText (0.493 vs 0.469) at 90–95%
of teacher, distilled from SAM3's PE-text on a broad open-vocab + RefCOCO prompt set. All variants
beat the released EfficientSAM3 (0.133 on this harness) by 1.9–3.7×.
Files
gitext_large_v4.pt— commercial-clean text tower, 63M (GIText-large; 0.493/94.6%).gitext_base_v3.pt— commercial-clean text tower, 44M (GIText-base, 0.489/93.9%).hiera_large_concept_trunk.pt— compressed vision trunk, SAM2-Hiera-L 150M (concept-distilled, 0.431/82.7% teacher) — the best vision-compression point; graft intomodel.backbone.vision_backbone.trunk. Preserves occlusion tracking (~teacher). Full pipeline (with the trunk TensorRT-compiled) measures 12.7 FPS on an idle A100 (vision+neck 29 ms / decode+mask 49 ms); the decode+mask head is the dominant floor, so ≥15 FPS at ≥80% needs a lighter decoder. Thor not yet measured — runscripts/bench_thor.pyon-device. Reproduction recipe + edge-speed detail:docs/HIERA_BACKBONE.mdin the GitHub repo.concept_vitb_trunk_step6000.pt— earlier ViT-B vision trunk (concept-distilled, 0.353/68%).vit_base_stageA.pt— ViT-B vision trunk (Stage-A feature distill).
Usage (sketch)
Build SAM3 (your own gated SAM3 checkpoint), swap in the GIText text tower for the recommended
"LiteText" variant. Full reproduction, training recipe, and eval harness:
https://github.com/william-Dic/InstinctSAM (see docs/MODEL_CARD.md, src/train_text.py,
src/eval_saco_cgf1.py).
⚠️ Licensing
- These weights are derived from SAM3 → SAM License with pass-through (research/eval OK; redistribute under the same terms + include the license).
- The GIText text tower architecture is our own (standard CLIP-BPE transformer) — no
apple-amlr/ MobileCLIP dependency, so it is commercially usable where SAM3's own license permits. - Commercial/production use of the combined system requires a separate license from General Instinct, Inc. — guanming@general-instinct.com.
- SA-1B (distillation data) — Meta research license; hold your own rights.