garment-detector-seg

Instance segmentation of garments, accessories, and garment components in photographs. RF-DETR-Seg (Apache 2.0) fine-tuned on Fashionpedia (CC BY 4.0), predicting all 46 Fashionpedia classes: 13 main garments (shirt/blouse … cape), 14 accessories, and 19 garment parts (collar, lapel, sleeve, pocket, neckline, zipper, buckle, and trim classes such as rivet, ruffle, sequin).

Built for wholesale apparel manufacturing QC, not e-commerce tagging: the segmentation masks feed deterministic color measurement, and the part classes map to BOM trim lines and construction details in a tech pack. It is one of two models in a pipeline (see garment-attributes for fine-grained construction attributes).

Intended use

• Locate garments and their visible components in sample/production photos.
• Provide masks for downstream color QC (ΔE2000 against a spec target).
• Verify component presence/count against a tech pack (e.g. "2 flap pockets, 1 center-front zipper, no hood").

Out of scope: fiber content, GSM, measurements, stitch class, interior construction — none of these are visually determinable and this model does not pretend to output them.

Training

• Base: RF-DETR-Seg-Small (Apache 2.0), DINOv2 backbone, 33.6M params.
• Data: Fashionpedia train2020 (45,623 images, 333,401 instances), converted by scripts/prepare_detector_data.py (attributes stripped; masks kept).
• Recipe: scripts/train_detector.py with configs/detector_mac.yaml (8 epochs, effective batch 16, lr 1e-4, fp16 AMP, trained on Apple-silicon MPS in ~37 hours).

Evaluation (Fashionpedia val2020, 1,158 images, epoch 8)

Metric	Value
mask mAP@[.5:.95]	0.352
mask mAP@.5	0.502
box mAP@[.5:.95]	0.419
box mAP@.5	0.549
box mAP, main garments (ids 0–12)	0.589
box mAP, accessories (ids 13–26)	0.534
box mAP, garment parts (ids 27–45)	0.204

Strongest classes: dress (0.87), pants (0.84), jacket and coat (approx. 0.75). Weakest: small trims — zipper (0.10), beads/appliqué (0.05) — and rare categories (cape, jumpsuit). Full per-class table ships in this repo.

Small trim classes (rivet, bead, sequin) score materially lower than garment classes in all published Fashionpedia baselines; expect the same here and check the per-class table in runs/detector before relying on a trim class for QC decisions.

Known limitations & biases

• Fashionpedia images are street/celebrity photos of worn garments; performance on flat-lay or on-hanger factory photos is lower. Fine-tune on in-domain photos for production deployment (the training script supports any COCO-format dataset).
• Occluded components (e.g. back pockets in a front photo) are not detected — the downstream validator reports missing components as REVIEW, not FAIL, for this reason.
• Small trims (<32 px) are frequently missed at 1024-px inference resolution.
• Dataset skews toward Western womenswear street fashion; expect weaker performance on technical/workwear categories.

License & attribution

Weights: Apache 2.0. Training data: Fashionpedia, CC BY 4.0 — cite Jia et al., Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset, ECCV 2020. Architecture: RF-DETR (Roboflow, Apache 2.0), ICLR 2026.

Usage

from rfdetr import RFDETRSegSmall
model = RFDETRSegSmall(pretrain_weights="checkpoint_best_ema.pth")
detections = model.predict("sample.jpg", threshold=0.5)

Full pipeline (attributes + color + tech pack validation): see the garment-classifier repository this model ships with.

---

⚠ This card predates our model-card standard

It does not report: a baseline comparison, a measured failure rate, or what it refuses.

Every model we publish should state (1) the cheap baseline and the margin over it, (2) at least one named failure mode with a measured rate, (3) what the model refuses and why, and (4) label provenance. This card was written before that rule and has not been retrofitted, because the measurements needed no longer exist. Treat its numbers with more caution than our newer cards, and prefer a model that meets the standard where one exists.