Code mirror of https://github.com/rudzz1950/Virtual-Try-On
Fitroom: Virtual Try-On
Find your style, try clothes on live with your phone camera, and get an honest, explainable score for every outfit.
Fitroom is an Android app (Kotlin + Jetpack Compose). The live try-on, the style profile and the outfit rating all run on the device. An optional backend adds photoreal HD stills through Google Vertex AI Virtual Try-On.
Status: beta. The app builds, its 19 unit tests pass, and every screen has been checked on an Android emulator. Live try-on on a real person and import of real garment photos still need testing on physical devices before a store release. See Roadmap.
Features
| What it does | Where it runs | |
|---|---|---|
| Style quiz | 7 questions build a blended profile across 10 style archetypes (Classic, Minimalist, Casual, Streetwear, Sporty, Preppy, Bohemian, Edgy, Romantic, Glam), plus the occasions you dress for, your preferred fit and colour palette | On device |
| Occasions | A dress-code formality scale (casual β smart casual β business casual β cocktail β formal) mapped to Everyday, Active, Work, Date night, Party, Formal event and Travel | On device |
| Closet | Photograph your clothes: the background is removed (ML Kit subject segmentation, with a plain-background fallback), fit points such as shoulders, cuffs, hems and waist are detected automatically, and you can drag them to adjust | On device |
| Live try-on | Tops, trousers, shorts and skirts follow your body in real time: sleeves and legs bend with your arms and legs, lighting from the camera shades the garment, and your hands stay in front | On device |
| Outfit rating | A 0β100 score with plain-English reasons: occasion fit, colour harmony, match with your style, how the pieces go together, and proportions measured live (for example "this top runs long on you"). Suggests swaps from your closet that score higher | On device |
| Outfit ideas | Pairs your tops and bottoms and ranks the combinations for the occasion you pick | On device |
| Saved looks | Snapshot plus rating, with favourites | On device |
| HD photo | Optional photoreal still via Vertex AI Virtual Try-On, through your own backend. Cloud credentials stay on the server and never ship in the app | Cloud (opt-in) |
How the live try-on works
flowchart LR
CAM[CameraX frame] --> POSE[MediaPipe Pose<br/>33 landmarks + person mask]
POSE --> SMOOTH[One Euro filter]
SMOOTH --> BODY[Body frame<br/>shoulders, hips, joints]
G[Garment photo<br/>+ fit points] --> RIG[Split into parts<br/>torso / sleeves / legs / skirt]
BODY --> FIT[Fit solver]
RIG --> FIT
FIT -->|torso, skirt| TPS[Thin-plate spline warp]
FIT -->|sleeves, legs| STRIP[Bend along arm / leg bones]
TPS --> GL[OpenGL ES 3 renderer]
STRIP --> GL
POSE -->|mask| GL
GL --> OUT[Screen: shading from camera,<br/>hands kept on top]
FIT --> MEASURE[Proportion measurements] --> RATE[Outfit rater]
It's 2D augmented reality: the garment photo is warped onto your body rather than simulated in 3D. That keeps it fast enough for a phone, and it looks best when you face the camera. For a photoreal result, the HD photo button sends a single frame to the generative model.
Repository layout
android/ Fitroom app (Kotlin, Jetpack Compose, CameraX, MediaPipe, OpenGL ES 3, Room)
backend/ FastAPI service for HD photo (Vertex AI Virtual Try-On), deployable to Cloud Run
tools/ make_starter_wardrobe.py: generates the sample wardrobe bundled with the app
docs/ screenshots
app.py, try_on.py, *.md guides the original Streamlit prototype (see below)
Inside android/app/src/main/java/com/fitroom/app/:
domain/style style archetypes, occasions and formality, quiz and scoring
domain/rating OutfitRater (facets and tips), ColorHarmony, OutfitIdeas
tryon/geometry pure-Kotlin fitting: TPS, strip skinning, body model, meshes, smoothing
tryon PoseTracker (MediaPipe), GarmentAsset, TryOnEngine
tryon/render TryOnRenderer (OpenGL ES 3)
garments import pipeline, AnchorEstimator, starter wardrobe
data Room (garments, looks), DataStore (profile, settings)
hd client for the HD backend
ui Compose screens: onboarding, home, closet/import, try-on, looks, profile
Build and run the app
Requirements: Android Studio (Koala or newer), JDK 17, Android SDK 36.
cd android
./gradlew :app:testDebugUnitTest # geometry, rating, quiz and fit-point detection tests
./gradlew :app:assembleDebug # APK: app/build/outputs/apk/debug/app-debug.apk
Or open android/ in Android Studio and press Run with a phone connected.
On first launch, take the style quiz, then tap Load sample wardrobe on the home screen to explore with 15 bundled garments. You can also photograph your own clothes from the Closet tab.
The APK ships ARM libraries only, because MediaPipe has no x86_64 build. x86_64 emulators run it through ARM translation. If the GPU delegate fails, pose tracking falls back to the CPU automatically.
HD photo backend (optional)
backend/ is a small FastAPI service that calls Vertex AI Virtual Try-On on the server side:
- the app authenticates with a random app token, and each token is rate limited
- a top and a bottom can be applied in one request
- results are returned as generated, with no cropping
Deployment to Cloud Run with a least-privilege service account and Secret Manager is described in backend/README.md. Paste the service URL and app token into the app under Profile β HD try-on.
Privacy
The style profile, closet, photos and ratings stay on the phone. A camera frame and garment images leave the device only when you tap HD photo, and then they go only to the backend you configured.
Roadmap
- Test live try-on on physical devices: fit quality, frame rate, low-end phones
- Test garment import with real clothing photos
- Deploy the HD backend and test the end-to-end flow
- Signed release build with R8, tested
- Crash reporting, privacy policy, Play Store listing and closed testing
- Dresses and outerwear layering
Original prototype
The first version of this project was a Streamlit web app (app.py, try_on.py) that sent uploaded photos and video frames to Vertex AI Virtual Try-On. It's kept for reference. Setup notes are in gcp_setup_guide.md and auth_guide.md, and n8n_guide.md covers automation.
pip install -r requirements.txt
streamlit run app.py
The Android app and backend replace it: try-on is now live and on-device, credentials no longer live on the client, and outputs are no longer cropped to remove the provider watermark.