Code mirror of https://github.com/rudzz1950/Virtual-Try-On

Fitroom: Virtual Try-On

Find your style, try clothes on live with your phone camera, and get an honest, explainable score for every outfit.

Fitroom is an Android app (Kotlin + Jetpack Compose). The live try-on, the style profile and the outfit rating all run on the device. An optional backend adds photoreal HD stills through Google Vertex AI Virtual Try-On.

Style quiz Style profile Outfit ideas Closet Outfit rating

Status: beta. The app builds, its 19 unit tests pass, and every screen has been checked on an Android emulator. Live try-on on a real person and import of real garment photos still need testing on physical devices before a store release. See Roadmap.

Features

What it does Where it runs
Style quiz 7 questions build a blended profile across 10 style archetypes (Classic, Minimalist, Casual, Streetwear, Sporty, Preppy, Bohemian, Edgy, Romantic, Glam), plus the occasions you dress for, your preferred fit and colour palette On device
Occasions A dress-code formality scale (casual β†’ smart casual β†’ business casual β†’ cocktail β†’ formal) mapped to Everyday, Active, Work, Date night, Party, Formal event and Travel On device
Closet Photograph your clothes: the background is removed (ML Kit subject segmentation, with a plain-background fallback), fit points such as shoulders, cuffs, hems and waist are detected automatically, and you can drag them to adjust On device
Live try-on Tops, trousers, shorts and skirts follow your body in real time: sleeves and legs bend with your arms and legs, lighting from the camera shades the garment, and your hands stay in front On device
Outfit rating A 0–100 score with plain-English reasons: occasion fit, colour harmony, match with your style, how the pieces go together, and proportions measured live (for example "this top runs long on you"). Suggests swaps from your closet that score higher On device
Outfit ideas Pairs your tops and bottoms and ranks the combinations for the occasion you pick On device
Saved looks Snapshot plus rating, with favourites On device
HD photo Optional photoreal still via Vertex AI Virtual Try-On, through your own backend. Cloud credentials stay on the server and never ship in the app Cloud (opt-in)

How the live try-on works

flowchart LR
    CAM[CameraX frame] --> POSE[MediaPipe Pose<br/>33 landmarks + person mask]
    POSE --> SMOOTH[One Euro filter]
    SMOOTH --> BODY[Body frame<br/>shoulders, hips, joints]
    G[Garment photo<br/>+ fit points] --> RIG[Split into parts<br/>torso / sleeves / legs / skirt]
    BODY --> FIT[Fit solver]
    RIG --> FIT
    FIT -->|torso, skirt| TPS[Thin-plate spline warp]
    FIT -->|sleeves, legs| STRIP[Bend along arm / leg bones]
    TPS --> GL[OpenGL ES 3 renderer]
    STRIP --> GL
    POSE -->|mask| GL
    GL --> OUT[Screen: shading from camera,<br/>hands kept on top]
    FIT --> MEASURE[Proportion measurements] --> RATE[Outfit rater]

It's 2D augmented reality: the garment photo is warped onto your body rather than simulated in 3D. That keeps it fast enough for a phone, and it looks best when you face the camera. For a photoreal result, the HD photo button sends a single frame to the generative model.

Repository layout

android/    Fitroom app (Kotlin, Jetpack Compose, CameraX, MediaPipe, OpenGL ES 3, Room)
backend/    FastAPI service for HD photo (Vertex AI Virtual Try-On), deployable to Cloud Run
tools/      make_starter_wardrobe.py: generates the sample wardrobe bundled with the app
docs/       screenshots
app.py, try_on.py, *.md guides   the original Streamlit prototype (see below)

Inside android/app/src/main/java/com/fitroom/app/:

domain/style      style archetypes, occasions and formality, quiz and scoring
domain/rating     OutfitRater (facets and tips), ColorHarmony, OutfitIdeas
tryon/geometry    pure-Kotlin fitting: TPS, strip skinning, body model, meshes, smoothing
tryon             PoseTracker (MediaPipe), GarmentAsset, TryOnEngine
tryon/render      TryOnRenderer (OpenGL ES 3)
garments          import pipeline, AnchorEstimator, starter wardrobe
data              Room (garments, looks), DataStore (profile, settings)
hd                client for the HD backend
ui                Compose screens: onboarding, home, closet/import, try-on, looks, profile

Build and run the app

Requirements: Android Studio (Koala or newer), JDK 17, Android SDK 36.

cd android
./gradlew :app:testDebugUnitTest   # geometry, rating, quiz and fit-point detection tests
./gradlew :app:assembleDebug       # APK: app/build/outputs/apk/debug/app-debug.apk

Or open android/ in Android Studio and press Run with a phone connected.

On first launch, take the style quiz, then tap Load sample wardrobe on the home screen to explore with 15 bundled garments. You can also photograph your own clothes from the Closet tab.

The APK ships ARM libraries only, because MediaPipe has no x86_64 build. x86_64 emulators run it through ARM translation. If the GPU delegate fails, pose tracking falls back to the CPU automatically.

HD photo backend (optional)

backend/ is a small FastAPI service that calls Vertex AI Virtual Try-On on the server side:

  • the app authenticates with a random app token, and each token is rate limited
  • a top and a bottom can be applied in one request
  • results are returned as generated, with no cropping

Deployment to Cloud Run with a least-privilege service account and Secret Manager is described in backend/README.md. Paste the service URL and app token into the app under Profile β†’ HD try-on.

Privacy

The style profile, closet, photos and ratings stay on the phone. A camera frame and garment images leave the device only when you tap HD photo, and then they go only to the backend you configured.

Roadmap

  • Test live try-on on physical devices: fit quality, frame rate, low-end phones
  • Test garment import with real clothing photos
  • Deploy the HD backend and test the end-to-end flow
  • Signed release build with R8, tested
  • Crash reporting, privacy policy, Play Store listing and closed testing
  • Dresses and outerwear layering

Original prototype

The first version of this project was a Streamlit web app (app.py, try_on.py) that sent uploaded photos and video frames to Vertex AI Virtual Try-On. It's kept for reference. Setup notes are in gcp_setup_guide.md and auth_guide.md, and n8n_guide.md covers automation.

pip install -r requirements.txt
streamlit run app.py

The Android app and backend replace it: try-on is now live and on-device, credentials no longer live on the client, and outputs are no longer cropped to remove the provider watermark.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using Anirudh1950/Virtual-Try-On 1