Instructions to use aipracticecafe/dual-stream-dit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use aipracticecafe/dual-stream-dit with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("aipracticecafe/dual-stream-dit", dtype=torch.bfloat16, device_map="cuda") prompt = "1girl, souryuu asuka langley, neon genesis evangelion, rebuild of evangelion, evangelion: 3.0 you can \\(not\\) redo, momoko \\(momopoco\\), toosaka asagi, ebifurya, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, solo, long hair, blue eyes, orange hair, breasts, parted lips, official alternate costume, pilot suit, red bodysuit, black eye patch, animal hat, cabbie hat, cat hat, black hat, gloves, hat, hat horns, jacket, mecha pilot suit, bodysuit under clothes, red gloves, red jacket, track jacket, turtleneck, turtleneck jacket, interface headset, multicolored clothes, interface headset \\(evangelion\\), upper body, simple background" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Update README.md
Browse files
README.md
CHANGED
|
@@ -47,6 +47,8 @@ control (panning and zoom) at inference time.
|
|
| 47 |
- **Objective**: Conditional Flow Matching (Logit-Normal timestep sampling)
|
| 48 |
- **License**: OpenRAIL++
|
| 49 |
|
|
|
|
|
|
|
| 50 |
---
|
| 51 |
|
| 52 |
## Key Architecture & Features
|
|
@@ -57,8 +59,8 @@ The base architecture is based on the results from (i1: A Simple and Fully Open
|
|
| 57 |
per-block AdaLN modulation, reducing parameter overhead.
|
| 58 |
- **Sandwich Normalization & SwiGLU**: RMSNorm placed before and after
|
| 59 |
attention and FFN operations ensures training stability in bfloat16.
|
| 60 |
-
- **SPRINT Acceleration**: Employs structured group-wise
|
| 61 |
-
dropping (
|
| 62 |
demands by half while retaining global semantic context.
|
| 63 |
|
| 64 |
### 2. HDM Continuous Aspect-Normalized RoPE (Camera Control)
|
|
@@ -80,8 +82,8 @@ framing at inference time:
|
|
| 80 |
### 3. Components
|
| 81 |
- **Text Encoder**: `LiquidAI/LFM2.5-230M` (unidirectional causal LM hidden
|
| 82 |
states passed via a 2-layer Transformer Text Adapter).
|
| 83 |
-
- **Autoencoder**: `kaiyuyue/FLUX.2-dev-vae` (
|
| 84 |
-
patch size
|
| 85 |
|
| 86 |
---
|
| 87 |
|
|
@@ -99,8 +101,8 @@ resolution with shifted square cropping and cached in-RAM latents.
|
|
| 99 |
|
| 100 |
- **Total Epochs**: 175
|
| 101 |
- **Optimizer**: 8-bit AdamW (`bitsandbytes`), $\beta=(0.9, 0.95)$, WD=0.01
|
| 102 |
-
- **Learning Rate**:
|
| 103 |
-
- **Effective Batch Size**: 256 (Batch size
|
| 104 |
- **Precision**: Mixed Precision (`torch.bfloat16` with `torch.compile`)
|
| 105 |
|
| 106 |
---
|
|
|
|
| 47 |
- **Objective**: Conditional Flow Matching (Logit-Normal timestep sampling)
|
| 48 |
- **License**: OpenRAIL++
|
| 49 |
|
| 50 |
+
<Gallery />
|
| 51 |
+
|
| 52 |
---
|
| 53 |
|
| 54 |
## Key Architecture & Features
|
|
|
|
| 59 |
per-block AdaLN modulation, reducing parameter overhead.
|
| 60 |
- **Sandwich Normalization & SwiGLU**: RMSNorm placed before and after
|
| 61 |
attention and FFN operations ensures training stability in bfloat16.
|
| 62 |
+
- **SPRINT Acceleration**: Employs structured group-wise 2x2 patch
|
| 63 |
+
dropping (75% drop ratio during early pretraining), cutting compute
|
| 64 |
demands by half while retaining global semantic context.
|
| 65 |
|
| 66 |
### 2. HDM Continuous Aspect-Normalized RoPE (Camera Control)
|
|
|
|
| 82 |
### 3. Components
|
| 83 |
- **Text Encoder**: `LiquidAI/LFM2.5-230M` (unidirectional causal LM hidden
|
| 84 |
states passed via a 2-layer Transformer Text Adapter).
|
| 85 |
+
- **Autoencoder**: `kaiyuyue/FLUX.2-dev-vae` (C=32, f=8,
|
| 86 |
+
patch size p=2, effective image token size 16x16).
|
| 87 |
|
| 88 |
---
|
| 89 |
|
|
|
|
| 101 |
|
| 102 |
- **Total Epochs**: 175
|
| 103 |
- **Optimizer**: 8-bit AdamW (`bitsandbytes`), $\beta=(0.9, 0.95)$, WD=0.01
|
| 104 |
+
- **Learning Rate**: 5e-4 with warmup and cosine annealing only for last stage
|
| 105 |
+
- **Effective Batch Size**: 256 (Batch size 64x4 gradient accumulation)
|
| 106 |
- **Precision**: Mixed Precision (`torch.bfloat16` with `torch.compile`)
|
| 107 |
|
| 108 |
---
|