Hugging Face's logo
Winky's picture
8

Winky

Winky24
·

AI & ML interests

None yet

Recent Activity

reacted to KingNish's post with 🔥 21 days ago
Muon vs MuonClip vs Muon+Adamw Muon has gone from an experiment to a mainstream optimizer, but does it hold up for fine‑tuning? We ran head‑to‑head tests on Qwen3‑4B (10k+ high‑quality instruction rows) to find out. Short story: Pure Muon converged fastest at the start, but its gradient‑norm spikes made training unstable. MuonClip (Kimi K2’s clipping) stabilizes long pretraining runs, yet in our small‑scale fine‑tune it underperformed, lower token accuracy and slower convergence. The winner was the hybrid: Muon for 2D layers + AdamW for 1D layers. It delivered the best balance of stability and final performance and even beat vanilla AdamW. Takeaway: for small-scale fine-tuning, hybrid = practical and reliable. Next Step: scale to larger models/datasets to see if Muon’s spikes become catastrophic or if clipping wins out. Full Blog Link: https://huggingface.co/blog/KingNish/optimizer-part1
reacted to toshas's post with 🔥 21 days ago
Introducing 🇨🇭WindowSeat🇨🇭 –– our new method for removing reflections from photos taken through windows, on planes, in malls, offices, and other glass-filled environments. Finetuning a foundation diffusion transformer for reflection removal quickly runs up against the limits of what existing datasets and techniques can offer. To fill that gap, we generate physically accurate examples in Blender that simulate realistic glass and reflection effects. This data enables strong performance on both established benchmarks and previously unseen images. To make this practical, the open-source Apache-2 model builds on Qwen-Image-Edit-2509, a 20B image-editing diffusion transformer that runs on a single GPU and can be fine-tuned in about a day. WindowSeat keeps its use of the underlying DiT cleanly separated from the data and training recipe, allowing future advances in base models to be incorporated with minimal friction. Try it out with your own photos in this interactive demo: 🤗 https://huggingface.co/spaces/toshas/windowseat-reflection-removal Other resources: 🌎 Website: https://huggingface.co/spaces/huawei-bayerlab/windowseat-reflection-removal-web 🎓 Paper: https://huggingface.co/papers/2512.05000 🤗 Model: https://huggingface.co/huawei-bayerlab/windowseat-reflection-removal-v1-0 🐙 Code: https://github.com/huawei-bayerlab/windowseat-reflection-removal Team: Daniyar Zakarin (@daniyarzt)*, Thiemo Wandel (@thiemo-wandel)*, Anton Obukhov (@toshas), Dengxin Dai. *Work done during internships at HUAWEI Bayer Lab
upvoted a paper 21 days ago
EtCon: Edit-then-Consolidate for Reliable Knowledge Editing
View all activity

Organizations

None yet