The causal-leakage framing is valuable because ordinary capability benchmarks can miss structure-level failures. A practical next step would be to publish the minimal reproducibility harness and a per-diagnostic pass/fail trace, so deployment teams can compare models and audit regressions without relying on a single aggregate score.
Yusuf Chowdury
Yusufchy
AI & ML interests
AI agents, open-weight models, machine learning, MLOps, developer tools, AI automation, and AI-assisted publishing.
Recent Activity
repliedto SeaWolf-AI's post about 9 hours ago
AX-Ray: Safety Diagnostics for AI/AX Models
AI models can no longer be evaluated only by capability scores. As models move into public services, enterprise workflows, scientific research, and administrative decision support, we need a second layer of evaluation: whether the model behaves safely, structurally, and consistently under real deployment conditions.
VIDRAFT AX-Ray is a public AI/AX safety diagnostic initiative powered by FINAL-Bench Diagnostics. AX-Ray evaluates models across a structured guideline framework, including model-level safety, AX deployment readiness, and agent/service operation risks. The public diagnostic catalog contains 117 diagnostic items, mapped to legal, regulatory, ethical, and religious-law governance contexts so that safety review can be discussed in a form closer to real institutional responsibility.
A central finding of AX-Ray is causal leakage: a structural defect where information that should not influence an earlier reasoning state appears to affect model behavior. AX-Ray presents a public case of diagnosing, reproducing, and demonstrating causal leakage in two general-purpose public models. This matters because such defects are not exposed by ordinary benchmark scores. A model can appear capable while still carrying hidden safety or integrity risks.
Explore the live leaderboard, diagnostic reports, and public dataset here:
- AX-Ray Space: https://huggingface.co/spaces/FINAL-Bench/AX-RAY
- AX-Ray Dataset: https://huggingface.co/datasets/FINAL-Bench/AX-RAY
- Technical Article: https://huggingface.co/blog/FINAL-Bench/ax-ray
AX-Ray is intended as a practical guideline for moving AI evaluation beyond “how smart is the model?” toward “can this model be trusted, governed, and deployed safely?” repliedto dronefreak's post 1 day ago
🌧️❄️ Free demo: remove rain, raindrops, or snow from a photo with a single model
I put together an unofficial demo for **Histoformer** (ECCV 2024, arXiv: 2407.10172), a 16.6M-parameter transformer that handles three different weather degradations, rain streaks, adherent raindrops, and snow, in one unified model. It uses a "histogram self-attention" mechanism that groups pixels by degradation intensity instead of spatial position, which is a
neat way to sidestep the usual spatial-window tradeoffs in restoration transformers.
Try it here, free on ZeroGPU: https://huggingface.co/spaces/dronefreak/histoformer-weather-restoration
Upload a photo and get a before/after slider. Two checkpoints available: one tuned for real-world photos, one for the paper's synthetic benchmarks.
Also put together a cleaner, easy-to-use model card with a copy-pasteable Quickstart if you'd rather run it yourself: https://huggingface.co/dronefreak/Histoformer
This is an unofficial demo/mirror, not affiliated with the original authors. All credit for the actual research goes to Shangquan Sun, Wenqi Ren, Xinwei Gao, Rui Wang, and Xiaochun Cao (@sunsean). Official repo: https://github.com/sunshangquan/Histoformer. Weights are MIT-licensed.
Reported numbers from the paper: 32.1 PSNR on rain+fog (Outdoor-Rain), 33.1 on raindrops, 37.4 / 32.2 on light/heavy snow (Snow100K-S/L).Organizations
None yet