🔄 In a Training Loop

Andy Chen

andynoodles

·

yi-hsiang-chen-tw

AI & ML interests

Feel free to contact me through Linkedin

Recent Activity

repliedto onekq's post 4 days ago

My take on device-side inference: it's all about high bandwidth memory (thinking about it, this holds for the cloud too). MacBooks enjoy incidental capacity of apple silicon, but per-device RAM is too low (16 to 24GB), only sufficient for a decent SLM. 512GB is the highest you can go (Kimi K2*). Counting MLX downloads of Kimi K2* on Huggingface, I estimate the user base to be <25K. On the other hand, the newly debuted DGX station (Nvidia) has 748GB, which can fit in the latest Kimi, DS, and Qwen. Also the quantization options of CUDA is way better than MLX. For high-end inferencing, I place my bet on workstations over Macs.

liked a model 4 days ago

zai-org/GLM-5.3

liked a model 4 days ago

zai-org/GLM-5.3-Flash

View all activity

Organizations

andynoodles 's collections 1