Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Standardization for hardware use for reported RTFx metrics
#29
by petersnias - opened
Was there a standard hardware used across the different models for the RTFx metrics? If so, what hardware was used? Thank you.
Hi there, thanks for your question! That info can be found in Section 3 of our preprint.
"All evaluation scripts, as described in Section 2.5, were conducted on an NVIDIA A100-SXM4-80GB GPU (driver 560.28.03, CUDA 12.6), using a batch size of 64 whenever memory allowed, and reduced adaptively (48, 32, 16, . . . ) when necessary to fit in device memory"
Note that RTFx is intended for relative comparison between models under fixed conditions, rather than as absolute performance claims across arbitrary hardware.