JMMMU-Pro_Leaderboard / result.csv
AtsuMiyai
update
47ff54a
model_name,prompt,Overall,culture-specific,culture-agnostic,Japanese_Art,Japanese_Heritage,Japanese_History,World_History,Accounting,Agriculture,Architecture_and_Engineering,Basic_Medical_Science,Biology,Chemistry,Clinical_Medicine,Computer_Science,Design,Diagnostics_and_Laboratory_Medicine,Economics,Electronics,Energy_and_Power,Finance,Manage,Marketing,Materials,Math,Mechanical_Engineering,Music,Pharmacy,Physics,Psychology,Public_Health
Aya-Vision-8B,Direct,22.424,23.833,21.25,26.0,26.667,20.0,22.667,30.0,20.0,20.0,30.0,16.667,20.0,26.667,13.333,23.333,23.333,26.667,10.0,23.333,26.667,30.0,16.667,10.0,20.0,20.0,16.667,20.0,16.667,20.0,30.0
Heron-NVILA-Lite-15B,Direct,26.97,26.667,27.222,26.0,27.333,31.333,22.0,23.333,30.0,30.0,33.333,26.667,20.0,23.333,33.333,40.0,30.0,30.0,10.0,30.0,26.667,26.667,16.667,26.667,30.0,23.333,13.333,26.667,30.0,23.333,50.0
Phi-4-multimodal,Direct,31.818,28.833,34.306,29.333,18.0,30.0,38.0,26.667,33.333,20.0,46.667,23.333,26.667,30.0,56.667,20.0,13.333,43.333,50.0,40.0,20.0,30.0,43.333,46.667,33.333,46.667,33.333,36.667,26.667,33.333,43.333
InternVL2.5-8B,Direct,25.076,23.833,26.111,26.0,18.0,24.667,26.667,30.0,13.333,30.0,30.0,23.333,23.333,20.0,30.0,23.333,33.333,26.667,26.667,36.667,26.667,30.0,33.333,16.667,23.333,36.667,26.667,13.333,30.0,23.333,20.0
Qwen2.5-VL-7B,Direct,44.697,50.167,40.139,42.667,44.0,38.667,75.333,30.0,53.333,30.0,60.0,40.0,36.667,36.667,50.0,53.333,40.0,30.0,43.333,33.333,46.667,23.333,36.667,33.333,36.667,33.333,36.667,53.333,33.333,50.0,43.333
Qwen3-VL-8B,Direct,45.833,47.0,44.861,39.333,37.333,40.667,70.667,50.0,53.333,46.667,73.333,40.0,26.667,40.0,43.333,53.333,40.0,53.333,43.333,33.333,36.667,43.333,50.0,40.0,33.333,53.333,40.0,46.667,43.333,53.333,40.0
Gemini3Pro,Direct,87.045,95.0,80.417,91.333,96.667,95.333,96.667,80.0,66.667,93.333,90.0,70.0,90.0,83.333,86.667,80.0,56.667,83.333,83.333,86.667,90.0,73.333,86.667,73.333,90.0,80.0,46.667,83.333,86.667,83.333,86.667
GPT-5.2,Direct,83.333,88.333,79.167,80.0,92.667,90.0,90.667,80.0,63.333,86.667,93.333,70.0,90.0,60.0,83.333,83.333,53.333,80.0,83.333,73.333,90.0,76.667,90.0,76.667,93.333,50.0,60.0,86.667,93.333,86.667,96.667
LLaVA-OV-1.5-8B,Direct,29.924,26.333,32.917,34.0,22.0,20.667,28.667,33.333,23.333,26.667,43.333,16.667,33.333,23.333,33.333,23.333,23.333,36.667,43.333,33.333,26.667,36.667,43.333,46.667,33.333,43.333,43.333,23.333,40.0,26.667,33.333
LLaVA-OV-7B,Direct,27.348,26.5,28.056,30.667,26.667,26.0,22.667,46.667,26.667,30.0,33.333,26.667,30.0,26.667,23.333,26.667,23.333,30.0,33.333,36.667,16.667,23.333,16.667,33.333,23.333,33.333,36.667,33.333,13.333,20.0,30.0
Pangea-7B,Direct,19.545,23.0,16.667,28.0,24.667,20.0,19.333,13.333,16.667,23.333,16.667,10.0,3.333,3.333,13.333,10.0,10.0,20.0,13.333,16.667,20.0,33.333,26.667,13.333,20.0,20.0,30.0,23.333,13.333,16.667,13.333
Sarashina2-V-14B,Direct,30.682,32.333,29.306,32.0,27.333,31.333,38.667,33.333,33.333,23.333,30.0,20.0,23.333,20.0,26.667,26.667,36.667,36.667,33.333,43.333,30.0,20.0,30.0,20.0,43.333,20.0,20.0,40.0,26.667,23.333,43.333
Sarashina2-V-8B,Direct,27.879,27.0,28.611,29.333,24.0,28.0,26.667,40.0,30.0,26.667,36.667,26.667,23.333,16.667,23.333,36.667,26.667,33.333,40.0,23.333,20.0,26.667,13.333,26.667,40.0,20.0,20.0,40.0,30.0,26.667,40.0
Sarashina2.2-V-3B,Direct,38.03,40.167,36.25,42.0,29.333,36.0,53.333,36.667,50.0,36.667,53.333,33.333,33.333,30.0,50.0,43.333,40.0,43.333,46.667,33.333,30.0,20.0,30.0,33.333,36.667,26.667,26.667,36.667,23.333,33.333,43.333
Aya-Vision-8B,CoT,26.742,27.0,26.528,22.0,30.667,26.667,28.667,23.333,13.333,23.333,40.0,26.667,13.333,13.333,30.0,36.667,26.667,30.0,26.667,33.333,26.667,43.333,30.0,30.0,16.667,36.667,26.667,26.667,16.667,20.0,26.667
Heron-NVILA-Lite-15B,CoT,5.303,1.0,8.889,1.333,1.333,0.667,0.667,10.0,13.333,0.0,3.333,10.0,6.667,0.0,10.0,6.667,3.333,3.333,23.333,6.667,13.333,0.0,10.0,10.0,10.0,23.333,10.0,3.333,0.0,16.667,20.0
Phi-4-multimodal,CoT,24.167,22.0,25.972,19.333,18.667,17.333,32.667,33.333,20.0,26.667,36.667,20.0,13.333,16.667,30.0,10.0,20.0,43.333,40.0,33.333,26.667,23.333,26.667,26.667,20.0,23.333,23.333,33.333,20.0,23.333,33.333
InternVL2.5-8B,CoT,31.212,29.0,33.056,29.333,22.667,28.0,36.0,33.333,26.667,23.333,43.333,36.667,26.667,43.333,40.0,40.0,23.333,40.0,33.333,33.333,36.667,26.667,36.667,16.667,30.0,40.0,13.333,40.0,40.0,23.333,46.667
Qwen2.5-VL-7B,CoT,45.0,46.667,43.611,34.0,41.333,43.333,68.0,43.333,30.0,33.333,83.333,40.0,36.667,53.333,36.667,33.333,33.333,46.667,46.667,26.667,53.333,33.333,50.0,36.667,43.333,36.667,40.0,60.0,43.333,43.333,63.333
Qwen3-VL-8B,CoT,47.273,47.5,47.083,42.667,34.0,44.667,68.667,53.333,53.333,10.0,63.333,36.667,43.333,40.0,43.333,46.667,36.667,60.0,33.333,30.0,43.333,56.667,80.0,23.333,56.667,36.667,26.667,63.333,66.667,66.667,60.0
LLaVA-OV-1.5-8B,CoT,31.97,28.0,35.278,28.0,20.667,29.333,34.0,30.0,16.667,40.0,46.667,36.667,23.333,30.0,50.0,30.0,20.0,36.667,43.333,43.333,60.0,23.333,30.0,40.0,36.667,46.667,30.0,26.667,33.333,26.667,46.667
LLaVA-OV-7B,CoT,14.091,14.333,13.889,11.333,14.0,12.667,19.333,13.333,13.333,6.667,13.333,6.667,13.333,13.333,10.0,23.333,0.0,20.0,3.333,10.0,33.333,3.333,13.333,13.333,26.667,16.667,20.0,20.0,10.0,10.0,20.0
Pangea-7B,CoT,23.409,21.667,24.861,22.0,22.0,24.0,18.667,13.333,26.667,26.667,33.333,16.667,10.0,23.333,23.333,40.0,20.0,26.667,33.333,30.0,30.0,20.0,30.0,36.667,23.333,13.333,16.667,36.667,20.0,20.0,26.667
Sarashina2-V-14B,CoT,30.0,30.5,29.583,30.0,26.0,29.333,36.667,36.667,33.333,13.333,40.0,23.333,13.333,13.333,30.0,33.333,36.667,43.333,36.667,46.667,33.333,20.0,23.333,30.0,36.667,20.0,23.333,43.333,26.667,20.0,33.333
Sarashina2-V-8B,CoT,27.273,25.333,28.889,24.0,24.667,31.333,21.333,36.667,23.333,26.667,36.667,23.333,23.333,16.667,30.0,36.667,26.667,30.0,43.333,23.333,30.0,36.667,16.667,26.667,30.0,20.0,16.667,40.0,23.333,30.0,46.667
Sarashina2.2-V-3B,CoT,42.879,54.0,33.611,48.0,51.333,51.333,65.333,36.667,40.0,23.333,63.333,30.0,13.333,23.333,30.0,33.333,26.667,50.0,36.667,23.333,43.333,16.667,26.667,33.333,30.0,20.0,23.333,53.333,43.333,40.0,46.667