sashakunitsyn/vlrm-blip2-opt-2.7b
Image-to-Text
•
4B
•
Updated
•
12
•
19
Alexander Kunitsyn
sashakunitsyn
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
22 days ago
Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation
upvoted
a
paper
5 months ago
nablaNABLA: Neighborhood Adaptive Block-Level Attention
posted
an
update
over 1 year ago
Introducing VLRM: Vision-Language Models act as Reward Models for Image Captioning
Paper:
https://arxiv.org/abs/2404.01911
Model weights and training logs:
https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b
We propose VLRM, a method for fine-tuning an existing image captioning model using reinforcement learning and vision-language models as reward models.
The method manages to significantly improve the generation quality without human-labeled data and is applicable to any image captioning model.
Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split.
Organizations
