VLM-R1 is a multimodal AI model developed by Om AI Lab based on the DeepSeek R1 methodology, with the core capability of accurately associating natural language commands with visual content. The project innovatively combines the architectural advantages of the Qwen2.5-VL model, and adopts the dual optimization strategies of reinforcement learning (R1) and supervised fine-tuning (SFT), which enables the model to perform well on the task of representing expression comprehension (REC). Typical examples include parsing instructions such as "Where is the red cup in the picture?" and accurately locating the target object in the form of a bounding box or coordinates.
In terms of technical implementation, the project adopts GRPO reinforcement learning algorithm to optimize the model parameters, with bfloat16 mixed-precision training to improve the computational efficiency. The open source community provides COCO and RefCOCO standard dataset support, including 340,000 training images and 120,000 finger annotations, to ensure that the model has excellent generalization ability. The project obtained nearly 2,000 starred labels within 3 months of open-sourcing in GitHub, which verifies the leading-edge of its technical solution.
This answer comes from the articleVLM-R1: A Visual Language Model for Localizing Image Targets through Natural LanguageThe
