Knowledge Hub Artificial-intelligence

MiniCPM-V 2.0 is a cutting-edge multimodal large language model (LLM) developed by OpenBMB, designed for efficient end-device deployment. This model, based on SigLip-400M and MiniCPM-2.4B, offers a range of powerful features suitable for tasks like visual question answering, feature extraction, and scene-text understanding in both English and Chinese.

Key Features:

• State-of-the-Art Performance: MiniCPM-V 2.0 excels on multiple benchmarks, including OCRBench, TextVQA, and OpenCompass, outperforming models like Qwen-VL-Chat 9.6B and Yi-VL 34B in terms of accuracy and efficiency.

• Trustworthy Behavior: Aligned with multimodal reinforcement learning (RLHF-V), MiniCPM-V 2.0 mitigates hallucinations, ensuring reliable performance similar to GPT-4V.

• Bilingual Support: With robust support for both English and Chinese, the model generalizes multimodal capabilities across languages.

• High-Resolution Image Processing: Capable of processing images with up to 1.8 million pixels, MiniCPM-V 2.0 provides excellent perception of fine-grained details, ideal for applications requiring detailed visual analysis.

• Efficient Deployment: The model is optimized for deployment on GPUs, Mac (with MPS support), and even mobile devices like Android and Harmony OS, making it versatile for a wide range of applications.

Usage and Inference

MiniCPM-V 2.0 can be easily integrated with HuggingFace Transformers, utilizing GPUs or Apple silicon for efficient inference. For deployment on mobile devices, the model demonstrates impressive performance even on consumer-grade hardware, such as Xiaomi 14 Pro.

Latest News:

• [2024.05.20] 🔥 MiniCPM-Llama3-V 2.5 release, reaching GPT-4V level multimodal capabilities.
• [2024.04.23] MiniCPM-V 2.0 now supports vLLM for faster inference.

For more detailed usage instructions, source code, and demo links, check the GitHub repository.

Citations

For academic use, please cite relevant papers from OpenBMB, including works published on arXiv:2403.11703 and arXiv:2408.01800.