LMMs-Lab
LMMs-Lab is an open research organization focused on building multimodal intelligence. It develops and openly shares multimodal models, evaluation benchmarks, datasets, training systems, and research tooling.
At a Glance
- Universities and academic research labs
- Open-source AI and multimodal-model developers
- Foundation-model and machine-learning research teams
- Researchers needing standardized evaluation across image, video, audio, and reasoning tasks
AI Tools by LMMs-Lab
(1)
LMMs-Eval
Multimodal Model Evaluation Toolkit
Discussions
No discussions yet
Be the first to start a discussion about LMMs-Lab
Latest News
LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
OneVision Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence
LLaVA-OneVision-1.5-RL: Unlocking Multimodal Reasoning via Lightweight Reinforcement Learning
LongVT: Incentivizing Thinking with Long Videos via Native Tool Calling
Products & Services
Unified, open evaluation toolkit for large multimodal models across text, image, video, audio, and reasoning benchmarks; it emphasizes reproducibility, efficiency, and statistically grounded comparisons.
Open large multimodal model family with a unified interface for single-image, multi-image, chart/document, and video understanding.
Video instruction-following model trained with the LLaVA-Video-178K synthetic dataset, covering detailed captioning, open-ended QA, and multiple-choice QA.
Sparse-autoencoder framework for interpreting open-semantic features in multimodal models, including cross-scale auto-explanation and feature steering.
Market Position
LMMs-Lab positions itself as a nonprofit, open-source alternative to closed multimodal-model research by releasing end-to-end artifacts rather than only checkpoints. Its differentiation is the combination of competitive open multimodal models, long-video and native-resolution research, a broad unified evaluation stack, and openly reproducible training infrastructure. Relevant comparison points include proprietary multimodal systems such as GPT-4o/Gemini and open model ecosystems such as Qwen-VL, as well as general evaluation frameworks such as EleutherAI's lm-evaluation-harness; LMMs-Lab focuses specifically on multimodal evaluation and open model development.
Leadership
Founders
Bo (Brian) Li
Researcher focused on multimodal models and AI systems; his prior experience includes ByteDance Seed, TikTok AI Innovation Center, Microsoft Research, Berkeley AI Research, and DiDi Visual Perception. He led or contributed to LLaVA-OneVision, LMMs-Eval, LMMs-Engine, and Multimodal-SAE.
Ziwei Liu
Professor at Nanyang Technological University and a computer-vision/machine-learning researcher. LMMs-Lab's official About page identifies him as the professor who heavily supported the organization founded by Bo Li and friends.
Executive Team
Bo (Brian) Li
Founder
Multimodal-model researcher and lead contributor to LLaVA-OneVision, LMMs-Eval, LMMs-Engine, and Multimodal-SAE; previously worked at ByteDance Seed, TikTok AI Innovation Center, Microsoft Research, Berkeley AI Research, and DiDi.
Ziwei Liu
Founder and academic supporter
Professor at Nanyang Technological University; the official LMMs-Lab About page identifies him as the principal academic supporter of the lab.
Founding Story
The lab was formed by Bo Li with friends and heavily supported by Prof. Ziwei Liu as a non-profit, open-source research community. Its initial vision was to advance multimodal intelligence through openly shared models, data, evaluation, and tools, ultimately pursuing a world model that goes beyond text to understand and act on the world.
Target Markets
- Universities and academic research labs
- Open-source AI and multimodal-model developers
- Foundation-model and machine-learning research teams
- Researchers needing standardized evaluation across image, video, audio, and reasoning tasks
- Academic and industrial research on large multimodal models
- Reproducible benchmarking and leaderboard evaluation
- Image, video, audio, chart, document, OCR, and spatial understanding
- Long-video question answering and evidence retrieval
- Multimodal reasoning and reinforcement-learning research
- Training and fine-tuning open vision-language and video-language models