MLCommons

MLCommons is an open AI engineering consortium that brings together industry leaders, academics, and researchers to build trusted, safe, and efficient AI systems. The organization develops industry-standard benchmarks and open datasets that measure quality, performance, and risk in machine learning systems, helping companies and universities worldwide build better AI that benefits society.

• MLPerf Benchmarks provide neutral, consistent measurements of AI system accuracy, speed, and efficiency across training, inference, storage, and specialized domains like automotive, mobile, and tiny ML applications.

• AILuminate offers comprehensive AI safety evaluation tools including safety benchmarks, jailbreak testing, and agentic AI assessment methodologies to help developers build more reliable AI systems.

• Open Datasets include People's Speech, Multilingual Spoken Words, Dollar Street, and other large-scale, diverse datasets that improve AI model training and evaluation.

• Croissant Metadata Standard serves as today's standard vocabulary for ML datasets, making machine learning work easier to reproduce and replicate across the research community.

• AI Risk & Reliability Working Group brings together a global consortium of AI industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach for safer AI.

• Collaborative Research supports scientific advancement through shared infrastructure and diverse community participation, enabling new breakthroughs in AI through working groups focused on algorithms, data-centric ML, and scientific applications.

To get started with MLCommons, organizations can join as members or affiliates to participate in working groups, contribute to benchmark development, access datasets, and collaborate on research initiatives. The consortium operates on principles of open collaboration, consensus-driven decision-making, and inclusive participation from startups, large companies, academics, and non-profits globally.