MLCommons

Want to learn more about the progress of our MLCommons Automotive task force benchmark suite? Join MLCommons automotive working group co-chairs Kasper Mecklenburg and James Goel for an informative webinar on August 13th. #selfdriving #benchmarks #automotive

Just announced! The Adversarial Nibbler Challenge is a red-teaming effort for crowdsourcing diverse implicitly adversarial prompts. Red teaming is an essential part of responsible ML development to help uncover harms and facilitate mitigations. But, existing efforts are privately done, and may not seek community input when determining adequate safety guardrails. This could lead to missed subtle or non-obvious harms. Community input is to critical to build a diverse set of implicitly adversarial prompts for capturing long-tail safety issues that may be overlooked in standard testing. The MLCommons Dynabench WG team is supporting the Adversarial Nibbler Challenge, with the goal of holistically evaluating model robustness against blind spots in harmful image generation or long-tail problems. Learn more and join the challenge today! https://lnkd.in/gFdr6Eiw

Today in Singapore, we signed an MOI with AI Verify Foundation to collaborate on developing a set of common safety testing benchmarks for generative AI models for the betterment of AI safety globally. This collaboration is a step-forward towards creating a truly global and inclusive standard for AI safety testing, with benchmarks designed to address safety risks across diverse contexts, languages, cultures, and value systems. Welcome to the MLCommons AI Safety working group AI Verify! #AISafety #ATxSG #ATxSummit #AI #SaferAIforEveryone https://lnkd.in/gNDqASgt

Building a shared understanding of AI safety testing was key to the recent MLCommons AI Safety v0.5 benchmark proof-of-concept. Learn more about the comprehensive test spec schema the AI Safety working group delivered that was applied to the POC benchmark results. https://lnkd.in/eZuTtW4j #AIsafety #testspec

MLCommons just announced the v4.0 release of our MLPerf Mobile benchmark application and the latest benchmark results using the updated app. “The MLPerf Mobile benchmark suite is continuously evolving. Each of the vision tasks–classification, object detection, and segmentation–has been updated since the first version of the benchmark application, and a new super-resolution task was added last year. Looking ahead, the MLPerf Mobile group is working to add generative AI tasks to a future version of the benchmark suite.” said Mostafa El-Khamy, co-chair of the MLPerf Mobile working group. Learn more in our latest blog - https://lnkd.in/gZPsuYRw #MLPerf #MachineLearning #AI #Mobile

Researchers at the University of Oxford in collaboration with MLCommons and others, released the PRISM alignment project – a new resource for the AI community to understand how humans from around the world perceive and interact with LLMs. It's more important than ever that a diversify of voices contributes to the human feedback used to train LLMs. PRISM aims to do just that. Learn more in our blog https://lnkd.in/gywzzVg3 #MachineLearning #data #LLMs #datadiversity #AI

Thanks Sajan Paul. We are thrilled to have Juniper Networks join the MLCommons consortium of leading industry and academic members committed to measuring and improving the accuracy, safety, speed, and efficiency of AI technologies.

Welcome to MLCommons Juniper Networks. We are thrilled to have you join our consortium of industry leaders & academics committed to measuring and improving the accuracy, safety, speed, and efficiency of AI technologies. https://lnkd.in/gAieey7z #Machinelearning #ai #MLPerf

Great writeup by Hackster.io on the #MLPerf Tiny v1.2 benchmark results. This was a very diverse set of submissions, which only showcases the importance of these kinds of #benchmarks.