MLCommons Issues New Testing For AI Chips And The Top AI System

MLCommons released a series of AI testing. (photo: x @MLCommons)

JAKARTA - On Wednesday, March 27, the intelligence benchmark group MLCommons released a series of new tests and results that assess the speed at which the best hardware can run artificial intelligence (AI) applications and respond to users.

Two new benchmarks added by MLCommons measure the speed at which AI chips and systems can generate responses from powerful AI models filled with data. The results roughly show how quickly an AI application such as ChatGPT can respond to user questions.

One new benchmark adds the ability to measure speed in question scenarios and answers to large language models. Known as Llama 2, it covers 70 billion parameters and is developed by Meta Platforms Inc.

MLCommons officials also added text generators to a second image into a suite of testing tools, called MLPerf, based on the XL Stable Diffusion Model of AI Stability.

Server backed by the Nvidia H100 chip built by companies like Alphabet, Supermicro, and Nvidia themselves easily won both new benchmarks in raw performance. Some server builders deliver designs based on L40S chips that are less powerful than the company.

The developer of the Krai server sends a design for image generation benchmarks with Qualcomm AI chips that use far less power than Nvidia's leading processor.

Intel also sends designs based on its Gaudi2 accelerator chip. The company described the results as "solid."

Raw performance is not the only critical measure when deploying AI applications. State-of-the-art AI chips consume huge amounts of energy and one of the biggest challenges for AI companies is to deploy chips that provide optimal performance with minimal amounts of energy.

MLCommons has a separate benchmark category to measure power consumption.