This AI Meta Software Can Accelerate Nvidia and AMD Chip Performance

Facebook Meta parent launched a new set of free software tools dubbed AITemplate (AIT). (photo: doc. unsplash)

JAKARTA - Facebook parent Meta has launched a new set of free software tools dubbed AITemplate (AIT), for artificial intelligence applications. This tool can help speed up the performance of the underlying chip.

AITemplate is based on the open-source PyTorch machine learning framework that converts AI models into high-performance GPU C++ template code to accelerate inference, and can help code run up to 12 times faster on Nvidia Corp's flagship A100 chip or up to four times faster on chips MI50 AMD.

At launch, AITemplate has two layers of template systems, the first is the Python Jinja2 template, and the second is the GPU Tensor Core/Matrix Core C++ template (CUTLASS for NVIDIA GPUs and Composable Kernel for AMD GPUs).

First, AITemplate runs profiling to find the best kernel configuration in Python, and then renders the Jinja2 template into C++ code.

After the model source code is generated, the GPU C++ compiler (NVIDIA NVCC and AMD HIPCC) compiles the source code into the final binary code for the model. With a front-end design, which is similar to PyTorch, users can easily convert their models to AITemplates from various frameworks, including PyTorch.

"AITemplate also provides widely used out-of-the-box models (e.g., VisionTransformer, BERT, Stable Diffusion, ResNet, and MaskRCNN). This simplifies the deployment process and allows practitioners to easily deploy PyTorch pre-workout models," says Meta in his official blog post.

Apart from that, AITemplate also reduces dependency on external libraries. That way, the software can make it easier for developers to switch between different base chips.

Software has become a major battleground for chipmakers looking to build an ecosystem of developers to use their chips in.

By far, Nvidia's CUDA Platform is the most popular for artificial intelligence work. However, once developers adapt their code for Nvidia chips, it's difficult to run them on the graphics processing unit, or GPU, of Nvidia's competitors like AMD.

Meta says the software is designed to easily swap between chips without being locked out. The company, which also uses Nvidia chips in its data centers, has long been a supporter of open source hardware and software.