AMD has just released the latest version of its open compute software, AMD ROCm™ 6.2.3 which supports Radeon GPUs on native Ubuntu® Linux® systems. Most notably, this new release gives incredible inference performance with Llama 3 70BQ4, and now allows developers to integrated Stable Diffusion (SD) 2.1 text-to-image capabilities in their own AI development.
“Following our previous release with AMD ROCm 6.1, we targeted specific features to accelerate Generative AI development. AMD ROCm 6.2 brings pro-level performance for Large Language Model inference via vLLM and Flash Attention 2. In addition, this release also includes beta support for the Triton framework enabling more users to develop AI functionality on AMD hardware”, says Erik Hultgren, Software Product Manager at AMD.
The four major feature highlights of AMD ROCm 6.2.3 for Radeon GPUs include the following:
- Official Support for Latest Version of Llama via vLLM – Incredible inference performance of AMD ROCm™ on Radeon with Llama 3 70BQ4
- Official Support for Flash Attention 2 “Forward Enablement” – Designed to help reduce memory requirements and speed up inference performance
- Official support for Stable Diffusion (SD) 2.1 – Integrate SD text-to-image model in your own AI development
- Beta Support for Triton – Leverage the Triton framework to easily write high-performance AI code with minimal expertise
AMD ROCm™ support for Radeon GPUs has come a long way since our initial 5.7 release just 12 months ago.
With version 6.0, we significantly expanded the capabilities of AMD ROCm by adding support for the popular ONNX runtime and formally qualified the use of more Radeon GPUs, including the Radeon PRO W7800 with 32GB.
The release of AMD ROCm 6.1 marked yet another important milestone where we announced official support for multi-GPU configurations and the TensorFlow framework as well as provided beta-level access to Windows® Subsystem for Linux® (WSL 2) which is now also officially qualified for use with 6.1.
With the latest release 6.2.3, the AMD ROCm™ solution stack for Radeon GPUs looks as followed:
While our focus with ROCm 6.2.3 was on Linux®, we will be releasing WSL 2 support soon.
In case you have missed our previous announcement on ROCm 6.1.3, make sure you check out our supporting video and blog.
It’s been a great year for ROCm on Radeon for AI and Machine Learning development, and we are looking forward to continue working closely with the community to further enhance our product stack and helping our system builders create compelling on-prem, client-based solutions.
Resources:
Visit the Documentation Portal >
David Diederichs is Product Marketing Manager, Workstation and AI
© 2024 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, AMD RDNA, AMD ROCm, Radeon, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries. Microsoft and Windows are registered trademarks of Microsoft Corporation in the US and/or other countries. PyTorch, the PyTorch logo and any related marks are trademarks of The Linux Foundation. TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc. Ubuntu and the Ubuntu logo are registered trademarks of Canonical Ltd. Other product names used in this publication are for identification purposes only and may be trademarks of their respective owners.