Written by Anshul Gupta, Sr. Product Marketing Manager -- AI Group
AMD just released Ryzen™ AI software 1.2 for developers and ISVs!. Ryzen AI software aims to offer a cohesive environment for developers and ISVs to build AI-enhanced applications while maximizing the performance benefits of the underlying Ryzen AI APU hardware architecture (CPU, iGPU, NPU). Ryzen AI software continues to champion the open-source community by delivering compatibility with top industry frameworks giving developers freedom to deploy AI inference applications on AMD Ryzen AI powered PCs. The Ryzen AI 1.2 software release highlights new capabilities including additional hardware and model support and new ease of use developer tools. Let's dive into the new features!
New Hardware Support:
- Enhanced NPU Support: The latest update introduces support for Strix Point NPUs, delivering outstanding performance:
- Ryzen AI 9 365 and Ryzen AI 9 HX 370: Up to 50 TOPS
- Ryzen AI 9 HX 375: Up to 55 TOPS
- Improved iGPU Integration: Developers can now seamlessly switch between integrated GPUs (iGPU) and NPUs based on their model requirements without the need to uninstall or reinstall the software.
Expanded Model Support:
- LLM Flow Support: Ryzen AI software 1.2 supports additional models in both PyTorch and ONNX frameworks for general purpose flows and performance optimized applications.
- General purpose unoptimized LLMs for experimental purposes can be found here. These models are not to be used for benchmarking purposes.
- Applicability: prototyping and early development with a broad set of LLMs
- Performance: Functional support only
- Supported platforms: AMD Ryzen 7040 Series, AMD Ryzen 8040 Series and AMD Ryzen AI 300 Series (and future Generations)
- Supported frameworks: PyTorch
- Supported models: Many
- Performance-optimized LLMs for production and benchmarking purposes are available upon request on the AMD secure download site.
- Applicability: benchmarking and deployment of specific LLMs
- Performance: highly optimized
- Supported platforms: AMD Ryzen AI 300 Series (and future generations)
- Supported frameworks: PyTorch and ONNX
- Supported models: Llama2, Llama3, Qwen1.5
- General purpose unoptimized LLMs for experimental purposes can be found here. These models are not to be used for benchmarking purposes.
New and Improved Ease of Use Developer Tools:
- AI Analyzer: A tool for analysis and visualization of model compilation and inference profiling, to understand bottlenecks and optimize model performance more effectively.
- Platform/NPU Inspection and Management: A new xrt-smi tool that seamlessly integrates three features: it examines the state of the AI PC and NPU, validates the NPU by executing sanity tests, and configures the NPU’s performance levels for optimal efficiency.
- Dynamic Power Management (DPM) Integration: Developers can now change the Power Slider in Windows to "Performance," "Balanced," or "Power saver," during NPU inference by configuring the status of the dynamic power management register. This integration provides flexibly for the developers to run inference in different power profiles based on the needs of the end application. Developers can also set the power level via xrt-smi tools.
- ONNX Benchmarking Tool: A tool for benchmarking ONNX models, facilitating performance assessment and optimization of ONNX-based workflows.
- Smart Installer: The new Smart Installer simplifies the Ryzen AI software setup process. With a “Single-click installation” of the NPU driver, the Smart Installer is now a Windows based MSI process through a unified executable instead of manual python script installation.
Through periodic software releases, we continue to provide cutting-edge tools and enhancements specifically designed to supercharge your AI development on AI PCs. Ryzen AI 1.2 software introduces updates that enhances ease of use, streamlines workflows for optimal performance, supports open frameworks and offers resources tailored to advance your AI capabilities.
For a comprehensive overview of the new features and improvements in the 1.2 software release, check out AMD’s official release notes.
Stay at the forefront of AI innovation by signing up for the latest Ryzen AI software updates to ensure you have access to the tools and resources that will help you push the boundaries of what’s possible on AI PCs. Sign Up
Learn more about Ryzen AI Software
Download Ryzen AI Software