Right now, the tech world is awash in discussion of AI. Huge data center chips with terabytes of bandwidth and multi-thousand-dollar price tags share the space with diminutive Internet of Things products designed for ultra-low power processing. The news cycle serves up an endless discussion of artificial intelligence, but many of these discussions don't explain the difference between platforms and capabilities. It's all well and good to say AMD has built an "integrated AI engine," but what does that actually mean and why should you care? What's the advantage to you?
A Superior Hybrid Solution
Today, the majority of AI processing takes place in the cloud, not on your local PC. This is particularly true of Large Language Models (LLMs) like ChatGPT. The various companies that deploy these services have built vast server backends to handle queries and improve response times. This makes good sense when relatively few PCs had the hardware to run AI workloads locally, but there's a real underlying problem with this approach: It's difficult to scale. According to an analysis earlier this year, incorporating ChatGPT into every Google search would cost roughly $36 billion per year. That's an approximation, but it makes the point: Only a few companies can afford to deploy their own AI solutions at this kind of scale. The "Build an enormous cloud service everyone else pays to access," model suits giant cloud providers well, but this top-down model isn't the only way to scale access to AI processing.
Select models of the AMD Ryzen 7040 Series of mobile processors with Ryzen AI presents a different model for AI deployments – hybrid AI. These processors integrate specialized silicon to process AI workloads locally instead of relying primarily on cloud services to do the heavy lifting. These AMD Ryzen 7040 Series mobile processors offer an integrated AI engine to process these workloads more quickly and efficiently on x86 platforms. This new on-die processor is powered by the AMD XDNA Architecture and offers the option to offload work that would have previously consumed CPU or GPU cycles.
Training AI models from scratch can require large processing clusters and a great deal of RAM, but running the models after training requires much less horsepower. By integrating AI functionality directly into the CPU, Ryzen AI creates a cost-effective solution that scales from the bottom-up. Instead of relying solely on ever-larger cloud deployments, workloads can be kept local. This is part of where the "hybrid" in "hybrid AI" comes from. The other component of the name, expected to be in expanded use in the future, refers to the idea of running part of a workload locally while the rest runs in the cloud. The AMD XDNA Architecture is flexible enough to support both types of operation.
There are several advantages to using a hybrid approach. Moving some work to the individual PC helps improve performance and reduce access latency compared to leaning on a cloud provider. It improves privacy by keeping data local to an individual system rather than transmitting everything across the cloud. Offloading AI workloads to a dedicated engine instead of relying on the CPU or GPU to execute them also can improve power efficiency.
Rapidly Improving Software Support
Hardware availability always precedes wide software support. Customers can't start developing software until they have hardware on which to prototype, and companies can't build features that rely on new hardware capabilities until those capabilities are widely available. This is part of why so much discussion around artificial intelligence has also focused on the cloud. Thus far, it has been easier for companies to centralize server infrastructure and then rent that infrastructure out to others than to deploy AI (and AI-enabled software) across consumer PCs.
The advent of the AMD XDNA architecture and the AI engine it powers is changing that narrative. As hardware availability improves for commercial and consumer users, the effective "install base" for AI applications is only going to grow.
Today, Microsoft Windows already supports using an integrated AI engine to handle background blur, eye contact correction, and camera framing. In the (very) near future, improving software support will allow dedicated accelerator blocks to handle more tasks. The power and performance savings unlocked by moving workloads to this type of dedicated processor and away from the CPU and GPU will give laptops equipped with a dedicated AI engine a leg up over equivalently configured systems that lack this feature.
We're still in the earliest days of broad AI adoption, but the technology represents one of the most exciting computing breakthroughs in decades. It's an adventure AMD is excited to be part of and a core component of our leadership across CPUs, GPUs, and dedicated AI accelerators.
