AI Discussions

[Ishigami]

<p>Hello, I have an AMD laptop with the following specs: <strong>AMD Ryzen 7 PRO 8840U w/ Radeon 780M Graphics 3.30 GHz</strong>.</p>
<p>I want to run LLMs, image generation models (like Stable Diffusion), vision models (image-to-text), and audio models locally using the system's <strong>NPU</strong> and <strong>integrated GPU</strong>. While exploring the AMD website, I came across <strong>Ryzen AI software</strong> and reviewed its capabilities. Based on my understanding, it appears that only a limited number of LLMs are currently supported on the NPU and GPU. I would like to confirm whether <strong>image generation and audio generation models</strong> are also supported on the NPU and GPU. Could you clarify this?</p>

[Uday_Das]

Yes, currently you can run generation model like Stable Diffusion on integrated GPU. 

[Ishigami]

Hi @Uday_Das  thanks for your response! Could you please provide a link to the steps for running the Stable Diffusion model on an AMD integrated GPU? I’d really appreciate it. Thank you!

[Ishigami]

Hi @Carteraiden907  I looked into the Ryzen AI software, which is designed for running generative AI models on the iGPU and NPU. However, I noticed limitations regarding support for other data modalities, and it is still under development. For image generation, they provided Stable Diffusion, but when I followed the steps mentioned in the blog, I encountered an error—Torch was unable to detect the GPU. Additionally, the blog referenced labels such as Adaptive AI (Versal, Zynq, Kria), AI Announcements, AI Software, Alveo AI, and EPYC AI, but did not mention Ryzen AI. This led me to the forums to confirm whether Stable Diffusion is optimized to run on an iGPU or requires a dedicated GPU.

Link Of Ryzen AI : https://ryzenai.docs.amd.com/en/latest/gpu/ryzenai_gpu.html 

Link Of the Blog : https://community.amd.com/t5/ai/how-to-automatic1111-stable-diffusion-webui-with-directml/ba-p/64902... 

[terrywashington]

Right now, AMD’s NPU mainly supports some LLMs, but image and audio generation models might not fully run on it—you may need to use the integrated GPU instead.

[Ishigami]

According to the document, only Stable Diffusion is supported for image generation on the iGPU, while Whisper is supported for audio processing. I followed the instructions from the blog, but many steps related to Olive are unclear, and I encountered multiple errors. I'm unsure if the documentation is accurate.

link : https://community.amd.com/t5/ai/how-to-automatic1111-stable-diffusion-webui-with-directml/ba-p/6490...

Additionally, I couldn't find any relevant information on running Whisper using Ryzen AI software.

If you have any links or resources on running these models (Stable Diffusion and Whisper) on an AMD iGPU, please share them.

[myriad97]

hi @Ishigami, 

i would recommend the following for usage on Windows 10/11 environment.

For LLM , you can use LLM studio
https://lmstudio.ai/

for Image generation (easy) you can use the AMUSE.AI desktop application
https://www.amuse-ai.com/

it does run on AMD hardware but the recommended minimum RAM is >= 32GB
16GB is OK for using LCM-based image generation but SD requires at least 32GB system RAM.
It does utilise the AI Accelerator on the 8840U processor to assist in image generation (it is called AMD XDNA architecture)
it is not similar to the AI Accelerator found on Radeon RX GPU but it is from the Xylink product (Alveo)
you can check the Alveo product line documentation if you are interested.

For Flux dev/schnell, unfortunately, you need a much bigger memory, internally 64GB so unless you are allocating at least 20GB++ shared memory to the GPU it will not work due to this application doesn't have yet support for tiled VAE.

For Audio I think you can start to look into any web-based (AUTOMATIC1111, Forge, ComfyUI) deployment that supports the AMD with MS OLIVE compiled model, but currently it is not guaranteed it will work without customization.