AI Discussions

[gemsooooo]

Hello Team,

I am not AI or ONNX expert but there is inquiry from customer for OGA Hybrid Execution support.

When I check AMD AI document, 8700G is not support for OGA Hybrid execution. > 

https://ryzenai.docs.amd.com/en/latest/llm/overview.html

My question is,

1.how LLM software company can use AMD Ryzen 8700G on OGA Hybrid execution on Windows. If possible, what will be the steps.

2.if not possible on Windows OS, what about Linux base? 

There is information that ROCm is fully supported only in Linux environments.

In the case of the 8700G, can LLM models be operated in a Linux environment with fully use of NPU acceleration? (what I found is that, on Linux, its possible but not support with NPU).

If it is supported in a Linux environment, after developing the LLM model in Linux, can the model be run in the Windows 11 WSL2 (Linux on Windows) environment while utilizing the NPU resources of the 8700G fully?

Would you please help on this? User is desperately looking for a solution on this.

Thank you very much in advance.

[Uday_Das]

8700 G has previous generation of NPU, which is Phoenix, only 16 Tops which is not interesting for LLM. Hence we dont support LLM on that NPU. The Strix Point are 50, 55 Tops are supported for LLM Hybrid execution mode. 

Also we dont have Linux support on 8700G. 

Please reach out to your Ryzen AI sales/marketing contact if you like to get more help on this. 

Thanks 

[gemsooooo]

Thanks for your reply. I understand that. Main things I would like to know is, is there any application or method that customer can see NPU utilize in LLM or non LLM apps.

There is we have additional findings regarding the 8700G CPU that I would like to share.

Based on the information provided in the link below, the tests conducted on the 8700G CPU resulted in a Passed outcome. We also confirmed that, although minimal, NPU resources are being utilized in the Windows 11 environment.

Installation Instructions — Ryzen AI Software 1.4 documentation

First, it identified the 8700G CPU as a PHX/HPT type.

Referring to the script content of quicktest.py, it used the following random image as input for the Test_model.onnx model:

image_array = np.random.rand(3, 32, 32).astype(np.float32)

Out of 398 NPU units, vector operations for the subgraph were actually performed on 1 NPU unit.

The output values were processed and generated correctly, and at the very end, "Test Passed" was displayed.

Even without a complex AI LLM model, if there are applications that can utilize NPU resources as in the above test scenario, we can highlight that the AMD 8700G CPU has a competitive edge in the AI field compared to Intel CPUs without NPU resources.

Would you please advise?