AI Discussions

[danielcor]

I've been trying to use my NPU in a chat bot application for some time...   I keep running into one challenge or limitation - not supported in WSL, RocM not supported, Ollama not supported, etc.    So many different common AI servers don't support it....

I've now installed the Ryzen AI software driver and libraries from here:  https://ryzenai.docs.amd.com/en/latest/inst.html

It was challenging because it says it's installing in one place, but installed in another place...    For whatever reason the directory path is different than what the installer was saying.   Certain .dll files were missing when I tried to run things, even though they were installed in C:\Program Files\RyzenAI\1.4.0.  -- so I had to copy them into the directory I am running python in.

I've downloaded models from AMD's section of hugging place, and now when I startup a server for these models I'm getting the error: Error loading model: VitisAI execution provider is not supported in this build.

C:\SavedDev\AI\NPU>python ryzen-npu-server.py --model "C:\Users\danie\models\Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix" --device HYBRID --port 8765
Loading model from: C:\Users\danie\models\Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix
Execution device: HYBRID
Error loading model: VitisAI execution provider is not supported in this build.

* Serving Flask app 'ryzen-npu-server'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on all addresses (0.0.0.0)
* Running on http://127.0.0.1:8765
* Running on http://192.168.1.163:8765

This is quite frustrating...    Can you please help me figure out where to go from here?    Is this NPU actually a supported product?

Thanks,

     - Daniel

[joegreene22]

<p>Ryzen AI support still feels pretty early-stage. Even with the official SDK and documentation, it's hard to tell what's actually working or supported. Hopefully, AMD provides better tooling and model compatibility over time.</p>

[Uday_Das]

It is not very clear what exactly you trying to do. You are showing you are running your own script  ryzen-npu-server.py, did you run provided example script first?

There is clear instruction here: https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html 

What you really need to do install driver, install lightweight installer (this installer is fine for running precompiled LLM) from the installation page https://ryzenai.docs.amd.com/en/latest/inst.html 

And then follow page every step which says how to run your model using python  code: 

https://ryzenai.docs.amd.com/en/latest/inst.html 

You should make sure what is written working for you before trying something else.

Thanks