AI Discussions

[Giganotauroghz]

How to run a Large Language Model (LLM) on your AM... - AMD Community

Do LLMs on LM studio work with the 7900xtx only on Linux? I have Windows and followed all the instructions to make it work as per the blog I'm sharing here and got this error that I tried to post here but apparently am not allowed to. The error basically stated that there was a problem with either the configuration or model itself but I even allowed the studio to load the respective configuration for each model I tried. It kept saying "Try a different model and/or config." I tried with different 7b models and made sure they were all the Q4M versions of the GGUF but could not get them to work. I would select them, they would try to load, and then give me the error.  I kept getting the same error. I will highlight that this is my first experience using LM Studio for LLMs.

[djxmdjxm]

I have the same problem. when I select openCL it work - but pretty slowly. When I select "ROCm" I get this error every time regardless of the number of layers - even 0: 

```json
{
  "cause": "(Exit code: 0). Some model operation failed. Try a different model and/or config.",
  "suggestion": "",
  "data": {
    "memory": {
      "ram_capacity": "31.91 GB",
      "ram_unused": "17.85 GB"
    },
    "gpu": {
      "gpu_names": [
        "AMD Radeon RX 7900 XTX",
        "AMD Radeon RX 7900 XTX"
      ],
      "vram_recommended_capacity": "47.97 GB",
      "vram_unused": "23.86 GB"
    },
    "os": {
      "platform": "win32",
      "version": "10.0.22631",
      "supports_avx2": true
    },
    "app": {
      "version": "0.2.24",
      "downloadsDir": "E:\\Users\\Chris\\.cache\\lm-studio\\models"
    },
    "model": {}
  },
  "title": "Error loading model."
}```

[Giganotauroghz]

You have to install ROCm, the HIP packages. It may have been from this page (https://rocm.docs.amd.com/en/docs-5.6.0/deploy/windows/quick_start.html). I don't remember well. Delete your LM Studio app if it's old and install the new one.