I know this is old but hopefully this will help somebody in the future as it was very difficult to figure out how. I was not able to get mxGPU to work either so I'm giving up and just passing through the GPUs. I found this page, https://xcp-ng.org/docs/develprocess.html#kernel-module-policy, which talks how to remove modules.
In short:
rmmod gim # to unload module if it is already loaded
yum remove mxgpu-4.19.0+1-modules.x86_64
# then reboot server, supplemental pack should no longer be there now