I have six 4k monitors, and i would like to attach them on my Linux(ubuntu/RHEL). I tried it with my old two nvidia NVS 510 cards, but failed. Although it works well on win10 with 6*4k. Does anyone has this experience or suggestions?Appreciate for your help!!!
In Linux you CAN do it but you can't easily do it with the official nvidia modules; you have to use noveau.
With an AMD card you should be able to do it, but I myself am having issues with multihead off of DisplayPort using the open source modules and have thus far received no real help nor attention; so you may have the same issues if this is an issue on cards other than the RX 5600 XT which I have or if you're using the closed source "pro" drivers which are not offered on non-LTS versions of Ubuntu so I have not been able to test them myself:
The real issue here, and the reason why you weren't able to do this with nvidia cards [assuming you are using the closed source modules] is that the graphics card "driver" modules must be integrated into the kernel/system in such a way that X/Wayland/etc. can address each display without having to take any consideration into what hardware that display is connected to. This essentially requires the display just be an "address" which the window manager or whatever can send information to, and that usually requires some abstraction to be implemented in the module. The AMD open source modules DO implement this and they implement it well, the noveau modules DO implement this but they do not support many other features nor do they provide much performance, the closed-source nvidia drivers DO NOT implement this, and because of this you need to run a completely separate X stack for each card if you have displays connected to more than one card or displays connect to a card and to your on-board graphics. In contrast, with an AMD card and the open source modules, it is trivial to connect many displays to many ports on many cards and/or your on-board graphics and everything works seamlessly and completely transparently.
As an anecdotal example: even though I am having issues with the 3 DisplayPorts on my card, I'm able to drive 3 displays with the single HDMI on my AMD card and off two of the onboard graphics ports I have on my motherboard - it's completely seamless and transparent and required no installation of additional software or specific tweaks.