TLDR: After the user clicks through the legal blurb to enable "Custom Resolutions" (aka use at your own risk), when attempting to create a valid mode for an attached CRT, the AMD driver returns "The custom resolution settings are not compatible with this display. Reconfigure your settings and try again.", making it impossible to build the desired custom resolution mode. The desired behavior is to match what NVIDIA does which is to NOT attempt to revalidate the mode, and instead use the settings "as is" from the user.
Details,
(1.) This is quite literally a one line fix in the driver. Just comment out the code which double checks the modeline against the display DCC EDID specifications, and alternatively against some specific hard coded minimums and maximums. The user had already clicked through the legal blurb to use at their own risk. This secondary check does nothing except disable legitimate usage.
(2.) Some HDMI to VGA converters report incorrect DCC EDID specs to the video card. So if the driver checks against the EDID it will disable modes which actually work on the attached display and converter. Also if the driver disables modes say under 480 lines, it will disable legitimate modes useful for vintage gaming (similar to these https://www.retrorgb.com/mister-240p-120hz-on-a-vga-crt-monitor.html). So the driver shouldn't be second guessing what the user sets.
(3.) This problem exists on current and many prior AMD drivers, as of latest on December 15 2022. I can repro on a 5500M laptop using an HDMI output. But GPU shouldn't matter.
(4.) This problem does not exist on NVIDIA drivers or hardware (all of them work just fine). This is probably why nearly everyone using CRTs for low latency gaming are on NVIDIA GPUs. Lets change that.
(5.) Here is a reproduction case that I use. Attach a standard VGA CRT (something that does 31Khz and above, like an old ViewSonic 17GS for example) to an HDMI to VGA converter like a HDfury Nano. Build a custom resolution for "720x240 @ 120 Hz" using CVT Reduced Blank as the timing generator (note this option is supported the same on both NVIDIA and AMD drivers). On AMD this will throw the error message and refuse to generate the mode, and on NVIDIA it works just fine. In theory using a 240 line mode at 120 Hz would hit two of the possible causes for the bug.
