Nits do not get enabled on your hdr tv. The tv is always ready to receive hdr at all times. Its only when the metadata tag is sent via hdmi that the tv switchs its color to bt2020 and sets the contrast and brightness off and lets the metadata tags add it to show the hdr picture. Windows 10 attempt to remove the contrast and brightness happens when you slide the hdr tab to on. The screen becomes pale. An hdr uhd file is pale at the start. Not sure if you understand how it works. Forget gamma curve and nits. HDR is 10 points contrast added to produces the dynamic range of the content. Take any uhd file and load it in xdr. Adjust the contrast up and brightness down. match the saturation with contrast leave the tint at 0 if you can force the advanced color to set it to bt2020. If not add 12 points tint to make up for the color shift as and hdr file appears dull in bt709 mode. Adding bt2020 if you can will give you the deeper blues and reds and greens. The tv can remain in vivid mode or whatever picture mode you use for normal bluray. As for Madvr its is totally wrong when using the dci options to convert hdr to sdr. It makes the shadows way to dark and blows out the darks. XDR leaves the whitelevel at the highest it can go while retaining the blacks through the entire spectrum. The only way to put this to rest is to have the original uhd disk and a captured copy. Run the original in hdr mode then run it on pc with xdr adjustments with hdr off on windows and hdr off on the hdr tv. Watch both. THEY ARE EXACT!!!
Does not work for me. I can set to 10 bit and get BT2020 playback if I set it in the xdr.cal file no problemo. But none of those sliders (or editing them in the file) gives me the light output that playing back HDR via HDR mode on the TV does. The light levels are SDR levels. I can push contrast and brightnes around as much as I like in XDR/MPC-BE64 and it makes not a hoot of difference.
EDIT: A new version of MadVR just came out which fixes the issue. HDR switching now works correctly so I have no need for your pgram. Thanks for trying though.