Problem solved so far.
The windows 10 settings were set to 59Hz instead 60Hz. But the Radeon driver software showed me in system settings a refresh rate of 59,939Hz, so I didn't bother. After a long night I finaly checked windows setting, instead believing in Radeon drivers systems information, and found 59Hz. Who needs 59Hz? I changed it to 60 Hz, now its running with 10bpc. And the radeon driver shows now a refresh rate of 59,996Hz. The difference between 59,939Hz to 59,996 Hz makes the difference? I can't believe.....