This is a well known problem acknowledged by Valve. It is particularly bad on AMD hardware:
Seems to be a poor implementation of interop2 on AMD hardware?
This is going to make VR on AMD hardware, with applications that run OpenGL, a poor experience. Are you working with Valve to resolve this?