Most likely it sound like Coil Whine if you have definitely verified it is coming from your new GPU Card.
Coil While occurs in High powered hardware like your PSU, GPU, or VRM on your Motherboard. There isn't anything you can do except RMA the GPU card if it is excessively loud or annoying.
When power flows through certain electronic components it causes those components to vibrate to produce the noise you are hearing. Generally under heavy power usage the Coil Whine will increase also as more power is applied to your GPU card.
NOTE: Coil Whine is harmless just annoying if it is loud enough.