It seems it was a cooling issue.
The case I have is simply too small and the GPU is just getting recycled hot air.
I opened the side of the case and everything is dandy now.
I think these cards are better with a water cooler.. The heat coming from the heat sink seems to rise and then get caught into my CPU fan.
I also set the fans to cut in earlier.
They are extremely quiet fans.. very impressive non the least.