First is this a Pre-built PC like HP or DELL OEM PC?
Possibly the PSU doesn't have enough power to boot with that new GPU card installed. Especially if it is fairly old PSU and not very efficient one at that.
But, in my opinion, it should at least boot up into Windows Desktop.
Does it boot up with your older weaker GPU card installed?
What Motherboard Trouble LED lights are on or Digital Code being shown or BEEP Sequence you are hearing?