Just stick to "-march=native -O2 -pipe"
All the rest of exotic flags will just screw your system and for most packages you won't even get more speed- they could be actually slower !
-O2 has the most sensible optimisation subset that works best normally.
On a few packages that you feel strongly about, you can use some other CFLAGS.
-flto and -fwhole-program are all the rage these days, but their use is linked to so many headaches these days that I would avoid them in a wide circle, IIWY.
They might work on some latter, stabilised gcc-4.6 but not now.
And don't use gcc-4.6 for your system yet. It still has plenty of critical bugs.
Use latest 4.5* instead.