The U3011 won't give you 2560x1600 without DP or DualLink DVI. HDMI 1.4 simply doesn't support that resolution, that's why you only see up to 1080p.
Not sure why the 4k 60Hz doesn't work for you on the other display though if you are using native DisplayPort. Have you tried a better/shorter DisplayPort cable?
I have three DisplayPort cables, and all of them work from MacBook to UP3216Q, so I would assume the cable isn't the problem. It's current connected from DisplayPort R9 380 to Mini DisplayPort UP3216Q. Any other ideas? It seems to be locked, i.e. cannot select 60Hz. Not sure why this is so frustrating.
Can you check if your monitor is set to DP 1.2 mode?
On the monitor: Menu -> Display Settings -> Display Port 1.2 -> Enable.
I just tried the same Dell 4k display and it works fine at 60Hz on my R9 360.
However if I disable Display 1.2 on the display I do get only 30Hz like you are seeing.
BTW, I am using the latest 16.1.1 hotfix driver on Win10 64.
I ended up getting it to work by plugging a different monitor into DVI-D
and then plugging the UP3216Q at the same time. A couple of flashes later,
and both were showing 60Hz. Very strange, but I hope AMD eventually reduces
the number of bugs in its software. This has been particularly frustrating.