Hi,
I'm planning to do an ISA optimization on my existing kernel, and just wondering if anybody have some info on how to feed the S and V ALU's to achieve maximum utilization.
On the GCN .ppt slides I've seen that there is an instruction 'arbitrator' which is feeding 4 Vector alu's and 1 Scalar alu. It also tells that the S alu is working 4x faster than the V alu's. In total it gives 1:1 V:S alu operation capacity.
For example: what if I interleave independent V_ and S_ instructions in a pattern like SVSVSVSV?
Will the instruction decoder in the SIMD engine be able to feed all four V alu's in every 4 cycles while also feeding the S alu with one instruction every cycle (for each of the 4 V alu's)?
What if I use larger VOP3 instructions and/or 32bit immediate constants, when the instruction decoder will have to look more dwords ahead? Is there any specifications on the capabilities/limitations of the instruction decoder/arbitration unit?
Why am I doing this: I have a quiet big kernel (25KB on 7970) which is working at 98% alu utilization on the 6970, but unfortunately on 7970 it runs out of VRegs (above 128 vgprs I noticed a 'task scheduler' bottleneck because it can't put 2 kernels in the queues. When a task is done, the simd engines can't start immediately another task. It means -30% performance loss in my case). My plan is to use more sgprs instead of some of the vgprs to get below the 128 vgprs 'limit'. I can swap many calculations out to S registers which are the same values for every 16 simd vector threads, I just have th learn, how effectively schedule the SOP's and VOP's to achieve maximum alu utilization.
In my worst expectations maybe there is no S/V paralellism, and in every 4 cycles either 1 VOP or 1 SOP can be executed. This way there is no need for hardwaredependency checks or special compiler instruction reordering.
I also noticed the s_buffer_load_dwordx16 thing, another reason to go down below CAL 
Thank you for your answers!
1 Solution
I can't tell sprofile values sorry. (not using VS neither OpenCl, and launching sperform manually on my .exe just fails)
According execution time measurements, I can utilize the V alu on 98-99% ,but only in special circumstances.
On the 7xxx you can try these hints:
- Unroll a big kernel (16KB ISA size is ok, max 32KB on 7xxx and max 48KB on 4xxx..6xxx)
- Optimize your algo in a way that doesn't use more than 64 VRegs (on the 4xxx..6xxx hardware 128 registers will do fine)
I guess that poor 24% alu utilization can come from using too much VRegs on the 7xxx
"Is there still a big benefit in using vector types?"
- on 4xxx..6xxx you only need to duplicate long dependency chains in order to let CalCompile feed the 4 or 5 VLIW lanes from them. If theoretically there is a way to utilize all vliw slots, then calCompile will find it
- on 7xxx there is no ALU benefit from vectorizing code.
- but of course still need to vectorize for effective memory access.
realhet schrieb:
According execution time measurements, I can utilize the V alu on 98-99% ,but only in special circumstances.
On the 7xxx you can try these hints:
- Unroll a big kernel (16KB ISA size is ok, max 32KB on 7xxx and max 48KB on 4xxx..6xxx)
- Optimize your algo in a way that doesn't use more than 64 VRegs (on the 4xxx..6xxx hardware 128 registers will do fine)
My Kernel is quite small, just under 1kB. And i use only 10 VRegs and 25 SRegs ~40 VALUInst and ~12 SALUInst.
realhet schrieb:
"Is there still a big benefit in using vector types?"
- on 4xxx..6xxx you only need to duplicate long dependency chains in order to let CalCompile feed the 4 or 5 VLIW lanes from them. If theoretically there is a way to utilize all vliw slots, then calCompile will find it
- on 7xxx there is no ALU benefit from vectorizing code.
- but of course still need to vectorize for effective memory access.
Exactly how i thought it should be...
My pogram is a heatblade simulation. So access patterns are allways in raws, one element next to the other.
The funny part is, when i use LDS, my runtime increase. My Cache hit goes from ~35% up to ~90%, but the runtime is 45% higher..
Also when i change the order of accesses to a inverted pattern (because i use 2D array in LDS), there change nothing.
Really strange... I hope AMD brings soon the optimization guide for GCN.
But please let's get back to the topic,
So far I've found out that the V and S instructions can be interleaved in an 1:1 ratio and they will execute simultaneously.
For example:repeating this over and over:
s_xor_b64 s0, s1, s2
v_xor_b32 v0, v1, v2
eats only 4 cycles, not 8.
another one:
s_mul_u32 s0, s1, s2
v_mad_u24_u32 v0, v1, v2,v3
also run in paralell (in this case there is an instruction decoder bottleneck because the two 64bit instructions, so the S:V ratio can be at most 3:4)
With this I was able to get additional 88 Gops of mul32 performance out of the 7970
Now it generates a question: Is there a hardware mechanism that ensures that no S and V instructions can be executed in paralell when they're depend on each or other? Or is it the compiler's responsibility that it never send register-dependent S/V code to the GPU, like in the example below?
s_xor_b64 s0(!!!!), s1, s2
v_xor_b32 v0, s0(!!!!), v2 ;does the instruction decoder delay the V instruction until the result is calculated by the S instruction? Or execute both using the previous s0 register value in the V instruction.
(My opinion is that there is no extra hardware for this (thus more transistors can do math), and the compiler must take care of these situations, but I'm not sure)
If anyone has accurate info on this, please tell me!
Let's say 256 vregs/wave -> 4bytes*256regs*64wavesize*4simd/cu=256KB which is exactly the amount of vregister ram in a CU. Exactly like in http://developer.amd.com/afds/assets/presentations/2620_final.pdf
But I'm pretty sure that its not good to use more than 128, and the best performance can be achieved when not using more than 64 vregs. Just don't know, why is it 
. I'll do some simple tests tomorrow to point out this behaviour.
Hello again,
I'm only testing the raw computing power, no lds/gds/uav at all, and my test are fitting well in all of the caches (instr and data).
I've ran some tests:
The test kernel was this in ISA:
s_mov_b32 s10,24
label_1:
;128 times repeated(unrolled) [inner_code]
s_add_u32 s10,s10,-1
s_cbranch_scc1 label_1 ;break on borrow, so it's a 25x loop
s_endpgm
inner_code was basically 4 V instructions interleaved with 0..3 S instructions
Trying a mixture of 32bit and 64bit S/V instructions. A capital letter V or S means 64bit.and small letters are 32bit.
For example: VSVsvv stands for
v_mad_i32_i24 v0,v0,v1,v2 ;big 3 operand V instr
s_mul_i32 s0,12345,s1 ;big 2 operand S instr with 32bit immed
v_mad_i32_i24 v0,v0,v1,v2
s_mul_i32 s0,s0,s1 ;small 2 operand S instr
v_xor_b32 v0,v0,v1 ;small 2 operand V instr
v_xor_b32 v0,v0,v1
The kernel was issued for 40 000 000 workitems (64 workitems/workgroup) and the second time measurement was taken.
Here are the diagrams:
http://x.pgy.hu/~worm/het/7970_isa_test/7970_SV_timings_4-12dwords.png
http://x.pgy.hu/~worm/het/7970_isa_test/7970_SV_timings_8-16dwords.png
Also the raw data in excel:
http://x.pgy.hu/~worm/het/7970_isa_test/7970_isa_speedtest.xls
My conclusions:
ONE CAN SIMPLY NOT interleave 4 S instructions with 4 V instructions, or it will slow down the V performance by a minimum of 1-2%
Also when sending more code dwords to the S alu, there is on higher numVGprs kernel settings:
It can be partitioned to 3 parts:
16..64 numVGprs -> excellent V and S paralellism (in my thoughts this is when only 4 waves are in the CU)
65..84 numVGprs -> this starts to hate when it gets many S instruction dwords (I think 3 waves can sit in a CU)
85..128 numVGprs -> 2 small S for 4 V is ok, bug starts to get slow (2 waves)
129..255 numVGprs -> try to avoid ALL S instructions if posibble, or else there will be terrible stalls (1 waves)
If there is a 4 stage pipeline thing in it, then multiply my waves_in_the CU expectations by 4 of course.
But I think all this S instruction 'sensitivity' is based on the complicated instruction decoding/scheduling mechanism. I'd love to know more about it, if it's not a secret
I don't even think about what things apply for the v_add instruction when it reads and writes from/to both S and V registers.
Anyways It's really a fun puzzle to find out these things. (I have still so many questions to ask lol, just don't wanna flood the forums)
