Read those architecture manuals:
For older Very Large Instruction Word hardware -> http://developer.amd.com/sdks/AMDAPPSDK/assets/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf
For Graphic Core Next -> http://developer.amd.com/sdks/AMDAPPSDK/assets/AMD_Southern_Islands_Instruction_Set_Architecture.pdf
^^^^ Finally it's out 
Read those architecture manuals:
For older Very Large Instruction Word hardware -> http://developer.amd.com/sdks/AMDAPPSDK/assets/AMD_HD_6900_Series_Instruction_Set_Architecture.pdf
For Graphic Core Next -> http://developer.amd.com/sdks/AMDAPPSDK/assets/AMD_Southern_Islands_Instruction_Set_Architecture.pdf
^^^^ Finally it's out
Hi jg1,
Note that the inner elf has about 5 'program' sections 3 of which overlap some of the normal text sections,
this is helpful when calculating the new section offsets and sizes. The program sections have their own
program headers instead of the regular section headers.
1. Replace the gcn binary code in Inner file: second text section.
2. Calculate and change all required sizes in section headers and program headers.
3. Change a few parameters in the inner file 4th (index 3) 'program' section.
(this includes things like the no. of SGPRs, VGPRs, lds and gds memory requirements)
After packing the inner elf
4. Change appropriate section sizes in outer elf section headers.
5. In the outer elf ".rodata" section change the lds memory maximum if needed (see attached notes/code snippets)
6. In the outer symbol table entries, change the size of the section with the kernel name (need to look at strtab for the name,
again there is a code snippet in the attached notes.)
Then, pack that thing! 
There is more info in the notes that I couldn't paste here because this editor keeps inserting everything into tables.
The notes also mention how to change the parameters in the 4th program section in the inner file. Basically you look
for a 32 bit key code first and then read/write the data in the following int. There is a table of important key codes in
the notes.
Note, some of the data output from my program does not need to be changed, i.e., the uav memr buffer register assignments and type.
These are just for reference and/or can be used for programming at a higher level, like a compiler.
Good luck, it's really not that difficult, only made a little complicated by the elf format.
Sorry for the jumbled notes.
drallan
The linkdat structure looks like the "device configuration table" mentioned in the CAL Programming Guide. [...]
typedef struct {
Elf32_Word address; /* device address */
Elf32_Word value; /* value */
} CALProgramInfoEntry;
It is probably used by the driver to program the GPU chips through device I/O ports.
Yes, I think that must be correct.
The "device address" also looks like a PCI bus IO transaction.
I checked, the entries in linkdat were all dug from code, I don't have any other docs on the device configuration table values.
drallan
Hi drallan and realhet,
My repacker is now working. Thank you for all your help. I really appreciate it.
For the purpose of repacking ISA code (with the same kernel function signature), drallan's instructions are good enough.
I also find good information in this file. Look for struct si_bin_enc_dict_entry_t, and enum _E_SC_USER_DATA_CLASS. You might want to look at other files at the web site to see how the dots are connected. I modified my readelf program again and its output is in the attachment.
I still have not found information about the use of GDS_MAX, LDS_MAX, and LDS_AUX. But they do not seem to matter with the level of complexity of my current kernel. I will post here if I find anything.
jg1
Wow! Nice find -> multi2sim
http://www.multi2sim.org/svn/multi2sim/trunk/src/arch/southern-islands/emu/machine.c
This seems to be an accurate 'documentation on every GCN instructions 
Also there's a complete* inverse assembler -> http://www.multi2sim.org/svn/multi2sim/trunk/src/arch/southern-islands/asm/asm.c
I think don't you wanna understand all the values in the .elf, only patch that few you really use and leave the rest for the ocl compiler.
GDS_MAX, LDS_MAX, and LDS_AUX: That's new to me too: I only tried LDS on CAL, but there the only thing I had to change was the control word marked with PGM2_COUMPUTE_RESOURCES in the disasm. If I recall bits8:15 (or something like this) represented the LDS allocation size. But seems like OCL stores it redundantly.
"as10.txt.zip" <- Well done! 
Edit: Oh noes, multi2sim is full of inumplemented instructions
Hi realhet,
You are right. Multi2sim can be a good reference when we need to find out execution details of an ISA instructions. I have not studied yet but it is good to know where to look to when I need.
Thanks for the info of LDS. They are bits 8:16 in COMPUTE_PGM_RSRC2. So you are only a tiny "bit" off. 
jg1 wrote:
Hi drallan and realhet,
My repacker is now working.
For the purpose of repacking ISA code (with the same kernel function signature), drallan's instructions are good enough.
I also find good information in this file. Look for struct si_bin_enc_dict_entry_t, and enum _E_SC_USER_DATA_CLASS. You might want to look at other files at the web site to see how the dots are connected. I modified my readelf program again and its output is in the attachment.
I still have not found information about the use of GDS_MAX, LDS_MAX, and LDS_AUX. But they do not seem to matter with the level of complexity of my current kernel. I will post here if I find anything.
That was fast! Congratulations.
I have the multi2sim files. They are very good for gcn code but they don't have the device configuration table info (Where is it!?
)
The _E_SC_USER_DATA_CLASS looks real interesting but the numbers don't seem to correlate with anything. Maybe specific to multi2sim?
The Linux Mesa/Gallium project also has some (too many ! ) files but they are a bit less clear.
You can google mesa southern islands.
I would be very interested if you or realhet find any more good info, and happy to share anything I find too...
drallan
Hey,
Do you wanna make your ISA wrapper to be able to define kernel parameters as well?
I think it gonna be really usefull, but where to get all the information for it from
I'm thinking it more simpler: Don't wanna understand how those 16 elements works and also understand HOW OPENCL PASSES BUFFER ptrs. I'll need to use them from the asm side also. And what if AMD changes that structure (the can rewrite it, and generate different amd_il code, you know)?!
I'm thinking about to compile skeleton opencl code and then patch it automatically.
Anyways, now I try to make a 99% OpenCl compatible patcher which works without LDS/GDS. And after I have some question marks about LDS/GDS.
realhet wrote:
Hey,
Do you wanna make your ISA wrapper to be able to define kernel parameters as well?
I think it gonna be really usefull, but where to get all the information for it from
I'm thinking it more simpler: Don't wanna understand how those 16 elements works and also understand HOW OPENCL PASSES BUFFER ptrs. I'll need to use them from the asm side also. And what if AMD changes that structure (the can rewrite it, and generate different amd_il code, you know)?!
I'm thinking about to compile skeleton opencl code and then patch it automatically.
Anyways, now I try to make a 99% OpenCl compatible patcher which works without LDS/GDS. And after I have some question marks about LDS/GDS.
Yes, I am interested in a more complete solution. I do not need it at this moment but knowing the use of various bits gives me more confidence that any hacking/patching will work in different conditions. I will look into it when I have more time.
jg1
Something's not round with gds/lds:
drallan:
{0x80000081,"GDS_MAX "}, //???
{0x80000082,"LDS_MAX "}, //also 80001045 see below
{0x80001045,"LDS_AUX "} //also 80001045 see below
+ .rodata
multi2sim:
{ "AMU_ABI_LDS_SIZE_AVAIL", 0x80000081 },
{ "AMU_ABI_LDS_SIZE_USED", 0x80000082 },
I say:
key 0x80000081 is always $80000, whatever I do o.o
key Computr_pgm_rsc2 bits[16:24] is the actual LDS size
What's the truth? On CAL my version works fine, I tested it. Doing some lds size comparison soon.
Other stuff:
There is the 0th program entry in the cal.elf. It's type in $70000002 and inside there are offset/size pairs referencing to the amd_il and isa sections offseted with a misterious offset. Do you know this? I can shift it anyways according to the new isa size. Maybe it's an offset relative to the OpenCl.elf image...
hi realhet,
Just curious what script language are you using in your attachment?
One question:
SetCalNote($00002e13,(opts.ldsSizeBytes+255)shr 8 shl 15,$FFF07FFF{and mask});
maybe it is a typo of {or mask}?
I verified this with my kernel. the LDS size is indeed in bits [15:23].
jg1
Something's not round with gds/lds:
drallan:
{0x80000081,"GDS_MAX "}, //???
{0x80000082,"LDS_MAX "}, //also 80001045 see below
{0x80001045,"LDS_AUX "} //also 80001045 see below
+ .rodata
multi2sim:
{ "AMU_ABI_LDS_SIZE_AVAIL", 0x80000081 },
{ "AMU_ABI_LDS_SIZE_USED", 0x80000082 },
multi2sim is more round!
0x80000082 LDS_SIZE_USED is the same as LDS_MAX (max used by a workgroup)
0x80000081 is LDS_SIZE_AVAIL (LDS hardware limit, so it affects nothing)
So GDS_MAX is wrong and may not exist because GDS is global and seen by all CUs and workgroups.
At least the question marks were right ! 
.rodata Under some conditions LDS values had problems. The value of LDS_USED also appears in the 8 integer block at the end of the .rodata section just after the text part. After changing both locations, the problems disappeared.
drallan
