cancel
Showing results for 
Search instead for 
Did you mean: 

OpenCL

amd4
Journeyman III

How to fill H264_picture_parameter_2 structure

Hi all,

I'm trying to implement an H264 decoder using opencl 1.2 and I'm currently studying the OVDecodeRender sample project.

SideNote: I have been trying to compile the OVDecodeRender.zip sample, but it seems like it is based on SDK 2.7 and it

failed with 2.8. Do you have any update of this sample? -I have manage to understand what I want from the sample, so this

is not really urgent matter.

I want to use the H264 decoder to decode an RTP stream in baseline profile. It appears that I need to decode myself all the SPS and PPS structure?

Is it possible to avoid this? (I worked with several h264 decoder and all of them just need the SPS and PPS NAL untouched which is far

easier....).

I have started to parse SPS and PPS, however, I'm not able to understand some of the value of H264_picture_parameter_2

What is the exact required values for "picture_parameter.level"?

    For example for level 4.1 -> picture_parameter.level = 41?

Is pic_width_in_mbs_minus1 the exact value for "picture_parameter.width_in_mb"?

Is  pic_heigtht_in_map_units_minus1 the exact value for "picture_parameter.height_in_mb"?

I have no idea what should be in "picture_parameter_2.picture_structure"?

I'm not sure at all how I should fill those last 3 values:

        picture_parameter_2.frame_num = 0; //?

        picture_parameter_2.intra_flag = 1; //?

        picture_parameter_2.reference = 1; //?

If you have any hints about what should be in "slice_data_control" and how I should know

the number of slice... I would also be greatfull.

Tks to all

0 Likes
Reply
35 Replies
himanshu_gautam
Grandmaster

Re: How to fill H264_picture_parameter_2 structure

SideNote: I have been trying to compile the OVDecodeRender.zip sample, but it seems like it is based on SDK 2.7 and it

failed with 2.8. Do you have any update of this sample? -I have manage to understand what I want from the sample, so this

is not really urgent matter.

I want to use the H264 decoder to decode an RTP stream in baseline profile. It appears that I need to decode myself all the SPS and PPS structure?

Is it possible to avoid this? (I worked with several h264 decoder and all of them just need the SPS and PPS NAL untouched which is far

easier....).

Hi amd,

What errors are you getting in compiling the sample with SDK 2.8.?

AFAIK, it is required to extract out the structures containing decoded information on a frame by frame basis, as shown in sample. Anyways I will ask someone more knowledgeable to help you

amd4
Journeyman III

Re: How to fill H264_picture_parameter_2 structure

Hi / Tks for your answer!

What errors are you getting in compiling the sample with SDK 2.8.?

First error was that the location of include

#include <OVDecode/OVDecodeTypes.h>

#include <OVDecode/OVDecode.h>

while the file exist in app SDK 2.8 in:

#include <OpenVideo/OVDecodeTypes.h>

#include <OpenVideo/OVDecode.h>

The second error was about usage of deprecated APIs.

Using #define CL_USE_DEPRECATED_OPENCL_1_1_APIS solved that one.

Then some link issue with checkVal, error, etc.. I have solved those ones // just removed the

macro to not spend time on it.

At least, then I have a binary compiled. I have no screen on my "asus HD 7750" so the app

won't work as it look for a display connected.

But it's not a big issue and not my real question!

AFAIK, it is required to extract out the structures containing decoded information on a frame by frame basis, as shown in sample. Anyways I will ask someone more knowledgeable to help you

Tks! I'll keep waiting for more!

Aymeric

0 Likes
Reply
amd4
Journeyman III

Re: How to fill H264_picture_parameter_2 structure

Hi Again,

I already did some progress and I'm able to run my application.

I have OVDecodePicture returning true: so I guess, I'm getting closer. However, I still

haven't been able to display the correct NV12 image.... I have pixel movings and I can

see a form in the displayed/decoded image which is clearly ME... but it's not yet working.

The image (my hand) is green and other part are grey.

First, these are the parameters I have extracted from my SPS/PPS: hardcoded currently

for testing purpose:

       ovd_picture_parameter    picture_parameter;

        H264_picture_parameter_2 picture_parameter_2;

        ovd_slice_data_control   slice_data_control[1]; /* with slicenum=1 */

        unsigned int num_event_in_wait_list=0;

        memset(&picture_parameter, 0, sizeof(ovd_picture_parameter));

        picture_parameter.codec_type = 1; //H264

        picture_parameter.profile = 1;

        picture_parameter.level = 40;

        //368x480

        picture_parameter.width_in_mb = 22+1; // from sps pic_width_in_mbs_minus1   

        picture_parameter.height_in_mb = 29+1; // from sps pic_heigtht_in_map_units_minus1

        picture_parameter.decode_flag = 0;

        memset(&picture_parameter_2, 0, sizeof(H264_picture_parameter_2));

        picture_parameter_2.sps_info.sps_flag.delta_pic_always_zero_flag = 0; //from sps (because pic_order_cnt_type == 2)

        picture_parameter_2.sps_info.sps_flag.gaps_in_frame_num_value_allowed_flag = 0; // from sps

        picture_parameter_2.sps_info.sps_flag.frame_mbs_only_flag = 1;

        picture_parameter_2.sps_info.sps_flag.mb_adaptive_frame_field_flag = 0; // from sps (because frame_mbs_only_flag==1)

        picture_parameter_2.sps_info.sps_flag.direct_8x8_inference_flag = 1; // from sps

        picture_parameter_2.picture_structure = 3; //no idea if this is right?

        //ALL 0 (and not present) if profile_idc=66

        picture_parameter_2.chroma_format = 1; //0 for baseline

        picture_parameter_2.sps_info.sps_flag.residual_colour_transform_flag = 0; //0 for baseline.

        picture_parameter_2.bit_depth_luma_minus8 = 0; //0 for baseline

        picture_parameter_2.bit_depth_chroma_minus8 = 0; //0 for baseline

        picture_parameter_2.log2_max_frame_num_minus4 = 0; // from sps

        picture_parameter_2.pic_order_cnt_type = 2; // from sps

        picture_parameter_2.log2_max_pic_order_cnt_lsb_minus4 = 0; // from sps, because pic_order_cnt_type == 2

        picture_parameter_2.num_ref_frames = 1; // from sps

        picture_parameter_2.reserved_8bit = 0;

        picture_parameter_2.pps_info.pps_flag.entropy_coding_mode_flag = 0; //from pps

        picture_parameter_2.pps_info.pps_flag.pic_order_present_flag = 0; //from pps

        picture_parameter_2.num_slice_groups_minus1 = 0; //from pps

        picture_parameter_2.slice_group_map_type = 0; //from pps (because num_slice_groups_minus1==0)

        picture_parameter_2.slice_group_change_rate_minus1 = 0; //from pps (because num_slice_groups_minus1==0)

        picture_parameter_2.num_ref_idx_l0_active_minus1 = 0; //from pps

        picture_parameter_2.num_ref_idx_l1_active_minus1 = 0; //from pps

        picture_parameter_2.pps_info.pps_flag.weighted_pred_flag = 0; //from pps

        picture_parameter_2.pps_info.pps_flag.weighted_bipred_idc = 0; //from pps

        picture_parameter_2.pic_init_qp_minus26 = 0; //from pps

        picture_parameter_2.pic_init_qs_minus26 = 0; //from pps

        picture_parameter_2.chroma_qp_index_offset = 0; //from pps

        picture_parameter_2.pps_info.pps_flag.deblocking_filter_control_present_flag = 1; //from pps

        picture_parameter_2.pps_info.pps_flag.constrained_intra_pred_flag = 0; //from pps

        picture_parameter_2.pps_info.pps_flag.redundant_pic_cnt_present_flag = 0; //from pps

        //from pps (if stop bit wasn't set)

        picture_parameter_2.pps_info.pps_flag.transform_8x8_mode_flag = 0;

        picture_parameter_2.second_chroma_qp_index_offset = 0;

        picture_parameter_2.reserved_16bit = 0;

        picture_parameter_2.frame_num = 0; //?

        picture_parameter_2.intra_flag = 1; //?

        picture_parameter_2.reference = 1; //?

        memset(&slice_data_control[0], 0, sizeof(ovd_slice_data_control));

        slice_data_control[0].SliceBytesInBuffer = end-p;

        slice_data_control[0].SliceBitsInBuffer = slice_data_control[0].SliceBytesInBuffer*8; //?

        slice_data_control[0].SliceDataLocation = 0;

Do I have to analyse each NAL (other than SPS/PPS) and extract some of the data? Or can

I just provide the other NAL as they are?

Are those values correct:

// in wireshark I see in sps pic_width_in_mbs_minus1    == 22

        picture_parameter.width_in_mb = 22+1;

// in wireshark I see in sps pic_heigtht_in_map_units_minus1== 29

        picture_parameter.height_in_mb = 29+1; // from sps

Is there any value that should be changed for each NAL in those structure?

Tks

Aymeric

0 Likes
Reply
amd4
Journeyman III

Re: How to fill H264_picture_parameter_2 structure

Hi All,

no answer yet - I'm doing -possible- progress myself so it might be usefull to post where I am.

1/ I'm still only able to view a broken frame where I clearly see part of the correct image (you can see my head, my hairs? I wish you could see better...)broken-video-openvideo.PNG

2/ For intra_flag and reference, I think i found the values:

For I-Frame:

picture_parameter_2.intra_flag = 1

picture_parameter_2.reference =1

For P-Frame:

picture_parameter_2.intra_flag = 0

picture_parameter_2.reference =1

3/ I'm not 100% sure of my other parameters, but both my ovd_picture_parameter and H264_picture_parameter_2 appears fine to me.

4/ After some testing, I understand that I'm receiving several I-slice or P-slice in separate RTP packets. I figured out that i should fill "ovd_slice_data_control" and this might be where I have some issue.

Here is the process, I use:

1/ each I-slice is combined into one buffer.

2/ before first I-slice data, I append 00 00 00 01

3/ the first octet of frst I-slice is removed (F-bit, nal_ref_idc and type of nal unit) I guess this is required in RTP only

4/ before each I-slice data, I append 00 00 01

I do the same for P-slice

For each of the slice, I fill

slice_data_control[count_slice].SliceDataLocation = 0 // for first slice

OR

slice_data_control[count_slice].SliceDataLocation = slice_data_control[count_slice-1].SliceDataLocation+slice_data_control[count_slice-1].SliceBytesInBuffer;

slice_data_control[count_slice].SliceBytesInBuffer = slice_len;

slice_data_control[count_slice].SliceBitsInBuffer = slice_data_control[*count_slice].SliceBytesInBuffer*8;

I hope someone is listening and can help me to get this working. I must be very close...

Regards

Aymeric

0 Likes
Reply
himanshu_gautam
Grandmaster

Re: How to fill H264_picture_parameter_2 structure

Hi Aymeric,

I am working on understanding these fields myself. I will post my learning in a few days.

0 Likes
Reply
himanshu_gautam
Grandmaster

Re: How to fill H264_picture_parameter_2 structure

Hi,

did you got any further?

Have you tried using some opensource libraries for parsing your video to SPS/PPS blocks? ffmpeg might help.

Still digging in, will keep you posted.

0 Likes
Reply
amd4
Journeyman III

Re: How to fill H264_picture_parameter_2 structure

Hi,

Tks for your answer.

Parsing SPS/PPS doesn't seems to be the issue. I'm pretty sure I'm extracting the correct

information from SPS and PPS. It's still possible that i have made error in converting values

from SPS/PPS to the structures... There is absolutly no reference code on this I could find.

I think my issue relates to formating the data and information to provide for ovd_slice_data_control.

I have tried every formating i could think of...


Regards

Aymeric

0 Likes
Reply
himanshu_gautam
Grandmaster

Re: How to fill H264_picture_parameter_2 structure

I am quite sure, you would have gone through this document: http://developer.amd.com/wordpress/media/2012/10/OpenVideo_Decode_API.PDF

But just adding for other reader's reference.

FYI I have a piece of code, that converts a *.264 file into the bit files which are fed as input in the OVDecodeRender Sample. If you can ask some pointed questions about how this parsing is happening, i may try to browse through the code, and try answering your question. I am not allowed to share the parsing code here.

0 Likes
Reply
amd4
Journeyman III

Re: How to fill H264_picture_parameter_2 structure

Hi Himanshu,

Tks for the document: I already used it, but the explanation on slice is light...

4.3 OVD_BITSTREAM_DATA and OVD_SLICE_DATA_CONTROL

OVD_BITSTREAM_DATA contains blocks of compressed bitstream data (see Figure 3). The Decoder

(host) stores the data blocks size/location information in OVD_SLICE_DATA_CONTROL. Every data

block has its own control data structure. OVD_BITSTREAM_DATA must be 128 byte aligned.

I was missing 128byte padding. So I have added it today at the end of my last slice. But it didn't helped.

I'm pretty sure I have trouble with formating my OVD_BITSTREAM_DATA and values for OVD_SLICE_DATA_CONTROL .... else, it could still be an error with SPS/PPS... who knows...

Questions:

1/ How is the OVD_SLICE_DATA_CONTROL filled where there are multiple slice?

2/ Do you have sample bit files for a BASELINE profile decoding?

Regards

Aymeric

0 Likes
Reply