History

vulkan_h264: reject end_frame being called without valid session parameters

When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame without calling start_frame. We are unable to decode this, as no frame state has been set. Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence the individual commit.

Lynne committed 3 years ago

Commit f701f081 parent 160a4151 file · +4 −0Browse files

Unified · Split

libavcodec/vulkan_h264.c+4−0
@@ -508,10 +508,14 @@ static int vk_h264_end_frame(AVCodecContext *avctx) · 507 unmodified lines
508 const H264Context *h = avctx->priv_data;508 const H264Context *h = avctx->priv_data;
509 H264Picture *pic = h->cur_pic_ptr;509 H264Picture *pic = h->cur_pic_ptr;
510 H264VulkanDecodePicture *hp = pic->hwaccel_picture_private;510 H264VulkanDecodePicture *hp = pic->hwaccel_picture_private;
511 FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data;
511 FFVulkanDecodePicture *vp = &hp->vp;512 FFVulkanDecodePicture *vp = &hp->vp;
512 FFVulkanDecodePicture *rvp[H264_MAX_PICTURE_COUNT] = { 0 };513 FFVulkanDecodePicture *rvp[H264_MAX_PICTURE_COUNT] = { 0 };
513 AVFrame *rav[H264_MAX_PICTURE_COUNT] = { 0 };514 AVFrame *rav[H264_MAX_PICTURE_COUNT] = { 0 };
514 515
516 if (!dec->session_params)
517 return AVERROR(EINVAL);
518
515 for (int i = 0; i < vp->decode_info.referenceSlotCount; i++) {519 for (int i = 0; i < vp->decode_info.referenceSlotCount; i++) {
516 H264Picture *rp = hp->ref_src[i];520 H264Picture *rp = hp->ref_src[i];
517 H264VulkanDecodePicture *rhp = rp->hwaccel_picture_private;521 H264VulkanDecodePicture *rhp = rp->hwaccel_picture_private;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t