History

vulkan_h264: reject end_frame being called without valid session parameters

When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame without calling start_frame. We are unable to decode this, as no frame state has been set. Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence the individual commit.

Lynne committed 3 years ago

Commit f701f081 parent 160a4151 file · +4 −0Browse files

Unified · Split

libavcodec/vulkan_h264.c+4−0
@@ -508,10 +508,14 @@ static int vk_h264_end_frame(AVCodecContext *avctx) · 507 unmodified lines
508508 const H264Context *h = avctx->priv_data;
509509 H264Picture *pic = h->cur_pic_ptr;
510510 H264VulkanDecodePicture *hp = pic->hwaccel_picture_private;
511+ FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data;
511512 FFVulkanDecodePicture *vp = &hp->vp;
512513 FFVulkanDecodePicture *rvp[H264_MAX_PICTURE_COUNT] = { 0 };
513514 AVFrame *rav[H264_MAX_PICTURE_COUNT] = { 0 };
514515
516+ if (!dec->session_params)
517+ return AVERROR(EINVAL);
518+
515519 for (int i = 0; i < vp->decode_info.referenceSlotCount; i++) {
516520 H264Picture *rp = hp->ref_src[i];
517521 H264VulkanDecodePicture *rhp = rp->hwaccel_picture_private;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t