It is possible for a format to be supported by hardware but not by av_hwframe_transfer for uploading. This breaks the best upload format selection. We first select the hardware input format and then choose the best software format that can be uploaded. In some cases, this may result in a format that is not uploadable at all, leading to an error. To solve this, we should avoid adding non-uploadable formats to the fmts table. For example, for format d3d11/yuv420p, av_hwframe_transfer_get_formats() returns empty list of formats.
Unified · Split
| @@ -486,7 +486,8 @@ static bool probe_formats(struct mp_filter *f, int hw_imgfmt, bool use_conversio · 485 unmodified lines | |||
| 486 | 486 | ||
| 487 | enum AVPixelFormat *fmts; | 487 | enum AVPixelFormat *fmts; |
| 488 | if (av_hwframe_transfer_get_formats(frames, | 488 | if (av_hwframe_transfer_get_formats(frames, |
| 489 | AV_HWFRAME_TRANSFER_DIRECTION_TO, &fmts, 0) >= 0) | 489 | AV_HWFRAME_TRANSFER_DIRECTION_TO, &fmts, 0) >= 0 && |
| 490 | fmts[0] != AV_PIX_FMT_NONE) | ||
| 490 | { | 491 | { |
| 491 | int index = p->num_fmts; | 492 | int index = p->num_fmts; |
| 492 | MP_TARRAY_APPEND(p, p->fmts, p->num_fmts, imgfmt); | 493 | MP_TARRAY_APPEND(p, p->fmts, p->num_fmts, imgfmt); |