Before this commit if allocation would fail in ff_add_channel_layout() function, function would return negative error code and this would cause wrong format pick up later. If allocation would not fail return code would be 0 and then format negotiation would simply fail as code would break from the loop but with wrong return code. Error was introduced in 6aaac24d72a7da commit. Fixes #6638
Showing the changes to libavfilter/avfiltergraph.c (1 of 1 file). Show all
Unified · Split
| @@ -748,8 +748,10 @@ static int reduce_formats_on_filter(AVFilterContext *filter) · 747 unmodified lines | |||
| 748 | (KNOWN(fmt) || fmts->all_counts)) { | 748 | (KNOWN(fmt) || fmts->all_counts)) { |
| 749 | /* Turn the infinite list into a singleton */ | 749 | /* Turn the infinite list into a singleton */ |
| 750 | fmts->all_layouts = fmts->all_counts = 0; | 750 | fmts->all_layouts = fmts->all_counts = 0; |
| 751 | if (ff_add_channel_layout(&outlink->incfg.channel_layouts, fmt) < 0) | 751 | ret = ff_add_channel_layout(&outlink->incfg.channel_layouts, fmt); |
| 752 | ret = 1; | 752 | if (ret < 0) |
| 753 | return ret; | ||
| 754 | ret = 1; | ||
| 753 | break; | 755 | break; |
| 754 | } | 756 | } |
| 755 | 757 | ||