History

avfilter/formats: Leave lists' ownership unchanged upon merge failure

ff_merge_formats(), ff_merge_samplerates() and ff_merge_channel_layouts() share common semantics: If merging succeeds, a non-NULL pointer is returned and both input lists (of type AVFilterFormats resp. AVFilterChannelLayouts) are to be treated as if they had been freed; the owners of the input parameters (if any) become owners of the returned list. If merging does not succeed, NULL is returned and both input lists are supposed to be unchanged. The problem is that the functions did not abide by these semantics: In case of reallocation failure, it is possible for these functions to return NULL after having already freed one of the two input list. This happens because sometimes the refs-array of the destined output gets reallocated twice to its final size and if the second of these reallocations fails, the first of the two inputs has already been freed and its refs updated to point to the destined output which in this case will be freed immediately so that all of the already updated pointers are now dangling. This leads to use-after-frees and memory corruptions lateron (when these owners get cleaned up, the lists they own get unreferenced). Should the input lists don't have owners at all, the caller (namely can_merge_formats() in avfiltergraph.c) thinks that both the input lists are unchanged and need to be freed, leading to a double free. The solution to this is simple: Don't reallocate twice; do it just once. This also saves a reallocation. This commit fixes the issue behind Coverity issue #1452636. It might also make Coverity realize that the issue has been fixed.

Andreas Rheinhardt committed 6 years ago

Commit 195a25a1 parent ae5026c1 file · +28 −12Browse files

Showing the changes to libavfilter (1 of 1 file). Show all

Unified · Split

libavfilter/formats.c+28−12
@@ -33,17 +33,11 @@ · 32 unmodified lines
33 33
34/**34/**
35 * Add all refs from a to ret and destroy a.35 * Add all refs from a to ret and destroy a.
36 * ret->refs must have enough spare room left for this.
36 */37 */
37#define MERGE_REF(ret, a, fmts, type, fail) \38#define MERGE_REF_NO_ALLOC(ret, a, fmts) \
38do { \39do { \
39 type ***tmp; \
40 int i; \40 int i; \
41 \
42 if (!(tmp = av_realloc_array(ret->refs, ret->refcount + a->refcount, \
43 sizeof(*tmp)))) \
44 goto fail; \
45 ret->refs = tmp; \
46 \
47 for (i = 0; i < a->refcount; i ++) { \41 for (i = 0; i < a->refcount; i ++) { \
48 ret->refs[ret->refcount] = a->refs[i]; \42 ret->refs[ret->refcount] = a->refs[i]; \
49 *ret->refs[ret->refcount++] = ret; \43 *ret->refs[ret->refcount++] = ret; \
@@ -54,6 +48,17 @@ do { \ · 4 unmodified lines
54 av_freep(&a); \48 av_freep(&a); \
55} while (0)49} while (0)
56 50
51#define MERGE_REF(ret, a, fmts, type, fail) \
52do { \
53 type ***tmp; \
54 \
55 if (!(tmp = av_realloc_array(ret->refs, ret->refcount + a->refcount, \
56 sizeof(*tmp)))) \
57 goto fail; \
58 ret->refs = tmp; \
59 MERGE_REF_NO_ALLOC(ret, a, fmts); \
60} while (0)
61
57/**62/**
58 * Add all formats common for a and b to ret, copy the refs and destroy63 * Add all formats common for a and b to ret, copy the refs and destroy
59 * a and b.64 * a and b.
@@ -61,6 +66,7 @@ do { \ · 1 unmodified line
61#define MERGE_FORMATS(ret, a, b, fmts, nb, type, fail) \66#define MERGE_FORMATS(ret, a, b, fmts, nb, type, fail) \
62do { \67do { \
63 int i, j, k = 0, count = FFMIN(a->nb, b->nb); \68 int i, j, k = 0, count = FFMIN(a->nb, b->nb); \
69 type ***tmp; \
64 \70 \
65 if (!(ret = av_mallocz(sizeof(*ret)))) \71 if (!(ret = av_mallocz(sizeof(*ret)))) \
66 goto fail; \72 goto fail; \
@@ -85,8 +91,13 @@ do { · 18 unmodified lines
85 if (!ret->nb) \91 if (!ret->nb) \
86 goto fail; \92 goto fail; \
87 \93 \
88 MERGE_REF(ret, a, fmts, type, fail); \94 tmp = av_realloc_array(NULL, a->refcount + b->refcount, sizeof(*tmp)); \
89 MERGE_REF(ret, b, fmts, type, fail); \95 if (!tmp) \
96 goto fail; \
97 ret->refs = tmp; \
98 \
99 MERGE_REF_NO_ALLOC(ret, a, fmts); \
100 MERGE_REF_NO_ALLOC(ret, b, fmts); \
90} while (0)101} while (0)
91 102
92AVFilterFormats *ff_merge_formats(AVFilterFormats *a, AVFilterFormats *b,103AVFilterFormats *ff_merge_formats(AVFilterFormats *a, AVFilterFormats *b,
@@ -238,8 +249,13 @@ AVFilterChannelLayouts *ff_merge_channel_layouts(AVFilterChannelLayouts *a, · 145 unmodified lines
238 ret->nb_channel_layouts = ret_nb;249 ret->nb_channel_layouts = ret_nb;
239 if (!ret->nb_channel_layouts)250 if (!ret->nb_channel_layouts)
240 goto fail;251 goto fail;
241 MERGE_REF(ret, a, channel_layouts, AVFilterChannelLayouts, fail);252
242 MERGE_REF(ret, b, channel_layouts, AVFilterChannelLayouts, fail);253 ret->refs = av_realloc_array(NULL, a->refcount + b->refcount,
254 sizeof(*ret->refs));
255 if (!ret->refs)
256 goto fail;
257 MERGE_REF_NO_ALLOC(ret, a, channel_layouts);
258 MERGE_REF_NO_ALLOC(ret, b, channel_layouts);
243 return ret;259 return ret;
244 260
245fail:261fail:

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t