History

avfilter/formats: Leave lists' ownership unchanged upon merge failure

ff_merge_formats(), ff_merge_samplerates() and ff_merge_channel_layouts() share common semantics: If merging succeeds, a non-NULL pointer is returned and both input lists (of type AVFilterFormats resp. AVFilterChannelLayouts) are to be treated as if they had been freed; the owners of the input parameters (if any) become owners of the returned list. If merging does not succeed, NULL is returned and both input lists are supposed to be unchanged. The problem is that the functions did not abide by these semantics: In case of reallocation failure, it is possible for these functions to return NULL after having already freed one of the two input list. This happens because sometimes the refs-array of the destined output gets reallocated twice to its final size and if the second of these reallocations fails, the first of the two inputs has already been freed and its refs updated to point to the destined output which in this case will be freed immediately so that all of the already updated pointers are now dangling. This leads to use-after-frees and memory corruptions lateron (when these owners get cleaned up, the lists they own get unreferenced). Should the input lists don't have owners at all, the caller (namely can_merge_formats() in avfiltergraph.c) thinks that both the input lists are unchanged and need to be freed, leading to a double free. The solution to this is simple: Don't reallocate twice; do it just once. This also saves a reallocation. This commit fixes the issue behind Coverity issue #1452636. It might also make Coverity realize that the issue has been fixed.

Andreas Rheinhardt committed 6 years ago

Commit 195a25a1 parent ae5026c1 file · +28 −12Browse files

Showing the changes to libavfilter (1 of 1 file). Show all

Unified · Split

libavfilter/formats.c+28−12
@@ -33,17 +33,11 @@ · 32 unmodified lines
3333
3434 /**
3535 * Add all refs from a to ret and destroy a.
36+ * ret->refs must have enough spare room left for this.
3637 */
37−#define MERGE_REF(ret, a, fmts, type, fail) \
38+#define MERGE_REF_NO_ALLOC(ret, a, fmts) \
3839 do { \
39− type ***tmp; \
4040 int i; \
41− \
42− if (!(tmp = av_realloc_array(ret->refs, ret->refcount + a->refcount, \
43− sizeof(*tmp)))) \
44− goto fail; \
45− ret->refs = tmp; \
46− \
4741 for (i = 0; i < a->refcount; i ++) { \
4842 ret->refs[ret->refcount] = a->refs[i]; \
4943 *ret->refs[ret->refcount++] = ret; \
@@ -54,6 +48,17 @@ do { \ · 4 unmodified lines
5448 av_freep(&a); \
5549 } while (0)
5650
51+#define MERGE_REF(ret, a, fmts, type, fail) \
52+do { \
53+ type ***tmp; \
54+ \
55+ if (!(tmp = av_realloc_array(ret->refs, ret->refcount + a->refcount, \
56+ sizeof(*tmp)))) \
57+ goto fail; \
58+ ret->refs = tmp; \
59+ MERGE_REF_NO_ALLOC(ret, a, fmts); \
60+} while (0)
61+
5762 /**
5863 * Add all formats common for a and b to ret, copy the refs and destroy
5964 * a and b.
@@ -61,6 +66,7 @@ do { \ · 1 unmodified line
6166 #define MERGE_FORMATS(ret, a, b, fmts, nb, type, fail) \
6267 do { \
6368 int i, j, k = 0, count = FFMIN(a->nb, b->nb); \
69+ type ***tmp; \
6470 \
6571 if (!(ret = av_mallocz(sizeof(*ret)))) \
6672 goto fail; \
@@ -85,8 +91,13 @@ do { · 18 unmodified lines
8591 if (!ret->nb) \
8692 goto fail; \
8793 \
88− MERGE_REF(ret, a, fmts, type, fail); \
89− MERGE_REF(ret, b, fmts, type, fail); \
94+ tmp = av_realloc_array(NULL, a->refcount + b->refcount, sizeof(*tmp)); \
95+ if (!tmp) \
96+ goto fail; \
97+ ret->refs = tmp; \
98+ \
99+ MERGE_REF_NO_ALLOC(ret, a, fmts); \
100+ MERGE_REF_NO_ALLOC(ret, b, fmts); \
90101 } while (0)
91102
92103 AVFilterFormats *ff_merge_formats(AVFilterFormats *a, AVFilterFormats *b,
@@ -238,8 +249,13 @@ AVFilterChannelLayouts *ff_merge_channel_layouts(AVFilterChannelLayouts *a, · 145 unmodified lines
238249 ret->nb_channel_layouts = ret_nb;
239250 if (!ret->nb_channel_layouts)
240251 goto fail;
241− MERGE_REF(ret, a, channel_layouts, AVFilterChannelLayouts, fail);
242− MERGE_REF(ret, b, channel_layouts, AVFilterChannelLayouts, fail);
252+
253+ ret->refs = av_realloc_array(NULL, a->refcount + b->refcount,
254+ sizeof(*ret->refs));
255+ if (!ret->refs)
256+ goto fail;
257+ MERGE_REF_NO_ALLOC(ret, a, channel_layouts);
258+ MERGE_REF_NO_ALLOC(ret, b, channel_layouts);
243259 return ret;
244260
245261 fail:

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t