History

vp9: recon: Use emulated edge to prevent buffer overflows

The arm/aarch64 horizontal filter reads one additional pixel beyond what the filter uses. This can become an issue if the application does not allocate larger buffers than what's required for the pixel data. If the motion vector points to the bottom right edge of the picture this becomes a read buffer overflow. This triggers segfaults in Firefox for video resolutions which result in a page aligned picture size like 1280x640. Prevent this by using emulated edge in this case.

Janne Grunau committed 1 year ago

Commit 06046411 parent f3662561 file · +8 −0Browse files

Unified · Split

libavcodec/vp9recon.c+8−0
@@ -319,7 +319,11 @@ static av_always_inline void mc_luma_unscaled(VP9TileData *td, const vp9_mc_func · 318 unmodified lines
319 // The arm/aarch64 _hv filters read one more row than what actually is319 // The arm/aarch64 _hv filters read one more row than what actually is
320 // needed, so switch to emulated edge one pixel sooner vertically320 // needed, so switch to emulated edge one pixel sooner vertically
321 // (!!my * 5) than horizontally (!!mx * 4).321 // (!!my * 5) than horizontally (!!mx * 4).
322 // The arm/aarch64 _h filters read one more pixel than what actually is
323 // needed, so switch to emulated edge if that would read beyond the bottom
324 // right block.
322 if (x < !!mx * 3 || y < !!my * 3 ||325 if (x < !!mx * 3 || y < !!my * 3 ||
326 ((ARCH_AARCH64 || ARCH_ARM) && (x + !!mx * 5 > w - bw) && (y + !!my * 5 + 1 > h - bh)) ||
323 x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {327 x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {
324 s->vdsp.emulated_edge_mc(td->edge_emu_buffer,328 s->vdsp.emulated_edge_mc(td->edge_emu_buffer,
325 ref - !!my * 3 * ref_stride - !!mx * 3 * bytesperpixel,329 ref - !!my * 3 * ref_stride - !!mx * 3 * bytesperpixel,
@@ -358,7 +362,11 @@ static av_always_inline void mc_chroma_unscaled(VP9TileData *td, const vp9_mc_fu · 32 unmodified lines
358 // The arm/aarch64 _hv filters read one more row than what actually is362 // The arm/aarch64 _hv filters read one more row than what actually is
359 // needed, so switch to emulated edge one pixel sooner vertically363 // needed, so switch to emulated edge one pixel sooner vertically
360 // (!!my * 5) than horizontally (!!mx * 4).364 // (!!my * 5) than horizontally (!!mx * 4).
365 // The arm/aarch64 _h filters read one more pixel than what actually is
366 // needed, so switch to emulated edge if that would read beyond the bottom
367 // right block.
361 if (x < !!mx * 3 || y < !!my * 3 ||368 if (x < !!mx * 3 || y < !!my * 3 ||
369 ((ARCH_AARCH64 || ARCH_ARM) && (x + !!mx * 5 > w - bw) && (y + !!my * 5 + 1 > h - bh)) ||
362 x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {370 x + !!mx * 4 > w - bw || y + !!my * 5 > h - bh) {
363 s->vdsp.emulated_edge_mc(td->edge_emu_buffer,371 s->vdsp.emulated_edge_mc(td->edge_emu_buffer,
364 ref_u - !!my * 3 * src_stride_u - !!mx * 3 * bytesperpixel,372 ref_u - !!my * 3 * src_stride_u - !!mx * 3 * bytesperpixel,

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t