History

swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input

Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int' Fixes: ticket8293

Michael Niedermayer committed 6 years ago

Commit e057e831 parent 49ba1871 file · +3 −3Browse files

Unified · Split

libswscale/output.c+3−3
@@ -1850,9 +1850,9 @@ static av_always_inline void yuv2rgb_write_full(SwsContext *c, · 1849 unmodified lines
18501850 Y -= c->yuv2rgb_y_offset;
18511851 Y *= c->yuv2rgb_y_coeff;
18521852 Y += 1 << 21;
1853− R = Y + V*c->yuv2rgb_v2r_coeff;
1854− G = Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff;
1855− B = Y + U*c->yuv2rgb_u2b_coeff;
1853+ R = (unsigned)Y + V*c->yuv2rgb_v2r_coeff;
1854+ G = (unsigned)Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff;
1855+ B = (unsigned)Y + U*c->yuv2rgb_u2b_coeff;
18561856 if ((R | G | B) & 0xC0000000) {
18571857 R = av_clip_uintp2(R, 30);
18581858 G = av_clip_uintp2(G, 30);

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t