This patch addresses issue #79, which is a regression since commit 28de670 "Fix RD bug." If the coded error value is zero, the iiratio calculation effectively multiplies by 1000000 by the DOUBLE_DIVIDE_CHECK macro. This can result in a value larger than INT_MAX, giving a negative ratio. Since the error values are conceptually unsigned (though they're stored in a double) this patch makes the iiratio values unsigned, which allows the clamping to work as expected.
Unified · Split
| @@ -523,8 +523,8 @@ typedef struct · 522 unmodified lines | ||
| 523 | 523 | int motion_lvl; |
| 524 | 524 | int motion_speed; |
| 525 | 525 | int motion_var; |
| 526 | − int next_iiratio; | |
| 527 | − int this_iiratio; | |
| 526 | + unsigned int next_iiratio; | |
| 527 | + unsigned int this_iiratio; | |
| 528 | 528 | int this_frame_modified_error; |
| 529 | 529 | |
| 530 | 530 | double norm_intra_err_per_mb; |