max_bin_idx can be at most LMCS_MAX_BIN_SIZE - 1 here, so pivot[LCMS_MAX_BIN_SIZE + 1] may be accessed, but pivot has only LCMS_MAX_BIN_SIZE + 1 elements (unless the values of pivot were so that it is always assured that pivot[LCMS_MAX_BIN_SIZE] is always < sample (which it is iff it is always < 2^bit_depth - 1)). So reorder the checks.
Showing the changes to libavcodec (1 of 1 file). Show all
Unified · Split
| @@ -652,7 +652,7 @@ static int lmcs_derive_lut(VVCLMCS *lmcs, const H266RawAPS *rlmcs, const H266Raw · 651 unmodified lines | |||
| 652 | i = lmcs->min_bin_idx; | 652 | i = lmcs->min_bin_idx; |
| 653 | for (uint16_t sample = 0; sample < max; sample++) { | 653 | for (uint16_t sample = 0; sample < max; sample++) { |
| 654 | uint16_t inv_sample; | 654 | uint16_t inv_sample; |
| 655 | while (sample >= lmcs->pivot[i + 1] && i <= lmcs->max_bin_idx) | 655 | while (i <= lmcs->max_bin_idx && sample >= lmcs->pivot[i + 1]) |
| 656 | i++; | 656 | i++; |
| 657 | 657 | ||
| 658 | inv_sample = lmcs_derive_lut_sample(sample, input_pivot, lmcs->pivot, | 658 | inv_sample = lmcs_derive_lut_sample(sample, input_pivot, lmcs->pivot, |