History

avfilter/palettegen: change cut score from ∑e² to max e²

This is following the results from personal research¹. ¹: https://github.com/ubitux/research/tree/main/color-quantization#results

Clément Bœsch committed 3 years ago

Commit dafd43b1 parent 187f5e73 files · +4 −3Browse files

Unified · Split

3 files
  1. libavfilter/vf_palettegen.c +2 −1
  2. tests/ref/fate/filter-palettegen-1 +1 −1
  3. tests/ref/fate/filter-palettegen-2 +1 −1
libavfilter/vf_palettegen.c+2−1
@@ -171,7 +171,8 @@ static void compute_box_stats(PaletteGenContext *s, struct range_box *box) · 170 unmodified lines
171171 if (er2[0] >= er2[1] && er2[0] >= er2[2]) box->major_axis = 0;
172172 if (er2[1] >= er2[0] && er2[1] >= er2[2]) box->major_axis = 1; // prefer green again
173173
174− box->cut_score = er2[0] + er2[1] + er2[2];
174+ /* The box that has the axis with the biggest error amongst all boxes will but cut down */
175+ box->cut_score = FFMAX3(er2[0], er2[1], er2[2]);
175176 }
176177
177178 /**
tests/ref/fate/filter-palettegen-1+1−1
@@ -3,4 +3,4 @@ · 2 unmodified lines
33 #codec_id 0: rawvideo
44 #dimensions 0: 16x16
55 #sar 0: 1/1
6−0, 0, 0, 1, 1024, 0x394ee723
6+0, 0, 0, 1, 1024, 0x21c6e6c4
tests/ref/fate/filter-palettegen-2+1−1
@@ -3,4 +3,4 @@ · 2 unmodified lines
33 #codec_id 0: rawvideo
44 #dimensions 0: 16x16
55 #sar 0: 1/1
6−0, 0, 0, 1, 1024, 0xc54d773d
6+0, 0, 0, 1, 1024, 0x630d76b1

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t