History

setlocalversion: work around "git describe" performance

Contrary to expectations, passing a single candidate tag to "git describe" is slower than not passing any --match options. $ time git describe --debug ... traversed 10619 commits ... v6.12-rc5-63-g0fc810ae3ae1 real 0m0.169s $ time git describe --match=v6.12-rc5 --debug ... traversed 1310024 commits v6.12-rc5-63-g0fc810ae3ae1 real 0m1.281s In fact, the --debug output shows that git traverses all or most of history. For some repositories and/or git versions, those 1.3s are actually 10-15 seconds. This has been acknowledged as a performance bug in git [1], and a fix is on its way [2]. However, no solution is yet in git.git, and even when one lands, it will take quite a while before it finds its way to a release and for $random_kernel_developer to pick that up. So rewrite the logic to use plumbing commands. For each of the candidate values of $tag, we ask: (1) is $tag even an annotated tag? (2) Is it eligible to describe HEAD, i.e. an ancestor of HEAD? (3) If so, how many commits are in $tag..HEAD? I have tested that this produces the same output as the current script for ~700 random commits between v6.9..v6.10. For those 700 commits, and in my git repo, the 'make -s kernelrelease' command is on average ~4 times faster with this patch applied (geometric mean of ratios). For the commit mentioned in Josh's original report [3], the time-consuming part of setlocalversion goes from $ time git describe --match=v6.12-rc5 c1e939a21eb1 v6.12-rc5-44-gc1e939a21eb1 real 0m1.210s to $ time git rev-list --count --left-right v6.12-rc5..c1e939a21eb1 0 44 real 0m0.037s [1] https://lore.kernel.org/git/20241101113910.GA2301440@coredump.intra.peff.net/ [2] https://lore.kernel.org/git/20241106192236.GC880133@coredump.intra.peff.net/ [3] https://lore.kernel.org/lkml/309549cafdcfe50c4fceac3263220cc3d8b109b2.1730337435.git.jpoimboe@kernel.org/

Rasmus Villemoes committed 1 year ago

Commit 523f3db1 parent e397a601 file · +38 −16Browse files

Showing the changes to scripts (1 of 1 file). Show all

Unified · Split

scripts/setlocalversion+38−16
@@ -30,6 +30,27 @@ if test $# -gt 0 -o ! -d "$srctree"; then · 29 unmodified lines
30 usage30 usage
31fi31fi
32 32
33try_tag() {
34 tag="$1"
35
36 # Is $tag an annotated tag?
37 [ "$(git cat-file -t "$tag" 2> /dev/null)" = tag ] || return 1
38
39 # Is it an ancestor of HEAD, and if so, how many commits are in $tag..HEAD?
40 # shellcheck disable=SC2046 # word splitting is the point here
41 set -- $(git rev-list --count --left-right "$tag"...HEAD 2> /dev/null)
42
43 # $1 is 0 if and only if $tag is an ancestor of HEAD. Use
44 # string comparison, because $1 is empty if the 'git rev-list'
45 # command somehow failed.
46 [ "$1" = 0 ] || return 1
47
48 # $2 is the number of commits in the range $tag..HEAD, possibly 0.
49 count="$2"
50
51 return 0
52}
53
33scm_version()54scm_version()
34{55{
35 local short=false56 local short=false
@@ -61,33 +82,33 @@ scm_version() · 25 unmodified lines
61 # stable kernel: 6.1.7 -> v6.1.782 # stable kernel: 6.1.7 -> v6.1.7
62 version_tag=v$(echo "${KERNELVERSION}" | sed -E 's/^([0-9]+\.[0-9]+)\.0(.*)$/\1\2/')83 version_tag=v$(echo "${KERNELVERSION}" | sed -E 's/^([0-9]+\.[0-9]+)\.0(.*)$/\1\2/')
63 84
85 # try_tag initializes count if the tag is usable.
86 count=
87
64 # If a localversion* file exists, and the corresponding88 # If a localversion* file exists, and the corresponding
65 # annotated tag exists and is an ancestor of HEAD, use89 # annotated tag exists and is an ancestor of HEAD, use
66 # it. This is the case in linux-next.90 # it. This is the case in linux-next.
67 tag=${file_localversion#-}91 if [ -n "${file_localversion#-}" ] ; then
68 desc=92 try_tag "${file_localversion#-}"
69 if [ -n "${tag}" ]; then
70 desc=$(git describe --match=$tag 2>/dev/null)
71 fi93 fi
72 94
73 # Otherwise, if a localversion* file exists, and the tag95 # Otherwise, if a localversion* file exists, and the tag
74 # obtained by appending it to the tag derived from96 # obtained by appending it to the tag derived from
75 # KERNELVERSION exists and is an ancestor of HEAD, use97 # KERNELVERSION exists and is an ancestor of HEAD, use
76 # it. This is e.g. the case in linux-rt.98 # it. This is e.g. the case in linux-rt.
77 if [ -z "${desc}" ] && [ -n "${file_localversion}" ]; then99 if [ -z "${count}" ] && [ -n "${file_localversion}" ]; then
78 tag="${version_tag}${file_localversion}"100 try_tag "${version_tag}${file_localversion}"
79 desc=$(git describe --match=$tag 2>/dev/null)
80 fi101 fi
81 102
82 # Otherwise, default to the annotated tag derived from KERNELVERSION.103 # Otherwise, default to the annotated tag derived from KERNELVERSION.
83 if [ -z "${desc}" ]; then104 if [ -z "${count}" ]; then
84 tag="${version_tag}"105 try_tag "${version_tag}"
85 desc=$(git describe --match=$tag 2>/dev/null)
86 fi106 fi
87 107
88 # If we are at the tagged commit, we ignore it because the version is108 # If we are at the tagged commit, we ignore it because the
89 # well-defined.109 # version is well-defined. If none of the attempted tags exist
90 if [ "${tag}" != "${desc}" ]; then110 # or were usable, $count is still empty.
111 if [ -z "${count}" ] || [ "${count}" -gt 0 ]; then
91 112
92 # If only the short version is requested, don't bother113 # If only the short version is requested, don't bother
93 # running further git commands114 # running further git commands
@@ -95,14 +116,15 @@ scm_version() · 1 unmodified line
95 echo "+"116 echo "+"
96 return117 return
97 fi118 fi
119
98 # If we are past the tagged commit, we pretty print it.120 # If we are past the tagged commit, we pretty print it.
99 # (like 6.1.0-14595-g292a089d78d3)121 # (like 6.1.0-14595-g292a089d78d3)
100 if [ -n "${desc}" ]; then122 if [ -n "${count}" ]; then
101 echo "${desc}" | awk -F- '{printf("-%05d", $(NF-1))}'123 printf "%s%05d" "-" "${count}"
102 fi124 fi
103 125
104 # Add -g and exactly 12 hex chars.126 # Add -g and exactly 12 hex chars.
105 printf '%s%s' -g "$(echo $head | cut -c1-12)"127 printf '%s%.12s' -g "$head"
106 fi128 fi
107 129
108 if ${no_dirty}; then130 if ${no_dirty}; then