History

Remove

The document was prepared to help me came up with a parallel build solution, but to no avail.

malc committed 5 years ago

Commit 2d25d3c1 parent 08c648d1 file · +0 −227Browse files

Showing the changes to misc/notes (1 of 1 file). Show all

Unified · Split

misc/notes/pbuild.txt+0−227
@@ -1,227 +0,0 @@
1−There is some unclaimed parallelism inside the build process:
2−
3− a) all C compilation can be done in parallel
4−
5− b) ditto all _independent_ OCaml compilation (i.e. things having the
6− same indentation (and growing from the same "root") in the
7− following build.bash output)
8−
9−The "neat" solution of how to get there is elusive though. Currently:
10−
11− - bash build.bash build
12− utils.ml -> utils.cmo
13− keys.ml -> keys.cmo
14− wsi.mli -> wsi.cmi
15− utf8syms.ml -> utf8syms.cmo
16− lablGL/raw.ml -> lablGL/raw.cmo
17− lablGL/gl.ml -> lablGL/gl.cmo
18− lablGL/glPix.ml -> lablGL/glPix.cmo
19− lablGL/glMisc.ml -> lablGL/glMisc.cmo
20− lablGL/glTex.ml -> lablGL/glTex.cmo
21− lablGL/glFunc.ml -> lablGL/glFunc.cmo
22− lablGL/glDraw.ml -> lablGL/glDraw.cmo
23− lablGL/glArray.ml -> lablGL/glArray.cmo
24− parser.ml -> parser.cmo
25− build/confstruct.ml -> confstruct.cmo
26− config.ml -> config.cmo
27− ffi.ml -> ffi.cmo
28− glutils.ml -> glutils.cmo
29− lablGL/glMat.ml -> lablGL/glMat.cmo
30− listview.ml -> listview.cmo
31− help.mli -> help.cmi
32− help.ml -> help.cmo
33− lablGL/glClear.ml -> lablGL/glClear.cmo
34− main.ml -> main.cmo
35− link.c -> link.o
36− cutils.c -> cutils.o
37− version.c -> version.o
38− lablGL/ml_gl.c -> lablGL/ml_gl.o
39− lablGL/ml_glarray.c -> lablGL/ml_glarray.o
40− lablGL/ml_raw.c -> lablGL/ml_raw.o
41− 9 sec
42−
43−IOW - whole build (including lablGL, but excluding mupdf/ocaml and its
44−dependencies) takes less than 10 seconds even on this circa 2014
45−MacMini.
46−
47−Making things parallel ought to bring some benefits to the build on
48−the Raspberry Pi3 and it might even benefit single core PPC MacMini
49−due to interleaving IO/CPU bound parts (but this is just that -
50−baseless speculation)
51−
52−=============================================================
53−
54−With following applied, build.bash will produce a Makefile that builds
55−llpp. Building things via this Makefile (i.e. `\time -pv make -jN')
56−seems to suggest that there is some (a lot) parallelism that can be
57−harvested even for OCaml code (parallelism for interface deprived
58−OCaml code (as is the case with llpp) is limited)
59−
60−diff --git a/build.bash b/build.bash
61−index e9a2919..37382df 100755
62−--- a/build.bash
63−+++ b/build.bash
64−@@ -1,6 +1,8 @@
65− #!/bin/bash
66− set -eu
67−
68−+:>Makefile
69−+
70− now() { date +%s; }
71− S=$(now)
72− vecho() { ${vecho-:} "$*"; }
73−@@ -159,6 +161,7 @@ bocaml2() {
74−
75− cmd="ocamlc $(oflags $o) -c -o $o $s"
76− keycmd="digest $o $s $(< $o.depl)"
77−+ printf "$o: $s $(< $o.depl)\n\t$cmd\n" >>Makefile
78− isfresh "$o" "$overs$cmd$(eval $keycmd)" || {
79− printf "%*.s%s -> %s\n" $n '' "${s#$srcd/}" "${o#$outd/}"
80− eval "$cmd || die '$cmd failed'"
81−@@ -191,6 +194,7 @@ bocaml() (
82− *) false;;
83− esac && {
84− local s1=${s#$srcd/}
85−+ echo "$outd/${s1%.ml}.cmo: $o" >>Makefile
86− bocaml1 $n "$s" "$outd/${s1%.ml}.cmo" "${o#$outd/}"
87− } || true
88− cycle=$cycle1
89−@@ -203,6 +207,7 @@ bocamlc() {
90− local cmd="ocamlc $cc-ccopt \"$(cflags $o) -MMD -MF $o.dep -MT_ -o $o\" $s"
91− test -r $o.dep && read _ d <$o.dep || d=
92− local keycmd='digest $o $d'
93−+ printf "$o: $s $d\n\t$cmd\n" >>Makefile
94− isfresh "$o" "$cmd$(eval $keycmd)" || {
95− printf "%s -> %s\n" "${s#$srcd/}" "${o#$outd/}"
96− eval "$cmd || die '$cmd failed'"
97−@@ -229,6 +234,7 @@ ver=$(cd $srcd && git describe --tags --dirty) || ver=unknown
98−
99− cmd="(. $srcd/genconfstr.sh >$outd/confstruct.ml)"
100− keycmd="digest $srcd/genconfstr.sh $outd/confstruct.ml"
101−+printf "$outd/confstruct.ml: $srcd/genconfstr.sh\n\t$cmd\n" >>Makefile
102− isfresh "$outd/confstruct.ml" "$cmd$(eval $keycmd)" || {
103− echo "generating $outd/confstruct.ml"
104− eval "$cmd || die genconfstr.sh failed"
105−@@ -295,6 +301,7 @@ fi
106− ord=$(grep -v \.cmi $outd/ordered)
107− cmd="ocamlc -custom $libs -o $outd/llpp $cobjs $(echo $ord) -cclib \"$clibs\""
108− keycmd="digest $outd/llpp $cobjs $ord $mulibs"
109−+printf "$outd/llpp: $cobjs $(echo $ord)\n\t$cmd\n" >>Makefile
110− isfresh "$outd/llpp" "$cmd$(eval $keycmd)" || {
111− echo linking $outd/llpp
112− eval "$cmd || die '$cmd failed'"
113−
114−========================================
115−~/xsrc/llpp
116−- rm build/*.cm* build/lablGL/*.cm* build/wsi/x11/*.cm* build/*.o build/lablGL/*.o build/wsi/x11/*.o
117−~/xsrc/llpp
118−- \time -pv bash build.bash build
119− utils.ml -> utils.cmo
120− keys.ml -> keys.cmo
121− wsi.mli -> wsi.cmi
122− wsi/x11/wsi.mli -> wsi/x11/wsi.cmi
123− wsi/x11/wsi.ml -> wsi/x11/wsi.cmo
124− utf8syms.ml -> utf8syms.cmo
125− lablGL/raw.ml -> lablGL/raw.cmo
126− lablGL/gl.ml -> lablGL/gl.cmo
127− lablGL/glPix.ml -> lablGL/glPix.cmo
128− lablGL/glMisc.ml -> lablGL/glMisc.cmo
129− lablGL/glTex.ml -> lablGL/glTex.cmo
130− lablGL/glFunc.ml -> lablGL/glFunc.cmo
131− lablGL/glDraw.ml -> lablGL/glDraw.cmo
132− lablGL/glArray.ml -> lablGL/glArray.cmo
133− parser.ml -> parser.cmo
134− build/confstruct.ml -> confstruct.cmo
135− config.ml -> config.cmo
136− ffi.ml -> ffi.cmo
137− glutils.ml -> glutils.cmo
138− lablGL/glMat.ml -> lablGL/glMat.cmo
139− listview.ml -> listview.cmo
140− help.mli -> help.cmi
141− help.ml -> help.cmo
142− lablGL/glClear.ml -> lablGL/glClear.cmo
143−main.ml -> main.cmo
144−link.c -> link.o
145−cutils.c -> cutils.o
146−version.c -> version.o
147−lablGL/ml_gl.c -> lablGL/ml_gl.o
148−lablGL/ml_glarray.c -> lablGL/ml_glarray.o
149−lablGL/ml_raw.c -> lablGL/ml_raw.o
150−wsi/x11/keysym2ucs.c -> wsi/x11/keysym2ucs.o
151−wsi/x11/xlib.c -> wsi/x11/xlib.o
152−10 sec
153− Command being timed: "bash build.bash build"
154− User time (seconds): 8.75
155− System time (seconds): 0.93
156− Percent of CPU this job got: 101%
157− Elapsed (wall clock) time (h:mm:ss or m:ss): 0:09.55
158− Average shared text size (kbytes): 0
159− Average unshared data size (kbytes): 0
160− Average stack size (kbytes): 0
161− Average total size (kbytes): 0
162− Maximum resident set size (kbytes): 77656
163− Average resident set size (kbytes): 0
164− Major (requiring I/O) page faults: 0
165− Minor (reclaiming a frame) page faults: 348362
166− Voluntary context switches: 3401
167− Involuntary context switches: 706
168− Swaps: 0
169− File system inputs: 0
170− File system outputs: 10272
171− Socket messages sent: 0
172− Socket messages received: 0
173− Signals delivered: 0
174− Page size (bytes): 4096
175− Exit status: 0
176−~/xsrc/llpp
177−- rm build/*.cm* build/lablGL/*.cm* build/wsi/x11/*.cm* build/*.o build/lablGL/*.o build/wsi/x11/*.o
178−~/xsrc/llpp
179−- \time -pv make -j4 build/llpp -s
180− Command being timed: "make -j4 build/llpp -s"
181− User time (seconds): 5.11
182− System time (seconds): 0.64
183− Percent of CPU this job got: 220%
184− Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.60
185− Average shared text size (kbytes): 0
186− Average unshared data size (kbytes): 0
187− Average stack size (kbytes): 0
188− Average total size (kbytes): 0
189− Maximum resident set size (kbytes): 77648
190− Average resident set size (kbytes): 0
191− Major (requiring I/O) page faults: 0
192− Minor (reclaiming a frame) page faults: 166642
193− Voluntary context switches: 303
194− Involuntary context switches: 369
195− Swaps: 0
196− File system inputs: 0
197− File system outputs: 89008
198− Socket messages sent: 0
199− Socket messages received: 0
200− Signals delivered: 0
201− Page size (bytes): 4096
202− Exit status: 0
203−~/xsrc/llpp
204−- \time -pv make -j1 build/llpp -s
205− Command being timed: "make -j1 build/llpp -s"
206− User time (seconds): 3.62
207− System time (seconds): 0.43
208− Percent of CPU this job got: 99%
209− Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.07
210− Average shared text size (kbytes): 0
211− Average unshared data size (kbytes): 0
212− Average stack size (kbytes): 0
213− Average total size (kbytes): 0
214− Maximum resident set size (kbytes): 77592
215− Average resident set size (kbytes): 0
216− Major (requiring I/O) page faults: 0
217− Minor (reclaiming a frame) page faults: 166577
218− Voluntary context switches: 176
219− Involuntary context switches: 138
220− Swaps: 0
221− File system inputs: 0
222− File system outputs: 89008
223− Socket messages sent: 0
224− Socket messages received: 0
225− Signals delivered: 0
226− Page size (bytes): 4096
227− Exit status: 0

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t