.TLMaintaining Files on Plan 9 with Mk.AUAndrew G. Humeandrew@research.att.comBob Flandrenabobf@plan9.bell-labs.com.AB.PP.CW Mkis a toolfor describing and maintaining dependencies betweenfiles.It is similar to theUNIX program.CW make ,but provides several new capabilities..CW Mk\fR'\fPsflexible rule specifications, implieddependency derivation, and parallelexecution of maintenance actions arewell-suited to the Plan 9 environment.Almost all Plan 9 maintenance proceduresare automated using.CW mk ..AE.NH 1Introduction.PP.CW Mkperforms the same task as.CW make[Feld79] but is different in many ways.This document describes.CW mk\fR'\fPscapabilities and discusses how they areused in Plan 9. There are occasional referencesto differences between .CW mkand.CW make ,butan appendix summarizes the details..PP.CW Mkworks well in the Plan 9 environment.It supports program constructionin a heterogeneous environment andexploits the power of multiprocessors by executingmaintenance actions in parallel.It interacts seamlessly with the Plan 9 commandinterpreter.CW rcand accepts pattern-based dependency specificationsthat are not limited to describingrules for program construction.The result is a tool that is flexible enough toperform many maintenance tasks includingdatabase maintenance,hardware design, and document production..PPThis document begins by discussing the syntax of the input file,the pattern matching capabilities, andthe special rules for maintaining archives.A brief description of.CW mk\fR'\fPsalgorithm for deriving dependenciesis followed by a discussionof the conventions used to resolve ambiguousspecifications. The final sectionsdescribe parallel executionand special features..PPAn earlier paper [Hume87]provides a detailed discussion of.CW mk\fR'\fPsdesign..NH 1The \f(CWMkfile\fP.PP.CW Mkreads a file describing relationships among filesand executes commands to bring the files up to date.The specification file, called a.CW mkfile ,contains three types of statements:assignments, includes, and rules.Assignment and include statements are similarto those in C.Rules specify dependencies between a.I targetand its.I prerequisites .When the target and prerequisites are files, theirmodification times determine if theyare out of date. Rules often contain a.I recipe ,an.I rc (1)script that produces the target fromthe prerequisites..PPThis simple.CW mkfileproduces an executablefrom a C source file:.P1CC=pccf1: f1.c $CC -o f1 f1.c.P2The first line assigns the name of the portable ANSI/POSIX compilerto the.CW mkvariable.CW CC ;subsequent references of the form.CW $CCselect this compiler.The only rule specifies a dependence between the target file.CW f1and the prerequisite file.CW f1.c .If the target does not exist or if theprerequisite has been modified more recently thanthe target,.CW mkpasses the recipe to.CW rcfor execution. Here,.CW f1.cis compiled and loaded to produce.CW f1 ..PPThe native Plan 9 environmentrequires executables forall architectures, not only the current one.The Plan 9 version of the same.CW mkfilelooks like:.P1</$objtype/mkfileCFLAGS=-w f1: f1.$O $LD $LDFLAGS -o f1 f1.$Of1.$O: f1.c $CC $CFLAGS f1.c.P2The first line is an include statementthat replaces itself with the contents of the file.CW /$objtype/mkfile .The variable.CW $objtypeis inherited from the environment andcontains the name of the target architecture.The prototype.CW mkfilefor that architecture defines architecture-specific variables:.CW CCand.CW LDare the names of the compiler and loader,.CW O is the code character of the architecture.The rules compile the source file into an objectfile and invoke the loader to produce.CW f1 .Invoking.CW mkfrom the command line as follows.P1% objtype=sparc mkkc -w f1.ckl $LDFLAGS -o f1 f1.k%.P2produces the.CW sparcexecutable of program.CW f1regardless of the current architecture type..PPWe can extend the.CW mkfileto build two programs:.P1</$objtype/mkfileCFLAGS=-wALL=f1 f2 all:V: $ALL f1: f1.$O $LD $LDFLAGS -o f1 f1.$Of1.$O: f1.c $CC $CFLAGS f1.cf2: f2.$O $LD $LDFLAGS -o f2 f2.$Of2.$O: f2.c $CC $CFLAGS f2.c.P2The target.CW all ,modified by the.I attribute.CW V ,builds both programs.The attribute identifies .CW allas a dummy target that isnot related to a file of the same name;its precise effect is explained later.This example describes cascading dependencies:the first target depends on another which depends on a third andso on.Here, individual rules build eachprogram; later we'll see how to do this with ageneral rule..NH 1Variables and the environment.PP.CW Mkdoes not distinguish between itsvariables and.CW rcvariables in the environment.When.CW mkstarts, it imports each environment variable into a.CW mkvariable of the same name. Before executing a recipe,.CW mkexports all variables, including thoseinherited from the environment,to the environment in which.CW rcexecutes the recipe..PPThere are several ways for avariable to take a value.It can be set with an assignment statement,inherited from the environment, or specifiedon the command line..CW Mkalso maintains several internal variables;these variables are described in.I mk (1).Assignments have the following decreasing order of precedence:.LP.in .7i1) Command line assignment.br2) Assignment statement.br3) Imported from the environment.br4) Implicitly set by \f(CWmk\fP.in 0.LPFor example, a command line assignment overridesa value imported from the environment..PPAll variable values are strings. They can beused for pattern matching andcomparison but not for arithmetic.A.I listis a string containing several values separated bywhite space. Each member ishandled individually during pattern matching,target selection, and prerequisite evaluation..PPA.I namelistis a list produced bytransforming the members of an existing list.The transform applies a pattern to each member,replacing each matched string with a new string,much as in the substitute command in.I sam (1)or.I ed (1).The syntax is.P1${\fIvar\fP:A%B=C%D}.P2where.I varis a variable.The pattern.CW A%Bmatches a member beginning with the string.I Aand ending with the string.I Bwith any string in between;it behaves like the regular expression.CW A.*B .When a member of the.I varlistmatches this pattern,the string.I Creplaces.I A ,.I Dreplaces.I B ,and the matched string replaces itself.Any of.I A ,.I B ,.I C ,or.I Dmay be the empty string. In effect, a namelist isgenerated by applying the.I ed (1)substitute command.P1 s/\fIA\fP(.*)\fIB\fP/\fIC\fP\e1\fID\fP/.P2to each member of a variable..PPNamelists are useful for generatinga list based on a predictable transformation.For example,.P1 SRC=a.c b.c c.c OBJ=${SRC:%.c=%.v}.P2assigns the list \f(CW(a.v b.v c.v)\fP to.CW OBJ .A namelist may be used anywhere a variable is allowedexcept in a recipe..PPCommand output is assigned to a variableusing the normal.CW rcsyntax:.P1 var=`{rc command}.P2The command executes in an environment populatedwith previously assigned variables, including thoseinherited from.CW mk\fR'\fPsexecution environment.The command maybe arbitrarily complex; for example,.P1 TARG=`{ls -d *.[cyl] | sed 's/..$//'}.P2assigns a list of the C, yacc, and Alef source files in the currentdirectory, stripped of their suffix, to the variable.CW TARG ..NH 1The include statement.PPThe include statementreplaces itself with the contents of a file.It is functionally similar to the C.CW #includestatement but uses a different syntax:.P1 <\fIfilename\fP.P2The contents of the file are evaluatedas they are read.An include statement may be used anywhere exceptin a recipe..PPUnlike.CW make ,.CW mkhas no built-in rules. Instead,the include statement allows generic rulesto be imported from a prototype.CW mkfile ;most Plan 9.CW mkfilesuse this approach [Flan95]..NH 1Rules.PPA rule has four elements: targets,prerequisites, attributes, and a recipe.It has the form:.P1\fItargets\fP:\fIattributes\fP:\fIprerequisites\fP \fIrecipe\fP.P2The first line, containing thetargets, attributes, and prerequisites isthe.I "rule header" ;itmust begin at the left margin.The recipe contains zero or more lines,each of which begins with white space.One or more targets must be specified but theattributes, prerequisites, and recipe are optional.A rule specifiesa dependency between the targets and the prerequisites;the recipe brings the targetsup to date with the prerequisites.Attributes modify.CW mk\fR'\fPsevaluation of a rule..PPNormally the target is a file that dependson one or more prerequisite files..CW Mkcompares the modification times of each targetand each prerequisite; a target is considered out of datewhen it does not exist or when a prerequisite has been modifiedmore recently.When a target is out of date,.CW mkexecutes therecipe to bring it up to date.When the recipe completes,the modification time of the target is checked andused in later dependency evaluations.If the recipe does not update the target,evaluation continues with the out of date target..PPA prerequisite of one rulemay be the target of another. Whenthis happens, the rules cascadeto define a multi-step procedure.For example,an executable target depends on prerequisiteobject files, each of which is a targetin a rule with a C source file as the prerequisite..CW Mkfollows a chain of dependencies until it encountersa prerequisite that is not a target of another ruleor it finds a target thatis up to date. It thenexecutes the recipes in reverse order to producethe desired target..PPThe rule header is evaluated when the rule is read.Variables are replaced by their values, namelists aregenerated, andcommands are replaced by theiroutput at this time..PPMost attributes modify.CW mk\fR'\fPsevaluation of a rule.An attribute is usually a single letter but someare more complicated.This paper only discusses commonly used attributes;see.I mk (1)for a complete list..PPThe.CW Vattribute identifies a.I virtual target;that is, a target that is not a file.For example,.P1clean:V: rm *.$O $O.out.P2removes executables and compiler intermediate files.The target is virtual because it does not refer to a file named.CW clean .Without the attribute, the recipe would not beexecuted if a file named.CW clean existed.The.CW Qsilences the default printing of a recipe beforeit is executed.It is useful when the output of a recipe issimilar to the recipe:.P1default:QV: echo 'No default target; use mk all or mk install'.P2.PPA recipe is an.CW rcscript. The recipe is optional; when it ismissing, the rule is handled specially, as described later.Unlike.CW make ,.CW mkexecutes recipes without interpretation.AfterLines 1–500 of 1539. Load more
.TL Maintaining Files on Plan 9 with Mk .AU Andrew G. Hume andrew@research.att.com Bob Flandrena bobf@plan9.bell-labs.com .AB .PP .CW Mk is a tool for describing and maintaining dependencies between files. It is similar to the UNIX program .CW make , but provides several new capabilities. .CW Mk\fR'\fPs flexible rule specifications, implied dependency derivation, and parallel execution of maintenance actions are well-suited to the Plan 9 environment. Almost all Plan 9 maintenance procedures are automated using .CW mk . .AE .NH 1 Introduction .PP .CW Mk performs the same task as .CW make [Feld79] but is different in many ways. This document describes .CW mk\fR'\fPs capabilities and discusses how they are used in Plan 9. There are occasional references to differences between .CW mk and .CW make , but an appendix summarizes the details. .PP .CW Mk works well in the Plan 9 environment. It supports program construction in a heterogeneous environment and exploits the power of multiprocessors by executing maintenance actions in parallel. It interacts seamlessly with the Plan 9 command interpreter .CW rc and accepts pattern-based dependency specifications that are not limited to describing rules for program construction. The result is a tool that is flexible enough to perform many maintenance tasks including database maintenance, hardware design, and document production. .PP This document begins by discussing the syntax of the input file, the pattern matching capabilities, and the special rules for maintaining archives. A brief description of .CW mk\fR'\fPs algorithm for deriving dependencies is followed by a discussion of the conventions used to resolve ambiguous specifications. The final sections describe parallel execution and special features. .PP An earlier paper [Hume87] provides a detailed discussion of .CW mk\fR'\fPs design. .NH 1 The \f(CWMkfile\fP .PP .CW Mk reads a file describing relationships among files and executes commands to bring the files up to date. The specification file, called a .CW mkfile , contains three types of statements: assignments, includes, and rules. Assignment and include statements are similar to those in C. Rules specify dependencies between a .I target and its .I prerequisites . When the target and prerequisites are files, their modification times determine if they are out of date. Rules often contain a .I recipe , an .I rc (1) script that produces the target from the prerequisites. .PP This simple .CW mkfile produces an executable from a C source file: .P1 CC=pcc f1: f1.c $CC -o f1 f1.c .P2 The first line assigns the name of the portable ANSI/POSIX compiler to the .CW mk variable .CW CC ; subsequent references of the form .CW $CC select this compiler. The only rule specifies a dependence between the target file .CW f1 and the prerequisite file .CW f1.c . If the target does not exist or if the prerequisite has been modified more recently than the target, .CW mk passes the recipe to .CW rc for execution. Here, .CW f1.c is compiled and loaded to produce .CW f1 . .PP The native Plan 9 environment requires executables for all architectures, not only the current one. The Plan 9 version of the same .CW mkfile looks like: .P1 </$objtype/mkfile CFLAGS=-w f1: f1.$O $LD $LDFLAGS -o f1 f1.$O f1.$O: f1.c $CC $CFLAGS f1.c .P2 The first line is an include statement that replaces itself with the contents of the file .CW /$objtype/mkfile . The variable .CW $objtype is inherited from the environment and contains the name of the target architecture. The prototype .CW mkfile for that architecture defines architecture-specific variables: .CW CC and .CW LD are the names of the compiler and loader, .CW O is the code character of the architecture. The rules compile the source file into an object file and invoke the loader to produce .CW f1 . Invoking .CW mk from the command line as follows .P1 % objtype=sparc mk kc -w f1.c kl $LDFLAGS -o f1 f1.k % .P2 produces the .CW sparc executable of program .CW f1 regardless of the current architecture type. .PP We can extend the .CW mkfile to build two programs: .P1 </$objtype/mkfile CFLAGS=-w ALL=f1 f2 all:V: $ALL f1: f1.$O $LD $LDFLAGS -o f1 f1.$O f1.$O: f1.c $CC $CFLAGS f1.c f2: f2.$O $LD $LDFLAGS -o f2 f2.$O f2.$O: f2.c $CC $CFLAGS f2.c .P2 The target .CW all , modified by the .I attribute .CW V , builds both programs. The attribute identifies .CW all as a dummy target that is not related to a file of the same name; its precise effect is explained later. This example describes cascading dependencies: the first target depends on another which depends on a third and so on. Here, individual rules build each program; later we'll see how to do this with a general rule. .NH 1 Variables and the environment .PP .CW Mk does not distinguish between its variables and .CW rc variables in the environment. When .CW mk starts, it imports each environment variable into a .CW mk variable of the same name. Before executing a recipe, .CW mk exports all variables, including those inherited from the environment, to the environment in which .CW rc executes the recipe. .PP There are several ways for a variable to take a value. It can be set with an assignment statement, inherited from the environment, or specified on the command line. .CW Mk also maintains several internal variables; these variables are described in .I mk (1). Assignments have the following decreasing order of precedence: .LP .in .7i 1) Command line assignment .br 2) Assignment statement .br 3) Imported from the environment .br 4) Implicitly set by \f(CWmk\fP .in 0 .LP For example, a command line assignment overrides a value imported from the environment. .PP All variable values are strings. They can be used for pattern matching and comparison but not for arithmetic. A .I list is a string containing several values separated by white space. Each member is handled individually during pattern matching, target selection, and prerequisite evaluation. .PP A .I namelist is a list produced by transforming the members of an existing list. The transform applies a pattern to each member, replacing each matched string with a new string, much as in the substitute command in .I sam (1) or .I ed (1). The syntax is .P1 ${\fIvar\fP:A%B=C%D} .P2 where .I var is a variable. The pattern .CW A%B matches a member beginning with the string .I A and ending with the string .I B with any string in between; it behaves like the regular expression .CW A.*B . When a member of the .I var list matches this pattern, the string .I C replaces .I A , .I D replaces .I B , and the matched string replaces itself. Any of .I A , .I B , .I C , or .I D may be the empty string. In effect, a namelist is generated by applying the .I ed (1) substitute command .P1 s/\fIA\fP(.*)\fIB\fP/\fIC\fP\e1\fID\fP/ .P2 to each member of a variable. .PP Namelists are useful for generating a list based on a predictable transformation. For example, .P1 SRC=a.c b.c c.c OBJ=${SRC:%.c=%.v} .P2 assigns the list \f(CW(a.v b.v c.v)\fP to .CW OBJ . A namelist may be used anywhere a variable is allowed except in a recipe. .PP Command output is assigned to a variable using the normal .CW rc syntax: .P1 var=`{rc command} .P2 The command executes in an environment populated with previously assigned variables, including those inherited from .CW mk\fR'\fPs execution environment. The command may be arbitrarily complex; for example, .P1 TARG=`{ls -d *.[cyl] | sed 's/..$//'} .P2 assigns a list of the C, yacc, and Alef source files in the current directory, stripped of their suffix, to the variable .CW TARG . .NH 1 The include statement .PP The include statement replaces itself with the contents of a file. It is functionally similar to the C .CW #include statement but uses a different syntax: .P1 <\fIfilename\fP .P2 The contents of the file are evaluated as they are read. An include statement may be used anywhere except in a recipe. .PP Unlike .CW make , .CW mk has no built-in rules. Instead, the include statement allows generic rules to be imported from a prototype .CW mkfile ; most Plan 9 .CW mkfiles use this approach [Flan95]. .NH 1 Rules .PP A rule has four elements: targets, prerequisites, attributes, and a recipe. It has the form: .P1 \fItargets\fP:\fIattributes\fP:\fIprerequisites\fP \fIrecipe\fP .P2 The first line, containing the targets, attributes, and prerequisites is the .I "rule header" ; it must begin at the left margin. The recipe contains zero or more lines, each of which begins with white space. One or more targets must be specified but the attributes, prerequisites, and recipe are optional. A rule specifies a dependency between the targets and the prerequisites; the recipe brings the targets up to date with the prerequisites. Attributes modify .CW mk\fR'\fPs evaluation of a rule. .PP Normally the target is a file that depends on one or more prerequisite files. .CW Mk compares the modification times of each target and each prerequisite; a target is considered out of date when it does not exist or when a prerequisite has been modified more recently. When a target is out of date, .CW mk executes the recipe to bring it up to date. When the recipe completes, the modification time of the target is checked and used in later dependency evaluations. If the recipe does not update the target, evaluation continues with the out of date target. .PP A prerequisite of one rule may be the target of another. When this happens, the rules cascade to define a multi-step procedure. For example, an executable target depends on prerequisite object files, each of which is a target in a rule with a C source file as the prerequisite. .CW Mk follows a chain of dependencies until it encounters a prerequisite that is not a target of another rule or it finds a target that is up to date. It then executes the recipes in reverse order to produce the desired target. .PP The rule header is evaluated when the rule is read. Variables are replaced by their values, namelists are generated, and commands are replaced by their output at this time. .PP Most attributes modify .CW mk\fR'\fPs evaluation of a rule. An attribute is usually a single letter but some are more complicated. This paper only discusses commonly used attributes; see .I mk (1) for a complete list. .PP The .CW V attribute identifies a .I virtual target; that is, a target that is not a file. For example, .P1 clean:V: rm *.$O $O.out .P2 removes executables and compiler intermediate files. The target is virtual because it does not refer to a file named .CW clean . Without the attribute, the recipe would not be executed if a file named .CW clean existed. The .CW Q silences the default printing of a recipe before it is executed. It is useful when the output of a recipe is similar to the recipe: .P1 default:QV: echo 'No default target; use mk all or mk install' .P2 .PP A recipe is an .CW rc script. The recipe is optional; when it is missing, the rule is handled specially, as described later. Unlike .CW make , .CW mk executes recipes without interpretation. After