.HTML "The Organization of Networks in Plan 9.TLThe Organization of Networks in Plan 9.AUDave PresottoPhil Winterbottom.sppresotto,philw@plan9.bell-labs.com.AB.FSOriginally appeared in.IProc. of the Winter 1993 USENIX Conf.,.Rpp. 271-280,San Diego, CA.FEIn a distributed system networks are of paramount importance. Thispaper describes the implementation, design philosophy, and organizationof network support in Plan 9. Topics include network requirementsfor distributed systems, our kernel implementation, network naming, user interfaces,and performance. We also observe that much of this organization is relevant tocurrent systems..AE.NHIntroduction.PPPlan 9 [Pike90] is a general-purpose, multi-user, portable distributed systemimplemented on a variety of computers and networks.What distinguishes Plan 9 is its organization.The goals of this organization were toreduce administrationand to promote resource sharing. One of the keys to its success as a distributedsystem is the organization and management of its networks..PPA Plan 9 system comprises file servers, CPU servers and terminals.The file servers and CPU servers are typically centrallylocated multiprocessor machines with large memories andhigh speed interconnects.A variety of workstation-class machinesserve as terminalsconnected to the central servers using several networks and protocols.The architecture of the system demands a hierarchy of networkspeeds matching the needs of the components.Connections between file servers and CPU servers are high-bandwidth point-to-pointfiber links.Connections from the servers fan out to local terminalsusing medium speed networkssuch as Ethernet [Met80] and Datakit [Fra80].Low speed connections via the Internet andthe AT&T backbone serve users in Oregon and Illinois.Basic Rate ISDN data service and 9600 baud serial lines provide slowlinks to users at home..PPSince CPU servers and terminals use the same kernel,users may choose to run programs locally ontheir terminals or remotely on CPU servers.The organization of Plan 9 hides the details of system connectivityallowing both users and administrators to configure their environmentto be as distributed or centralized as they wish.Simple commands support theconstruction of a locally represented name spacespanning many machines and networks.At work, users tend to use their terminals like workstations,running interactive programs locally andreserving the CPU servers for data or compute intensive jobssuch as compiling and computing chess endgames.At home or when connected overa slow network, users tend to do most work on the CPU server to minimizetraffic on the slow links.The goal of the network organization is to provide the sameenvironment to the user wherever resources are used..NH Kernel Network Support.PPNetworks play a central role in any distributed system. This is particularlytrue in Plan 9 where most resources are provided by servers external to the kernel.The importance of the networking code within the kernelis reflected by its size;of 25,000 lines of kernel code, 12,500 are network and protocol related.Networks are continually being added and the fraction of codedevoted to communicationsis growing.Moreover, the network code is complex.Protocol implementations consist almost entirely ofsynchronization and dynamic memory management, areas demanding subtle error recoverystrategies.The kernel currently supports Datakit, point-to-point fiber links,an Internet (IP) protocol suite and ISDN data service.The variety of networks and machineshas raised issues not addressed by other systems running on commercialhardware supporting only Ethernet or FDDI..NH 2The File System protocol.PPA central idea in Plan 9 is the representation of a resource as a hierarchicalfile system.Each process assembles a view of the system by building a.I "name space[Needham] connecting its resources.File systems need not represent disc files; in fact, most Plan 9 file systems have nopermanent storage.A typical file system dynamically representssome resource like a set of network connections or the process table.Communication between the kernel, device drivers, and local or remote file servers uses aprotocol called 9P. The protocol consists of 17 messagesdescribing operations on files and directories.Kernel resident device and protocol drivers use a procedural versionof the protocol while external file servers use an RPC form.Nearly all traffic between Plan 9 systems consistsof 9P messages.9P relies on several properties of the underlying transport protocol.It assumes messages arrive reliably and in sequence andthat delimiters between messagesare preserved.When a protocol does not meet theserequirements (for example, TCP does not preserve delimiters)we provide mechanisms to marshal messages before handing themto the system..PPA kernel data structure, the.I channel ,is a handle to a file server.Operations on a channel generate the following 9P messages.The.CW sessionand.CW attachmessages authenticate a connection, established by means external to 9P,and validate its user.The result is an authenticatedchannelreferencing the root of theserver.The.CW clonemessage makes a new channel identical to an existing channel, much likethe.CW dupsystem call.Achannelmay be moved to a file on the server using a.CW walkmessage to descend each level in the hierarchy.The.CW statand.CW wstatmessages read and write the attributes of the file referenced by a channel.The.CW openmessage prepares a channel for subsequent.CW readand.CW writemessages to access the contents of the file..CW Createand.CW removeperform the actions implied by their names on the filereferenced by the channel.The.CW clunkmessage discards a channel without affecting the file..PPA kernel resident file server called the.I "mount driver"converts the procedural version of 9P into RPCs.The.I mountsystem call provides a file descriptor, which can bea pipe to a user process or a network connection to a remote machine, tobe associated with the mount point.After a mount, operationson the file tree below the mount point are sent as messages to the file server.Themountdriver manages buffers, packs and unpacks parameters frommessages, and demultiplexes among processes using the file server..NH 2Kernel Organization.PPThe network code in the kernel is divided into three layers: hardware interface,protocol processing, and program interface.A device driver typically uses streams to connect the two interface layers.Additional stream modules may be pushed ona device to process protocols.Each device driver is a kernel-resident file system.Simple device drivers serve a single leveldirectory containing just a few files;for example, we represent each UARTby a data and a control file..P1cpu% cd /devcpu% ls -l eia*--rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia1--rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia1ctl--rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia2--rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia2ctlcpu%.P2The control file is used to control the device;writing the string.CW b1200to.CW /dev/eia1ctlsets the line to 1200 baud..PPMultiplexed devices presenta more complex interface structure.For example, the LANCE Ethernet driverserves a two level file tree (Figure 1)providing.IP \(budevice control and configuration.IP \(buuser-level protocols like ARP.IP \(budiagnostic interfaces for snooping software..LPThe top directory contains a.CW clonefile and a directory for each connection, numbered.CW 1to.CW n .Each connection directory corresponds to an Ethernet packet type.Opening the.CW clonefile finds an unused connection directoryand opens its.CW ctlfile.Reading the control file returns the ASCII connection number; the userprocess can use this value to construct the name of the proper connection directory.In each connection directory files named.CW ctl , .CW data , .CW stats ,and .CW typeprovide access to the connection.Writing the string.CW "connect 2048"to the.CW ctlfile sets the packet type to 2048andconfigures the connection to receiveall IP packets sent to the machine.Subsequent reads of the file.CW typeyield the string.CW 2048 .The.CW datafile accesses the media;reading itreturns thenext packet of the selected type.Writing the filequeues a packet for transmission afterappending a packet header containing the source address and packet type.The.CW statsfile returns ASCII text containing the interface address,packet input/output counts, error statistics, and general informationabout the state of the interface..so tree.pout.PPIf several connections on an interfaceare configured for a particular packet type, each receives acopy of the incoming packets.The special packet type.CW -1selects all packets.Writing the strings.CW promiscuousand.CW connect.CW -1to the.CW ctlfileconfigures a conversation to receive all packets on the Ethernet..PPAlthough the driver interface may seem elaborate,the representation of a device as a set of files using ASCII strings forcommunication has several advantages.Any mechanism supporting remote access to files immediatelyallows a remote machine to use our interfaces as gateways.Using ASCII strings to control the interface avoids byte order problems andensures a uniform representation fordevices on the same machine and even allows devices to be accessed remotely.Representing dissimilar devices by the same set of files allows common toolsto serveseveral networks or interfaces.Programs like.CW sttyare replaced by.CW echoand shell redirection..NH 2Protocol devices.PPNetwork connections are represented as pseudo-devices called protocol devices.Protocol device drivers exist for the Datakit URP protocol and for each of theInternet IP protocols TCP, UDP, and IL.IL, described below, is a new communication protocol used by Plan 9 fortransmitting file system RPC's.All protocol devices look identical so user programs contain nonetwork-specific code..PPEach protocol device driver serves a directory structuresimilar to that of the Ethernet driver.The top directory contains a.CW clonefile and a directory for each connection numbered.CW 0to.CW n .Each connection directory contains files to control oneconnection and to send and receive information.A TCP connection directory looks like this:.P1cpu% cd /net/tcp/2cpu% ls -l--rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 ctl--rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 data--rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 listen--r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 local--r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 remote--r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 statuscpu% cat local remote status135.104.9.31 5012135.104.53.11 564tcp/2 1 Established connectcpu%.P2The files.CW local ,.CW remote ,and.CW statussupply information about the state of the connection.The.CW dataand.CW ctlfilesprovide access to the process end of the stream implementing the protocol.The.CW listenfile is used to accept incoming calls from the network..PPThe following steps establish a connection..IP 1)The clone device of theappropriate protocol directory is opened to reserve an unused connection..IP 2)The file descriptor returned by the open points to the.CW ctlfile of the new connection.Reading that file descriptor returns an ASCII string containingthe connection number..IP 3)A protocol/network specific ASCII address string is written to the.CW ctlfile..IP 4)The path of the.CW datafile is constructed using the connection number.When the.CW datafile is opened the connection is established..LPA process can read and write this file descriptorto send and receive messages from the network.If the process opens the.CW listenfile it blocks until an incoming call is received.An address string written to the.CW ctlfile before the listen selects theports or services the process is prepared to accept.When an incoming call is received, the open completesand returns a file descriptorpointing to the.CW ctlfile of the new connection.Reading the.CW ctlfile yields a connection number used to construct the path of the.CW datafile.A connection remains established while any of the files in the connection directoryare referenced or until a close is received from the network..NH 2Streams.PPA.I stream [Rit84a][Presotto] is a bidirectional channel connecting aphysical or pseudo-device to user processes.The user processes insert and remove data at one end of the stream.Kernel processes acting on behalf of a device insert data atthe other end.Asynchronous communications channels such as pipes,TCP conversations, Datakit conversations, and RS232 lines are implemented usingstreams..PPA stream comprises a linear list of.I "processing modules" .Each module has both an upstream (toward the process) anddownstream (toward the device).I "put routine" .Calling the put routine of the module on either end of the streaminserts data into the stream.Each module calls the succeeding one to send data up or down the stream..PPAn instance of a processing module is represented by a pair of.I queues ,one for each direction.The queues point to the put procedures and can be usedto queue information traveling along the stream.Some put routines queue data locally and send it along the stream at somelater time, either due to a subsequent call or an asynchronousevent such as a retransmission timer or a device interrupt.Processing modules create helper kernel processes toprovide a context for handling asynchronous events.For example, a helper kernel process awakens periodicallyto perform any necessary TCP retransmissions.The use of kernel processes instead of serialized run-to-completion service routinesdiffers from the implementation of Unix streams.Unix service routines cannotuse any blocking kernel resource and they lack a local long-lived state.Helper kernel processes solve these problems and simplify the stream code..PPThere is no implicit synchronization in our streams.Each processing module must ensure that concurrent processes using the streamare synchronized.This maximizes concurrency but introduces thepossibility of deadlock.However, deadlocks are easily avoided by careful programming; todate they have not caused us problems..PPInformation is represented by linked lists of kernel structures called.I blocks .Each block contains a type, some state flags, and pointers toan optional buffer.Block buffers can hold either data or control information, i.e., directivesto the processing modules.Blocks and block buffers are dynamically allocated from kernel memory..NH 3User Interface.PPA stream is represented at user level as two files, .CW ctland.CW data .The actual names can be changed by the device driver using the stream,as we saw earlier in the example of the UART driver.The first process to open either file creates the stream automatically.The last close destroys it.Writing to the.CW datafile copies the data into kernel blocksand passes them to the downstream put routine of the first processing module.A write of less than 32K is guaranteed to be contained by a single block.Concurrent writes to the same stream are not synchronized, although the32K block size assures atomic writes for most protocols.The last block written is flagged with a delimiterto alert downstream modules that care about write boundaries.In most cases the first put routine calls the second, the secondcalls the third, and so on until the data is output.As a consequence, most data is output without context switching..PPReading from the.CW datafile returns data queued at the top of the stream.The read terminates when the read count is reachedor when the end of a delimited block is encountered.A per stream read lock ensures only one processcan read from a stream at a time and guaranteesthat the bytes read were contiguous bytes from thestream..PPLike UNIX streams [Rit84a],Plan 9 streams can be dynamically configured.The stream system intercepts and interpretsthe following control blocks:.IP "\f(CWpush\fP \fIname\fR" 15adds an instance of the processing module .I nameto the top of the stream..IP \f(CWpop\fP 15Lines 1–500 of 1336. Load more
.HTML "The Organization of Networks in Plan 9 .TL The Organization of Networks in Plan 9 .AU Dave Presotto Phil Winterbottom .sp presotto,philw@plan9.bell-labs.com .AB .FS Originally appeared in .I Proc. of the Winter 1993 USENIX Conf., .R pp. 271-280, San Diego, CA .FE In a distributed system networks are of paramount importance. This paper describes the implementation, design philosophy, and organization of network support in Plan 9. Topics include network requirements for distributed systems, our kernel implementation, network naming, user interfaces, and performance. We also observe that much of this organization is relevant to current systems. .AE .NH Introduction .PP Plan 9 [Pike90] is a general-purpose, multi-user, portable distributed system implemented on a variety of computers and networks. What distinguishes Plan 9 is its organization. The goals of this organization were to reduce administration and to promote resource sharing. One of the keys to its success as a distributed system is the organization and management of its networks. .PP A Plan 9 system comprises file servers, CPU servers and terminals. The file servers and CPU servers are typically centrally located multiprocessor machines with large memories and high speed interconnects. A variety of workstation-class machines serve as terminals connected to the central servers using several networks and protocols. The architecture of the system demands a hierarchy of network speeds matching the needs of the components. Connections between file servers and CPU servers are high-bandwidth point-to-point fiber links. Connections from the servers fan out to local terminals using medium speed networks such as Ethernet [Met80] and Datakit [Fra80]. Low speed connections via the Internet and the AT&T backbone serve users in Oregon and Illinois. Basic Rate ISDN data service and 9600 baud serial lines provide slow links to users at home. .PP Since CPU servers and terminals use the same kernel, users may choose to run programs locally on their terminals or remotely on CPU servers. The organization of Plan 9 hides the details of system connectivity allowing both users and administrators to configure their environment to be as distributed or centralized as they wish. Simple commands support the construction of a locally represented name space spanning many machines and networks. At work, users tend to use their terminals like workstations, running interactive programs locally and reserving the CPU servers for data or compute intensive jobs such as compiling and computing chess endgames. At home or when connected over a slow network, users tend to do most work on the CPU server to minimize traffic on the slow links. The goal of the network organization is to provide the same environment to the user wherever resources are used. .NH Kernel Network Support .PP Networks play a central role in any distributed system. This is particularly true in Plan 9 where most resources are provided by servers external to the kernel. The importance of the networking code within the kernel is reflected by its size; of 25,000 lines of kernel code, 12,500 are network and protocol related. Networks are continually being added and the fraction of code devoted to communications is growing. Moreover, the network code is complex. Protocol implementations consist almost entirely of synchronization and dynamic memory management, areas demanding subtle error recovery strategies. The kernel currently supports Datakit, point-to-point fiber links, an Internet (IP) protocol suite and ISDN data service. The variety of networks and machines has raised issues not addressed by other systems running on commercial hardware supporting only Ethernet or FDDI. .NH 2 The File System protocol .PP A central idea in Plan 9 is the representation of a resource as a hierarchical file system. Each process assembles a view of the system by building a .I "name space [Needham] connecting its resources. File systems need not represent disc files; in fact, most Plan 9 file systems have no permanent storage. A typical file system dynamically represents some resource like a set of network connections or the process table. Communication between the kernel, device drivers, and local or remote file servers uses a protocol called 9P. The protocol consists of 17 messages describing operations on files and directories. Kernel resident device and protocol drivers use a procedural version of the protocol while external file servers use an RPC form. Nearly all traffic between Plan 9 systems consists of 9P messages. 9P relies on several properties of the underlying transport protocol. It assumes messages arrive reliably and in sequence and that delimiters between messages are preserved. When a protocol does not meet these requirements (for example, TCP does not preserve delimiters) we provide mechanisms to marshal messages before handing them to the system. .PP A kernel data structure, the .I channel , is a handle to a file server. Operations on a channel generate the following 9P messages. The .CW session and .CW attach messages authenticate a connection, established by means external to 9P, and validate its user. The result is an authenticated channel referencing the root of the server. The .CW clone message makes a new channel identical to an existing channel, much like the .CW dup system call. A channel may be moved to a file on the server using a .CW walk message to descend each level in the hierarchy. The .CW stat and .CW wstat messages read and write the attributes of the file referenced by a channel. The .CW open message prepares a channel for subsequent .CW read and .CW write messages to access the contents of the file. .CW Create and .CW remove perform the actions implied by their names on the file referenced by the channel. The .CW clunk message discards a channel without affecting the file. .PP A kernel resident file server called the .I "mount driver" converts the procedural version of 9P into RPCs. The .I mount system call provides a file descriptor, which can be a pipe to a user process or a network connection to a remote machine, to be associated with the mount point. After a mount, operations on the file tree below the mount point are sent as messages to the file server. The mount driver manages buffers, packs and unpacks parameters from messages, and demultiplexes among processes using the file server. .NH 2 Kernel Organization .PP The network code in the kernel is divided into three layers: hardware interface, protocol processing, and program interface. A device driver typically uses streams to connect the two interface layers. Additional stream modules may be pushed on a device to process protocols. Each device driver is a kernel-resident file system. Simple device drivers serve a single level directory containing just a few files; for example, we represent each UART by a data and a control file. .P1 cpu% cd /dev cpu% ls -l eia* --rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia1 --rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia1ctl --rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia2 --rw-rw-rw- t 0 bootes bootes 0 Jul 16 17:28 eia2ctl cpu% .P2 The control file is used to control the device; writing the string .CW b1200 to .CW /dev/eia1ctl sets the line to 1200 baud. .PP Multiplexed devices present a more complex interface structure. For example, the LANCE Ethernet driver serves a two level file tree (Figure 1) providing .IP \(bu device control and configuration .IP \(bu user-level protocols like ARP .IP \(bu diagnostic interfaces for snooping software. .LP The top directory contains a .CW clone file and a directory for each connection, numbered .CW 1 to .CW n . Each connection directory corresponds to an Ethernet packet type. Opening the .CW clone file finds an unused connection directory and opens its .CW ctl file. Reading the control file returns the ASCII connection number; the user process can use this value to construct the name of the proper connection directory. In each connection directory files named .CW ctl , .CW data , .CW stats , and .CW type provide access to the connection. Writing the string .CW "connect 2048" to the .CW ctl file sets the packet type to 2048 and configures the connection to receive all IP packets sent to the machine. Subsequent reads of the file .CW type yield the string .CW 2048 . The .CW data file accesses the media; reading it returns the next packet of the selected type. Writing the file queues a packet for transmission after appending a packet header containing the source address and packet type. The .CW stats file returns ASCII text containing the interface address, packet input/output counts, error statistics, and general information about the state of the interface. .so tree.pout .PP If several connections on an interface are configured for a particular packet type, each receives a copy of the incoming packets. The special packet type .CW -1 selects all packets. Writing the strings .CW promiscuous and .CW connect .CW -1 to the .CW ctl file configures a conversation to receive all packets on the Ethernet. .PP Although the driver interface may seem elaborate, the representation of a device as a set of files using ASCII strings for communication has several advantages. Any mechanism supporting remote access to files immediately allows a remote machine to use our interfaces as gateways. Using ASCII strings to control the interface avoids byte order problems and ensures a uniform representation for devices on the same machine and even allows devices to be accessed remotely. Representing dissimilar devices by the same set of files allows common tools to serve several networks or interfaces. Programs like .CW stty are replaced by .CW echo and shell redirection. .NH 2 Protocol devices .PP Network connections are represented as pseudo-devices called protocol devices. Protocol device drivers exist for the Datakit URP protocol and for each of the Internet IP protocols TCP, UDP, and IL. IL, described below, is a new communication protocol used by Plan 9 for transmitting file system RPC's. All protocol devices look identical so user programs contain no network-specific code. .PP Each protocol device driver serves a directory structure similar to that of the Ethernet driver. The top directory contains a .CW clone file and a directory for each connection numbered .CW 0 to .CW n . Each connection directory contains files to control one connection and to send and receive information. A TCP connection directory looks like this: .P1 cpu% cd /net/tcp/2 cpu% ls -l --rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 ctl --rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 data --rw-rw---- I 0 ehg bootes 0 Jul 13 21:14 listen --r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 local --r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 remote --r--r--r-- I 0 bootes bootes 0 Jul 13 21:14 status cpu% cat local remote status 135.104.9.31 5012 135.104.53.11 564 tcp/2 1 Established connect cpu% .P2 The files .CW local , .CW remote , and .CW status supply information about the state of the connection. The .CW data and .CW ctl files provide access to the process end of the stream implementing the protocol. The .CW listen file is used to accept incoming calls from the network. .PP The following steps establish a connection. .IP 1) The clone device of the appropriate protocol directory is opened to reserve an unused connection. .IP 2) The file descriptor returned by the open points to the .CW ctl file of the new connection. Reading that file descriptor returns an ASCII string containing the connection number. .IP 3) A protocol/network specific ASCII address string is written to the .CW ctl file. .IP 4) The path of the .CW data file is constructed using the connection number. When the .CW data file is opened the connection is established. .LP A process can read and write this file descriptor to send and receive messages from the network. If the process opens the .CW listen file it blocks until an incoming call is received. An address string written to the .CW ctl file before the listen selects the ports or services the process is prepared to accept. When an incoming call is received, the open completes and returns a file descriptor pointing to the .CW ctl file of the new connection. Reading the .CW ctl file yields a connection number used to construct the path of the .CW data file. A connection remains established while any of the files in the connection directory are referenced or until a close is received from the network. .NH 2 Streams .PP A .I stream [Rit84a][Presotto] is a bidirectional channel connecting a physical or pseudo-device to user processes. The user processes insert and remove data at one end of the stream. Kernel processes acting on behalf of a device insert data at the other end. Asynchronous communications channels such as pipes, TCP conversations, Datakit conversations, and RS232 lines are implemented using streams. .PP A stream comprises a linear list of .I "processing modules" . Each module has both an upstream (toward the process) and downstream (toward the device) .I "put routine" . Calling the put routine of the module on either end of the stream inserts data into the stream. Each module calls the succeeding one to send data up or down the stream. .PP An instance of a processing module is represented by a pair of .I queues , one for each direction. The queues point to the put procedures and can be used to queue information traveling along the stream. Some put routines queue data locally and send it along the stream at some later time, either due to a subsequent call or an asynchronous event such as a retransmission timer or a device interrupt. Processing modules create helper kernel processes to provide a context for handling asynchronous events. For example, a helper kernel process awakens periodically to perform any necessary TCP retransmissions. The use of kernel processes instead of serialized run-to-completion service routines differs from the implementation of Unix streams. Unix service routines cannot use any blocking kernel resource and they lack a local long-lived state. Helper kernel processes solve these problems and simplify the stream code. .PP There is no implicit synchronization in our streams. Each processing module must ensure that concurrent processes using the stream are synchronized. This maximizes concurrency but introduces the possibility of deadlock. However, deadlocks are easily avoided by careful programming; to date they have not caused us problems. .PP Information is represented by linked lists of kernel structures called .I blocks . Each block contains a type, some state flags, and pointers to an optional buffer. Block buffers can hold either data or control information, i.e., directives to the processing modules. Blocks and block buffers are dynamically allocated from kernel memory. .NH 3 User Interface .PP A stream is represented at user level as two files, .CW ctl and .CW data . The actual names can be changed by the device driver using the stream, as we saw earlier in the example of the UART driver. The first process to open either file creates the stream automatically. The last close destroys it. Writing to the .CW data file copies the data into kernel blocks and passes them to the downstream put routine of the first processing module. A write of less than 32K is guaranteed to be contained by a single block. Concurrent writes to the same stream are not synchronized, although the 32K block size assures atomic writes for most protocols. The last block written is flagged with a delimiter to alert downstream modules that care about write boundaries. In most cases the first put routine calls the second, the second calls the third, and so on until the data is output. As a consequence, most data is output without context switching. .PP Reading from the .CW data file returns data queued at the top of the stream. The read terminates when the read count is reached or when the end of a delimited block is encountered. A per stream read lock ensures only one process can read from a stream at a time and guarantees that the bytes read were contiguous bytes from the stream. .PP Like UNIX streams [Rit84a], Plan 9 streams can be dynamically configured. The stream system intercepts and interprets the following control blocks: .IP "\f(CWpush\fP \fIname\fR" 15 adds an instance of the processing module .I name to the top of the stream. .IP \f(CWpop\fP 15