\chapter{Background}\label{chap:background} \ifpdf	\graphicspath{{Chapter1/Chapter1Figs/PDF/}{Chapter1/Chapter1Figs/PNG/}		{Chapter1/Chapter1Figs/}}\else	\graphicspath{{Chapter1/Chapter1Figs/EPS/}{Chapter1/Chapter1Figs/}}\fi There have been work going on to open Blue Gene for more generic applications,and Plan 9 has been ported to Blue Gene\cite{Van07} as part of that effort.This has lead to exploration of alternatives for workload distribution on thisplatform. Following are the concepts/work that inspired our exploration.  We have borrowedmany concepts from these works and used them to meet the requirements of dataflow applications.  \section{Plan 9}Plan 9 is a distributed operating system which is able to simplify development ofdistributed applications. The Plan 9 design is based on three primary design principles.\begin{enumerate}	\item \textbf{Everything is file} : Plan 9 presents everything as file. This includesdevices and the processor itself.  All these resources are accessible in a hierarchicalnamespace.	\item \textbf{9P: Standard communication protocol} All files are accessed using the9P protocol. This allows accessing both local and remote resources in same way.	\item \textbf{Namespaces} Each process has it's own namespace view.  This namespaceview can be separately built for every process, allowing every process to havea different view of the world.\cite{namespace}\end{enumerate} As everything in Plan 9 is a file, it is easy to share everything as file over network.  It alsoallows transparently sharing its resources as files over network.  The 9P protocolremoves the differentiation between local and remote resources by providing the sameinterface for accessing both of them.  Private namespaces allow a process to buildits own namespace using any resources anywhere over the network.  These features allowarranging any remote resources in any view and using them transparently.  In the followingsections, we will see how these concepts can be used to simplify the deploymentof applications on remote nodes.  \subsection{CPU}CPU\cite{plan9-cpu} is Plan 9's concept for using remote compute resources.  It is different from traditional rlogin, telnet or ssh.  Most of these protocols work by logging in on a remote node and providing the environment of the remote node to user.Only the local terminal is used while all other resources like the processor, disk and filesystem are from the remote node. In contrast, CPU works by bringing remote resources to the user.  Figure 2.1 presentsCPU.\begin{figure}[h]  \begin{center}    \leavevmode    \ifpdf      \includegraphics[height=0.4\textheight,width=0.6\textwidth]		{CPU}    \fi    \caption{CPU}    \label{fig:CPU}  \end{center}\end{figure} It works by creating new private namespace\cite{namespace} with alocal filesystem view, disk, network, console, etc on the client side.  This namespace is mounted by the remote processor and used for all the processes started there as part of the CPU session.  These processes have the view of the client system while the execution is still performed on a remote node.  One of the keyadvantages of this model is that now users don't have to worry about the environmentof the remote node when running any job on it.  Any execution requested on a remotenode will see the same local view of the user's environment, simplifying thedeployment of applications. Even though CPU does simplify the life of the developer, it does not solve all ofour problems.  Firstly, it is Plan 9 specific and does not work with any otheroperating system.  This keeps it out of realm of most of the users.Another limitation is that CPU works in one-to-one fashion.  This limits theusefulness of it in cluster like environments.  Controlling each and every nodein a cluster individually does not scale well for larger numbers of nodes. \section{XCPU}XCPU\cite{ron-xcpu} \cite{lucho-xcpu} was an attempt to bring the functionalityof CPU into mainstream operating systems like Linux.  It aims at providing theremote resources using a \textbf{filesystem interface} and improve thescalability.  XCPU can be seen as the remote process executionsystem presenting execution and control services in the form of files ina hierarchical filesystem.  This filesystem can be exported over the network so thatother nodes can use it.  As XCPU presents remote compute resources as files,deployment of applications does not need any special privileges as long as user can read/write these files. XCPU has two types of nodes, the controller node which is responsible forstarting and controlling the job and compute nodes which actually do the computations. In addition to execution of the program, XCPU is also responsible for distributing them to remote nodes. XCPU differs from CPU in that it does not use the private namespaces to create local environments on remote nodes.  It uses the \textbf{push model} and automatically pushes the executable and all the dependencies of it to the remote node.  The dependencies are analyzed using static analysis of the executable requested by the user. To achieve scalable pushing of programs and their dependencies, XCPU uses a fewcompute nodes for distributing the programs and dependencies during the programstartup time.  This is known as the \textbf{tree spawn mechanism} and it improves thescalability for XCPU to larger numbers of nodes.  Figure 2.2 presents atypical tree hierarchy used by XCPU for better scalability. \begin{figure}[h]  \begin{center}    \leavevmode    \ifpdf      \includegraphics[height=0.3\textheight,width=0.6\textwidth]		{xcpu-tspawn}    \fi    \caption{XCPU tree spawn mechanism}    \label{fig:XCPU}  \end{center}\end{figure} With automatic pushing and the tree spawn mechanism, XCPU not only simplifies the deploymentof applications but it also scales them better with quicker job startup time.  \subsection{Drawbacks}One major drawback of XCPU is the result of the push model.  Alldependencies are needed to be pushed to compute nodes.  The difficulty liesin detecting the dependencies of the requested executable.  Static analysis isnot capable of detecting dependencies like configuration files which are knownonly at runtime.  The user needs to explicitly provide information about thesedependencies so that XCPU can push them.  XCPU is designed for executing one application on the cluster at one time.This is not very useful when multiple users want to use different partsof the cluster at the same time.  XCPU does not aim for such situations, leadingto limited flexibility.  XCPU has properties like quick job startup time and a simple filesysteminterface which makes it a good first step towards scalable dataflow applicationdeployment.  But there is still a lot of gap to cover for meeting all the requirementsof dataflow applications.  \section{XCPU2}XCPU2 \cite{lucho-xcpu2} is evolved from XCPU and is aimed to provide more flexibility.  XCPU2 was inspired by the use of \textbf{private namespace} of CPU.  Recently the Linux kernel has added support for private namespace\cite{linuxns}, and this facilitated thedevelopmentof XCPU2.  The much needed flexibility is provided by the ability of building a separateview of the filesystem for each process using these private namespaces. XCPU2 works by dividing the nodes into three categories.  Each type is responsiblefor different functionality.\begin{enumerate}	\item \textbf{A Control Node} is responsible for handling \textbf{reservation} requests for compute nodes.There will  be a single control node per cluster for this role.  XCPU2 does not take responsibility of reservation.  This part is typically delegatedto some other software which accepts the reservation requests and returns thelist of nodes assigned for reservation request. 	\item \textbf{A Job Control Node} is responsible for \textbf{managing a single job}.The nodes provided by the control node for reservation requests are then managed by a job control node which is responsible for starting, stopping the job and manipulatingthe filesystem view of compute nodes.  Users will typically interact with thesejob control nodes for running their applications. 	\item \textbf{A Compute Node} is responsible for \textbf{actual computation}.These nodes communicate with the job control nodes and accepts the application namesthat are to be executed.  These nodes also allow job control node to modify thefilesystem view of process executing the application by using private namespaces. \end{enumerate} A user typically starts by requesting the reservation with the control node.  Once thereservation is done, the user will get an access to the job control node which can be usedto manage the job.  The user can export his local filesystem view to the job control nodewith the name of the executable to execute.  All compute nodes mount the user's localfilesystem via the job control node.  All compute nodes then create processes withthe namespace modified in such a way that it will inherit the user's filesystem viewinstead of the compute node's filesystem view.  This way, processes running on a compute nodewill see the same uniform view as processes in the user environment.  In contrast toXCPU which pushes the executables and dependencies, XCPU2 uses the \textbf{pull model}where any dependency is automatically pulled because of the inherited user filesystemview using private namespace. This propagation of the user's environment on the compute nodes simplifies the applicationdeployment to a great extent.  If an application is properly running on the user environment, thenit will also run on the compute nodes as they have the same filesystem view.  This isa big improvement over other environments where developers are forced to developthe applications for an environment of the compute nodes which may differ significantly from the user environment.  Also, developers can use any libraries or customized tools within a distributed application without worrying much about availability of thosein the compute environment.  Users neither need special privileges nor do they need to makespecial requests for installing special libraries and modified tools on every nodesof the cluster.  This model also simplifies the maintenance of the cluster as there isno need to maintain every tool and every version of the libraries on each compute node. \begin{figure}[h]  \begin{center}    \leavevmode    \ifpdf      \includegraphics[height=0.3\textheight,width=0.6\textwidth]		{xcpu2}    \fi    \caption{XCPU2 in action}    \label{fig:XCPU2}  \end{center}\end{figure} The diagram 2.3 shows how multiple users can use the cluster at the same time for executingdifferent applications with contradicting requirements on the filesystem view.  The control nodeprovides the partitioned set of compute nodes with one job control node to eachuser requested reservation.  Now each user can execute its own applicationwith  it's own view of filesystem without any interference from other users whoare running their applications at the same time on the same cluster.  This provides addedflexibility and better utilization of resources. XCPU2 also inherits scalability from XCPU by using the tree spawning mechanism in computenodes within a single reservation.  This hierarchical structure reduces the loadon the job control node and provides better job-startup time.  XCPU2 is definitely a forward step toward providing the flexibility needed to deploydataflow applications.  It provides flexibility in controlling the namespace view and it allows multiple users to run their applications simultaneously with goodjob startup time. \subsection{drawbacks} The fundamental drawback with XCPU2 is the granularity of control.The control provided by the XCPU2 on the filesystem view of compute nodes is not finegrained.  Users can control the filesystem view of all nodes together, but theycan not control each and every compute node individually. This lack of control on every node is a major deterrent for dataflow applications.As these applications can be best scheduled by treating them as DAG and mappingeach computational vertex to one compute node.  As each computational vertex canhave different requirements about environment and filesystem view, control ofeach compute node is important for successfully mapping dataflow applicationon compute resources.  \textit{XCPU2 fails to provide this much needed fine grained controlon all nodes}. This limitation was major motivation for us to develop XCPU3 which can handleall needs of dataflow applications. % ------------------------------------------------------------------------%%% Local Variables: %%% mode: latex%%% TeX-master: "../thesis"%%% End: