#+TITLE:     Graph 500 Benchmark 1 ("Search")#+AUTHOR:    Graph 500 Steering Committee#+EMAIL:     sc@graph500.org#+LANGUAGE:  en#+OPTIONS:   H:3 num:t toc:t \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t#+OPTIONS:   TeX:t LaTeX:t skip:nil d:nil todo:t pri:nil tags:not-in-toc#+STYLE: <style>body {margin-left: 10%; margin-right: 10%;} table {margin-left:auto; margin-right:auto;}</style> Contributors: David A. Bader (Georgia Institute of Technology),Jonathan Berry (Sandia National Laboratories), Simon Kahan (PacificNorthwest National Laboratory and University of Washington), RichardMurphy (Sandia National Laboratories), E. Jason Riedy (GeorgiaInstitute of Technology), and Jeremiah Willcock (Indiana University). Version History: - V0.1 :: Draft, created 28 July 2010 - V0.2 :: Draft, created 29 September 2010 - V0.3 :: Draft, created 30 September 2010 - V1.0 :: Created 1 October 2010 - V1.1 :: Created 3 October 2010 Version 0.1 of this document was part of the Graph 500 communitybenchmark effort, led by Richard Murphy (Sandia NationalLaboratories).  The intent is that there will be at least threevariants of implementations, on shared memory and threaded systems, ondistributed memory clusters, and on external memory map-reduceclouds. This specification is for the first of potentially severalbenchmark problems. References: "Introducing the Graph 500," Richard C. Murphy, KyleB. Wheeler, Brian W. Barrett, James A. Ang, Cray User’s Group (CUG),May 5, 2010. "DFS: A Simple to Write Yet Difficult to Execute Benchmark," RichardC. Murphy, Jonathan Berry, William McLendon, Bruce Hendrickson,Douglas Gregor, Andrew Lumsdaine, IEEE International Symposium onWorkload Characterizations 2006 (IISWC06), San Jose, CA, 25-27 October2006. * Brief Description of the Graph 500 Benchmark Data-intensive supercomputer applications are an increasinglyimportant workload, but are ill-suited for platforms designed for 3Dphysics simulations. Application performance cannot be improvedwithout a meaningful benchmark. Graphs are a core part of mostanalytics workloads. Backed by a steering committee of 30international HPC experts from academia, industry, and nationallaboratories, this specification establishes a large-scale benchmarkfor these applications. It will offer a forum for the community andprovide a rallying point for data-intensive supercomputingproblems. This is the first serious approach to augment the Top 500with data-intensive applications. The intent of this benchmark problem ("Search") is to develop acompact application that has multiple analysis techniques (multiplekernels) accessing a single data structure representing a weighted,undirected graph. In addition to a kernel to construct the graph fromthe input tuple list, there is one additional computationalkernel to operate on the graph. This benchmark includes a scalable data generator which produces edgetuples containing the start vertex and end vertex for eachedge. The first kernel constructs an /undirected/ graph in a formatusable by all subsequent kernels. No subsequent modifications arepermitted to benefit specific kernels. The second kernel performs abreadth-first search of the graph. Both kernels are timed. There are five problem classes defined by their input size:  - toy :: 17GB or around 10^10 bytes, which we also call level 10,  - mini :: 140GB (10^11 bytes, level 11),  - small :: 1TB (10^12 bytes, level 12),  - medium :: 17TB (10^13 bytes, level 13),  - large :: 140TB (10^14 bytes, level 14), and  - huge :: 1.1PB (10^15 bytes, level 15). Table \ref{tbl:classes} provides the parameters used by the graphgenerator specified below. ** References D.A. Bader, J. Feo, J. Gilbert, J. Kepner, D. Koester, E. Loh,K. Madduri, W. Mann, Theresa Meuse, HPCS Scalable Synthetic CompactApplications #2 Graph Analysis (SSCA#2 v2.2 Specification), 5September 2007. * Overall benchmark The benchmark performs the following steps:   1. Generate the edge list.  2. Construct a graph from the edge list (*timed*, kernel 1).  3. Randomly sample 64 unique search keys with degree at least one,     not counting self-loops.  4. For each search key:     1. Compute the parent array (*timed*, kernel 2).     2. Validate that the parent array is a correct BFS search tree        for the given search tree.  5. Compute and output performance information. Only the sections marked as *timed* are included in the performanceinformation.  Note that all uses of "random" permit pseudorandomnumber generation. * Generating the edge list ** Brief Description The scalable data generator will construct a list of edge tuplescontaining vertex identifiers. Each edge is undirected with itsendpoints given in the tuple as StartVertex and EndVertex.  The intent of the first kernel below is to convert a list with nolocality into a more optimized form.  The generated list of inputtuples must not exhibit any locality that can be exploited by thecomputational kernels.  Thus, the vertex numbers must be randomizedand a random ordering of tuples must be presented to kernel 1.The data generator may be parallelized, but the vertex namesmust be globally consistent and care must be taken to minimize effectsof data locality at the processor level. ** Detailed Text Description The edge tuples will have the form <StartVertex, EndVertex> whereStartVertex is one endpoint vertex label and EndVertex is theother endpoint vertex label.  The space of labels is the set of integersbeginning with *zero* up to but not including the number of vertices N(defined below).  The kernels are not provided the size N explicitlybut must discover it. The input values required to describe the graph are:  - SCALE :: The logarithm base two of the number of vertices.  - edgefactor :: The ratio of the graph's edge count to its vertex count (i.e.,                 half the average degree of a vertex in the graph). These inputs determine the graph's size:  - N :: the total number of vertices, 2^SCALE. An implementation may   use any set of N distinct integers to number the vertices, but at   least 48 bits must be allocated per vertex number. Other parameters   may be assumed to fit within the natural word of the machine. N is   derived from the problem’s scaling parameter.  - M :: the number of edges. M = edgefactor * N. The graph generator is a Kronecker generator similar to the RecursiveMATrix (R-MAT) scale-free graph generation algorithm [Chakrabarti, etal., 2004]. For ease of discussion, the description of this R-MATgenerator uses an adjacency matrix data structure; however,implementations may use any alternate approach that outputs theequivalent list of edge tuples. This model recursively sub-divides theadjacency matrix of the graph into four equal-sized partitions anddistributes edges within these partitions with unequalprobabilities. Initially, the adjacency matrix is empty, and edges areadded one at a time. Each edge chooses one of the four partitions withprobabilities A, B, C, and D, respectively.  These probabilities, theinitiator parameters, are provided in Table \ref{tbl:initiator}. #+CAPTION: Initiator parameters for the Kronecker graph generator#+LABEL:   tbl:initiator| A = 0.57 | B = 0.19             || C = 0.19 | D = 1-(A+B+C) = 0.05 | The next section details a high-level implementation for thisgenerator.  High-performance, parallel implementations are included inthe reference implementation. The graph generator creates a small number of multiple edges betweentwo vertices as well as self-loops. Multiple edges, self-loops, andisolated vertices may be ignored in the subsequent kernels but mustbe included in the edge list provided to the first kernel. Thealgorithm also generates the data tuples with high degrees oflocality. Thus, as a final step, vertex numbers must be randomlypermuted, and then the edge tuples randomly shuffled. It is permissible to run the data generator in parallel. In this case,it is necessary to ensure that the vertices are named globally, andthat the generated data does not possess any locality, either in local memoryor globally across processors. The scalable data generator should be run before starting kernel 1, storing itsresults to either RAM or disk.If stored to disk, the data may be retrieved beforestarting kernel 1. The data generator and retrieval operations need not betimed. ** Sample high-level implementation of the Kronecker generator The GNU Octave routine in Algorithm \ref{alg:generator} is anattractive implementation in that it is embarrassingly parallel anddoes not require the explicit formation of the adjacency matrix. #+CAPTION: High-level generator code#+LABEL: alg:generator#+BEGIN_SRC Octavefunction ij = kronecker_generator (SCALE, edgefactor)%% Generate an edgelist according to the Graph500%% parameters.  In this sample, the edge list is%% returned in an array with two rows, where StartVertex%% is first row and EndVertex is the second.  The vertex%% labels start at zero.%%%% Example, creating a sparse matrix for viewing:%%   ij = kronecker_generator (10, 16);%%   G = sparse (ij(1,:)+1, ij(2,:)+1, ones (1, size (ij, 2)));%%   spy (G);%% The spy plot should appear fairly dense. Any locality%% is removed by the final permutations.   %% Set number of vertices.  N = 2^SCALE;   %% Set number of edges.  M = edgefactor * N;   %% Set initiator probabilities.  [A, B, C] = deal (0.57, 0.19, 0.19);   %% Create index arrays.  ij = ones (2, M);  %% Loop over each order of bit.  ab = A + B;  c_norm = C/(1 - (A + B));  a_norm = A/(A + B);   for ib = 1:SCALE,    %% Compare with probabilities and set bits of indices.    ii_bit = rand (1, M) > ab;    jj_bit = rand (1, M) > ( c_norm * ii_bit + a_norm * not (ii_bit) );    ij = ij + 2^(ib-1) * [ii_bit; jj_bit];  end   %% Permute vertex labels  p = randperm (N);  ij = p(ij);   %% Permute the edge list  p = randperm (M);  ij = ij(:, p);   %% Adjust to zero-based labels.  ij = ij - 1;#+END_SRC ** Parameter settings The input parameter settings for each class are given in Table \ref{tbl:classes}. #+CAPTION: Problem class definitions and required storage for the edge list assuming 64-bit integers.#+LABEL: tbl:classes| Problem class     | SCALE | edge factor | Approx. storage size in TB ||-------------------+-------+-------------+----------------------------|| Toy (level 10)    |    26 |          16 |                     0.0172 || Mini (level 11)   |    29 |          16 |                     0.1374 || Small (level 12)  |    32 |          16 |                     1.0995 || Medium (level 13) |    36 |          16 |                    17.5922 || Large (level 14)  |    39 |          16 |                   140.7375 || Huge (level 15)   |    42 |          16 |                  1125.8999 |#+TBLFM: $4=2^$2*$3*2*8/1e12;%.4f ** References D. Chakrabarti, Y. Zhan, and C. Faloutsos, R-MAT: A recursive modelfor graph mining, SIAM Data Mining 2004. Section 17.6, Algorithms in C (third edition). Part 5 GraphAlgorithms, Robert Sedgewick (Programs 17.7 and 17.8) P. Sanders, Random Permutations on Distributed, External andHierarchical Memory, Information Processing Letters 67 (1988) pp305-309. * Kernel 1 – Graph Construction ** Description The first kernel may transform the edge list to any data structures(held in internal or external memory) that are used for the remainingkernels. For instance, kernel 1 may construct a (sparse) graph from alist of tuples; each tuple contains endpoint vertex identifiers for anedge, and a weight that represents data assigned to the edge. #+ XXX: Unsure about this part.  I'd rather permit structural#+ optimizations here so long as they're used for *all* kernels.#+   - However, any layout or arrangement computations intended to#+    increase locality, improve load-balance, or reduce hot-spotting#+    must be counted in the benchmark execution time.  Therefore, /no#+    comparison of or binning by vertex degrees is permitted during this#+    kernel 1 (graph construction)/. The graph may be represented in any manner, but it may not be modifiedby or between subsequent kernels. Space may be reserved in the datastructure for marking or locking.Only one copy of a kernel will berun at a time; that kernel has exclusive access to any such marking orlocking space and is permitted to modify that space (only). There are various internal memory representations for sparse graphs,including (but not limited to) sparse matrices and (multi-level)linked lists. For the purposes of this application, the kernel isprovided only the edge list and the edge list's size.  Furtherinformation such as the number of vertices must be computed within thiskernel.  Algorithm \ref{alg:kernel1} provides a high-level sampleimplementation of kernel 1. The process of constructing the graph data structure (in internal orexternal memory) from the set of tuples must be timed. #+CAPTION: High-level implementation of kernel 1#+LABEL: alg:kernel1#+BEGIN_SRC Octavefunction G = kernel_1 (ij)%% Compute a sparse adjacency matrix representation%% of the graph with edges from ij.   %% Remove self-edges.  ij(:, ij(1,:) == ij(2,:)) = [];  %% Adjust away from zero labels.  ij = ij + 1;  %% Find the maximum label for sizing.  N = max (max (ij));  %% Create the matrix, ensuring it is square.  G = sparse (ij(1,:), ij(2,:), ones (1, size (ij, 2)), N, N);  %% Symmetrize to model an undirected graph.  G = spones (G + G.');#+END_SRC ** References Section 17.6 Algorithms in C third edition Part 5 Graph Algorithms,Robert Sedgewick (Program 17.9) * Sampling 64 search keys The search keys must be randomly sampled from the vertices in thegraph.  To avoid trivial searches, sample only from vertices that areconnected to some other vertex.  Their degrees, not counting self-loops,must be at least one.  If there are fewer than 64 such vertices, runfewer than 64 searches.  This should never occur with the graph sizesin this benchmark, but there is a non-zero probability of producing atrivial or nearly trivial graph.  The number of search keys used isincluded in the output, but this step is untimed. * Kernel 2 – Breadth-First Search ** Description A Breadth-First Search (BFS) of a graph starts with a single sourcevertex, then, in phases, finds and labels its neighbors, then theneighbors of its neighbors, etc.  This is a fundamental method onwhich many graph algorithms are based. A formal description of BFS canbe found in Cormen, Leiserson, and Rivest.  Below, we specify theinput and output for a BFS benchmark, and we impose some constraintson the computation.  However, we do not constrain the choice of BFSalgorithm itself, as long as it produces a correct BFS tree as output. This benchmark's memory access pattern (internal or external) is data-dependentwith small average prefetch depth.  As in a simpleconcurrent linked-list traversal benchmark, performance reflects anarchitecture's throughput when executing concurrent threads, each oflow memory concurrency and high memory reference density.  Unlike sucha benchmark, this one also measures resilience to hot-spotting whenmany of the memory references are to the same location; efficiencywhen every thread's execution path depends on the asynchronousside-effects of others; and the ability to dynamically load balanceunpredictably sized work units.  Measuring synchronization performanceis not a primary goal here. You may not search from multiple search keys concurrently. *ALGORITHM NOTE* We allow a benign race condition when vertices at BFSlevel k are discovering vertices at level k+1.  Specifically, we donot require synchronization to ensure that the first visitor mustbecome the parent while locking out subsequent visitors.  As long asthe discovered BFS tree is correct at the end, the algorithm isconsidered to be correct. ** Kernel 2 Output For each search key, the routine must return an array containing validbreadth-first search parent information (per vertex).  The parent ofthe search_key is itself, and the parent of any vertex not included inthe tree is -1.  Algorithm \ref{alg:kernel2} provides a sample (andinefficient) high-level implementation of kernel two. #+CAPTION: High-level implementation of kernel 2#+LABEL: alg:kernel2#+BEGIN_SRC Octavefunction parent = kernel_2 (G, root)%% Compute a sparse adjacency matrix representation%% of the graph with edges from ij.   N = size (G, 1);  %% Adjust from zero labels.  root = root + 1;  parent = zeros (N, 1);  parent (root) = root;   vlist = zeros (N, 1);  vlist(1) = root;  lastk = 1;  for k = 1:N,    v = vlist(k);    if v == 0, break; end    [I,J,V] = find (G(:, v));    nxt = I(parent(I) == 0);    parent(nxt) = v;    vlist(lastk + (1:length (nxt))) = nxt;    lastk = lastk + length (nxt);  end   %% Adjust to zero labels.  parent = parent - 1; #+END_SRC * Validation It is not intended that the results of full-scale runs of thisbenchmark can be validated by exact comparison to a standard referenceresult. At full scale, the data set is enormous, and its exact detailsdepend on the pseudo-random number generator and BFS algorithm used. Therefore,thevalidation of an implementation of the benchmark uses soft checking ofthe results. We emphasize that the intent of this benchmark is to exercise thesealgorithms on the largest data sets that will fit on machines beingevaluated. However, for debugging purposes it may be desirable to runon small data sets, and it may be desirable to verify parallel resultsagainst serial results, or even against results from the executablespecification. The executable specification verifies its results by comparing themwith results computed directly from the tuple list. Kernel 2 validation: after each search, run (but do not time) afunction that ensures that the discovered breadth-first tree iscorrect by ensuring that:   1) the BFS tree is a tree and does not contain cycles,  2) each tree edge connects vertices whose BFS levels differ by     exactly one,  3) every edge in the input list has vertices with levels that differ     by at most one or that both are not in the BFS tree,  4) the BFS tree spans an entire connected component's vertices, and  5) a node and its parent are joined by an edge of the original graph. Algorithm \ref{alg:validate} shows a sample validation routine. #+CAPTION: High-level implementation of kernel 2 validation#+LABEL: alg:validate#+BEGIN_SRC Octavefunction out = validate (parent, ij, search_key)  out = 1;  parent = parent + 1;  search_key = search_key + 1;   if parent (search_key) != search_key,    out = 0;    return;  end   ij = ij + 1;  N = max (max (ij));  slice = find (parent > 0);   level = zeros (size (parent));  level (slice) = 1;  P = parent (slice);  mask = P != search_key;  k = 0;  while any (mask),    level(slice(mask)) = level(slice(mask)) + 1;    P = parent (P);    mask = P != search_key;    k = k + 1;    if k > N,      %% There must be a cycle in the tree.      out = -3;      return;    end  end   lij = level (ij);  neither_in = lij(1,:) == 0 & lij(2,:) == 0;  both_in = lij(1,:) > 0 & lij(2,:) > 0;  if any (not (neither_in | both_in)),    out = -4;    return  end  respects_tree_level = abs (lij(1,:) - lij(2,:)) <= 1;  if any (not (neither_in | respects_tree_level)),    out = -5;    return  end#+END_SRC * Computing and outputting performance information

Lines 1–500 of 673. Load more