Next: Results Up: NetPIPE Previous: Introduction

NetPIPE Design

NetPIPE consists of two parts: a protocol independent driver, and a protocol specific communication section. The communication section contains the necessary functions to establish a connection, send and receive data, and close a connection. This part is different for each protocol. However, the interface between the driver and protocol module remains the same. Therefore, the driver does not have to be altered in order to change communication protocols.

The driver is based on the principles presented by the HINT [1] computer performance metric(See Appendix A). Just as a computer's performance cannot be accurately described using a single sized computation, neither can the performance of a network be described using a single sized communication transfer. NetPIPE increases the transfer block size k from a single byte until transmission time exceeds 1 second. Hence, NetPIPE is a variable time benchmark and will scale to all network speeds. Unlike fixed size benchmark tests, NetPIPE will not become outdated and inaccurate as technology advances (see Gustafson [6]). To increase the universality of NetPIPE, information is measured in bits rather than bytes. The definition of byte varies more than one might think.

For each block size c, three measurements are taken: c - p bytes, c bytes, and c + p bytes, where p is a perturbation factor with a default value of 3. This perturbation allows analysis of block sizes that are possibly slightly smaller or larger than an internal network buffer. For each measurement, NetPIPE uses the following algorithm:

 /* First set T to a very large time. */
   T = MAXTIME
   For i = 1 to NTRIALS

  		t0 = Time()
   		For j = 1 to nrepeat
   		    if I am transmitter
   		        Send data block of size c
   		 		 Recv data block of size c
   		 	 else
   		 		 Recv data block of size c
   		 		 Send data block of size c
   		 	 endif
   		endFor
   		t1 = Time()
   /* Insure we keep the shortest trial time. */
	  T = MIN(T, t1p;t0)

   endFor
	T = T/(2 * nrepeat)

The variable nrepeat is calculated based on the time of the last data transfer. The intent is to repeat the experiment enough times such that the total time for the experiment is far greater than timer resolution. The default target time is 0.5 seconds. For most modern computers, this provides a sufficiently precise data transfer time. Given that the last transfer time was tlast seconds for a block size bsz1, the value of nrepeat for block size bsz2 is approximated as:

nrepeat = TARGET/((bsz2/bsz1)*tlast)

NetPIPE uses a ping-pong transfer like Hockney [11] uses for each block size. This forces the network to transmit just the data block without streaming other data blocks in with the message. The result is the transfer time of a single block, thus providing the information necessary to answer which block size is best, or what is the throughput given a block of size k..

NetPIPE produces a file that contains the transfer time, throughput, block size, and transfer time variance for each data point and is easily plotted by any graphing package. For instance, Figure 1 presents the throughput versus the transfer block size for a typical Ethernet link. This graph is referred to as the throughput graph. From this graph, it is easy to see the maximum throughput for this network is approximately 7.8 Mbps. However, it is difficult to analyze the latency, an equally important statistic.

Figure 1: Ethernet Throughput

A graph that is easier to read and analyze is the network signature graph. One such graph is shown in Figure 2. It depicts the transfer speed versus the elapsed time; hence it represents a network ``acceleration'' graph. This graph is a new and unique way of viewing network performance data; the key is to use a logarithmic time scale horizontally instead of the transfer block size. In this graph, as in all graphs presented, time is plotted in seconds. It is very similar to the way computer performance is presented by the HINT performance metric. Although unconventional, this graph represents perhaps a better approach to visualizing network performance. All the necessary data are clearly visible and easy to extrapolate. The network latency coincides with the time of the first data point on the graph. The maximum attainable throughput is clearly shown as the maximum point on the graph.

Figure 2: Ethernet Signature Graph

Plotting the block size versus the transfer time on a logarithmic scale for both the x and y axis, as in Figure 3, reveals what we define as the saturation point. This is the point after which an increase in block size results in a near-linear increase in transfer time, effectively the knee of the curve. The time interval between the saturation point and the end of the recorded data is referred to as the saturation interval. In this interval, the graph monotonically increases at a constant rate i.e., the network throughput cannot be improved upon by increasing the block size.

Figure 3: Ethernet Saturation Graph

Using maximum effective bandwidth to compare networks (or even worse, nominal bandwidth) is much like using peak megahertz ratings to compare computers. While it may be correct for ranking certain applications, in general, its accuracy leaves much to be desired. A given network may have a high maximum effective bandwidth but also have a high latency. So a network with a lower latency would possibly be better for small messages even though it has a much lower maximum effective bandwidth. This effect can be readily observed when comparing ATM with Ethernet, as shown below. Any ranking based on a single number does not provide sufficient insight for accurate network comparison. For network tuning and comparison, we recommend taking the entire NetPIPE signature graph together with application specific information into consideration.

Next: Results Up: NetPIPE Previous: Introduction

Quinn Snell
Fri Apr 5 12:14:32 CST 1996