Notes on Richard Stevens' "Unix Network Programming, Volume 1"

We owe W. Richard Stevens quite a debt; his books are a great resource for Unix programmers.

Resources at W. Richard Stevens' site

www.kohala.com -- W. Richard Stevens' home page (and shrine)
www.kohala.com/start/unpv12e.html -- Home page for "Unix Network Programming, Volume 1"
www.kohala.com/start/unpv12e/unpv12e.tar.gz -- Source code for "Unix Network Programming, Volume 1"
www.kohala.com/start/unpv12e.problems.html -- Reported problems building the above source.

Related Patches

Here are some patches and scripts I've put together which add support for Linux to some of the book's source code (notably to the benchmarks in chapter 27).

They can be downloaded as a tarball named unpv12ep (with a trailing p for 'patches', to distinguish it from the original):

unpv12ep-0.2.tar.gz

or as individual files:

unpv12e.tar.gz -- Richar Stevens' original tarball
build.sh -- shell script to apply all patches and build (well, build the part I'm interested in, anyway)
test.sh -- shell script to run server benchmarks to reproduce Stevens' figure 27.1
dodiff.sh -- shell script used to make patches by comparing book's original source code to local tree
clean.patch -- "make clean" and "make distclean" clean up better (useful when making patches)
pktinfo.patch -- struct in_pktinfo from system headers now used if available
servmake.patch -- build serv00 and serv09 by default
servrusage.patch -- work around broken getrusage on Linux
setpshared.patch -- programs that require _POSIX_THREAD_PROCESS_SHARED now fail at runtime rather than compile time on systems where that is not supported

Usage

To more or less reproduce figure 27.1 of "Unix Network Programming, Volume 1, 2nd edition" on Linux, download the above tarball, and do:

tar -xzvf unpv12ep-0.2.tar.gz
cd unpv12ep-0.2
sh build.sh
cd unpv12e/server
sh ../../test.sh PORT NITER
grep time *.out

where PORT is some port outside your local ephemeral IP port range, and NITER is just less than your local ephemeral IP port range divided by 10.

For instance, if /proc/sys/net/ipv4/ip_local_port_range is

  1024	32767

you'd want to set ITERS to 3000, and PORTNUM to e.g. 55000. You may want to increase your local port range to [1024,52000] so you can achieve longer run times and more accuracy.

Preliminary results

The results listed below appear to be repeatable to about 10% accuracy.

On a Pentium III, 450 MHz, running Red Hat 6.2, client running on same machine as server, with 32000 local ports available, running

test.sh 3000 53000

yields

serv00: user time = 0.5, sys time = 5.0
serv01: user time = 3.7, sys time = 17.8
serv02: user time = 0.6, sys time = 7.1
serv03: user time = 0.8, sys time = 7.8
serv04: N/A (Linux doesn't support pthread_mutexattr_setpshared())
serv05: user time = 0.9, sys time = 6.4
serv06: N/A (Linux doesn't support process CPU time accounting for short-lived threads)
serv07: user time = 0.8, sys time = 7.1
serv08: user time = 1.0, sys time = 5.5
serv09: user time = 0.7, sys time = 7.3

On a dual Pentium III 650 MHz, booted with a non-SMP Red Hat 7.1, client running on same machine as server, with 51000 local ports available, running

test.sh 5000 53000

yields

serv00: user time = 0.3, sys time = 3.8
serv01: user time = 8.3, sys time = 35.1
serv02: user time = 0.5, sys time = 4.8
serv03: user time = 0.4, sys time = 5.5
serv04: N/A (Linux doesn't support pthread_mutexattr_setpshared())
serv05: user time = 0.8, sys time = 6.2
serv06: N/A (Linux doesn't support process CPU time accounting for short-lived threads)
serv07: user time = 0.9, sys time = 5.0
serv08: user time = 1.0, sys time = 5.5
serv09: user time = 0.6, sys time = 4.8

Same machine, same conditions except booted with a SMP red hat 7.1 kernel:

serv00: user time = 0.3, sys time = 8.2
serv01: user time = 9.9, sys time = 56.8
serv02: user time = 0.8, sys time = 9.9
serv03: user time = 0.6, sys time = 10.9
serv04: N/A (Linux doesn't support pthread_mutexattr_setpshared())
serv05: user time = 0.9, sys time = 11.9
serv06: N/A (Linux doesn't support process CPU time accounting for short-lived threads)
serv07: user time = 1.0, sys time = 10.5
serv08: user time = 1.2, sys time = 11.0
serv09: user time = 0.5, sys time = 9.8

Issues

The SMP kernel has noticable overhead. I suppose that's expected.
Linux has two pthreads compatibility problems that prevented me from running two of the programs:
- Linux does not support pthread_mutexattr_setpshared() yet.
- Under Linux, getrusage(USAGE_CHILDREN, buf) does not yet report time spent by child threads created with pthread_create(). I was able to work around this for the servers that create long-lived threads with a kludge that reads child threads' cpu time from /proc (see getrusage_children.c), but this won't work well for short-lived threads (e.g. as used by program serv06.c).
The above timings were measured with both client and server on a single machine, instead of running the clients on two separate machines, as in Stevens. This means the test can't run as long, as it runs out of local port numbers faster. I don't have a script for running client on separate machines yet.
Output not normalized or presented the same way as in the book (I don't sum user and sys, and I don't subtract serv00's time from all results)
Haven't verified that my patches don't break Solaris, BSD, or OSF/1 compatibility yet
Stevens' example server/client.c uses gethostbyname() heavily; if you're not careful, this can make the program run very slowly.

Last updated: 22 Aug 2001