Remote testing of gcc3

Since I'm building cross-compiling versions of gcc3, I need to be able to do remote testing (where you build on one machine, and run on another). The gcc3 testcases allow this, thanks to dejagnu, a powerful but intricate, confusing, poorly documented, and fragile test framework written in Tcl and Expect.

Since dejagnu's learning curve is steep, I picked the easiest possible remote testing situation: building a non-cross-compiling gcc3 on one x86 redhat box, and running the testcases remotely on another. Once that's working, it ought to be fairly easy to switch to building a cross-compiler and testing on the embedded system of interest.

Dejagnu often misbehaves mysteriously; its error handling is slim. Here are some techniques I had to use to get unstuck:

• Give multiple -v flags to runtest, referring to the dejagnu source code (in /usr/share/dejagnu) as needed to understand the output, and look for problems.
• Add debug prints to the dejagnu source code to help figure out why something isn't working
• Use tcpdump to understand where remote connections were going wrong.

I hope the following recipe helps those who just want to test gcc without having to learn the internals of dejagnu.

Resources

• The Dejagnu Manual
• Niklaus Geiger's rough draft dejagnu tutorial (local copy)
• My notes re Niklaus Geiger's dejagnu tutorial

Simple steps to test gcc on a remote system

In this example, the build machine is called 'krunch'; the machine that will run the binaries is called 'dual'. Warning: this is just a recipe I cooked up from a position of abject ignorance, so it might not be the right way to do things.

1. Reformat the hard drive and install a fresh copy of Linux. (Well, you don't have to, but these instructions should work even if you do.)
2. Install gcc if it isn't already. (It's usually installed by default with Linux.)
3. Install expect and dejagnu, e.g.

   rpm -i /mnt/cdrom/RedHat/RPMS/{expect,dejagnu}*
   

4. Unpack gcc3.0.x source tarball and build gcc, e.g.

   rpm -i /mnt/cdrom/RedHat/SRPMS/gcc3-3.0.1-3.src.rpm
   cd /usr/src/redhat/SPECS
   rpm -bp gcc3.spec
   cd /usr/src/redhat/BUILD/gcc-3.0.1-20010905/obj-i386-redhat-linux
   make
   

5. Verify that you can run the gcc tests locally, e.g.

   cd /usr/src/redhat/BUILD/gcc-3.0.1-20010905/obj-i386-redhat-linux/gcc/testsuite
   runtest --tool=g++
   

6. Configure remote testing machine as follows:
   • Make sure you can telnet, ftp, and rsh to your testing machine from your build machine. If you can, skip the next step.
   • By default, Red Hat doesn't come with ftp or telnet services. Here's one way to install servers for ftp, telnet, and rsh:

     rpm -i /mnt/cdrom/RedHat/RPMS/wu-ftpd-2.6.2-5.i386.rpm
     vi /etc/xinetd.d/wu-ftpd    (and change 'disable=yes' to 'disable=no')
     rpm -i /mnt/cdrom/RedHat/RPMS/telnet-server-0.17-20.i386.rpm
     vi /etc/xinetd.d/telnet    (and change 'disable=yes' to 'disable=no')
     rpm -i /mnt/cdrom/RedHat/RPMS/rsh-server-0.17-5.i386.rpm
     vi /etc/xinetd.d/rsh    (and change 'disable=yes' to 'disable=no')
     /etc/init.d/xinetd restart
     

     Then turn off your firewall, or punch holes through it for telnet and ftp. On RedHat, the basic interactive tool for configuring the firewall is

     /usr/sbin/lokkit
     

   • Enable remote telnet logins by adding the line

     telnet: krunch
     

     to /etc/hosts.allow.
   • Enable remote rsh logins by editing the file /etc/pam.d/rsh to remove the line containing the word 'nologin', and by doing lots of other stuff I haven't figured out yet. (Or by figuring out how to get dejagnu to use telnet instead.)
   • Make sure you can connect using telnet, rsh, and ftp to your testing machine from your build machine. If you can't, try looking in /var/log/messages on the testing machine for permission error messages.
   • Because wu-ftpd appears to run chroot to your home directory (!) these days, and dejagnu will try to upload test executables to /tmp, make a tmp subdirectory in your home directory on the test system, e.g.

     mkdir ~/tmp
     

7. Configure dejagnu for remote testing as follows:
   • Set up ~/site.exp and ~/.dejagnurc:

     echo 'puts "this is ~/site.exp"' > ~/site.exp
     echo 'set target dual' > ~/.dejagnurc
     

   • Create a board file in ~/boards named after the remote system that will be running the tests. For instance, if the testing machine is named 'dual', create ~/boards/dual.exp with the following lines:

     load_generic_config "unix";
     process_multilib_options "";
     set_board_info compiler  "[find_gcc]";
     set_board_info bmk,use_alarm 1;
     set_board_info hostname        "dual"
     set_board_info telnet_username "dank"
     set_board_info telnet_password "xxMyPasswordxx"
     set_board_info shell_prompt    "dank@dual"
     set_board_info file_transfer   "ftp"
     set_board_info ftp_username "dank"
     set_board_info ftp_password "spam,bot"
     

8. Run test cases:

   # Tell dejagnu about ~/site.exp and ~/boards
   export DEJAGNU=/home/dank/site.exp
   # cd to wherever your gcc setup puts its obj files
   cd /usr/src/redhat/BUILD/gcc-3.0.1-20010905/obj-i386-redhat-linux/gcc/testsuite
   # And finally, run the tests of interest.
   runtest --tool g++ --target_board dual