Software Engineering with Wine

Contents

Introduction

This document describes a series of projects suitable for use in an undergraduate Software Engineering course, designed to give students a modicum of real-world experience.

This page is being reorganized. The 2009 version of this page is here.

Slides prepared for UCLA's CS130 are online: Winter 2009, Winter 2007, magicpoint source, Winter 2006, html/ Winter 2006, magicpoint source.

Mailing list

The mailing list ucla-cs130-wine is set up for students currently enrolled in the class.

Lei's Documentation

Lei Zhang has set up a great web page at linux.ucla.edu/~leiz/software/wine/cs130_w07.php. Go check it out.

The Wine Project

The projects are centered around the Wine Project, an ambitious effort to create a clean-room, open source, and free reimplementation of the Win32 API on top of the Unix API. The Wine developers use publically available Microsoft documentation, together with the observed behavior of Windows, to implement each Windows API call and a compliance test for those calls. The compliance test suite is crucial, as without it, adding features to or even fixing bugs in Wine without breaking it would be extremely difficult.

Many companies and governments are currently planning to start using Linux on the desktop, but use several legacy Windows applications that are not yet available for Linux. The Wine project could be an important part of these organizations' transition plans. For instance, Munich is converting about 14000 computers from Windows to Linux by 2008, and will use VMWare, Terminal Servers, or Wine when Linux versions of applications are unavailable.

By contributing improvements to Wine, students will help reduce software costs for millions of people around the world.

Working Environment

Students should have access to both Linux and Windows XP (either by using two computers, or by dual booting). The student should not need root or administrator power to build or run Wine.

Don't use an old versions of Linux (e.g. Red Hat 9, Fedora Core 1); it's best to use a fairly recent mainstream, e.g. Ubuntu 6.06 or Ubuntu 6.10. Also, use the 32 bit version of the OS; Wine does not yet play well with 64 bit software. The Linux system should have the full suite of GNU developer tools (gcc, binutils, etc.) as well as git installed. The wiki page RecommendedPackages describes how to add the needed packages to the Linux system. The easiest way to do this is to run my canned script, e.g. if you're using Ubuntu 6.06, download dapper.sh and run it like this: 

wget http://kegel.com/wine/dapper.sh
sudo sh dapper.sh
Or if you're using Ubuntu 6.10, do: 
wget http://kegel.com/wine/edgy.sh
sudo sh edgy.sh
(Thanks to Lei for updating edgy.sh.)

The Windows system should have either Cygwin's C compiler (freely downloadable from cygwin.com) or the Microsoft C compiler and Platform SDK installed. The latter are freely downloadable from microsoft.com: first grab Visual C++ Express, then install Platform SDK.

If the school does not provide a lab set up with such computers, the instructor may have to provide the students with Linux CD-ROMs and have them spend the first project setting up their own computers. This should be coordinated with the school's Linux User Group, which may be able to host an Installfest to assist the students.

The students will also need reasonably fast Internet connections. This document will assume everyone has cable modem, DSL, or T1 access. (Students without fast access can also participate, though the procedures will be a bit different, and will not be covered here.)

Read the Fine Manual

www.winehq.org has a fair amount of documentation. You should at least have a look at the following:

Get and Build Wine on Linux

Assuming one has a reasonably fast Internet connection, it's easy to use git to download the wine sources. See winehq's git page or winehq's wiki's git page for more info. Git has some rough edges, but once you learn it, it's great. (It's kind of like the vi of source code control systems.)

To download Wine from git, do: 

git clone git://source.winehq.org/git/wine.git wine-git
This takes a couple minutes if you have a fast connection. (It even works on Windows under Cygwin.)

To build Wine, just use configure and make. (You can give the command 'git pull' before making to be sure you have the very latest wine sources.) 

cd ~/wine-git
git pull
./configure
make depend
make
Note: If you're using ubuntu 6.10, you may need to disable a C compiler option when building wine: 
CFLAGS=-fno-stack-protector ./configure

If you have a ~/.wine directory left over from an older version of Wine, you should delete or rename it (since Wine is still changing rapidly, it often breaks backwards compatibility with old ~/.wine contents): 

cd ~
mv .wine .wine.old
(If you aren't running X already, e.g. if you're running Wine under VMWare and/or your copy of linux isn't set to use graphics on bootup, start X now with the command startx.)

Once that's done, try running some of the programs that come with Wine, e.g. 

cd ~/wine-git
./wine programs/notepad/notepad.exe.so

If notepad works, try running winefile, e.g. 

programs/winefile/winefile
That should give you an idea of how Wine maps drive letters.

Then try installing and running a real Windows program using Wine. One good program to try is WinZip. Download the latest version, save the file in your home directory, then install it with Wine like this: 

cd ~
wine-git/wine winzip100.exe
and then run it, e.g. 
wine-git/wine 'c:\Program Files\WinZip\WINZIP32.EXE'
to make sure it works as you'd expect.

Troubleshooting

You may see, when you quit winzip, that a tiny window is left on the screen, and a unix process keeps running an instance of wine. I suspect wine is showing a winzip window that is supposed to be hidden, and for some reason doesn't get cleaned up. To see it, try 
ps augxw | grep wine
To kill it (and all other wine processes), try 
killall wine-preloader
or, if you actually installed Wine (not recommended while developing it), 
wineserver -k

How To Submit Changes To Wine

Once you have made a change to Wine or its test suite that you're happy with, create a patch (see here and here for how), and make sure the patch applies cleanly to a fresh checkout of the Wine sources, and still builds and runs properly. If you used git to retrieve the sources, you should be able to use 'git diff > my.patch' to create a patch.

Once your patch works, add a changelog entry at the top, giving your name and the license (LGPL), and saying very briefly what it does. Then have a friend review the patch for errors, and clean up anything they found confusing or wrong.

If you're a student working in a structured project, send it to your project leader for review, and clean up any issues they find.

Finally, post it to the wine-patches mailing list, and watch for replies on the wine-devel mailing list.

If the patch isn't accepted right away, don't worry, just address any feedback you're given, and resubmit it once a week or so. Ask on wine-devel if you're unsure what's happening. Continue resubmitting until it makes it in. (Sometimes the Wine maintainer's mailbox overflows, so don't expect approval to be quick or easy.)

Project idea: Write a Conformance Test

As of this writing, there are many DLLs that lack a conformance test. Read Chapter 5. Writing Conformance tests in the Wine Developers Guide if you haven't already, then pick a DLL that doesn't have a tests subdirectory yet, and write a simple conformance test that exercizes at least some of its basic functions. (Here's an example.) This can take as little as a day to write, for a simple DLL, or could be a monster project. Try to keep the test very simple; the goal is to not spend more than a couple days on the first version.

Note: do not include any third-party code (e.g. from Microsoft) in your tests! You have to write everything from scratch yourself (or get the author's permission to contribute his/her code to Wine under the LGPL), otherwise the Wine project won't be able to accept your code!

If the DLL has many functions, that's ok, just test the most important or most commonly used ones. Later, you or others can add tests for more of its functions.

If you need a data file, create it during the run, as the test must be fully self-contained. Make sure the test runs properly on at least Windows XP, and preferably also on Windows 98, before running it under Wine.

I find it easiest to compile the test standalone on Wine using Microsoft's C++ toolkit using commands like (assuming I'm trying to build the 'file.c' conformance test for msvcrt): 

cd dlls/msvcrt/tests
cl -DSTANDALONE -D_X86_ -I../../../include file.c
This lets me compile the test once, and run it on both Wine and Windows.

Once it runs properly on both Windows and Wine, copy it into the tests subdirectory of the DLL in question, add the neccessary changes to configure.ac and Makefile.in (see the lzexpand test patch for an example), run autoconf to regenerate configure from configure.ac, and make sure your test builds properly.

If the DLL you're testing is nontrivial, you probably want to plan on submitting an initial version that only tests a subset of the DLL. While you're waiting for that to be accepted, continue adding coverage to your tests. Getting a partial test into the Wine source tree can sometimes make it easier and quicker to get the full test in later.

For UCLA cs130 Winter 2007 term, see Lei Zhang's great web page at linux.ucla.edu/~leiz/software/wine/cs130_w07.php for the specific DLL we plan to work on.

Project idea: Implement A Missing Feature in RichEd20

Many small Windows applications use RichEd20.dll to display a rich text editor window, but at the moment, Wine's implementation of RichEd20 doesn't implement all the features needed by these programs. See Tommy Kho's "riched20: rich edit control in Wine" page for a list of features used by real programs but missing from Wine's implementation (now somewhat dated). Try implementing one of these features.

Project idea: Implement A Missing Feature in msxml3

Many Windows applications use msxml3 to parse XML files, but at the moment, Wine's implementation of msxml3 doesn't implement all the features needed by these programs. See for instance Try implementing one of these features.

Project idea: Take Care of a Janitorial Project

http://wiki.winehq.org/TodoList lists a few chores that need taking care of, e.g. Try doing a little bit of that cleanup work, and submitting your changes. The wine developers will love you...

Optional: Run The Conformance Test Suite under Wine

To run the entire conformance test suite under Linux, follow the instructions at www.winehq.org/site/docs/winedev-guide/testing-wine. For example: 
cd ~/wine-git
make -k test  > log 2>&1
(The -k makes it continue past failing tests; the "2>&1 log" redirects both stdout and stderr to a file named "log".) This should take about five minutes. If a test crashes, you may need to click 'cancel' to make the crash dialog box go away.

To see a list of failures, just look for lines with "Test failed" in them, e.g. 

grep "Test failed" log | more
You can get a count of failures with wc: 
grep "Test failed" log | wc

or a breakdown by test source file with sed, uniq, and sort (I admit, I love the Unix shell!): 

grep "Test failed" log | sed 's/:.*//' | uniq -c | sort
On my machine, I got the following breakdown of errors in 2004: 
      2 typelib.c
      4 capture.c
      4 wave.c
      7 vartype.c
If you see more errors than this, something's probably wrong, and you need to troubleshoot a bit before going on.

Troubleshooting the Wine tests

If you see lots of errors of the form 
metafile.c:91: Test failed: pass 0: dx[0] (1079638356) didn't match 6
it probably means you don't have any TrueType fonts installed, and Wine didn't install any for you by default. To fix this, install a TrueType font, e.g. download and run http://prdownloads.sourceforge.net/corefonts/courie32.exe.

2004 Harvey Mudd College Wine clinic postmortem

I worked with four students at HMC in 2004-2005 to enhance LCOV for use with Wine and then use it to improve Wine's test coverage. Some patches did get accepted: but a bunch more didn't: I feel I picked a difficult approach, and the students would have been more effective working on Wine more directly rather than working on a somewhat arcane tool.

2005 UCLA Software Engineering with Wine postmortem

Since the time scale of the UCLA course was so short (three months), I asked the students to pick a DLL that lacked good conformance tests, and implement or improve the tests. The focus on testing wasn't bad, and some tests were committed: However, since only one student managed to get code into CVS, I clearly was still doing something wrong.

2006 UCLA Software Engineering with Wine postmortem

This year's "software engineering with wine" project at UCLA went rather more smoothly than last year's. I think what made the difference is: but maybe Linux and Wine are just in better shape this year, too.

This year, everybody was asked to implement a windows message currently unimplemented by riched20, starting with the conformance tests. The unofficial TA forged ahead and wrote an initial test suite for riched20 that the students could add to; he then went and implemented a couple features himself. It took much persistance and many rewrites, but his patches made it into winehq.org's CVS in time:

Six of the seven students have had changes accepted into the Wine tree during the school term: A few other patches were committed in July and August, thanks to Matt Finnicum and others dusting them off and reposting them with updates (the students who wrote them probably could have gotten them in with a bit more persistence): Two more changes are still pending as of 1 Sept 2006 (and probably need rewriting):