The Wine QA Project

Contents

Wine is now at version 0.9.31, and is looking for volunteers to help test it.

Wine QA Tools

Before you start testing Wine, you should know about a few ways you can communicate with the Wine developers:

The Life Cycle of a Bug

The normal life cycle of an individual bug in Bugzilla is described at mozilla.org/bugs. For Wine, it looks something like this:
  1. UNCONFIRMED - somebody has reported a bug
  2. NEW - a sufficiently empowered Wine QA person or developer agrees that it's a good bug report
  3. ASSIGNED - somebody's working on it
  4. RESOLVED - they're done, and the bug is either fixed or has been marked invalid, but no monthly snapshot with the fix has been released yet.
  5. CLOSED - same as RESOLVED, but the fix is now available in a monthly snapshot or release. All RESOLVED bugs should eventually be CLOSED (assuming they really are still fixed).
Note that a bug should not be marked as fixed if the "fix" is to use native DLLs instead of ones that come with Wine. Wine should be able to run any Windows application without relying on nonredistributable Windows DLLs. (Redistributables like MFC40.DLL and the like are another matter -- it's ok to require those, since applications are allowed to bundle them.)

Watching the Bug Parade

It's interesting to see how many bugs are in each stage of the bug lifecycle. Here are the counts as of various dates, and links to click on to get the current count:
Date Unconfirmed New Assigned Resolved Closed
2005.09.30824413714311478
2005.10.08531457527641494
2005.10.14519470608171496
2005.10.22496483558791499
2005.12.15601584682122456
2006.08.011022811609062716

It's even more interesting to see where the action is. Here are the bugs which have been updated in the last 7 days. It's worth browsing through the comments for a few of these.
Date Unconfirmed New Assigned Resolved Closed
2005.09.309888161014
2005.10.082973023034922
2005.10.141028518752
2005.10.22558512695
2005.12.1552559272
2006.08.011131223799

Getting Ready to Test Wine

Here's how to get ready to test Wine:
  1. Read the 5 minute intro to Wine development
  2. Join the wine-devel mailing list
  3. Read the introduction into how to use Bugzilla with Wine
  4. Read WineHQ's tips on bug hunting and support
  5. Create a login account at bugs.winehq.org
  6. Remove any old Wine installation on your machine, and remove the directory ~/.wine if it exists. (It's better to start with a clean slate when testing.)
  7. Install Wine from the sources in git (don't panic!).
    (Usually it's enough to build with the latest snapshot, but using the latest sources from git keeps you from wasting effort in case somebody's been fixing these bugs lately.) This also puts you in good position to try out patches sent to you by Wine developers to see if they fix the bug in question.

    You'll need to have many of your Linux distribution's development tools (e.g. gcc, the openssl and cups headers, ...) installed to do this properly.
    Here's an example of the commands needed to install all the needed development tools, get the Wine source, and build (assuming you're running Ubuntu Dapper):

    $ cd ~
    $ wget kegel.com/wine/dapper.sh
    $ sudo sh dapper.sh
    $ git clone git://source.winehq.org/git/wine.git wine-git
    $ cd wine-git
    $ ./configure 
    $ make depend && make
    
    There's no need to actually install wine; it's easiest to just run it from the directory where it was built.

    You can then run an app using your newly built version of wine with the commands

    $ cd /path/to/where/the/app/is/installed
    $ ~/wine-git/wine app.exe
    
    You probably want to remove your ~/.wine directory before each installation of a Windows app, to make sure your results are repeatable. (The ~/.wine directory is recreated from scratch automatically if it doesn't already exist when you run wine.)

    For instance, if you are trying to test Google's Picasa, you might download and install it with

    $ wget http://dl.google.com/picasa/picasa2-current.exe
    $ rm -rf ~/.wine
    $ ~/wine-git/wine picasa2-current.exe
    
    and run it with
    $ cd ~/.wine/drive_c/Program\ Files/Picasa2
    $ ~/wine-git/wine Picasa2.exe
    
Now you're ready to start testing Wine applications!

What is Triage?

In a way, the Wine project already has enough good bug reports -- they're just mixed in with a lot of bug reports that aren't as useful as they could be. Wine users (not neccessarily developers!) who want to help the project should therefore consider triaging existing problem reports. That will help the Wine developers focus on fixing bugs instead of trying to find good bug reports.

Triage is what medics do when wounded soldiers arrive at a field hospital so fast they can't all be helped. They decide one of three things about each arriving patient, namely: he/she is either

Similarly, triage is also what QA people do when bug reports come in too fast for developers to handle by themselves. They look at each incoming bug report, and decide whether it:

How to help: triage bugzilla problem reports

Here's how to help triage problem reports in Wine's Bugzilla:
  1. Follow the instructions in Getting Ready to Test Wine above to install a fresh copy of Wine from the latest snapshot tarball.
  2. Look through the bug reports to find one that has not had attention from a developer recently, and involves a Windows app you have or can download.
    Here are some canned Bugzilla queries that pull up different flavors of inactive bug reports: Note: When reproducing bugs, please practice safe computing -- don't run programs downloaded from sites you don't trust. The hard drive you save may be your own!
  3. Triage the bug as follows:
    1. If it's an unconfirmed bug, see if you can find a similar bug by searching Bugzilla. If you think it's a duplicate of bug 12345, say something like "Looks like a duplicate of bug 12345" in the Comment box. (The word 'bug' followed by a number gets turned into a handy hyperlink, so people who look at the bug report can click on the bug number to see how that bug was solved.)
      Bug 2950 and bug 3389 are good examples of how to tell somebody they may have filed a duplicate bug.
    2. Try to reproduce the bug on your computer with the recent snapshot of Wine you installed.
      • If there isn't enough information for you to reproduce the problem, add a comment asking him to try the latest Wine and/or provide the missing details.
      • If the app works for you (or at least gets past the bug in question), add a comment saying you can't reproduce the bug, and mention which version of Wine and which version of the app you tried.
      • If you get the same problem as the original poster, add a comment saying you did reproduce the bug, mention which version of Wine and which version of the app you tried, and give an easy-to-follow recipe for reproducing the bug (if there isn't already one there). And congratulations, you've just confirmed the bug!
    3. If you want to get email when someone replies (and you probably do), put your email address in the CC: field in the upper right.
    4. Then press the Commit button.
    5. If the bug is real, check the Wine App Database page for the affected application. If it doesn't already list the bug in question, please log in and add it. That way, users will know which Bugzilla bugs affect each application.
Simply combing through the bug reports and helping get them in shape for developers like this can really help.

How to help: Triage appdb problem reports

The Wine App Database contains many comments from users, some of which are useful descriptions of real bugs. Unfortunately, the Wine developers are unlikely to see them unless they're looking at a particular application.

Here's how to help make these bug reports more useful:

  1. Follow the instructions in Getting Ready to Test Wine above
  2. Look through the Wine App Database for problem reports in user comments that haven't had a good answer
  3. Try to find the problem in Bugzilla.
  4. Triage the bug in Bugzilla as described above
  5. Reply to the original problem report in the App Database and point the user to the URL of the bugzilla bug.

How to help: Triage complaints on wine user forums

There are a number of online forums devoted to users of Wine, e.g. You can help by triaging recent posts in these forums just like described above in "Triage appdb problem reports". Not only will the users appreciate the attention, but by distilling the problems they report and that you can reproduce into concise, easy-to-repeat bug reports, you'll help get the problems fixed faster.

How to help: QA a 'must-have' Windows application

Many organizations would like to switch some of their desktops to Linux, but can't because a few Windows applications don't run on Linux.

Wine is pretty good these days. Chances are, there are only a couple showstopper Wine bugs that keep any particular application from running under Wine. Once these bugs are filed in Wine's Bugzilla, the Wine developers can find and fix them.

You can help by testing one of these apps. (Some of them have free demo versions.)

First check for existing App DB and Bugzilla entries (you don't want to duplicate any existing bug reports). Then file a bug report in bugzilla for each problem you find, taking care to give a minimal recipe for how to repeat the problem, and giving the name of the app in the summary.

Here are a few market segments and some of the "must have" apps that keep each from switching to Linux:

K-12 educational software

I asked the k12osn mailing list which packages they needed Wine to work with; here are the Windows programs they thought were "must-have's" (along with links to their AppDB and Bugzilla entries, if any):

AdobeAcrobat 7AppdbBugzilla
AdobePagemakerAppdb
AdobePhotoshopAppdbBugzilla
AlphasmartAlphabeam
AutoskillAcademy of Reading
AutoskillAcademy of Math
BroderbundKid PixAppdbBugzilla
Harmonic VisionMusic Ace 2
Hunter SystemsSchool Minder
InspirationInspirationAppdb
JASCPaint Shop Pro 7AppdbBugzilla
JASCAnimation Shop 3
Jackson SoftwareGradeQuickAppdb
RenaissanceRead Naturally
RenaissanceAccelerated ReaderAppdb
RenaissanceAccelerated Math
RenaissanceSTAR Reading
RenaissanceSTAR Math
RIM"Blackberry" software
RiverdeepDestination Math
RiverdeepDestination Reading
RiverdeepLearn to Read with Phonics
RiverdeepMavis Beacon Teaches TypingAppdbBugzilla
RiverdeepMighty Math
RiverdeepReader Rabbit
RiverdeepReading Builder
ScholasticReading Counts
ScholasticType to Learn
SunburstHyperstudio
Thomson South-WesternMicroType 4

I have created a tracking bug in Bugzilla, bug 3554, to track the showstopper bugs in the above applications.

Church software

(Note: I'd like to list "city government software", but none of the apps they need are on the market; they're all homegrown VB apps. "Church" is the closest market segment I can find; the church management app I'm testing is a VB app, and is at least a straw man standin for those unobtainable government apps.)

I'm asking the iccm-tech and Linux4Christians mailing lists which Windows applications their churches need to run on Linux.
Church WindowsChurch WindowsBugzilla
Coda MusicFinale 2005AppDBBugzilla

Redistributable Libraries from Language Vendors

Programming languages like Visual C++, Visual Basic, and Visual Foxpro use runtime libraries that are not on all user's computers; the library vendors solve this problem by providing little installers for each set of runtime libraries. (See e.g. Microsoft's web pages "Redistributing Microsoft Visual C++ 6.0 Applications" and Platform SDK Redistributables for more information.) It is important that Wine run the installers for the commonly used runtime libraries properly.

(Some languages come with packaging wizards that package up both the runtime libraries and the user's applications; see e.g. Microsoft's web pages Deploying Complex Microsoft Office Access Runtime-Based Solutions and Packaging Access 2003 Solutions. There are no simple installers just for the runtimes for these languages, so they're not listed in the table below. Not sure what to do about that yet.)

 
AppleQuicktimeAppDBBugzilla
CorelParadox 10 RuntimeAppDB
MicrosoftVisual C++ redistributables
MicrosoftVisual Basic Runtime
MicrosoftMDACBugzilla
MicrosoftJet 4.0
OracleOracle 10g Instant Client

Home Office Software

(Until I have a larger sample of users, here's a sample of one:) The Windows apps my wife (a writer and web developer) would need on Linux include:
Microsoft Office '97AppDBBugzilla
Adobe Photoshop ElementsAppDBBugzilla
Ulead PhotoImpactAppDBBugzilla
Macromedia DreamweaverAppDBBugzilla
Musicmatch JukeboxAppDBBugzilla
FamilyTreeMakerFamilyTreeMakerAppDBBugzilla

See also Novell's list of must-have windows software.

See Also


Last updated: 17 Feb 2007
Copyright 2005, 2006, 2007 Dan Kegel