Anet exception handler
Last updated: 8 February 1998
The Anet Exception Handler, or crash logger for short, is a convenient
way for game programmers to collect information about crashes and warnings
logged by dpReportCrash() experienced by QA testers, beta testers,
or even end users.
Note: all the following regarding crashes also hold true for warnings
logged by dpReportAssertionFailure().
Note: dpReportAssertionFailure() is currently stubbed out.
When a program using it crashes, the helper app sendcrsh.exe is run,
which prompts the user for a comment, then uploads the crash description to
a central server.
The developer can then download the crash description from the central server
and use the program analcrsh.exe to display the call
stack a the time of the crash.
The tester can wait until the end of the day to send the file,
because each crash appends to the file. After the file is sent, the tester
should delete it.
Because it is a pain to constantly be emailing crash files to the
developer, the Anet libray will automatically upload the log file the
next time the user connects to an Anet game server. If the upload
succeeds, the log file is deleted, to prevent duplication.
Binary and text files for crashes received by Anet game servers
are displayed in table format at
http://iserv.activision.com/anet2/merglogs along with a count of the total
number of crashes recorded in the given file.
Currently, the crash logger only supports Windows 95 and Windows NT 5.0.
If run under Windows NT 4.0, it will not crash, but many of its methods
will just return an error code.
How to Use It
How to Call dpReportAssertionFailure()
Redefine your asserts to something similar to DP_ASSERT in
anet.h
#define DP_ASSERT(exp) \
(void)((exp) || dpReportAssertionFailure(__LINE__, __FILE__, #exp), assert(exp), exit(1), 0)
In your exception filter, check if the exception return code is
aeh_ASSERTION_CODE (defined in anet.h);
if yes, return EXCEPTION_CONTINUE_EXECUTION from your filter. If this check
is not done, all dpReportAssertionFailure() calls will be fatal.
How to Call dpReportCrash()
From a Win32 exception handler, call
the Win32 function GetExceptionInformation() and pass its result to
dpReportCrash(LPEXCEPTION_POINTERS pException).
This creates a record describing the crash, including the stack trace,
info from the game's anet.inf, and the user's graphics driver name and version
number. The record is appended to the file atvilog.bin on the central
server.
For example,
#ifndef _DEBUG
LONG __cdecl Debug_ExceptionFilter(LPEXCEPTION_POINTERS pException)
{
/* your own stuff */
dpReportCrash(pException);
/* more of your own stuff */
/* include the following 2 lines if using dpReportAssertionFailure or
* if using activenet stuff since activenet uses it */
if (pException->ExceptionRecord->ExceptionCode == aeh_ASSERTION_CODE)
return (EXCEPTION_CONTINUE_EXECUTION);
return EXCEPTION_CONTINUE_SEARCH; /* or EXCEPTION_EXECUTE_HANDLER etc */
}
#endif
/* A wrapper around the original WinMain */
int PASCAL WinMain(HANDLE hInstance, HANDLE hPrevInstance, LPSTR lpszCmdParam, int nCmdShow)
{
#ifndef _DEBUG
__try {
#endif
return oldWinMain(hInstance, hPrevInstance, lpszCmdParam, nCmdShow);
#ifndef _DEBUG
} __except (Debug_ExceptionFilter(GetExceptionInformation())) {
/* your exception handling code; should restore display mode. */
}
#endif
}
For a complete example, see the chat demo application in the chat directory
(chat.c and a batch file to compile it).
Compile and run time specifics
- Compile your code without enabling frame pointer omission. If you're
compiling with /O2 optimization to maximize speed, you need to add /Oy- option
after it. In Visual C 5.0, you turn on stack frames in Project / Settings
/ C/C++ / Category:Optimization as follows: choose Optimizations:Customize,
then scroll down to the bottom of the list and make sure "frame pointer
omission" is not checked.
Note: you should verify that turning off this optimization does not
decrease your game's frame rate. If it does, you may need to turn this
optimization on for a few critical modules.
- When linking .exe's or .dll's, be sure to generate a mapfile.
This is needed later to get the function names for the symbolic stack dump.
- Set up a Mapfile Directory as described below.
- Test your crash handler by setting fault=1 in
dp.ini. This should cause an exception when you call dpCreate.
- Find the resulting atvilog.bin; it should be appended after approximately
one minute to the file on the central server (currently
aeh.activision.com at
http://aeh.activision.com/anet2/atvilog.bin).
- Run c:\maps\analcrsh.exe -e atvilog.bin
This will produce a human readable crash log in atvilog.txt. Your crash
should be one of the last in the file.
- If you are looking for a crash that was uploaded some time ago, you can
search for the comment you entered in the sendcrsh.exe dialog box.
The Mapfile Directory
Games using the crash logger put numerical info about the crash into
atvilog.bin, and you use the program analcrsh.exe to convert this to
a human-readable form showing the crash address.
To get the names of the functions involved in the crash, you need
the executables and dll's that were involved in the crash, and their
.mapfiles. Because you might need to interpret crashes from several
releases of a game, all with slightly different versions of the .exe's and
.dll's, analcrsh supports a Mapfile Directory scheme that lets
.mapfiles for different versions of the same program coexist.
Here's how to set one up:
- Make a directory named c:\maps, and copy the programs modcrc.exe and
analcrsh.exe there. You might even want to put c:\maps on your
PATH, by adding the line PATH %path%;c:\maps in your AUTOEXEC.BAT.
- Underneath c:\maps, create another directory for each release of your program,
and copy the .dll's, .exe's, and their .mapfiles into that directory.
The name of this directory doesn't matter;
modcrc.exe will look in all subdirectories
of c:\maps to find the right .mapfile for you.
A good choice for the name might be the build number or the date.
(You need to repeat this step every time you ship a new build to your
testers.)
- Run c:\maps\modcrc.exe. This creates a catalog of the contents of c:\maps
needed by analcrsh.exe. (You need to repeat this step whenever you add new
versions of your .dll .exe .mapfiles to c:\maps.)
Programs Used to Manipulate Mapfiles and Crash Files
The following three programs are used to manipulate mapfiles and crash files.
They should be copied from win\bin into to a mapfile directory where you
keep all the .mapfiles from the programs you want to analyse crash dumps for.
Modcrc.exe: Mapfile Catalog Builder
Modcrc.exe should be run every time you add a new mapfile/executable to the
mapfile directory.
It assumes that it is located in the mapfile directory.
It maintains a catalog of .mapfiles for use by analcrsh.exe.
The catalog is kept in the file modcrc.txt, also in the mapfile directory.
It contains one line for each mapfile found in the mapfile directory
or any subdirectory thereof, but only if the corresponding executable or
dll has the same name and is in the same directory as the mapfile.
It prints a brief usage message if invoked with command-line argument -h.
Analcrsh.exe: Crash File Analyser
The program analcrsh.exe reads the crash file and the mapfile catalog
modcrc.txt (in the mapfile directory), finds and reads the mapfiles for
the .dll's and .exe involved in the crash, looks up the function names
in the call stack, and outputs an ASCII file.
It prints a brief usage message if invoked with command-line argument -h.
Procserv.exe: Crash File Compacter
The program procserv.exe compacts binary crash files, and filters and sorts
crash records according to command-line arguments
This is generally only needed when combining crash files from different
machines, or on crash files downloaded from the game server.
It prints a brief usage message if invoked with command-line argument -h.
Example
You can try out the exception handler using the files in anetsdk.zip as
follows.
- Make a Mapfile Directory
See Building a Mapfile Directory above.
- Create the directory c:\maps and copy these files into it:
- win/bin/anetdrop.exe
- win/bin/modcrc.exe
- win/bin/analcrsh.exe
- then create a subdirectory in c:\maps using the date of anetsdk.zip
(eg. c:\maps\00-01-01) and copy these files into it:
- win\bin\*.*
- win\bin\chat\*.*
- win\dll\*.*
- In c:\maps, run modcrc.exe.
This will create a mapfile catalog named modcrc.txt,
which should look something like this:
1cf85718 00-01-01\anet2.map
97610581 00-01-01\anet2d.map
4e52497c 00-01-01\winets2.map
f43b0762 00-01-01\winets2d.map
b0f25ab6 00-01-01\gtest2dd.map
976d5f0b 00-01-01\gtest2dr.map
5d2c19d4 00-01-01\anetdrop.map
1a984d63 00-01-01\anetdropd.map
93f49896 00-01-01\chat.map
49d46fe4 00-01-01\chatd.map
- Add c:\maps to your path
by running PATH %path%;c:\maps , or, if you hate command
lines, set Windows Explorer to open .bin files with c:\maps\analcrsh.exe.
- Make a directory somewhere, e.g. \demo, and copy these files into it:
- win/bin/anetdrop.exe
- win/bin/chat/anet.inf
- win/bin/chat/chat.exe
- win/dll/anet2.dll
- win/dll/wipx2.dll
- In \demo, run "anetdrop -n=wipx2.dll -h".
This runs anetdrop.exe which reads anet.inf and launches chat.exe.
In the chat program, type Ctrl-A.
Chat will call dpReportAssertionFailure() which should
create the crash file atvilog.bin (usually in c:\windows\temp on Windows 95
machines; you can look for it if you like but it's not necessary for you to
know where it is, unless you're going to email it to somebody). The assertion
warning is recorded and chat's exception handler allows the program to continue
executing.
Still in the chat program, type Ctrl-X.
Chat should try to divide by zero, and crash.
On its way to the grave, chat will call dpReportCrash(), which should
append the crash record to atvilog.bin.
- Run analcrsh.exe. This will read the crash file (atvilog.bin, wherever
it is) that was created by dpReportCrash, and write the file atvilog.txt
into the current directory. atvilog.txt should look something like this:
******* Crash stat *******
Number of cases: 1
excDesc: assertion failure, retAddr:bff9a07c
assert: line 299, file chat.c, text ASSERT!
SessType: 65, Plat: 1, Lang: 1, Vers: 0.9
System Description:
ATI Graphics Pro Turbo PCI (mach64): atim64.drv (4.10.0.1555)
Call stack:
abs_addr mod_off func_off
bff9a07c 0 0 (kernel) (kernel) (unknown)
1000d77f d77f 2f ANET2.DLL _dpReportAssertionFailure 00-01-01
401879 1879 39 CHAT.EXE _main 00-01-01
401aec 1aec fc CHAT.EXE _mainCRTStartup 00-01-01
******* Crash stat *******
Number of cases: 1
excDesc: Integer divide by zero, retFlag:0, retAddr:4014ee
SessType: 65, Plat: 1, Lang: 1, Vers: 0.9
System Description:
ATI Graphics Pro Turbo PCI (mach64): atim64.drv (4.10.0.1555)
Call stack:
abs_addr mod_off func_off
4014ee 14ee 21e CHAT.EXE @chat@8 00-01-01
401879 1879 39 CHAT.EXE _main 00-01-01
401aec 1aec fc CHAT.EXE _mainCRTStartup 00-01-01
This is read as follows:
- Number of cases - number of instances of this crash in atvilog.bin.
If you re-run anetdrop.exe as before and cause another crash, then run analcrsh,
you will see this number becomes 2.
- excDesc, retFlag, retAddr - this is info about the crash; excDesc
describes the exception, retFlag is a Microsoft defined variable and retAddr
is the address where the crash occurred.
- SessType, Plat, Lang, Vers - this is information from the anet.inf file of
the crashing application for the session type, platform, language
and version number.
- System Description - this contains graphics driver info and 3D card info
for the machine where the crash occurred.
- Call stack - this is the stack trace for the crash. The first column
if the absolute address; the second is the address offset into the module; the
third is the address offset into the function; the fourth is the module name;
the fifth is the function name; and the last is the mapfile directory
containing the module used. If any of the values in the first three columns
can't be determined, 0 is put into that column; similarly, if any of the
values in the last three columns can't be determined, unknown or kernel is
placed there.
Internal Implementation Details
For hardcore hackers only. Most game programmers won't need to read this.
Refer to aeh.h and
aehlog.h for function definitions.
The crash handling/analyzing process can be divided up into three parts:
- one that directly uses the crash data (aeh.c)
- one that transfers the crash data to/from file (aehlog.c)
- one that transfers the crash data to/from network (dpexcept.c)
The latter two ignore the actual contents of the data so it's the first part
that does most of the work in getting the data during the crash and in
analyzing it afterwards.
What dpReportCrash calls:
- aehCreate - designed to be called by an exception handler, it returns
information about the crash included exception code, stack trace, and modules
involved in the crash in an aeh_t structure that is used by all other functions.
Inputs to aehCreate include information about the application that crashed
(gotten for instance by a call to dpReadAnetInf) and information about the
user's system (gotten by a call to getDispProfileInfo in getdisp.c).
- aeh_writeOutputStream - converts information in aeh_t structure to binary
form that can be used by file or network operations
- aehlog_writeExceptionRecord - takes the information in binary form and
writes it to file.
What the crash analyzer calls:
- aehlog_readExceptionRecord - reads binary info on one crash from file
- aeh_readInputStream - converts information in binary form to aeh_t
structure
- aeh_getAllInfo - this gets the module functions involved in a crash using
the crash information already available along with information from the
mapfiles for the modules involved in the crash.
- aeh_toString - if you're too lazy to format the info in aeh_t, this will
print the info out in an ASCII string
What current versions of anet do internally to send a packet to the game
server, and what happens afterwards:
- When dpReportCrash() is called, the helper application sendcrsh.exe is
run, and the crash description and central server hostname are passed on the
command line.
- sendcrsh prompts the user for a comment. When the user clicks OK or hits
enter, it connects to the game server specified on the command line
(currently aeh.activision.com), and uploads the crash description via
TCP.
- The game server appends the record to atvilog.bin within a minute. This
can be downloaded from the central server (currently at
http://aeh.activision.com/anet2/atvilog.bin).
- Nightly, the central server will copy the crash files it received
that day to a file [date].bin; these can be downloaded using a web browser at
[server_URL]/anet2/atvilog,
eg.
http://aeh.activision.com/anet2/atvilog.
iserv.activision.com grabs these daily .bin files from each game server
and merges them together. It then uses the merged files to create daily
.bin files for each session type as well as a merged .bin file covering all
dates for each session type. The log files are converted to text format also.
What old versions of anet do internally to send a packet to the game
server, and what happens afterwards:
- When dpSetGameServer() is called on a game server, dp_publishExceptions()
is called which reads in a specified number of bytes from the exception log
file and uses the info to set a dptab entry in the exception table as well as
setting the game server as a subscriber. On the other hand, when the game
server is notified of a new host, it subscribes to that host's exception table
and when it receives a dptab exception entry, it stores the data in its own
exception log file, deleting the table entry afterwards.
- Nightly, each Anet game server will copy the crash files it received
that day to a file [date].bin; these can be downloaded using a web browser at
[server_URL]/anet2/atvilog,
eg.
http://california3.activision.com/anet2/atvilog.
A local report server grabs these daily .bin files from each game server
and merges them together. It then uses the merged files to create daily
.bin files for each session type as well as a merged .bin file covering all
dates for each session type. The log files are converted to text format also.
Localizing the Helper App
The helper app gets its dialog box description, including strings and layout,
from a windows resource compiled into sendcrsh.exe. If you have access to the
source, this is at src/win/sendcrsh/sendbox.rc. Since windows can retrieve
different resources based on the current language, we should be able to just
keep adding languages to sendbox.rc, and end up with a sendcrsh.exe which has
all languages compiled into it, and can be shipped anywhere in the world.