Anet Reliable Packet Delivery API

Copyright 1995-2001, Activision

Last updated: 7 May 1998

Dpio: lightweight reliable and unreliable packet delivery

Purpose

The dpio package is not normally used directly by the programmer; it is normally used internally by the DP package. It handles loading and unloading transport DLLs, sending and receiving packets between nodes, retransmitting packets if desired, and timing out inactive nodes. It does not distinguish between different endpoints at a single node.

The term "node" will be used in this document to denote a computer running an Anet game or an Anet server. Other documents may refer to nodes by any of the synonyms "host", "computer", or "machine". They all mean the same thing, since this is a peer-to-peer oriented system. The terms "Master" and "Slave" will be used when there is any asymmetry between the machines.

Where does it fit in?

dpio sits between the transport DLL's and upper level code (such as the dp package or a user program). It was created to provide a tidy wrapper around the transport DLL functions, and to allow the dp package to use reliable packet transmission for its own private system messages. However, it is quite practical to use it and not the rest of the Anet API, if you don't need the player and session management functions. You can even use the full Anet API in your game setup shell, and just the dpio package in your game proper, if you want to keep overhead to a minimum.

The dpio package is a part of the AneAnet library, because it is used by the Anet API.

The dpio package doesn't quite stand on its own; you still use dpEnumTransports(path, callback, context) to list available transport DLL's, and commScanAddr(), commPrintAddr(), and commPlayerInfo() to get access to network addresses and convert them between ASCII and binary form.

Function Reference

For the moment, refer to dpio.h.

Overview

The following paragraphs describe the major functions of the dpio package. All calls to the library are nonblocking; that is, they return within a few hundred microseconds.

Startup and Shutdown

Once a transport DLL has been chosen, the DPIO package is initialized by calling dpio_create(&dpio, transportName, params, &hMaster, &now, NULL) This loads the selected transport DLL, and passes it the given params. (Dpio will use &hMaster and &now as pointers to where it can look for the comm handle of the 'master' and the current return value of clock(). You should update now before each call to most dpio functions. Your code should set hMaster to PLAYER_ME if you are the master, or the comm handle for the master if it is some other node. Yes, this is a bit messy.)

When the user wants to quit the program, the dpio package should be shut down by calling dpio_destroy(dp). This unloads the transport DLL and frees memory.

Opening and closing connections

A connection to another computer is represented by a 32-bit playerHdl_t, known as a 'comm handle' or 'connection handle'.

dpio_openHdl() takes a node's network address, establishes a connection to it, and returns a playerHdl_t. (Note: dpio_get() will call dpio_openHdl() if needed when a new node tries to connect to this node using dpio_openHdl(), so servers may not need to call dpio_openHdl().)

dpio_closeHdl() frees up the connection represented by the given playerHdl_t.

dpio_openHdl2() takes a pair of network addresses, establishes a connection to it and returns a playerHdl_t. A pair of addresses is useful when the underlying network can be multi-homed. The Internet is the good example, in the case where a machine has multiple Ethernet cards, an Ethernet card and a dial-up modem or multiple dial-up modems. By supplying two addresses, dpio will attempt to connect to both addresses, the first address to respond to will be the one which is used. It may be possible to see both addresses on the network but by using the one which responds first, fastest connection will established.

Even if you are only opening a handle to one network address, dpio_openHdl2() can still be used by setting the second network address to NULL.

There are a number of special modifications to winets to make use of two addresses. When two addresses are supplied to winets via dpio_openHdl2(), packets will be sent to both addresses. When a packet is retrieved from one of the addresses, packets will only be sent to that address henceforth. There are only two packet types, which allowed to be sent to multiple addresses, these are SYN and ACK packets, which implies that two addresses can only exist whilst a connection is being established. 

Receiving packets

To receive a packet, set size to sizeof(buffer), then call dpio_get(dpio, &src, flags, buffer, &size). The return value will be dp_RES_OK if a packet was received. In that case, msglen will contain the number of bytes received, and idFrom and idTo will be set to the id's of the sender and recipient.

Sending packets

To send a packet unreliably but quickly to one or more nodes, call dpio_put_unreliable( dpio, &dests[0], nDests, buffer, size, &errDest) where dests is an array of playerHdl_t's you'd like to send the packet to, nDests is how many places you'd like to send it, buffer and size are the data to send and its size, and errDest is either NULL, or the address of a playerHdl_t where the handle of the offending partner will be placed upon error.

To send a packet reliably but slowly to one or more nodes, call dpio_put_reliable( dpio, &dests[0], nDests, buffer, size, &errDest) instead.

Currently, the first byte of buffer must not be ASCII 'd'.

Most game data should be sent unreliably, as it gives the best performance. Packets sent unreliably will only be sent once, might arrive in the wrong order, and might not arrive at all. An upper level protocol should be used to recover from lost packets; for instance, the game should be written to guess reasonable values by interpolation.

Packets sent reliably are guarenteed to be delivered correctly, and in the order originally sent. Only very important non-recurring data should be sent in reliable packets. There is a limited amount of space for buffering outgoing packets (currently, 16 packets' worth). Be sure to check the return value of dpio_put_reliable; if it returns dp_RES_FULL, your packet has not been accepted for transmission, and you need to send it later, after letting your main loop call dpio_get() and dpio_update() for a while to empty the outgoing buffer.

If you send two packets, one reliably and one unreliably, they may arrive in either order.

Reliable mode retransmissions are currently restricted to a small part of available bandwidth. The retransmission algorithm is currently naive.

Timeouts

The only difference between Master and Slave in the dpio package is that the Master times out any node which does not send it packets often enough, whereas the Slaves only time out the Master if it does not send packets often enough. This is how the Anet library likes it. It makes sense because in a normal game, Slaves don't always have anything to say to each other (e.g. when they are on opposite sides of a simulated mountain).

In practice, each node calls dpio_findTimedOutHost() periodically to notice nodes which have gone dead, as mentioned below in Housekeeping. It is the duty of the Master to somehow inform the other nodes of dead players.

Passing an Open Session Between Programs

A program can save the state of a session by calling dpio_freeze(dpio_t *dpio, FILE *fp). To restore the state of the session, pass a file pointer to the same file as the last parameter to dpio_create() instead of NULL.

This makes it possible to create separate lobby programs that set up games, and pass them to the game program proper.

Programmers wanting to avoid the overhead of the full Anet API will be happy to know that sessions saved with dpFreeze(dp, fname) can be restored with dpio_create(). This lets the lobby program use dpFreeze() to pass a session to a game regardless of whether the game uses the full Anet API or just the dpio package. However, the opposite is not the case; a game cannot freeze the state with dpio_freeze() and later restore it with dpCreate(). For example, suppose a custom shell program wishes to launch the game, and then provide a debriefing room after the game is done. If the game uses dpio instead of Anet, and does a dpio_freeze() to save its state before returning control to the shell for debriefing, the shell must use dpio_create() to restore state.

Housekeeping

Housekeeping must be done manually, and includes

All this is normally done inside dp.c, and is only mentioned here for the benefit of programmers writing test programs or games that for some reason want to bypass the Anet API and write straight to the dpio api.

Future enhancements

The retransmission algorithm will be made smarter. A side effect will be that an estimate of the current round-trip latency will be available for each host, which may be quite useful for games.

The keepalive method will probably be rewritten in terms of the reliable transmission stuff.


Dan Kegel, et. al.