Jeff Harris

Software engineer, hobbyist game developer

A talk about RESTful DOOM

I spoke about RESTful-DOOM at an API meetup a couple of weeks ago!

RESTful-DOOM embeds an API into the classic 1993 first person shooter game, allowing the it to be queried and controlled using HTTP and JSON. More details here.


RESTful DOOM

TL;DR I embedded a RESTful API into the classic 1993 game DOOM, allowing the game to be queried and controlled using HTTP and JSON.

“We fully expect to be the number one cause of decreased productivity in businesses around the world.”

   - ID Software press release (1993).


1993

1993 was an exciting year - Sleepless in Seattle opened in theatres, Microsoft shipped Windows NT 3.1, and Whitney Houston’s ‘I Will Always Love You’ was the best selling song for 2 straight months. Oh, and a game called Doom was released!

Doom was created by a small team at ID Software. Wikipedia describes it as one of the most significant and influential titles in video game history, and growing up I loved playing it. As an adult I couldn’t put down a book called Masters of DOOM, which describes the back story of ID Software.

ID Software has a super cool practice of releasing source code for their games. For the kind of hackers who lurk on /r/gamedev, an ID Software engine is an amazing resource to learn from. And lo, in 1997, the Doom engine source code was released, causing much happiness!

2017

I was having trouble finding a fun API to use in a talk I had to do. I had spent the normal amount of time procrastinating and stressing about having to give the talk, and wasn’t making any progress on building a compelling demo.

Late one night, out of the blue, I had the idea to create an API for Doom, now 24 years old(!), and obviously never designed to have an API. I could have some fun digging around the Doom source code and solve my API problem at the same time!

My random idea became RESTful-DOOM - a version of Doom which really does host a RESTful API! The API allows you to query and manipulate various game objects with standard HTTP requests as the game runs.

There were a few challenges:

  • Build an HTTP+JSON RESTful API server in C.
  • Run the server code inside the Doom engine, without breaking the game loop.
  • Figure out what kinds of things we can manipulate in the game world, and how to interact with them in memory to achieve the desired effect!

I choose chocolate-doom as the base Doom code to build on top of. I like this project because it aims to stick as close to the original experience as possible, while making it easy to compile and run on modern systems.

Hosting an HTTP API server inside Doom

chocolate-doom already uses SDL, so I added an -apiport <port> command line arg and used SDLNet_TCP_Open to open a TCP listen socket on startup. Servicing client connections while the game is running is a bit trickier, because the game must continue to update and render the world many times a second, without delay. We must not make any blocking network calls.

The first change I made was to edit D_ProcessEvents (the Doom main loop), to add a call to our new API servicing method API_RunIO. This calls SDLNet_TCP_Accept which accepts a new client, or immediately returns NULL if there are no clients.
If we have a new client, we add its socket to a SocketSet by calling SDLNet_TCP_AddSocket. Being part of a SocketSet allows us to use the non-blocking SDLNet_CheckSockets every tic to determine if there is data available.
If we do have data, API_ParseRequest attempts to parse the data as an HTTP request, using basic C string functions. I used cJSON and yuarel libraries to parse JSON and URI strings respectively.

Routing an HTTP request involves looking at the method and path, then calling the right implementation for the requested action. Below is a snippet from the API_RouteRequest method:

if (strcmp(path, "api/player") == 0)
{
    if (strcmp(method, "PATCH") == 0) 
    {
        return API_PatchPlayer(json_body);
    }
    else if (strcmp(method, "GET") == 0)
    {
        return API_GetPlayer();
    }
    else if (strcmp(method, "DELETE") == 0) {
        return API_DeletePlayer();
    }
    return API_CreateErrorResponse(405, "Method not allowed");
}

Each action implementation (for example API_PatchPlayer) returns an api_response_t containing a status code and JSON response body.

Putting it all together, this is what the call graph looks like when handling a request for PATCH /api/player:

D_ProcessEvents();
  API_RunIO();
    SDLNet_CheckSockets();
    SDLNet_TCP_Recv();
    API_ParseRequest();
    API_RouteRequest();
      API_PatchPlayer();
    API_SendResponse();

Interfacing with Doom entities

Building an API into a game not designed for it is actually quite easy when the game is written in straight C. There are no private fields or class hierarchies to deal with. And the extern keyword makes it easy to reference global Doom variables in our API handling code, even if it feels a bit dirty ;)

cJSON library is used to generate the JSON formatted response data from API calls.

We want the API to provide access to the current map, map objects (scenery, powerups, monsters), doors, and the player. To do these things, we must understand how the Doom engine handles them.

The current episode and map are stored as global int variables. By updating these values, then calling the existing G_DeferedInitNew, we can trigger Doom to switch smoothly to any map and episode we like.

Map objects (mobj_t) implement both scenery items and monsters. I added an id field which gets initialized to a unique value for each new object. This is the id used in the API for routes like /api/world/objects/:id.

To create a new map object, we call the existing P_SpawnMobj with a position and type. This returns us an mobj_t* that we can update with other properties from the API request.

The local player (player_t) is stored in the first index of a global array of players. By updating fields of the player, we can control things like health and weapon used. Behind the scenes, a player is also an mobj_t.

A door in Doom is a line_t with a special door flag. To find all doors, we iterate through all line_t in the map, returning all lines which are marked as a door. To open or close the door, we call the existing EV_VerticalDoor to toggle the door state.

API Specification

An API spec describes the HTTP methods, routes, and data types that the API supports. For example, it will tell you the type of data to send in a POST call to /api/world/objects, and the type of data you should expect in response.
I wrote the API spec in RAML 1.0. It is also hosted in a public API Portal for easier reading.

Putting it all together

So now we have an HTTP+JSON server inside Doom, interfacing with Doom objects in memory, and have written a public API specification for it. Phew!
We can now query and manipulate this 24 year old game from any REST API client - heres a video proving exactly that! Enjoy ;)

restful-doom on GitHub



Slack client for Commodore 64

Slack is great. Many smarter people than me also think that Slack is great. Slack is great because its simple and easier to deal with than emails. With all the time it saves me on emails, I   relax   go the beach   write more code   send messages via Slack instead.

But while Slack might be great, it does not have a great native client for the Commodore 64. In fact, they have no client for Commodore 64 at all!

This is clearly a problem.


Reasoning that “a pull request is better than a complaint”, I’m happy to present the first (and most likely only) Slack client for Commodore 64!

“Team communication for the 21st century” … now backwards compatible with 1985!

The C64 has an extension port called the Userport which, via an adapter, can communicate over RS-232 serial. I connected the Userport to a Raspberry Pi with a artisanal, locally sourced, homemade cable with the UserPort connector on one end, and a usb TTL-RS-232 converter on the other. The fastest I have been able to run this reliably is a solid 1200 baud / 150 bytes per second.

On the Commodore, I wrote an application in 6502 assembly. It uses built-in Kernal ROM functions to read and write the serial port and update the screen.

On the Pi, I wrote a NodeJS app which talks to the Slack RTM API. It is reponsible for translating a simple RPC protocol between the C64 and itself and connecting to the outside world (in this case, Slack). It uses serialport to talk to the USB serial driver.

RPC API

A simple RPC message format is defined between the C64 and Pi. The description below is all C-style, but of course on the C64 this is all implemented in 6502 assembly.

struct rpc_message_t {
  char command_id;
  void *payload;
  char message_end_marker (0x7e);
}

struct channel_t {
  char len;
  char[9] channel_id;
  char[len-9] channel_name;
}

When the Pi gets a connection to Slack’s RTM api, it kicks off communication with the C64. The protocol follows like this:

[Pi >> ] HELLO {user: jeff}
[Commodore >> ] REQUEST_CHANNEL_LIST
[Pi >> ] CHANNELS_HEADER { channel_count, list_size_in_bytes }
[Pi >> ] CHANNELS_LIST { channel_data }

// user selects a channel on the C64
[Commodore >> ] CHANNEL_SELECT { channel_id }

// message arrives via Slack RTM api in the selected channel
[Pi >> ] MESSAGE_HEADER_LINE { ascii_data }
[Pi >> ] MESSAGE_LINE { ascii_data }
[Pi >> ] MESSAGE_LINE { ascii_data }
...

// user sends message from C64
[Commodore >> ] SEND_MESSAGE { message }

When the channel data has finished streaming, we switch to the Channels screen. This shows a scrollable list of all channels and groups, ordered by unread_count, name. Hitting [RETURN] on a channel name sends channel_id to the Pi and switches to the Messages screen.

When a channel is joined, the Pi app sends the last couple of messages in the channel to the C64. Subsequently, whenever a message for that channel is recieved via the Slack websocket connection, it gets converted into a multiple lines of C64-compatible characters and sent across the serial connection.

Slash commands work too! We use the undocumented chat.command API found with help from Chrome dev tools :)


Heres an example of the kind of code running on the C64. This is the top level processing loop:

Want to do this yourself? Of course you do. Why else are you still reading?

Hardware needed:

  • 1 x Commodore64
  • 1 x Raspberry Pi (really anything with usb which can run NodeJS)
  • 1 x homemade C64 Userport <-> USB serial cable. instructions

Software:

It goes without saying (surely?!) that the company named ‘Slack’ is not affiliated in any way with this project. I’m not sure if they would be happy or horrified to see this creation.