Defeating Checksums

This tutorial was written by (Jazz) Jackalope, with much-appreciated advice from ValleyBell.

For today's recipe, you will need:

MAME
A hex editor, such as XVI32 or Translhextion. (If you have a need to defeat a checksum, you proooobably already have used one.)

First, you have to run your target game with MAME in debugging mode. You can do that via the command line:

mame.exe -debug genesis -cart a_game_goes_here.ext

And by using the program Kaijuu as I describe with a pattern of *.md;*.smd;*.bin, you can make that command accessible by right-click:

mame.exe -debug genesis -cart {file}

As you can see below, Mame's debugger has a profoundly friendlier UI than any other part of it. (This should be a big relief for those with MAME allergies, like me.)

You first need to find out the game's checksum. (Or, you can assume the Checksum appears at hex address 018E and skip ahead to "Return to the main MAME debug window." If that fails, come back here.) In the debugger window, press Ctrl+L or use Debug -> New Error Log Window. Despite the name, this contains general info as well, and it'll tell us the checksum. Have an example:

[:mdslot] INTERNAL HEADER
[:mdslot] ===============
[:mdslot] Console: SEGA GENESIS

[:mdslot] Checksum: 2078
[:mdslot]  - Calculated Checksum: 9877
[:mdslot] Supported I/O Devices: J

Smashing! You want to ignore the "Calculated Checksum", which is no help to us. So in this case, the number we want is 2078.

Now it's time for the hex editor to come in. You should be able to use its Find feature to find the checksum in code, which will appear very early (coming soon after the intelligible header info). As you've possibly guessed from the advice above, the example Checksum indeed appears at hex address 018E.

Return to the main MAME debug window. At the bottom, there's a field for entering commands. You now want to use wpset, as you can see from the screencap above:

>wpset ####,1,r

The four # signs will be replaced by the Checksum's location in hex.
The second number is just a range for watching addresses ahead of the first number; if you don't quite grasp that, don't worry. We found exactly where the address we needed is, so whatever number you give makes no difference (as long as it's bigger than zero...).
The r means to watch for anything that tries to read from those addresses. (You can also watch for writes or both, but that won't help us here.)

If you did it right, the debugger reports that a Watchpoint was set. Now hit F5 or use Debug -> Run. Soon it should stop and tell you that the Checksum was read from its address, and tell you the address that did so (in parentheses).

Very soon after that line (probably highlighted), either a bne command (as shown) or a beq appears. What you'll need to change it to depends on which command:

BEQ: becomes BRA (60 xx, where xx can be whatever's already there)
BNE: becomes NOP (4E 71)

In your hex editor, go to the address (in this case, 036E) of the bne/beq command. Change the numbers there to those of the target command. In this case, 66 00 will be transformed into 4E 71.

Save your changes. If you haven't modified anything else about your target ROM yet, you won't see a difference right away. But if you have, you should find to your delight that the annoying red screen of blockage has gone away!

Note: In the small handful of games I tested writing this tutorial, the Checksum address always happened to be 018E. It may be different for different games, but the exceptions are probably dwarfed by the non-exceptions. I think.

Now that we've seen success, let's review the process:

Open game in MAME debugger
Use the Error Log window to find out the Checksum
Search for the Checksum in a hex editor, which will tell you its address
Use the debug command wpset ####,#,r with appropriate numbers to set a watchpoint on the Checksum's address (most likely, wpset 018E,1,r)
Continue execution with Run
Back in the hex editor, go to the address of the soonest (probably highlighted) beq/bne command after the checksum-reading address reported by MAME
Change it to 60 xx or 4E 71 accordingly
Save
?
Profit!

You're welcome, and happy ripping!

Wait, but Can I Work This Sorcery in BlastEm?

Absolutely yes, but it's more tedious: BlastEm doesn't currently have watchpoints, and if there's an alternate way, I'm not able to find it.

You have to enter the debugger before the checksum is read (use the -d option to do this most reliably; hitting u immediately after loading the game is fast enough for Dynamite Headdy, putting me right before where I need to look, but this may be game-dependent). Once there, enter the command over once. You can then hit Return upwards of possibly 30 times or more to continue using over until you finally see a command like this fly past:

366: move.l #$18E, a1

See a familiar number? It's the location of the checksum for Dynamite Headdy, as we (I) discovered from the hex editor earlier. It's accompanied by its retainers:

366: move.l #$18E, a1
> over
36C: cmp.w (a1), d1
>
36E: nop

(Actually, this is my patched ROM, so I've turned the bne into a nop(e) already. You can see that the system works!)

Indeed, if you look back up at the MAME image, you can see that these are the very same lines, aside from my change.

But here's the problem: to find the Checksum-reading this manually, you have to already know where the Checksum is located, and the quickest way to do that... seems to be using MAME's debugger. Otherwise, you can make a change to the ROM so the Checksum is bad, use the dozens of overs until you hit the Bad Checksum loop, then follow the commands upward until you find what you think is the correct beq or bne command to change. (It won't always be as obvious as it is for Dynamite Headdy.)

Without a watchpoint or ability to simulate them, the only advantage this supplies is the ability to avoid MAME allergies. Because debugger use sidesteps those nasty allergies, this method is incomparably worse.