Sega Genesis Native Hardware Chiptune Synthesizer

jarek319 · Post by **jarek319** » 2016-08-28, 22:43:33

Hi everyone! I want to create a complete kit that anyone can use to synthesize VGM files on the native Sega Genesis sound chips, and I need someone who is familiar with the VGM format to help me out.

I uploaded the schematics and source code I currently have to https://hackaday.io/project/13361-sega- ... ynthesizer , and I really need some help understanding how to read, validate, and parse VGM files, so I can translate the files directly to my helper functions:

sn76489.setFrequency( Channel, Frequency )
sn76489.setRegister( Register, Data )
ym2612.send( Address, Data )

ctr · Post by **ctr** » 2016-08-28, 22:50:07

VGM files are literally just a stream of commands sent to the chips. There are also timing commands, which tells the player to wait "n/44100" seconds. The hardest thing with parsing is the version differences of the format, but for Mega Drive VGMs this is fairly simple. The latest format specification is found here or here.

Notice that VGM files usually get quite large, possibly too large to fit inside the RAM of an atmega microcontroller. In that case you will have to stream data from external memory, a PC or similar. This is possible with VGMs, but keep in mind files that use DAC data stored inside datablocks.

jarek319 · Post by **jarek319** » 2016-08-28, 23:13:10

These file descriptors will definitely help me get started, thanks!

Thankfully I can read Streams out from the SD card, so I only need a buffer in RAM to store the next fifty or so instructions for skipless playback.

bmos · Post by **bmos** » 2016-08-31, 5:06:03

awesome project. i had both genesis chips + teensy 3 spilled across several breadboards for a while under it took up too much space on my desk. this looks rad

Post by **vampirefrog** » 2016-08-31, 16:39:53

That looks great! You can easily stream VGM commands from a filesystem or any sort of byte stream.

I've written a couple of C classes that do just that, and you might be able to use them, or at least take some ideas from them.

a byte-by-byte header parser:
https://github.com/vampirefrog/libvgm/b ... erparser.c

a byte-by-byte command parser:
https://github.com/vampirefrog/libvgm/b ... gmparser.c

They use macros called from separate header files.

There is one catch, some commands declare data blocks (the YM2612 part sometimes uses these), and you'll have to store those in RAM, or perhaps a temporary file (some are compressed) and stream from that.

You'll also need decompression code for some of the compressed data blocks, I think you can find it here.

jarek319 · Post by **jarek319** » 2016-09-02, 3:01:04

Thanks to everyone for the guidance, samples, and suggestions! I've compiled all the info into the following results:

I posted the code for this under vgmTest.ino at http://hackaday.io/project/13361-sega-g ... ynthesizer

https://youtube.com/watch?v=8vRRA7wkYX8 A short video of the current state of the project.

Can anyone can help me figure out what that "blip" at the beginning is? I have a feeling it has something to do with the parser, so I took a log of everything the parser was doing, careful it's a 3MB text file: soniktech.com/putty.log

Here is the file I am using for testing: http://soniktech.com/SONIC.VGM

It seems that the Atmega1284 chip I chose for this is too slow to keep the 44.1KHz DAC on the YM2612 fed, even when running at 20MHz. Reading from the SD card is slow too, and I doubt I'll have the memory to decompress vgz files, so I'm switching over to a higher power Arm Cortex chip. This should give me the headroom to expand the code as far as the VGM format requires. That way the hardware won't have to change if updates come out.

I'll keep you guys updated

you can also follow the project on hackaday.io for notifications

Post by **ValleyBell** » 2016-09-02, 7:08:55

I edited the post to make the links "real" links.

This looks nice so far.
A few things I noticed:

the "blip" is caused by the way VGMs were trimmed in 2005 - if you trimmed the 1 second off, VGMTool only removes the delays (and keeps all commands), so there's lots of gargabe at sample 0.
You can work around this if you save all commands of the first VGM sample into a virtual YM2612 state and send that state to the sound chip before playing the rest of the VGM.
You decompressed the VGZ incorrectly - it removed all 0D bytes. (or maybe it's just the uploaded VGM) This causes all offsets to be invalid and breaks a few commands. SONIC.VGM should have 58 452 bytes.

The DAC in Sonic 1 runs with up to 20 KHz, btw. (i.e. it's not updated at every VGM sample - the YM2612 can't do that anyway) Timpanis are usually around 8-10 KHz.

jarek319 · Post by **jarek319** » 2016-09-03, 5:46:11

Thanks for the tip about the blip! I'll see if there's a way I can implement that workaround

I uncompressed it using 7zip... is there a recommended way to extract them on a PC? I plan on implementing the decompression algorithm after I add some sort of memory to the chip, so I can read from the SD card, decompress and save to memory, then play from memory. It would be helpful to have a "known good" copy to make sure I'm doing it all right though

by the way, does anyone know what the largest VGM file is? I want to pick an inexpensive memory chip but I'm not sure what the scene would expect from its capabilities. I think the largest I found was from Xmen 2: Clone Wars, second track is 8.07MB when uncompressed

Post by **ValleyBell** » 2016-09-03, 9:19:04

Using 7zip or gzip should work well. As I said, it could've been the FTP upload that corrupted the file as well.

The largest song I've found (I had only a quick look) is King Colossus: Silent Fog (Staff Roll) (13.2 MB uncompressed, vgm_cmp can reduce it to 3.6 MB though)

Post by **vampirefrog** » 2016-09-03, 19:55:53

You sure the Sd card would be too slow? I'd calculate at 44100 samples per second and usually samples are 8 bits or less, let's assume it's 16 bits, so 44100 * 2 = 88200 bytes per second, and afaik sd cards can do megabytes per second, so thay's my two cents.

jarek319 · Post by **jarek319** » 2016-09-04, 7:03:50

using a tiny microcontroller that comes in at 48MHz, sd cards only reach about 200kB/s, and having to seek through a vgm file for pcm data makes performance really poor. what i can do is copy the uncompressed vgm file to a faster on-board memory chip, and read the vgm file from there where I can access arbitrary points in memory quickly.

Post by **ValleyBell** » 2016-09-04, 12:58:23

You could also copy only the PCM data blocks to the on-board memory. Then you wouldn't need to seek back and forth.
Data blocks for YM2612 shouldn't get a lot larger than they are in the PCM-only Toy Story songs.

jarek319 · Post by **jarek319** » 2016-09-05, 23:46:46

That was exactly what I was doing for the first version

the Sonic Stage Clear PCM data comes in at just under 7KB, which fits nicely into microcontoller RAM, but even the most expensive microcontrollers have like 128kB of RAM. I'm planning on putting the entire decompressed VGM file in some sort of external memory that I can read arbitrary locations from at 44.1kHz, which requires a different microcontroller.

Currently I've made the code as fast as possible reading from internal memory, and the bottleneck now is the rate at which I push serial data out to the chips which handle my microcontroller -> sound chip conversion. I need to redesign my circuit with a microcontroller that will let me flip all those 16 bits at once instead of one at a time, which would give me the constant 44.1khz sample rate I need to accurately reproduce samples.

youtube.com/watch?v=4KVDb4aGcvQ here's the latest version that I can code into this circuit board.

jarek319 · Post by **jarek319** » 2016-10-02, 20:44:29

Using a combination of suggestions from you guys and some sneaky tricks, I've managed to get a pretty accurate recreation of VGM files on native hardware. There are still a few things I need to iron out, but one thing I stumbled on that I was confused about regarding VGM files: for some reason, any 0x80-0x8F commands had to have their pause counts decremented or everything would sound funny. Meaning, 0x80 samples play for one "beat", and 0x81 samples play for one "beat", but 0x82 samples plays for 2 beats. The part I was always struggling with is whether or not delay timers are supposed to be inclusive of the current instruction, or...?

youtube.com/watch?v=YA_0UVSEqgo

Post by **ValleyBell** » 2016-10-03, 0:14:38

None of the "normal" commands causes any delays.
Delay commands have an effect immediately when processing it. (i.e. after all previous and before all following commands)
Commands 7n cause a delay of (n+1) VGM samples, commands 8n are DAC writes followed by a delay of just n VGM samples. (i.e. 0x80 is a DAC write without delay)