Compression Issues
7-Zip + VGZs make larger archives?
Technical discussion about the VGM format, and all the software you need to handle VGM files.
Moderator: Staff
- RichterEX2 Offline
- Posts: 191
- Joined: 2012-03-30, 9:28:37
- Location: Georgia, USA
Compression Issues
Something I'm curious about. If I have a bunch of un-VGZed VGMs and put them into an archive with 7-Zip, I get a much smaller archive (985kb) than if I convert them all the VGZ and do the same (3MB). Is there a way I can have both small individual VGM files as well as having smaller archive sizes?
I'd say this is limited to the way compression of this nature works. 0Try compressing each individual VGM with 7-zip and then pack them inside another 7-zip.
You're getting bigger sizes because for any common data shared across the separate VGMs, because vgz already compresses everything then their representative form isn't necessarily useful for a group of different files to compress, simply because the compressed form is already different data in its own right.
You're getting smaller file sizes with uncompressed VGMs compressed together in one hit, most likely because of the following trait: http://en.wikipedia.org/wiki/Run-length_encoding
You're getting bigger sizes because for any common data shared across the separate VGMs, because vgz already compresses everything then their representative form isn't necessarily useful for a group of different files to compress, simply because the compressed form is already different data in its own right.
You're getting smaller file sizes with uncompressed VGMs compressed together in one hit, most likely because of the following trait: http://en.wikipedia.org/wiki/Run-length_encoding
-dj.tuBIG/MaliceX
- ValleyBell Offline
- Posts: 4787
- Joined: 2011-12-01, 20:20:07
- Location: Germany
- RichterEX2 Offline
- Posts: 191
- Joined: 2012-03-30, 9:28:37
- Location: Georgia, USA
Why is this even being questioned? The standard has always been VGZ files in a ZIP archive, and this isn't up for debate. We still lack a proper tutorial, but you can look at the one on project2612.
Also known as nineko.
I didn't want to state it again (which is why I didn't post this earlier), but I might as well do it now. My personal opinion about VGM7z is that it's a very bad idea for most purposes. True, it reduces the size of the global archive, but it introduces an unneeded layer of complexity for the player, and the individual files are much bigger than they should once they're extracted. For the same reason, I don't like the RSN and RGM "formats" either, since they're only RAR files in a wig (in fact, I always rename them to .RAR and use them as such); as a matter of fact, a quick research on Google further proves my point (some guy having problems with RGM files on the Winamp forum).
As of now, only smspower officially supports (and encourages) VGM7z; we don't, and Project2612 afaik still provides only the ZIP files, even though they are considering the possibility to provide VGM7z files too.
Furthermore: everyone and their moms know what a ZIP file is and how to handle it, while 7z is still relatively obscure to the occasional guy. Even though the ZIP compression is much worse than RAR and 7z, it is one of our goals to be as cross compatible as possible, across every operating system. In fact, I can tell you something. Windows 7's built-in ZIP handler is broken (unsurprisingly, like 99% of the things by Microsoft), and it refused to open one of the ZIP packs I uploaded on this website (even though every reasonable ZIP manager, including the 1993 version of pkunzip for ms-dos, could open it just fine): ValleyBell recompressed it with a different program, and the ZIP file is now slightly bigger, but those people who (for some reason) use Windows 7 can now open it too.
tl;dr reducing the file size is good but increasing the compatibility is better.
As of now, only smspower officially supports (and encourages) VGM7z; we don't, and Project2612 afaik still provides only the ZIP files, even though they are considering the possibility to provide VGM7z files too.
Furthermore: everyone and their moms know what a ZIP file is and how to handle it, while 7z is still relatively obscure to the occasional guy. Even though the ZIP compression is much worse than RAR and 7z, it is one of our goals to be as cross compatible as possible, across every operating system. In fact, I can tell you something. Windows 7's built-in ZIP handler is broken (unsurprisingly, like 99% of the things by Microsoft), and it refused to open one of the ZIP packs I uploaded on this website (even though every reasonable ZIP manager, including the 1993 version of pkunzip for ms-dos, could open it just fine): ValleyBell recompressed it with a different program, and the ZIP file is now slightly bigger, but those people who (for some reason) use Windows 7 can now open it too.
tl;dr reducing the file size is good but increasing the compatibility is better.
Also known as nineko.
Interesting. You'd think Microsoft are great at keeping backwards compatibility with even features like that (considering the number of band-aids their operating system has :V) also ahh, good old PKUNZIP.Tom wrote:...increasing the compatibility is better.
Also yeah, with the discussion on complexity, for the (extremely) impatient users, 7z can take much longer than ZIP, and certainly much longer than a solid RAR on best settings, to extract if you have a massive file archive.
But yeah, aside the pros and cons on the usability side of things, data bandwidth and storage space shouldn't even be an issue in this day and age.
Last edited by MaliceX on 2012-10-17, 14:51:38, edited 2 times in total.
-dj.tuBIG/MaliceX
- ValleyBell Offline
- Posts: 4787
- Joined: 2011-12-01, 20:20:07
- Location: Germany
There's one thing that annoys me about VGM7z:
Typing del *.vgm in the commandline deletes all VGMs and VGM7zs. (And yes, I know that this is because of the DOS 8.3 filenames Windows generates.)
I hope to be able to make in_vgm/VGMPlay support reading from zip-files sometime, btw.
I have nothing against vgm7zs and 7zs files in the Submissions Forum, since I'm repacking them anyway.
One thing though: vgm7z files are less handy than 7zs, because WinRAR doesn't let me unpack vgm7z with a right click, so 7z is preferred.
Typing del *.vgm in the commandline deletes all VGMs and VGM7zs. (And yes, I know that this is because of the DOS 8.3 filenames Windows generates.)
I hope to be able to make in_vgm/VGMPlay support reading from zip-files sometime, btw.
I have nothing against vgm7zs and 7zs files in the Submissions Forum, since I'm repacking them anyway.
One thing though: vgm7z files are less handy than 7zs, because WinRAR doesn't let me unpack vgm7z with a right click, so 7z is preferred.
Another 'Hidden Benefit' of VGZ's in ZIP's
Hey guys! Figured I'd sorta explain another reason that Tom mandates the VGZ in ZIP method of distributing VGM's. Bear with me here...
I use XMPlay. XMPlay can semi-natively use Winamp plugins as their own. There is a native XMPlay plugin that deals with Zip files, that enables adding zipfiles to add all the playable contents within to the playlist.
In short, VGM's can be played from within zipfiles, given knowledge of said methods!
I use XMPlay. XMPlay can semi-natively use Winamp plugins as their own. There is a native XMPlay plugin that deals with Zip files, that enables adding zipfiles to add all the playable contents within to the playlist.
In short, VGM's can be played from within zipfiles, given knowledge of said methods!
Until next post...
Re: Another 'Hidden Benefit' of VGZ's in ZIP's
I don't really follow your reasoning.JFD62780 wrote:Hey guys! Figured I'd sorta explain another reason that Tom mandates the VGZ in ZIP method of distributing VGM's. Bear with me here...
I use XMPlay. XMPlay can semi-natively use Winamp plugins as their own. There is a native XMPlay plugin that deals with Zip files, that enables adding zipfiles to add all the playable contents within to the playlist.
XMPlay has also an archive plugin for both 7z and rar archives, so nothing's really stopping you from using VGM in 7z/rar. Unless you want much larger VGM archives, but honestly, why would you want that?