Tool RPGM Ren'Py Cruncher - Ren'Py & RPGM MV Game Compactor [v0.4.1] [bas]

[bas]

Overview:
​

A batch file utility for simple automated LOSSY compressing of image, audio, and video files in Ren'Py & RPGM MV games without any updating of script/json files.

IMPORTANT: THIS TOOL OVERWRITES FILES IN THE GAME - IT DOES NOT MAKE BACKUPS OR SEND TO RECYCLE BIN

DO NOT run this on your only copy of the game if you don't want this.

Updated: 2019-08-06
Author: @bas
Tool Version: v0.4.1
Language: English

Thanks to @Sam for UnRen, from which I shamelessly stole the menu framework for this.

Features:

Spoiler

You don't have permission to view the spoiler content. Log in or register now.

Installation/Usage:

Spoiler

You don't have permission to view the spoiler content. Log in or register now.

Change-Log:

Spoiler

You don't have permission to view the spoiler content. Log in or register now.

Download:

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

Linux/Mac Port (Made/maintained by @goobdoob):

You must be registered to see the links

​

 

[N7]

work like a charm thanks for making this.

 

[Katgza]

God Bless you.

 

[CobraPL]

View attachment 76681

Overview:​

A batch file utility for simple automated LOSSY compressing of image, audio, and video files in Ren'Py games without any updating of .rpy and .py files.

Any chances for lossless WebP recompressor?
EDIT: I made link to your thread there:

You must be registered to see the links

 

[bas]

Any chances for lossless WebP recompressor?
EDIT: I made link to your thread there:

You must be registered to see the links

Do you have a text editor that shows line numbers like notepad ++ or Atom? If you do, edit the batch file line 75 and replace "-q 85" with "- lossless".

 

[CobraPL]

Do you have a text editor that shows line numbers like notepad ++ or Atom? If you do, edit the batch file line 75 and replace "-q 85" with "- lossless".

* lossless performance and compression improvements + a new 'cruncher' mode
(-m 6 -q 100)

EDIT: It looks like that for lossless it is better to use:
"-lossless -m 6 -z 9 -mt" or "-m 6 -z 9 -mt" since -z causes lossless

EDIT2:

-q float

Specify the compression factor for RGB channels between 0 and 100. The default is 75.

In case of lossy compression (default), a small factor produces a smaller file with lower quality. Best quality is achieved by using a value of 100.

In case of lossless compression (specified by the -lossless option), a small factor enables faster compression speed, but produces a larger file. Maximum compression is achieved by using a value of 100.

-z int
Switch on lossless compression mode with the specified level between 0 and 9, with level 0 being the fastest, 9 being the slowest. Fast mode produces larger file size than slower ones. A good default is -z 6. This option is actually a shortcut for some predefined settings for quality and method. If options -q or -m are subsequently used, they will invalidate the effect of this option.

so "-lossless -q 100 -m 6 -mt"

-lossless -q 100 -m 6 gives ~0,6% gain over -lossless -q 100
Not much LOL

 

[CobraPL]

@bossapplesauce one thing could be good - to compare output with input and save smaller one. Sometimes lossless WebP is larger than jpg. Sometimes not.

 

[gue5t]

Unfortunately the tests so far show that there is no such thing as loss-less WebP compression
Here is an example:

You must be registered to see the links

P.S. Files are in ZIP because the forum is doing some "optimization" and downsizing of big picture files that add artifacts and increase the file sizes (for example 22KB 1920x1080 file results in 130KB 1600x900 file).

P.P.S. @CobraPL Converting JPG files to loss-less format usually results in a bigger file (in rare cases like UI elements it can produce smaller results). In principle once you go the lossy compression route there is no going back (there is no way to recover the missing data).

 

[CobraPL]

P.P.S. @CobraPL Converting JPG files to loss-less format usually results in a bigger file (in rare cases like UI elements it can produce smaller results). In principle once you go the lossy compression route there is no going back (there is no way to recover the missing data).

You are right! "USUALLY"! This is optimizer (script). So, IF WebP lossless is smaller than jpg (rare, rather small files or poorly made jpgs), then script should convert to WebP. That is why size comparing should be used.
And if ANY uotput file is larger than input one - do store smaller.

EDIT:

Unfortunately the tests so far show that there is no such thing as loss-less WebP compression
Here is an example:

You must be registered to see the links

Well, not true. File supplier used lossy option and it looks like he thought "-q 100" alone is lossless.

You must be registered to see the links

 

[CobraPL]

@bossapplesauce
Hmm, I thought a little about this script. And I have some ideas for it:

TL;DR: To make REN'PY and RPGMaker MV or any folder optimization - automated, with lossy -q 100/lossy -q 85/lossless/custom commandline.

So, totally powerful script could do sth like this:
1. Starting, general disclaimer to not use on CMYK files.
2. Select general preset - lossy -q 100/lossy -q 85/lossless/custom commandline
3. Selecting RenPY or RPG Maker or any other folder.
4a. Renpy dectected - unrenpy on all RPA files, delete RPA files. Prepare for optimziation.
4b. RPGMaker (MV) detected - invoke decrypt program, delete archives, prepare files for optimization.
5a. In case of RPGMaker there is simple thing - PNGs must remain! So, either borrow/stole solution form TheOptimizer or invoke TheOptimizer for it. Default preset lossless.
5b. RenPY detected...
6. Start optimization as for point 2. Also convert WAV to FLAC (lossless) or Ogg Vorbis (lossy)
6a. During optimization always compare input and output, leave smaller one. Funny thing: sometimes PNGs/WebP(lossless) are smaller than jpgs (for small files). Sometimes wav is smaller than mp3/ogg (also for very small files).
7. Use TheOptimizer/borrowed programs on jpgs and mp3s (NOT on Ogg Vorbis) with lossless option.
8. Hunter-seeker script edit for RenPY - during point 6/6a maintain true file suffix, so there will be example.webp not example.png. That is why there should be simple script to replace example.png with example.webp inside RenPy files.
Why? Because this solution could be total one. Developer uses it, chooses lossless, hit START, wait, and has professionally optimized game. Powered by F95 Zone LOL

 

[bas]

@bossapplesauce
Hmm, I thought a little about this script. And I have some ideas for it:

TL;DR: To make REN'PY and RPGMaker MV or any folder optimization - automated, with lossy -q 100/lossy -q 85/lossless/custom commandline.

So, totally powerful script could do sth like this:
1. Starting, general disclaimer to not use on CMYK files.
2. Select general preset - lossy -q 100/lossy -q 85/lossless/custom commandline
3. Selecting RenPY or RPG Maker or any other folder.
4a. Renpy dectected - unrenpy on all RPA files, delete RPA files. Prepare for optimziation.
4b. RPGMaker (MV) detected - invoke decrypt program, delete archives, prepare files for optimization.
5a. In case of RPGMaker there is simple thing - PNGs must remain! So, either borrow/stole solution form TheOptimizer or invoke TheOptimizer for it. Default preset lossless.
5b. RenPY detected...
6. Start optimization as for point 2. Also convert WAV to FLAC (lossless) or Ogg Vorbis (lossy)
6a. During optimization always compare input and output, leave smaller one. Funny thing: sometimes PNGs/WebP(lossless) are smaller than jpgs (for small files). Sometimes wav is smaller than mp3/ogg (also for very small files).
7. Use TheOptimizer/borrowed programs on jpgs and mp3s (NOT on Ogg Vorbis) with lossless option.
8. Hunter-seeker script edit for RenPY - during point 6/6a maintain true file suffix, so there will be example.webp not example.png. That is why there should be simple script to replace example.png with example.webp inside RenPy files.
Why? Because this solution could be total one. Developer uses it, chooses lossless, hit START, wait, and has professionally optimized game. Powered by F95 Zone LOL

My intention was never for this to be used for lossless, just FYI. I wanted it to be very simple to use file size reduction of Ren'Py games.

And I have zero desire to make it work with RPGM games because I really don't care for them at all.

One thing that makes this script unique is that it takes a single format and converts all files to that format regardless of their original extension/format. Example: all image files are converted to webp, but the name and file extension is not changed. Ren'Py doesn't care if the format matches the extension, as long as it's a readable format. Here's how I was forced to do that...

At first I had the script change the file extensions as files were converted. It included a powershell script to parse through the Ren'Py text and change all the file extensions to webp for images, etc. But in Ren'Py some of the gui/buttons images MUST be PNG. And since a developer can rename these files and/or have them in different folders, it became impossible to keep the script both simple and functional - it would break the game changing these file extensions.

You've definitely got some good ideas, but far surpass the scope that I intended for this tool. To do all the stuff you're mentioning, it would really need to be a GUI tool in something like .net or C#, and I don't want to mess with trying to build it.

 

[CobraPL]

@bossapplesauce WebP 1.0.0 released

 

[konstant61]

Please make this Tool for 32-bit Windows....

 

[goobdoob]

Yep. Was a lot of work :test:

I'm a little surprised that worked. Windows and OSX/Linux use different permission schemes. I've seen Mac versions of Ren'Py games built on Windows machines that didn't have the executable bits set on the programs in the app bundle. I assumed that was because they'd been unzipped/rezipped on a Windows box, but I guess I was wrong.

You should be able to just copy the whole game folder over, replacing the one in the Mac app. That way you don't need to hunt for changed assets. The scripts are all the same, it's just the programs and libraries that are different. And the file structure, given the way a Mac app is put together.

 

[bas]

I'm a little surprised that worked. Windows and OSX/Linux use different permission schemes. I've seen Mac versions of Ren'Py games built on Windows machines that didn't have the executable bits set on the programs in the app bundle. I assumed that was because they'd been unzipped/rezipped on a Windows box, but I guess I was wrong.

You should be able to just copy the whole game folder over, replacing the one in the Mac app. That way you don't need to hunt for changed assets. The scripts are all the same, it's just the programs and libraries that are different. And the file structure, given the way a Mac app is put together.

That's what I meant - I unpacked the Mac version, deleted the rpas except scripts.rpa, and then copied the images/videos/sounds to the correct paths in \game. Left everything else alone and then rezipped.

 

[goobdoob]

That's what I meant - I unpacked the Mac version, deleted the rpas except scripts.rpa, and then copied the images/videos/sounds to the correct paths in \game. Left everything else alone and then rezipped.

You should be able to delete \game in the mac version, and copy the crunched pc \game over. Everything in \game is identical between the two versions. That might be a little easier than monkeying around with a bunch of files.

The issue I was talking about comes in <app>/Contents/MacOS. There's a Unix executable (for example, Milfy_City in Milfy City). Its the equivalent of the .exe on Windows. OSX/Linux use permission bits - read/write/execute for owner, group, everyone. Windows uses ACLs (access Control Lists). If the execute bit isn't set, the program won't run, and the app won't launch. I thought that unzipping a Mac program on Windows might strip the permission bits, but I guess not.

 

[bas]

You should be able to delete \game in the mac version, and copy the crunched pc \game over. Everything in \game is identical between the two versions. That might be a little easier than monkeying around with a bunch of files.

The issue I was talking about comes in <app>/Contents/MacOS. There's a Unix executable (for example, Milfy_City in Milfy City). Its the equivalent of the .exe on Windows. OSX/Linux use permission bits - read/write/execute for owner, group, everyone. Windows uses ACLs (access Control Lists). If the execute bit isn't set, the program won't run, and the app won't launch. I thought that unzipping a Mac program on Windows might strip the permission bits, but I guess not.

Ok, I'm with you now. I use 7zip, so not sure how any other zip program would behave. But yeah, glad it worked

 

[Ryahn]

Please make this Tool for 32-bit Windows....

32 bit windows in being phased out and more applications are not supporting it.

 

[goobdoob]

@bossapplesauce to use this I should run UnRen to exract RPAs, then delete the RPAs that have images/video/audio?

 

[bas]

@bossapplesauce to use this I should run UnRen to exract RPAs, then delete the RPAs that have images/video/audio?

yes