Tool RPGM Ren'Py Cruncher - Ren'Py & RPGM MV Game Compactor [v0.4.1] [bas]

[goobdoob]

I just made a test using few random file samples from the game and come to the conclusion it goes to compress all images again. On .jpg files, the size difference after second iteration and 10th seems to be the same in most cases I tried. Only thing that could actually be breaking the game is the .png images transparency. First iteration seems to be ok and quality decrease is hardly noticeable while size decrease is significant. Second compression on png's seems to remove transparency since everything is now white (image for example taken from renp'y with background being shown behind usually, but not now since png is over the whole screen) Since we use png's to combine characters I will just have to make sure to not compress png images multiple times, while jpg's in my opinion don't matter as much. So in conclusion, if anyone is using just .jpg format for their game, compressing multiple times doesn't seem to hurt that much.
View attachment 642036

Since you're technically savvy enough to use git, I suggest you look at the bat file/shell script, pull out the call to cwebp and write your own script based on it. Put the new files in a separate directory, run on that, then put the newly compressed files in your directory structure with the rest of the images.

Another suggestion - make a sig image, and put it and a link to your F95zone game thread and patreon in your signature.

 

[PearlNecklaceProductions]

Since you're technically savvy enough to use git, I suggest you look at the bat file/shell script, pull out the call to cwebp and write your own script based on it. Put the new files in a separate directory, run on that, then put the newly compressed files in your directory structure with the rest of the images.

Another suggestion - make a sig image, and put it and a link to your F95zone game thread and patreon in your signature.

Well i went and tried to modify the bat file a bit. I was my first time doing it, so I believe better job could be done, however this suits my needs perfectly so someone else might need it too.
Basicially what i added is a function that compares files that are yet to be compressed with the .txt file that lists all compressed files. If the file does not exist, it is created when first entry/compression is being done and also writes down the full file location in the txt file (rencrunch_compressed.txt is name set on line 8).
If same file was already compressed once, it will skip it.

As for the suggestion regarding the sig image, thank you. will do in the upcoming days.

 

[hngg]

Thanks bas & PearlNecklaceProductions for your work on the batch files. I'd like to share an updated version of the scripts that I've been working on here.

You must be registered to see the links

Features include:

1. Updated cwebp.exe, ffmpeg.exe, nconvert.exe, and libwebp.dll to their latest versions.
2. Tweaked settings to look better at the cost of slightly slower encode speed.
3. Two quality levels to choose from, LQ(Low Quality) & HQ(High Quality). LQ should be 20-50% smaller.
4. A new "AudioRipper" script that removes audio tracks from videos to save space.

The AudioRipper is useful for games that have blank/silent audio tracks taking up space in the videos. It works by simply copying the video portion to a new file and renaming it to the original name. This means the videos are exactly the same quality and the whole thing takes seconds, not hours. To use it, create a folder titled "ripme" located in the same directory as the scripts and copy any video you want to remove audio from to there. When you're done and happy with the results, move the videos back to where they were and overwrite.

 

[CobraPL]

Thanks bas & PearlNecklaceProductions for your work on the batch files. I'd like to share an updated version of the scripts that I've been working on here.

You must be registered to see the links

Features include:

1. Updated cwebp.exe, ffmpeg.exe, nconvert.exe, and libwebp.dll to their latest versions.
2. Tweaked settings to look better at the cost of slightly slower encode speed.
3. Two quality levels to choose from, LQ(Low Quality) & HQ(High Quality). LQ should be ~20% smaller.
4. A new "AudioRipper" script that removes audio tracks from videos to save space.

The AudioRipper is useful for games that have blank/silent audio tracks taking up space in the videos. It works by simply copying the video portion to a new file and renaming it to the original name. This means the videos are exactly the same quality and the whole thing takes seconds, not hours. To use it, create a folder titled "ripme" located in the same directory as the scripts and copy any video you want to remove audio from to there. When you're done and happy with the results, move the videos back to where they were and overwrite.

Is there option to force lossless and option to retain smaller file in case of bigger optimized one?

 

[Abhai]

Thanks bas & PearlNecklaceProductions for your work on the batch files. I'd like to share an updated version of the scripts that I've been working on here.

You must be registered to see the links

Features include:

1. Updated cwebp.exe, ffmpeg.exe, nconvert.exe, and libwebp.dll to their latest versions.
2. Tweaked settings to look better at the cost of slightly slower encode speed.
3. Two quality levels to choose from, LQ(Low Quality) & HQ(High Quality). LQ should be ~20% smaller.
4. A new "AudioRipper" script that removes audio tracks from videos to save space.

The AudioRipper is useful for games that have blank/silent audio tracks taking up space in the videos. It works by simply copying the video portion to a new file and renaming it to the original name. This means the videos are exactly the same quality and the whole thing takes seconds, not hours. To use it, create a folder titled "ripme" located in the same directory as the scripts and copy any video you want to remove audio from to there. When you're done and happy with the results, move the videos back to where they were and overwrite.

good thinking man
tested it a bit, went through a few animated games, and in few of them there were animations with useless/mute sounds...
integrating your audio-ripping into cruncher saves 10% of the space/size on averages prior to compressing even starts.
the only thing i could change a bit are those settings of yours with hq/lq cruncher versions...going with audio 64k for "Lq" is a bit too much imo (32k, or sometimes even 24k works there),..similar goes to "Hq" - I would use 48k there. Yet, Im one of those who usually dont play/listen avn games music/sounds, so I could be a bit biased there.
And one more thing, regarding webp, "Hq" imo it would be better with 85-90q (than 80q, which is default settings), and in "lq" i would turn af off and instead m6 go with m5 (just for those with a bit older pc-s and w/o ssd-s)

 

[goobdoob]

Is there option to force lossless and option to retain smaller file in case of bigger optimized one?

You must be registered to see the links

 

[hngg]

Is there option to force lossless and option to retain smaller file in case of bigger optimized one?

That's an interesting idea. What you're asking for is something that compares the file size of the input and output and keeps whichever is smaller? That's certainly doable. It'd be a good option for Devs to use on their own games to keep size down without effecting quality. A "Master Quality" preset, perhaps? I'll mess around with that idea when I have the time. Could be a while.

good thinking man
tested it a bit, went through a few animated games, and in few of them there were animations with useless/mute sounds...
integrating your audio-ripping into cruncher saves 10% of the space/size on averages prior to compressing even starts.
the only thing i could change a bit are those settings of yours with hq/lq cruncher versions...going with audio 64k for "Lq" is a bit too much imo (32k, or sometimes even 24k works there),..similar goes to "Hq" - I would use 48k there. Yet, Im one of those who usually dont play/listen avn games music/sounds, so I could be a bit biased there.
And one more thing, regarding webp, "Hq" imo it would be better with 85-90q (than 80q, which is default settings), and in "lq" i would turn af off and instead m6 go with m5 (just for those with a bit older pc-s and w/o ssd-s)

I'm an audio buff, so it hurts me deeply to hear 32k music. 48k shouldn't be too bad.
Maybe I'll rename the existing profile to "MQ" and make a new "LQ" that drops everything even lower. Should I enable down-mixing the audio to mono for that lowest quality setting? Opus makes some trade-offs once you go really low in bit-rate and I'm not sure what would sound better to most people; stereo with 12k low-pass filter, or mono with 20k. It'd depend on the music, I guess. And how old the listener is(unable to hear high pitched noises) .

The "HQ" script is really only meant for games with lossless PNG and 256k+ MP3 files. Q80 looks fine coming from PNG, especially with -af and -m6, but from JPG it might be too much. I think I'll change it so Q80 is used for PNG and Q90 for everything else.

As far as speed for those of us with old PCs(ME being one!) I think I want to work on implementing Parallel processing. None of the encoders are particularly well multi-threaded, which means people with CPUs with more than 2 cores aren't getting full speed. I'd like to fix that, but that will probably take a while.

 

[Abhai]

...I'm an audio buff, so it hurts me deeply to hear 32k music. 48k shouldn't be too bad.
Maybe I'll rename the existing profile to "MQ" and make a new "LQ" that drops everything even lower. Should I enable down-mixing the audio to mono for that lowest quality setting? Opus makes some trade-offs once you go really low in bit-rate and I'm not sure what would sound better to most people; stereo with 12k low-pass filter, or mono with 20k. It'd depend on the music, I guess. And how old the listener is(unable to hear high pitched noises) .
.....

well, i'm not sure how, yet i've guessed you could be one of those "audio-freaks"
so whatever rumble your ea...headphones man
as for the rest, i agree.
the beauty of bas's tool, apart its wide applicability, is the easiness of tweaking/upgrading it.
(this one has to be said, as i know bas is watching this thread, so just to be sure we stay on the good side of bas)

 

[hngg]

...

Okay, I've updated my mod. Both quality sets drop the -af and -m6 options for speed. HQ now uses Q90 for lossy input pictures, Q80 for lossless input(with sharp RGB->YUV conversion enabled).
LQ now uses Q70 for all input pictures, but adds lossy alpha compression to lossless inputs for even more space savings. LQ also drops audio bit-rate to 48k.

I have no intention of stepping on anyone's toes here. If bas has any issue I'll gladly capitulate.

Also: My Beyerdynamic DT 770 pros will never leave my head!

 

[Abhai]

Okay, I've updated my mod. Both quality sets drop the -af and -m6 options for speed. HQ now uses Q90 for lossy input pictures, Q80 for lossless input(with sharp RGB->YUV conversion enabled).
LQ now uses Q70 for all input pictures, but adds lossy alpha compression to lossless inputs for even more space savings. LQ also drops audio bit-rate to 48k.

I have no intention of stepping on anyone's toes here. If bas has any issue I'll gladly capitulate.

Also: My Beyerdynamic DT 770 pros will never leave my head!

you did good job there, those few tweaks and audio-ripper are/could be really useful
i was bas-joking only, no worries there...
...at least till the time comes for ya to finally remove those beloved dt770

 

[hngg]

Is there option to force lossless and option to retain smaller file in case of bigger optimized one?

Well, it's taken a while but I've finally gotten around to implementing those features. My YAC script now compares file sizes(crunched vs original) and keeps the smaller of the two. It now also uses lossless compression for images if you set image quality to 100. Let me know what you think.

 

[Peserval]

hello I have weird problem with both Cruncher and YAC, after interrupting and closing compression of game midway through I can't anymore compress games in folder wich that game was (also affects sub-folders) no matter if I'm trying to use bas Cruncher or hngg YAC I guess something must be messed up in windows? (compression works if I pull game out of that folder or sub-folder)

 

[bas]

hello I have weird problem with both Cruncher and YAC, after interrupting and closing compression of game midway through I can't anymore compress games in folder wich that game was (also affects sub-folders) no matter if I'm trying to use bas Cruncher or hngg YAC I guess something must be messed up in windows? (compression works if I pull game out of that folder or sub-folder)

If you happen to be on Windows 10, I've had that happen before where Windows Defender decides that I'm not allowed to run .bat files in a given folder anymore.

Try running it in a completely different folder. If that works, then add to your exceptions list. And I dunno what YAC is?

 

[Abhai]

...And I dunno what YAC is?

YetAnotherCruncher (the latest addition to crunchers scripts list)

 

[bas]

YetAnotherCruncher (the latest addition to crunchers scripts list)

Derp, I remember now. Thanks bud

 

[recreation]

macadam why do you need to unpack your own game to use the cruncher?

 

[macadam]

macadam why do you need to unpack your own game to use the cruncher?

i follow the tutorial...

it say you need to unren the .rpa, then use the cruncher.

i dot want to take the risk to use it directly on what i m working on. soo i used it with a copy of the game builded.

 

[recreation]

i follow the tutorial...

it say you need to unren the .rpa, then use the cruncher.

i dot want to take the risk to use it directly on what i m working on. soo i used it with a copy of the game builded.

Okay, but why not use a copy of the game not build? It would at least cut the unpacking part^^

 

[macadam]

Okay, but why not use a copy of the game not build? It would at least cut the unpacking part^^

unsure if that would work, 'cause no .exe. And maybe it's required to work...

i dont know, i just give a try for first time lol, like i say, i m noob at this shit XD

 

[macadam]

the problem wasn't the rpa extract , but more the compression itself as the tool do each images and it took lots of time on my side.