Tool Others HTML Ren'Py YAC - Yet Another Cruncher (A Game Compactor) [V3.5]

[hngg]

hngg Since some devs use it to crunch their games, you may want to:
- add -exact in lossy image compression too (at least for lossy quality 90% and higher)
- change the -m value from 5 to 6 (or allow the user to change the value)
- maybe add another lossy 100% quality preset.
- add an option to disable the filesize comparison (before replacing the original file) or disable it by default, for loseless and 100% lossy compressions.

ps. here you can find examples & summarized the most important webp compression options/settings... for those that want to finetune the compression of their images

Thanks for your input.

• It is not necessary to preserve RGB values in transparent areas. There may be some good reasons to do that for archival storage but this tool is for distribution, not archiving.
• In the first versions of YAC I used the -m 6 switch but saw very little benefit to justify the speed hit.
• Lossy 100% is already used with the "ultra" preset on lossy inputs. The custom mode set to 99% would give lossy output for everything is you want that. I don't want to have more presets, four is already pushing it for me.
• I'm not sure what the benefit to disabling the file size check would bring. Do you have an example use case where that'd make sense? Why would you want a compression tool to give you files bigger than what you started with?

If you can show me the benefit to a change I'll gladly implement it. As it stands, I want this to to be as simple as possible, so I have to weigh each new feature against how many more things my users will have to think about.

 

[lewdgamerxxx]

Thanks for your input.

• It is not necessary to preserve RGB values in transparent areas. There may be some good reasons to do that for archival storage but this tool is for distribution, not archiving.
• In the first versions of YAC I used the -m 6 switch but saw very little benefit to justify the speed hit.
• Lossy 100% is already used with the "ultra" preset on list inputs. The custom mode set to 99% would give list output for everything is you want that. I don't want to have more presets, four is already pushing it for me.
• I'm not sure what the benefit to disabling the file size check would bring. Do you have an example use case where that'd make sense? Why would you want a compression tool to give you files bigger than what you started with?

If you can show me the benefit to a change I'll gladly implement it. As it stands, I want this to to be as simple as possible, so I have to weigh each new feature against how many more things my users will have to think about.

There are many instances where the compressed size is bigger than the original size, and even empty files are generated sometimes, so I don't see any benifit of disabling size check.

 

[pk2000]

Thanks for your input.

• It is not necessary to preserve RGB values in transparent areas. There may be some good reasons to do that for archival storage but this tool is for distribution, not archiving.
• In the first versions of YAC I used the -m 6 switch but saw very little benefit to justify the speed hit.
• Lossy 100% is already used with the "ultra" preset on list inputs. The custom mode set to 99% would give list output for everything is you want that. I don't want to have more presets, four is already pushing it for me.
• I'm not sure what the benefit to disabling the file size check would bring. Do you have an example use case where that'd make sense? Why would you want a compression tool to give you files bigger than what you started with?

If you can show me the benefit to a change I'll gladly implement it. As it stands, I want this to to be as simple as possible, so I have to weigh each new feature against how many more things my users will have to think about.

You are welcome
- I disagree, especially when overlapping transparent images RGB values can and will effect the final result; but also depends on how good the color perception of each viewer/player is and the dimension of the image.
- Still this should be for the user to decide. The same argument can be used between 4 and 5 and for all other settings.
- Are you sure? Because -lossless -exact is not lossy; and lossy 99% gives lower quality results than lossy 100% (the quality is more like 90%).
- Some may want format consistency or to convert in a few seconds/minutes faster (depends on the number of files that are checked, cpu speed,etc.)

As I said those suggestions were made for devs that use your tool and not for compressors.
(I would expect a dev to want his/her game to be distributed with the maximum possible quality even when compressed, but I could be wrong)

 

[hngg]

You are welcome
- I disagree, especially when overlapping transparent images RGB values can and will effect the final result; but also depends on how good the color perception of each viewer/player is and the dimension of the image.
- Still this should be for the user to decide. The same argument can be used between 4 and 5 and for all other settings.
- Are you sure? Because -lossless -exact is not lossy; and lossy 99% gives lower quality results than lossy 100% (the quality is more like 90%).
- Some may want format consistency or to convert in a few seconds/minutes faster (depends on the number of files that are checked, cpu speed,etc.)

As I said those suggestions were made for devs that use your tool and not for compressors.
(I would expect a dev to want his/her game to be distributed with the maximum possible quality even when compressed, but I could be wrong)

I'm sorry for my horrid spelling before. What I meant to say was that the ultra preset uses lossless mode on lossless inputs, and lossy 100% on lossy inputs. Using lossless compression on files that were already lossy compressed makes no sense. You won't get that quality back; you'll just end up with bigger files.

The file size check takes milliseconds; the risk of having bigger files after spending sometimes hours compressing is not something I'd inflict on my users. Not to save maybe 1% of encode speed.

Again, this tool is for distribution; not archiving. As long as the files work, they should be in the smallest/most efficient format possible. Believe it or not, PNG is actually more efficient on some edge cases! Plus, going from a 70%jpg to a 90%webp can sometimes give a bigger file AND lower quality so in that case It'd make more sense to just keep the jpg file. It shouldn't matter if the game uses jpg, webp, or png; as long as each individual file is as efficient as possible.

As far as the speed/quality switches, here's my promise; I'll add options for slower encoding once I figure out multithreading the image encoding. Then, the speed won't be impacted as much.

 

[pk2000]

I guess you meant not multi-threading (since you already have it enabled) but multi-task processing.

For that
1st you'll have to disable multithreading
2nd either disable the file-size check or mark/log the files already processed (otherwise you risk entering in loops)
3rd use a shell tool that allows parallel processing like Gnu Parallel or Rush

You must be registered to see the links

You must be registered to see the links

In the following link you can find some examples on how to use Gnu parallel

You must be registered to see the links

 

[lewdgamerxxx]

I guess you meant not multi-threading (since you already have it enabled) but multi-task processing.

For that
1st you'll have to disable multithreading
2nd either disable the file-size check or mark/log the files already processed (otherwise you risk entering in loops)
3rd use a shell tool that allows parallel processing like Gnu Parallel or Rush

You must be registered to see the links

You must be registered to see the links

In the following link you can find some examples on how to use Gnu parallel

You must be registered to see the links

Multi-Task Processing also uses multi-threading (as each thread is assigned to a task). And yeah, I think YAC uses multi-threading for audio and video processing but image processing is single threaded.
All of the three process are seperate from each other. And I think the most hngg can do is to create three seperate batch files for all of the three types (image, audio and video), configure multi-threading for them seperately, and then use a master batch script which will call them using the call function.
EDIT: I think that GNU parallel seems the easiest option right now. It only needs few changes in the current script.
This might be of some help:

You must be registered to see the links

 

[hngg]

News!
I've been working toward more parallelization of the script. To that end, I'm branching the development of this script now.

• For now, the 2.x builds will live on as a stable and predictable option for those who want it.
• For the more adventurous, I'm now introducing 3.x builds which will be taking things in a bit of a new direction.

Until now, my script has been a big, monolithic file that looks at one file at a time, crunches it, then keeps whichever version of the file is smaller. It's a simple concept and is reliable, but it's slow. If you have a bunch of CPU cores they will sit mostly idle while one image at a time compresses, even if they could be doing something useful (like crunching video!). With 3.x this will change.

Images, audio, and video will now be compressed in parallel; which should improve compression speed significantly. Eventually, I'd like to start compressing multiple images at the same time; I still haven't gotten that to work reliably(IE not breaking files, producing multiples, or skipping files) so that may have to wait for an eventual 4.x branch.

I've already re-worked a lot of the script and broken it into multiple pieces for future goodness (and my own sanity). Some parts are not working yet— like file size check — but I'm going to upload this first BETA build anyway since in my own testing it does at least crunch files(and quickly, too!). I hope to get all of the old 2.x features implemented soon, but I want to get to bug testing as well, so have at it guys!

Spoiler: How to use 3.x

You don't have permission to view the spoiler content. Log in or register now.

Download V3.0BETA2

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

Edit: BETA2 added, fixed some errors that caused files to be named incorrectly.

 

[lewdgamerxxx]

View attachment 1164415
News!
I've been working toward more parallelization of the script. To that end, I'm branching the development of this script now.

• For now, the 2.x builds will live on as a stable and predictable option for those who want it.
• For the more adventurous, I'm now introducing 3.x builds which will be taking things in a bit of a new direction.

Until now, my script has been a big, monolithic file that looks at one file at a time, crunches it, then keeps whichever version of the file is smaller. It's a simple concept and is reliable, but it's slow. If you have a bunch of CPU cores they will sit mostly idle while one image at a time compresses, even if they could be doing something useful (like crunching video!). With 3.x this will change.

Images, audio, and video will now be compressed in parallel; which should improve compression speed significantly. Eventually, I'd like to start compressing multiple images at the same time; I still haven't gotten that to work reliably(IE not breaking files, producing multiples, or skipping files) so that may have to wait for an eventual 4.x branch.

I've already re-worked a lot of the script and broken it into multiple pieces for future goodness (and my own sanity). Some parts are not working yet— like file size check — but I'm going to upload this first BETA build anyway since in my own testing it does at least crunch files(and quickly, too!). I hope to get all of the old 2.x features implemented soon, but I want to get to bug testing as well, so have at it guys!

Spoiler: How to use 3.x

You don't have permission to view the spoiler content. Log in or register now.

Download V3.0BETA2

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

Edit: BETA2 added, fixed some errors that caused files to be named incorrectly.

I don't know much coding, but I have an idea for size check. How about saving all compressed file with .temp extension (as in earlier version) in the same folder and after all files are compressed, initiate size check and rename/delete the .temp files accordingly??

 

[EdogawaKonan]

Menu item "MUTE" missed in 2.x and 3.x
How restore this?

 

[hngg]

Menu item "MUTE" missed in 2.x and 3.x
How restore this?

MUTE option has been added to V3 BETA3, just for you

You must be registered to see the links

 

[EdogawaKonan]

MUTE option has been added to V3 BETA3, just for you

You must be registered to see the links

Thanks wery much

 

[Circe21]

thx for your work. it's very usefull.
but somtimes when tiny png file get reduced , renpy game cant recognize it and makes error('Unsupported image format',)
i dont know why.

 

[Techstar]

How about extra menu item for html game without converting .gif as all

 

[hngg]

thx for your work. it's very usefull.
but somtimes when tiny png file get reduced , renpy game cant recognize it and makes error('Unsupported image format',)
i dont know why.

I haven't run into that yet. Can you give me the name of a game that I can test?

How about extra menu item for html game without converting .gif as all

I'll add that.

 

[Circe21]

I haven't run into that yet. Can you give me the name of a game that I can test?


I'll add that.

https://f95zone.to/threads/the-stargazers-lupiesoft.1263/
this is one of many. i dont know why it happens and how to solve

 

[hngg]

https://f95zone.to/threads/the-stargazers-lupiesoft.1263/
this is one of many. i dont know why it happens and how to solve

I think I found the issue. That game uses a very old version of RenPy. It will work if you upgrade the "lib" and "renpy" folders by running this .exe inside the game's folder(select "yes to all"). I've zipped and attached the file to this post.

 

[Circe21]

I think I found the issue. That game uses a very old version of RenPy. It will work if you upgrade the "lib" and "renpy" folders by running this .exe inside the game's folder(select "yes to all"). I've zipped and attached the file to this post. View attachment 1183062

Thanks. It works good.

but i found another one that coulnt works with your solution.

It says " An exception has occurred."

https://f95zone.to/threads/witch-trainer-final-akabur.763/

and these also dosent works

https://f95zone.to/threads/planet-stronghold-v1-5-5-final-winter-wolves.601/
https://f95zone.to/threads/amatas-journey-v2-1-nikraria.2554/
https://f95zone.to/threads/delirium-nikraria.3415/

and this one when i try to extract images.rpa file. it says "Could not extract file from archive: [Errno 2] the requested file. does not exist in the given Ren'Py archive"
https://f95zone.to/threads/the-menagerie-final-lupiesoft.4083/
https://f95zone.to/threads/tears-of-yggdrasil-moonstar.5228/
and this [Errno 22]
https://f95zone.to/threads/his-chuunibyou-cannot-be-cured-tamaya-kagiya.12851/
https://f95zone.to/threads/hikari-love-potion-final-toffer-team.36092/

and i dont know how to compress this type of renpy game. It says "Cannot locate python.exe"
https://f95zone.to/threads/happy-valentines-day-final-tusia-arashi-leonhart.34063/
https://f95zone.to/threads/to-love-ru-trial-trouble-v2-0-ninjagrass.63321/

 

[hngg]

Thanks. It works good.

but i found another one that coulnt works with your solution.

It says " An exception has occurred."

https://f95zone.to/threads/witch-trainer-final-akabur.763/

and these also dosent works

https://f95zone.to/threads/planet-stronghold-v1-5-5-final-winter-wolves.601/
https://f95zone.to/threads/amatas-journey-v2-1-nikraria.2554/
https://f95zone.to/threads/delirium-nikraria.3415/

and this one when i try to extract images.rpa file. it says "Could not extract file from archive: [Errno 2] the requested file. does not exist in the given Ren'Py archive"
https://f95zone.to/threads/the-menagerie-final-lupiesoft.4083/
https://f95zone.to/threads/tears-of-yggdrasil-moonstar.5228/

Thanks for the help! I'll take a look at those soon. For now I've been focusing on getting the 3.x branch stable and up to feature parity with 2.x.

In fact, today I am announcing YAC version 3.1, the first non-beta release in the 3.x branch. It brings with it a return of all the features in 2.1, plus some extras that have been requested recently (MUTE and .GIF skipping). I am still conducting my own testing but I feel it's ready for others to start putting it through its paces.

 

[Techstar]

Thanks for the help! I'll take a look at those soon. For now I've been focusing on getting the 3.x branch stable and up to feature parity with 2.x.

In fact, today I am announcing YAC version 3.1, the first non-beta release in the 3.x branch. It brings with it a return of all the features in 2.1, plus some extras that have been requested recently (MUTE and .GIF skipping). I am still conducting my own testing but I feel it's ready for others to start putting it through its paces.

YAC Version 3.1 Download

You must be registered to see the links

-

You must be registered to see the links

-

You must be registered to see the links

Going to test out, will give you feedback after testing.

 

[hngg]

Sorry guys, I uploaded the wrong version before.
I fixed an issue with all videos getting muted but I forgot to include the fix in v3.1.
v3.1b with the fix is up.