Reducing size of renders

[Agent HK47]

@

You must be registered to see the links

Ah, now I understand what you mean. I thought you were expecting me to upload my entire render library for the game when it was finished. I have to go on an errand, but I will be back soon and will upload a couple of my renders, so you can get some hands on experience.

Also, I have studied reviews and people who has used the progam for months before settling on it. I wouldn't use it if I felt it wouldn't be able to do the job ^^

That being said, I will almost certainly switch to Ren'py at some point, I just dont know when. However, it will probably not be for this game.

 

[Agent HK47]

The idea is that most if not all rendering tools are producing big unoptimized files regardless of the format you choose and you have to do the optimization afterward - that's why I've asked you to upload a sample we can use to test optimizations with.

VNM is a bad idea at the moment, you should check

You must be registered to see the links

and the problems it's authors and users had with VNM ...

Alright, here are 3 of the test renders. If you can find a way to reduce them a bit in filesize, let me know. Also, I know that the lighting isn't optimal at all, but I haven't really messed around with lights much.

 

[gue5t]

If you haven't bought VNM yet I'd suggest you download the pirate copy and test it first.

On the render side I'd suggest you read the

You must be registered to see the links

topic because it'll help with the quality and the size reduction of the renders.

I'll report on the size reduction in a while but I expect ~10% from my experience so far.

[EDIT]
Lossless optimization resulted in ~13% size reduction.
Lossy optimization resulted in ~70% size reduction and less than 5% quality loss.

 

[NandabaCanti]

I recently came across a really nice utility:

You must be registered to see the links

I run it with just the --quality=high flag and get files out that look identical but end up like 10-30% of their original size! Obviously this isn't much help if you need transparency, in which case you would need to look into lossy PNG compressors if you want to get nice and big file size reductions on that front. It has been ages since I last messed around with this, so I can't tell you what would end up being ideal settings, but you can look into something like:

You must be registered to see the links

 

[gue5t]

I recently came across a really nice utility:

You must be registered to see the links

I run it with just the --quality=high flag and get files out that look identical but end up like 10-30% of their original size! Obviously this isn't much help if you need transparency, in which case you would need to look into lossy PNG compressors if you want to get nice and big file size reductions on that front. It has been ages since I last messed around with this, so I can't tell you what would end up being ideal settings, but you can look into something like:

You must be registered to see the links

You do realize that both utilities are throwing away a lot of information to fit the image into a small file?
And to recommend using this on photos ... I'm speechless as an amateur photographer and don't even want to imagine the reaction of a pro on the jpeg-archive author's claims. Keep in mind that the average human eye can't easily notice "small" difference in colors especially when they are surrounded by similar colors unless they are too different. That's why we use software to point and measure them out. Here is an interesting article you might want to red:

You must be registered to see the links

You can easily test that by opening one of the images from

You must be registered to see the links

and comparing it with the same image from

You must be registered to see the links

then you can open the corresponding DIFF.PNG file and try to locate the difference in color in one of the blue pixels in it between the two previous images.

P.S. I've used pngquant as one of the tools while making Lossy file.

 

[NandabaCanti]

I'm not suggesting using them to clobber your source files, you would run them as a last step to produce the versions you push into your game. You keep the originals around untouched for backup purposes.

 

[Agent HK47]

If you haven't bought VNM yet I'd suggest you download the pirate copy and test it first.

On the render side I'd suggest you read the

You must be registered to see the links

topic because it'll help with the quality and the size reduction of the renders.

I'll report on the size reduction in a while but I expect ~10% from my experience so far.

[EDIT]
Lossless optimization resulted in ~13% size reduction.
Lossy optimization resulted in ~70% size reduction and less than 5% quality loss.

Yeah, I read something about the lighting affecting the grain, which was already on my list of things I was going to look into, but like I already explained, I haven't played around too much with lighting. So, if anyone has any tips on how to do outdoor/indoor lighting pretty easily, feel free to let me know.

As for the renders... I would be able to live with the 5% loss in quality, in return for the 70% reduction on size, but only if it doesnt result in further degradation of the image down the line. The reason I like PNG is that it retains quality no matter how many times you screw around with it, which is something jpg can't do.
Would you say that the lossy option is worth it? If it comes down to having to choose between quality and size, I will definately choose quality, and go for the lossless optimization.

 

[gue5t]

If you prefer quality your best bet is using Lossy png files and re-rendering the scenes each time you wish to change something in them because once you go the Lossy way there is no turning back and each time you change something in the image there is going to be quality loss like with the jpg files.

As for the light issues I recommended that topic because people more knowledgeable than me have given there advice in it.

 

[NandabaCanti]

I'd say end users have no need of high quality lossless images, unless you intend to release your assets as a pack to be used by other game devs for their projects, which I am guessing most here would not want. As such, jpeg is just fine for final output you package with your releases (as long as you don't need transparency). Any time you want to change things down the line, work off of your original renders saved in your image editor of choices native format, so you are always working with the best quality. But once you are done, re-export a fresh jpeg or png (if you need transparency) and clobber them with lossy compressors as much as you can get away with to reduce the final file sizes. This way we can stop wasting multiple GB on games that can easily be a few hundred MB.

 

[gue5t]

@uradamus I fully support the idea of not wasting space but it's quite obvious you have not dealt with a few gigs of images you have to optimize. No offense intended but considering the quality lost to save the space I don't think its worth it + once @Agent HK47 fixes the grain problem the png file size will go further down.

P.S. Each red pixel in the DIFF files is the cost of the size reduction. The JPG files ware made at 95% but the quality loss goes up to ~20. Also Motorhome chill.png does contain transparency so the saved space by it shouldn't be counted.

 

[NandabaCanti]

That jpeg-archive package offers a multi-threaded batch processing option (assuming you install the extra dependency to support that feature). So it can chew through a ton of images in very little time. Even if you don't take advantage of that, it is fairly trivial to write a script to batch process whole directory trees. (In fact I am pretty sure I've wrote several in the past in both Bash and Python for similar tasks.)

Even if you don't want to go through the extra final compression step, photos with lots of complex gradients - such as your typical Daz renders - will pretty much always be significantly smaller as JPEGs than their PNG equivalent. So backgrounds and CGs that are full screen, with no transparency needs, should still be output as jpeg. It is best to reserve PNG for the things it's particularly good at: UI elements, anything with text or lots of sharp edges you need to keep crisp, flat shaded graphics, things with smooth predictable gradients and anything that needs transparency.

Here's a good read comparing JPEG and PNG. Even though it focuses on web use cases, it's basically just as applicable here when it comes to determining the best format for a given use case to get the smallest files.

You must be registered to see the links

I should note however, if you do plan to use something like jpeg-archive, it is best to keep your initial JPEG output fairly high quality (like 90+%). The better the source quality, the better the results will be. Though as you can see in the above link, even just going with a fairly drastic 60% quality setting using your typical exporter, you will still end up with a rather high quality image that is a fraction of the size.

 

[gue5t]

Nice article but it's oversimplified and doesn't even mention the possibility of optimizing the files before comparing them.

Original Files:
flower-jpeg.jpg 66 264 bytes
flower-png.png 412 666 bytes

Optimized Files:
flower-jpeg.jpg 63 393 bytes
flower-png.png 388 388 bytes
flower-lossy.png 187 705 bytes
flower-95%.jpg 94 056 bytes

And here are the DIFF images:

Original PNG vs Original JPG^

Original PNG vs JPG@95%^

Original PNG vs Lossy PNG^

P.S. Attachment contains optimized files + DIFFs, original files can be downloaded from authors site in the comments.

 

[NandabaCanti]

I really don't know why you are so obsessed over the diff maps. Of course there will be real differences between a lossy compressed image and its original, but in most cases, those red dots represent changes so small your eye can't perceive them on its own, tiny little changes made to allow the image to compress better. For whatever reason you view them as practically an unforgivable sin against art, while I view them as welcome changes when they mean big savings in download times and storage use with no perceived loss of quality. As long as you don't push things to the point of noticeable compression artifacts, lossy compression is a great and desirable thing and all those diff maps prove is that the compressors are doing their job.

 

[gue5t]

The black dots mean that nothing has changed, blue dots mean the change is below tolerance level and the red dots mean it's above tolerance level... in the Original PNG vs JPG@95% we have differences up to 20 which means you can't trust the value used as quality criteria and have to verify each picture for quality loss thus the image diffs. They are a tool to easily display the location and range of the differences in the images so the observer can focus on the changes instead of on the whole image and decide if the quality loss is acceptable or not.

P.S. If you push the compression level to the point where you have visible artifacts you have went way above acceptable levels of compression.

 

[NandabaCanti]

As a practical example of what jpeg-archive is capable of, here is a PNG background ripped from a game and a JPEG I generated with the most extreme compression settings jpeg-recompress had to offer (--quality low --method smallfry). I am sure if you generate a diff map of it, it will light up like a christmas tree, but to the naked eye they are virtually indistinguishable. The PNG weighs in at ~3200 KB while the JPEG is only ~240 KB.


EDIT: For the sake of completeness, here is also a JPEG generated with just the --accurate flag if you don't mind jumping up a fair bit in size for minor gains in detail (~675 KB).

 

[gue5t]

No offense but I can easily spot the artifacts in both .jpg files, could you zip all 3 files and attach them because the forum is known to modify the images uploaded to it and we can't reliably use the files in your post for comparison because they ware probably damaged by the lossy optimizations used when uploading them here.

 

[NandabaCanti]

There is a very slight (and mostly insignificant) quality loss, mainly in the form of slightly softer details in the background foliage and the brightest highlights in the water are ever so slightly dulled. Those are pretty much the only really noticeable differences and they are slightly more pronounced in the first really small JPEG. IMO, those little differences are well within the realm of acceptable quality loss for the huge savings in size; something like ~93% compression and yet at a quick glance they look pretty much exactly the same.

With the zip, I've also tossed in the PPM version I generated with ImageMagick from the PNG in order to run it through the utility to get the JPEGs.

 

[gue5t]

PNG vs bg backyard_daytime.jpg is horrid I went step by step up to 50 tolerance level and there was plenty of red so I went directly to 80 and still there ware red pixels + I can see the differences (nowhere near as pronounced as the images here but still noticeable - the forum resized them down to 1600x900px)...

PNG vs bg backyard_daytime-acc.jpg is better I went step by step to 50 tolerance and there was still red - I had to go to 60 before the red pixels disappeared but the differences are difficult to notice ...

PNG vs bg backyard_daytime_lossy.png on the other hand is blue at 3 tolerance level and I can't spot the differences even when looking at the exact pixel at 500% zoom, the drawback is the size (1 557 275 bytes) but that's the situation each author need's to find the balance of quality vs size he/she can live with.

P.S. I have trained eyes and calibrated monitor so noticing the color differences is easier and more noticeable for me - I guess that's an occupational hazard or whatever the correct English term for it is ...

 

[Studio Errilhl]

If you're concerned about file-sizes, my suggestion would be (if your engine supports them, Renpy does, for instance) to use .webp - it's lossless compression by default, but can be set to whatever lossy compression you would like (depending on the converter-program used). You need a codec to be able to see them in Windows explorer, apart from that they work wonders, and can cut your game down by more than 50% (depending on how much of the size is images).

.webp is, to me, mostly the same as the original .pngs. I've attached a file with a .png file, and its converted .webp counterpart.

 

[gue5t]

@Studio Errilhl I don't know what settings are you using for the compression but I can tell you outright that your numbers are wrong ...