Guidance and feasibility of modifying Renp'y ffmpeg to support next-gen codecs(JpegXL/AVIF)

[mav99]

Jpeg XL seems about ready for prime time Firefox supports it now and it is pretty much superior to both WEBP and AVIF in every way. The onerous over-head of AVIF and its AV1 parent really make it a no go but JpegXL has rapidly improved and so too the point anyway to shoe-horn it it into Renp'y? Thanks

 

[MLocke]

Call me crazy, but you could do something convoluted to get brunsli WASM with a V8 JS engine running even in Python 2. Then, you can use the missing image callback to decode the jxl files and return something with im.Data. If doing everything with code is too hard, you could include brunscli executables for all platforms except maybe Android. I just read some code online though, and I don't have a real familiarity with the limitations of Renpy. The Python 3 port of Renpy may be happening in less than a year, and it's not that long to wait for such improvements to sort everything out.

 

[EternitysInfinite]

I'd just ask the Renpy developers on their github to add support. See

You must be registered to see the links

for a similar issue that you can base yours on.
I think they'd automatically gain support once their libraries update to support it, whenever that happens.

 

[MLocke]

It looks like they are deferring to the SDL2_image project. So, I suggest making the request there.

I don't know what you will make of this, but I actually got JXL images working by dropping in the Python wand module and ImageMagick shared libraries. I'll include my work at the end of the post, but please be careful about trying anything I describe out if you aren't experienced with Ren'py.

BTW, I confused Brunsli for something else because it was part of the larger JPEG-XL project. I should have said libjxl. Still, the WASM route is interesting because many related tools like libjxl, imagemagick, and squoosh are already being compiled to it. Python 2 is being phased out in January, so I think the Renpy devs are kind of being pushed into upgrading sooner rather than later. Python 3 has the wasmer module, so it's probably better to wait for that, especially when it comes to full cross-platform support.

I am not sure which imagemagick library file works for Android builds. Mac users can be told to use brew to install imagemagick. Linux has ImageMagick Easy Install if you wanted to put together the latest libs for Linux users. I included the windows ones that I downloaded from the official website.

Instructions: Extract libs.zip to renpy engine path so that it comes with the existing lib directory. Put jpeg_xl.rpy in the game folder.

You must be registered to see the links

Spoiler: jpeg_xl.rpy

You don't have permission to view the spoiler content. Log in or register now.

 

[f95zoneuser463]

I just requested JPEG XL support for SDL_image on GitHub.

You must be registered to see the links

... then I searched for 'JPEG XL' on F95zone and to my surprise I found this thread. Last time I checked I got nothing. Nice to see at least some people care about using efficient file formats.

There should really be no excuse to not implement this into SDL_image so that RenPy can inherit the support.
All mayor browsers have JPEG XL support already implemented - just not enabled yet by default.
A few days ago the ISO standard for the format was formally approved ... in the past so many devs used this as excuse to not implement this ... even though to format was pretty much final 1 year ago (bitstream frozen).

MLocke
The link to anonfiles in your post 404ed. Is this something you created or are these libs from magick? I'd like to try this.
I tried loading .jxl files in RenPy myself - but gave up after I found out it's SDL based. Reup the lib?

Please leave a comment or thoughts on GitHub to push this up in priority. I feel like this kind of stuff requires some convincing 'lobby' work.

Imagine 4K Visual Novels or 1080p at half the size ...

 

[MLocke]

I just requested JPEG XL support for SDL_image on GitHub.

You must be registered to see the links

... then I searched for 'JPEG XL' on F95zone and to my surprise I found this thread. Last time I checked I got nothing. Nice to see at least some people care about using efficient file formats.

There should really be no excuse to not implement this into SDL_image so that RenPy can inherit the support.
All mayor browsers have JPEG XL support already implemented - just not enabled yet by default.
A few days ago the ISO standard for the format was formally approved ... in the past so many devs used this as excuse to not implement this ... even though to format was pretty much final 1 year ago (bitstream frozen).

MLocke
The link to anonfiles in your post 404ed. Is this something you created or are these libs from magick? I'd like to try this.
I tried loading .jxl files in RenPy myself - but gave up after I found out it's SDL based. Reup the lib?

Please leave a comment or thoughts on GitHub to push this up in priority. I feel like this kind of stuff requires some convincing 'lobby' work.

Imagine 4K Visual Novels or 1080p at half the size ...

They were DLL files from ImageMagick but they were reorganized so that they could be dropped in / bundled with the game. If you use their Windows binary installer (usually the x64 version and it should be the installer with "dll" in the name as well), the script will still be able to find everything.

 

[Transmutator]

You must be registered to see the links

and

You must be registered to see the links

support was added in SDL image library

 

[f95zoneuser463]

I honestly did not expect this - usually my feature requests lead to nothing.

This is great news and a big step towards getting official support into RenPy!
I find it noteworthy to point who did add this to SDL.
It is 'slouken' aka

You must be registered to see the links

- an ex

You must be registered to see the links

programmer now working at Valve with an impressive history.
Very smart guy!

It is a shame that 99.99% of people do not understand the importance of JPEG XL for RenPy.
Even though the major part of a visual novels size comes from images usually.
If they would understand - this would be a pinned top post on the dev forum and there would be a feature request for it on the RenPy GitHub already.

One has to spell it out simplified for everybody understand:
DO YOU WANT YOUR RENPY GAME TO BE HALF THE SIZE FOR THE SAME QUALITY?
[ ] Yes ?
[ ] Yes ?!
[x] Yes !!!

With RenPy 8 in the works I think it's a good time to request JPEG XL support in RenPy now.
Seriously wtf! - I feel like a politician who has to do lobby work here to push this.
The beauty of open source software is that you can request new features even if you're not a coder.

 

[megalol]

I wonder will Ren'Py dev finally add support for avif (not rly much interested in JPEG XL coz when I've compared it with avif before and avif won the battle for me) images (and maybe av1 videos too) this year or not - I've created again

You must be registered to see the links

(it's an image library that used by Ren'Py and maintened by same dev) some time ago but he didn't answered yet...

 

[mav99]

killer feature of jxl is butteraugli lossless mode. Automatically calculated visually lossless mode. But yeah since Intel resurrected AV1 months ago since they were putting in hardware support with their ARC gpus they dumped huge resources into SVT-AV1 which went from a CPU fire-risk/joke compared to x265 to now beating it in pretty much every way. Incidentally I just compiled Libavif and Libjxl from yesterday 5/30/22 and low res avif does now beat jxl-it's very close but going lossless jxl still superior as with my 62 MP png test image jxl compressed to 6.5 MB lossless and AVIF was 7.7 MB. The image was 8.8 MB in max lossless compressed PNG. jxl visually lossless was 357 KB ha again AVIF has no mode so you would have to guess over and over to match that. So also Renp'y Dev should listen-I will put in a git and here is 100kb comparison of both-the jxl bt-lossless-and the original lossless PNG-to check out see your opinion again these are cutting edge libavif .10 compiled with SVT-AV1 and libjxl .70.(wow I went hardcore-nerd there but what can I do I find it fascinating) Try Nomacs as a viewer as Xnview dev still hasn't updated from libjxl.6 and NoMacs uses .7 so there ya go.

 

[mav99]

So on 6/20 the beta SDL_image release code/dlls were released on github with JXL and Avif support so here we go again-If god damn Renpy Tom is still asleep at the wheel can this be shoe-horned into Renpy8 Beta(py3)? I am going to work on it now Ill keep apprised of any progress.

 

[MLocke]

In my opinion, if you waited this long, you should wait a little longer to see if the Renpy team will pick up the changes once it's out of beta. Although, I think that there is a route with the PySDL2 python package (which says it works in pure python). It only has to load the SDL2_image module (DLL/so). That should be compatible with all platforms (even Android possibly) and it should work with all the variants of Renpy (as they move to v8).

 

[megalol]

Avif support finally was added in nighly builds about week ago and since it comes with libaom I was able to add AV1 (+ compiled HEVC support also) support too (in unreleased yet) v8.1 so if anybody wanna test it here are SDK links (60Mb)

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

P.S. Tutorial game images and video was converted to avif/AV1.

 

[mav99]

Thanks Mega! You have my gratitude for your great work! Now I have a 7900xt I for science just re-encoded Milfy City v71B and updated it to Renpy 8.1 encoded with SVT-AV1 141 for avif images and man is the AMF encoder trash so I did some hardware AV1 intro for example and went to HEVC for most. 3080 NVENC still miles ahead. avif images visually lossless -video vbr at 2k and overall size down from 4.6 GB to 1.7! with again only some video loss! Hope a hardware avif solution appears like NVJPEG which is like never used. So far AV1 really not blowing my doors off compared to HEVC especially this AMF encoder what are your thoughts? Ill post it over at MC thread

 

[megalol]

Thanks Mega! You have my gratitude for your great work! Now I have a 7900xt I for science just re-encoded Milfy City v71B and updated it to Renpy 8.1 encoded with SVT-AV1 141 for avif images and man is the AMF encoder trash so I did some hardware AV1 intro for example and went to HEVC for most. 3080 NVENC still miles ahead. avif images visually lossless -video vbr at 2k and overall size down from 4.6 GB to 1.7! with again only some video loss! Hope a hardware avif solution appears like NVJPEG which is like never used. So far AV1 really not blowing my doors off compared to HEVC especially this AMF encoder what are your thoughts? Ill post it over at MC thread

I'm looking forward to NVENC-AV1 in NVIDIA 40xx series and planning to buy such videocard this year. Not much fun of AMD AMF coz it already showed bad results with AMF+HEVC before so I'm not surprised that it's not very good with AV1 too.

 

[mav99]

I'm looking forward to NVENC-AV1 in NVIDIA 40xx series and planning to buy such videocard this year. Not much fun of AMD AMF coz it already showed bad results with AMF+HEVC before so I'm not surprised that it's not very good with AV1 too.

Mega it's fricking awful! no joke like my girlfriend bought this for Christmas for me so I kinda feel obligated to use it but that 3080 is calling my name-Tensor-RT/CUDA/NVENC! haha

Oh I just posted my AV1 version of Milfy City with your help check it out!

 

[megalol]

Mega it's fricking awful! no joke like my girlfriend bought this for Christmas for me so I kinda feel obligated to use it but that 3080 is calling my name-Tensor-RT/CUDA/NVENC! haha

Oh I just posted my AV1 version of Milfy City with your help check it out!

Ok, I'll try to check it in about 8h when I get home from work and I suggest u to swith to NVIDIA RTX 4080 if still possible (explain her that u needs/missing CUDA for ur work/hobby/etc) but if not there are still chances that AMD will improve AV1. Btw what tool did u used for AMF AV1 encoding: "Shutter Encoder" or maybe "FastFlix"? There are not much info about GPU AV1 encoding, for NVENC-AV1 I've found such interesting video

You must be registered to see the links

Upd: images looks fine (actually I'm not fan of lossless conversion, especially in this case where strong side of avif is very lossy compression at quality ~ 60% that should look like ~ 80% of quality for webp). If judging by main menu, video background has visible artifacts (look at clouds in window, it becames very pixelated sometimes) so either its poor AMF+AV1 implementation, bad encoding software and/or low quality/profile settings but certainly not RenPy problem coz in PotPlayer it looks the same as in it, compressed & attached (1st file) main menu video with UAGC (using 1 pass ffmpeg + SVT-AV1 encoder) and it looks much better than urs with even a bit lower size. NVENC+HEVC at speed 0 have worse quality than SVT-AV1 for me but maybe also coz my old NV 1060 GTX

You must be registered to see the links

You must be registered to see the links

in NVENC (urs 3080 RTX supported).
Btw looks like libvpx-VP9 at default speed (0) is quite similar to SVT-AV1 regarding quality at same bitrate but much slower.
P.S. regarding my modified SDK and RenPy updating as u maybe already know for now it's worth only for completed/abandoned games coz saves created on RenPy v8.1 nightly would not work on older versions. Also there are no guarantee that updating RenPy to v8.1 would not break some scripts compatibility. I've added in UAGC v2.2.4 separate tool that would do RenPy v8.1 game building/updating and compressing video/images to AV1/avif.

More info about SVT-AV1 commands

You must be registered to see the links

and

You must be registered to see the links

.

You must be registered to see the links

).

 

[megalol]

Avif support finally was added in nighly builds about week ago and since it comes with libaom I was able to add AV1 (+ compiled HEVC support also) support too (in unreleased yet) v8.1 so if anybody wanna test it here are SDK links (60Mb)

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

P.S. Tutorial game images and video was converted to avif/AV1.

Uploaded newer SDK, nothing changed - only SDK update (minus included demo games so now it has smaller size).

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links