Tool Ren'Py UnRenGUI, UnRen-forall(v9.4), UnRen-Powershell-forall(v9.4), UnRen-old

[⎰ ↜ ✰ BAD 75 ✰ ↜ ⎱]

Hello everyone
I am not an expert on unren
But I follow this thread because I use it everyday
but I am stuck on version 7 because in more than 100 games where I have done extraction and the only one that didn't give me any problem
for my own convenience i just translated unren in my language
i have seen multiple errors being reported here
a few times out of curiosity I downloaded the games mentioned that were giving problems and 95% of the time I extracted them without problems
as I did now with Another Chance
so I wonder if maybe that version is the most complete version
I turn to you who are more experienced
thanks

 

[VepsrP]

Hello everyone
I am not an expert on unren
But I follow this thread because I use it everyday
but I am stuck on version 7 because in more than 100 games where I have done extraction and the only one that didn't give me any problem
for my own convenience i just translated unren in my language
i have seen multiple errors being reported here
a few times out of curiosity I downloaded the games mentioned that were giving problems and 95% of the time I extracted them without problems
as I did now with Another Chance
so I wonder if maybe that version is the most complete version
I turn to you who are more experienced
thanks View attachment 2200655

I still didn't understand what exactly your problem is, but here's what confused me: the progress.rpyc file weighs 12 kilobytes, while progress.rpy weighs 32. Decompiled files weigh less than compiled ones. A rather strange problem that appeared only in one file.
I can only say that sirpedro7 has already made about 100 translations of games into Spanish, including using my version of the tool for unpacking and decompiling.

 

[⎰ ↜ ✰ BAD 75 ✰ ↜ ⎱]

I still didn't understand what exactly your problem is, but here's what confused me: the progress.rpyc file weighs 12 kilobytes, while progress.rpy weighs 32. Decompiled files weigh less than compiled ones. A rather strange problem that appeared only in one file.
I can only say that sirpedro7 has already made about 100 translations of games into Spanish, including using my version of the tool for unpacking and decompiling.

maybe I explained myself wrongly
mine was just curiosity since I use your V7 version and have had no problems with decompialation
and that's why I was asking if it's better to change version for me

 

[VepsrP]

maybe I explained myself wrongly
mine was just curiosity since I use your V7 version and have had no problems with decompialation
and that's why I was asking if it's better to change version for me

Well, it depends on the circumstances or your desire. The new versions have fixes not only in terms of decompiling but also unpacking. It's pretty strange that you didn't have any problems in the same games, considering how many versions have already been released after. In any case, do what you think is best.

 

[⎰ ↜ ✰ BAD 75 ✰ ↜ ⎱]

Well, it depends on the circumstances or your desire. The new versions have fixes not only in terms of decompiling but also unpacking. It's pretty strange that you didn't have any problems in the same games, considering how many versions have already been released after. In any case, do what you think is best.

I will also try the 'latest version and as soon as I have time I will compare the extractions to see if there are differences
indeed many versions have passed
but never had a problem with any renpy version
congratulations again for your outstanding work

 

[VepsrP]

Will not decompile .rpyc files for this game Here

The files were decompiled calmly. There is only one error in the code, but it cannot be fixed automatically, because the roots come from the compiled file itself. Probably a compilation error.

 

[Madeddy]

Will not decompile .rpyc files for this game Here

I did try it with my version of the decompiler and also not encountered problems:
...here are 900+ lines more of this...

Decompiling /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/phone_apps/contacts.rpyc to /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/phone_apps/contacts.rpy...
Decompiling /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/bird/bird.rpyc to /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/bird/bird.rpy...
Decompiling /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/engine/quick_start.rpyc to /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/engine/quick_start.rpy...
Decompilation of 942 script files successful.

Decompiled files weigh less than compiled ones.

That's a no-no.
Compiling and pickling reduces normally the data-size.
- comments are removed
- data is binary; text takes always more space
- the whole goal of serialization is space saving and fast data transfer; its in a way like compression but not really
See:

You must be registered to see the links

Some stuff cannot be pickled, so its related to the content if we save space or even use more space as the original plain text. Renpy is there very wasteful i think. I mean, the Renpy devs limit it to pickle protocol 2, while we have now 5(!) available.

A rather strange problem that appeared only in one file.

Sorry, but also incorrect. You accidentally overlooked some files... or in my test its different.
In the game mentioned i have seen over 100 file pairs where the rpy bigger is as the rpyc. For many of them its very little, like barely 10%. Others are more as double the size of the compiled one. e.g. in .../1000_base_game/game/quests/act_one.* and others...

Overall its true what you said if you have Renpy games. Most of the compiled ones are strangely bigger as the text source. Why ever ...?

 

[VepsrP]

I did try it with my version of the decompiler and also not encountered problems:
...here are 900+ lines more of this...

That's a no-no.
Compiling and pickling reduces normally the data-size.
- comments are removed
- data is binary; text takes always more space
- the whole goal of serialization is space saving and fast data transfer; its in a way like compression but not really
See:

You must be registered to see the links

Some stuff cannot be pickled, so its related to the content if we save space or even use more space as the original plain text. Renpy is there very wasteful i think. I mean, the Renpy devs limit it to pickle protocol 2, while we have now 5(!) available.

Sorry, but also incorrect. You accidentally overlooked some files... or in my test its different.
In the game mentioned i have seen over 100 file pairs where the rpy bigger is as the rpyc. For many of them its very little, like barely 10%. Others are more as double the size of the compiled one. e.g. in .../1000_base_game/game/quests/act_one.* and others...

Overall its true what you said if you have Renpy games. Most of the compiled ones are strangely bigger as the text source. Why ever ...?

Well, I assumed that the compiled file stores the entire ast tree. Given that there is much more information in it than in a regular code file, more weight after compilation is reasonable. But the reverse option I met for the first time.
Due to the fact that there is only one file, I only talked about what I saw in the screenshot, I did not investigate this case in the game itself.
By the way, did the game start normally after decompilation? One of Isabella's files was not decompiled correctly by me, outputting at transform: and parallel: on the same line.

 

[Madeddy]

Well, I assumed that the compiled file stores the entire ast tree. Given that there is much more information in it than in a regular code file, more weight after compilation is reasonable. ...

Yeah, like said: I think its the data content of your text source. If you have many stuff thats not pickleable and use a old pickle protocol your get bigger compiled files. You could run a test(just for fun) with the different pickle protocols and see what happens.

Byy the way: I didn want to come across like a "i know better". Just wanted to be helpfully clear a missunderstanding to avoid problems from it.

...did the game start normally after decompilation? One of Isabella's files was not decompiled correctly by me, outputting at transform: and parallel: on the same line.

I did not test the game. But you're right, i have the the same code mistake.
Reappearance of a known bad boy, unfixed:

You must be registered to see the links

You must be registered to see the links

If you did read the issues, you noticed he points to the

You must be registered to see the links

" code part as the culprit. But i am not sure, because other stuff after this statement is correctly placed. The same goes for the

You must be registered to see the links

. In other places its on the next line...

Well, seeing the main dev don't doing any bugfixing at the moment... i guess its on us again. Oh so much fun with the headache.

 

[VepsrP]

Oh so much fun with the headache.

Oh, I already have. And I haven't even started. I had enough problems at work, I'm afraid to imagine what would happen if I added this too.

 

[VepsrP]

Yeah, like said: I think its the data content of your text source. If you have many stuff thats not pickleable and use a old pickle protocol your get bigger compiled files. You could run a test(just for fun) with the different pickle protocols and see what happens.

Byy the way: I didn want to come across like a "i know better". Just wanted to be helpfully clear a missunderstanding to avoid problems from it.
I did not test the game. But you're right, i have the the same code mistake.
Reappearance of a known bad boy, unfixed:

You must be registered to see the links

You must be registered to see the links

If you did read the issues, you noticed he points to the

You must be registered to see the links

" code part as the culprit. But i am not sure, because other stuff after this statement is correctly placed. The same goes for the

You must be registered to see the links

. In other places its on the next line...

Well, seeing the main dev don't doing any bugfixing at the moment... i guess its on us again. Oh so much fun with the headache.

                                    <renpy.sl2.slast.SLBlock
                                        .atl_transform = <renpy.atl.RawBlock
                                            .animation = False,
                                            .loc = (
                                                <nicode>,
                                                9
                                            ),
                                            .statements = [<renpy.atl.RawParallel
                                                .loc = (
                                                    <nicode>,
                                                    9
                                                )
                                            >]
                                        >,

Actually, that's the reason. As you can see, both constructs logically were on the same line when this code was compiled, but this should not be. I don't know how to get around it yet.

 

[Madeddy]

Actually, that's the reason. As you can see, both constructs logically were on the same line when this code was compiled, but this should not be. I don't know how to get around it yet.

Well... isn't it interesting. How do you print your AST version out? I use just the dump from unrpyc...
I sit also for hours on this and go through the line-numbers and put some control prints in there:

Decompiling /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpyc to /home/olli/.x
lib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy...
LINE at_transform after write: 19
LINE callback begin: 19
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 9)
LINENUM parallel beg: 19
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 10)
LINENUM parallel end: 23
LINENUM parallel beg: 23
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 15)
LINENUM parallel end: 26
LINE callback end: 26
LINE at_transform after write: 33
LINE callback begin: 33
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 25)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 26)
LINE callback end: 37
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 316)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 471)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 477)
Decompilation of 1 script file successful.

This looks perhaps a bit confusing for the moment... but the point is, the ast loc(=linenumber) is in other cases always after the linenum positions before them. Here is the linenum completely gone crazy and is more and more lines ahead of the ATL. I think its accidental or luck he prints the ATL nodes then after the next colon instead several lines behind the current line of the parent elements like "at transform".
I dont know if i managed to explain understandable, so in short: self.linenumber jumps ahead of the linenumber in ast.location

And by the way... my AST dump is slightly different from yours:

Spoiler: AST code...

You don't have permission to view the spoiler content. Log in or register now.

Its time:

 

[VepsrP]

Well... isn't it interesting. How do you print your AST version out? I use just the dump from unrpyc...

I also used a dump, I just cut it a little, leaving the most important.

I sit also for hours on this and go through the line-numbers and put some control prints in there:

Decompiling /home/olli/.xlib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpyc to /home/olli/.x
lib/RPG/_test/AnotherChance-v1.24-pc/game/rpakit_out/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy...
LINE at_transform after write: 19
LINE callback begin: 19
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 9)
LINENUM parallel beg: 19
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 10)
LINENUM parallel end: 23
LINENUM parallel beg: 23
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 15)
LINENUM parallel end: 26
LINE callback end: 26
LINE at_transform after write: 33
LINE callback begin: 33
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 25)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 26)
LINE callback end: 37
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 316)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 471)
AST loc: ('game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy', 477)
Decompilation of 1 script file successful.

This looks perhaps a bit confusing for the moment... but the point is, the ast loc(=linenumber) is in other cases always after the linenum positions before them. Here is the linenum completely gone crazy and is more and more lines ahead of the ATL. I think its accidental or luck he prints the ATL nodes then after the next colon instead several lines behind the current line of the parent elements like "at transform".
I dont know if i managed to explain understandable, so in short: self.linenumber jumps ahead of the linenumber in ast.location

And by the way... my AST dump is slightly different from yours:

                   <renpy.sl2.slast.SLDisplayable
                        .child_or_fixed = False,
                        .children = [<renpy.sl2.slast.SLIf
                            .entries = [
                                (
                                    <renpy.ast.PyExpr
                                        .filename = 'game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy',
                                        .linenumber = 7,
                                        .py = 3
                                    > = 'note_visible',
                                    <renpy.sl2.slast.SLBlock
                                        .atl_transform = <renpy.atl.RawBlock
                                            .animation = False,
                                            .loc = (
                                                'game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy',
                                                9
                                            ),
                                            .statements = [<renpy.atl.RawParallel
                                                .blocks = [
                                                    <renpy.atl.RawBlock
                                                        .animation = False,
                                                        .loc = (
                                                            'game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy',
                                                            10
                                                        ),
                                                        .statements = [
                                                            <renpy.atl.RawMultipurpose
                                                                .circles = '0',
                                                                .duration = '0',
                                                                .expressions = [],
                                                                .loc = (
                                                                    'game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy',
                                                                    10
                                                                ),
                                                                .properties = [(
                                                                    'alpha',
                                                                    <renpy.ast.PyExpr
                                                                        .filename = 'game/1000_base_game/game/characters/isabelle/q_isabelle_piano.rpy',
                                                                        .linenumber = 10,
                                                                        .py = 3
                                                                    > = '0.0'
                                                                )],
                                                                .revolution = None,
                                                                .splines = [],
                                                                .warp_function = None,
                                                                .warper = None
                                                            >,

Its time: View attachment 2204903

Well, I kind of got the idea. It turns out that ast loc should have been printed first, and then linenum. Probably.

 

[Madeddy]

... It turns out that ast loc should have been printed first, and then linenum. Probably.

We where wrong i think. Total. I believe i figured nearly everything out. Nearly... Also, i don't know if these strangeness with the linenumbers and AST.loc cause some problems at some place, but its not our culprit. I did sit into the night nearly till morning for this shit.

I hope i explain understandable:
Well.. The reason is a call from the method which prints "at transform:" to the method self.print_atl_callback()(inside another method in the Decompiler class in __init__.py). Now, we have a bool object called self.skip_indent_until_write and inside method indent()(in util.py) its checked. Only if False does the indentation code run. NOTE: The method indent() does also a newline ...

And this object is somehow set to True as the method print_atl_callback() is called, or shortly after, and the indent inside the method which prints "parallel:" is not executed. Question is now what the heck activates the skip_indent_until_write or rather where? Its easy to find the places where we have skip_indent_until_write = True, but which one and why? These lines are somewhere else and should not be touched. "Should not"!

I think it could be from the different instances. Somehow another instance gets dragged in and there is the state of skip_indent_until_write set to true. Or some other method with a skip_indent_until_write = True gets somewhere called and i missed this.

Its easy to put a self.skip_indent_until_write = False into parallel directly before the indent() call, which fixes it actually. But thats just a HACK and not a solution. Its happened with other code like "on show"also and we would need another HACK. And another... and... And then is the question what happens if its needed to skip the indent in some cases. Right, we broke the other thing with the hack. Great.

 

[VepsrP]

We where wrong i think. Total. I believe i figured nearly everything out. Nearly... Also, i don't know if these strangeness with the linenumbers and AST.loc cause some problems at some place, but its not our culprit. I did sit into the night nearly till morning for this shit.

I hope i explain understandable:
Well.. The reason is a call from the method which prints "at transform:" to the method self.print_atl_callback()(inside another method in the Decompiler class in __init__.py). Now, we have a bool object called self.skip_indent_until_write and inside method indent()(in util.py) its checked. Only if False does the indentation code run. NOTE: The method indent() does also a newline ...

And this object is somehow set to True as the method print_atl_callback() is called, or shortly after, and the indent inside the method which prints "parallel:" is not executed. Question is now what the heck activates the skip_indent_until_write or rather where? Its easy to find the places where we have skip_indent_until_write = True, but which one and why? These lines are somewhere else and should not be touched. "Should not"!

I think it could be from the different instances. Somehow another instance gets dragged in and there is the state of skip_indent_until_write set to true. Or some other method with a skip_indent_until_write = True gets somewhere called and i missed this.

Its easy to put a self.skip_indent_until_write = False into parallel directly before the indent() call, which fixes it actually. But thats just a HACK and not a solution. Its happened with other code like "on show"also and we would need another HACK. And another... and... And then is the question what happens if its needed to skip the indent in some cases. Right, we broke the other thing with the hack. Great.

Actually, I found a place where is the problem due to which parallel is not printed from a new line, but I'm not sure that it is there that the code needs to be changed. Rather, we need to find a place where skip_indent_until_write should become false.
super(Decompiler, self).dump(ast, indent_level, skip_indent_until_write=True) : 124 __init__.py
If you replace it with false here, then decompilation is normal, but it may affect other files.

 

[Madeddy]

If you replace it with false here, then decompilation is normal, but it may affect other files

Exactly. Thats what i meant with the talk about a hack. And i have my eye also for some time on the Decompiler.dump code part you cited.... it could be the reason.
I did now the following:
I moved the hack line self.skip_indent_until_write = False from the print_atl_rawparallel() method step for step backwards through the code in direction of the "at transform:" code part(if atl_transform is not None at the end of sl2decompiler. From parallel to print_atl and his sub calls and now i am inside print_atl_callback().

    def print_atl_callback(linenumber, indent_level, atl):
        # control print
        print(f"INDENT skipinside atl callback:"
              f" {self.skip_indent_until_write}")
        self.skip_indent_until_write = False
        ...

I wanted to see where exactly its set to true.

        # end of sl2decompiler.py
        self.write("at transform:")
        # control print
        print(f"LINE at_transform after write: {self.linenumber}"
              f" INDENT skip in transform: {self.skip_indent_until_write}")
        self.linenumber = self.print_atl_callback(
            self.linenumber, self.indent_level, atl_transform)

Try this. You see the value is False after "at transform" and before the last codeline(!) and True at the begin of method print_atl_callback. The Decompiler instance with the dump() method gets dragged in there i'd say. Like you posted up there.

So, how to fix this? Deactivating it in dump() could produce unwanted outcomes.

 

[Madeddy]

Very interesting: Search inside for print_atl_callback. (File is unpacked a bit bigger; don't get scared... its just text)

 

[VepsrP]

Exactly. Thats what i meant with the talk about a hack. And i have my eye also for some time on the Decompiler.dump code part you cited.... it could be the reason.
I did now the following:
I moved the hack line self.skip_indent_until_write = False from the print_atl_rawparallel() method step for step backwards through the code in direction of the "at transform:" code part(if atl_transform is not None at the end of sl2decompiler. From parallel to print_atl and his sub calls and now i am inside print_atl_callback().

    def print_atl_callback(linenumber, indent_level, atl):
        # control print
        print(f"INDENT skipinside atl callback:"
              f" {self.skip_indent_until_write}")
        self.skip_indent_until_write = False
        ...

I wanted to see where exactly its set to true.

        # end of sl2decompiler.py
        self.write("at transform:")
        # control print
        print(f"LINE at_transform after write: {self.linenumber}"
              f" INDENT skip in transform: {self.skip_indent_until_write}")
        self.linenumber = self.print_atl_callback(
            self.linenumber, self.indent_level, atl_transform)

Try this. You see the value is False after "at transform" and before the last codeline(!) and True at the begin of method print_atl_callback. The Decompiler instance with the dump() method gets dragged in there i'd say. Like you posted up there.

So, how to fix this? Deactivating it in dump() could produce unwanted outcomes.

I tested this on the rest of the game files. The game started without any problems. However, it is unknown whether something will appear during the progress. I'm also downloading the latest version of SummerTime Sega to test it. It was in this game that I periodically had problems similar to the current one. I'll see how the decompilation will go now.

 

[VepsrP]

Very interesting: Search inside for print_atl_callback. (File is unpacked a bit bigger; don't get scared... its just text)

It's lovely, but in my opinion, too detailed.

 

[VepsrP]

Exactly. Thats what i meant with the talk about a hack. And i have my eye also for some time on the Decompiler.dump code part you cited.... it could be the reason.
I did now the following:
I moved the hack line self.skip_indent_until_write = False from the print_atl_rawparallel() method step for step backwards through the code in direction of the "at transform:" code part(if atl_transform is not None at the end of sl2decompiler. From parallel to print_atl and his sub calls and now i am inside print_atl_callback().

    def print_atl_callback(linenumber, indent_level, atl):
        # control print
        print(f"INDENT skipinside atl callback:"
              f" {self.skip_indent_until_write}")
        self.skip_indent_until_write = False
        ...

I wanted to see where exactly its set to true.

        # end of sl2decompiler.py
        self.write("at transform:")
        # control print
        print(f"LINE at_transform after write: {self.linenumber}"
              f" INDENT skip in transform: {self.skip_indent_until_write}")
        self.linenumber = self.print_atl_callback(
            self.linenumber, self.indent_level, atl_transform)

Try this. You see the value is False after "at transform" and before the last codeline(!) and True at the begin of method print_atl_callback. The Decompiler instance with the dump() method gets dragged in there i'd say. Like you posted up there.

So, how to fix this? Deactivating it in dump() could produce unwanted outcomes.

I tested with SummerTime Saga. There was only one error - unknown encoding cp866. The only solution I could find was unicode decoding.
Also available is another displayable_name.
I solved the problem with connecting additional python libraries on old engines (7.3-) in the most banal way - by duplicating lines of code. This was required because the game runs on this engine.
After that, I tested it on a game with an 8 engine and got a new error in the Revertable section - now it's AtributeError.
The most interesting thing is that I have previously decompiled this game and there was no such error, why it got out now is not clear.
All changes can be viewed in the last comment.