Tool Ren'Py UnRen.bat v1.0.11d - RPA Extractor, RPYC Decompiler, Console/Developer Menu Enabler

[eike2000]

thanks, finally

 

[creedgeezer]

creedgeezer taki :::
There are like 20 possible error reasons... How the heck should we find perhaps out, if you do NOT tell us stuff like which RenPy version the game you did try to work on runs or which target game (name) or which UnRen version you used?
Please add more infos.

Oh...Sorry my bad

Game: Deadlock in Time
Ren'py vr: 7.3.5.6060

Unren vr: 0.91

Problem: everything works except the rpyc decompilation..If i pick the option 2(decompile rpyc) it quits immediately....

 

[Madeddy]

Thanks for the additional info.
I guess the game name is "Deadlocked in Time". Just make sure i have the right one...

..If i pick the option 2(decompile rpyc) it quits immediately....

That statement is not really right. I get this:



And even the message is correct.
You need first to unpack the .rpa files to get the games scripts(.rpyc files...), then you can decompile. Some games like this have no/few files outside the rpa left(nice & clean like this) and others have rpa or not and look like a dumpster.

 

[VepsrP]

Well, kids, Dad's in the building. It's been a long time. It's been 5 days since I wanted to see why the decompiler doesn't work on the third version of python. Having no knowledge of decompiling and python in general, I decided to take this challenge. And today I am ready to present you a working version of the decompiler, which not only works on the python 3, but has already been tested on 2 games: Mythos and WVM. Both games run and work perfectly, especially the first one. Perhaps there are still some possible problematic options, but I'm too lazy to conduct large-scale tests on my own, it's time for me to go through the games that I put off until this moment. I suggest that the interested individuals themselves conduct a check and send the problems found to me for further work. I hope RenPy won't want to change the compilation much in the near future, except for the reason of switching to Python 3. In fact, I'm shocked, it feels like it's been more than a week since I've been sitting with this seemingly trivial task. There is still a deobfuscation file left, but there is too much to edit in it, all these exceptions and raising them no longer work, perhaps later I will see what needs to be done with it, and now, until there is an urgent need, you just should not use the key to use it.

 

[creedgeezer]

Thanks for the additional info.
I guess the game name is "Deadlocked in Time". Just make sure i have the right one...
That statement is not really right. I get this:

View attachment 1076997

And even the message is correct.
You need first to unpack the .rpa files to get the games scripts(.rpyc files...), then you can decompile. Some games like this have no/few files outside the rpa left(nice & clean like this) and others have rpa or not and look like a dumpster.

Oh..Sorry my bad...I didn't make it more clear....
I already know how to use this tool (since 2018)
.
.
I already extracted the rpa files but after it finished extracting.... And if i pick the option 2...It quits immediately..No errors or anything

 

[VepsrP]

Oh..Sorry my bad...I didn't make it more clear....
I already know how to use this tool (since 2018)
.
.
I already extracted the rpa files but after it finished extracting.... And if i pick the option 2...It quits immediately..No errors or anything

I checked yesterday, personally I didn't have any problems with this game. Can you try downloading the tool again? Maybe it's damaged.

 

[Madeddy]

... today I am ready to present you a working version of the decompiler, which not only works on the python 3, but has already been tested on 2 games: Mythos and WVM. ....

Congrats. That's really, really great.
Although i am stumped how ever this works, without trowing errors left and right, if you use illegal code. For the buildin string method str.encode() exists not a 'bytes' value as encoding type. Not to my knowledge . Encoding to "bytes" is done with str.encode('utf-8'). The values you changed in the unrpyc files have already utf-8 as encoding argument.
Encoding type "bytes" is AFAIK a python2 remnant and just legit as encoding in stuff like the pickle module... and there we are...

On a side note: Setting the string encoding to "UTF-8" is not even needed because its the default. So `str.encode()` is enough.

You must be registered to see the links

I already extracted the rpa files but after it finished extracting.... And if i pick the option 2...It quits immediately..No errors or anything

Strange. Something must be wrong on your side. For me its without issues and for VepsrP also. Check like he said the tool, unpack the game again and check the access rights to this directory.

 

[VepsrP]

Congrats. That's really, really great.
Although i am stumped how ever this works, without trowing errors left and right, if you use illegal code. The buildin string method `encode('bytes')` doesn't exist to my knowledge. String encoding to "bytes" is done with str.encode('UTF-8') and the values you changed like cited did have already "UTF-8" as argument.
Encoding type "bytes" is AFAIK a python2 remnant and just legit as encoding in stuff like the pickle module... and there we are...

On a side note: Setting the encoding to "UTF-8" is not even needed because its the default. So `str.encode()` is enough.

You must be registered to see the links

I adhere to the doctrine: what works - don't touch it.
And what is the illegal code?

 

[anne O'nymous]

The buildin string method `encode('bytes')` doesn't exist to my knowledge. String encoding to "bytes" is done with str.encode('UTF-8') and the values you changed, like i cited, did have already "UTF-8" as argument.

The three occurrence of encode are clearly a call to the str method:

translate.py:67

            if isinstance(i, renpy.ast.Say):
                code = say_get_code(i)
            elif isinstance(i, renpy.ast.UserStatement):
                code = i.line
            else:
                raise Exception("Don't know how to get canonical code for a %s" % str(type(i)))
            md5.update(code.encode("bytes") + "\r\n")

code is the "canonical code", therefore pure text.

codegen.py:799

            newline_count = node.s.count('\n'.encode('bytes'))

It's an anonymous string, but still a string.

astdump.py:148

            if isinstance(name, str):
                name = name.encode('bytes')

And here name is explicitly validated as string.

 

[Madeddy]

I adhere to the doctrine: what works - don't touch it.
And what is the illegal code?

Like explained... i thought. Do I so bad explain? I mean i am not english and never got a teacher for it, so possible. Maybe someone else can do it better. I edit it a little - maybe it helps.
By the way: I tested it without your changes to str. encode('bytes') and it works. So far.

I did try last spring to refactor unrpyc to py3 and got then stuck. You did basically the same changes as i am, but found and corrected two error sources more:

• In magic.py you replaced the encoding (What interestingly enough works: from line 592)

# py3 compat: at least for py2 src needet (also py3 renpy ?)
def safe_loads(string, class_factory=None, safe_modules=(), use_copyreg=False,
               encoding="utf-8", errors="errors"):

This goes over some classes in the end into the pickle module and should have ascii or bytes as encoding, but if not bytes it goes into str.decode:

You must be registered to see the links

• In sl2decompiler.py you replaced a None with a line value from ast (after line 100)

# py3 compat: at least for py2 src needet (also py3 renpy ?)
@dispatch(sl2.slast.SLBlock)
def print_block(self, ast):
    self.print_keywords_and_children(
        ast.keyword, ast.children, ast.location[1], atl_transform=getattr(ast, 'atl_transform', None))

This did trow a error if lines where compared: TypeError: '>' not supported between instances of 'int' and 'NoneType' Why ever this changed in py3...

Bottom line: Good work anyway. I did not find this two.

 

[VepsrP]

Like explained... i thought. Do I so bad explain? I mean i am not english and never got a teacher for it, so possible. Maybe someone else can do it better. I edit it a little - maybe it helps.
By the way: I tested it without your changes to str. encode('bytes') and it works. So far.

str.encode('bites')

is not my change. In the latest version, I only replaced 'bytes' with 'utf-8' in one place - in the safe_loads function, because otherwise the attributes become byte-like, as shown in the attached screenshot, the rest of the lines I kind of left in order.

 

[VepsrP]

I did try last spring to refactor unrpyc to py3 and got then stuck. You did basicly the same changes as i am, but found and corrected two error sources more:

• In magic.py you replaced the encoding (What interestingly enough works)
  

  # py3 compat: at least for py2 src needet (also py3 renpy ?)
  # encoding="bytes", errors="errors"):
  def safe_loads(string, class_factory=None, safe_modules=(), use_copyreg=False,
                 encoding="utf-8", errors="errors"):

• In sl2decompiler.py you replaced a None with a line value from ast (after line 100)
  

  @dispatch(sl2.slast.SLBlock)
  def print_block(self, ast):
  self.print_keywords_and_children(
  ast.keyword, ast.children, ast.location[1],
          atl_transform=getattr(ast, 'atl_transform', None))

These two things are the most important thing that was the most troublesome. It was on these two things that I was stuck. And if the logic of the first problem as a whole can be understood, because the second python simply ignored the letter 'b' before the name of the attributes, then that's why in cases with the display of blocks in the second python rolled None, and in the third there is no longer, for now a mystery to me.
Update: And one more interesting thing at the end. for some reason, in the IF and USE blocks, I gave out two colons in a row, also a strange moment, but I quickly fixed it there.

 

[Madeddy]

I only replaced 'bytes' with 'utf-8' in one place - in the safe_loads function

I downloaded your unrpyc v0.3 and there are the changes form utf-8 to encode('bytes') like said. Maybe you made a mass replace and forgot?

Anyway: You meant these colons? I figured you made a mistake with them, because i believed the colons are important... but with this it prints them double?

    def _print_if(self, ast, keyword):
        # the first condition is named if or showif, the rest elif
        keyword = First(keyword, "elif")
        for condition, block in ast.entries:
            self.advance_to_line(block.location[1])
            self.indent()
            # if condition is None, this is the else clause
            if condition is None:
                self.write("else:")
            else:
                self.write("%s %s:" % (keyword(), condition))

If you're right with the change, we have the question to ask if this also in python2 happens? Would be a bug.

 

[VepsrP]

I downloaded your unrpyc v0.3 and there are the changes form utf-8 to encode('bytes') like said. Maybe you made a mass replace and forgot?

Anyway: you mean these colons? I figured you made a mistake with them, because i believed the colons are important... but with this it prints them double?

    def _print_if(self, ast, keyword):
        # the first condition is named if or showif, the rest elif
        keyword = First(keyword, "elif")
        for condition, block in ast.entries:
            self.advance_to_line(block.location[1])
            self.indent()
            # if condition is None, this is the else clause
            if condition is None:
                self.write("else:")
            else:
                self.write("%s %s:" % (keyword(), condition))

If your're right with the change, we have the question to ask if this also in python2 happens? Would be a bug.

Yes, this is the right place. And no, on the python 2, it didn't print the colon twice. Otherwise, RenPy would have immediately said that there was an error, in fact, it was through him that I learned about colons, since an error immediately popped up at startup.

 

[Madeddy]

Ok. Thanks. This means in py2 it needs the colons there and in py3 not.
Along with all the other changes we have now the choice to

• make this compatible to py2 AND py3 (putting switches in) or
• to remove the remaining code which is "only for py2" and make a pure py3 version

There are also two bugs(so far) in deobfuscate.py, but easy to fix i think. I will upload my version of unrpyc_py3 later to my unrpyc fork. I did for this no format corrections or other changes. Just whats for the py3 version really needed is.
A pep8 compatiple format refactor or use of pathlib and some other are for later on my TODO.

 

[guroth]

Small suggestion, change the currentdir var so u can execute it thru path.
something like set "currentdir=%cd%\"

 

[VepsrP]

Ok. Thanks. This means in py2 it needs the colons there and in py3 not.
Along with all the other changes we have now the choice to

• make this compatible to py2 AND py3 (putting switches in) or
• to remove the remaining code which is "only for py2" and make a pure py3 version

There are also two bugs(so far) in deobfuscate.py, but easy to fix i think. I will upload my version of unrpyc_py3 later to my unrpyc fork. I did for this no format corrections or other changes. Just whats for the py3 version really needed is.
A pep8 compatiple format refactor or use of pathlib and some other are for later on my TODO.

I don't know what bugs you have in deobfuscation, but I have a problem with exceptions there. It can be easy to fix them, but there are a whole cloud of them, which does not look at all fun. At the expense of everything else, I would make different versions, so that if anything, decompilation on the python 2 can always be easily replaced, and with the 3 already according to the situation.

 

[VepsrP]

Small suggestion, change the currentdir var so u can execute it thru path.
something like set "currentdir=%cd%\"

Can you specify: who is it for, what is it for?

 

[Madeddy]

I don't know what bugs you have in deobfuscation, but I have a problem with exceptions there.

I did so far not even test the deobfuscation... he, he. Its in the unused function 'assert_is_normal_rpyc', so its without value.
Regardless i spotted these 2 mentioned problems:

• line 236 and 239 should be the same variable/name i think: 'header' -> 'header_data'
• line 247/251 and 259 the same: 'uncompressed' -> 'raw_contents'

And more of your exceptions after a test. Two of them, many repeats:

• Line 71 needs a byte prefix b'RENPY RPYC' literal on the string. Thats one of your errors.
• Line 291 and 308: ValueError has no "message" member so remove it from 'e.message'. Just 'e' Thats the other error i got.

With summertimesaga as test the deobfuscation did now run without problems.

However another problem: We missed some of the double colons and in one case its even missing. Summertimesaga produced a bunch of them. See attached file.

I would make different versions, so that if anything, decompilation on the python 2 can always be easily replaced, and with the 3 already according to the situation.

This was/is exactly my thought. Py2 is end of life anyway and already supported by the unrypc dev. I will see if he takes the work from here for a official py3 version. Very possible we overlooked some problems and he sees them immediately.

Can you specify: who is it for, what is it for?

For the normal unren.bat. This idea pops every 2 months up. Not a bad idea per se, but people missing the fact this app isnt really maintained anymore and makes it for more unknowing users harder to use.

 

[yoyomistro]

Awesome work VepsrP , and awesome discussion even though I'm not fully following lol. I know you built this for Python 3, but I'm gonna ask what I always ask when someone fiddles with un'ren, which is does it work for Summertime Saga ?