Ren'Py Tutorial [How-To] Variables and save compatibility...

[anne O'nymous]

So to reproduce the exception, one can add config.rollback_length+1 dialogue lines with no intermediate label in your sample script, and save at the last dialogue line.

It just need to move and rename the label for the last demonstration.

 

[Tacito]

All authors have to deal with the same problem, at some point of the development of their game, they have to add new variables.

Very helpfull for a beginner like me

Looking in a game I found an interesting method to add new task:

label after_load:
    python:
        try:
            come_anna
        except:
            come_anna = Task(_("Come to Anna"))
            TASK.append(come_anna)

So if the task is not present is added for compatibility with old save.
The problem is ... the new task needed is created only if you load (at least in this game).
If you start a new game and reach the point where the task or control is needed ... error.

Is there a better point to put a Call to the label after_load if I start a new game ? or label start: is ok ?

I see also you can have more than one after_load labels
label after_load_00001
label after_load_00002
.....

Sorry if they are stupid questions

 

[anne O'nymous]

Very helpfull for a beginner like me

It was the goal, so I'm glad to hear that it worked

label after_load:
    python:
        try:
            come_anna
        except:
            come_anna = Task(_("Come to Anna"))
            TASK.append(come_anna)

It's a matter of taste more than anything else, but I really dislike the useless use of try here. The code is read like,"try to access a variable named 'come_anna', and if you failed to do it, create the variable". It's a Python paradigm (named "

You must be registered to see the links

"), but sometimes people over abuse of it. The old man I am prefer an approach like this :

label after_load:
    if not hasattr( store, "come_anna" ):
        $ come_anna = Task(_("Come to Anna"))
        $ TASK.append(come_anna)

which read like, "if the variable named 'come_anna' don't exist, create it" ; and in this particular case let you use Ren'py statements if needed.
In the end the result is the same, that's why it's not especially an error. But the couple try/except should be kept for cases where there isn't a command specifically designed to do the exact same thing ; here it's like using a nuclear weapon to kill an ant, when you have in hand a hammer that can do exactly the same thing faster and more easily.

The problem is ... the new task needed is created only if you load and save (at least in this game).
If you start a new game and reach the point where the task or control is needed ... error.
Is there a better point to put a Call to the label after_load if I start a new game ?

Calling after_load from the start label isn't a bad idea (I'll even add it in the How-To). But it can have side effects if you do more than just adding variables. The best approach is probably to split the after_load label like this :

label after_load:
    call updateVariables
    call correctError
    [...]

then also call updateVariables from the start label.

I see also you can have more than one after_load labels
label after_load_00001
label after_load_00002
.....

I can have missed something, but I don't think it's a regular Ren'py feature. It seem more like a way to split the after_load label like I did above.

Sorry if they are stupid questions

You must be registered to see the links

, yours don't fall under this category.

 

[Tacito]

It was the goal, so I'm glad to hear that it worked

All very clear, thanks again.
Just out of curiosity I tried the after_load_0001 0002
They are executed sequentially after the load, but it is not that essential.

 

[anne O'nymous]

Just out of curiosity I tried the after_load_0001 0002
They are executed sequentially after the load, but it is not that essential.

It's the part I dislike with Ren'py, even if you follow the official announce on the forum official, read all the change log and the doc, there's sometimes things that you don't know. So, if you miss a message time to time, you end knowing a lot, and in the same time ignoring a lot...
Well, until someone tell you about it, so thanks.

 

[Tacito]

Well, until someone tell you about it, so thanks.

Sorry I see that works only in that specific game ... there is a code for that , I don't know why ...

You are right ... as usual

 

[anne O'nymous]

You are right ... as usual

Can I quote you next time a co-worker complain ?

 

[botc76]

I'm just a player, not a dev, but I've been linking to this thread in every game that kills saves each update, I play or used to play.
I feel it should be required reading for any new dev working with ren'py.

Thanks for your work, anne.

 

[anne O'nymous]

Thanks for your work, anne.

No, thanks to you for spreading the word.
I can write as many helping guides as I want, if devs don't even know about them, it's useless. It's because there's users like you, that my works take all its sense.

 

[ZanithOne]

Hey! Great topic, super useful for developers of all kinds and experience levels, thank you for putting so much time and effort into it.

I've been struggling with save compatibility in my game for a long time, and I finally found out what was wrong when I dug a little deeper into the RenPy core. So I want to chime in on this:

You can safely delete the .rpyc file, as long as you don't rename a label in this file ;

This particular point is not correct, at least, not fully so. If you delete .rpyc file, RenPy can and will generate new namemap/location entries for the lines within the blocks. If you use from clauses in your calls or just don't have a big return stack, this may not cause much harm, but at the same time, it just might wreck your saves.

For reference, RenPy call/return stacks are saved like this:

Return stack: [u'navigator_scripts.navigator_room_enter_helper', (u'game/scripts/repeatable_actions/cait/massage_sit/cait_massage_level_three.rpy', 1575100391, 28397), u'screen_utils.window_hide_pause']
Call location stack:[(u'game/scripts/navigation/navigator_scripts.rpy', 1575100391, 2322), (u'game/scripts/repeatable_actions/cait/massage_sit/cait_massage_level_three.rpy', 1575100391, 28396), u'screen_utils.window_hide_pause']

Note, the labels are stored as plain strings, but plain call directives are stored as opaque tuples.

And now to the .rpyc files. The thing is, .rpyc compilation is not deterministic. I will demonstrate the namemap dump diffs for one particular file. Namemap is the main index of all the addressable nodes in the RenPy script. Please note that the source .rpy file was not changed at all between runs.

First diff is between the .rpyc file that is currently stored in git and the one that was generated after I checked "force recompile" in the build dialog.

Note that while some lines are rearranged (diff viewer is not smart enough to align them), the indices are the same. This means that namemap stays consistent and if you take that return stack reference tuple from before, you will be able to find the corresponding node.

Now, I remove the cait_massage_level_three.rpyc file without changing anything in .rpy file and launch the game.

And all tuples (well, triples, but who's counting) are changed now. And that means that return stack is no longer valid and your players will be getting Could not find return label exception when they try to load the game!

And you can't really do anything about that if you don't have your old .rpyc files, because this line location generation is very state-dependent. You can try looking into the RenPy parser yourselves, I just had a cursory glance, but as far as I can see, it just increments a counter while going through the script and then slaps this number as a reference number for a node, along with the unixtime of compilation.

So, if you can at all help it, do not delete your .rpyc files. Store them in your VCS, or at least back them up regularly if you don't use VCS (but you totally should).

 

[anne O'nymous]

If you delete .rpyc file, RenPy can and will generate new namemap/location entries for the lines within the blocks. If you use from clauses in your calls or just don't have a big return stack, this may not cause much harm, but at the same time, it just might wreck your saves.

No, it's the opposite. Using the from clause will prevent you to have any problems, as long as you don't ask Ren'py to generate the said clause for you.
In fact, the from clause is here exactly for this case. It create a virtual label that Ren'py will use as return point, instead of the reference used by default.

Try this :

init python:
    config.label_overrides["_callMe"] = "hijacked"
    config.label_overrides["realReturn"] = "_callMe"

label start:
    "Let's go for a ride."
    call callMe from _callMe
    "Welcome back to the start label."
    "End."

label callMe:
    "You're in the regular label, but where will you return ?"
    return

label hijacked:
    "I wanted to say hi to you."
    jump realReturn

You'll see this :

• Let's go for a ride.
• You're in the regular label, but where will you return ?
• I wanted to say hi to you.
• Welcome back to the start label.

Therefore, as long as it's you who write the from clause, there's absolutely no risk in the fact that you removed the rpyc files. Ren'py will not return to an entry point, but jump to a label ; technically it's not just a jump, but the result is the same.

But effectively, if you let Ren'py generate those clause for you, it will lead to problems. This because the return labels will not have the same name, and here Ren'py can struggle to find the return point, or return to a totally different place.

You can also see it this way :

label start:
    "Let's go for a ride."
    call callMe from _callMe
    "Welcome back to the start label."
    "End."

label callMe:
    "You're in the regular label, but where will you return ?"
    "Please save here."
    return

Save where asked, then edit the source this way :

label start:
    "Let's go for a ride."
    call callMe
    "Welcome back to the start label."
    "End."

label callMe:
    "You're in the regular label, but where will you return ?"
    "Please save here."
    return

label _callMe:
    "Well, it's not where I expected to be."
    "End."
    return

Then load your save and follow the lead. What you'll see this time will be :

• Let's go for a ride.
• You're in the regular label, but where will you return ?
• Please save here.
• Well, it's not where I expected to be.
• End.

And the return in "_callMe" will end the game as expected.

 

[ZanithOne]

No, it's the opposite. Using the from clause will prevent you to have any problems, as long as you don't ask Ren'py to generate the said clause for you.
In fact, the from clause is here exactly for this case. It create a virtual label that Ren'py will use as return point, instead of the reference used by default.

Yes that's what I meant, if you use from clauses, you are mostly safe from return stack mismatches. Maybe I should have formulated it differently.

The problem is, sometimes you can't add from clauses to calls. I believe there is no way to do this when you use $ renpy.call(...) or Call action*, that's why I think it's important for people to do their best to keep their .rpyc files consistent.

* - as there is no RenPy AST node to return to**. I don't remember off the top of my head how the return stack looks for those cases, and you can rewrite the return stack yourself, but I do believe that for most folks, especially those with less coding experience, it would be better to just not delete the .rpyc files.

Edit:
** - this is not correct, dunno why I wrote this. There is an AST node, Python Statement node and call screen statement probably can accept from keyword. Still, AFAIK, there is no way to add a simple from to python node, so you'll have to rewrite return stack to explicitly define return label in this case.

 

[anne O'nymous]

[...] that's why I think it's important for people to do their best to keep their .rpyc files consistent.

What I totally agree with. It's important not only for the return point, but also for few other things, like the skip feature by example.
This said, my point wasn't to say that it's totally safe to delete the rpyc files, but that it's not unsafe to do it and/or to play with the label ; when done properly. Some devs struggle with their code because they fear to finally split their code in multiple files and things like this. And my focus was to explain them how they can do it safely. But I can have failed to express this correctly.

Still, AFAIK, there is no way to add a simple from to python node, so you'll have to rewrite return stack to explicitly define return label in this case.

You shouldn't use the renpy.call(), and anyway shouldn't have a reason to use it. It's a function dedicated for user defined statements (and possibly screen actions), therefore advanced Ren'py coding.
If you have to use it outside of one of those case, then the design of your code have a problem.

 

[AmazonessKing]

Necrobumping this (insanely useful) thread because I have a question.

Let's say you have a class in version 0.1:

init python:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0

But you add a line in version 0.2:

init python:

    class character_class(object):
        def __init__(self, name, type, outfit, contact_pic=False, gps=False):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0
            self.horny = 0

How would that reflect between saves? Is there a way to update those classes using something like after_load ?

 

[ZanithOne]

How would that reflect between saves? Is there a way to update those classes using something like after_load ?

If you just update the constructor, new attributes in older instances will remain undefined and will raise attribute exception unless you assign them manually. You can do it in after_load with something like this:

if not hasattr(char_class_obj, "horny"):
    char_class_obj.horny = 0

You don't need the hasattr check if you're only going to do that once and you are sure you won't call this ever again, but better safe than sorry, IMO.

 

[AmazonessKing]

If you just update the constructor, new attributes in older instances will remain undefined and will raise attribute exception unless you assign them manually. You can do it in after_load with something like this:

if not hasattr(char_class_obj, "horny"):
    char_class_obj.horny = 0

You don't need the hasattr check if you're only going to do that once and you are sure you won't call this ever again, but better safe than sorry, IMO.

Alright. And there's no way to add a default with after_load, right? Something like

label after_load:
    default record = 1

Because being a label, it only accepts $ which is basically define, right?

 

[ZanithOne]

Alright. And there's no way to add a default with after_load, right? Something like

label after_load:
    default record = 1

Because being a label, it only accepts $ which is basically define, right?

I'll be honest, I still don't completely understand the define/default behaviour. When I add new fields to an existing class I just use the code I posted earlier and run it on all instance of those classes. And it's just an attribute assignment at this point.

 

[anne O'nymous]

You can do it in after_load with something like this:

Since Ren'Py rely on pickle, there's another way to do this, more technical, but also more efficient. It rely on Python's magic methods, here the __setstate__( self, state ) one.

Basically speaking:
By default, when an object is created by pickle, therefore when a save file is loaded, the value saved are directly assigned to the object __dict__.
When the __setstate__ magic method exist, it will be called automatically by Python, in place of this direct assignation. Then, state will be a dictionary corresponding to __dict__ when the object was saved.

Therefore, the behavior by default is this one:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0

        def __setstate( self, state ):
            self.__dict__ = state

This mean that you can interfere with the assigned values to add a new one:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0
            self.horny = 0   # This is new.

        def __setstate( self, state ):
            #  Verify if the added attribute was part of the save.
            if not "horny" in state:
                # Add it with its default value if it wasn't the case.
                state["horny"] = 0
            # Then 'load' as usual.
            self.__dict__ = state

But you can go further than this, if by example you changed the name of an attribute:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.class = type   # This is renamed
            self.level = 0
            self.exp = 0


        def __setstate( self, state ):
            #  Verify the old name is used for the attribute
            if "type" in state:
                #  Create an entry with the new name, while
                # keeping the saved value.
                state["name"] = state["type"]
                # Remove the entry with the old name.
                del state["type"]
            # Then 'load' as usual.
            self.__dict__ = state

In extreme cases, you can even do evil things, starting with this:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0
            self.love = 0
            self.lust = 0

then deciding that separating battle and relationship stats is a better idea:

    class Battle_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0

    class Relation_class(object):
        def __init__(self, name):
            self.name = name
            self.love = 0
            self.lust = 0

    class character_class(object):
        def __init__(self):
            # You don't need this anymore

        def __setstate( self, state ):
            #  After this update, the object will be blank with no attributes.
            # If it's the case, just return.
            if not "name" in state: return

            #  Else it mean that you need to split the object.

            #  Ensure that the object exist, else create it. Not really sure
            # how /default/, /define/ and this interfere, so better safe than
            # sorry.
            #  Firstly the battle object
            if not hasattr( store, state["name"][:3] + "_battle" ):
                setattr( store, state["name"][:3] + "_battle", Battle_class( state["name"], state["type"] ) )

            # Then 'load' it
            obj = getattr( store, state["name"][:3] + "_battle" )
            obj.level = state["level"]
            obj.exp = state["exp"]

            # Do the same for the relation object.
            if not hasattr( store, state["name"][:3] + "_relation" ):
                setattr( store, state["name"][:3] + "_relation", Relation_class( state["name"] ) )

            obj = getattr( store, state["name"][:3] + "_relation" )
            obj.love = state["love"]
            obj.lust = state["lust"]

            # Then do nothing else, this object is now irrelevant, keep it blank.

Edit: A typo in the code, sorry.

 

[AmazonessKing]

I'll be honest, I still don't completely understand the define/default behaviour. When I add new fields to an existing class I just use the code I posted earlier and run it on all instance of those classes. And it's just an attribute assignment at this point.

That's easy, I always forget, but I just came across it yesterday. Try doing this: Define a random variable, make it a list, then default another random variable, and make it a list.

define random_list1 = []
default random_list2 = []

label start:
    $ random_list1.append("test")
    $ random_list2.append("test")
    "Hi."
    "Save the game here. Then reload renpy through the developer console or Shift + R."
    "If you did, the random_list2 should retain the value, the other one doesn't."
    return

Basically, both define and default retain the values using ren'py's loads. But if you close the game or reload it to test more code, the values from the defined variable will be gone.

You must be registered to see the links

just to D'oh! myself a few minutes later.

Since Ren'Py rely on pickle, there's another way to do this, more technical, but also more efficient. It rely on Python's magic methods, here the __setstate__( self, state ) one.

Basically speaking:
By default, when an object is created by pickle, therefore when a save file is loaded, the value saved are directly assigned to the object __dict__.
When the __setstate__ magic method exist, it will be called automatically by Python, in place of this direct assignation. Then, state will be a dictionary corresponding to __dict__ when the object was saved.

Therefore, the behavior by default is this one:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0

        def __setstate( self, state ):
            self.__dict__ = state

This mean that you can interfere with the assigned values to add a new one:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0
            self.horny = 0   # This is new.

        def __setstate( self, state ):
            #  Verify if the added attribute was part of the save.
            if not "horny" in state:
                # Add it with its default value if it wasn't the case.
                state["horny"] = 0
            # Then 'load' as usual.
            self.__dict__ = state

But you can go further than this, if by example you changed the name of an attribute:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.class = type   # This is renamed
            self.level = 0
            self.exp = 0


        def __setstate( self, state ):
            #  Verify the old name is used for the attribute
            if "type" in state:
                #  Create an entry with the new name, while
                # keeping the saved value.
                state["name"] = state["type"]
                # Remove the entry with the old name.
                del state["type"]
            # Then 'load' as usual.
            self.__dict__ = state

In extreme cases, you can even do evil things, starting with this:

    class character_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0
            self.love = 0
            self.lust = 0

then deciding that separating battle and relationship stats is a better idea:

    class Battle_class(object):
        def __init__(self, name, type):
            self.name = name
            self.type = type
            self.level = 0
            self.exp = 0

    class Relation_class(object):
        def __init__(self, name):
            self.name = name
            self.love = 0
            self.lust = 0

    class character_class(object):
        def __init__(self):
            # You don't need this anymore

        def __setstate( self, state ):
            #  After this update, the object will be blank with no attributes.
            # If it's the case, just return.
            if not "name" in state: return

            #  Else it mean that you need to split the object.

            #  Ensure that the object exist, else create it. Not really sure
            # how /default/, /define/ and this interfere, so better safe than
            # sorry.
            #  Firstly the battle object
            if not hasattr( store, state["name"][:3] + "_battle" ):
                setattr( store, state["name"][:3] + "_battle", Battle_class( state["name"], state["type"] ) )

            # Then 'load' it
            obj = getattr( store, state["name"][:3] + "_battle" )
            obj.level = state["level"]
            obj.exp = state["exp"]

            # Do the same for the relation object.
            if not hasattr( store, state["name"][:3] + "_relation" ):
                setattr( store, state["name"][:3] + "_relation", Relation_class( state["name"] ) )

            obj = getattr( store, state["name"][:3] + "_relation" )
            obj.love = state["love"]
            obj.lust = state["lust"]

            # Then do nothing else, this object is now irrelevant, keep it blank.

Edit: A typo in the code, sorry.

Legit amazing. Hopefully, I won't need to do this in the future, but holy fuck.

 

[anne O'nymous]

Basically, both define and default retain the values using ren'py's loads. But if you close the game or reload it to test more code, the values from the defined variable will be gone.

Even more explicit:

define random_list1 = []
default random_list2 = []

label start:
    $ random_list1.append("test")
    $ random_list2.append("test")
    "list1 [random_list1] \n list2 [random_list2]"
    "boo"

Go to the main menu, and restart the "game". Do it again, and again, and again, and... Well, everyone should have understood before reaching this level of "again".


And there's also this post that explain the differences ; the two previous posts on the said thread are also worth been read.