Ren'Py [solved] Is proper savegame locking even possible?

[f95zoneuser463]

What's save locking?
Prevent incompatible savegames from loading.

The (simplified) problem:

1. dev creates a game using a complex custom python class/object, lets call it A
2. game version 1 gets published, players play it, savegames contain class A-data and the info that it's version 1
3. dev decided to remove/replace/ class A in version 2 with the new class B and knows that versions 1 savegames will be incompatible
4. dev adds savegame version-check to the
   You must be registered to see the links
   -label, that check is suppose to kick the player back to the main menu if he tries to load an incompatible savegame
5. solved ?
6. No. Attempting to load an old savegame will trigger an exception before the code even reaches the version-check because it tries to load object A data
7. prediction: player tries version 2, tries to load his old save, sees exception instead of proper warning about incompatible savegame, uninstalls game
8. result: usability is garbage, player is mad, dev is sad

So this kind of version checking will not work reliable. If classes got modified in a way that is incompatible with older object-data it will fail. Yet I cannot find any examples that does it in any other way. Everything I've found is similar and has the same problem. (Plus I'm shocked how bad some scripts are.)

Theory for proper savegame check:

• 'read' the version from the savegame before it actually loads
• check the savegame version:
  • block load if incompatible savegame and display a message
  • load if compatible

I cannot think of any solution to this because there is no such thing like a "before_load"-label or "read specific data from savegame" function. Any ideas?

 

[79flavors]

I don't know if this is the answer to your question... but it's something I did when I was trying to patch Sister, Sister, Sister's Truth or Dare minigame.

I created two variables. One as a define, one as a normal saveable variable. Then I would compare the two to see if they matched in the after_load:

Let's call my two variables my_savegame_vers and my_currgame_vers
At the start of the game and at the end of after_load, I would set the savegame_vers = currgame_vers. That way, any saved games that load while the version matches... just skips my future checks.

But if the version numbers don't match... then I start "upgrading" my variables depending on the version in my load file.

Something like this:

default my_savegame_vers = 0
define my_currgame_vers = 4

label start:

    $ my_savegame_vers = my_currgame_vers
    scene black
    "just a test"
    return

label after_load:

    if my_savegame_vers == my_currgame_vers
        return

    # I should probably check if my_savegame_vers  == 0 too. For implementing this system AFTER adding these two variables.
    # I was just too lazy to type it.

    if my_savegame_vers == 1:
        call upgr_save_vers_1_to_2

    if my_savegame_vers == 2:
        call upgr_save_vers_2_to_3

    if my_savegame_vers == 3:
        call upgr_save_vers_3_to 4

    $ my_savegame_vers = my_currgame_vers
    return

label upgr_save_vers_1_to_2

    if pc_name == "":
        $ pc_name = mc

    my_savegame_vers = 2

    return

# etc, etc, etc.

Upgrading those variables might not always be obvious. But this is at least a start.
And by checking and upgrading multiple levels... it gets around the problem if what to do if a player last saved a "vers 1" save, but the game is currently version 4.

You could make use of the config version number variable instead... but when I was thinking about this, I knew that most of my savegame files would be compatible with the previous version... So decided to keep the actual game version number separate from my imaginary save game versioning. It also avoided the problem of game versions being "0.1.1" or "0.1a" - something a little less easy to do a greater than or equal check against.

In the case where I used it, I was quite lucky that the game constantly looped back to a common point.
So I was able to create two almost identical variables as a sort of "truth or dare version" comparison too... independent of the save versioning.

That basically looked like this:

default truthordare_saved_vers = 0

# [...]

label truth_or_dare:

    $ truthordare_saved_vers = 13
    define truthordare_latest_vers = 13

    # [...]

label ourturnstart:

    if truthordare_latest_vers > truthordare_saved_vers:
        jump truthordare_version_conflict

    # [...]

label truthordare_version_conflict:

    scene bg_vi_logo
    "DEVELOPER NOTE: Truth or Dare has been altered significantly recently. Your save may not work quite how you expect it should."
    "To get around this, it's probably safer to restart at the beginning."
    "{i}Sorry.{/i}"
    jump truth_or_dare

 

[Epadder]

What's save locking?
Prevent incompatible savegames from loading.

The (simplified) problem:

1. dev creates a game using a complex custom python class/object, lets call it A
2. game version 1 gets published, players play it, savegames contain class A-data and the info that it's version 1
3. dev decided to remove/replace/ class A in version 2 with the new class B and knows that versions 1 savegames will be incompatible
4. dev adds savegame version-check to the
   You must be registered to see the links
   -label, that check is suppose to kick the player back to the main menu if he tries to load an incompatible savegame
5. solved ?
6. No. Attempting to load an old savegame will trigger an exception before the code even reaches the version-check because it tries to load object A data
7. prediction: player tries version 2, tries to load his old save, sees exception instead of proper warning about incompatible savegame, uninstalls game
8. result: usability is garbage, player is mad, dev is sad

So this kind of version checking will not work reliable. If classes got modified in a way that is incompatible with older object-data it will fail. Yet I cannot find any examples that does it in any other way. Everything I've found is similar and has the same problem. (Plus I'm shocked how bad some scripts are.)

Theory for proper savegame check:

• 'read' the version from the savegame before it actually loads
• check the savegame version:
  • block load if incompatible savegame and display a message
  • load if compatible

I cannot think of any solution to this because there is no such thing like a "before_load"-label or "read specific data from savegame" function. Any ideas?

My thought is that doing #3, except at the very start of development, is a disservice to your player base.

Instead of discarding class A immediately, it should be used to generate a class B replacement in the after_load part of your code for at least a few updates.

I know that can't be done all the time unfortunately.

I don't know a perfect code solution to this issue, but here are the two ways I would approach it.

1. At first I would go into options.rpy and change "define config.save_directory" so that people won't pull old saves from their AppData folder (this fails if people download the game and just copy it into the old directory though).
2. The other thing I would do is add a warning message into the splashscreen label so that people are much more likely to read, at least compared to patch notes/thread update.

The only other thing I thought of would be to use the splashscreen label to execute a function that gets a list of files in the saves directory under the game folder and then looks at the dates of the files and then deletes files that are older than the latest update.

I have no clue how to code that or whether ren'py would run into permission issues trying to do that.

 

[f95zoneuser463]

@79flavors
That is indeed a neat way to patch and upgrade things. Unfortunately, if I understand it correctly, it has the same problem: the exception (due to the incompatible data) is already triggered before reaching after_load.

In my case the 'real world' example is this:

• old version had a class named State to hold a bunch of character-data
• this got split into 2 new classes named MC and NPC with completely different properties (inheritance would have made no sense)

loading an old save triggers this before reaching after_load:

Now that I think about it, maybe I could try to wrap the loading-function on the load screen in an exception handler ... but that seems to be inside the Ren'Py source code edit: nope, is in screens.rpy ... *need to investigate*

My thought is that doing #3, except at the very start of development, is a disservice to your player base.

I agree and a I try to avoid it at all costs obviously. I'd say I'm still before the 5% completion mark. To avoid frustration at this point in development I can only provide a quick start, cheats and workaround-code for small updates if necessary. Still people will complain anyway. But I don't think even the most experienced Python- and Ren'Py-coder can do a big project without breaking savegames, especially if it's not linear. A good example is 'Summertime Saga' with a large team of experienced coders. I completely understand why they use save locking.

 

[79flavors]

@79flavors
That is indeed a neat way to patch and upgrade things. Unfortunately, if I understand it correctly, it has the same problem: the exception (due to the incompatible data) is already triggered before reaching after_load.

You may well be right. I'm only a novice RenPy programmer.
I've used something similar for swapping strings and boolean, and strings and integers. Not the more complex class/object types. (Primarily fixing someone's code who used the same variable names for both the Character object and the string that contained the character's name).

'twas mostly wishful thinking on my part.
My solution might be okay if you were creating a new object and copying the data from the old to the new, then effectively abandoning the old. No conflict would exist, since both structures would both exist.
All that said, I've never coded anything in RenPy more complex than simple variable types and never run into the exception you're trying to solve.

 

[aroha]

@79flavors

• old version had a class named State to hold a bunch of character-data
• this got split into 2 new classes named MC and NPC with completely different properties (inheritance would have made no sense)

You need to keep the State class around, use MC/NPC in your code, and copy data from State to MC/NPC in after_load (I think this is what 79flavors was implying above).

This means that you need these objects around for as long as you want to support that save version.

If you want to avoid after_load becoming enormous, you can add your own callback functions to

You must be registered to see the links

.

 

[Epadder]

I agree and a I try to avoid it at all costs obviously. I'd say I'm still before the 5% completion mark. To avoid frustration at this point in development I can only provide a quick start, cheats and workaround-code for small updates if necessary. Still people will complain anyway. But I don't think even the most experienced Python- and Ren'Py-coder can do a big project without breaking savegames, especially if it's not linear. A good example is 'Summertime Saga' with a large team of experienced coders. I completely understand why they use save locking.

It's that a lot of developers have a different priority than their users...

A lot of developers want to make the whole game the best thing they can, even if it breaks what came before. While a good portion of users want each update to make the new part of the game really good, but the stuff they already did isn't very interesting anymore. It's especially frustrating to users the more complex the requirements are to fulfill to get back to where they were.

I completely overhauled the original version of my project, which of course broke save compatibility, but I've made it my priority to keep save compatibility from the current released version to all the future versions.

I don't have the pressure of people paying me to release in a timely manner, only the pressure I put on myself to do as much as I can.

I've just been that frustrated user on the other side of a game breaking it's save compatibility and given up on things that do it often.

 

[anne O'nymous]

No. Attempting to load an old savegame will trigger an exception before the code even reaches the version-check because it tries to load object A data

It happen because the error don't come from Ren'py code, but directly from pickle. It could be possible to catch the exception with a try/except block, but it need too much changes.

Theory for proper savegame check:

Practice for a proper savegame check: Do not check, make it effectively compatible.

The problem is simple: In your save file, you have a not used anymore class, while in you game, you have a now used class.
Then the solution is as simple as the problem : Keep the definition of your not used anymore class, and use after_load to do the conversion.

[wrote on the fly, there's a better way to do it]

init python:

    class OldClass():
        def __init__( self ):
            self.someVar = 1
            self.anotherVar = 2

    class NewClass():
        def __init__( self ):
            self.newSomeVar = 'a'
            self.newAnotherVar = 'b'

        def convert( self, obj ):
            self.newSomeVar = chr( 60 + obj.someVar )
            self.newAnotherVar = chr( 60 + obj.anotherVar )
            

# Previous way to do
#default myStats = OldClass()
default myStats = NewClass()


label after_load:

    if isinstance( myStats, OldClass ) is True:
        $ tmp = NewClass()
        $ tmp.convert( myStats )
        $ myStats = tmp

 

[f95zoneuser463]

It happen because the error don't come from Ren'py code, but directly from pickle. It could be possible to catch the exception with a try/except block, but it need too much changes.

You are right. I did not pay close attention the the exception.

Last night I modified the 'file_slots' screen in screens.rpy like this:

Spoiler: bad pointless code

You don't have permission to view the spoiler content. Log in or register now.

That was a waste of time and did not change anything. The exception gets raised from deep within the Ren'Py source-code. Just like after_load the renpy.jump will never be executed. So the way I see it, I can only catch this if I'd modify the Ren'Py source-code ... a terrible idea.

Do not check, make it effectively compatible.

That's practically impossible in my case. The example with the class/object-data is just the tip of a VERY large iceberg with many fundamental changes. There is just no way this can be made compatible.

Meanwhile I've chatted with another developer about this and his solution was 'brutal':
-keep track of the last played version in the persistent data
-when the game starts check the last played version and if it's incompatible delete all savegames without ever asking the user
*sigh* ... awful solution

At first I would go into options.rpy and change "define config.save_directory" so that people won't pull old saves from their AppData folder (this fails if people download the game and just copy it into the old directory though).

I'm changing the save_directory with this update, but for other reasons unrelated to the save-locking problem. This should help with this specific update. I'd just like to have a solution for save-locking that is "clean", so even if users manually copy saves the game should display message and prevent loading it.

In the past I did show a screen from after_load with a compatibility warning in big red text. From my experience it's pointless to just show a warning, people ignore it, I guess some players don't speak englisch. So I ended up getting messages about bugs caused by very old saves in very bad english. For the future I have a list of compatible versions. example ["0.10.0","0.10.1","0.10.2"]. I no longer want this just to be a warning that can be ignored. I want to block loading old saves when I know they will cause problems. In other words: save-locking.
Just checking the savegame-version in after_load like I did in previous version will not work due to the exception from cpickle being raised before that.

I think this is a problem with Ren'Py. There is just no clean way to handle problems that occur during a load, only after.

 

[anne O'nymous]

That was a waste of time and did not change anything. The exception gets raised from deep within the Ren'Py source-code.

Yep, probably somewhere in loadsave.py. That's why I said that it's a solution that need too much changes. Unless you edit directly the core, Ren'py will catch the exception before it reach your own code.

That's practically impossible in my case. The example with the class/object-data is just the tip of a VERY large iceberg with many fundamental changes. There is just no way this can be made compatible.

Don't take me wrong, I'm sure that you thought about it a lot, but are you really sure ? Not that I don't believe you, but I've heard this so many time that now I tend to doubt.
Unless if it's a radical change on the game mechanism, where you can't make association between an old value and a new one, there's almost always a way to do it. Which didn't mean that it's an easy way and that forcing a restart isn't the simplest way to do, both for you and the player.

Meanwhile I've chatted with another developer about this and his solution was 'brutal':

There's a softer method.

Firstly, keep the definition of your classes to prevent pickle to complain. You don't even need to keep the full definition of the classes, pickle don't care about the methods. And like my memory tell me that the variables attributes are created on the fly, this should be enough to prevent pickle to complain :

    class State(renpy.store.object):
         pass

If it's not enough, just keep the __init__ method and use it to create (with a None value) every attributes.

So, now old save can load, which solve half of the problem. The second half is solved more or less like I said in my previous comment. Except that this time you don't even try to correct the problem. You just use one test to define if it's a save from the now incompatible version, or not.
And one test is really enough. Just be sure to test something that can only be found in the old version of the code, and that will be present whatever the said version. The best way being probably something like :

label after_load:
    # Case one : A variable will never exist in the new version.
    if hasattr( store, "tracy" ) is True:
        "Sorry, due to deep changes, you need to start a new game."
        $ renpy.full_restart()

    # Case two : The variable will exist, but be something different.
    if isinstance( tracy, store.State ) is True:
        "Sorry, due to deep changes, you need to start a new game."
        $ renpy.full_restart()

Obviously, keep only one of the two cases, depending of what you intend to do. And it also mean that the replacement class need to have another name that "State".


I can't really test it, but I'm 99% sure that it's enough to do the trick and let the player try to load an incompatible save file, without other result than you sending him back to the main menu.

 

[f95zoneuser463]

Firstly, keep the definition of your classes to prevent pickle to complain.

Yes that works to get past pickle. I only did a quick test, the empty class wasn't enough for pickle to stop complaining. But with additional dummy-vars/functions it works.

But it doesn't stop there. If I'd only have to keep a few of old "dummy"-classes around that would be okay. Over 100 screens got changed in various ways (merged/deleted/replaced/renamed), only god knows how many labels I've renamed, probably over 100 too. Obviously I don't have exact numbers for the changes, I can only guess them from the lint statistics and they are not an exaggeration. That triggers a never ending flood of exceptions before I can even reach after_load.


Meanwhile I came up with this code to catch the exception inside the Ren'Py source code:

## loads() can be found in <RENPY SDK>/Renpy/Common/loadsave.py

## ORIGINAL CODE
def loads(s):
    if renpy.config.use_cpickle:
        return cPickle.loads(s)
    else:
        return pickle.loads(s)

## MODIFIED CODE
def loads(s):
    try:
        if renpy.config.use_cpickle:
            return cPickle.loads(s)
        else:
            return pickle.loads(s)
    except:
        # We're in Python, Ren'Py code cannot be executed directly from here
        # jump to label "bad_savegame" deal with it
        renpy.python.py_exec("renpy.jump(\"bad_savegame\")")

I've tested this and it works.

Assuming I want to publish the game with this modified Ren'Py version ... do I just make the changes and hit "Build Distributions" and it builds a zip with the modified version? I hope ... I'm not sure how the Ren'Py Launchers builds the distributions.

 

[anne O'nymous]

Assuming I want to publish the game with this modified Ren'Py version ... do I just make the changes and hit "Build Distributions" and it builds a zip with the modified version? I hope ... I'm not sure how the Ren'Py Launchers builds the distributions.

As far as I understood it, the SDK take the content of its own renpy and lib directories. But there's a better way to do it, one that don't need you to effectively change the code :

init python:
    def loadsModded(s):
        try:
            if renpy.config.use_cpickle:
                return cPickle.loads(s)
            else:
                return pickle.loads(s)
        except:
            renpy.python.py_exec("renpy.jump(\"bad_savegame\")")

    renpy.loadsave.loads = loadsModded

It will replace the regular loads function by your own. So, the change, without the need to effectively change the source of the core.

 

[f95zoneuser463]

@anne O'nymous
Perfect, that works.
I had to add imports for cPickle and pickle. Otherwise it would always trigger an *invisible* exception because it wasn't able to find the pickle-modules and jump to "bad_savegame". Or use "return renpy.cPickle.loads(s)". I don't know where Ren'Py stores the pickle module, it's not in renpy.pickle ... *meh* doesn't matter. Maybe it's not even loaded if renpy.config.use_cpickle is True or something ... it's 4:00 nighttime, I rly need to sleep.

Thanks for the help.

 

[anne O'nymous]

I had to add imports for cPickle and pickle.

Oops Was already half asleep, sorry.

I don't know where Ren'Py stores the pickle module, it's not in renpy.pickle ... *meh* doesn't matter.

For later use :
In Python everything is object, even modules. So you have the renpy attribute in the 'store', which represent everything in the core. Then behind it you'll find every single modules, like in this case renpy.loadsave. And each one of them have an attribute representing the modules they loaded.
Therefore, among the different way to access them, you can find both pickle and cpickle, in renpy.loadsave.pickle and renpy.loadsave.cpickle.

Thanks for the help.

You're welcome.

 

[Rich]

Just to toss something in here, of late I've been experimenting with avoiding having class instances saved as part of my game data at all. In my latest approach, all my classes get created at "init" time, and use a slightly twisty approach to creating named dictionaries (nested, in some cases) in the Ren'py store, into which they read/write their values at runtime. As a result, if I make changes to the class names, etc, nothing breaks. If I need to restructure the data, all I have to do is make sure the later code can get back to the original dictionaries in the store, which is straightforward.

I'm still fiddling with the technique a bit to cut down on the amount of code that needs to be written to use it, but it seems to work. When I get it refined somewhat more, I'll do a post on it.

 

[anne O'nymous]

I'm still fiddling with the technique a bit to cut down on the amount of code that needs to be written to use it, but it seems to work. When I get it refined somewhat more, I'll do a post on it.

Have you tried the

You must be registered to see the links

and

You must be registered to see the links

meta methods ? They serve as interface between the class and pickle/cpickle. By default pickle use the __dict__ dictionary to retrieve/restore the data, but if one of these methods exist, they'll be used instead. __getstate__ can return whatever you want (a single value, a tuple, a list, a dictionary, some nested structure), and __setstate__ will receive the exact same thing.

So by example you can clean the __dict__ if it include data that aren't meant to be saved (like temporary ones or computed ones) :

class MyClass():
    def __getstate__( self ):
        tmp = self.__dict__
        del tmp["computedValue"]
        del tmp["temporaryValue"]
        return tmp

Then pickle will restore the data by filling directly __dict__. So just be sure to have something like this in your class:

    def someMethod( self ):
        if not "computedValue" in self.__dict__ or self.computedValue is None:
            [compute again]

You can also use it to effectively save a class that use unpickable values. By example, for a trigger class I use renpy.python.py_eval_bytecode to optimize the eval process, but obviously the result isn't pickable. So my class have something that looks like this :

class Trigger():
    def __getstate___( self ):
        return self.__rawEval

    def __setstate___( self, data ):
        self.__rawEval = data
        self.evalString = renpy.python.py_eval_bytecode( data )

Then finally you can use __setstate__ to handle changes in your class :

class MyClass():
    def __setstate__( self, data ):
        # Assume that you didn't thought about it before, so pickle will
        # give you the __dict__ of the class.
        if "oldName" in data:
            data["newName"] = chr( 60 + data["oldName"] )
            del data["oldName"]
        if "nowAProperty" in data:
            data["_nowAProperty"] = data["nowAProperty"]
            del data["nowAProperty"]
        self.__dict__ = data

As far as I understood the way you use the external dictionaries, it let you do more or less the same things, without the need to keep an external copy of the values. You can even rewrite totally the class. As long as the name of the class stay the same and you have a __setstate__ method to handle the conversion, it should work.

 

[Rich]

Have you tried the

You must be registered to see the links

and

You must be registered to see the links

meta methods ?

Yes, I'm well aware of those methods, but thanks (as always) for pointing them out for the benefit of others here. In my case, I'm trying to accomplish a number of things simultaneously, which is one reason I didn't go that route. It's quite possible that I could have gone that way - it's just that my alternate method was what occurred to me first, and "if it ain't broke..."

Plus, (as I understand it) that approach still requires class information to be part of the save, which means that you don't escape the "can't get rid of the class definition" problem. Although, of course, again the "class that does nothing other than satisfy the save requirements" approach does work, and implementing a dummy __setstate__ for "dead" classes is certainly one way of handling that.

 

[f95zoneuser463]

Marking post as solved. Posting cut down copypasta for reference other devs. If you see something bad or improve it report back please.

Important changes:

• added flag to disable the safelock, pretty much required during development
• bad_save label is being called instead of jumped (this is to make sure the game continues from where the user tries to load and it fails due to the safelock)

The safelock can be triggered in 3 ways:

• during the load with a pickle exception
• after the load if the save had no 'save_version' variable
• after the load if the save had a 'save_version' but it's not in the list of supported versions

## current version:
define config.version = "1.10.2"
## list of compatible versions:
define compatible_versions = ["1.09.0","1.09.1","1.09.2","1.10.0","1.10.1",config.version]

## to debug and for safelock screen
default bad_save_reason = "unknown"

init -3 python:
    if persistent.savelock is None:
        persistent.savelock = True

init python:
    def loadsModded(s):
        if persistent.savelock:
            try:
                if renpy.config.use_cpickle:
                    return renpy.loadsave.cPickle.loads(s)
                else:
                    return renpy.loadsave.pickle.loads(s)
            except:
                store.bad_save_reason = "pickle exception"
                renpy.python.py_exec("renpy.call(\"bad_save\")")
        else:
            if renpy.config.use_cpickle:
                return renpy.loadsave.cPickle.loads(s)
            else:
                return renpy.loadsave.pickle.loads(s)

    renpy.loadsave.loads = loadsModded

    def compatible_version(v):
        if v in compatible_versions:
            return True
        return False

    def compatible_versions_string():
        s = ""
        for v in compatible_versions:
            s += v + "  "
        return s

    def check_save():
        if persistent.savelock:
            try:
                if not compatible_version(store.save_version):
                    # check fail due to incompatible save_version
                    store.bad_save_reason = "save_version is not in compatible_versions-list"
                    renpy.call("bad_save")
                # check okay, set new savegame version
                store.save_version = config.version
            except AttributeError:
                # check fail due to very old save without save_version variable
                store.bad_save_reason = "save_version does not exist"
                renpy.call("bad_save")

    config.after_load_callbacks = [check_save]

################################################################################
## SAFELOCK SCREEN (fix design to match your project)
################################################################################
screen s_bad_savegame_version():
    zorder 200
    modal True
    text "{color=#F00}BAD SAVEGAME VERSION{/color}" size 60
    text "compatible versions:\n{color=#0F0}"+compatible_versions_string()+"{/color}" size 30
    hbox:
        if not renpy.context()._main_menu:
            textbutton _("Main menu"):
                action Hide("s_bad_savegame_version"),MainMenu(confirm=False)

        textbutton _("Return"):
            action Hide("s_bad_savegame_version"),Return()

    if config.developer:
        text "DEBUG bad_save_reason: [bad_save_reason]" align 0.5,0.95 size 30 color "#F00"

################################################################################
## BAD SAVE
################################################################################
## If persistent.savelock is enabled it can be called from:
## -> check_save()
## -> loadsModded()
label bad_save:
    scene black
    play sound audio.denied
    call screen s_bad_savegame_version
    return

################################################################################
## START
################################################################################
label start:
    default save_version = config.version
    ## optinal to keep track of the version where a save was started:
    default save_version_initial = config.version