Ren'Py Lists and persistent data

Vernam

Newbie
Jan 10, 2021
58
172
I've been trying for a while without success, maybe someone can help me. :(

My goal is to add an "Achievements" system to the game so that, at some point in the VN, a character (male female or both together) appears and notifies the player he unlocked an "Achievement".
The unlocked one are added to a menu as thumbnails and I want them to be permanent, even after restart.

To do so I define the Achiev class and a function to check if it's already in the unlocked list, call the animation and to fill the unlocked Achiev list if necessary:
You don't have permission to view the spoiler content. Log in or register now.

I create a full list of possible Achiev and an empty list of unlocked one:
You don't have permission to view the spoiler content. Log in or register now.

I create a screen for the Achievements menu:
You don't have permission to view the spoiler content. Log in or register now.

I add some Achiev and run the game to test it:
You don't have permission to view the spoiler content. Log in or register now.


The animations works fine, Achievements menu is filled properly and if I rollback or I load an older savegame the thumbnails stay there but when I reload the game the gallery is always empty...

I am a total beginner and probably I am overcomplicating it but shouldn't persistent.AchUnlocked stay there even after reload?
I have to add something else to make it persistent?
Can lists be persistent?

I feel I am missing something stupid. :(
 

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
In general, define is used for data which is only held in memory while the game is running.

You probably want to try default.

That is...

Python:
default persistent.AchUnlocked = [] # Persistent list of unlocked Achievements
#default AchievList = []            # only if it is to be saved (and potentially overwritten when someone loads an old save game file).
# -or-
define AchievList = []              # if it is data that will change as the game is developed and shouldn't be saved.

Edit2: Based on usage, AchievList() should definitely still be a define.

That said, persistent. is kinda special, and I'm surprised it's getting lost. I haven't tested your code, but I can imagine that AchievList() could easily get lost.

This is my quick answer, while I throw your code into a testbench project to see if I can find a better answer.
 
Last edited:

anne O'nymous

I'm not grumpy, I'm just coded that way.
Modder
Donor
Respected User
Jun 10, 2017
10,957
16,191
Code:
init python:
    def achiecall(n,sex):                        #to chose Achiev number n with appropriate animation
        [...]
        persistent.AchUnlocked.append(AchievList[n])      #adds the unlocked Achiev to the list
I'm tired and my brain is fried, but it really seem that there's a big "boom" here.
Try with :
Code:
init python:
    import copy

    def achiecall(n,sex):                        #to chose Achiev number n with appropriate animation
        [...]
        persistent.AchUnlocked.append( copy.copy( AchievList[n] ) )      #adds the unlocked Achiev to the list
 
  • Like
Reactions: Vernam

Vernam

Newbie
Jan 10, 2021
58
172
Thank you for your help!

I already tried both default and define with no success.
I even tried an explicit $renpy.save_persistent() but obviously didn't change anything.

With:
Code:
init python:
    import copy

    def achiecall(n,sex):                        #to chose Achiev number n with appropriate animation
        [...]
        persistent.AchUnlocked.append( copy.copy( AchievList[n] ) )      #adds the unlocked Achiev to the list
the game runs, unlocks the gallery but still resets after reload.

I was thinking of using some kind of parallel simple true/false flag, after that there's only a shaman with some vodoo.:(

Ps: If it helps I use Renpy 7.4.2.1292 .
 

Rich

Old Fart
Modder
Donor
Respected User
Game Developer
Jun 25, 2017
2,565
7,378
Persistent data is kind of special. If you're doing
Code:
default persistent.AchUnlocked = []
I suspect the "default" statement may be resetting the persistent list every time the game starts, because persistent data isn't treated the same way that "regular" variables are.

persistent will always be there, whether or not you initialize it.

persistent.anything will always be None unless you've assigned a value to it.

So the only issue is making sure that it has the AchUnlocked list on it. To ensure that, I'd use:
Code:
if persistent.AchUnlocked is None:
    persistent.AchUnlocked = []
You could put those two lines of code right after your "start" label (for someone who's starting the game fresh) and also in "afterLoad" (to pick up someone who already has a save).

After that, you should be fine, I think. But don't use "default" or "define" with "persistent."
 

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
Code:
default persistent.AchUnlocked = []
I suspect the "default" statement may be resetting the persistent list every time the game starts, because persistent data isn't treated the same way that "regular" variables are.

Nahh. I actually tested that before I suggested it.

I realize persistent. is another of RenPy's exceptions, but in this case default works how I would expect it to. The default value is only overwritten if the variable doesn't exist in the first place (just like other uses of default).

This is the script I used to prove it:

Python:
default persistent.testarray = ["Apples", "Bananas", "Cars", "Dogs", "Elephants"]

label start:

    scene black with fade

    "Start:.... "

    $ lenarray = len(persistent.testarray)
    "Length of array is [lenarray]"

    $ persistent.testarray.append("Frogs")

    $ lenarray = len(persistent.testarray)
    "New length of array is [lenarray]"

    "*** THE END ***"
    return

First time through, the length of the array is 5. Each time the game is run, it increases by 1. Including if you exit the game completely and run it from the command prompt.

I should point out I'm still using RenPy 7.3.5, in case that matters.
 

Rich

Old Fart
Modder
Donor
Respected User
Game Developer
Jun 25, 2017
2,565
7,378
Nahh. I actually tested that before I suggested it.
OK, I stand corrected. I ran into a problem like this at one point in the past, but that was well in the past, so perhaps it's been corrected since then.
 

Vernam

Newbie
Jan 10, 2021
58
172
The more I test it the less I understand it...

with:
Code:
if persistent.AchUnlocked is None:
    persistent.AchUnlocked = []
if it's inside the start label if the player goes directly in the gallery without starting the game it gives error, so i try use:
Code:
init python:
    if persistent.AchUnlocked is None:
        persistent.AchUnlocked = []
and works but resets after restart...


I tried with a direct $ persistent.AchUnlocked.append(AchievList[0]) to avoid use the achiecall() function but it resets after reload.

I've tried also using, instead of an Achiev object, a list of numbers or text (like persistent.AchUnlocked.append([n])) but that's not the problem, behaves perfectly during the game (number, text or class) and resets on exit.


I found out that starting with a non-null list, like in persistent.testarray :

default persistent.AchUnlocked = [Achiev("Achiev0name",Image("Achiev0.png"),"Achiev0desc1","Achiev0desc2")]
or
define persistent.AchUnlocked = [Achiev("Achiev0name",Image("Achiev0.png"),"Achiev0desc1","Achiev0desc2")]
or
Code:
init python:

    if persistent.AchUnlocked is None:

            persistent.AchUnlocked = [Achiev("Achiev0name",Image("Achiev0.png"),"Achiev0desc1","Achiev0desc2")]
The achievement is in the gallery even after restart.
I think that, as stated, for some reason it overwrite the list every time the game starts but i don't understand why...

default persistent.testarray = ["Apples", "Bananas", "Cars", "Dogs", "Elephants"] works and the Frogs stay there...

There's a game with a similar feature so i can copy/study it?
I really feel I am missing something stupid... :confused:
 

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
Stab in the dark (sorry, still haven't gotten to grips with your code in my testbench code).

Try:
class Achiev (renpy.python.RevertableObject):
class Achiev (renpy.python.RevertableList):

Edit: Nevermind. It is just a temporary workspace for initializing the main AchUnlocked() array. As such, it probably won't feature in the main game .

... which will inherit from the RevertableObject thingy that makes classes support rollback. I don't think it's that, but it's been bugging me since I first saw your message - and it might be related.

Honestly, I've never gotten to grips with classes yet. But since my example is an array of strings (works) and your example is an array of your custom class Achiev() (doesn't work)... I can only presume it's a custom class thing... and my one tiny bit of knowledge of custom classes in RenPy games is that they should be tied into the RevertableObject type objects.

Edit2: This is a red herring. Ignore me. It is already revertable.
 
Last edited:

anne O'nymous

I'm not grumpy, I'm just coded that way.
Modder
Donor
Respected User
Jun 10, 2017
10,957
16,191
Code:
init python:
    if persistent.AchUnlocked is None:
        persistent.AchUnlocked = []
and works but resets after restart...
Just change it into:
Code:
init python:
    if persistent.AchUnlocked is None:
        print "AchUnlocked created"
        persistent.AchUnlocked = []
It will print on the console when the if block is proceeded. But I assume that it shouldn't appear since AchUnlocked is supposed to be in the persistent file since a long time now.



I tried with a direct $ persistent.AchUnlocked.append(AchievList[0]) to avoid use the achiecall() function but it resets after reload.
Try to force the save by adding $ renpy.save_persistent() right after it.


I've tried also using, instead of an Achiev object, a list of numbers or text (like persistent.AchUnlocked.append([n])) but that's not the problem, behaves perfectly during the game (number, text or class) and resets on exit.
No, no and no. The persistent pseudo store can't be broke, it would be known :/ But it also have no reason to reset by itself.
Therefore, either you're deleting the persistent data, there's something in your code that delete its value, or it's not effectively empty and the error is in the way you works with it.

There's something that I haven't noted at first, but that is really troubling. In your initial message, you wrote :
Code:
label start:
    [...]
    "[persistent.AchUnlocked[0]]" # For testing, returns <store.Achieve object at 00...>
What mean that persistent.AchUnlocked was NOT empty when the game start in your initial tries.
 

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
Okay. Finally getting somewhere with this... nowhere nice, but somewhere.

For some reason I can't figure out, when you test if persistent.AchUnlocked is None: during init python:, the result is TRUE... even if a value has be loaded into persistent.AchUnlocked during the previous playthrough. I can sort of explain it, but I'd rather stick with things I can prove at this stage...

If I use if persistent.AchUnlocked is None: or default persistent.AchUnlocked = [], the variable is ALWAYS initialized, regardless of any previous values.

I (eventually) tested this by running my test script, then removed my default persistent.AchUnlocked = [] line from the code before running it again. With the line removed, the persistent values were retained. (The same is true for the if persistent.AchUnlocked is None: check too, before anyone jumps on that).

A quick look in the log.txt file and I found:
Code:
AttributeError: 'StoreModule' object has no attribute 'Achiev'

I came to the conclusion that because AchUnlocked is a list of Achiev() objects, there was somehow a problem during initialization referencing a class that doesn't actually exist yet.

I tried creating the class during init -1 python:. That didn't work.

I then found python early: ... and that does work. If you create the Achiev() class during "python early", it lets you use any method you like to initialize the array/list in anyway you like. And yes, default persistent.AchUnlocked = [] works just fine.

Now you start to run into more problems. As long as you don't quit out from the game, everything runs fine. But when you do exit the game completely, the next time it runs it effectively creates a new AchievList array. Well, not exactly new... but stored at a different location in memory. As a result, when you check to see if the achievement has already unlocked, even though the values are the same, the memory location is different and the test fails (at least that is what I assume is happening)... which results in the same achievement being added to the unlocked list again... and again... and again... and again, depending on how many times you run that code.

Let me come back to that one...

Meanwhile, there's another issue. Because of the way you are initializing the AchievList, it changes the value of an already existing variable... which results in that variable being saved as part of the save game data. So currently there are two possible achievements. If you run the game and save, those two values are saved. Then, if later, you add a third and fourth achievement but load a saved game from when only two existed... those 3rd and 4th will be lost for that playthrough.

The answer is to NOT save AchievList by making it a constant. That is... use define and NEVER change the values while the game is running. Nice and simple...

Python:
define  AchievList = [
            Achiev("Achiev0name",Image("Achiev0.jpg"),"Achiev0 desc1","Achiev0 desc2"),
            Achiev("Achiev1name",Image("Achiev0.jpg"),"Achiev1 desc1","Achiev1 desc2"),
        ]

If you do already have some saves, you may want to delete them - since they may already include AchievList.

Again, the problem here is that each time the game is run from scratch, the lists are stored in different locations in memory (again, I assume) and python/RenPy cares about that when it does checks like if not AchievList[n] in persistent.AchUnlocked:.

The answer (in my opinion) is to match the achievement name rather than doing a direct compare of the Achiev() objects.

This seems to work for me:

Python:
    def achiecall(n,sex):     # sex is ignored here, because I don't know how it is used.
        foundmatch = False
        for x in persistent.AchUnlocked:
            if x.name == AchievList[n].name:
                foundmatch = True
        if not foundmatch:
            persistent.AchUnlocked.append( AchievList[n] )

My concern is that the achievement numbers could change over time. So I think a better way would be to add achievements based on their name rather than their array number.

My final test script looks like this:

Python:
python early:

    class Achiev:
        def __init__(self, name, thumb,  desc1, desc2):
            self.name = name
            self.thumb = thumb         #thumbnail image of the gallery
            self.desc1 = desc1
            self.desc2 = desc2

        def __eq__(self, other):       #to compare Achievs, without it gives error
            if not isinstance(other, Achiev):
                return False

            return self.name is other.name



init python:

    def achiecall(achname,sex):     # sex is ignored here, because I don't know how it is used.
        for ach in AchievList:
            if achname == ach.name:
                foundmatch = False
                for x in persistent.AchUnlocked:
                    if x.name == ach.name:
                        foundmatch = True
                if not foundmatch:
                    persistent.AchUnlocked.append( ach )

    gallery_page = 0


define  AchievList = [
            Achiev("Achiev0name",Image("Achiev0.jpg"),"Achiev0 desc1","Achiev0 desc2"),
            Achiev("Achiev1name",Image("Achiev0.jpg"),"Achiev1 desc1","Achiev1 desc2"),
        ]

default persistent.AchUnlocked = []


label start:

    scene black with fade

    "Start:.... "

    $ templen = len(persistent.AchUnlocked)

    if templen > 0:
        $ tempach = persistent.AchUnlocked[0].name
    else:
        $ tempach = "<not yet set>"

    "AchUnlocked is [templen] objects. The first achievement is [tempach]."


label add_achievment:

    menu:
        "Unlocked Achievement ZERO":
            $ achiecall("Achiev0name",3)
            "Unlocked Achievement 'Achiev0name'."
            jump add_achievment

        "Unlocked Achievement ONE":
            $ achiecall("Achiev1name",2)
            "Unlocked Achievement 'Achiev1name'."
            jump add_achievment

        "I'm finished here":
            pass

    $ templen = len(persistent.AchUnlocked)
    if templen > 0:
        $ tempach = persistent.AchUnlocked[len(persistent.AchUnlocked)-1].name
    else:
        $ tempach = "<not yet set>"

    "AchUnlocked is [templen] objects. The last achievement is [tempach]."

    menu:
        "Delete achievements before ending?"
        "No":
            pass

        "Yes (reset)":
            $ persistent.AchUnlocked = []

        "Yes (full delete)":
            $ del persistent.AchUnlocked
            "NOTE: Game will now exit (otherwise errors happen)."
            $ renpy.quit()

    "*** THE END ***"
    return

Part of me thinks that I too am missing something here. Probably something to do with def __eq__(self, other): and how classes are compared. But honestly, other than knowing that python create pointers to other variables when you do stuff like $ x = y, where x is just a pointer/reference to y, I'm so far out of my depth.

In addition, part of me wants to say that the achievement system should be some form of rather than a list. But that is taking me even further out of my depth.

Perhaps anne O'nymous or Rich could correct my thinking.


Edit: The more I think about this, the more I'm doubting my "it's something to do with memory locations" things. Maybe someone could take a closer look at:

Python:
       def __eq__(self, other):       #to compare Achievs, without it gives error
            if not isinstance(other, Achiev):
                return False

            return self.name is other.name

... when used in relation to something like...

Python:
   def achiecall(n,sex):
        if not AchievList[n] in persistent.AchUnlocked:
            persistent.AchUnlocked.append(AchievList[n])
 
Last edited:
  • Like
Reactions: anne O'nymous

Rich

Old Fart
Modder
Donor
Respected User
Game Developer
Jun 25, 2017
2,565
7,378
Okay. Finally getting somewhere with this... nowhere nice, but somewhere.

For some reason I can't figure out, when you test if persistent.AchUnlocked is None: during init python:, the result is TRUE... even if a value has be loaded into persistent.AchUnlocked during the previous playthrough. I can sort of explain it, but I'd rather stick with things I can prove at this stage...
It could have something to do with when - exactly - during the initialization process persistent is set up by Ren'py. When it actually inits that variable's contents.

I've never used persistent in an init block. If I was coding it, I'd probably do something like:

Code:
def is_item_in_persistent(x):
    if persistent.item_list is None:
        return False
    return x in persistent.item_list

def add_item_to_persistent(x):
    if persistent.item_list is None:
        persistent.item_list = []
    persistent.item_list.append(x)
In other words, just deal with the possible None at runtime, rather than trying to init it.

I came to the conclusion that because AchUnlocked is a list of Achiev() objects, there was somehow a problem during initialization referencing a class that doesn't actually exist yet.
Yes, that makes sense. When Ren'py goes to "unpickle" the file that stores the persistent data, it needs the class to exist.

Now you start to run into more problems. As long as you don't quit out from the game, everything runs fine. But when you do exit the game completely, the next time it runs it effectively creates a new AchievList array.
Yes - every time you run, all the objects get recreated from scratch. This means that when the persistent data is unpickled, if it contains references to objects that are defined at init time, things won't work. The unpickling operation will create its own set of nested objects, and they won't be the same physical objects as the ones you created at init time.

Part of me think that I too am missing something here. Probably something to do with def __eq__(self, other): and how classes are compared. But honestly, other than knowing that python create pointers to other variables when you do stuff like $ x = y, where x is just a pointer/reference to y, I'm so far out of my depth.

In addition, part of me wants to say that the achievement system should be some form of rather than a list. But that is taking me even further out of my depth.
You CAN probably get around your problems by overriding __eq__. One way of doing this would be to assign a unique "id" to each class instance at the time you create it (i.e. pass that value into the constructor) and then have __eq__ check the ID's.

But, to be frank, the way I'd go about it would be to do that (i.e. assign ID's to all your "init"-ed objects), but then just store the ID's (strings?) in the list in persistent. Then you don't have to worry about creating the classes early, about dealing with different instances before and after restarting the game, etc.
 
  • Like
Reactions: 79flavors

anne O'nymous

I'm not grumpy, I'm just coded that way.
Modder
Donor
Respected User
Jun 10, 2017
10,957
16,191
I came to the conclusion that because AchUnlocked is a list of Achiev() objects, there was somehow a problem during initialization referencing a class that doesn't actually exist yet.
[...]
I then found python early: ... and that does work.
Well, it's explicitly said in : "As persistent data is loaded before init python blocks are run, persistent data should only contain types that are native to Python or Ren'Py. Alternatively, classes that are defined in python early blocks can be used, provided those classes can be pickled and implement equality."

Should have thought about that, instead of seeing the problem from my own eyes :(
 
  • Like
Reactions: 79flavors

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
Yeah. I'm wondering if the persistent data should just be a list of achievement names instead of a copy of the whole Achiev() object.

As simple strings... most of this would go away.

i.e.

persistent.AchUnlocked would be equal to [ u'Achiev0name', u'Achiev1name' ]
... once those two achievement have been unlocked.
 

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
Well, it's explicitly said in : "As persistent data is loaded before init python blocks are run, persistent data should only contain types that are native to Python or Ren'Py. Alternatively, classes that are defined in python early blocks can be used, provided those classes can be pickled and implement equality."

Yeah... But who ever reads those? :devilish:
 
  • Thinking Face
Reactions: anne O'nymous

Vernam

Newbie
Jan 10, 2021
58
172
Stab in the dark
You hit for triple damage :)

Thank you all for your help, this thing is the wall between me and a [complete] game.

I did some test, if someone is interested is my achievements project (warning: NSFW animations).

I try to use not only complex Objects but also some simple text or numbers, everything always resets after restart.
The only thing doesn't resets is a simple flag so I presume the problem isn't in the object itself or the content of the list.

What mean that persistent.AchUnlocked was NOT empty when the game start in your initial tries.
I add the objects ($achiecall(0,3) means add Achiev 0 with animation 3)and after that i check what's there and everything stay there until I restart the game.

Actually every time i look at the unlocked list (like "[persistent.AchUnlocked[0]]") it points to a different memory area (I assume they are the numbers after store.Achieve object at ... ) but, if this is the problem, why the frogs stay there in the other test program?
Why default persistent.testarray = ["Apples", "Bananas", "Cars", "Dogs", "Elephants"] is different?

I tried also with something simple like with a parallel list:
Code:
define persistent.TestText = ["TestText"]

init python:
    def achiecall(n,sex):
        ...
        persistent.TestText.append("TestText")
To use only simple text but it still resets after restart.


I suspect the "default" statement may be resetting the persistent list every time the game starts
I have a feeling something like this happens and resets everything at restart but I don't understand where and why.
I am a beginner so it can also be a stupid error.
I'll look over Dic() and the other code and see what happens.

For now, other possible reasons for me are::)
- forgot to buy the persistent. DLC from Ren'py.
- menu->options-> enable persistent.
 

Rich

Old Fart
Modder
Donor
Respected User
Game Developer
Jun 25, 2017
2,565
7,378
I tried also with something simple like with a parallel list:
Code:
define persistent.TestText = ["TestText"]

init python:
    def achiecall(n,sex):
        ...
        persistent.TestText.append("TestText")
To use only simple text but it still resets after restart.
"define" puts something a statement in "init". "init" runs every time you start a game, not just when you fire the program up. So, in this case, the "define" is overwriting the list.

To what @79 Flavors and anne O'nymous said, before I came back to this thread, I found a statement by PyTom basically saying the same thing on the lemmasoft forums:

Ah! I realized what the problem is.

The problem is that the persistent data is loaded before the init code runs. So testObject is undefined when the persistent data is loaded, persistent loading fails, and Ren'Py resets the persistent data.

If you really want to, you can define testObject in a python early block. But honestly, it's probably best to only store built-in data types in the persistent data.
"resets the persistent data" (possibly) being the key phrase when you were dealing with classes.

I still think the best solution is to handle the "the list might be None" in the functions that access it, rather than using "default" (wrong) or "define" (maybe correct). But that's just my HO.
 

anne O'nymous

I'm not grumpy, I'm just coded that way.
Modder
Donor
Respected User
Jun 10, 2017
10,957
16,191
I still think the best solution is to handle the "the list might be None" in the functions that access it, rather than using "default" (wrong) or "define" (maybe correct). But that's just my HO.
Or your ID approach. Something like :
Code:
init python:
    if persistent.AchUnlocked is None:
        persistent.AchUnlocked = []

define AchievList = [ "Achiev0name": Achiev("Achiev0name",Image("Achiev0.png"),"Achiev0desc1","Achiev0desc2"), 
                               "Achiev1name": Achiev("Achiev1name",Image("Achiev1.png"),"Achiev1desc1","Achiev1desc2"),
                               [...] ]

label whatever:
    $ persistent.AchUnlocked.append( "Achiev1name" )

screen Achievements:
    [...]

        for i in range(start, end + 1):
            imagebutton :
                idle AchievList[persistent.AchUnlocked[i]].thumb
    [...]
A persistent list that limit to basic type, while still having the benefit from the object.
 
  • Like
Reactions: Rich

79flavors

Well-Known Member
Respected User
Jun 14, 2018
1,607
2,256
For now, other possible reasons for me are:

Okay. Slightly concerned you quoted my post where I was on the wrong track and completely seem to have missed the huge post where I go through things step by step explaining what is going wrong and what doesn't work and how it could be fixed.

Not that my fixed version is ideal and does go against pytom's advice for not using non-predefined classes within persistent data. But it does actually work.

Then Anne and Rich's followup posts go into a bit more accurate detail.

Meanwhile, downloading TestPers-1.0-pc.zip now. Let's see if I can get that working AND keep on the right side of .