Ren'Py Do 'python early' blocks mess with the lifecycle of Ren'Py somehow? Persistent variables contained in custom classes are ignored

[EviteGames]

Hey,

I have the most weird bug I've encountered so far.

I'm building a scene gallery with support for multi-part scenes. Multi-part scenes basically mean that you can obtain only certain parts of a scene and miss others. You might not have had enough points for... stuff.

The checkpoints are stored in persistent variables, along with the SceneItem classes as well. This class holds every data related to a scene.

The checkpoints are displayed under the thumbnail of the scene in the gallery. This can either be a checkmark or an X, for simplicity's sake.

Since I'm using custom classes to store the scene item's information, I had to declare the class in an init python early block (to be able to persist the class). I imagine this is causing my problem right now.

The problem is that when the gallery screen is parsing through the scene item's multi_part_unlocks list (which is a list of persisted boolean variables), it seemingly ignores the persistent variable's value and defaults to false.

Here is the code, for clarity:

This is the Gallery implementation, the key point is how it parses through the multi_part_unlocks

init 501 screen scenegallery():
    tag menu

    python:
        import math

        GALLERY_SIZE = len(persistent.scene_list)
        GALLERY_COLUMNS = 3
        GALLERY_ROWS = int(math.ceil((float(GALLERY_SIZE) / GALLERY_COLUMNS)))
        EMPTY_COUNT = GALLERY_ROWS * GALLERY_COLUMNS - GALLERY_SIZE

    use game_menu(_("Scenene Gallery"), scroll="viewport"):
        style_prefix "underconst"

        $ print ("-----------------------")
        $ print("Adding scene gallery")
        $ print ("-----------------------")
        vbox:
            label "Replay your favourite moments"
            null height 50
            grid GALLERY_COLUMNS GALLERY_ROWS:
                spacing 30
            
                for scene_element in persistent.scene_list:
                    if scene_element.is_locked:
                        # Add locked image with no actions.
                        vbox:
                            text _(scene_element.item_name) xalign 0.5
                            imagebutton:
                                idle scene_element.locked_image
                    else:
                        # Add unlocked hover/idle image and action
                        vbox:
                            text _(scene_element.item_name) xalign 0.5
                            imagebutton:
                                idle scene_element.unlocked_idle_image
                                hover scene_element.unlocked_hover_image
                                action Replay(scene_element.label_name, locked = False)
                            if len(scene_element.multi_part_unlocks) > 0:
                                $ print("The following scene element IS multi-part: ", scene_element.item_name)
                                hbox:
                                    for persistent_variable in scene_element.multi_part_unlocks:
                                        if persistent_variable == True:
                                            $ print("Adding UNLOCKED button with scene element ", scene_element.item_name, "boolean value is", (persistent_variable))
                                            add "multi_part_unlocked"
                                        else:
                                            $ print("Adding LOCKED button with scene element ", scene_element.item_name,"boolean value is", (persistent_variable))
                                            add "multi_part_locked"
                            else:
                                $ print("The following scene element IS NOT multi-part: ", scene_element.item_name)

                for i in range(EMPTY_COUNT):
                    null

This is the SceneItem implementation:

# The SceneItem class is the basic model of a single element in the scene gallery
# item_id - Unique ID
# item_name - Item name shown in the Scene Gallery
# is_unlocked_by_default - Determines whether the item should be unlocked by default or not.
# unlocked_idle_image - Unlocked image resource, idle state
# unlocked_hover_image - Unlocked image resource, hover state
# locked_image - Locked image resource, has no states
# label_name - The label's name referred to, for replay.
# multi_part_unlocks - A list persistent boolean variables that control how much of the multi-part scene has been unlocked
init python early:
    class SceneItem:
        def __init__(self, item_id, item_name, is_unlocked_by_default, unlocked_idle_image, unlocked_hover_image, locked_image, label_name, multi_part_unlocks):
            self.item_id = item_id
            self.item_name = item_name
            self.unlocked_idle_image = unlocked_idle_image
            self.unlocked_hover_image = unlocked_hover_image
            self.locked_image = locked_image
            self.label_name = label_name
            self.is_locked = not is_unlocked_by_default
            self.multi_part_unlocks = multi_part_unlocks

        def __eq__(self, other):
            return self.item_id == other.item_id

        def unlock(self):
            self.is_locked = False

GalleryItems are created and appended as such (check the last item of the constructor, the list of variables):

    if persistent.scene_list is None:
        persistent.scene_list = []
        persistent.scene_list.append(SceneItem(1, "Sicily", True, "gallery_item_sicily_idle", "gallery_item_sicily_hover", "gallery_item_locked", "IntSicilyLewd", [persistent.fiona_other_stuff, persistent.fiona_another_stuff]))

The weird part is that when running this code, the behaviour is SUPER awkward. I dunno why. Since the screen 'functions' run multiple times, the print statements also do. And the results are inconsistent. No clue why. Here is the latest log:



As you can see, during the second pass, the iterations run more then they should and the code somehow enters the "unlocked" statement, even when the boolean value is false (which it shouldn't be). You can see it at the end of the sentence. "Adding UNLOCKED button with scene element Sicily, boolean value is False". This should never happen.

I'd appreciate any help which would bring me closer to figuring out this issue.

 

[anne O'nymous]

Since I'm using custom classes to store the scene item's information, I had to declare the class in an init python early block (to be able to persist the class).

Absolutely not.
The early init state is dedicated to user defined statements, in order for them to exist when Ren'py will parse the files, and it's its only role. It can happen that a class is defined at this moment, but only because it will be used by the said user defined statements, but it's the only reason why the class is defined in an init early block.

    python:
        import math

        GALLERY_SIZE = len(persistent.scene_list)
        GALLERY_COLUMNS = 3
        GALLERY_ROWS = int(math.ceil((float(GALLERY_SIZE) / GALLERY_COLUMNS)))
        EMPTY_COUNT = GALLERY_ROWS * GALLERY_COLUMNS - GALLERY_SIZE

It's not the place for a module importation. Importation should be global, not local, especially not local to a screen since they have their own context ; I'm not even sure that because of this question of context, it don't have some influence.
And also what's the interest to have something like int(math.ceil(...)), since math.ceil is specifically made to return an integer ?

                                            add "multi_part_unlocked"
[...]
                                            add "multi_part_locked"

What are you trying to achieve here ? The add screen statement is to add a displayable, either an image or a user defined displayable.
If you want to add an image, then the extension is missing, if you want to add a displayable, then it shouldn't be a string, and if you wanted to do something else, then it's absolutely not how you should do it.

I'd appreciate any help which would bring me closer to figuring out this issue.

Well, firstly simplify your code.

By example, you don't need to have things like :

           self.unlocked_idle_image = unlocked_idle_image
            self.unlocked_hover_image = unlocked_hover_image
            self.locked_image = locked_image

especially when the value are "gallery_item_sicily_idle", "gallery_item_sicily_hover", "gallery_item_locked"
According to its name, locked_image is a constant, you don't need to store it in each object. As for the two firsts, it can be "gallery_item_sicily_%s", that would permit to replace :

                               idle scene_element.unlocked_idle_image
                                hover scene_element.unlocked_hover_image

by :

                               auto scene_element.unlocked_image

And obviously, solve this add thing before everything.

 

[EviteGames]

Thanks for your reply!

Absolutely not.

I'm sorry but you are wrong here. If you want to persist data not native to Python or Ren'Py, you have to declare that class in an early Python block. It clearly says so in the

You must be registered to see the links

:

As persistent data is loaded before init python blocks are run, persistent data should only contain types that are native to Python or Ren'Py. Alternatively, classes that are defined in python early blocks can be used, provided those classes can be pickled and implement equality.

If you fail to do so, the data won't be persisted. You can check this directly with the code I provided. If you unlock a scene item with $ unlock_scene_item(ID), the item will unlock and will remain persisted if you restart the game. If you fail to declare the SceneItem class in an early block, you'll lose the data once you restart the game.

It's not the place for a module importation. Importation should be global, not local, especially not local to a screen since they have their own context

I'm not sure I follow you here. Isn't it supposed to be bound to the python block, and not the screen? My understanding of blocks in Ren'Py might be wrong, but when the game compiles, won't all python blocks (unless given an order prefix) run asynchronously (when needed in this specific scenario)? If you have a resource on this which I could read upon, that'd be great.

Where do you suggest the import should be?

And also what's the interest to have something like int(math.ceil(...)), since math.ceil is specifically made to return an integer ?

That's not the behaviour of math.ceil(float) in Python 2.7. In this version, the float value is rounded upwards to the nearest integer value, but there is no type conversion. 2.33333 will become 3.0, not 3. The behaviour change was made in Python 3.0. I didn't know this originally, but you can read up on this

You must be registered to see the links

.

What are you trying to achieve here ? The add screen statement is to add a displayable, either an image or a user defined displayable.
If you want to add an image, then the extension is missing, if you want to add a displayable, then it shouldn't be a string, and if you wanted to do something else, then it's absolutely not how you should do it.

I'm trying to add X number of locked/unlocked icons below the SceneItem's thumbnail, in a hbox. I thought that I'd simply use the add statement since the image is not interactive.

The raw PNG files are under game\images\gallery, named as multi_part_unlocked.png and multi_part_locked.png. My issue doesn't seem to be with displaying the icons but rather the code itself ignoring the content of the multi_part_unlocks list of a SceneItem and falling back to the False value, always.

This is how it looks, right now:


Note how the little markers are both Xs. That's the result of the code somehow not reading the persistent variables correctly. This is what I can't figure out.

According to its name, locked_image is a constant

It's not a constant. The reason I left it in the the constructor is that I wanna give different scene items different locked thumbnails, depending on various gameplay elements. It'd probably be better to make the locked_image an optional parameter which falls back to the default value, but that's just something I haven't done yet.

As for the two firsts, it can be "gallery_item_sicily_%s", that would permit to replace

I've never heard of this auto thing yet. I'll definitely check this out later, thank you!

However, I'm not closer to my solution

My hunch is that something is either wrong with the for loop which checks the multi_part_unlocks list inside a SceneItem, or something completely different.

But I've been stuck on this for the past few days when it comes to feature programming, and I've probably looked at that UI code like 200+ times. I might have just ran into a Ren'Py bug as well.

 

[anne O'nymous]

I'm sorry but you are wrong here. If you want to persist data not native to Python or Ren'Py, you have to declare that class in an early Python block.

My bad ; forgot that persistent are loaded by anticipation due to its nature.

I'm not sure I follow you here. Isn't it supposed to be bound to the python block, and not the screen?

Module importation are always global, so available whatever in the code once done ; unless the importation is made inside a function or class. By example, renpy/common/00achievement.rpy import a module under the name "steam" into the achievement store. You can address it, by example from the console, through achievement.steam. This despite the fact that you obviously aren't anymore in the init block that imported it, and that this importation is made from an alternate store. Therefore, just have an init block at the top of the file to import the module once for all.

It also imply that, no, the importation isn't bound to the Python block. But if it was, then it would be bound to the screen, since the said Python block is itself bound to this screen. And it's where it can, possibly, generate some problems since screen have their own context.
Normally everything should be alright, due to the global nature of importation, but context are really sensible, so I don't guaranty it.

My understanding of blocks in Ren'Py might be wrong, but when the game compiles, won't all python blocks (unless given an order prefix) run asynchronously (when needed in this specific scenario)?

They will, but it don't mean that they are isolated piece of code.
Ren'py is nothing more than a big and complex Python script. Anything, whatever it's a statement, an inline Python line, or a Python block, is just one piece of this big entity. Whatever happen in a Python block will have an impact on the rest of the script ; else the said blocks would be totally useless.

The behaviour change was made in Python 3.0. [...]

The fuck :/ It's the first time I mess between the two branches My bad again.

I'm trying to add X number of locked/unlocked icons below the SceneItem's thumbnail, in a hbox. I thought that I'd simply use the add statement since the image is not interactive.

You can, but it's really unusual to address the image entity in place of the file itself ; mostly because added images tend to be generic and so can have conflictual names.

My issue doesn't seem to be with displaying the icons but rather the code itself [...]

I know, but by itself your code is correct, so I focused on the wrong or unusual parts of the code, since they could have been the source of Ren'py confusion.

However, I'm not closer to my solution

What happen when you proceed the elements outside of a screen ? Something like :

label start:
    while True:
        call whatever
        "next loop"

label whatever:
    # There's no /for/ statement outside of screens :(
    python:
        for scene_element in persistent.scene_list:
            # If the list is empty, the loop will just be skipped.
            for persistent_variable in scene_element.multi_part_unlocks:
                if persistent_variable is True:
                     narrator( "{} value is {}".format( scene_element.item_name, persistent_variable ) )
                else:
                    narrator( "{} value is {}".format( scene_element.item_name, persistent_variable ) )
    return

Are the results constant, or do you also have some inconstancy like for the screen ?

My hunch is that something is either wrong with the for loop which checks the multi_part_unlocks list inside a SceneItem, or something completely different.

It looks more like a problem of context. It's totally not natural that something like your

                                       if persistent_variable == True:
                                            $ print("Adding UNLOCKED button with scene element ", scene_element.item_name, "boolean value is", (persistent_variable))

have a different value for persistent_variable in the if statement then right after in the print function.
More than "not natural", it should be totally impossible.

Therefore, my guess is that there's something that mess with the context, processing the statement in a context, and the function in a different one ; with the variable value being different in each context.

So, firstly you should eliminate the problem of the console. Normally the print function should be proceeded in the actual context, but the console having its own, perhaps that...
Replace those two print by text ( "Adding UNLOCKED button with scene element {} boolean value is {}".format( scene_element.item_name, persistent_variable ) )
Do you still have the inconstancy ?

If yes, then do not put your code as inclusion to the use statement. Or better, use the "navigation" screen in place of "game_menu" one ; still without including your code as block for the statement. Something like :

[...]
   use navigation

   frame:
        [probably need a positioning here]
        style_prefix "underconst"

        $ print ("-----------------------")
        $ print("Adding scene gallery")
        $ print ("-----------------------")
        vbox:
            label "Replay your favourite moments"
[...]

Variables sharing between an use[icode]d screen and its calling screen are really capricious. Therefore it's not impossible that there's effectively a double context because of this.

 

[EviteGames]

My bad ; forgot that persistent are loaded by anticipation due to its nature.
....

Hey! Sorry for the late reply, I was super busy in the past few days.

I've managed to figure out the issue. It was a wild ride, let me tell you that...

Apparently, if you do:

 persistent.scene_list.append(SceneItem(1, "Sicily", False, "gallery_item_sicily_idle", "gallery_item_sicily_hover", "gallery_item_locked", "IntSicilyLewd", [persistent.fiona_other_stuff, persistent.fiona_another_stuff]))

The values of those variables inside the list get 'snapshotted' upon creation. They are not updated after the fact.

Ren'Py seems to store their values, not their references.

They were False or None after they were created, and never updated.

I worked around this by storing only the names of the variables inside the SceneItem class, such as:

persistent.scene_list.append(SceneItem(1, "Sicily", False, "gallery_item_sicily_idle", "gallery_item_sicily_hover", "gallery_item_locked", "IntSicilyLewd", ["fiona_other_stuff", "fiona_another_stuff"]))

I needed to make the following changes to the logic which parsed the booleans:

                            if len(scene_element.multi_part_unlocks) > 0:
                                $ print("The following scene element IS multi-part: ", scene_element.item_name)
                                hbox:
                                    for persistent_variable_name in scene_element.multi_part_unlocks:
                                        if getattr(persistent, persistent_variable_name) == True:
                                            add "multi_part_unlocked"
                                        else:
                                            add "multi_part_locked"

And then it works, flawlessly.

Ren'Py is weird. I don't see a reason why it should copy the value instead of holding a reference to the object, but oh well.

Thank you so much for your help!

 

[anne O'nymous]

Ren'Py seems to store their values, not their references.

And then I also missed the end of your assignation line, [persistent.fiona_other_stuff, persistent.fiona_another_stuff] which is the problem ; clearly it wasn't the good day for me to answer, sorry.

Ren'Py is weird. I don't see a reason why it should copy the value instead of holding a reference to the object, but oh well.

Still there's a reason, and a particularly good one:
Due to the immutable nature of variables in Python, the notion of reference do not exist in this language ; at least not like you imagine it. Everything is a value, or more precisely an object representing this value ; and I mean really everything, even strings, numbers and boolean are an object.
Because of that, if effectively two variables can point to the same object, because you assigned a variable to another one, change the value of one, and they'll stop to be linked.

[in the console]

a = "string"
b = a
id( a ) == id( b )
b = "different"
id( a ) == id( b )

 

[EviteGames]

clearly it wasn't the good day for me to answer, sorry.

Don't apologize, you still helped me a ton, even with code quality and such

The feature works perfectly now, so thank you again for all your generous help!