Ren'Py Can't Pickle when saving

[obsessionau]

So I found what seemed to be a fairly robust and slightly overkill event handler system at:

You must be registered to see the links

After spending two days transferring my game into this more powerful event system I have found that the code has a pickling error when you try to save the game.

I downloaded the original code, and sure enough, get far enough into the game and it does it in that too.

I believe it has something to do with a single call to a lambda function but I am not sure how easy it is to fix and have no clue about lexer parsing?

While running game code:
File "renpy/common/00action_file.rpy", line 372, in __call__
renpy.save(fn, extra_info=save_name)
PicklingError: Can't pickle <function <lambda> at 0x06C83030>: it's not found as store.<lambda>


The developer has notes in his game that suggests it could be much improved but has since abandoned it.

Hope a more experienced coder is interested enough to have a look?

 

[anne O'nymous]

I believe it has something to do with a single call to a lambda function but I am not sure how easy it is to fix and have no clue about lexer parsing?

Pickle can not proceed code, and lambda are code.
Either you refactor your code to not use lambda, or you refactor it for them to be transparent during the save process. For the later you can, by example, store the lambda into an object, and use the

You must be registered to see the links

and

You must be registered to see the links

Python meta methods to save them as string, and revert them into lambda during the load process.
Something like this :

init python:
    class lambdaStore( renpy.python.RevertableObject ):

        def __init__( self ):
            # Save proof copy of the lambda.
            self.asString = renpy.python.RevertableDict( {} )

        # Add a lambda to the store.
        def __setattr__(self, aName, aValue):
            # Special case for the internal attribute
            if aName == "asString": return super( lambdaStore, self ).__setattr__( aName, aValue )

            # The name of the lambda MUST be a string.
            if not isinstance( aName, basestring ): return
            # The lambda MUST be passed as string.
            if not isinstance( aValue, basestring ): return
            # The name can NOT be duplicated.
            if aName in self.asString: return

            # Store the lambda as string for save purpose.
            self.asString[aName] = aValue

            # Add the lambda as attribute of the store.
            return super( lambdaStore, self ).__setattr__( aName, eval( aValue ) )

        # Pickle ask for the attributes to save and their values.
        def __getstate__( self ):
            # Return just the lambda as string.
            return self.asString

        # Pickle give you the attributes that were saved.
        def __setstate__( self, state ):
            # Store the lambda as string.
            self.__dict__["asString"] = state

            # Then for each one add them as attribute of the store.
            for n, v in self.asString.items():
                setattr( self, n, v )

# Create the store
default myLambdaStore = lambdaStore()

label start:

    # Add a lambda named "aLambda" that have "x+1" as code
    $ myLambdaStore.aLambda = "lambda x: x+1"

    # Execute a lambda.
    $ a = myLambdaStore.aLambda( 1 )
    "[a]"

You can also assign one of the stored lambda to a variable, by using $ b = myLambdaStore.aLambda, but the said variable must be the attribute of a none savable object, else you'll fallback to the problem you actually have.
The best way to use this lambdaStore in you code is probably to use the name of the lambda in your own part. Something like :

    class Whatever():
        def whatever( self, lbd ):
             self.whatever = getattr( myLambdaStore, lbd)( 42 )

[...]
define whatever = Whatever()
[...]
label whatever:
    $ whatever.whatever( "lambdaName )

While running game code:
File "renpy/common/00action_file.rpy", line 372, in __call__
renpy.save(fn, extra_info=save_name)
PicklingError: Can't pickle <function <lambda> at 0x06C83030>: it's not found as store.<lambda>

This don't help, it just say where the error happen (in the code that save the game), not where the error is.
Add this into your code, using legacy Pickle instead of CPickle will give you a better view of the problem :

init 100:
    $ config.use_cpickle = False

 

[obsessionau]

Thanks,

There is only one line in 01event_handler that mentions "lambda".

python early:
    import re
    EVENT_TAG_PREFIX = "event"
    # Lexer code to handle the various 'event' tags
    # it would be nicer to do these with renpy.screenlang.FunctionStatementParser like other things do
    # Until I find a better way...
    # label_regex = re.compile( r'^\s+(label)[\t ]+([a-z_0-9]+)', re.I )

    file_line_label_map = {}
    def get_label_for_node(filename, linenumber):
        """ Return label name relevant for the filename and linenumber
        """
        if not len(file_line_label_map):
            poss_labels = [ k for k in renpy.get_all_labels() if k[0] != '_' ]
            for k in poss_labels:
                fn = renpy.game.script.namemap[k].filename
                ln = renpy.game.script.namemap[k].linenumber
                if not fn in file_line_label_map:
                    file_line_label_map[ fn ] = []
                file_line_label_map[ fn ].append( ( ln, k ) )
            for fn in file_line_label_map:
                # sort reverse
                file_line_label_map[ fn ].sort( key=lambda r:-r[0] )
        if filename in file_line_label_map:
            for lineno, label in file_line_label_map[ filename ]:
                if lineno < linenumber:
                    return label
        return False

The error message provided was with the legacy Pickle with CPickle I get

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

 

[obsessionau]

The code looks like it does need a fair bit of work as I have en-counted a couple of other problems (ie starting a new game I need to reset things but I have to go through it all and work out what).

I might try and create my own modified version of DSE so that it gives me a list of choices for that period instead of popping up a day planner and introduce myself to LEXER evaluation a little bit more gently.

 

[anne O'nymous]

There is only one line in 01event_handler that mentions "lambda".

Yeah, I saw it when looking at the original code, but it can't be the cause of the error :

            for fn in file_line_label_map:
                # sort reverse
                file_line_label_map[ fn ].sort( key=lambda r:-r[0] )

The lambda is only used internally to sort the list. Once the for loop is finish, the lambda is totally forgotten. So, it can't be the cause of the error.

The error message provided was with the legacy Pickle with CPickle I get

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Hm... __builtin__ refer to the legacy code of Python, which is really strange.

[...] introduce myself to LEXER evaluation a little bit more gently.

There's not much games who use

You must be registered to see the links

; at least "the good way", some try to mimic the way Ren'py create its own statements, which is risky. But it's not something really complicated, and if you need a less complex example, my dynamic strings snippet use one.

This said, using creator defined statements isn't a mandatory thing, even for what you want. It's more a way to simplify your works than anything else. By example, with them you can have something like :

label whatever:
    myEvent eventID:
        condition  myVariable >= 12
        action      jump
        target      someLabel

and without them it would be something like that :

label whatever:
    $ eventList["eventID"] = { condition: "myVariable >= 12", action: "jump", target: "someLabel" }

The creator defined statements is just a transparent way to assign the value to the right variables, and at the right place. It generally lead to a more natural and intuitive writing, and can also perform some validation of the values (verify that the label "someLabel" exist by example), but mostly everybody achieve to write working code without using them.

It don't mean that they are to ignore, when you know how to use them it can really ease your works from update to update, but not using them isn't an obstacle either.