Is there a way to see and possibly edit the Persistent file?

[Imnus]

Is there a way to see and possibly edit the Persistent file?

While testing and modding some games, the persistent file get constantly corrupted and I want to check it to see what's happening exactly but I cannot see how to check and even less edit this file.

 

[79flavors]

I'm not sure it's everything you are looking for, but anne O'nymous 's Extended Variable Viewer shows program controlled persistent variables.

https://f95zone.to/threads/extended...hrough-authors-game-authors-and-modders.5676/

You can, of course, change the values of persistent variables from the developer console.
Just press {SHIFT+o} and then enter commands like persistent.myvariable = True.
(The console is available when config.developer and config.console are set to True... usually by running the project via the RenPy launcher).

 

[anne O'nymous]

I'm not sure it's everything you are looking for, but anne O'nymous 's Extended Variable Viewer shows program controlled persistent variables.

Not sure it will really help since it limits to the game values, hiding those coming from Ren'py's core. What, for the persistent file, imply that 99% of the content is hidden.

While testing and modding some games, the persistent file get constantly corrupted [...]

What do you mean by "corrupted", and how you know it happen ?

I mean, I mod for Ren'py since four years, and played/tested probably near to 1.000 Ren'py games, and never encountered a case where this file was corrupted. I know that in my early Ren'py modding years, I happened to corrupt the save files, and more than once, trying this or that, but never ever the "persistent" one was impacted. As I implied above, it mostly serve to handle Ren'py's internals, and they are relatively tied and secured.

So, knowing what you're effectively looking for would help to answer you accurately.

 

[Imnus]

What do you mean by "corrupted", and how you know it happen ?

I mean, I mod for Ren'py since four years, and played/tested probably near to 1.000 Ren'py games, and never encountered a case where this file was corrupted. I know that in my early Ren'py modding years, I happened to corrupt the save files, and more than once, trying this or that, but never ever the "persistent" one was impacted. As I implied above, it mostly serve to handle Ren'py's internals, and they are relatively tied and secured.

So, knowing what you're effectively looking for would help to answer you accurately.

Because first the game force quits and then when I try to start it again, it starts loading but then quits again, and when I delete the persistent file it gets fixed.

 

[rayminator]

Because first the game force quits and then when I try to start it again, it starts loading but then quits again, and when I delete the persistent file it gets fixed.

that doesn't really help it's quitting for a reason tells us what game it is and post the log/traceback/error txt file

 

[79flavors]

[...] since [the extended variable viewer] limits to the game values, hiding those coming from Ren'py's core. What, for the persistent file, imply that 99% of the content is hidden.

Feature request... toggle for "Show internal RenPy Persistent Variables".

Because first the game force quits and then when I try to start it again, it starts loading but then quits again, and when I delete the persistent file it gets fixed.

As rayminator says, I think some examples might help here.
Is it just one game? Based on your initial "While testing and modding some games", that sounds like it's happening fairly often to multiple games.

Have you ever published any of your mods? i.e. Is there somewhere we could download one of them? I can see messages here on F95 where you have discussed other peoples mods for Big Brother: Another Story, Long Short Story and Milfy City - but nothing concerning your own mods. I only ask, because perhaps it's something your mod(s) is doing that causes the game to initially "force quit". If we could see a recent mod that has caused you problems, perhaps we can see something that's causing it.

Have you ever had RenPy games "force quit" on you without you trying to mod or unpack them? Again, I only ask because it's possible that it isn't RenPy or your mod(s) and is your computer setup itself. If unmodded RenPy games crash for you but not the rest of the community - that points to something unique to you. If only modded games crash, then it possible something you are doing is the cause.

The need to delete the persistent file to get the game running again makes me think that the initial crash of the game is the real source of your issues. Perhaps it's something as simple as getting a disk write error while trying to write the save/persistent data.

One practical thing you could check is whether both copies of the persistent file are been kept in line.
Save and persistent files are stored in two places on your computer.
Firstly the game folder's own save folder. Perhaps D:\Games\{game-ID}\game\saves\
... and a common RenPy folder. Likely C:\Users\{username}\AppData\Roaming\RenPy\{game-ID}\.

The persistent file in both these folders should be the same date/time and be the same size.

 

[anne O'nymous]

Because first the game force quits and then when I try to start it again, it starts loading but then quits again, and when I delete the persistent file it gets fixed.

Same than the others, it's too few information. And since you talk about trying to mod, I would add to their queries: When is the game force quitting the first time, what is the last modification you did before that, and in what file.

There's (exceptional) times when Ren'py crash silently, but for this to happen the problem need to appear before Ren'py install its own error handler ; therefore the error have to happen early when Ren'py is still trying to load its core.
It doesn't necessarily mean that it come from the core itself, but it really limits the number of case where it can happen.

Feature request... toggle for "Show internal RenPy Persistent Variables".

Sorry, you'll have to wait for the two others feature request: Make it Python 3 ready and make it works with the last found bug.

The need to delete the persistent file to get the game running again makes me think that the initial crash of the game is the real source of your issues. Perhaps it's something as simple as getting a disk write error while trying to write the save/persistent data.

Personally I see it more as a compatibility issue.
Except at my early Ren'py modding age, when I did really dirty (even in my book) things, I never had to delete the persistent file to solve an issue. Perhaps would it had solved the issue, but changing the libraries and core to another version of Ren'py always solved the issue for me.
But well, he is at his early Ren'py modding age, so, everything is possible.

 

[Imnus]

that doesn't really help it's quitting for a reason tells us what game it is and post the log/traceback/error txt file

Because first the game force quits and then when I try to start it again, it starts loading but then quits again, and when I delete the persistent file it gets fixed.

It has happened with several games, but it has happened a lot with Big Brother: Another Story, which is why ended making this thread.

This may be the traceback of the last time it failed:

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "game/tl/english/dialogues/breakfast.enhanced.rpy", line 16, in script
    old "Ну да, на словах все такие храбрые. А на деле..."
  File "game/tl/english/dialogues/breakfast.enhanced.rpy", line 16, in script
    old "Ну да, на словах все такие храбрые. А на деле..."
Exception: A translation for "Ну да, на словах все такие храбрые. А на деле..." already exists at game/tl/english/dialogues/breakfast.enhanced.rpy:5.

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "renpy/bootstrap.py", line 326, in bootstrap
    renpy.main.main()
  File "renpy/main.py", line 515, in main
    renpy.game.context().run(node)
  File "game/tl/english/dialogues/breakfast.enhanced.rpy", line 16, in script
    old "Ну да, на словах все такие храбрые. А на деле..."
  File "game/tl/english/dialogues/breakfast.enhanced.rpy", line 16, in script
    old "Ну да, на словах все такие храбрые. А на деле..."
  File "renpy/ast.py", line 2470, in execute
    renpy.translation.add_string_translation(self.language, self.old, self.new, newloc)
  File "renpy/translation/__init__.py", line 453, in add_string_translation
    stl.add(old, new, newloc)
  File "renpy/translation/__init__.py", line 394, in add
    quote_unicode(old), fn, line))
Exception: A translation for "Ну да, на словах все такие храбрые. А на деле..." already exists at game/tl/english/dialogues/breakfast.enhanced.rpy:5.

Windows-10-10.0.19041
Ren'Py 7.4.4.1439
Большой брат: Другая история 0.06.5.09
Wed Jun 23 02:11:36 2021

As rayminator says, I think some examples might help here.
Is it just one game? Based on your initial "While testing and modding some games", that sounds like it's happening fairly often to multiple games.

Have you ever published any of your mods? i.e. Is there somewhere we could download one of them? I can see messages here on F95 where you have discussed other peoples mods for Big Brother: Another Story, Long Short Story and Milfy City - but nothing concerning your own mods. I only ask, because perhaps it's something your mod(s) is doing that causes the game to initially "force quit". If we could see a recent mod that has caused you problems, perhaps we can see something that's causing it.

Have you ever had RenPy games "force quit" on you without you trying to mod or unpack them? Again, I only ask because it's possible that it isn't RenPy or your mod(s) and is your computer setup itself. If unmodded RenPy games crash for you but not the rest of the community - that points to something unique to you. If only modded games crash, then it possible something you are doing is the cause.

The need to delete the persistent file to get the game running again makes me think that the initial crash of the game is the real source of your issues. Perhaps it's something as simple as getting a disk write error while trying to write the save/persistent data.

One practical thing you could check is whether both copies of the persistent file are been kept in line.
Save and persistent files are stored in two places on your computer.
Firstly the game folder's own save folder. Perhaps D:\Games\{game-ID}\game\saves\
... and a common RenPy folder. Likely C:\Users\{username}\AppData\Roaming\RenPy\{game-ID}\.

The persistent file in both these folders should be the same date/time and be the same size.

Yes it has happened with several games but only often with Big Brother: Another Story.

I don't actually make mods, except for once I updated the Love Season - OscarSix's Mod because he wasn't available.

I only mod a few things that annoy me in some games, but since I don't have the time or the passion to keep with it, I never post them. But here it's the Big Brother - Another Story v0.06.5.09 Extra Mod in case you wanna check.

The errors happen when I'm modding, usually there is some mistake that causes the game to crash and ends up corrupting the persistent file, I would like to see exactly what's happening with it.

It never has happened with unmodded games or with proper mods, only when modding, but I unpack all games and enable the Console and Developer Menu.

Yes, it always the same persistent file in both folders, I always check that to see if one of them is not broken. I use UltraCompare binary comparison to check.

 

[rayminator]

the traceback is telling why you getting that error

Exception: A translation for "Ну да, на словах все такие храбрые. А на деле..." already exists at game/tl/english/dialogues/breakfast.enhanced.rpy:5.

are you trying to make a translation for the game?

 

[anne O'nymous]

This may be the traceback of the last time it failed:

This traceback do not correspond to the mod you joined, and also it explicitly tell that the problem do not come from the persistent file.

The errors happen when I'm modding,

By doing what ? Assuming that it's effectively a corrupted persistent file, the corruption didn't happened by itself.

usually there is some mistake that causes the game to crash and ends up corrupting the persistent file,

How do you know that it's a corruption of the persistent file ? And, no, "deleting the persistent file solve the problem" is not a proof that the file is corrupted.
The SDK offer the possibility to delete it for any project for a good reason. It reset the global context of the game, what can solve a lot of external errors and potential conflict ; especially when you make too radical change. But this absolutely don't mean that the persistent file is corrupted.

It never has happened with unmodded games or with proper mods, only when modding, but I unpack all games and enable the Console and Developer Menu.

What make my question even more important: What are you doing, and where...

Your change are the reason of those crash, it's there that you should look, and not in a file that you would probably don't understand even if you add access to its content in a human readable way.

 

[Imnus]

the traceback is telling why you getting that error

Exception: A translation for "Ну да, на словах все такие храбрые. А на деле..." already exists at game/tl/english/dialogues/breakfast.enhanced.rpy:5.

are you trying to make a translation for the game?

Yes, I know why I got the error in the first place, what I don't know is why the persistent file got corrupted.

I was adding some translations that were not in the game, editing some that were, etc.

In this case it was some duplicate lines in the 'breakfast.enhanced.rpy file:

    old "Ну да, на словах все такие храбрые. А на деле..."
    new "Yeah, like you'd have the guts..."

This traceback do not correspond to the mod you joined, and also it explicitly tell that the problem do not come from the persistent file.

'breakfast.enhanced.rpy' is one of the files I added, I never say the problem was caused by the persistent file, I say after the game crashed because I was messing with it, the only way it would start without force quitting was if I deleted the persistent file, hence I assumed the persistent file was corrupted.

By doing what ? Assuming that it's effectively a corrupted persistent file, the corruption didn't happened by itself.

I know the corruption didn't happen by itself, in this case it was some duplicate lines in the 'breakfast.enhanced.rpy file:

    old "Ну да, на словах все такие храбрые. А на деле..."
    new "Yeah, like you'd have the guts..."

Those lines were duplicated and caused the game to crash, but even after correcting the problem the game wouldn't start properly unless I deleted the persistent file, so I assumed it was corrupted. But I don't have a way to know why, which is why I wanted to look into the file to see if I could discover what was happening.

How do you know that it's a corruption of the persistent file ? And, no, "deleting the persistent file solve the problem" is not a proof that the file is corrupted.
The SDK offer the possibility to delete it for any project for a good reason. It reset the global context of the game, what can solve a lot of external errors and potential conflict ; especially when you make too radical change. But this absolutely don't mean that the persistent file is corrupted.

I don't know if it was corrupted, since the beginning I explained that only deleting the persistent file would let me start the game, which is why I assumed it was corrupted.

I don't know what would constitute a too radical change, in this case it was just adding some missing translations, editing some, etc. like I just said.

What make my question even more important: What are you doing, and where...

Your change are the reason of those crash, it's there that you should look, and not in a file that you would probably don't understand even if you add access to its content in a human readable way.

I already posted all the changes I did: Big Brother - Another Story v0.06.5.09 Extra Mod.

I know the changes are the reason for the crash, what I don't know it's why after correcting the problem that caused the crash, the game won't start unless I delete the persistent file. I mean the cause of the crash is no longer in the files I uploaded (it was fixed), but those were the files I was messing with.

That's why I want to access the persistent file, to see what's in it that won't let the game start. Again, solving what caused the crash doesn't help with the fact that I still have to delete the persistent file for the game to start.

Of course I started just backing up the persistent file before messing with the game, but I still would like to access it to be able to fix it, because I know eventually I will forget to back it up in the future before messing around. And I just want to know what's happening with it.

 

[rayminator]

there are 2 locations for the persistent files

C:\Users\Your_name\AppData\Roaming\RenPy\Game-XXXX
D:\DOWNLOAD\Bright_Past-0.84.0-pc\Bright_Past-0.84.0-pc\game\saves

delete both of them

just a question

have you tried using a old save for any renpy games that was updated?
because that can cause problems.

 

[Imnus]

there are 2 locations for the persistent files

C:\Users\Your_name\AppData\Roaming\RenPy\Game-XXXX
D:\DOWNLOAD\Bright_Past-0.84.0-pc\Bright_Past-0.84.0-pc\game\saves

delete both of them

Yes, that's what I've been doing, it's the only way for the game to start.

just a question

have you tried using a old save for any renpy games that was updated?
because that can cause problems.

Nope, just new saves, but since I'm messing around and modding the game, it sometimes crashes and of those crashes sometimes the persistent file gets corrupted and I have to delete it.

I would like to know what's happening with it, what exactly corrupts it but it seems there's no way to do it.

 

[darkhound1]

Yes, that's what I've been doing, it's the only way for the game to start.






Nope, just new saves, but since I'm messing around and modding the game, it sometimes crashes and of those crashes sometimes the persistent file gets corrupted and I have to delete it.

I would like to know what's happening with it, what exactly corrupts it but it seems there's no way to do it.

Pretty sure by now that it's a bug in
Ren'Py 7.4.4.1439

Since I've switched to this version, I'm getting complaints from user that their game won't start any more.
Once they delete the persistent file, it works again. There is nothing in the log that helps.

 

[anne O'nymous]

Pretty sure by now that it's a bug in
Ren'Py 7.4.4.1439

Since I've switched to this version, I'm getting complaints from user that their game won't start any more.
Once they delete the persistent file, it works again. There is nothing in the log that helps.

What don't prove that there's a bug in the version 7.4.4.

As said since more than one year now, the versions 7.4.x mark the start of the effective port to Python 3.x. While the possibility to effectively write your own code in a Python 3.x compatible way appeared with the version 7.4.5, for the core itself this transition started with the version 7.4.0.
And obviously, it's the responsibility of each dev to ensure that his code is (and will be) compatible, each time he switch to a 7.4.x, whatever version he previously used. This even in a forward way, since the save files, and the persistent file, store data that can lead to some incompatibility and make the process fail.

Anyway, the whole 7.4.x branch shouldn't be used for production.
While the releases are mostly stable (minus few regressions time to time), there's no effective compatibility guaranties. Everyone should assume that the famous backward compatibility of Ren'py have stopped with the version 7.3.5 ; and will comeback only when the port will be finished (with the 8.0.x ?).
PyTom, and those who now help him on the project, do their best, while refactoring the code. But Ren'py being sometimes really lax, there's no guaranty that the way you used a statement until the 7.3.5 will continue to works with a 7.4.x version ; at least unless you strictly follow the usage described in the documentation, what is rarely the case, even for experienced coders.

 

[Rich]

Anyway, the whole 7.4.x branch shouldn't be used for production.

Without disagreeing with your general statement, there is one big problem. Unfortunately, without using the 7.4.x, you lose the ability to produce Android builds. It was an unfortunately choice by PyTom-and-company "back when" to rely on a third-party server to provide required content for that. (That's not being accusatory - as developers, we've all made bad choices now and again. I'm sure PyTom, in retrospect, would agree.)

Obviously, one of the things you can do to maximize the chance of surviving version updates is to be very careful what you store, both in your saves and also in the persistent data. For example, I completely avoid saving classes as part of saves (for a lot of reasons). Everything in my saves or persistent data is a Python built-in. (Primitive, list, dict, etc.)

Pretty sure by now that it's a bug in
Ren'Py 7.4.4.1439

Since I've switched to this version, I'm getting complaints from user that their game won't start any more.
Once they delete the persistent file, it works again. There is nothing in the log that helps.

One of the best things you could do would be to capture a copy of a persistent data file from someone having the problem (before they delete it), prove to yourself that it crashes on your own machine and then open an issue at

You must be registered to see the links

. If there is a bug lurking, this is the best way to get it squashed.

 

[anne O'nymous]

Unfortunately, without using the 7.4.x, you lose the ability to produce Android builds.

Obviously, one of the things you can do to maximize the chance of surviving version updates is to be very careful what you store, both in your saves and also in the persistent data. For example, I completely avoid saving classes as part of saves (for a lot of reasons). Everything in my saves or persistent data is a Python built-in. (Primitive, list, dict, etc.)

For a good jump to a 7.4.x, I advice having a last update with a none 7.4.x, in which you clean up your variables.
This way, the save files should be clean even the part that is still present only in the rollback, while you increase the chance for the persistent data to also be clean.

But well, obviously this don't solve the "sorry, no Android port for this version" issue.

 

[darkhound1]

You're missing the point. It's not a one time thing during the version switch loading an old save. It just happens randomly even when all saves including the persistent file have been deleted.
My game code doesn't use the persistent file except to store a value from the preferences screen.
So it's a fair assumption that it's a general renpy issue of that specific version.

It's not a code compatibility problem. It took a lot of time to fix these, but has been done.
You can play the game for hours without any problem. You close it down and try to start it again some time later and it won't start until you delete the persistent file.

 

[Rich]

You can play the game for hours without any problem. You close it down and try to start it again some time later and it won't start until you delete the persistent file.

Ah - OK. I misunderstood. I thought you'd updated from an earlier version to 7.4.x, and the upgraded game was having issues reading persistent data from a previous version. Clearly that's not what's happened.

I'm sure you know that the persistent data files stores quite a bit of stuff other than your explicit persistent data - things like "I've seen this image" and "I've played past this label." So it does update a lot as the game is played.

I would repeat my suggestion - if you can get to the point where a particular persistent data file won't load, file a bug report. You might not be the only developer with the issue, but PyTom doesn't hear about it unless there's an issue created, and having a file that's broken really helps them debug.

 

[anne O'nymous]

Ok, so right now I hate Ren'Py.

There's tools to see the content of a pickled file, but the persistent file is not pure pickle. It's "zlib" encoded before being saved. This mean that you can't open it as an archive, because it's not an archive, and you also can't open it with one of the said tools, because it's not a pickle file...
And obviously, once I achieved to generate an effective pure pickle file, I have the (totally unsurprising) joy to discover that all those tools are designed to works with pure Python pickle files, and fail to load the "persistent" file, because they can't find the "renpy" module.

And now I struggle because even when doing this from a Ren'py script, it fail to load a module, this time the "persistent" one :/ But the module exist, and even if I ensure that the path to Ren'py core files is in the modules path of Python, it don't works

Personally I quit. It's near midnight here, I had a fucking hard day and lost enough time with this, especially since I don't need it.

If someone want to give it a try, here's the code:

# Ensure that it works with Python 2.x
#rpy python 3

init python:

    import pickle

    # Ensure that the path to Ren'py's core is known by Python.
    PATH_TO_RENPY_IN_SDK = "[Path to Ren'py's SDK]\renpy"
    if not PATH_TO_RENPY_IN_SDK in renpy.sys.path: renpy.sys.path.append( PATH_TO_RENPY_IN_SDK )

    def unpickle( path ):
        # Decode the 'persistent' file
        FH = open( renpy.os.path.join( path, "persistent." ), "rb")
        s = FH.read().decode("zlib")
        FH.close()
        # Save as pure pickle file
        FH = open( renpy.os.path.join( path, "persistent.RAW" ), "a" )
        FH.write( s.encode( encoding='UTF-8' ) )
        FH.close()
        # Load the pickled file
        FH = open( renpy.os.path.join( path, "persistent.RAW" ), "rb" )
        # Here you should have a "print pickle.....", but it would log all this on
        # the console, what risk to be too many information. Therefore I 
        # decided to load it on a variable, and from there to try to proceed
        # whatever will be loaded.
        raw = pickle.load( FH )
        FH.close()
        # Here come the processing of /raw/ content... if it load

label start:
    # /!\ WARNING - You can NOT open the persistent file of the current project /!\
    $ unpickle( "PATH TO THE persistent file]" )
    "END"

This being said, picklingtools have a

You must be registered to see the links

. Technically it should works independently to the module (since it should save the name and value, not create the variable), so perhaps that it can be the solution. At least once you've generated the pure pickle file.