Ren'Py RenPy Autosave Issue - Creates Empty Slots on choice / Autosaves on Quit work

[BloodyMares]

I hope someone here would be able to find the answer to my problem. The autosave feature in the game that our team is working on doesn't save properly, specifically on my end. I can't replicate this issue playing other games, so my suspicion falls on my Renpy build.

Problem:
- Autosave on choices instead of saving creates Empty Slots. The existing autosaves are renamed but no save files are created.
- Autosave upon quitting the game or qutting to Main Menu saves the game properly, with a screenshot and all the data, etc, the autosaves are physically present in the save folder.
- This happens on my end only, other teammates can't replicate this issue.
- This issue is exclusive to the game /Renpy version, other games can autosave on choice with no problem.
- Tried deleting the persistent file, as well as updating the Renpy version. Same result.

IIRC, normally autosaves can only disappear if you hit the same choice but on a different branch, the newer save deletes the outdated / inaccessible one. I suspect something happens now in the autosave script that it triggers the command to create the save but an issue prevents the save from being written or it gets instantly deleted on the final step.

I'm not really an experienced coder, I mainly work on the story scripts, without touching the functionality of the game so I would really appreciate it anyone here is either familiar with this issue or knows how to fix it. I'll stress again, since the autosave works fine for my teammates, I doubt the issue has anything to do with the project, but the renpy itself or its work on my end. What renpy scripts do I need to look into?

 

[anne O'nymous]

- Autosave on choices instead of saving creates Empty Slots. The existing autosaves are renamed but no save files are created.
- Autosave upon quitting the game or qutting to Main Menu saves the game properly, with a screenshot and all the data, etc, the autosaves are physically present in the save folder.

Hmm, autosave and save on quit use the exact same function, called with the exact same arguments (none at all). Therefore, there's absolutely no reason for them to act differently.
Of course, they are called from different parts of the core, but cycling the file names is one of the last steps before the save is effectively done. The only thing in between is the screenshot, but it's skipped due to the arguments past to the function.
Therefore, if the file names are cycled, and you said they are, the save will starts immediately. And there's no way to exist this part of the code without throwing an exception, therefore you would have an error screen. I assume that you would have told it if it was the case. What mean that the save file are created, and created normally.

This being said, what about the save files once rotated ? Have you looked if they are still empty or filed ?
Since, as implied above, the autosave rely on a thread, you can have looked at them before Ren'Py finished to proceed them. Or the system can have some slowdown.
And do this concern the two files ? The one in the save directory and the copy on the system directory ?

[...] I doubt the issue has anything to do with the project, but the renpy itself or its work on my end.

Once the file names are cycled, either you get an error screen, or the file save is created and will contain a screenshot, a file named "extra_info" (potentially empty), a file named "json" (that can't be empty), a file named "renpy_version" (that can't be empty) and a file named "log" (potentially empty if Ren'Py silently failed to get the game state).
Unless I missed something, but I really doubt, there's no other possibilities. Therefore, it can't be more than a guess, but it really looks like an issue on your end, probably from the OS.

 

[BloodyMares]

Okay, so here's the screenshot of how it looks in the game. The saves that have the name and screenshots were all created when I quit to the main menu. The empty slots were all made when I encountered the choices naturally. No error screens or anything like that.

Once the file names are cycled, either you get an error screen, or the file save is created and will contain a screenshot, a file named "extra_info" (potentially empty), a file named "json" (that can't be empty), a file named "renpy_version" (that can't be empty) and a file named "log" (potentially empty if Ren'Py silently failed to get the game state).

Where can I find all these files? I only have the log.txt in the main renpy folder as well as the project folder.

 

[anne O'nymous]

The empty slots were all made when I encountered the choices naturally.

Ok, I did some tests, and what you get is not possible. I believe you, it's what you get, but you really shouldn't have this.

1) Ren'Py will automatically name the autosave file accordingly to the last name. Therefore the only way to not have an "auto-2" file is if the said file is deleted after the creation of "auto-3".
Side note: If the file is missing in one directory and not the other, Ren'Py will end out of sync, because it rename once by directory, not knowing the existing names in the other directory.

2) Ren'Py synchronize the content of the system directory and the one of the save directory. Therefore, if one file is missing in one, but exist in the other, it will be shown on the save page.

3) If one of the two copy of the save file is corrupted, then Ren'Py will leave a hole at its place. This whatever if the corrupted file is on the system directory, or on the save directory.
Side note: Corrupted mean that the file is not a valid zip file. Else, even if the archive is totally empty, Ren'Py will show the slot as being used.

Therefore, yes, what happen to you shouldn't happen and it's near to impossible that it happen.

It need that either the autosave file is deleted after the creation of another autosave file, and this in both directory. Or it need that one of the copy for this autosave file is an invalid zip file, while they are created at the same time, one being the copy of the other.
If both have a reasonable probability to happen, this probability drop really near to 0 when you add "this happen for all autosave files, but never happen for the save on quit files, while they all are exactly the same thing and created in the exact same way".

Where can I find all these files?

The system directory is "C:\Users\Techurious\AppData\Roaming\RenPy". Here you'll find a directory by game, search the name of your project, and you'll find the save files inside. As for the "save" directory it's "[path to the project]\game\saves".
All ".save" files are in fact zip archive. So you can open them with 7z, WinZip, WinRar, or whatever software you use to manage archives. And, as I implied previously, the autosave files have named that starts with "auto-", followed by the slot number.

The question is, are the files really missing, and if not, is there invalid zip archives among them.

 

[BloodyMares]

The question is, are the files really missing, and if not, is there invalid zip archives among them.

Yes, the files are really missing, in both directories, and there's no trace of any corrupted archive, etc. And even if they were hidden, I'd still be able to see them but they're not there. There are only auto-1, 3, 5, 7, 10, and 12. All created upon quitting to main menu, all of them working without issues. No other files named "auto" in either of the save locations and no other suspicious files. Only persistent, normal saves, and quick saves as well as navigation.json that we use for a custom dev tool.

 

[BloodyMares]

Okay, update. I downloaded a fresh sdk of Renpy 7.5.1 (my teammates are using that version), ran the project from there and the issue was completely fixed without me having to do anything else. I'm glad it's gone but the annoying thing is that I won't know what was wrong with my previous Renpy build in the first place, that it caused such a weird and as you say impossible bug.

 

[gojira667]

Okay, update. I downloaded a fresh sdk of Renpy 7.5.1...

I have what I'll call a similar issue. What engine version did you have the problem in?

In my case I'm seeing it in renpy-7.5.3. Though it's different as it's triggering a non-blocking autosave at the start of a turn via:

$ renpy.force_autosave(take_screenshot=True)

It works great as long as their is enough time for it to complete the save. If the player hits end turn right away I see the same behavior you were having.

Trying to read

You must be registered to see the links

both autosave() and force_autosave() return early if the game is in the middle of an autosave. Testing for if not renpy.loadsave.autosave_not_running.is_set(): unsurprising doesn't help, but does show it isn't trying a second autosave.

Which suggests to me that it's retrying the initial (force_)autosave in autosave_thread_function() if it was interrupted/incomplete. One of the first things it does is cycle_saves() which renames all autosaves by incrementing them up 1 slot. This will create the empty slots when done repeatedly as the saves start to fall off the end.

 

[anne O'nymous]

Trying to read

You must be registered to see the links

both autosave() and force_autosave() return early if the game is in the middle of an autosave.

What is the expected behavior. It obviously can lead to a save not being performed (but without impact on the files name), but it prevent the game to saturate the computer with threads stuck in an endless loop or really long process.

Which suggests to me that it's retrying the initial (force_)autosave in autosave_thread_function() if it was interrupted/incomplete.

Since it's a thread, it can't be interrupted.
Not completed, it's something else. From the game itself (because the console is seen as a main menu and will prevent the autosave to works), try $ force_autosave(block=True). It will force the autosave to be performed from the main thread, therefore it guaranty that, whatever happen, if the save process is interrupted something will happen. It can be Ren'Py showing the error code due to an exception, or the game silently crashing because Python itself had an issue.

One of the first things it does is cycle_saves() which renames all autosaves by incrementing them up 1 slot. This will create the empty slots when done repeatedly as the saves start to fall off the end.

As I said above, no it will not. Ren'Py do not internally keep track of the (supposedly) used slot. Therefore if the list of autosave file end at "auto-2", the next file will be named "auto-3". This even if there were 10 failed autosave attempt in between.



As for your issue:

Though it's different as it's triggering a non-blocking autosave at the start of a turn via: [...]
It works great as long as their is enough time for it to complete the save.

The creation of a save file is really fast. I can't find back the thread where I had to do the tests, but if my memory don't betray me, globally it's below two seconds for most of the games.
Plus, it's in a thread, therefore it will always have all the time it need to complete the save. Perhaps that a running autosave will prevent the next one to happen, but that's all.

Testing for if not renpy.loadsave.autosave_not_running.is_set(): unsurprising doesn't help, but does show it isn't trying a second autosave.

Then Ren'Py should proceed the autosave, unless it believe that you are in the main menu, are in a rollback, or are replaying a scene.
Side note:
By "not trying a second autosave", you mean that you get True from renpy.loadsave.autosave_not_running.is_set(), right ?
I ask because this flag is a pure none sense. As its name say, it works at the opposite of what is expected in such case, being raised 99% of the time, and lowered only when you should be cautious.

 

[gojira667]

Since it's a thread, it can't be interrupted.
Not completed, it's something else. From the game itself (because the console is seen as a main menu and will prevent the autosave to works), try $ force_autosave(block=True). It will force the autosave to be performed from the main thread, therefore it guaranty that, whatever happen, if the save process is interrupted something will happen.

Blocking force_autosave(), $ renpy.force_autosave(take_screenshot=True, block=True), works every time. With its corresponding delay as it saves.

As I said above, no it will not. Ren'Py do not internally keep track of the (supposedly) used slot. Therefore if the list of autosave file end at "auto-2", the next file will be named "auto-3". This even if there were 10 failed autosave attempt in between.
...
The creation of a save file is really fast. I can't find back the thread where I had to do the tests, but if my memory don't betray me, globally it's below two seconds for most of the games.
Plus, it's in a thread, therefore it will always have all the time it need to complete the save. Perhaps that a running autosave will prevent the next one to happen, but that's all.

Attached is a reproduction case to try if you like. Well, an attempted one anyway. It may very well be somewhat PC dependent and mine isn't high-end. I'm assuming it's related to how quickly it can execute the autosave.

The start is a bit slow as I'm, possibly in-artfully, increasing the save size. Once you are at the screen it will have done the initial autosave. "End Turn" will trigger a new autosave as it comes back around.

Not all the time, yet sometimes, smashing "End Turn" repeatedly will result in a "missed" autosave and an empty slot with no file on disk. Nothing in log.txt, besides the initial things & the debug prints, for the whole session even.

What did seem to reliably trigger the issue for me is to:

1. "End Turn"
2. Immediately click "Screen B"
3. Immediately click "Screen A" to go back

As long as I'm quick I get an empty slot. YMMV and all that.

Download You must be registered to see the links Download You must be registered to see the links

SHA256:
8de715faac7fa229aa61635dbabccabccac999f13c46a9d6e3713838fa3d62eb autosave_mul_ti_pass.zip

 

[anne O'nymous]

Blocking force_autosave(), $ renpy.force_autosave(take_screenshot=True, block=True), works every time. With its corresponding delay as it saves.

Hmm... So it's definitively not something that break the save.
It's also not a question of configuration (Ren'Py not doing an autosave because the previous one happened not long ago) since you're explicitly forcing this save.

What bother me is your "if the player hit end turn immediately". The autosave process starts at parts of the main thread, therefore the thread is already started when the player have the possibility to end the turn. What mean that it's something in the thread, but something that isn't an error.

Hmm, could it be due to the mutate_flag ?
It is raised when Ren'Py starts a new interaction, or when a list, dict, set or object change. Among other things, this offer to Ren'Py the possibility to not proceed to a save, because the state of the game have changed while he was build the content for the save.
Side note: *sigh* It's a no benefit but high risks design. The consequences of a state change for an autosave is negligible, especially in regard of the consequences if the said autosave do not happen.

It seem plausible due to the moment where the autosave fail. When the turn end, a lot of things change in the game context.
Therefore, what you need to do is to prevent the player to end the turn, until Ren'Py past this exit point. And it happen that it's perfectly possible, thanks to the way the save is designed. There's callbacks related to the save,

You must be registered to see the links

. They are called after the mutation validation, therefore they can be diverted from their primary use.

The principle is to prevent the player to end a turn until the autosave is finished, without blocking him more than it's really needed ; therefore without the use of a blocking autosave. And for this you need to have a flag, lowered right before the autosave starts, and raised when the autosave can not anymore prematurely end.
Something like:

# Flag telling you if it's safe, or not, to end a turn.
define canEndTurn = False

init python:
    #  The callback that will change your life.
    def autosaveWatchdog( JSON ):
        #  It's now safe to end a turn, Ren'Py will not stop the autosave
        # process below this point.
        store.canEndTurn = True

# The label where you call /force_autosave/
label whatever:
    #  It's not safe to end a turn
    $ canEndTurn = False
    #  Starts the autosave
    $ renpy.force_autosave()

# The screen that include the "end turn" button.
screen whatever():
    [...]

    #  Show the "end turn" button only when it's safe to end the turn.
    showif canEndTurn:
        textbutton "End Turn":
            [...]

It will slowdown the starts of a turn, but a bit less than a full blocking autosave.
It's alas all that can be done. Well, except a redesign of the autosave for it to not exit when it's not needed, but it's something else.