Tool Others F95Checker [WillyJL]

[WillyJL]

i didnt find this error when i searched... but this popped up this morning:
{
"INDEX_ERROR": "PARSING_FAILED"
}

Yep, I have the same error
View attachment 4434199

Ditto like the rest today. Parsing failed error...

Hello

same here : when doing full refresh :
{
"INDEX_ERROR": "PARSING_FAILED"
}

when manually updating one game no issue

Thank

So the change I made yesterday did fix the logic issue as I expected, now it notices there is an issue, instead of just marking the thread as missing. I will check the logs when I'm in pc in a few hours.

 

[WillyJL]

WillyJL Any chance the server is down?

No. I heard f95zone itself was down for a while, maybe that's what you mean
If you mean the error that the others posted, I will look at it in logs in a few hours

 

[simple_human]

11.0.2 beta 1437

I found a couple of bugs.
When I move a game to a 'new tab' (a new tab is created, and the game is moved there), it becomes impossible to switch between tabs.

I restart the program, and everything works fine. I go into the 'new tab' move the game back, and when I try to close that tab, the program crashes with the this error (the tab does get removed though):

 

[FaceCrap]

As always, I can't fetch all threads always from f95zone.

I understand, but that means the 'Full Refresh' option no longer works like before. Because when it went directly, that would have been picked up.
So you don't store the datetime data for when a thread was last queried?
Otherwise you could compare it to the date value in the json result to see if it's been modified since.
Also, this was an actual update, wouldn't it had to have the thread parsed in that case anyways instead of relying on the expiration cycle?

DDL goes directly to f95zone, it's not cached on my end. If the issue also happened with DDL, this sounds to me like something actually went wrong with that thread on f95zone end itself.

That was just extraneous info. Some mod probably forgot to add the mirrors. It's what lead to noticing the mirror difference

11.0.2 beta 1437
I found a couple of bugs.

confirmed, same here. But the tab does get removed though.

 

[simi00]

When I check one by one, then it works, but error shows up on game ID 74065 ( https://f95zone.to/threads/the-hard-way-v0-38-1-muffin-maker.74065/ ). Other games (checked on 50 games) works. I have around 400 games in list.

Other game with parse error:
https://f95zone.to/threads/the-innocent-mother-ch-2-spies.4211/

 

[WillyJL]

and if the actual underlying issue persists (as we see from people below your post) now I will see the logs for it

So the change I made yesterday did fix the logic issue as I expected, now it notices there is an issue, instead of just marking the thread as missing. I will check the logs when I'm in pc in a few hours.

doctorx6969 simi00 Baalzie triste_lune turns out that this error in particular was due to f95zone database downtime, not to the previous issue with threads missing. it didnt receive valid data from f95zone so of course parsing wasnt successful. it would have attempted again in 6 hours, i fixed it now (5 hours after it happened). if a databse error like this ever happens again, it will be caught as a F95ZONE_UNAVAILABLE error instead and retried after 15 minutes.
i have invalidated cache for all the affected threads (3983 3959 136922 151469 79941 1132 1224 5541 76200 785 2796 6996 1297 135918 42937 47958 10703 4431 126890 20812 210467 202 727 26392 30383 4211 132123 214170 74065), should be fine now

 

[WillyJL]

When I check one by one, then it works, but error shows up on game ID 74065 ( https://f95zone.to/threads/the-hard-way-v0-38-1-muffin-maker.74065/ ). Other games (checked on 50 games) works. I have around 400 games in list.

Other game with parse error:
https://f95zone.to/threads/the-innocent-mother-ch-2-spies.4211/

when you get an actual error reported like this (saying the f95checker cache api returned an error) it means it knows there was an issue and logged it for me to inspect, no need to investigate further, just notify me
it was different last week with the missing threads, those went unnoticed (considered thread as missing, but not an api error), and needed info from users to diagnose

 

[WillyJL]

I understand, but that means the 'Full Refresh' option no longer works like before. Because when it went directly, that would have been picked up.
So you don't store the datetime data for when a thread was last queried?
Otherwise you could compare it to the date value in the json result to see if it's been modified since.

of course it doesnt work like before, basically nothing works like before, it *cannot* work like before.
your f95checker client never talks to f95zone directly when it comes to getting thread info.
as stated in the github readme, now the full refresh means syncing data from the cache api. it doesnt force the cache api to recache the thread. if it did, we would be back at square 1: everyone forcing a full refresh and blasting f95zone till we DDoS it.

i do store the timestamp for last time a game was refreshed, on both sides:
- your client records the last time the thread data changed on the cache api. it works kinda like a "version" of cached data. any time the cache api updates the cached data, it saves the timestamp, then when your client refreshes it checks all the timestamps, if the cache api's cached data timestamp is different than what your client has saved, it syncs the whole thread info from the cache api.
- and on the cache api, i record when each thread was last fetched from f95zone, when the last time any of the contained data actually changed (this is to notify client that they should sync the new data, and avoids syncing if after cache expiration the new data is the exact same), and the time when cache is due to expire (not all threads have same cache expiry, like some errors can have as little as 15 minutes of cache, and non-existent threads are cached for 14 days)

Also, this was an actual update, wouldn't it had to have the thread parsed in that case anyways instead of relying on the expiration cycle?

yes. and it did. the issue is that mods pushed the update, someone refreshed the thread in f95checker, it was cached, and only *after* the downloads were changed. at that point, nothing changed that the cache api could detect.
its always human error. i cannot fix the 0.01%. sorry.

 

[simi00]

doctorx6969 simi00 Baalzie triste_lune turns out that this error in particular was due to f95zone database downtime, not to the previous issue with threads missing. it didnt receive valid data from f95zone so of course parsing wasnt successful. it would have attempted again in 6 hours, i fixed it now (5 hours after it happened). if a databse error like this ever happens again, it will be caught as a F95ZONE_UNAVAILABLE error instead and retried after 15 minutes.
i have invalidated cache for all the affected threads (3983 3959 136922 151469 79941 1132 1224 5541 76200 785 2796 6996 1297 135918 42937 47958 10703 4431 126890 20812 210467 202 727 26392 30383 4211 132123 214170 74065), should be fine now

Thank you. It works.

 

[simi00]

when you get an actual error reported like this (saying the f95checker cache api returned an error) it means it knows there was an issue and logged it for me to inspect, no need to investigate further, just notify me
it was different last week with the missing threads, those went unnoticed (considered thread as missing, but not an api error), and needed info from users to diagnose

Heh, I have been working as a programmer/analyst and as a support manager for 27 years, so it will probably be my nature to look for and identify the source of problems

 

[Blither]

Looks like 240210 and 209886 are still not refreshing WillyJL

 

[WillyJL]

Looks like 240210 and 209886 are still not refreshing WillyJL

those were others still affected from the not found bug, like i said i couldnt detect which ones were affected besides user reports. regardless it will not happen to new ones, just need to invalidate cache for those. done.

 

[Blither]

those were others still affected from the not found bug, like i said i couldnt detect which ones were affected besides user reports. regardless it will not happen to new ones, just need to invalidate cache for those. done.

Thanks, I appreciate all the work you put into this.

 

[WillyJL]

11.0.2 beta 1437

I found a couple of bugs.
When I move a game to a 'new tab' (a new tab is created, and the game is moved there), it becomes impossible to switch between tabs.

I restart the program, and everything works fine. I go into the 'new tab' move the game back, and when I try to close that tab, the program crashes with the this error (the tab does get removed though):
View attachment 4434539

confirmed, same here. But the tab does get removed though.

good catch, not sure how i missed it. fixed now, made a new beta build

 

[WillyJL]

i think ill push out 11.0.2 shortly. ASTC image compression has been included in recent beta builds but will not make 11.0.2, i still need to fix some issues with image error handling and state management, so it will probably be in 11.1 instead.