Tool Others F95Checker [WillyJL]

[WillyJL]

i didnt find this error when i searched... but this popped up this morning:
{
"INDEX_ERROR": "PARSING_FAILED"
}

Yep, I have the same error
View attachment 4434199

Ditto like the rest today. Parsing failed error...

Hello

same here : when doing full refresh :
{
"INDEX_ERROR": "PARSING_FAILED"
}

when manually updating one game no issue

Thank

So the change I made yesterday did fix the logic issue as I expected, now it notices there is an issue, instead of just marking the thread as missing. I will check the logs when I'm in pc in a few hours.

 

[WillyJL]

WillyJL Any chance the server is down?

No. I heard f95zone itself was down for a while, maybe that's what you mean
If you mean the error that the others posted, I will look at it in logs in a few hours

 

[FaceCrap]

As always, I can't fetch all threads always from f95zone.

I understand, but that means the 'Full Refresh' option no longer works like before. Because when it went directly, that would have been picked up.
So you don't store the datetime data for when a thread was last queried?
Otherwise you could compare it to the date value in the json result to see if it's been modified since.
Also, this was an actual update, wouldn't it had to have the thread parsed in that case anyways instead of relying on the expiration cycle?

DDL goes directly to f95zone, it's not cached on my end. If the issue also happened with DDL, this sounds to me like something actually went wrong with that thread on f95zone end itself.

That was just extraneous info. Some mod probably forgot to add the mirrors. It's what lead to noticing the mirror difference

11.0.2 beta 1437
I found a couple of bugs.

confirmed, same here. But the tab does get removed though.

 

[simi00]

When I check one by one, then it works, but error shows up on game ID 74065 ( https://f95zone.to/threads/the-hard-way-v0-38-1-muffin-maker.74065/ ). Other games (checked on 50 games) works. I have around 400 games in list.

Other game with parse error:
https://f95zone.to/threads/the-innocent-mother-ch-2-spies.4211/

 

[WillyJL]

and if the actual underlying issue persists (as we see from people below your post) now I will see the logs for it

So the change I made yesterday did fix the logic issue as I expected, now it notices there is an issue, instead of just marking the thread as missing. I will check the logs when I'm in pc in a few hours.

doctorx6969 simi00 Baalzie triste_lune turns out that this error in particular was due to f95zone database downtime, not to the previous issue with threads missing. it didnt receive valid data from f95zone so of course parsing wasnt successful. it would have attempted again in 6 hours, i fixed it now (5 hours after it happened). if a databse error like this ever happens again, it will be caught as a F95ZONE_UNAVAILABLE error instead and retried after 15 minutes.
i have invalidated cache for all the affected threads (3983 3959 136922 151469 79941 1132 1224 5541 76200 785 2796 6996 1297 135918 42937 47958 10703 4431 126890 20812 210467 202 727 26392 30383 4211 132123 214170 74065), should be fine now

 

[WillyJL]

When I check one by one, then it works, but error shows up on game ID 74065 ( https://f95zone.to/threads/the-hard-way-v0-38-1-muffin-maker.74065/ ). Other games (checked on 50 games) works. I have around 400 games in list.

Other game with parse error:
https://f95zone.to/threads/the-innocent-mother-ch-2-spies.4211/

when you get an actual error reported like this (saying the f95checker cache api returned an error) it means it knows there was an issue and logged it for me to inspect, no need to investigate further, just notify me
it was different last week with the missing threads, those went unnoticed (considered thread as missing, but not an api error), and needed info from users to diagnose

 

[WillyJL]

I understand, but that means the 'Full Refresh' option no longer works like before. Because when it went directly, that would have been picked up.
So you don't store the datetime data for when a thread was last queried?
Otherwise you could compare it to the date value in the json result to see if it's been modified since.

of course it doesnt work like before, basically nothing works like before, it *cannot* work like before.
your f95checker client never talks to f95zone directly when it comes to getting thread info.
as stated in the github readme, now the full refresh means syncing data from the cache api. it doesnt force the cache api to recache the thread. if it did, we would be back at square 1: everyone forcing a full refresh and blasting f95zone till we DDoS it.

i do store the timestamp for last time a game was refreshed, on both sides:
- your client records the last time the thread data changed on the cache api. it works kinda like a "version" of cached data. any time the cache api updates the cached data, it saves the timestamp, then when your client refreshes it checks all the timestamps, if the cache api's cached data timestamp is different than what your client has saved, it syncs the whole thread info from the cache api.
- and on the cache api, i record when each thread was last fetched from f95zone, when the last time any of the contained data actually changed (this is to notify client that they should sync the new data, and avoids syncing if after cache expiration the new data is the exact same), and the time when cache is due to expire (not all threads have same cache expiry, like some errors can have as little as 15 minutes of cache, and non-existent threads are cached for 14 days)

Also, this was an actual update, wouldn't it had to have the thread parsed in that case anyways instead of relying on the expiration cycle?

yes. and it did. the issue is that mods pushed the update, someone refreshed the thread in f95checker, it was cached, and only *after* the downloads were changed. at that point, nothing changed that the cache api could detect.
its always human error. i cannot fix the 0.01%. sorry.

 

[simi00]

doctorx6969 simi00 Baalzie triste_lune turns out that this error in particular was due to f95zone database downtime, not to the previous issue with threads missing. it didnt receive valid data from f95zone so of course parsing wasnt successful. it would have attempted again in 6 hours, i fixed it now (5 hours after it happened). if a databse error like this ever happens again, it will be caught as a F95ZONE_UNAVAILABLE error instead and retried after 15 minutes.
i have invalidated cache for all the affected threads (3983 3959 136922 151469 79941 1132 1224 5541 76200 785 2796 6996 1297 135918 42937 47958 10703 4431 126890 20812 210467 202 727 26392 30383 4211 132123 214170 74065), should be fine now

Thank you. It works.

 

[simi00]

when you get an actual error reported like this (saying the f95checker cache api returned an error) it means it knows there was an issue and logged it for me to inspect, no need to investigate further, just notify me
it was different last week with the missing threads, those went unnoticed (considered thread as missing, but not an api error), and needed info from users to diagnose

Heh, I have been working as a programmer/analyst and as a support manager for 27 years, so it will probably be my nature to look for and identify the source of problems

 

[Blither]

Looks like 240210 and 209886 are still not refreshing WillyJL

 

[WillyJL]

Looks like 240210 and 209886 are still not refreshing WillyJL

those were others still affected from the not found bug, like i said i couldnt detect which ones were affected besides user reports. regardless it will not happen to new ones, just need to invalidate cache for those. done.

 

[Blither]

those were others still affected from the not found bug, like i said i couldnt detect which ones were affected besides user reports. regardless it will not happen to new ones, just need to invalidate cache for those. done.

Thanks, I appreciate all the work you put into this.

 

[WillyJL]

11.0.2 beta 1437

I found a couple of bugs.
When I move a game to a 'new tab' (a new tab is created, and the game is moved there), it becomes impossible to switch between tabs.

I restart the program, and everything works fine. I go into the 'new tab' move the game back, and when I try to close that tab, the program crashes with the this error (the tab does get removed though):
View attachment 4434539

confirmed, same here. But the tab does get removed though.

good catch, not sure how i missed it. fixed now, made a new beta build

 

[WillyJL]

i think ill push out 11.0.2 shortly. ASTC image compression has been included in recent beta builds but will not make 11.0.2, i still need to fix some issues with image error handling and state management, so it will probably be in 11.1 instead.

 

[jinnhihho]

just skimmed through the last few msg, I think https://f95zone.to/threads/tales-of-legendary-lust-aphrodisia-build-2-f-crimsondelightgames.79920/

is still throwing the caching error, I think you have to invalidate cache for that if I understood that right.

 

[WillyJL]

just skimmed through the last few msg, I think https://f95zone.to/threads/tales-of-legendary-lust-aphrodisia-build-2-f-crimsondelightgames.79920/
View attachment 4435353
is still throwing the caching error, I think you have to invalidate cache for that if I understood that right.

done

 

[triste_lune]

doctorx6969 simi00 Baalzie triste_lune turns out that this error in particular was due to f95zone database downtime, not to the previous issue with threads missing. it didnt receive valid data from f95zone so of course parsing wasnt successful. it would have attempted again in 6 hours, i fixed it now (5 hours after it happened). if a databse error like this ever happens again, it will be caught as a F95ZONE_UNAVAILABLE error instead and retried after 15 minutes.
i have invalidated cache for all the affected threads (3983 3959 136922 151469 79941 1132 1224 5541 76200 785 2796 6996 1297 135918 42937 47958 10703 4431 126890 20812 210467 202 727 26392 30383 4211 132123 214170 74065), should be fine now

OK, noted THANK YOU!

so now we will saw the error : F95ZONE_UNAVAILABLE which will make it more clear to just wait.

i tested and now it work fine

thank for the support and follow up.

 

[WillyJL]

just pushed 11.0.2, next up will be 11.1 when im done working on image compression. latest beta includes image compression if youve been using it, while 11.0.2 does not.

 

[jinnhihho]

Just tried to update to 11.0.2 and windows defender is not happy about it, says that shit is the coming of the devil himself

e: still updated it and it seems to work fine so far

 

[WillyJL]

Just tried to update to 11.0.2 and windows defender is not happy about it, says that shit is the coming of the devil himself

e: still updated it and it seems to work fine so far

Not sure, didn't change anything that would affect that really. As with all executables, getting a certificate and signing them is expensive and really the only way to get a chance to shut up antiviruses for good, and I'm not gonna do that, so yeah we're at the mercy of shitty corporations. Antiviruses are useless nowadays, they can only detect known signatures, if something was dangerous it wouldn't use pre-existing malware. The only good antivirus is your brain. I always disable defender whenever I use windows (which isn't often).