so, i checked logs, there were some errors:
- some generic connection errors i hadnt accounted for, they retried after 6 hours now i made them retry after 15 mins like they should
- ddos-guard being a bitch 2 times in total, added its new message to list of temporary errors that retry after 15 mins
- 1 php error on latest updates, told sam about it as it seems worth taking a look at just to be sure
- 1 unexpected error fetching a thread, added to list of errors to retry after 15 mins
but, all of those were caught even if not handled optimally (they will be now if they happen again), and none of those are what caused the games that "arent refreshing".
the games that "arent refreshing" dont have anything weird in logs (if there was an error, it wouldve told end users too actually). how it detects these is if the thread page returns a 403 or a 404, so it actually did receive such a response for those games.
i checked with the user account the api uses to fetch threads, they are all visible to it, so yeah it was something temporary.
i have 2 theories:
- mods privated the thread temporarily, it was refreshed at that time, then opened it back up
- maybe ddos-guard sends 403 when it wants to be a bitch?
the only other 2 instances of ddos-guard *that i see in logs* are on 2024-12-29 which i feel is about when y'all started reporting the missing threads, and it was in the latest updates watcher that it was caught, which doesnt have any 403/404 logic. when fetching game threads, it first checks for known error strings (ratelimit, f95zone unavailable, login required...) and then checks for 403/404, so it never got to actually try to use the received data to parse the thread and just assumed it was gone.
i have now added the new ddos-guard message string to the prior check, so it should be able to detect it and just retry in 15 minutes instead of marking it as missing for 14 days.
also changed how it detects missing threads:
- checks known error strings
- tries to parse the thread
- if cant find OP in response, checks if response is from f95zone (looks for
.p-header-logo img[alt="F95zone"])
- if cant find OP, can find f95zone logo, and status is 403 or 404, marks as missing with no log messages and retries in 14 days
- if cant find OP and cant find f95zone logo, gives a parsing error (gets logged on my end and sent to users too so someone notices) and retries in 6 hours
in the case of these missing threads, i assume it would now fall at the first step checking for known error strings, and if in the future ddos-guard decides to change the error strings again it will fall into the parsing error situation instead of silently marking as missing.
still, i find it weird that ddos-guard would return 403...
You must be registered to see the links
this states that 403 is only returned for block lists, tor nodes and geoblocking, and i doubt any of these is the case... if it were, no games at all would be refreshing...? so weird.
if it indeed is ddos-guard (as odd as that would be to give a 403) then would also be useful to see if sam can add an exception for my server's ip, but before that i want to see if these changes catch the issue now
EDIT: actually,
You must be registered to see the links
this here shows a ddos-guard browser verification that returned 403, so it really does look like its whats happening. the 2 ddos-guard errors that i did see in my logs had a different text content but seems reasonable that different ddos-guard pages with similar purpose would give the same response code. also, that issue 5 years old so also possible they change the look of the page (would also explain why the previous error string i had for ddos-guard didnt match, it had changed slightly with different capitalization)
