- Mar 23, 2021
- 158
- 138
- Apr 14, 2022
- 28
- 23
I tried rate limiter (asynciolimiter instead of aiolimiter though, it's still in my fork), but I still was getting errors even with like 20 to 5 requests per minute. As I see it whole problem not is per-user, it's overall forum load (not "ratelimiting you" but "ratelimiting everyone at the moment"). For sure it's better than firing requests without any limits at all, but not sure problem can be solved without explicit retries on 429.
upd: place you'd put limiter restriction is better than what I did, now I'm embarrased
Last edited:
- Apr 14, 2022
- 28
- 23
- Aug 21, 2020
- 1,266
- 1,790
yes, as i mentioned here - it works with proxy in your modded version
i'm using local Tor for it (SOCKS5 localhost:9150)
- Apr 14, 2022
- 28
- 23
Then I think Sam is probably right. Your browser is proxied somehow while checker app is not. It's either try ask/force your ISP to unblock attachments domain or use proxy/vpn in both.
- Dec 19, 2020
- 443
- 1,559
Do you ever plan to release an offical one on the main github page rather than requiring users to login and hunt through extra menus? It's 18 months out of date, it's time to push a proper update.
- Dec 21, 2019
- 168
- 187
From what i can tell, he is releasing only the final version of the checker, leaving beta and alpha builds in the action tab for people that want it and to be fair, i find it ok. That way you have a stable release, but if you feel adventurous you can go on Buggy side of the deal, albeit with a couple hoops to pass by. What i mean is that 10.2 is stable release even if old, and in action tab is 11 but beta build. Considering how many changes to get it back to stable are needed in the past week, to me at least it makes sense why he is doing this way.
- Mar 7, 2019
- 1,408
- 1,205
My priority these days is another project. I wanted to finally release an update here but wanted to fix Linux Wayland issues. Seems like that not gonna happen easily, and now there's this huge new problem, once this current issue is fixed properly I'll make a release
- Mar 7, 2019
- 1,408
- 1,205
Looks like I'll end up with a mixed solution then, use aiolimiter but retry indefinitely with some delay if a rate limited response is received. Do you have your rate limited fork code anywhere?I tried rate limiter (asynciolimiter instead of aiolimiter though, it's still in my fork), but I still was getting errors even with like 20 to 5 requests per minute. As I see it whole problem not is per-user, it's overall forum load (not "ratelimiting you" but "ratelimiting everyone at the moment"). For sure it's better than firing requests without any limits at all, but not sure problem can be solved without explicit retries on 429.
upd: place you'd put limiter restriction is better than what I did, now I'm embarrased
- Mar 7, 2019
- 1,408
- 1,205
just checked back thought the logs for the night, no errors at all, and no activity currently. Looks like it's working well and so far it's been managing to keep up with demand (if it hadn't, it would've been going through a backlog of threads to cache slower than it is asked to cache them)
- Mar 23, 2021
- 158
- 138
Over 580 full refreshes without an error. Was decently fast considering I had forgot to bump the number of workers back up.
- Mar 23, 2021
- 158
- 138
- Mar 7, 2019
- 1,408
- 1,205
Hmmm line 514 is in import_f95_watched_threads()... are you on latest build?
Also please send that broken file it saved
haha yea i edited my thoughts as i was brainstorming and even im struggling to read it now.
looks like you already implemented the main points
The one i was suggesting at the end was:
when user fast checks a game, return the timestamp of when the cache on server is due to expire (i.e. time when the server will attempt checking for changes as the cache would be stale). and if the next user fast check happens before that timestamp, do not include the game in the list of fast checks.
slightly reduces the load on your server as the data locally and on server are guaranteed to already be in sync until that timestamp
- Mar 7, 2019
- 1,408
- 1,205
that would be a great idea, except for one thing: game updates. kinda the whole point of all this really lmao
if an update happens before cache expiration, server invalidates the cache and will re-fetch the thread from f95zone the next time someone asks for it. but if instead, the client doesnt ask again until cache expiration because it doesnt even know it was updated, then it would take up to 7 days to find the update for users.
- Dec 21, 2019
- 168
- 187
Now I am just curious about something. Would it be possible to add a new tab like description etc, when you click on a game that pulls the screen's attachments in it? The ones from OP's post showcase the game. I know is not really needed, but I think for a completionist look, would be ok.
- Mar 23, 2021
- 158
- 138
LOL - I forgot to exit before updating so files were locked. I'm an idiot basically. Updated again to Beta 1227 and the sync took awhile (it is 2,539 games though and it just ran in background).
Some unusual behaviour - I updated just before exiting and installing update, yet got 38 updates that seem bogus (version change to N/A) and a few games that it says are no longer available on F95 (like https://f95zone.to/threads/quest-failed-chapter-2-christmas-special-frostworks.9774/ and https://f95zone.to/threads/sunny-love-to-cheat-v0-3-1-satanica-hime.195485/ ).
- Mar 7, 2019
- 1,408
- 1,205
- Mar 7, 2019
- 1,408
- 1,205
It's planned, I was working on it long ago but didn't get that far. Considering the new rate limits as of last week, it seems even less feasible now.
- Dec 21, 2019
- 168
- 187
Ah, I see. It makes sense that now it is even harder to implement it with the new rate limits. It would probably slow it down considerably if not even crashing it and the cache would be, I guess, considerably higher as well, at least on games that have 10-20 screens. Now, I have no idea how the app is working on these requests, so I might be completely off on this. but in my head, what I said makes sense.