Tool Others F95Checker [WillyJL]

5.00 star(s) 23 Votes

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Well, I launch F95Checker.exe through a shortcut on the desktop.
By logs, do you mean error notifications? If so, no, there aren’t any notifications.
would be really useful if you can run the debug executable, but not by double clicking, instead running it in a cmd. this way, when it crashes, you can see output in console. this will be nearly impossible to fix without an error log
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Is it possible that I had an old shortcut on my desktop? When I installed a new build, I deleted all the files of the old build and threw the new one into the folder, but the path remained the same.
that shoudl be fine

Well, if you tell me how to do it, no problem. I know what cmd is, but how to run F95Checker-Debug.exe through it?
open cmd, cd to the folder of your f95checker install, then just type F95Checker-Debug.exe
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Unable to import OpenGL.arrays.numpymodule.NumpyHandler: No numpy module present: No module named 'numpy'
thats normal, the fucker writing the opengl library made it where it just prints that with no option to disable it, and it doesnt matter to normal usage

Now when it crashes, will there be info in the command prompt? “sorting problems”, will that be there too?
depends whats happening. if its an exception, there should be a traceback. if the sorting issues are related to the exception causing the crash, this will help. if its a kernel level process crash, like a segmentation fault, then might be less useful but hopefully should still give some hint
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
i just managed to replicate the sorting issue myself with some logging. first of all, my gui code was converting imgui sorts to a list of sort specs that the tool can understand, but every frame, this should be fixed. but also, while logging this, i noticed that it suddenly changed from index=12 that i was sorting by to index=613. gonna try to understand why, but could very well be some bug in imgui
EDIT: triggered it again, now index=1450, then index=0. looks like memory corruption
EDIT: or maybe not a bug in imgui, but a bug with how i interact with imgui via the python wrapper that causes memory corruption. leading theory rn is that the table sort specs object that pyimgui gives me is being invalidated and mvoed to another place, but i keep an older reference. gonna try to get a fresh pointer each time i interact with the sort specs, and also do it inside the recalculate_ids lock
EDIT: been trying to find whats going wrong by exclusion, no luck so far. tried not setting specs_dirty = False in case that was writing in the wrong palce in imgui memory space, tried disabling everything to do with DB writes in case something went wrong in C code, tried disabling image rendering which could be finicky, nothing, still manage to reproduce it after a while
EDIT: yeah, im at a loss here. nothing in my code seems wrong and nothing im trying seems to affect it. ill leave it be for now, maybe try to find a way to force imgui to recalculate the proper sorting when i receive bogus indexes. as for what you mentioned about build 1174 and 1166, i dont see anything of relevant in
EDIT: only way i can think of to trigger imgui to send the sort specs again is to render a frame with no imgui commands at all, tried this and when it triggers it skips a frame, other imgui context get reset (like popups get closed) but sorts remain corrupted
 
Last edited:

FaceCrap

Ghost of torrents passed
Donor
Oct 1, 2020
1,305
886
i just managed to replicate the sorting issue myself with some logging.
Ah, good to hear, I was just about to come back to post it happened to me in 1301.

And the way I ran into it was by assigning executables in a batch. But... so far only through a very specific workflow.

If I used the context menu to add an exe, sort order stayed the same.
If I used the More Info popup AND navigated to the next game with the left/right arrows on the side, sort order stayed same.

Shit hit the fan if I closed the More Info popup after setting the executable and reopening it on the next game.
Took me only assigning executables this way to 4 games to see sort order changed on the 5th.
.
I do need to stress that I've never ever seen sort order change after a Refresh. I so far have only experienced it with setting an executable for a set of games in succession in the way I described.

I also need to stress that the context menu method and the left/right button method on the More Info popup aren't foolproof free of this. I had it happen with these methods too, but it was way less often and took more than a few executable assignments before it happened. But so far in this build haven't seen it happen with these methods. Only the "open More Info/Assign exe/close More Info/open More Info on next game by clicking it in the list" workflow.

And yes, I know that this is a inefficient way of doing things. I just am not used to those left/right buttons and right-clicking. Muscle memory prefers left-click on game, left click outside popups to close them.
 
Last edited:

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
I do need to stress that I've never ever seen sort order change after a Refresh. I so far have only experienced it with setting an executable for a set of games in succession in the way I described.
ive seen it corrupt the sorting 4 times in the past hour right after a refresh. its seemingly random. just decides to corrupt the memory where imgui stored sorting info after certain actions.
 

FaceCrap

Ghost of torrents passed
Donor
Oct 1, 2020
1,305
886
ive seen it corrupt the sorting 4 times in the past hour right after a refresh. its seemingly random. just decides to corrupt the memory where imgui stored sorting info after certain actions.
Manual refreshing or auto-refresh in the system-tray? I never use auto-refresh and only manual, but also with hours in between each next refresh, maybe that's why I haven't seen it happen then?
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Manual refreshing or auto-refresh in the system-tray? I never use auto-refresh and only manual, but also with hours in between each next refresh, maybe that's why I haven't seen it happen then?
No, it was manual refreshing. Sometimes it's switching tab, sometimes making installed, sometimes changing labels, something archiving, sometimes changing executables. Seems like anything that triggers a resort/refilter has a chance to cause a memory corruption to the sort specs
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
man if this was the issue...

"deadlock" totally fits the description
View attachment 4273732

or could even be this...
View attachment 4273724

who wouldve guessed that updating shit fixes bugs xD
1 good piece of news atleast, the timeout happened again tonight and it recovered successfully. so it was in fact just a bug with aiohttp that had already been fixed upstream. i have now also added proper parsing of timeout errors, so they are considered "normal" (as in just a connection issue to be retried shortly, not an error, so now if a timeout like this happens while updating cache for a certain game, it will be marked with error f95zone unavailable and be retried after 15 mins ( as opposed to the generic exception's 6 hour retry timer)
 
  • Heart
Reactions: Red Sand

FaceCrap

Ghost of torrents passed
Donor
Oct 1, 2020
1,305
886
so now if a timeout like this happens while updating cache for a certain game, it will be marked with error f95zone unavailable
Is this the message you refer to?
1732789806380.png
Coz I thought the whole purpose of caching was to prevent this?
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Is this the message you refer to?
View attachment 4278042
Coz I thought the whole purpose of caching was to prevent this?
No. That's client side. Rate limit is very different than a timeout. Timeout means no response within the designated time, rate limit means the server responded that you are asking too much and need to slow down. The timeout refers to the issue that was happening on my server, the rate limit you see is happening between your pc and the forum. You requests to f95zone are already slowed down to 1 every 2 seconds, and when a rate limit response is received it retries up to 10 times. So when you see this message is because you're rate limited and it will not get better soon. It has already tried for 10 times over 20 seconds, continuing to make the request hoping that the rate limit passes is pointless. And that's just for getting the notifications, which is just 2 requests: check if logged in, request notification counts. And you're getting fully rate limited for over 20 seconds. That just shows how aggressive the rate limit is and how not using the new cache api was impossible moving forward.
 

FaceCrap

Ghost of torrents passed
Donor
Oct 1, 2020
1,305
886
That just shows how aggressive the rate limit is and how not using the new cache api was impossible moving forward.
Ok, but I thought that the new cache api would prevent this from happening. e.g. the checker would not be accessing the forum at all during a refresh apart from checking for notifications or when doing a quick search.
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
Ok, but I thought that the new cache api would prevent this from happening. e.g. the checker would not be accessing the forum at all during a refresh apart from checking for notifications or when doing a quick search.
that is precisely what i said. the ratelimit popup you got is due to being ratelimited from simply checking the notifications. disable checking for notifications, and you will not get any ratelimits. and again, you got that popup because simply checking the notifications was rate limited for 10 times in a row over 20 seconds.
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
WillyJL
Well, the program just crashed, but there's nothing on the command prompt View attachment 4278641

I wasn’t doing anything with the program at that moment. I was just checking stuff online.
possibly ran out of memory, seems like the only possible explanation to me. drawing images with opengl is memory intensive, and browsers are huge memory hogs too. when you run out of memory and dont have swap space the kernel just kills programs at random i believe
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
that is precisely what i said. the ratelimit popup you got is due to being ratelimited from simply checking the notifications. disable checking for notifications, and you will not get any ratelimits. and again, you got that popup because simply checking the notifications was rate limited for 10 times in a row over 20 seconds.
on another note, i may have found a way to reduce the impact, where notifications are less expensive to check (no strict need for xftoken), and quick search doesnt require login (can use latest updates api).
at that point, the only actions that would require an f95zone account are:
- download links in any form (xpath direct, masked, donor ddl)
- notifications
- import watched/bookmarked threads
- opening threads when browser > download pages is enabled (otherwise there would be no links or spoiler content)
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
I have 32 GB of RAM
then im sorry, i have no clue what is going wrong or any way to find out more. if more users report these random crashes i will try to look into it.
as for the sorting issues, yeah seems like a memory corruption with pyimgui / imgui itself, not much i can do from python, and its gonna be insanely difficult to debug this.
 

WillyJL

Veni, vidi, vici
Donor
Respected User
Mar 7, 2019
1,318
1,121
on another note, i may have found a way to reduce the impact, where notifications are less expensive to check (no strict need for xftoken), and quick search doesnt require login (can use latest updates api).
at that point, the only actions that would require an f95zone account are:
- download links in any form (xpath direct, masked, donor ddl)
- notifications
- import watched/bookmarked threads
- opening threads when browser > download pages is enabled (otherwise there would be no links or spoiler content)
and search has more options as a nice bonus of this change:
1732841364691.png
still triggered the same way, but similar to rpdl search you can change query afterwards and even change category or look in author instead of title
 
  • Heart
  • Crown
Reactions: Red Sand and Sam
5.00 star(s) 23 Votes