Tool Others F95Checker [WillyJL]

5.00 star(s) 23 Votes

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
if you go back to 8.5, make sure to delete the db.sqlite before using 9.0 again
Was thinking of reducing that number by only adding relevant games and animations thread only. I have come up with some solutions but with a "it's 7am and I haven't gone to sleep" brain rn

What are the step if I was to start over and redo the games list on 8.5?
Or (sorry to make you repeat the programming base of 8.5 and 9.0 if I make a mistake) instead of doing a fresh list, could I theoretically go in the folder path of 9.0, copy (or move) the files and drop them in the 8.5 folder Or should I not bother with the extra work and wait till a more stable release exists and use 8.5 in the meantime?
 

forgetmaster

Active Member
Oct 14, 2020
633
395
It’s like, you have to check 100 games, and you give them to some people to check them. If you divide them between 100 people it will take less, if you divide them between 20 people it will take more. However when there are more people working it can be an issue with your computers capabilities, so they fail. Try 20, if that doesn’t work lower again and again until you get to 1
50 is the sweet spot for me, its faster and still stable. Thanks
 
  • Like
Reactions: WillyJL

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
Or (sorry to make you repeat the programming base of 8.5 and 9.0 if I make a mistake) instead of doing a fresh list, could I theoretically go in the folder path of 9.0, copy (or move) the files and drop them in the 8.5 folder Or should I not bother with the extra work and wait till a more stable release exists and use 8.5 in the meantime?
doesnt really work like that... 8.5 uses a .json file, plain text based, old data structure. 9.0 uses a database, not plain text, in sqlite3 format. its not a folder that keeps the data, its a file. and since they are fundamentally different in every way possible its not really doable
 

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
doesnt really work like that... 8.5 uses a .json file, plain text based, old data structure. 9.0 uses a database, not plain text, in sqlite3 format. its not a folder that keeps the data, its a file. and since they are fundamentally different in every way possible its not really doable
With all due respect and the last of my sober/sane brain cell, I didn't post 3 alternative only to be explained a 4th time the difference between the file format used for 8.5 and 9.0.

What can I do to temporarily fix this while you work on a stable release?
 

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
What can I do to temporarily fix this while you work on a stable release?
as far as i can tell the issue is that in your case it has to download like 1gb of images and at times f95zone disconnects or it timeouts.

what i changed:
- requests are retried up to 3 times before failing
- full checks use 2 times more timeout
- image downloads use 4 times more timeout
- max amount of simultaneous image downloads is 1/10 of the workers you set
- fixed an issue with it hanging and stopping downloads completely

it will still take a while because, well, it has to download 1gb of images...
but hopefully it will not fail anymore (it hasnt so far for me)
i will add a toggle to disable images altogether in the future

for now you should download and use:
- 20 or less workers
- 30 or more timeout
(it will download all images only the first time, after it has the images it will refresh way faster)
 
Last edited:

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
as far as i can tell the issue is that in your case it has to download like 1gb of images and at times f95zone disconnects or it timeouts.

what i changed:
- requests are retried up to 3 times before failing
- full checks use 2 times more timeout
- image downloads use 4 times more timeout
- max amount of simultaneous image downloads is 1/10 of the workers you set
- fixed an issue with it hanging and stopping downloads completely

it will still take a while because, well, it has to download 1gb of images...
but hopefully it will not fail anymore (it hasnt so far for me)
i will add a toggle to disable images altogether in the future

for now you should download and use:
- 20 or less workers
- 30 or more timeout
(it will download all images only the first time, after it has the images it will refresh way faster)
Hello IT, Is this a virus?
1655680019402.png
:ROFLMAO: I didn't expect to test my luck and hope the problem is solved, only to be interrupted by something as minor as daily backups. 42% and going strong

==========IT WORKED 99% ============
Update: Good news is I didn't get any errors
Bad news is I didn't see any changelog (what game updated to which version) and such

update 2: major fix: empty lines formed
minor fix: add a sound when the updates are 100% done
optional: multiple choice/customised sound
 
Last edited:
  • Haha
Reactions: WillyJL

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
Good news is I didn't get any errors
:love:

Bad news is I didn't see any changelog (what game updated to which version) and such
it doesnt do that on the first successful refresh on 9.0 because, since i changed how version detection works (before it used the title, now it uses the OP content), basically all games would count as updated (version text is slightly different between title and content, content one is usually more detailed). from now on if a game is updated you will get a popup.
(also of course if you just added the game, on its first refresh to get its information it will not count as updated!)

major fix: empty lines formed
?????????

minor fix: add a sound when the updates are 100% done
optional: multiple choice/customised sound
i can kinda understand this from your perspective, but im not too sure its needed. right now if it finishes refreshing with no updates or alerts it will stay silent. however if games receive updates or you have unread alerts, you will have a popup in the interface, and if the interface is not focused (maybe youre using another program) you will receive a desktop notification saying "123 games have been updated, click here to see them" or "you have 123 unread notification, click here to see them". also goes for errors, youll have a notification saying "something went wrong, click here to view the issue". basically any time the interface needs your attention, but it is not focused, you will receive a notification.

if you still feel like you need to always know, i might make it a toggle in the future, but do tell me if thats the case. also in that case it probably will not be a sound, but just a standard desktop notification like all the others, since its already implemented
 

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
?????????
sry forgot to post picture
1655683503348.png

if you still feel like you need to always know, i might make it a toggle in the future, but do tell me if thats the case. also in that case it probably will not be a sound, but just a standard desktop notification like all the others, since its already implemented
nah I just had a funny idea at the time and thought about smartphone who have sound notifications that you can customise for each app and each notifications category. plus there's already notifications implemented using default windows sound
 
Last edited:
  • Like
Reactions: WillyJL

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
sry forgot to post picture
ah yeah that makes sense, as (almost) always a small overlook in my code, doesn't check for new lines. new build should be ready in maybe 15 minutes, when youre on the new build right click the refresh button and do a full refresh (or do a full refresh for each broken game)
 

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
ah yeah that makes sense, as (almost) always a small overlook in my code, doesn't check for new lines. new build should be ready in maybe 15 minutes, when youre on the new build right click the refresh button and do a full refresh (or do a full refresh for each broken game)
Sorry if I'm late but that fixes it.
I've put the pc to hibernate after the full refresh is done (also I have even more features to discover with the right click),
Came back later, woke it up and a screen appeared showing new game got updated. I'm so happy it works!

I remember you mentionned bg mode doesn't work or something similar, is that still the case?
Also what are the next plans of development now?
 

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
Came back later, woke it up and a screen appeared showing new game got updated. I'm so happy it works!
:love:

I remember you mentionned bg mode doesn't work or something similar, is that still the case?
implemented about a week ago, works perfectly (at least in my experience, and we know how that can go xD)
just like when unfocused (actually its the same code for that) if the interface requires your attention, may it be due to an update, some unread alerts or some error, you will receive a notification that when clicked will show the interface

Also what are the next plans of development now?
right as we speak i am fixing some little things with the ui that were yes small, but still quite annoying and felt unfinished:
- now hovering over stuff and scrolling while the checker is not focused will still react as if it was focused.
- you can now press escape to close weak popups (those that dont require much attention, the same popups that you can close by clicking outside them)
- drag values (the settings that show a numeric value you can drag to edit) can now be edited numerically and arbitrarily by single clicking on them, before it was double click

and right before i saw your message i was finishing up another one: reactive cursors, which means pointer when hovering a button / interactable, ibeam when over a text input and normal arrow otherwise. it wont be perfect because detecting the interactable items is a bit of a pain, but still a much needed improvement.

PS: those changes are implemented on dev branch but still not in any build!


all thats left is:
- cleaning up connection error popups, right now they spit out a traceback twice the size of the window. ill make it a dropdown
- fixing start with system on macos
- implementing the autoupdate system for the tool itself, i was almost forgetting that one

after those id say its about ready for release. of course more testing would be good, but since you say youre not having too many issues anymore im guessing it will be mostly fine xD
 

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
:love:


implemented about a week ago, works perfectly (at least in my experience, and we know how that can go xD)
just like when unfocused (actually its the same code for that) if the interface requires your attention, may it be due to an update, some unread alerts or some error, you will receive a notification that when clicked will show the interface


right as we speak i am fixing some little things with the ui that were yes small, but still quite annoying and felt unfinished:
- now hovering over stuff and scrolling while the checker is not focused will still react as if it was focused.
- you can now press escape to close weak popups (those that dont require much attention, the same popups that you can close by clicking outside them)
- drag values (the settings that show a numeric value you can drag to edit) can now be edited numerically and arbitrarily by single clicking on them, before it was double click

and right before i saw your message i was finishing up another one: reactive cursors, which means pointer when hovering a button / interactable, ibeam when over a text input and normal arrow otherwise. it wont be perfect because detecting the interactable items is a bit of a pain, but still a much needed improvement.

PS: those changes are implemented on dev branch but still not in any build!


all thats left is:
- cleaning up connection error popups, right now they spit out a traceback twice the size of the window. ill make it a dropdown
- fixing start with system on macos
- implementing the autoupdate system for the tool itself, i was almost forgetting that one

after those id say its about ready for release. of course more testing would be good, but since you say youre not having too many issues anymore im guessing it will be mostly fine xD
Ooh that's nice, very nice actually. Almost salivating by reading the backend stuff

I either misread or overlooked it but I did post some bugs I noticed before and wanted to remind you of some more

Game thread: some changelog and game descriptions won't show properly even if the game thread has a changelog/desc present

Toggling column on but has nothing to show (that was the case for date column I think

Other: imported discussion thread will not have pictures, I suggest having a category different than the green Other used for game engines for non game/art thread (could make a single one like Misc. Or separate it as multiple since there are tools, mod, etc.. )
 

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
Game thread: some changelog and game descriptions won't show properly even if the game thread has a changelog/desc present
unfortunately theres not much i can reasonably do about that... f95zone is designed as a forum, and game threads are just standard user submitted messages that tend to follow some sort of convention. however, while they might seem very easy to identify as a human, writing it in code is not so trivial. yo ucan see for yourself in inspect element if you know the basics. each piece of text is inside a <p> tag (paragraph). bold text is inside <b> tags. the problem is that if there is no bold text, then the different attributes (the "Version:" or "Overview:" text) all count as one. on the other hand, if there is bold text inside the attribute value (for example say on the first line of the overview there is a character's name in bold) then it will be split into multiple elements.
so yeah technically it is possible to fix this issue but it would require way more string analysis and manipulation than my brain can handle; for now the best way is to grab all the text after the attribute tag until a bold tag is found.

if you find some thread that isnt correctly formatted and really want to make OP fix it, ask for all the attribute names (for example "Version:") to be bold and the attribute values (like "v12.5") to not be bold.

(btw bold tags nested inside paragraphs or other elements, like say for example a spoiler, dont count for this. it proceeds laterally by looking at the sibling elements, not at the child elements)

Toggling column on but has nothing to show (that was the case for date column I think
do you have an example / steps to reproduce? i never encountered this...

Other: imported discussion thread will not have pictures, I suggest having a category different than the green Other used for game engines for non game/art thread (could make a single one like Misc. Or separate it as multiple since there are tools, mod, etc.. )
thats true, i did come into that problem a few times myself. its a bit of a trade off because theres like maybe a hundred total post prefixes, and by how i designed my implementation each game can only have one, while on the website you can have multiple.
so how i do it now is it prioritizes content types:
- cheat mods, collections, mods, tools
then post types:
- read me, request, tutorial
finally game engines:
- adrift, flash, html...

also since i never explored much outside games and collections, im not experienced with the prefixes like manga, siterip, comics, cg, pinup, video, gif... so i put them all under the single "collection" type, but at that point it could even not be a collection... i was thinking of maybe renaming it "media"? what do you think?

but yes "others" as game engine should be separate from misc threads that dont fit any of the above criteria, thats an easy fix

so to recap currently all the possible types are these:
1655742234051.png
(since i moved the others type and added the misc type, youll need to do a full refresh otherwise all your types will be off by 1 (internally they are handled numerically for better performance))
do let me know about renaming "collection" to "media" or if you have something better in mind

EDIT: i think i found a final solution, and committed it to upstream:
- "Collections" is now "Media". the color is a bit darker too (before it was the same gray as flash type)
- "Unreal Engine" is now "Unreal Eng". it is still the longest type, but just about the same as "Cheat Mod". this means that now all type labels (the small colored rectangles) are a lot smaller and reasonable.
- as already mentioned above i added the "Misc" type for posts that dont fit any other type.

final look:
1655754769274.png
 
Last edited:

GAB

Salty Montrealer
Donor
May 10, 2017
2,402
18,132
ok so far this is what you have to deal with, according to your existing filters and adding the the latest updates prefixes available, I added some main category to make it easier to read
Tagsv1.png
you'll quickly notice that some duplicates are present, you could keep them the same colors as on the forum but then the name might be confusing

Could also have different shade of the same colors

Also you could group them and pick the same shade of colors for each, excluding the games, cause you're already pretty advanced with it. ex:
comics are all diff shades of pink,
animations are shades of red,
posts are shades of green
contents are shades of blue
Tagsv2.png

if you want to implement your Media idea, that's how I understood it:
Tagsv3.png
 

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
ALL developers show as unknown? cos they dont on my end, very few (4/298) are infact, and its the same issue as the oveview and changelog, bad post formatting (or the thread really does not have an author, like misc threads)

Also is it possible to size all columns to fit?
F95Checker_swERB5T7HY.gif
well you cant size them all to fit if these isnt enough space... either way right click on the header allows you to size that specific column to fit, or you can size all columns to default. default is fit for all resizable columns, apart from name which is flexible. keep in mind that i made it so it only draws the text for the games that are visible at one time, so that in cases like yours it doesnt have to render 1k games at the same time. this means that it will fit based on what is currently visible, not all values.
 
  • Like
Reactions: GAB

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
if you want to implement your Media idea, that's how I understood it:
Tagsv3.png
basically yeah, thats how it ends up. i get the problem of the duplicates, but then it can get quite spaghetti codey trying to check which one to apply, and also the shades could be a good idea but they would end up being different from on the website...
to keep things simple what i did is:
1655832249679.png
(the grey lines prefixed with # are comments and just for my own sake, they do nothing in practice, types are not actually grouped in any way)
so basically going from top to bottom, as soon as it finds one of the prefixes it gets assigned that type. i used that order because for example you have therads that are marked [tutorial] [rpgm] or [tool] [others] (now that i think about it my own post is prefixed wrong lol)
in the end all colors are the same as on the forum, apart from misc (yellow) and media (dark grey) which of course are not present on the website. chose those colors because they are the least used across the other prefixes
 

WillyJL

Well-Known Member
Respected User
Mar 7, 2019
1,116
930
btw also added a open thread button on the updated games popup 1655833049412.png
thats the last thing to do that came to mind, now only need to fix that thing on macos and make the update system. release should be quite soon
 
5.00 star(s) 23 Votes