New Game Manager

[bakerdenn]

I am in the early phases of developing a web-based game manager applications to manage local game libraries. I realize that a similar project already exists and has been in development for some time now, and my aim is not to usurp that work but to use this project to build my development skills. In my professional life, I am a semi-developer (meaning that I develop, but it isn't my primary job role). To date, all of my development has been in an enterprise environment where my ability to innovate is somewhat stifled. This project will be a chance for me to explore new techniques that are off-limits to me in my professional work.

I'm posting this thread in hopes that this community will provide me with some of the features they would like to see in a management application. My plans at the moment include a web-based manager that can retrieve games from the site on demand. It will allow for private game rankings and notes, personal tag management (tags will be loaded from F95 initially, but can be modified as the user sees fit), and a locally installed service for Windows (initially) that will allow for launching games and tracking statistics on time played, managing library folders, and archiving/deleting old versions/unplayed games.

I would be very interested to hear your thoughts/ideas on what I have and what you would like. Please keep in mind that this is very much a back-burner/after-hours/as I feel like it kind of project and any deliverable will probably be some time off.

Thanks in advance for your feedback!

PS: While I have already developed a method to retrieve data from this site for games in real-time, the code is somewhat fragile at the moment. Does anyone know of a good way to access game-related data programmatically (and API perhaps)?

 

[Madmanator99]

I like the idea and the features you are planing, it sounds interesting to me. All I can think of as far as features go, is maybe try to retrieve the games' reviews/score as well? It can help a user pick a game, (while keeping the private ranking for his own list of games).
But I don't know about the "possible/allowed" side of that, I'm just a user hehe.

 

[bakerdenn]

I've considered that as well. However, I feel it is proper to leave the searching/discovering aspects to the F95 site and the library management aspects to my application. I wouldn't want to remove interactions with F95, but would rather enhance what is already there.

Thanks for the feedback.

 

[Madmanator99]

Alright, how about retrieving the screeshots? Would that be possible? I mean, how much info will be available to the user. For example, do I have to already know the name of the game, or will there be maybe some kind of random discovery banner thingy that offers different games? Maybe a rolling thing that shows games at random based on ... Ah no reviews scores, gotcha. Well, then maybe based on how many downloads within a time frame? Or answers to the main post? (sorry when I don't know how to code something, I call it a thingy).
Basically, what do you think about the idea that the game manager should allow for the discovery/search of games (by tags, most active/recent posts, etc).

The idea/feeling I am getting so far is that it is a tool for the established users of F95, and that is totally fine with me, but I'm just curious (I'm not a web dev but I know alot can be done with it)

 

[bakerdenn]

Since F95 doesn't have a public API for me to pull data from, my options are somewhat limited. I hope to include the screenshots in some manner, but I cannot say for certain at this point. Right now, the most effective means of adding a game is by using what I call the thread ID that is listed in the URL:



Those are easy to find and type and do not change if the game version changes (unlike the rest of the URL that includes the version).

My plan is to use the games that are managed collectively by all of my users to make suggestions on what is popular based on my data, but it will take some time to develop a critical mass of users for that to be useful. Again, I'm not trying to replace F95's logic (mainly so I don't get into trouble with them).

In the beginning, I really plan to rely on my users to find the games they want to manage with my app. As it grows, what I'm capable of doing will also grow.

 

[MillenniumEarl]

Hi bakerdenn!

Maybe I can help you out.

To make my life a little easier in managing the updates of the various games I have on the PC I have developed a game manager in Electron (I come from C# but I wanted to experiment with something new).

I found myself in the same situation as you: given the lack of APIs so I started building them from scratch and developed

You must be registered to see the links

.

These APIs are fast (<3 sec per game) and use

You must be registered to see the links

(HTTP request) to get platform game data (games/mods only) from both URL and search by name.

As you are developing a web-based game manager you could use the package directly without having to reinvent the wheel (assuming you are working with Node JS/Javascript, otherwise feel free to look at the repository and see how it works).

 

[bakerdenn]

I'll take a look. I'm actually writing the app in Dot NET Blazor (I need to learn that for work and this is as good a reason as any to use it). I'm sure your work will be helpful.

 

[bakerdenn]

Hi bakerdenn!

Maybe I can help you out.

To make my life a little easier in managing the updates of the various games I have on the PC I have developed a game manager in Electron (I come from C# but I wanted to experiment with something new).

I found myself in the same situation as you: given the lack of APIs so I started building them from scratch and developed

You must be registered to see the links

.

These APIs are fast (<3 sec per game) and use

You must be registered to see the links

(HTTP request) to get platform game data (games/mods only) from both URL and search by name.

As you are developing a web-based game manager you could use the package directly without having to reinvent the wheel (assuming you are working with Node JS/Javascript, otherwise feel free to look at the repository and see how it works).

In looking at your code, you and I approached the issue of retrieving data in much the same way. I don't have my code in a public repo yet (that's coming) as I haven't merged all of my testing bits into a single project (this is still very early in the process for me). Thanks for sharing!

 

[jababda]

For anyone who is interested I have made my repo public. At this point, there is very little there, but if you really want to see...

You must be registered to see the links

Is the Repo still public/ did you get any further with this?

 

[bakerdenn]

It isn't public at the moment as I have to make some changes in that department for reasons I'd rather not go into. I have been working on this project as time has allowed me to. I hope to get something going reasonably soon, but, as I stated in the first post, this is on the furthest of back burners for me since I don't see it helping to pay any bills I hope you understand.

Once I have more finished, I will add to this post with links to the manager site and the repo.

 

[bakerdenn]

Hi bakerdenn!

Maybe I can help you out.

To make my life a little easier in managing the updates of the various games I have on the PC I have developed a game manager in Electron (I come from C# but I wanted to experiment with something new).

I found myself in the same situation as you: given the lack of APIs so I started building them from scratch and developed

You must be registered to see the links

.

These APIs are fast (<3 sec per game) and use

You must be registered to see the links

(HTTP request) to get platform game data (games/mods only) from both URL and search by name.

As you are developing a web-based game manager you could use the package directly without having to reinvent the wheel (assuming you are working with Node JS/Javascript, otherwise feel free to look at the repository and see how it works).

I finally got to where your code is very helpful. I was trying to figure out how to authenticate to this site programmatically and you had the answer! I just need to convert your JS code to C#. Should be simple enough!

 

[bakerdenn]

I have made my new repo public for this project. At this point, there is only code in the repo. When I get closer to my launch, I will add a read me and whatnot.

You must be registered to see the links

 

[bakerdenn]

Hi bakerdenn!

Maybe I can help you out.

To make my life a little easier in managing the updates of the various games I have on the PC I have developed a game manager in Electron (I come from C# but I wanted to experiment with something new).

I found myself in the same situation as you: given the lack of APIs so I started building them from scratch and developed

You must be registered to see the links

.

These APIs are fast (<3 sec per game) and use

You must be registered to see the links

(HTTP request) to get platform game data (games/mods only) from both URL and search by name.

As you are developing a web-based game manager you could use the package directly without having to reinvent the wheel (assuming you are working with Node JS/Javascript, otherwise feel free to look at the repository and see how it works).

Did you run into issues getting 401 Bad Request status messages after you authenticated to this site when you wrote your code? If so, how did you correct them? The HTML that is coming back states that my browser must accept cookies, but I can see the cookies in my C# cookie container and everything appears to be working as expected. Sadly, there is zero support on this so any guidance you may have is greatly appreciated.

 

[MillenniumEarl]

Did you run into issues getting 401 Bad Request status messages after you authenticated to this site when you wrote your code? If so, how did you correct them? The HTML that is coming back states that my browser must accept cookies, but I can see the cookies in my C# cookie container and everything appears to be working as expected. Sadly, there is zero support on this so any guidance you may have is greatly appreciated.

Unfortunately I also had some problems about cookies, honestly I don't remember ever had a 401.

Does this happen immediately after login? Are you looking to re-use previously obtained cookies or is it a new session?

Also it could be a problem of user agent, try to see if setting it differently change something.

 

[bakerdenn]

I apologize for my delayed response. Apparently, I mistyped my response code. I'm getting a 400, not a 401 (that happens when you are working on too many projects at once). Either way, it isn't working correctly. I'm attaching a bit more context to this message in the hope that someone can help spot my error.

Based on the code in MillenniumEarl's repo, I'm posting the following string to {https://f95zone.to/login/login} (obviously I'm replacing the un/pw with my credentials).

login=*****&url=&password=********&password_confirm=&additional_security=&_xfRedirect=https%3A%2F%2Ff95zone.to%2F&website_code=&_xfToken=1615668870%2C8345322ebab5401ff7b9f46c7b214f80

The token value is found on the website prior to the post by scrapping the following value where HTML represents the login page:

token = html.DocumentNode.Descendants("input")
.Where(i => i.Attributes.Where(a => a.Value == "_xfToken").Any())
.FirstOrDefault()
.Attributes
.Where(a => a.Name == "value")
.FirstOrDefault()
.Value;

The response is
{StatusCode: 400, ReasonPhrase: 'Bad Request', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
Date: Sat, 13 Mar 2021 21:06:33 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=dfaa49fb5cf95ccecc3ab4f8ab827e1bd1615669592; expires=Mon, 12-Apr-21 21:06:32 GMT; path=/; domain=.f95zone.to; HttpOnly; SameSite=Lax; Secure
Set-Cookie: xf_csrf=8iRfSg9BbekdHuUl; path=/; secure
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Cache-Control: no-cache, max-age=0, private
Vary: Accept-Encoding
CF-Cache-Status: DYNAMIC
cf-request-id: 08cf029a350000f3453531e000000001
Expect-CT: max-age=604800, report-uri="

You must be registered to see the links

"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=fI%2BgHU16hO5MTRVIJlE29OSOtOfwlowIzTibNWHHIJTjwqu0AEKHXsULVOKVv6euBWfmkoIB4YDB1uUob%2FgaiaEmkfpDUoYbuz%2FC"}],"max_age":604800,"group":"cf-nel"}
NEL: {"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 62f83a09ee6af345-ATL
Alt-Svc: h3-27=":443"
Alt-Svc: h3-28=":443"
Alt-Svc: h3-29=":443"
Content-Type: text/html; charset=utf-8
Last-Modified: Sat, 13 Mar 2021 21:06:33 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
}}

The body of the HTML states that my browser doesn't accept cookies. However, two cookies are in the CookieContainer after the request that corresponds to the two set-cookie headers in the response.

 

[bakerdenn]

I just realized after posting that I never answered Earl's questions. This happens after I post the login request. If I try to use the cookies that I have, I'm told that I haven't authenticated. That is confirmed when I look at the opening HTML tag in the site:

<html id="XF" lang="en-US" dir="LTR" style="font-size: 62.5%;" data-app="public" data-template="thread_view" data-container-key="node-44" data-content-key="thread-65532" data-logged-in="true" data-cookie-prefix="xf_" data-csrf="1615668957,e917bb90190e073eb9000a4215e3356f" class="has-js template-thread_view uix_page--fixed uix_hasSectionLinks uix_hasCrumbs uix_hasPageAction has-no-touchevents has-passiveeventlisteners has-no-hiddenscroll has-pointer-nav">

That snippet is from my current state as I type this. You can see that I'm clearly logged in. When I try that from my code, the data-logged-in is false.

As far as my user agent, I have tried the agent that you have in your code and copying and pasting the one that my browser posted the form with and got the same result either way.

 

[bakerdenn]

I finally figured this out. Apparently, I wasn't capturing the cookies from my original request to get the token value. As a result of this, the cookie wasn't being sent with the subsequent POST request, prompting the no cookies message and the 400 return code.

Now, to do awesome things with this!

 

[bakerdenn]

So, I have now reached the point where I need to begin focusing on designing UX/UI for this application (admittedly my weakest skill in web development). With that in mind, I'm turning to my potential users for ideas about what people really like, while trying to keep my personal preferences out of the equation. Do you have and favorite a website from a design/usability perspective (the content of the site doesn't particularly matter here)? Do you prefer dark or light themes? How important is the ability to choose between the two? Is there some trend in UI design that drives you crazy and you hope to never see again (the site I'm posting this on is one of them for me. Love the content, hate the UI. Maybe that's why I'm building this )?

Please, send me some links to good, bad, and ugly sites, along with a note as to why you are sending it to me, that I can digest for my design. I truly look forward to any feedback you guys may have.

I hope to get a mockup/prototype done in Figma fairly soon that I can share before I begin coding.

On an unrelated, but, I feel, fairly impressive note, I populated ~8,400 games into my dev database from F95 in under 5 minutes. I guess my new scrapper is working pretty well

 

[bakerdenn]

I'm placing this on hold for the time being as I have other priorities. I am working on a temporary solution that will be a text-based interface run from the Windows command line to manage games. Frankly, I want something to use and I'm somewhat proud of the sync engine that I wrote.

As I mentioned in the first post, I'm using this project to play with new things and Microsoft is about to release MAUI, which I will build this with! Sorry to anyone who was looking forward to it, but it will come eventually.

 

[bakerdenn]

So, as I promised back in April, my text-based manager is almost ready!

I am putting some finishing touches on the documentation so I can publish a release on

You must be registered to see the links

. My hope is that I will finish it up sometime this weekend! I will post again when it is ready!