Others Taming AI Text Generator

[The Rogue Trader]

I was poking around my nose in the various AI content generators (mostly because the very idea makes me seriously uneasy), and I stumbled on this project

You must be registered to see the links

It looks fascinating, but my attempts to test it failed because "Servers are overloaded... Please retry in a few moments."

The developer anticipated this problem "maybe you don't have enough computing power to stay under the frontend's timeout delays. "
Now, my current computer works perfectly fine for all I want it to do, but I wouldn't call it "powerful" by any stretch, so yes, that's possibly the issue. Again, the dev gave us a solution:
"You might want to grab your copy of the frontend's source code to modify it:

You must be registered to see the links

. "
(In case you're wondering, yes, I have the last Python build up and running on my system, so that's not the problem)

I grabbed it, and (I think) installed it, but even just launching the front goes beyond my capabilities. I was going to dump the whole experience when it dawned on me that the project can be trained with classic porn books from the 70s/80s to make some interactive adult fiction game. There are hundreds of such books floating around the Net in txt and HTML formats. I alone have about 500 saved locally.

My problem is that even with the dev's (a bit scant) instructions, I'm completely at a loss about how it's supposed to be accomplished. I'm not a coder and have only a vague notion of what Javascript and Neural Nets are, let alone program with them.
Does somebody have clues for an absolute beginner (or even better wants to jump in)?

 

[VentureZapatist]

To run the back-end, you need to install all the dependencies first, which you can do by running pip install -r requirements after cloning the repository. You might have trouble here because of pytorch (I certainly did), so in that case I suggest following the instructions from the pytorch

You must be registered to see the links

. After successfully doing that, you can run any of the launch scripts to launch the back-end that is appropriate to your needs.

To run the front-end, you'll need the npm package manager. It comes with nodejs and you can get it from

You must be registered to see the links

. Install it and then follow the instructions from their github page.

 

[The Rogue Trader]

Thanks VentureZapatist!
Unfortunately

You might have trouble here because of pytorch

turned out to be quite an understatement.
You were right about me not having all the dependencies installed (I naively assumed that, using the github desktop application, they would get automatically installed when cloning). But even after you pointed the correct way, neither git nor pip installed pytorch correctly.
After several hours of fiddling and a substantial amount of swearing, I finally realized that the reason why I couldn't manage to force pytorch to work was that it's compatible only up to Python 3.9 and my system defaulted to 3.10 (I have both 3.9 and 3.10 installed, I'm not sure why).
Tomorrow I'll uninstall python 3.10 and see if it solves the issue.

P.S.
Weirdly, even using conda and creating an environment where pytorch was installed, activated and verified, the backend still refused to import the module 'torch', so I'm not confident about being able to get to the end of this.

 

[VentureZapatist]

After several hours of fiddling and a substantial amount of swearing, I finally realized that the reason why I couldn't manage to force pytorch to work was that it's compatible only up to Python 3.9 and my system defaulted to 3.10 (I have both 3.9 and 3.10 installed, I'm not sure why).

In my case, I was using python 3.8, which I thought was supported, but didn't work for some reason. So I switched to python 3.7 and was able to install pytorch using pip3 install torch torchvision torchaudio (this is the stable build, if it doesn't work try the LTS build from the website). The remaining dependencies installed just fine.

I couldn't test it because I didn't get the NER model. Make sure you grab that if you plan to use it.

 

[xj47]

What are you trying to accomplish exactly?
Are you focused on learning or creating a game/project?

Neural nets like GPT are quite cool but they need to be large to give good results. And their size is limited by the time it takes to train and run them. That's why projects like Ai Dungeon all use remote servers, the system is too expensive to run on local PCs.

So if your planning on training your own model and building a game around it, I think you'll be disappointed. Your model would have to be tiny compared to what the online services offer and the quality will suffer.

 

[The Rogue Trader]

What are you trying to accomplish exactly?

Ah... good question.
At the time I opened this thread I was entertaining the idea of starting some project. I soon realized that it was a goal born from ignorance: what I had in mind was well beyond what was reasonable, as you're pointing out.
At the moment I think my purpose could be better described as "stubbornly refuse to give up". Let's call it "learning", but if I manage to get the program running I hope that I'll be able to use it as a tool for my writing.

I understand, now, what you're saying about the model being doomed to be too small, so I'll content myself with just experimenting and understanding and maybe getting a tool able to give me some pointers when I'm stuck by writer's block, using the original project's NER.
Well, maybe in a few years here in F95 we'll be able to run a larger project able to support a game like Storium is trying to do with non-adult narrative games.
So what now is the tiny step of a man could turn into a giant leap for... well for porn

 

[xj47]

Righto
I went on a somewhat similar journey myself a year ago or so after learning about AI Dungeon

For running something on your own computer, I'd recommend looking at the GPT-2 repo on github. There are a lot of forks of the original, one that I found myself using was this one:

You must be registered to see the links

Try to get their smallest model (117M) to work on your PC. It's small enough that you can train & modify it without it taking ages.

For understanding the theory behind the model, first read up on machine learning in general a bit.
To start with I'd recommend learning about "Perceptrons", they are about as simple as ML gets and help you understand the underlying principles.
To give my own summary, ML is roughly about setting up a system (model) for crunching some input values to produce an output. Basically a massive math equation with a ton of unknown constants. The idea is that if all constants were set correctly, then your system would give you the results you want (perform perfect language translation for example). The process of trying to set the constants correctly is called "training". A technique called backwards propagation can be used for most models that works by comparing the currently computed result with the correct result and nudging the constants slightly in the right direction.
Taking a step back, when designing/understanding a model you just concern yourself with a process that *should* work with the right values, even if you have no idea what those values are. The training stage will (hopefully) find the correct values.

Once you have the basics down, read about neural networks and maybe some of the easier use-cases for ML. One example that ML people love to death is handwriting recognition. A lot has been written about the topic so it's a good example to learn from.
Moving on to text-stuff (ie. Natural Language Processing aka NLP), GPT uses a transformer architecture. I found this video that explains it:

You must be registered to see the links

While I think the video explains it well, it is a really complicated thing to cover in 10 minutes. Probably need to watch it multiple times or take a break to read up on a sub-component.

My own high-level explanation of how this works is:
This stuff uses a encoder-decoder model. The purpose of an "encoder" is to take an input and try to break it down into something that is more easily processed by the computer. Say the system is processing the sentence "Bob slaps Ben". Ideally you'd want to convert that into a fixed-size set of numbers that perfectly captures the meaning of the sentence and where minor changes to the encoded data translate to minor changes in the meaning. So if we fed in "Bob punches Ben" then the resulting encoding should be numerically similar. It's kind of like the computer digesting the input and turning it into a soup of meaning it can use directly. This abstract-encoding/soup is then used to perform the operation we actually care about, like predicting the next word in a sentence. This operation is easier to perform correctly since the data is in a form that the computer finds easy to use. However the output will also be a in highly abstract form. This is where the "decoder" comes in. Once we have performed the operations, we use a decoder to convert the abstract representation back into a form that humans can understand.

Anyways, take my own explanations with a grain of salt. I took a few courses on ML in university and read up on it a bit myself, but I'm hardly an expert on the topic.

 

[The Rogue Trader]

Wow @xj47, that's a lot of info, much more than I expected to find on a thread on F95.
Thanks! I'll delve deeper into the sources you gave me, but I'm afraid much will fly straight over my head, since my understanding of programming goes barely beyond the IF-THEN and I needed the better part of a week just to install pytorch.

GPT-2 was the base used for the project I was trying to run (they fine-tuned it on fiction, that was what caught my eye), so I hope that going straight to the source like you say I'll be able to find better documentation about how it works and how to make it work.

 

[The Rogue Trader]

In my case, I was using python 3.8, which I thought was supported, but didn't work for some reason. So I switched to python 3.7 and was able to install pytorch using pip3 install torch torchvision torchaudio (this is the stable build, if it doesn't work try the LTS build from the website). The remaining dependencies installed just fine.

I couldn't test it because I didn't get the NER model. Make sure you grab that if you plan to use it.

In the end, I uninstalled both Python 3.10 (that didn't go down without putting up a fight) and 3.9 and found a 3.8 installation on my computer, and that one did the trick: I installed torch, I installer transforms, I installed torch-transforms (at that point I was seriously believing I was being bullshitted) and I ran the backend, that this time gave the message "serving on port 7777".
I already had the NER, so I returned on the website and, as I expected, I got the same "server overloaded" message.

So I'm back to the problem of post 1: it should be solvable by running the frontend locally, but that's where my understanding stops.

 

[VentureZapatist]

I already had the NER, so I returned on the website and, as I expected, I got the same "server overloaded" message.

You should be able to use their website by selecting the checkbox that says "Run on local server?", so it connects to your back-end. Was there any error message on the console (where the "serving on port 7777" message is shown)?


In my case, the front-end from their website was able to connect to the local server, but there is an error when loading the model (because I don't have it). I got the server overload message just the same, though. So maybe it's just a generic way of saying that something went wrong.

So I'm back to the problem of post 1: it should be solvable by running the frontend locally, but that's where my understanding stops.

You could try, that. If anything, you'll be a bit less dependent on external services. It should also not be as painful as installing pytorch. Famous last words.

 

[The Rogue Trader]

You should be able to use their website by selecting the checkbox that says "Run on local server?"

Yep, that went without saying. The allinone launcher was way less verbose with me than it had been with you. I thought it was a sign that everything was running correctly but I'm no longer too sure.



I also agree that the "server overloaded" is most likely to be a generic timeout message.

I didn't go further last week because I had some deadlines to meet with the game I'm collaborating to, but I don't even understand whether the front-end is a local program or I've to install a local webserver for it.