Request Tool KoboldAI + Erebus Model for text-based adventure

[smirk]

Does anyone have any experience using

You must be registered to see the links

and the

You must be registered to see the links

model locally to create text-based lewd adventure games? Planning to try to set it up to run in GPU mode (on an RTX 3060, hopefully it'll do) but wondering if anyone's tried it and has any experience with it's limitations, performance and such. The model is ~64GB and I don't have a super connection so would be good to know if it's worth it, and if so, what works well and what might not work so well.

 

[smirk]

So I downloaded one of the smaller Erebus models and ran some tests. And it works surprisingly well, at least in Novel mode. Adventure mode is a bit hit and miss. Even without defining a very detailed prompt it generates nicely coherent stories for the most part, picks up on hints and goes along with it etc. I can imagine this really shining if it's set up with a well defined world.

Some tweaking is necessary unless you have ungodly amounts of vram, and it's not super quick. But I haven't tried tweaking a lot of parameters so YMMV. Kobold lets you export your world information and story in json format, so I can see this being great for sharing.

Curious to know if anyone else has tried this.

 

[Virt-A-MILF]

Yeah Erebus is pretty good for it's limitations, even the 6B is all I could run before switching to NovelAI finally. Does your GPU have 8GB VRAM? Mine has 11GB... So you're going to be limited and have to share threads with your CPU... It will be pretty slow to generate... Even then you might need to refresh the dialogue to keep it spicy or understand which person is which sometimes, especially with different genders being mixed up for your character, even with the 20B version...

Your other option is to use Google colab version which I did for awhile, the pro version is $10. Free is pretty shitty...

You must be registered to see the links

Nerys worked well for me adventuring and sex surprisingly, while Erebus is more more smutty and seems to work better for my mommy kink.

I use Novel AI now, surprisingly it's pretty good with smut! I love the Hypebot companion of which you can actually image generate an avatar for and add it into the CSS editor. Hopefully soon we can make our characters using custom bot avatars and image generation for change of background scenes!

Since you're speed is limited I'd say go with the online version if you aren't afraid of uploading smutty degenerate dialogue like I am. I'm actually buying two K80s and rigging a custom old server build PC to solely run the models threaded at home on my linux build with 48GB VRAM at my disposal!

 

[Sir.Fred]

I'm a total AI noob but for the last few days I've been experimenting with KoboldAI running Erebus 13B on a 24GB 3090 with wildly varying levels of success.

I have a beefy CPU too so I split the layers between GPU and CPU - I even had some very limited success with the 20B model but tinkering with settings I don't yet understand broke things to the point where I completely reinstalled to take more careful note of defaults.

The 6.7B parameter variant is quick and stable - seemingly with little sensitivity towards settings or demand - but the output quality difference between 6.7B and 13B has, admittedly totally subjectively and unscientifically, been noticeable.

Erebus on KoboldAI definitely seems like the way forward for local story generation at the moment but it's still hugely unapproachable even for the fairly technical among us.

I don't want to put anyone off - the actual download and installation process has actually been made very slick and friendly - great work by the project maintainers.

But it's once you're up and running that things rapidly become more under-documented and opaque.

I would, for example, really appreciate a decent guide/tutorial on what settings (with higher end hardware in mind). All the talk of generating soft prompts seems to assume a great deal of existing domain knowledge and there are precious few examples of prompts, settings or soft prompts out there that seem to be suitable.

Most of the stuff that is out there seems to assume that you're using NovelAI, Kobold on Colab or perhaps even something from the legendary, seemingly mythical era of an AI Dungeon golden age. Some of us missed that boat entirely.

So if you do have any tips on settings for Erebus 13B or indeed where to find soft prompts or more helpful soft prompt creation guides, please do share; there's a real drought of compatible fresh advice - particularly about the settings.

As an aside, InvokeAI is a not dissimilar image generation rabbit hole I stumbled down while researching this topic. Funnily enough, the image generation tool seems much less resource hungry than the writing tool.

I fear I have scratched the surface of something way bigger than I can handle. So for now, noob tips on WTF to do with the Repetition sampler in my sampler order, would be great. As an example.

In my most recent prompt I mistyped widow as window. It got weird real quick.

 

[smirk]

Your other option is to use Google colab version which I did for awhile, the pro version is $10. Free is pretty shitty...

You must be registered to see the links

Nerys worked well for me adventuring and sex surprisingly, while Erebus is more more smutty and seems to work better for my mommy kink.

I use Novel AI now, surprisingly it's pretty good with smut! I love the Hypebot companion of which you can actually image generate an avatar for and add it into the CSS editor. Hopefully soon we can make our characters using custom bot avatars and image generation for change of background scenes!

The reason I wanted to test KoboldAI in the first place was that I wanted to avoid any solutions hosted or managed by 3rd parties. I especially don't feel like sharing my smut with Google/Alphabet/the New World Order or whatever they go by these days - but aside from that, I also want to avoid another AI Dungeons situation. From the experiments I've done so far I am convinced that with a little tweaking and guidance this can become a viable solution for interactive stories or adventures/games.

Erebus on KoboldAI definitely seems like the way forward for local story generation at the moment but it's still hugely unapproachable even for the fairly technical among us.

Yeah I have a bachelors degree in this crap and still haven't figured out all the parameters I love how customizable it is but realize that this will be a no-go for people just looking to have a good time with a raunchy story. Another issue is the hardware requirements and the size of the model, most people would probably not want to go there for a text-based game. The lack of any usable documentation doesn't help.

So if you do have any tips on settings for Erebus 13B or indeed where to find soft prompts or more helpful soft prompt creation guides, please do share; there's a real drought of compatible fresh advice - particularly about the settings.

Been looking for this myself. I've found some references to

You must be registered to see the links

, but haven't tried it. Apparently it lets you import/export prompts and world data from a variety of formats, so if you find something usable for another platform you might be able to adapt it to Kobold.

 

[Skalli]

I've played around with it too. My success varies. Sometimes it comes up with great ideas and moves the stories in fun directions that were unexpected. Sometimes it suddenly goes with "and then he walked away, got to bed and slept... Next day: ...", so then it's retry time.
My biggest gripe is, that the consistency of characters is sometimes very bad. Body attributes and clothing states are all over the place.
So one minute a character is naked, the next they take of their blouse... Often I just fix the small mistakes in Edit Mode, but other times it's so far off that Retry is the only way to go.
I try to use the Memory and World Info more, but it's also not always successful.
If one has specific directions in mind, it can be difficult to get the AI in that way. E.g. I love pregnancy content, but the AI seems to have a very strong bias against it. Even when flooding Memory, Genres, Notes and World Info with it, the characters always ensure that it's safe because they take the pill. What a bummer.
Playing with heat has varied results, I also tried different models, over all, I'm very excited what the future will bring.
I'd love a chat mode with more actions and story, but haven't figured out how to do that yet.

 

[smirk]

I've noticed some oddities as well; sometimes it looks like it just reaches and impasse and keeps repeating the same things over and over again. The inconsistencies in attributes, relationships etc. can be fixed in world data, I've noticed that it does a pretty good job of keeping track of things that are spelled out for it in there.

I'd love a chat mode with more actions and story, but haven't figured out how to do that yet.

The Erebus model is tuned specifically for novels so I don't think any amount of tweaking will give very good results with Adventure or Chat mode. Will probably have to re-train it, but who knows, I haven't really tried.

 

[Skalli]

The Erebus model is tuned specifically for novels so I don't think any amount of tweaking will give very good results with Adventure or Chat mode. Will probably have to re-train it, but who knows, I haven't really tried.

Mostly with Erebus and Shinen 13B, I haven't noticed much difference when it comes to these points, no matter what mode, Novel, Adventure, Chat. They all worked mostly fine, but the inconsistencies stayed.
How do you put the info in the world data?

 

[smirk]

How do you put the info in the world data?

Hit the "W.Data" button above the input box. It isn't terribly well explained but there are some hints in the mouse-over tooltips in the interface. It's essentially a key-value store where the keys are the names of entities used in the story, and the value is a natural language description of the thing you're describing. For instance, you can create an entity with the key "Bob" and a value of "Bob is a 185 cm tall man. He's 24 years old. Bob has brown hair and brown eyes. He has a huge dick." and any references to Bob in your story should apply these attributes to it. I say "should" as it has mostly been the case for me, but there have been episodes where it has glitched out.

You can use this to describe anything mentioned in the story - people, locations, objects etc. I've used it to describe the layout of a house for instance, and the AI mostly got it right when describing characters moving around.

 

[Skalli]

Yeah, it's how I used it then. Mostly it's fine, but still gets things wrong. I tried different formats, but couldn't spot much difference between them.

 

[Sir.Fred]

You must be registered to see the links

has an interesting interface for characters, relationships and other world info. It's not local so it'll not replace Kobold but there are lessons to be learned from the UI.

 

[Virt-A-MILF]

Novel AI has a lorebook with location,people,object references to coax the AI to head in the direction I want it. I can also use the lorebook as a wiki or bestiary and item index! It has a generator for people,monsters,location,faction,general ect... I really do use it to help me with world building. Most importantly it gives you the ability EASILY UI wise alter phrase bias and activation selection... It will definitely follow through and it really works amazingly well..! These things are the primary reason I use it.

I really hope Kobold tries to add these features soon, soft prompts seem troublesome and time consuming to make.
But for the more depraved and spicy content I rather use Kobold...

 

[smirk]

Yeah I'm hoping Kobold will evolve to achieve some feature parity with these hosted solutions. Novel AI, Dreamily, AI Dungeons all have a lot going for them but I won't be using them for a number of reasons.

1. Cost. I get it, running an advanced AI takes a lot of resources but I'm not willing to pay a monthly subscription.
2. Privacy. This is not the sort of content I am comfortable sharing with a 3rd party, and combined with #1 means that it will be linked to my real identity (payment information). These sorts of services get compromised all the time, and that is one info dump I would not love to see on some dark web marketplace.
3. Control. As AI Dungeons have taught us, the terms regarding what content is allowed on these services can change quickly, sometimes due to decisions by the services themselves, sometimes due to pressure from 3rd parties like payment providers. I want something that works consistently for creating the content I want and won't be subject to the whims of PayPal, Patreon, VISA/MasterCard or anyone else.

 

[Lanerun]

Really interesting to hear about these things! Shame you need a powerful GPU to run them. I'll be buying a 4090/4080 soon, preferably the cheaper one since the difference here is about $700. Do we think the 4080s 16gb limitations is going to affect AI very heavily?

 

[Sir.Fred]

Really interesting to hear about these things! Shame you need a powerful GPU to run them. I'll be buying a 4090/4080 soon, preferably the cheaper one since the difference here is about $700. Do we think the 4080s 16gb limitations is going to affect AI very heavily?

VRAM is all that matters. I mean, the number of CUDA cores counts of course, but without enough VRAM you'll not be using all those cores properly anyway. FWIW I wish my 3090 had more than 24GB tbh.

 

[smirk]

VRAM is all that matters. I mean, the number of CUDA cores counts of course, but without enough VRAM you'll not be using all those cores properly anyway. FWIW I wish my 3090 had more than 24GB tbh.

Yeah I'll second this - VRAM is most definitely the bottleneck for sure. Once you have to resort to disk caching it slows to a crawl real quick if it runs at all.

 

[smirk]

Stumbled across another one -

You must be registered to see the links

. Looks like sort of a hybrid between AI Dungeons and KoboldAI, in that it gives you a text adventure framework to work with, but you run the model locally like in KoboldAI. It looks like it may have some online integrations too, haven't played around with it that much so can't tell if you can hook it up to the Kobold Horde or other services. First impressions are not bad, tested running it with GPT2-XL and it works pretty well. Some nonsensical stuff, but you get to edit the AI response somewhat before it's committed. Oh yeah, and no filter for lewd stuff

You need to pay for it but there is a free demo on steam. Also looks like it's on sale atm. so you can pick it up for around $6 if you want to. I wouldn't expect too much as it is maintained by a single developer and F95 knows all too well how many ambitious single-dev projects actually make it beyond v.0.1.1

 

[fass]

This stuff looks amazing but I'd say I like the look of the local run stuff the best. I don't suppose I mind if it is slow to reply or generate story text. It's better than nothing in that sense. At least with the local version you don't need to worry about bullshit censors being brought in and lobotomizing your characters. Unless we had some explicit "DegenerateAI" it's bound to end up being censored at some point and so not attractive in the long term to invest in, that's ignoring the privacy risks mentioned above though, which are legitimate.

Stumbled across another one -

You must be registered to see the links

. Looks like sort of a hybrid between AI Dungeons and KoboldAI, in that it gives you a text adventure framework to work with, but you run the model locally like in KoboldAI. It looks like it may have some online integrations too, haven't played around with it that much so can't tell if you can hook it up to the Kobold Horde or other services. First impressions are not bad, tested running it with GPT2-XL and it works pretty well. Some nonsensical stuff, but you get to edit the AI response somewhat before it's committed. Oh yeah, and no filter for lewd stuff

You need to pay for it but there is a free demo on steam. Also looks like it's on sale atm. so you can pick it up for around $6 if you want to. I wouldn't expect too much as it is maintained by a single developer and F95 knows all too well how many ambitious single-dev projects actually make it beyond v.0.1.1

It's 50/50 if you get one update every six months at best or if they update randomly but frequently for a while, get a big following and then just ghost the project with people still paypigging for months and even years after the silence, always assuming the best, like some kind of battered wife syndrome. Unless you get that rare unicorn project that actually works out...here's hoping. It's hard not to get cynical after seeing the same cycles so many times. The AI scene looks like it's really the big frontier for developers to grab at. Seeing as the big ones are afraid of lewd content it's a good niche to aim for for smaller ones.

 

[fass]

I tried NovelAI. It was pretty good for lewd. A little generic and lacking in detail and dialogue though and a tendency to forget details and get lost quite quickly or with along story it would loop in a confused manner back to a near clone of the start if it went on a long while. Short sentences with each addition too, which does limit story creating or consistency, I suppose. When Novel AI wants to work though sometimes it can do some really good shit and surprise me. Kobold looks interesting but I think it's going to need to be streamlined or I'll have to do a special educational course on it before I can get so much as a sentence to get off to.

Tried Dreamily too. Seemed demented. Had my character being covered in snot and all sorts of other weird shit going on except the lewd that I kept trying to direct it to. At one point when it looked like it was almost about to get to something lewd it then had my character go into some memory of college or something and back off the lewd topic. I guess I am missing something or it doesn't like to make lewd.

Tried the demo of AIdventure and it was surreal to the point of being funny. Nothing really made sense. Maybe playing with the settings would help but it was so random it's so far only good for accidental humor. I will give it a chance though and see if I can try out different models and how to get it to output longer lengths of text and more in line with what I want. There's going to be a learning curve for sure to get this working.

 

[smirk]

This stuff looks amazing but I'd say I like the look of the local run stuff the best. I don't suppose I mind if it is slow to reply or generate story text. It's better than nothing in that sense. At least with the local version you don't need to worry about bullshit censors being brought in and lobotomizing your characters. Unless we had some explicit "DegenerateAI" it's bound to end up being censored at some point and so not attractive in the long term to invest in, that's ignoring the privacy risks mentioned above though, which are legitimate.

Yeah my thoughts exactly, nothing I hate more than being at some company/dev's mercy as to whether I can play the game I want or not. As for speed when running locally, if you have enough VRAM to fit the model you should get as good (or better) performance as any of the commercial offerings. AIDventure runs just as well as AI Dungeons on my rig. Granted, AI Dungeons are probably running a larger, more complex model than me, the more layers you need to put on the CPU the worse it gets.

When it comes to how janky and random the results get, I've found that a lot comes down to picking the right model and settings for what you're trying to do, as well as writing good world-info, genre and memory. There are some tips on the KoboldAI wiki but a lot of it is guesswork or checking what others have done on places like aetherroom. Some models only work well for stories, some are made especially for adventure games etc. If you're going for a lewd adventure game, pick something like "Nerys FSD". KoboldAI's default settings are mostly sane, but I find that responses can be a little too random. Try lowering temperature and tweaking or changing sampling methods. If you have a comprehensive world-info section written and the AI ignores it, try turning on "Dynamic WI scan." It might eat up some extra VRAM but it will be more consistent.