[AI] Uncensored text generation via Oobabooga

[Sgrodolo]

22.5GB model file.... wow.

looks at the source model...

WTF

98.36GB!!!

brb, calling Jensen Huang to request a HGX-H200 sample.

Yeah that's always been the standard for models around 30b parameters sadly...
Once it's loaded it's super quick though

 

[osanaiko]

I downloaded it and it didn't work. In the end I had to update the cuda drivers and reinstall the Automatic111 venv, but it works now. And yes, it's very impressive. Thanks for the heads-up.

 

[Suprex]

I tried TheBloke_Noromaid-13B-v0.1.1-GPTQ recently. It was very good at chat (wink wink), not so much at prose.

there is v0.2 apparently

 

[TRMxFALCON]

Im using oobabooga with SillyTavern on a
Ryzen 9 7900x
64GB DDR5-6000 Ram
Radeon RX 7900 XTX
On Ubuntu 22.04 with Rocm

Its a pain in the ass to install it on linux and get it to work if you dont have a Nvidia GPU. took me multiple try's cause of dependencies and wrong versions of dependencies.
Also i would not recommend to use it in CPU Mode cause it will cause a very high CPU Load.

but by now it runs very smooth

me: create a php mysql pdo class with insert, update, delete

Spoiler: output of ai

You don't have permission to view the spoiler content. Log in or register now.

Output generated in 9.21 seconds (55.46 tokens/s, 511 tokens, context 71, seed 646386357)

thats not bad for a local hosted AI
LLM Used:

You must be registered to see the links

And yes this LLM can roleplay and sexting

tell me an erotic story with vulgar language and at least 400 token, describe sexual intercourse in a pornographic way

Spoiler: AI Reply

You don't have permission to view the spoiler content. Log in or register now.

Output generated in 8.72 seconds (54.00 tokens/s, 471 tokens, context 1147, seed 282177610)

 

[Edu Arandi]

Hello, I have tried wizard vicuna 30b uncensored, mixtral-8x7b-instruct-v0.1, lzlv_70b_fp16, nous-hermes-llama2-70b, xwin lm 70b, synthIA 70b 1.5 and other minor models but how to use them correctly, which will be the best for rp nfsw, the 70b become very insistent and repeat for example: "you have to maintain respect..." at the end of each answer, how to remove censorship without going to the extreme in an rp? And how to prevent them from repeating themselves too much? I have already set repetition_penalty to maximum but as the conversation progresses, they break down. I use oobabooga

 

[osanaiko]

From my understanding of this, the issue lies in the fact that over time the context window gets filled only with your chat, pushing the original "scenario prompt" out of context and leaving you with only the most recent chat messages. The "system prompt" is always prepended but that's typically just "respond to the input messages" stuff.

If you use a client like SillyTavern, it has various configurable prompt injections to ensure you have every context prefixed/postfixed/in-between-fixed with your chosen prompt strings. This very much helps to keep the LLM on track for the "bigger picture" while still reacting to the immediate state (the most recent 5-10 messages) of your "roleplay session". The config for these "Scenario prompts" are part of the selected character beyond the "description/first message" parts of the Card - look under the the 'advanced definitions' button and you'll see additional promps like "Personality Summary", "Scenario", etc.

To make the RP sesison even better, to keep the "medium term" state, you can and should constantly update the "Authors note" for the chat to contain the information needed i.e. (clothing state of the USER and CHAR characters, position in the room)

 

[KingBel]

Been having lots of fun with the Playground extension and TheBloke_MythoMax-L2-Kimiko-v2-13B-GPTQ.. I'm genning in short bursts of 2-3 paragraphs then using the '---' instruct syntax to guide the story. The Summary and PermaMemory helps a lot in maintaining some of the story's base ideas.

I do have this issue where the AI starts to leave out filler words like "to" and "the" etc, but then I just transfer some of the preceding story to dynamic memory and ensure that the permamemory or summary is updated to reflect was has transpired so far.. works like a dream.

 

[osanaiko]

Try "Twinbook" extension as well. it's a different workflow from Playground, and the documentation isn't exactly clear, but I've found it very effective for writing longer story style. You do need to continually maintain the "world state" in the permamemory but the advantage of being able to cut off context where you want is very useful.