Yeah that's always been the standard for models around 30b parameters sadly...
Once it's loaded it's super quick though
[AI] Uncensored text generation via Oobabooga
- Thread starter Deleted member 1121028
- Start date
Once it's loaded it's super quick though
- Nov 6, 2018
- 75
- 73
Im using oobabooga with SillyTavern on a
Ryzen 9 7900x
64GB DDR5-6000 Ram
Radeon RX 7900 XTX
On Ubuntu 22.04 with Rocm
Its a pain in the ass to install it on linux and get it to work if you dont have a Nvidia GPU. took me multiple try's cause of dependencies and wrong versions of dependencies.
Also i would not recommend to use it in CPU Mode cause it will cause a very high CPU Load.
but by now it runs very smooth
thats not bad for a local hosted AI
LLM Used:
And yes this LLM can roleplay and sexting
Output generated in 8.72 seconds (54.00 tokens/s, 471 tokens, context 1147, seed 282177610)
Ryzen 9 7900x
64GB DDR5-6000 Ram
Radeon RX 7900 XTX
On Ubuntu 22.04 with Rocm
Its a pain in the ass to install it on linux and get it to work if you dont have a Nvidia GPU. took me multiple try's cause of dependencies and wrong versions of dependencies.
Also i would not recommend to use it in CPU Mode cause it will cause a very high CPU Load.
but by now it runs very smooth
You don't have permission to view the spoiler content.
Log in or register now.
Code:
Output generated in 9.21 seconds (55.46 tokens/s, 511 tokens, context 71, seed 646386357)LLM Used:
You must be registered to see the links
And yes this LLM can roleplay and sexting
You don't have permission to view the spoiler content.
Log in or register now.
Last edited:
Hello, I have tried wizard vicuna 30b uncensored, mixtral-8x7b-instruct-v0.1, lzlv_70b_fp16, nous-hermes-llama2-70b, xwin lm 70b, synthIA 70b 1.5 and other minor models but how to use them correctly, which will be the best for rp nfsw, the 70b become very insistent and repeat for example: "you have to maintain respect..." at the end of each answer, how to remove censorship without going to the extreme in an rp? And how to prevent them from repeating themselves too much? I have already set repetition_penalty to maximum but as the conversation progresses, they break down. I use oobabooga
- Jul 4, 2017
- 2,588
- 4,719
From my understanding of this, the issue lies in the fact that over time the context window gets filled only with your chat, pushing the original "scenario prompt" out of context and leaving you with only the most recent chat messages. The "system prompt" is always prepended but that's typically just "respond to the input messages" stuff.
If you use a client like SillyTavern, it has various configurable prompt injections to ensure you have every context prefixed/postfixed/in-between-fixed with your chosen prompt strings. This very much helps to keep the LLM on track for the "bigger picture" while still reacting to the immediate state (the most recent 5-10 messages) of your "roleplay session". The config for these "Scenario prompts" are part of the selected character beyond the "description/first message" parts of the Card - look under the the 'advanced definitions' button and you'll see additional promps like "Personality Summary", "Scenario", etc.
To make the RP sesison even better, to keep the "medium term" state, you can and should constantly update the "Authors note" for the chat to contain the information needed i.e. (clothing state of the USER and CHAR characters, position in the room)
If you use a client like SillyTavern, it has various configurable prompt injections to ensure you have every context prefixed/postfixed/in-between-fixed with your chosen prompt strings. This very much helps to keep the LLM on track for the "bigger picture" while still reacting to the immediate state (the most recent 5-10 messages) of your "roleplay session". The config for these "Scenario prompts" are part of the selected character beyond the "description/first message" parts of the Card - look under the the 'advanced definitions' button and you'll see additional promps like "Personality Summary", "Scenario", etc.
To make the RP sesison even better, to keep the "medium term" state, you can and should constantly update the "Authors note" for the chat to contain the information needed i.e. (clothing state of the USER and CHAR characters, position in the room)
- Nov 12, 2017
- 429
- 3,492
Been having lots of fun with the Playground extension and TheBloke_MythoMax-L2-Kimiko-v2-13B-GPTQ.. I'm genning in short bursts of 2-3 paragraphs then using the '---' instruct syntax to guide the story. The Summary and PermaMemory helps a lot in maintaining some of the story's base ideas.
I do have this issue where the AI starts to leave out filler words like "to" and "the" etc, but then I just transfer some of the preceding story to dynamic memory and ensure that the permamemory or summary is updated to reflect was has transpired so far.. works like a dream.
I do have this issue where the AI starts to leave out filler words like "to" and "the" etc, but then I just transfer some of the preceding story to dynamic memory and ensure that the permamemory or summary is updated to reflect was has transpired so far.. works like a dream.
- Jul 4, 2017
- 2,588
- 4,719
Try "Twinbook" extension as well. it's a different workflow from Playground, and the documentation isn't exactly clear, but I've found it very effective for writing longer story style. You do need to continually maintain the "world state" in the permamemory but the advantage of being able to cut off context where you want is very useful.