I also have a 3090. But I do not understand how to set a different model in settings. I seem to have only one option to choose. It's called "deep cognito something or other."
Is there a step by step set of instructions anywhere for how to get this started? This entire thread appears to be written by people who already understand how to do everything in AI. I'm a total noob and I don't know where to begin.
To quote someone a couple of posts earlier, I don't know how to correctly do anything like this guy did: " Launched Kobold with my model, confirmed everything was working in the web ui, launched multiic, went to settings and picked local and looked up my model from the menu on the left, set a context value, then started the game." That's not me. I just started the game and hoped for the best.
For local AI text generation the game uses koboldcpp, which is a back-end application that runs GGUF quantized models. You can download models from huggingface.com, such as
You must be registered to see the links
You must be registered to see the links
(these are not the latest, just two I had around - you can do a little research and find others that may suit you)
The default model is 14b, which is pretty high, and Q4_K_M which is a compromise between speed and quality. The higher the "b" and "q" numbers, the more complex the model will be, potentially giving higher quality and more nuanced responses but eating up more resources.
You should copy the model (.gguf file) that you download to the resources/koboldcpp folder in your game, and you'll be able to choose a different model at game start.
This is supposed to be all you need to run the local model using the setup that the game provides. If you want fine tune things, you'll need to learn how to launch koboldcpp separately and configure it in the splash screen; then you keep it running in the background and copy the address they give you in the shell window (usually localhost) to the API address in the game's configuration.
I'm not an expert and also rely on wikis and installation instructions, if you play Skyrim and want to try LLM powered dialogue with NPCs you can check out this mod -
You must be registered to see the links
- they have detailed instructions and videos on how to launch and configure koboldcpp (and a bunch of other tools) that would hopefully give you a better idea of what you're doing.
But, again, this game works out of the box, just click on the model in that pre-menu window and then let a few seconds for the model to load, avoid starting the game while it shows "loading model" on the top left
and the game pulls you back to the story, but the chat moments still add a little flavour, I'm enjoying it.
