Others Multiic [v0.4.81] [Stvoler]

[Jinkxy]

Having alot of problems with that new koboldcpp local model in this version.
Responses take alot of time ( around 30s ) and alot of the times instead of a response i get a bullet list of character traits and suggestions what my answer should look like.

Switched back to the Gemma-2-Ataraxy-v4d-9B.i1-Q4_K_M.gguf from the previous local game version and it worked fine.

 

[francisdrake]

Is this game set up for Vulkan/AMD on Linux? Tried doing local kobold instead of web tunneling to try and get faster responses but the log is all messed up and it just loads forever. Maybe I did the setup wrong but I Launched Kobold with my model, confirmed everything was working in the web ui, launched multiic, went to settings and picked local and looked up my model from the menu on the left, set a context value, then started the game. Then when I get to the bedroom description the log looks like this, tries to run CPU only with 4k context instead of using my GPU.

 

[reidanota]

Local AI is working on my end, with the provided model. Sometimes the AI chat seems awkward if you go a little off-topic or into double-entendre and the game pulls you back to the story, but the chat moments still add a little flavour, I'm enjoying it.

 

[Ogre]

Game uses a 14B model, I'm not sure this is it but my RTX 3090 sometimes chokes on 12B models or even 7B when the prompt gets too long. Don't now how this game handles context and prompting but when I last played, a good few months back, it worked for a while but after a few interactions the llm stopped responding. I'm downloading now to check it, maybe you can set a different model in settings and use something lighter, sacrificing complexity in responses to a faster and perhaps more streamlined experience?

I also have a 3090. But I do not understand how to set a different model in settings. I seem to have only one option to choose. It's called "deep cognito something or other."

Is there a step by step set of instructions anywhere for how to get this started? This entire thread appears to be written by people who already understand how to do everything in AI. I'm a total noob and I don't know where to begin.

To quote someone a couple of posts earlier, I don't know how to correctly do anything like this guy did: " Launched Kobold with my model, confirmed everything was working in the web ui, launched multiic, went to settings and picked local and looked up my model from the menu on the left, set a context value, then started the game." That's not me. I just started the game and hoped for the best.

 

[reidanota]

I also have a 3090. But I do not understand how to set a different model in settings. I seem to have only one option to choose. It's called "deep cognito something or other."

Is there a step by step set of instructions anywhere for how to get this started? This entire thread appears to be written by people who already understand how to do everything in AI. I'm a total noob and I don't know where to begin.

To quote someone a couple of posts earlier, I don't know how to correctly do anything like this guy did: " Launched Kobold with my model, confirmed everything was working in the web ui, launched multiic, went to settings and picked local and looked up my model from the menu on the left, set a context value, then started the game." That's not me. I just started the game and hoped for the best.

For local AI text generation the game uses koboldcpp, which is a back-end application that runs GGUF quantized models. You can download models from huggingface.com, such as

You must be registered to see the links

You must be registered to see the links

(these are not the latest, just two I had around - you can do a little research and find others that may suit you)
The default model is 14b, which is pretty high, and Q4_K_M which is a compromise between speed and quality. The higher the "b" and "q" numbers, the more complex the model will be, potentially giving higher quality and more nuanced responses but eating up more resources.

You should copy the model (.gguf file) that you download to the resources/koboldcpp folder in your game, and you'll be able to choose a different model at game start.



This is supposed to be all you need to run the local model using the setup that the game provides. If you want fine tune things, you'll need to learn how to launch koboldcpp separately and configure it in the splash screen; then you keep it running in the background and copy the address they give you in the shell window (usually localhost) to the API address in the game's configuration.

I'm not an expert and also rely on wikis and installation instructions, if you play Skyrim and want to try LLM powered dialogue with NPCs you can check out this mod -

You must be registered to see the links

- they have detailed instructions and videos on how to launch and configure koboldcpp (and a bunch of other tools) that would hopefully give you a better idea of what you're doing.

But, again, this game works out of the box, just click on the model in that pre-menu window and then let a few seconds for the model to load, avoid starting the game while it shows "loading model" on the top left

 

[Ogre]

Thank you! I will try to comprehend and apply this guidance.

 

[Jinkxy]

Thanks for that reidanota, been trying alot but i cant get it to run with my own model. When i start koboldcpp on my own it instantly closes the cmd window as soon as i open multiic. And the multiic cmd window shows that he is always looking for his own koboldcpp.exe in his game path doesnt matter what i choose in the settings.

I can use different models by copying them into the game folder like you said but whatever i try i cant get it to run with my own koboldcpp at localhost:5001.

 

[reidanota]

Thanks for that reidanota, been trying alot but i cant get it to run with my own model. When i start koboldcpp on my own it instantly closes the cmd window as soon as i open multiic. And the multiic cmd window shows that he is always looking for his own koboldcpp.exe in his game path doesnt matter what i choose in the settings.

I can use different models by copying them into the game folder like you said but whatever i try i cant get it to run with my own koboldcpp at localhost:5001.

I was able to make it work using remote tunnel


I opened multiic first, chose "use models via api" on the splash screen. Only then I loaded koboldcpp, chose the model and ticked "remote tunnel"

Why I tried that? No idea whatsoever, just think I saw that written womewhere. The console window gives you the address



I'm sure there's missing information here, I don't really know what I'm doing but the game seemed to interact with the LLM, but miss some functionality. For example, while using the local model from the game menu, the game's progression took over from the AI chat, often interrupting the conversation but keeping the story flowing. With a remote API configured like this, it felt like you had no limitations in your chat interaction, but also there wasn't an obvious way to end the conversation and move on with the story. The responses also made much less sense.

Perhaps someone can explain why, and give us better instructions. For now, using koboldcpp locally from the game works best for me overall

 

[Jinkxy]

Thanks, the tunnel was what made it work for me too.
But after trying it i will continue using the in game setting for now. Using the same text model the tunnel variant was alot slower for me.

 

[aleksandr unconditional]

Does that mean I can use any gguf model that I want to with the game?

y, u can use any model. on windows it’s easier - u just need to put the gguf file in the resources/koboldcpp/ folder inside the game.
the only thing is: sometimes, certain newer models might hallucinate with koboldcpp, but usually this can be fixed by updating kobold

 

[Jogy1959]

Sorry but I don't run Apps or Games, that want to have the ownership over my server. I will look for the game, when it is really local.

 

[aleksandr unconditional]

Thanks for that reidanota, been trying alot but i cant get it to run with my own model. When i start koboldcpp on my own it instantly closes the cmd window as soon as i open multiic. And the multiic cmd window shows that he is always looking for his own koboldcpp.exe in his game path doesnt matter what i choose in the settings.

I can use different models by copying them into the game folder like you said but whatever i try i cant get it to run with my own koboldcpp at localhost:5001.

if u launch koboldcpp separately from the game, then u need to select the model and settings in the koboldcpp client itself - wait until localhost:5001 opens in your browser.
in the game, just choose koboldcpp in the settings and don’t select a model (otherwise it will launch the koboldcpp embedded in the game)

 

[reidanota]

Sorry but I don't run Apps or Games, that want to have the ownership over my server. I will look for the game, when it is really local.

I'd like to understand what you mean. I don't have a server, and koboldcpp doesn't "take ownership" of anything on my PC that I'm aware of. It's just opening a virtual port for the front-end (the game) to communicate with the back-end (koboldcpp). If you're worried about someone remotely hacking this setup to have access to your PC, then you can disconnect it from the internet and even from your local network. This game doesn't need you to be online, works 100% locally.

 

[Jinkxy]

if u launch koboldcpp separately from the game, then u need to select the model and settings in the koboldcpp client itself - wait until localhost:5001 opens in your browser.
in the game, just choose koboldcpp in the settings and don’t select a model (otherwise it will launch the koboldcpp embedded in the game)

Yeah doesnt work for me as the game instantly tries to launch its own version and terminates mine and then will of course never find a model to use so its keeps "loading" forever.

 

[aleksandr unconditional]

Yeah doesnt work for me as the game instantly tries to launch its own version and terminates mine and then will of course never find a model to use so its keeps "loading" forever.

ah, on the main screen there's an icon in the bottom right corner that opens the selection between local models and api models - uncheck the box that remembers yr choice

 

[Jinkxy]

ah, on the main screen there's an icon in the bottom right corner that opens the selection between local models and api models - uncheck the box that remembers yr choice

Yeah thought that was the problem too, but that still didnt change anything. Maybe i have to delete all remembered settings from my registry.

 

[aleksandr unconditional]

Yeah thought that was the problem too, but that still didnt change anything. Maybe i have to delete all remembered settings from my registry.

hmm, actually, try the opposite: on that screen, select api models and check the box to remember yr choice. then, just in case, go into the settings and pick any api model.
once you’ve launched koboldcpp separately, go back into the game settings and select koboldcpp there

 

[Jinkxy]

N

hmm, actually, try the opposite: on that screen, select api models and check the box to remember yr choice. then, just in case, go into the settings and pick any api model.
once you’ve launched koboldcpp separately, go back into the game settings and select koboldcpp there

No still doesnt help. Also if i start my own instance of koboldcpp after starting the game it will not be recognized even with the tunneling that works otherwise.