ASERDDERERT
Member
- Jun 8, 2019
- 207
- 149
Models 7B parameter and lower not good enough. Something bigger than it starts to actually require good PC. Other way around is API, and I talked about it in general in my pervious post.Have you managed to miss llama, alpaca and a ton of models that were then fine tuned from llama? Google gpt4all for example, the inference is by default done on the CPU and is pretty fast on my 8th gen i5
On top of that running actual local model on CPU make it very slow. It is why normal ppl use GPU if they have good one or again paying $$$ for online rent of GPU.
To sum up, u either don't really know what you talk. Or just want to gacha here.