Tool Mtool offline sugoi translation

james448

Newbie
Aug 13, 2017
48
242
Anyone here ever use ai to translate text file, any successful one on it?
I thought about it and it seem possible but a bit of a pain to set up, if someone is ever interested to try here where to start:

use a model like

There may be better model, up to you find one

To run it you are supposed to run it on a linux obviously that a pain, so a solution exist to run it locally whiteout linux and that to use docker

if someone can set it up and give simple instruction how to replicate it be great.

Obviously you have to do some code (In python) but it shouldn't not be to difficult, you just read the file line by line and give a copy file _output with the translation given by it.

I don't know the specific of how it handle context and chat but it should give good result in theory if it can remember previous text.
 

np305

Newbie
Nov 5, 2019
71
25
I thought about it and it seem possible but a bit of a pain to set up, if someone is ever interested to try here where to start:

use a model like

There may be better model, up to you find one

To run it you are supposed to run it on a linux obviously that a pain, so a solution exist to run it locally whiteout linux and that to use docker

if someone can set it up and give simple instruction how to replicate it be great.

Obviously you have to do some code (In python) but it shouldn't not be to difficult, you just read the file line by line and give a copy file _output with the translation given by it.

I don't know the specific of how it handle context and chat but it should give good result in theory if it can remember previous text.
cloud solution ? through api key based program like DazedMTL ?
 

james448

Newbie
Aug 13, 2017
48
242
cloud solution ? through api key based program like DazedMTL ?
The solution i said would be free and offline which is what i would prioritize.

If you are fine using something like chatgpt, remember to jailbreak as it may refuse it a sentence is too erotic or something and ready to pay for what is ask, it won't be free to use.

here the code you need to do it, i can't really test it because i don't want to make an account.



once you got the set up for translation, check on how to make a chat and context then look at my code and just read all file and send the line one by one to make the translated one, just name them the same way i do and add _output to it.
 
  • Like
Reactions: np305

Naiv

Newbie
Dec 26, 2018
76
58
is there a way to use the translated json without the MTool? Right now Mtool makes my system chugs after like 15 minutes
 

james448

Newbie
Aug 13, 2017
48
242
is there a way to use the translated json without the MTool? Right now Mtool makes my system chugs after like 15 minutes
You can try Translator++, i think you can export the text that need to be translated in json
 

normalperson492

New Member
Oct 30, 2018
1
0
Hi, just in case anyone else is trying to use this, I believe this method doesn't work when the "extract" files number over 100 (yes I was trying to translate MGQ paradox). The file is output fine but the translation doesn't match the original text at some points. My solution was to split the ManualTransFile into two and run Convert twice, first with extract01-99, second with extract100-111 renamed to extract1-12 (I had 111 extract files total). This seems to have produced a working translation for me.
 

james448

Newbie
Aug 13, 2017
48
242
Hi, just in case anyone else is trying to use this, I believe this method doesn't work when the "extract" files number over 100 (yes I was trying to translate MGQ paradox). The file is output fine but the translation doesn't match the original text at some points. My solution was to split the ManualTransFile into two and run Convert twice, first with extract01-99, second with extract100-111 renamed to extract1-12 (I had 111 extract files total). This seems to have produced a working translation for me.
Just so you know, in the extract file you can replace all the
sprintf('%02d' by sprintf('%03d'
That would make all the file have a padding in the one hundred, I never tough you could have the need to translate more than 500000 lines
in that code the number before d represent the number you want as padding so that as an example file number 3 is named 03 or 003

As an alternative you can also change the number of lines at in extract file too at
if ($currentLine >= 5000) {
change it to a higher number and you will get less file
 
  • Like
Reactions: normalperson492

Naiv

Newbie
Dec 26, 2018
76
58
You can try Translator++, i think you can export the text that need to be translated in json
Yeah, I was thinking that too. But I'm on Translator++ and haven't seen any option for json. I probably missed it
 

james448

Newbie
Aug 13, 2017
48
242
Yeah, I was thinking that too. But I'm on Translator++ and haven't seen any option for json. I probably missed it
I just checked and I haven't seen one either, but it is possible to export to a excel sheet, so it possible to make a program for it, and extract it. I could do it but it would take time so tell me if you are interested otherwise i won't bother to do it.
 

Naiv

Newbie
Dec 26, 2018
76
58
Oh definitely. Unless there's a way to translate RPG Maker game using Sugoi/more easily than Mtool. The current Mtool Bing translation model isn't that good so putting it through Sugoi is way better quality wise
 

james448

Newbie
Aug 13, 2017
48
242
Oh definitely. Unless there's a way to translate RPG Maker game using Sugoi/more easily than Mtool. The current Mtool Bing translation model isn't that good so putting it through Sugoi is way better quality wise
I am busy this month but after that I should have time, i make a program so that when you export the project of Translation++ in excel, the php code extract it, make you use sugoi file translation on extracted then put it on a copy of excel with the translation that you are able to import. I make a new thread for that tool when it done and reply to let you know too.
 
  • Like
Reactions: Naiv

firegame9231

New Member
Jul 28, 2019
2
2
Thank you for the guide.

On the topic of LLMs someone had a guide to using LM Studio and Luna Translator together on r/visualnovels. I don't think i can link the thread yet.
LM Studio creates a offline LLM with an endpoint that you can feed the extract output to. I created a Translate.php that seems to work, but barely. Here is what i have while trying to translate 【浮遊魔法都市のレイ】~直系の魔導士~

Some notes:
Your entirely at the mercy of how well your LLM behaves.
aya-expanse-8b is heavily restrictive and will not translate lewd content
aya-23-8b is less restrictive but at anypoint can basically ignore your system prompt and add its own dialog
vntl-llama3-8b seems to be behaving well enough for now, but is still prone to randomly dividing lines or mismatching the current line.

If it fails to translate correctly you can kind of bootstrap context by adding translations to the system prompt. Because of the high rate of failure i would recommend having smaller files sizes so it easier to continue close to where you left off. I would also run it with xdebug to workout any kinks missed.

How to use:
Theoretically you can just click Translate.php while on localhost/json_line
Set $model to what you download
Edit $systemPrompt with rules and context
Set $contextLength within your token limit a context of 20 lines use about 1000 tokens
$file_skip_amount is there incase the script fails and you have successful files of translations you don't want to erase when you rerun the script

You don't have permission to view the spoiler content. Log in or register now.
 

james448

Newbie
Aug 13, 2017
48
242
Thank you for the guide.

On the topic of LLMs someone had a guide to using LM Studio and Luna Translator together on r/visualnovels. I don't think i can link the thread yet.
LM Studio creates a offline LLM with an endpoint that you can feed the extract output to. I created a Translate.php that seems to work, but barely. Here is what i have while trying to translate 【浮遊魔法都市のレイ】~直系の魔導士~

Some notes:
Your entirely at the mercy of how well your LLM behaves.
aya-expanse-8b is heavily restrictive and will not translate lewd content
aya-23-8b is less restrictive but at anypoint can basically ignore your system prompt and add its own dialog
vntl-llama3-8b seems to be behaving well enough for now, but is still prone to randomly dividing lines or mismatching the current line.

If it fails to translate correctly you can kind of bootstrap context by adding translations to the system prompt. Because of the high rate of failure i would recommend having smaller files sizes so it easier to continue close to where you left off. I would also run it with xdebug to workout any kinks missed.

How to use:
Theoretically you can just click Translate.php while on localhost/json_line
Set $model to what you download
Edit $systemPrompt with rules and context
Set $contextLength within your token limit a context of 20 lines use about 1000 tokens
$file_skip_amount is there incase the script fails and you have successful files of translations you don't want to erase when you rerun the script

You don't have permission to view the spoiler content. Log in or register now.
I checked the code and it look good, if you want me to include it in the github you can do a pull request so you can get the credit, and for all setting option add it too, to the default with extract when creating setting.json and in the setting.json itself just put an example.
 

firegame9231

New Member
Jul 28, 2019
2
2
Okay i create a pull request for my changes.

Looking at the project by dazedanon it looks like you'll get a better translation by having a large ruleset and a structured input, right now it's line by line, batching it so that multiple lines are translated together can probably stop some of the randomness i see with certain translations. I'll leave that for another day.
 

james448

Newbie
Aug 13, 2017
48
242
Okay i create a pull request for my changes.

Looking at the project by dazedanon it looks like you'll get a better translation by having a large ruleset and a structured input, right now it's line by line, batching it so that multiple lines are translated together can probably stop some of the randomness i see with certain translations. I'll leave that for another day.
Thanks, got it done.
Yeah, sadly this is the limitation of Sugoi, there is no particular way you can divide the line inside the text to keep the exact format of the original text. With Ai it should be capable of keeping the text format and for this we can just make a setting option when using translate ai instead of sugoi, won't need to separate each line inside the text. I do it so some time soon.