Tool RPGM SLR Translator - Offline JP to EN Translation for RPG Maker VX, VX Ace, MV, and MZ

iYuma

Member
Nov 17, 2017
370
906
Quick question. Is there a way for this tool to instead use an LLM translation model from LM studio? I'm assuming I just need to configure the settings with the port and localhost stuff. As much as I like sugoi, LLM's has developed quite well now.
 

natsugajeel2000

Active Member
Jan 4, 2018
708
356
Sorry to request again, but for this DQ style game, I extracted the .rgssad file then ran the slr translator and put the files into the game folder, but it gives me this error after starting the game:
1743621967064.png
From translating it seems to have something to do with the script saying the script hung? What did I do wrong?

 

fantasmic

Member
Modder
Nov 3, 2021
434
1,059
Quick question. Is there a way for this tool to instead use an LLM translation model from LM studio? I'm assuming I just need to configure the settings with the port and localhost stuff. As much as I like sugoi, LLM's has developed quite well now.
The way I use SLR with LLM translations was to copy the sheetParser add-on from T++ to SLR (to enable export/import from spreasheets), then export as xlsx -> translate -> import from xlsx -> clear translations on red-/darkred-tagged cells. I don't translate any scripts, but those are manually done anyway.
 
  • Like
Reactions: iYuma

natsugajeel2000

Active Member
Jan 4, 2018
708
356
Anyone know what engine this is, or rather can SLR translate it: 1744891404811.png ]

Tried to unpack it doesnt work, game is DOREI QUEST from pyramidhouse:
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
I've released v1.128.
Added support for a newish MZ plugin that places notes stacked on top of dialogue. It's kinda hard to read with Latin characters because it's super small, but I guess still better than leaving it Japanese.
ex1.png

I've also added preview screenshots of the tool, which I should have probably done earlier, but I basically thought everyone knows what Translator++ looks like and I kept the core design largely the same. (Except I changed the color palette to no longer burn your eyes out.)
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
The way I use SLR with LLM translations was to copy the sheetParser add-on from T++ to SLR (to enable export/import from spreasheets), then export as xlsx -> translate -> import from xlsx -> clear translations on red-/darkred-tagged cells. I don't translate any scripts, but those are manually done anyway.
It took me till now to actually read that, I'm a bit surprised that you say that works.
I assume you mean this thing:
https://f95zone.to/threads/linguaga...-almost-everything-free-multilanguage.247102/
And I also assume you also exclude blue, aqua, yellow, and gold tags.

The issue is that my green tags are based on the current SLR escaper and the post processing making sure those specific scripts and commands aren't damaged.
Which means those tags aren't necessarily accurate for linguacha because it has to use it's own escaping and processing.

Essentially it is just luck whether or not green tagged cells are being processed correctly, or if they are being processed at all even though they would be processed correctly, because linguagacha could already have automation for some kind of plugin or script that I still tag red or yellow or not have automation for something I tag green.
 

fantasmic

Member
Modder
Nov 3, 2021
434
1,059
It took me till now to actually read that, I'm a bit surprised that you say that works.
Do you mean copying sheetParser into SLR, or the whole process with importing from sheets? I vaguely remember you mentioning that T++ add-ons should work with SLR, though it's possible that only applied way back then and T++ may have diverged too far by now to make it really reliable. Still, the sheetParser is an older add-on, so it works fine.

Yeah, that's the LinguaGacha I use. I don't exclude any tags when translating, since there's always a chance those will be relevant when editing. Even if I delete the translations in-project after importing, I've already run into situations where is was handy to have the translations available in the spreadsheet. However, I don't translate every file from the project. I translate it using SLR, save, "select all edited objects", then clear the translations and export those files. I don't save the project after clearing the translations, though it wouldn't really matter if I accidentally did since everything is saved to the cache.

So far, green-tagged cells have been handled properly (though I've mostly been doing MV/MZ games; who knows if it'll be good with the wild west of VXace). I'm not sure if it's LinguaGacha's preservations rules or DeepSeek being smart though. LG actually has support for translating .trans files directly these days, though I still prefer my spreadsheet method. It does some stuff with tagging when translating .trans files, but it's not well documented. The error reports are also less useful since the errors are arranged by file, which in this case is all the .trans file instead of individual map files. Having the errors be grouped by map makes it easier to see similarities between the errors, and that maybe it's one specific script call or map which is causing issues.
...
Oh wait, I remember I did have some VXace projects primed. For those, it handled benign and if-statement green cells properly, but bungled those using scripts. It's not really surprising, but that's why I still always run projects through SLR first and line up the LLM translation next to them.
 

SakuraKoi

Member
Oct 22, 2020
294
856
I really need to read some more into these tags of yours some of these days. There is so much to learn still... :HideThePain:

While I deliberately have made a quick and dirty LLM TL, in the future I have every intention to actual polish the works I have chosen. The TL I have done was more of a proof-of-concept for myself to see how, how well and if it works in the first place. It was already MTL translated and the game caught my eyes. I didn't even expect it to succeed so I found no harm done in releasing it.

Currently I am 'busy' (playing) and that quick & dirty TL was more of a little break that escalated. After all at the very least I should have used the wordwrap myself... but it turned out well. LinguaGacha inserted empty lines which resulted in empty text boxes once wrapped.
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
I've released v1.129.
"Normal" Automatic Word Wrapping will no longer preserve empty lines at the end of a cell.
Meaning if the cell to be wrapped for some reason has a bunch of random linebreaks at the end it will no longer keep them and make empty dialogue boxes for them. I specify "normal" because I purposely did not apply it for word wrapping of scripts/plugins, just in case there's a weird one out there that needs a linebreak at the end. And this will not touch purposely empty cells by the dev.

If you use the Automatic Word Wrapping button, you can now type in "cwrapable" to copy all original text to Initial Translations if it matches the requirements for "normal" word wrapping.
The point of that is so you can wrap dialogue boxes you didn't actually translate/change, for example because you are parsing the translation project of someone else.
I've also added the option to type in "ceverything", which will copy ALL original text to Initial Translations in case you want that for whatever reason. (It will also copy scripts, red tagged cells, etc.)

Edit: I forgot to mention, "cwrapable" and "ceverything" will only process empty rows, meaning you can still use them even if you've already translated/changed stuff. It will not overwrite your changes.
 
Last edited:

fantasmic

Member
Modder
Nov 3, 2021
434
1,059
Concerning DeepSeek, are you by chance using this?

It's something I've been meaning to look into, but never found the time/motivation.
I just pay for DeepSeek. I wanted to try out a llm and I'd heard it's cheap... and it turns out if you only use it during off-peak hours to take advantage of the off-peak discount then it's really cheap. Like, I've run over 25 games through it and only spent a little over $7. Even the most expensive games are just under a dollar, and smaller ones will often be ~$0.15. Most projects are done quickly, though I have had two which took 4 hours for some reason. Dunno if it's weird code or everyone else getting in on the discount or what.
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
On a whim of "why the fuck not" I asked deepseek to rewrite my main translation engine file to work with the deepseek api instead of the local sugoi server.
This is what it spit out:
JavaScript:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.DeepSeekEngine = void 0;
const TranslationEngineOption_1 = require("./TranslationEngineOption");
const TranslationEngineWrapper_1 = require("./TranslationEngineWrapper");

class DeepSeekEngine extends TranslationEngineWrapper_1.TranslationEngineWrapper {
    constructor(processor, thisAddon) {
        super(processor, thisAddon, {
            id: 'deepseek',
            name: 'DeepSeek',
            languages: {
                en: 'English',
                ja: 'Japanese'
            },
            batchDelay: 1,
            description: thisAddon.package.description,
            mode: 'rowByRow'
        });
        this.aborted = false;
        this.paused = false;
        this.pauseQueue = [];
       
        // Replace URLs option with API key option
        this.optionApiKey = new TranslationEngineOption_1.TranslationEngineOption({
            wrapper: this,
            id: 'apiKey',
            default: '',
            description: "Your DeepSeek API key. You can get one from the DeepSeek website.",
            name: 'API Key',
            category: TranslationEngineOption_1.TranslationEngineOptionCategories.LIMITS,
            priority: -10,
            formType: 'password'
        });

        this.optionModel = new TranslationEngineOption_1.TranslationEngineOption({
            wrapper: this,
            id: 'model',
            default: 'deepseek-chat',
            description: "The DeepSeek model to use for translation.",
            name: 'Model',
            category: TranslationEngineOption_1.TranslationEngineOptionCategories.OPTIONS
        });

        this.optionTemperature = new TranslationEngineOption_1.TranslationEngineOption({
            wrapper: this,
            id: 'temperature',
            default: 0.3,
            description: "Lower values make translations more consistent, higher values more creative.",
            name: 'Temperature',
            category: TranslationEngineOption_1.TranslationEngineOptionCategories.OPTIONS
        });

        this.optionMaxLength = new TranslationEngineOption_1.TranslationEngineOption({
            wrapper: this,
            id: 'maxTranslationLength',
            default: 5000,
            description: "Maximum amount of characters that will be sent per request to DeepSeek API.",
            name: 'Request Character Limit',
            category: TranslationEngineOption_1.TranslationEngineOptionCategories.LIMITS,
            priority: -9
        });

        this.optionConcurrency = new TranslationEngineOption_1.TranslationEngineOption({
            wrapper: this,
            id: 'maxTranslationJobs',
            default: 1,
            description: 'Maximum concurrent requests to DeepSeek API.',
            name: 'Maximum Request Count',
            category: TranslationEngineOption_1.TranslationEngineOptionCategories.LIMITS,
            priority: -8
        });
    }

    async doTranslate(toTranslate, options) {
        this.resetStatus();
        let returnTranslations;
        let returnError;
       
        if (options.sl != 'ja') {
            this.error(`[DeepSeek] The project specifies the source language as not being Japanese (${options.sl}). Since this engine is optimized for Japanese to English, we will use Japanese instead.`);
        }
        if (options.tl != 'en') {
            this.error(`[DeepSeek] The project specifies the destination language as not being English (${options.tl}). Since this engine is optimized for Japanese to English, we will use English instead.`);
        }

        let translations = new Array(toTranslate.length);
        let translatingIndex = 0;
        let completeThreads = 0;
        let totalThreads = this.optionConcurrency.getValue();
        totalThreads = totalThreads < 1 ? 1 : totalThreads;
        let maximumBatchSize = this.optionMaxLength.getValue();

        let complete = () => {
            if (++completeThreads == totalThreads) {
                returnTranslations(translations);
            }
        };

        let translationProgress = document.createTextNode('0');
        let translatedCount = 0;
        let errorProgress = document.createTextNode('');
        let errorCount = 0;
       
        this.print(document.createTextNode('[DeepSeek] Translating texts: '), translationProgress,
                  document.createTextNode('/' + toTranslate.length), errorProgress);

        const updateTranslatedCount = (count) => {
            translatedCount += count;
            translationProgress.nodeValue = translatedCount.toString();
            options.progress(Math.round((100 * translatedCount) / toTranslate.length));
        };

        const updateErrorCount = (count) => {
            errorCount += count;
            errorProgress.nodeValue = ' (' + errorCount.toString() + ' failed translations)';
        };

        const startThread = async () => {
            if (this.aborted) {
                returnError('Aborted.');
                return;
            }
            if (this.paused) {
                this.pauseQueue.push(() => {
                    startThread();
                });
                return;
            }

            let batchLength = 0;
            let batch = [];
            let batchIndexes = [];
           
            while (translatingIndex < toTranslate.length) {
                let index = translatingIndex;
                if (toTranslate[index] !== undefined &&
                    (batchLength == 0 || batchLength + toTranslate[index].length <= maximumBatchSize)) {
                    batch.push(toTranslate[index]);
                    batchIndexes.push(index);
                    batchLength += toTranslate[index].length;
                    translatingIndex++;
                } else {
                    break;
                }
            }

            if (batch.length == 0) {
                complete();
                return;
            }

            try {
                const apiKey = this.optionApiKey.getValue();
                if (!apiKey) {
                    throw new Error('API key is required');
                }

                const response = await fetch('https://api.deepseek.com/v1/chat/completions', {
                    method: 'POST',
                    headers: {
                        'Content-Type': 'application/json',
                        'Authorization': `Bearer ${apiKey}`
                    },
                    body: JSON.stringify({
                        model: this.optionModel.getValue(),
                        messages: [
                            {
                                role: "system",
                                content: "You are a professional translator. Translate the following Japanese text to natural, fluent English. Maintain the original meaning and tone."
                            },
                            {
                                role: "user",
                                content: `Translate these Japanese phrases to English:\n${batch.join('\n')}`
                            }
                        ],
                        temperature: this.optionTemperature.getValue()
                    })
                });

                if (!response.ok) {
                    throw new Error(`${response.status} - ${response.statusText}`);
                }

                const result = await response.json();
                const translatedText = result.choices[0]?.message?.content;
               
                // Split the response back into individual translations
                const translatedLines = translatedText.split('\n');
               
                if (translatedLines.length !== batch.length) {
                    throw new Error(`Received ${translatedLines.length} translations for ${batch.length} inputs`);
                }

                for (let i = 0; i < batch.length; i++) {
                    translations[batchIndexes[i]] = translatedLines[i];
                }

                updateTranslatedCount(batch.length);
            } catch (error) {
                updateErrorCount(batch.length);
                this.error(`[DeepSeek] Error during translation: ${error.message}`);
                console.error('[DeepSeek] Translation error:', error);
            } finally {
                startThread();
            }
        };

        return new Promise((resolve, reject) => {
            returnTranslations = resolve;
            returnError = reject;
            for (let i = 0; i < totalThreads; i++) {
                startThread();
            }
        });
    }

    resetStatus() {
        this.aborted = false;
        this.paused = false;
        this.pauseQueue = [];
    }

    abort() {
        this.aborted = true;
    }

    pause() {
        this.paused = true;
    }

    resume() {
        this.paused = false;
        this.pauseQueue.forEach((action) => {
            action();
        });
        this.pauseQueue = [];
    }
}

exports.DeepSeekEngine = DeepSeekEngine;
Besides the prompt obviously not being good enough I wonder if this would actually work. lol
I've actually never used AI for coding before.

If someone wants to volunteer to burn money to test it, tell me. :KEK:
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
Jokes aside if someone actually "really" cares about support for DeepseekV3, which of the "big" LLMs seems like the best option right now, then I would need a working official api key and I would need to actually put significant traffic on it, because I would need to test all kinds of script and placeholder constellations with different prompts and escapers to ensure the same stability people are used to with SLR. And obviously the whole deal could fail to work and you just loose money for no gain. (Although if communicating with the api doesn't work at all, it would also not charge you for it.)

There's more than enough ways to run LLMs on Jsons, I only see a point in adding a new translation engine if I can actually have it maintain the stability and ease of use of SLR. Without crashes or tons of missing translations.
Especially if it actually costs the user money to use.

Alternatively if someone can help me to actually get those reverse engineered apis to work that would be the "free" option. Although, who knows how long those options would still exist if we actually put attention and tons of traffic on it.
Examples:



TLDR: I'm still not a fan of the idea, I'm just exploring the option because using LinguaGacha with current SLR Translator just does not seem like a good idea.


Edit: To also preempt questions about smaller LLMs that could be hosted locally, they're all shit.
I've tested a bunch and I haven't found a single LLM small enough to run on "normal" hardware that doesn't make shit up.
They all suddenly put stuff in the translation that is clearly not present in the original text. They look like they are doing a better job than Sugoi because they get pronouns and grammar right more often, but who cares if the English looks good if it's not actually an accurate translation of the input?
 
Last edited:

gantos

Member
Feb 24, 2018
104
160
Edit: To also preempt questions about smaller LLMs that could be hosted locally, they're all shit.
I've tested a bunch and I haven't found a single LLM small enough to run on "normal" hardware that doesn't make shit up.
They all suddenly put stuff in the translation that is clearly not present in the original text. They look like they are doing a better job than Sugoi because they get pronouns and grammar right more often, but who cares if the English looks good if it's not actually an accurate translation of the input?
Define 'normal hardware'?
If you have 8-12 gb of vram there are a few good models, and if you have 20+GB of vram there are lots of choices that can even start to do parsing for you.
I think this is the best one right now: lmg-anon/vntl-llama3-8b-v2-hf
That one is custom trained on VN's, surprisingly accurate as long as you feed it a glossary for names/places, and really good at handling syntax.

Also mingshiba (sugoi's author) is going to be releasing a small and a big local model for sugoi:
The 14b model supposedly only needs 10 gb of vram. Given how good the sugoi models are for their size I think they could be really good.

I think this diagram thats been floating around might be why people are trying to use SLR with linguagacha:

TranslationTooling.png
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,284
5,861
Define 'normal hardware'?
With normal hardware I mean something an average person would have. Which has dropped quite a bit since people rather spend a bunch of money on their phone.
I would also argue that it's more likely that someone has $5 to spend on deepseek translations than $600 to spend on a GPU, when their goal is to play ancient RPG Maker games.

We are talking about a niche audience in general because I would assume that most people are perfectly happy with the normal performance of SLR using SugoiV4.


And to give a bit of an update, fantasmic seriously gave me his api key to burn, and I've already made a somewhat working prototype.
It's super slow right now and I've seen some cases in which it breaks and just returns the input, but for something I cobbled together in a few hours it's running pretty okay already. Although way too early to share, because it costs money and I want it to consistently work first.