RPGM WIP Shisaye's Project List

Silentce

Quiet Eldritch Horror
Modder
Uploader
Jan 2, 2018
4,274
26,034
A translation was already completed for Sheryl. it is being reviewed for Rule 7.

I think there is a tranlsation out there for the first one too.
 

lolchito

Member
Sep 15, 2019
247
137
A translation was already completed for Sheryl. it is being reviewed for Rule 7.

I think there is a tranlsation out there for the first one too.
Ty and nice, I don't know if it's gonna pass, but if you suddenly have the link to download it can you send it?
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
third game: https://f95zone.to/threads/translation-requset-sheryl-mega-download.87836/

second and third one doesn't have any translation.

Edit 2: Sheryl has now some community translation beign made now, one even beign in the test of the rule 7 so good I guess, now the only game that doesn't have any near translation it's the third one
action game with pixel art? +1 for sheryl
Erm... I'm a bit confused what you mean with it doesn't have any translation.
Sheryl is already on my SLR list.
https://f95zone.to/threads/translation-requset-sheryl-mega-download.87836/post-10585186

I stopped working on it because a manual translator was working on it instead.
https://f95zone.to/threads/translation-requset-sheryl-mega-download.87836/post-11464863

Said translator finished the translation but the thread has not been released, yet, because staff still needs to vote whether or not it's actually allowed here.
(Rule 7 Issues.)

Edit: Didn't see the edits before posting. it just showed me your original post.
 
Last edited:

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
Hi, I would like to make 3 request

first game: https://f95zone.to/threads/translation-requset-rpg-konoha-lolique2gaiden-mega-download.103577/

Edit: first game it's actually having an official translation for steam, it's official annunced by otaku plan it doesn't have any date for now, but for what I've seen it gotta be a censored version for steam, and l don't know at the same time or later, but a +18 patch
The thread is full of weird translation attempts and if it's getting an official translation anyway I don't really want to work on it.

I'll look into it. Edit: Done.
 
Last edited:
  • Like
Reactions: lolchito

yurum

Newbie
Jun 2, 2018
58
485
Alrighty. I had to implement it replacing the alpha channel before scanning, but now it's somewhat working.
I've attached the result. Please let me know what you think.

But ofc keep in mind that this happened in like 40 seconds with offline neural networks that I'm running in my ram, so obviously you can't favorably compare it to something like Google Lens or a manual translation.

Edit: I should also mention that your example was pretty much the worst case scenario of half transparent short text. It works a lot better for bigger blocks of text.
Understandable that it's a work in progress. Looking through the results, it seems like it really struggled with smaller fonts, especially when complicated kanji are involved, or distinguishing between something like ぽ and ぼ. I'm guessing that's one point of difficulty, especially if it fails in a 1600xsomething game; most rpgmaker games are already tiny.
It also struggles with 0, some of the images from a-1_18 to 34 drop the zero.

When the ocr does successfully recognize the text, the en text returned at least somewhat catches the gist of the text (stripped of context sure, but that's something all mtls struggle with).

If you're open to trying something out, could you run the same texts again with the 4x upscaled ones below? I don't expect it to be wildly successful since the images have artifact/strokes blending together issues, but it's worth checking out I think.

The workflow I'm thinking of for this is to upscale tiny images using chainner, apply your ocr, then downscale them again with chainner.

Upscaled images:
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
Understandable that it's a work in progress. Looking through the results, it seems like it really struggled with smaller fonts, especially when complicated kanji are involved, or distinguishing between something like ぽ and ぼ. I'm guessing that's one point of difficulty, especially if it fails in a 1600xsomething game; most rpgmaker games are already tiny.
It also struggles with 0, some of the images from a-1_18 to 34 drop the zero.

When the ocr does successfully recognize the text, the en text returned at least somewhat catches the gist of the text (stripped of context sure, but that's something all mtls struggle with).

If you're open to trying something out, could you run the same texts again with the 4x upscaled ones below? I don't expect it to be wildly successful since the images have artifact/strokes blending together issues, but it's worth checking out I think.

The workflow I'm thinking of for this is to upscale tiny images using chainner, apply your ocr, then downscale them again with chainner.

Upscaled images:
I've tried it, but since the issues are with the OCR and not SugoiV4(The MTL) here's the actual Japanese output of each picture instead of it being drawn on the images. (The numbers are the coordinates of the corners around the detected text block.)
If you want the actual MTL results instead just tell me, I just thought this would work better for you.

I really appreciate the help.
 

yurum

Newbie
Jun 2, 2018
58
485
I've tried it, but since the issues are with the OCR and not SugoiV4(The MTL) here's the actual Japanese output of each picture instead of it being drawn on the images. (The numbers are the coordinates of the corners around the detected text block.)
If you want the actual MTL results instead just tell me, I just thought this would work better for you.

I really appreciate the help.
Yeah, I have experience with Sugoi and it's a decent mtl tool. I got that the ocr had trouble recognizing the texts, which is why I thought it might work better with the upscaled images. I have absolutely no programming experience, so improving the image input (via upscaling) was my suggestion to improve the ocr process.

Glancing through the json files though, I'm pretty sure what I wrote below is happening.
Understandable that it's a work in progress. Looking through the results, it seems like it really struggled with smaller fonts, especially when complicated kanji are involved, or distinguishing between something like ぽ and ぼ. I'm guessing that's that the blurry texts are one point of difficulty, especially if it fails in a 1600xsomething game; most rpgmaker games are already tiny.
Upscaled images:
I'm not sure what feedback you need exactly to improve/train the ocr. What do you need?
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
Yeah, I have experience with Sugoi and it's a decent mtl tool. I got that the ocr had trouble recognizing the texts, which is why I thought it might work better with the upscaled images. I have absolutely no programming experience, so improving the image input (via upscaling) was my suggestion to improve the ocr process.

Glancing through the json files though, I'm pretty sure what I wrote below is happening.


I'm not sure what feedback you need exactly to improve/train the ocr. What do you need?
Honestly I just wanted to know now if you think that upscaling it actually improved it now or not. (Those Json were made using your upscaled versions.)
Or if you would say the detection is pretty much the same.
Also if you see potential in general, have ideas, before I run it on all the picture based text missing from my SLRMTL projects.

I honestly thought I was pretty much done with it because my tests before that actually worked really well, while you now managed to make it look like garbage. :KEK:

So now I'm trying to come up with ways to improve the OCR on those types of pictures.


To explain how much of a difference I mean, here's some examples from the game "Figure of a Hero" which I originally used to test the system.
 
Last edited:

yurum

Newbie
Jun 2, 2018
58
485
Honestly I just wanted to know now if you think that upscaling it actually improved it now or not. (Those Json were made using your upscaled versions.)
Or if you would say the detection is pretty much the same.
Also if you see potential in general, have ideas, before I run it on all the picture based text missing from my SLRMTL projects.

I honestly thought I was pretty much done with it because my tests before that actually worked really well, while you now managed to make it look like garbage. :KEK:

So now I'm trying to come up with ways to improve the OCR on those types of pictures.


To explain how much of a difference I mean, here's some examples from the game "Figure of a Hero" which I originally used to test the system.
Ohhh , that's my bad then, I'm the one who misunderstood. I thought the json files you sent were from the original images.

Could you send over the mtl overlaid on the upscaled images? It still struggled with the first image, but it got more accurate readings for a few that were mistranslated in the original. I want to see if the final results actually got better.


As for ideas, one would be to reduce the tolerance(?) of blocking texts together. You've probably already noticed that the two separate bottom text in Commentary21.png got mushed together in the en tl. Honestly, other than that, the tool looks like it works fine. Like you said, my images are the worst case scenarios for it. The issues I think are that:
1) For the first image especially, the font is pixelated and small, which confuses the OCR. Upscaling doesn't help at all, because it really only makes bigger blurrier text. This is more relevant for kanji.
2) The font has uneven stroke widths - vertical strokes are thick, and horizontal ones are tiny af. This makes the OCR miss some horizontal strokes of a character, especially for katakana and hiragana (e.g. it mistakes で for じ because it doesn't see the top line).
3) For the first image, the text is overlaid over the lines of the diagram, which again, confuses the OCR. Most of the inaccuracies I see there are on characters that are written above the line.
4) It can kind of work with handwritten fonts (at least the one used here), but it'll fuck up sometimes. Not as much as with issue 1 and 2, but it still happens often enough.

Not sure how helpful it would be but maybe you could use some kind of tool/program that can identify white (or other) colors enclosed in black and make everything other than the identified areas black before running the ocr. Unlikely that there's something like this out there, but who knows. Useful in cases of issue 1 and 3 above.
Also, what about cropping only the text parts out, testing different ocr models on them, then restitching the image together? Not sure how easy that would be though. The best ocr I came across was like that, but it was an online service and it stopped working a while ago.

TLDR is the tech likes things clean and discrete. Upscaling might have only a marginal improvement at best. The OCR tool is pretty much good to go as long as the text is as clear and uniform as in the example you gave.
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
Ohhh , that's my bad then, I'm the one who misunderstood. I thought the json files you sent were from the original images.

Could you send over the mtl overlaid on the upscaled images? It still struggled with the first image, but it got more accurate readings for a few that were mistranslated in the original. I want to see if the final results actually got better.
I've applied it to the original pictures because that's what would have to happen anyway, but I used the upscaled version for the scanning. (I just divided all box coordinates by 4, it's not perfect, but kinda worked.)
I used both SugoiV4 and DeepL so you also have a comparison there.
 
Last edited:

yurum

Newbie
Jun 2, 2018
58
485
After looking through the images, here're my thoughts.
1. Upscaling - Out of all the mistranslated lines in the original ocr, about 30-40% were fixed in the upscaled ocr. It also improved some lines that were tled somewhat accurately. However, the upscaled ocr did sometimes get something wrong that the og ocr got right, but less often than the corrections it gave, and even then only for katakana stuff. The zeros were also accounted for better here, but that's probably an issue of the character used for them, and not size/blurriness.
Workflow suggestion: Upscale large images with different blocks of texts in them. Images that contain only texts (e.g. attached below) are okay to use as is, unless really blurry/low-res.
2. Deepl vs Sugoi - Deepl does better translating UI labels like variable or item labels (Affection, level, time of day, sex toy, etc.). When the OCR failed to give the proper kanji for a word, Deepl was able to correct it somehow. For sentences though, sugoi does better.
3. The overlaid text is too small on some of the upscaled images for some reason, e.g. attached below.

I used ChaiNNer to upscale the images since it was able to preserve the alpha channels in the images. It's python-based, so you could automate that too, and it has native support for batch upscaling images.
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
After looking through the images, here're my thoughts.
1. Upscaling - Out of all the mistranslated lines in the original ocr, about 30-40% were fixed in the upscaled ocr. It also improved some lines that were tled somewhat accurately. However, the upscaled ocr did sometimes get something wrong that the og ocr got right, but less often than the corrections it gave, and even then only for katakana stuff. The zeros were also accounted for better here, but that's probably an issue of the character used for them, and not size/blurriness.
Workflow suggestion: Upscale large images with different blocks of texts in them. Images that contain only texts (e.g. attached below) are okay to use as is, unless really blurry/low-res.
2. Deepl vs Sugoi - Deepl does better translating UI labels like variable or item labels (Affection, level, time of day, sex toy, etc.). When the OCR failed to give the proper kanji for a word, Deepl was able to correct it somehow. For sentences though, sugoi does better.
3. The overlaid text is too small on some of the upscaled images for some reason, e.g. attached below.

I used ChaiNNer to upscale the images since it was able to preserve the alpha channels in the images. It's python-based, so you could automate that too, and it has native support for batch upscaling images.
Hm... I wonder if ChaiNNer does a better job than RealESRGANx4plus. That's what I used for upscaling till now.

As for the font size, that's a long story and most likely not actually a problem with the fontsize but with the box size because I have to try to make it always fit the box regardless of the overall scale.
I guess that box somehow got messed up and turned out way too small.

If you're curious this is how my script for determining linebreaks and font size looks like. (Taken from the Wand documentation.)
Python:
def word_wrap(image, ctx, text, roi_width, roi_height): #this likes crashing :C
    """Break long text to multiple lines, and reduce point size
    until all text fits within a bounding box."""
    mutable_message = text
    iteration_attempts = 100

    def eval_metrics(txt):
        """Quick helper function to calculate width/height of text."""
        metrics = ctx.get_font_metrics(image, txt, True)
        return (metrics.text_width, metrics.text_height)

    while ctx.font_size > 0 and iteration_attempts:
        iteration_attempts -= 1
        width, height = eval_metrics(mutable_message)
        if height > roi_height:
            ctx.font_size -= 0.75  # Reduce pointsize
            mutable_message = text  # Restore original text
        elif width > roi_width:
            columns = len(mutable_message)
            while columns > 0:
                columns -= 1
                mutable_message = '\n'.join(wrap(mutable_message, columns))
                wrapped_width, _ = eval_metrics(mutable_message)
                if wrapped_width <= roi_width:
                    break
            if columns < 1:
                ctx.font_size -= 0.75  # Reduce pointsize
                mutable_message = text  # Restore original text
        else:
            break
    if iteration_attempts < 1:
        raise RuntimeError("Unable to calculate word_wrap for " + text)
    return mutable_message
 
Last edited:

yurum

Newbie
Jun 2, 2018
58
485
Hm... I wonder if ChaiNNer does a better job than RealESRGANx4plus. That's what I used for upscaling till now.

As for the font size, that's a long story and most likely not actually a problem with the fontsize but with the box size because I have to try to make it always fit the box regardless of the overall scale.
I guess that box somehow got messed up and turned out way too small.

If you're curious this is how my script for determining linebreaks and font size looks like. (Taken from the Wand documentation.)
Yeah... I'm not a programmer so I really don't get any of the lines of code there.
As for upscaling, ChaiNNer isn't an upscaling model itself, it's the GUI used to apply the upscaling models to images. I used 4x_-Ultrasharp since it was best at preserving kanji strokes out of those I tested. There're models better for specialized use out there, and Ultrasharp or Animesharp will probably be better than realesregan.





There's also this specialized for lowres text, , but it wasn't any better than Ultrasharp.
 

Shisaye

Engaged Member
Modder
Dec 29, 2017
3,181
5,505
Yeah... I'm not a programmer so I really don't get any of the lines of code there.
As for upscaling, ChaiNNer isn't an upscaling model itself, it's the GUI used to apply the upscaling models to images. I used 4x_-Ultrasharp since it was best at preserving kanji strokes out of those I tested. There're models better for specialized use out there, and Ultrasharp or Animesharp will probably be better than realesregan.





There's also this specialized for lowres text, , but it wasn't any better than Ultrasharp.
Oh neat. I'll try those out.
I never had to upscale text so I don't really have a clue there.
I have a separate project going that is trying to automatically decensor pictures using stable diffusion. That's why I knew that one upscaler and that's all my knowledge there. lol