slick97
Active Member
- Dec 2, 2021
- 733
- 1,963
Bring this to the discord server and tag Starbuck, she would be thrilled to have someone go through and redo all of the character models. If you can create a solid workflow as well you could bring on additional help with modeling/rendering.I actually tried to generate different faces using Stable Diffusion; at this stage it's just a proof of concept, if anything; but my rough workflow appears to have a somewhat consistent result. However, it's not great, and I believe some more adjustment to the prompts will be needed to improve it (not to mention make them look better)
The idea so far is to use unren to extract image.rpa, then unzip all the character image zip files. Starting with character's faces by going into each pose and take crop the faces from each face pose image (since the body is separate, the face image contents on other information except bland spaces)
Use OpenPose to extract face pose image and use depth to generate a depth map (for control of the face shape)
Then use these two maps in Automatic1111 with openpose as first slot and depth as second (logic being it's the facial expression that's more important).
Then input descriptions into prompt, for example, with the angry face
With Clara MA being a LoRA trained with images of Caroline from the game Milfy City, I'll try mixing it with other LoRAs later.
I do 2 batches of size 2 to generate 4 images, and use the results in GIMP to extract the face, matching the position of the original, and save as a new face. I will probably play around with color map at some point to see if that can fix the color inconsistencies.
Original
View attachment 2946262
Test result (I will see if some image-to-image inpainting can fix the odd patches of hair later, and will have do something about the depth map which isn't optimal -- maybe using the depth map plugin instead of OpenPose depth map -- or try a different preprocessor). The SD model is also too biased towards realistic images, have to find another one
View attachment 2946275
I will see if I can write a costume class that loads extra faces and body types if this is successful; I know there's one script for loading extra images outside the RPA file, so I'll be looking into that if the SD method turns out hopeful.
The main to keep in mind when using AI generated models it to ensure consistency between textures. A few modders for various different games (i.e. Skyrim) have taken this approach and failed to properly adjust their textures to remove obvious seams.