Help with AI Backgrounds

[Deepfaceboy]

Anyone what any good loras and checkpoints for background art in a VN game? Specifically looking for modern interiors and exteriors. Right now I'm using Pony with Versatile Backgrounds, but the results are very iffey. The images really want to close off the rooms and make mistakes.
The kitchen image is more or less what I'm talking about. The bedroom image is what I managed after many hours of editing and recreating.

 

[osanaiko]

the results are very iffey.

after many hours of editing and recreating.

"Wanna know how I got these scars?"

Welcome to the happy world of trying to use Stable Diffusion as replacement for an artist.

My tips, as worthless as they might be:
- hack together your source images from "found" graphics sources / simple block-line art
- use controlnet canny/depth with relevant source images, at least to lock in the initial design of the background.
- don't be afraid to switch models/loras - the SD1.5 controlnets work much better than SDXL, and you can upscale and use SD1.5 outputs into a SDXL model to get the visual style you want at the very end.
- use LESS prompt terms, not more
- heavily use a repeated cycle of inpaint -> external edit -> img2img -> inpaint as you gradually fix all the issues that LEAP OUT AT YOUR EYES but normal people might not recognise
- come to a zen like understanding that the background details DO NOT MATTER to players, beyond a certain minimum "it feels right". Also accept that there will be anti-AI art / trolls who will still complain and point out all the issues in your game threads / give you bad reviews.
- optional: curse yourself for making a game design where background details actually matter to the gameplay sometimes

 

[osanaiko]

btw the bedroom looks very good... until one looks closer
- office style roof squares / fluro light fitting
- chair is weird geometry
- sun light on the wall should also continue onto the bedhead

but it doesn't matter - if you put some cute animetiddies girl sprites in front of the background, I wouldn't be looking at the background.

 

[Deepfaceboy]

- hack together your source images from "found" graphics sources / simple block-line art

Didn't think of this. Great idea, thanks.

- don't be afraid to switch models/loras - the SD1.5 controlnets work much better than SDXL, and you can upscale and use SD1.5 outputs into a SDXL model to get the visual style you want at the very end.

This was most of what I was really looking for, checkpoints and loras. My biggest consern is that I don't want to drift too far away from the character art-- it's about right as is, but it really doesn't want to make things that look like reasonable rooms.

- use LESS prompt terms, not more

They are a bit baren from what I remember, but I could be wrong, and even if I'm not I'm sure I can trim then down.

- heavily use a repeated cycle of inpaint -> external edit -> img2img -> inpaint as you gradually fix all the issues that LEAP OUT AT YOUR EYES but normal people might not recognise

so many hours

- come to a zen like understanding that the background details DO NOT MATTER to players, beyond a certain minimum "it feels right".

OCD hurts

Also accept that there will be anti-AI art / trolls who will still complain and point out all the issues in your game threads / give you bad reviews.

Yeah, I'm well aware. I argue with people about it without them even seeing my art, just on principles. All good.

curse yourself for making a game design where background details actually matter to the gameplay sometimes

Nooo... Not nearly good enough at coding for that. Gunna be a very basic renpy game with a few choices that probably just summate to "do you want this scene? y/n"

 

[osanaiko]

I'm glad I could help.

I'm not a pro by any means, but have been (and continue to) struggle along a similar journey in making assets for my game project.

In case it wasn't clear, one of the things I was suggesting is that using a specific Model and LORA combo to do the conversion to a certain art style can happen quite late in the process. So I've used several different sources to get the basics and some details "physcially" laid out, and then a few img2img passes at the end to bring it into the same linestyle/colorspace as the rest of the game assets.

Oh, a couple more tips:
- using photoshop's "perspective paste" stuff is great for transforming found images into the right angles for your design
- i've had success using PS color correction tools to fix mismatches between a character image generated in different poses. this matters the most when switching between poses of a character sprite mid-dialogue - if the colorspace or shading is different it is really obvious.
- prompts: you might already know this, but when doing inpaint, the prompt only needs to apply to the part you are replacing and its general place within the image. i.e no need to say "blonde short hair" if you are reworking the hand, unless the hand is next to the hair.


Regarding the idea of making a frankenstein-ed base images from lots of "found" sources:

here's the start image i put together for a specific room background (a VR relaxation pod)
- random image of a tiny home that i edited all the furniture out of with Paint.net clone tool
- render of a daz3d futuristic chair asset
- a dentist's chair work light arm
- a white occulus rift (or similar, can't remember)
- a wall and door I whipped up in blender



and here's the (not yet final) result: