[Stable Diffusion] Prompt Sharing and Learning Thread

[Gtg Gtg]

As someone who have trained plenty of Dreambooth and LoRA models based on cosplayers, I can advice you to stick with the most natural looking pictures, and avoid the actual cosplay ones. You want variety, yes, but of lighting conditions, poses, environment, etc. If you try to use pictures with heavy makeup or clothing that occludes too much of their natural body shape you will have a hard time getting consistency from the model. The point is to teach the model how to draw that person, not how they look in cosplay; that you'll get from prompting.

Also avoid images that are too dark, too low resolution, too grainy, etc. You want high quality, not high volume. You'll need images cropped to 512px square if you're training for Stable Diffusion 1.5 models, and 1024px square if you're training for SDXL, so make sure that the pictures have big enough resolution so that when you crop the desired area from them, the resulting image is already equal or larger than those resolutions, meaning NEVER UPSCALE your source images.

And yes, you probably want to stick with about 30-50 pictures total, more or less in a proportion of 60% of headshots, 35% of medium shots (waist up), and 15% of full body shots -- that's a good ratio for images in my experience.

I'm just curious now about the regularisation images, do I really need them? what should they be? how do I set it up? how many of these do I even need? so when I do repeats like 50_filename for images folder, I would do the same for regularisation folder? I've never actually touched on regularisation that much and was just shoving random images

 

[hkennereth]

I'm just curious now about the regularisation images, do I really need them? what should they be? how do I set it up? how many of these do I even need? so when I do repeats like 50_filename for images folder, I would do the same for regularisation folder? I've never actually touched on regularisation that much and was just shoving random images

I have trained models with and without regularization images, and honestly it's hard to really see any major difference that you can point as a direct result of using them. It's an optional step, and I'd say you can skip it; all my LoRAs were trained without it, and I can safely say that whenever I got a bad model it was due to my source images, not the lack of regularization images.

That said, if you do want to use them, I would recommend just finding a pack of regularization images somewhere online and using that, it's not worth the trouble to try to create a set of images yourself.

When training LoRAs for SD 1.5, I mostly followed this particular tutorial to the letter, and got some great results:

You must be registered to see the links

 

[Gtg Gtg]

I have trained models with and without regularization images, and honestly it's hard to really see any major difference that you can point as a direct result of using them. It's an optional step, and I'd say you can skip it; all my LoRAs were trained without it, and I can safely say that whenever I got a bad model it was due to my source images, not the lack of regularization images.

That said, if you do want to use them, I would recommend just finding a pack of regularization images somewhere online and using that, it's not worth the trouble to try to create a set of images yourself.

When training LoRAs for SD 1.5, I mostly followed this particular tutorial to the letter, and got some great results:

You must be registered to see the links

with your previous advice on the 512x squares, sometimes its hard to find full body images, how do you deal with it? the cosplayer I'm doing usually does vertical images and rarely ever does full body images.

 

[hkennereth]

with your previous advice on the 512x squares, sometimes its hard to find full body images, how do you deal with it? the cosplayer I'm doing usually does vertical images and rarely ever does full body images.

You do the best with what you got. If the person doesn't have full body pictures available you get the closest you can find to that, and I suppose you won't get accurate depictions of their toes. All you are training is how that specific person looks like but SD already knows how "people" look like. You want full body pictures to make sure that their general body is portrayed with accurate proportions, but if you fail to provide pictures that show how their feet look SD will just extrapolate from what you did provide and draw "generic" legs.

To give you a better idea of what a training set looks like, this is (almost; it's missing like 6 headshots that didn't fit on my screen) the entire set I used to train a model of cosplayer Rolyat:


And here are couple of pictures I managed to make with the resulting LoRA. Now, I wouldn't call them flawless depictions of her likeness, but it's in the range of how accurate one can get, it depends a lot on the prompt you use as well, and here I was more concerned about making the images interesting than being 100% accurate.

 

[Sepheyer]

with your previous advice on the 512x squares, sometimes its hard to find full body images, how do you deal with it? the cosplayer I'm doing usually does vertical images and rarely ever does full body images.

There's a mini-thread on training a Lara Croft LORA, with all the usual suspects from this thread: https://f95zone.to/threads/loras-for-wildeers-lara-croft-development-thread.173873/

There might be a few tidbits of wisdom. I saw you already encountered one earlier, which is: ~20 images is all you need. And in that thread I posed above we kinda empirically confirm that statement. There are quite a few other things that were stumbled upon during the discussion, I just don't recall what they were. Oh, I remember - I realized LORAs are a bit of a deadend for me personally and started looking into offshoots.

 

[Gtg Gtg]

There's a mini-thread on training a Lara Croft LORA, with all the usual suspects from this thread: https://f95zone.to/threads/loras-for-wildeers-lara-croft-development-thread.173873/

There might be a few tidbits of wisdom. I saw you already encountered one earlier, which is: ~20 images is all you need. And in that thread I posed above we kinda empirically confirm that statement. There are quite a few other things that were stumbled upon during the discussion, I just don't recall what they were. Oh, I remember - I realized LORAs are a bit of a deadend for me personally and started looking into offshoots.

offshoots? like other ways to replicate somebody/something better?

 

[me3]

Having nearly 500 images from the same "series" i posted a while ago, i thought i'd test some inpainting things.
Starting image and, well so far, end result. There's still things that should have been fixed...something for another day i guess...

Spoiler: Just an upscale to keep up with the boob ratio

You don't have permission to view the spoiler content. Log in or register now.

 

[Sepheyer]

offshoots? like other ways to replicate somebody/something better?

For a few evenings we had the IPAdapter hopes flying high: https://f95zone.to/threads/stable-diffusion-prompt-sharing-and-learning-thread.146036/post-11985258

But alas, we conclusively proved it can't replace LORAs.

 

[hkennereth]

Not sure if this is something anyone wants to see on this thread, but since it's been a bit quiet I didn't think you guys would mind much. Here is a set of images I created to celebrate Halloween featuring some cosplayers and internet models of whom I have created LoRAs before.

(Bonus points to anyone who can recognize them lol)

 

[Sharinel]

Not sure if this is something anyone wants to see on this thread, but since it's been a bit quiet I didn't think you guys would mind much. Here is a set of images I created to celebrate Halloween featuring some cosplayers and internet models of whom I have created LoRAs before.

(Bonus points to anyone who can recognize them lol)

View attachment 3059650 View attachment 3059651 View attachment 3059652 View attachment 3059653 View attachment 3059654 View attachment 3059655 View attachment 3059656 View attachment 3059657 View attachment 3059658 View attachment 3059659 View attachment 3059660 View attachment 3059661 View attachment 3059662 View attachment 3059663 View attachment 3059664 View attachment 3059665 View attachment 3059666 View attachment 3059667 View attachment 3059668

I think the 3rd one is Olivia Casta? Which is some sort of LoRa-ception, as she is already an AI of another model

 

[hkennereth]

I think the 3rd one is Olivia Casta? Which is some sort of LoRa-ception, as she is already an AI of another model

Yes, it is. And... yes, she is. Although I think she just uses like Snapchat filters or something, she is "pre-AI"

 

[me3]

For those that's been avoiding updating their nvidia drivers due to not wanting to deal with the potential usage of shared memory:

You must be registered to see the links

 

[me3]

Which of these do ppl prefer/find the best/least bad? and short note on why if possible.

Something along the process for 2 of the images has removed a strain of hair but left half the shadow which has made a somewhat interesting "running makeup" effect which is not intentional. Fixing it will literally that hours to do just to generate the images so try to ignore that fault.



(I'll update the post with the base prompt later as atm i don't want that to influence things in some way.)

Spoiler: Base prompt

You don't have permission to view the spoiler content. Log in or register now.

 

[Jimwalrus]

Which of these do ppl prefer/find the best/least bad? and short note on why if possible.

Something along the process for 2 of the images has removed a strain of hair but left half the shadow which has made a somewhat interesting "running makeup" effect which is not intentional. Fixing it will literally that hours to do just to generate the images so try to ignore that fault.

View attachment 3065005 View attachment 3065004 View attachment 3065003 View attachment 3065002

(I'll update the post with the base prompt later as atm i don't want that to influence things in some way.)

I'd personally say #4 - although the lighting is a little more stark resulting in darker shadows, the 'modeling'* it produces is more subtle and much more realistic.

*I think I'm using that correctly, it's a photography term I seem to have picked up from 'reading' my Dad's books about photography as a teenager, especially the glamour sections...

 

[hkennereth]

Which of these do ppl prefer/find the best/least bad? and short note on why if possible.

Something along the process for 2 of the images has removed a strain of hair but left half the shadow which has made a somewhat interesting "running makeup" effect which is not intentional. Fixing it will literally that hours to do just to generate the images so try to ignore that fault.

View attachment 3065005 View attachment 3065004 View attachment 3065003 View attachment 3065002

(I'll update the post with the base prompt later as atm i don't want that to influence things in some way.)

#1 and #2 have a warmer color tone to the overall image, which I think fits the image better, and out of those I think that #2 doesn't crunch the blacks as much. But I still like the face definition of #1 better -- on this one she is looking straight at the camera, while #2 she's looking slightly off to the right. So I'd pick #1.

But that's very nitpicky, they all seem very similar.

 

[Sepheyer]

Which of these do ppl prefer/find the best/least bad? and short note on why if possible.

Something along the process for 2 of the images has removed a strain of hair but left half the shadow which has made a somewhat interesting "running makeup" effect which is not intentional. Fixing it will literally that hours to do just to generate the images so try to ignore that fault.

View attachment 3065005 View attachment 3065004 View attachment 3065003 View attachment 3065002

(I'll update the post with the base prompt later as atm i don't want that to influence things in some way.)

Welp, from time to time I have these streak effects - I think of them as scars in anime or Witcher. So, I always discarded such images, sorry, I just can't recommend keeping any. Otherwise, ignoring the scar, keep #3.

 

[DD3DD]

Which of these do ppl prefer/find the best/least bad? and short note on why if possible.

Something along the process for 2 of the images has removed a strain of hair but left half the shadow which has made a somewhat interesting "running makeup" effect which is not intentional. Fixing it will literally that hours to do just to generate the images so try to ignore that fault.

View attachment 3065005 View attachment 3065004 View attachment 3065003 View attachment 3065002

(I'll update the post with the base prompt later as atm i don't want that to influence things in some way.)

#1 for stand alone due to eye contact with a serious tone
#2 for a scene not involving viewer
#4 has that subtle smile which draws you in with a sexy tone

 

[sharlotte]

For those that's been avoiding updating their nvidia drivers due to not wanting to deal with the potential usage of shared memory:

You must be registered to see the links

Well spotted and thanks for the info me3 ! I could not figure out why my generation of one image suddenly went to 35+it/s. Back to the usual 1.2it/s thanks to that change!

 

[rogue_69]

Is it possible to train a Lora model for a room? Like for instance, you have a room with images taken from different angles. You get several of the couch, the TV, the windows, etc. Then you use that Lora to place a character into that room. For instance, you have someone sit on the couch that was trained in the Lora.

 

[DreamingAway]

Is it possible to train a Lora model for a room? Like for instance, you have a room with images taken from different angles. You get several of the couch, the TV, the windows, etc. Then you use that Lora to place a character into that room. For instance, you have someone sit on the couch that was trained in the Lora.

Yes.. I've experimented with perspective training and actually had really good results.
As long as you don't overcook your train, the diffuser will understand the perspective your giving it then take over with the art style / context you want. This is the perfect use case for a LORA.

(although you can sometimes accomplish this with depth maps instead, as a sort of lazy way.. but I think that's too biased)