[Stable Diffusion] Prompt Sharing and Learning Thread

[me3]

Clothing, hairstyle/length, "sizes" and loads of such things won't be consistent between seeds on the same model so only way they'll be consistent to a different model is if they are built on the same data and even then it's a matter of what else is involved.
Generate a few thousand images on the same prompt and you'll see there can be a massive variance within the same model.
It's also very common that words will be completely ignored, it doesn't mean the model "doesn't know them", if you look at the images and prompts shared just in this read you'll see it's rather common that there's things clearly specified in the prompt that's not in the image, but if you keep those in the prompt and remove other elements it'll "pop up".

If you suspect the model might not "understand" something you just need to run just that as a prompt (in some cases you might need a very minimal additional prompting), generate a somewhat large amount of images (can be small and low quality for speed) and see what the more consistent imagery is. You might end up being pretty surprised what actually works, i've had some more "technical/obscure" things show very well, while things we'd normally consider extremely basic fail completely.

If you want absolute consistency you're gonna need something trained, but relatively basic/common concept should work across models, specially if they are within the same "family". Generating across multiple models with multiple images in each, you'll see more variance within the seeds than models that's built on the same data. Unfortunately massive grids like that isn't easily shared here, but you're probably better off generating them yourself anyway so it fits your preferred models and way of prompting

 

[Mr-Fox]

Don't overlook a tool that is included in SD by default, prompt matrix. It's in the scripts menu. Very similar to xyz plot but instead of comparing generation settings you will compare prompt tags, lines or styles or artists names etc. Write your prompt and have the variables separated with vertical lines ( "|").

Example: |variable1|variable2|variable3| etc.

You must be registered to see the links

You must be registered to see the links

Spoiler: large image

You don't have permission to view the spoiler content. Log in or register now.

 

[devilkkw]

Clothing, hairstyle/length, "sizes" and loads of such things won't be consistent between seeds on the same model so only way they'll be consistent to a different model is if they are built on the same data and even then it's a matter of what else is involved.
Generate a few thousand images on the same prompt and you'll see there can be a massive variance within the same model.
It's also very common that words will be completely ignored, it doesn't mean the model "doesn't know them", if you look at the images and prompts shared just in this read you'll see it's rather common that there's things clearly specified in the prompt that's not in the image, but if you keep those in the prompt and remove other elements it'll "pop up".

If you suspect the model might not "understand" something you just need to run just that as a prompt (in some cases you might need a very minimal additional prompting), generate a somewhat large amount of images (can be small and low quality for speed) and see what the more consistent imagery is. You might end up being pretty surprised what actually works, i've had some more "technical/obscure" things show very well, while things we'd normally consider extremely basic fail completely.

If you want absolute consistency you're gonna need something trained, but relatively basic/common concept should work across models, specially if they are within the same "family". Generating across multiple models with multiple images in each, you'll see more variance within the seeds than models that's built on the same data. Unfortunately massive grids like that isn't easily shared here, but you're probably better off generating them yourself anyway so it fits your preferred models and way of prompting

For consistent hairstyles or clothes, a good way is use BREAK.
A simple example:
blonde braid haired BREAK woman BREAK wearing blue summer dress BREAK at the sea BREAK high quality

This work like a charm keeping hairstyles and dress. remember the BREAK need used with uppercase char.
Also use it in your negative give really different result, especially if you use long negative.
BREAK is use to subdivide concept, and in many of my test it works like these order:

hairstyles BREAK subject BREAK clothes BREAK background BREAK image details (cinematic, film still, etc...)

this is a simple sample and maybe it need adjusted by checkpoint you are using.

 

[Mr-Fox]

Don't overlook a tool that is included in SD by default, prompt matrix. It's in the scripts menu. Very similar to xyz plot but instead of comparing generation settings you will compare prompt tags, lines or styles or artists names etc. Write your prompt and have the variables separated with vertical lines ( "|").

Example: |variable1|variable2|variable3| etc.

You must be registered to see the links

You must be registered to see the links

Spoiler: large image

You don't have permission to view the spoiler content. Log in or register now.

I wanted to compare a few different media types or "styles" for a prompt. When attempting to use prompt matrix I discovered that it creates very many images (32). This would take too long so I reverted to something I have used in the past, S/R prompt in xyz plot. S/R (search and replace), it's an option in xyz plot that you can find in scripts menu. It finds a prompt tag or token in your prompt and replaces it with variations that you decide. It generates an image for each variation in a grid and induvidual images as well.

Example

Prompt: blue sailboat, ocean, sunset.

You set the first tag in the script that it should find in the prompt and then replace and the variations after separated with comma.

S/R: blue, red, wood, old pirate, small, large,

gives these variations:

-blue sailboat, ocean, sunset.
-red sailboat, ocean, sunset.
-wood sailboat, ocean, sunset.
-old pirate sailboat, ocean, sunset.
-small sailboat, ocean, sunset.
-large sailboat, ocean, sunset.

It doesn't matter where the tag or token is in the prompt, or if it's one word or a phrase.

 

[me3]

For consistent hairstyles or clothes, a good way is use BREAK.
A simple example:
blonde braid haired BREAK woman BREAK wearing blue summer dress BREAK at the sea BREAK high quality

This work like a charm keeping hairstyles and dress. remember the BREAK need used with uppercase char.
Also use it in your negative give really different result, especially if you use long negative.
BREAK is use to subdivide concept, and in many of my test it works like these order:

hairstyles BREAK subject BREAK clothes BREAK background BREAK image details (cinematic, film still, etc...)

this is a simple sample and maybe it need adjusted by checkpoint you are using.

Logically that would only really make a difference if there was something else in the prompt affecting it, IE in the sense of things getting the wrong color, your white top becoming blue, because of the blue shorts. It shouldn't really affect that the simple bikini keeps changing "style", nor that things like a braid or split in the hair keeps changing sides. Nor that your chin length hair changes length drastically between seeds.
Try a relatively simple thing like ombre hair and see how that changes between seeds.

A bit of a warning about using BREAK, there's a lot of extensions that highjack that word so make sure to check which of those are active. Most commonly i think this applies to extensions that allows sectioning or composing to render different prompts in different areas etc.

 

[me3]

As a followup, i'd say it doesn't seem like it has any real impact and definitely not what i'd class as "consistent".
Granted it's only 240 images, but there seems to be a fairly low amount of images that visibly has the three elements hair, top and pants correct
(recommend opening the image in a separate tab due to size, tried to keep it as clear as possible while keeping within upload limits)

Spoiler: prompt

You don't have permission to view the spoiler content. Log in or register now.

 

[me3]

For the sake of completeness, i ran basically the same prompt using an extension partitioning the image into 3 areas, top, middle, bottom.
Generally this make it easier to "force" things assuming what ever you are creating can actually fit with the "pieces" you make.
While it's considerably more consistent with the colors of top and pants, you see there's still some problems, hair is better too, but still quite a few misses. You can probably fix this with weighting as the prompts are limited to each section, unlike normal, so the weighting wouldn't be as likely to bleed into other areas. I didn't do that in this case because i tried to keep the prompt (and weighting) the same for comparison.
Image width (aspect) was changed because there was a lot of decapitations which made seeing the hair a bit of an issue.

 

[PandaRepublic]

Does anyone have a method using stable diffusion to upscale and denoise low quality photos?

 

[devilkkw]

try some upscaler in extra tab, some in this

You must be registered to see the links

maybe help you.
Step i use:
1)using a denoise model in extra tab and set upscale at 1.1 (using different denoise model based on type of image)
2)resend output image to extra tab and upscale it at 2x with good upscaler model
3)if look good keep, else resend 2x image to extra and redo step 1.

The trick is use denoise with low ratio upscale(1.1) and upscale 2x every time.
So if you want 6x upscale, you have to repeat step 3 times.

 

[Mr-Fox]

Does anyone have a method using stable diffusion to upscale and denoise low quality photos?

I would rather go with SD upscale extension than the normal upscale. It's very similar to hiresfix in that it not only makes th e image larger but also improves it and it's details. Going with a somewhat low denoise strength is probably necessary in order to keep as much of the original image as possible. Outside of SD I can recommend photoshop and it's smart sharping filter. This also has a denoising setting. Then there is Topaz Gigapixel, that is "AI" based photo enhancement software. It works but I'm not that impressed with it. It's worth a try though.

 

[Mr-Fox]

For the sake of completeness, i ran basically the same prompt using an extension partitioning the image into 3 areas, top, middle, bottom.
Generally this make it easier to "force" things assuming what ever you are creating can actually fit with the "pieces" you make.
While it's considerably more consistent with the colors of top and pants, you see there's still some problems, hair is better too, but still quite a few misses. You can probably fix this with weighting as the prompts are limited to each section, unlike normal, so the weighting wouldn't be as likely to bleed into other areas. I didn't do that in this case because i tried to keep the prompt (and weighting) the same for comparison.
Image width (aspect) was changed because there was a lot of decapitations which made seeing the hair a bit of an issue.

View attachment 2859189

There is an extension focused on colorbleed, SD webui Cutoff. It works but is not without flaws.

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

(tutorial)

 

[Synalon]

Just for those that haven't noticed yet, if you are using automatic1111 check extensions for StyleSelectorXL. It doesn't just work with SDXL, it lets you select styles and although you don't see it until the image is generated and you check the info it adds the prompts to turn it into the selected style.

If you are using SDXL but don't understand comfyui yet, there is also an extension to add the refiner. That way you can select the base model generate the image and have the refiner selected to add the final steps. Instead of having to switch between them.

 

[Artiour]

I'm thinking about training LoRA with only characters (png format background removed), has anyone tried it

 

[Dagg0th]

I'm thinking about training LoRA with only characters (png format background removed), has anyone tried it

Any particular reason why?

I mean, if you describe the background when you are captioning your images, the AI can discard it for you, and replace it for whatever you want when prompting, assuming you did not overcook the Lora, of course.

But as an experiment it should be very interesting, my Abbigail Lora then to put a lot of wooden walls when indoors, because the original images have a lot of wooden walls, they then to bleed it depending on weight and model, should be interesting to see what happens when no background is present in the original image.

 

[Artiour]

Any particular reason why?

I mean, if you describe the background when you are captioning your images, the AI can discard it for you, and replace it for whatever you want when prompting, assuming you did not overcook the Lora, of course.

But as an experiment it should be very interesting, my Abbigail Lora then to put a lot of wooden walls when indoors, because the original images have a lot of wooden walls, they then to bleed it depending on weight and model, should be interesting to see what happens when no background is present in the original image.

you pretty much explained why, but another reason is when resizing images to 1:1, I red an article that says to add white space to the short side of the image but I don't want that space to be in the generated images (I haven't tried it tho)

 

[rogue_69]

I'm thinking about training LoRA with only characters (png format background removed), has anyone tried it

Isn't it important to have things in the image that aren't the Lora subject? I've only trained a couple of embeddings, so I'm not sure.

 

[Mr-Fox]

It all comes down to what you want to do with the Lora. The use scenarios you have in mind dictates how you will train it. I don't see any need for having a background for the sake of the character or the Lora other than if it's helps with placing the character in 3d space. The checkpoint you are using can provide the background. For anyone who wants more separated control over both the character and the background as well as the general composition, latent couple is great for this. If you also combine it with for instance openpose then you have all the control you could ever need. Sounds very interesting to try a Lora without any background. I'd say go for it and let us know how you get on with it. If it doesn't work out you can always revert back to images with a background but with this gained knowledge.

 

[FreakyHokage]

anyone have a bestiality lora model to bless me with?

The only place that actually has any bestiality lora is pixai but all of their downloads are disabled by whoever created them.

Honestly, I hate when people do that shit. They make a lora or checkpoint that they know people will like and then disable it so nobody can use it.

 

[me3]

Due to taking (bad) advice when starting to train embeddings and loras, the images i used had a "fixed" background. The claimed reason was for consistency and to make it easier to "filter out". HOWEVER, due to the fact i've run hundres of training runs on those images trying to find the best settings and settings that would work for multiple sets, i easily started to notice things that remained the same over all of them. One of those things were the background. Yes you can filter it out in the training captions, but despite that there was clear signs that the background information was included in the finished training. In the case of background you can start noticing it if you generate images where a background isn't specified directly or by "scene". So in the case of having a "neutral" plain simple background the images would very often have a much of the same. IE just a plain single colored wall, making the images seem empty and dull.

When i changed to having random backgrounds, that was more natural and life like, the default backgrounds in generation also became more lively and detailed. It also seemed to have an effect on better lighting in images but probably mostly due to the more varied lighting in the images and the more natural setting there.
Also, when training, you don't actually need to caption things that is different in each image, but you do need to caption everything that is the same that you DON'T want to train. So in the case of a character/person, if the background is the same (even just empty) in all your images you have to caption it otherwise you get that linked to the character, but if the background is different in all of them the AI sees that this is a person in multiple different locations.
The same applies to other things as well, like expressions, poses, and even cloths in a lesser extent.

So as a tldr; keep everything you don't want to learn different in each image and the things you want to learn as close to the same as possible

 

[me3]

Out of boredom i thought i'd give Fooocus a try as it claims to run SDXL on pretty low specs and it seems to be true.
Images all got the same basic prompt, there's nothing in the filedata (far as i've seen atm), not much relevant info to give, if you just look at the ui it shouldn't be too hard to see why that is.
Anyway just thought i'd share some of the test results. Similar results can probably be created with other UIs on the SDXL model as there's nothing much besides a really simplified UI being used.

Spoiler: prompt

You don't have permission to view the spoiler content. Log in or register now.