[Stable Diffusion] Prompt Sharing and Learning Thread

[Fuchsschweif]

I added "sticking tongue out" and "public hair" to see if the model could do that, the tongue (kinda) worked on the first one, public hair wasn't considered at all. How can I make SD not ignore prompts?

 

[Fuchsschweif]

Don't worry about merging checkpoints yet, before you get really familiar and proficient with SD. Instead download others merged checkpoints on civitai for now.

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:




It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:




So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

 

[Mr-Fox]

So the checkpoints are basically addons, and the loras are something like "prompt collections"?

I think your reply did overlap with my edit on my post above yours. It tried it first with only carrying the prompts over from civitai, but did get different results. Now with the data I extracted from your PNG the result is way closer (and takes way longer though).

How did you know what to add? I see you had the upscaler, denoising, refiner and all of that in your settings. Or, to put my question different: How would I gather these information from the original model page on civitai? Just the same way, downloading the picture and extracting the settings?

Funny is, in my own version (as above) when I added "nipples" they didn't show. Now with the extracted setting from the original it does. Any idea why?

I am asking because I of course want to understand the mechanisms behind it so that I can create more freely and not just copy and paste prompts all the time. So I have to understand for instance what changed with the extracted settings that made SD listen more closely to my input.

Aaaaand a last question: I can basically turn of the upscaler and only upscale pictures I like later on the extra page in order to safe time?

As I am typing this, the generation with the extracted settings from your PNG is done. It's quite close now, but the colors are still faded!

View attachment 2993328

You need this vae also. Forgot to tell you, sorry.

You must be registered to see the links

Place it inside Stable-Diffusion\stable-diffusion-webui\models\VAE .
It is for improving the face and eyes but it also can affect light and color.
On civitai you can find the generation data by enlarging the image and simply look on your right.

If you scroll all the way down you can find a button to copy all the generation data or you can choose to only copy the positive prompt for example by pressing the copy button to the right of the positive prompt. On civit the image doesn't contain the data unfortunately so you have to paste it in a txt document and then copy all the settings manually.
I have no idea why the nipples didn't show before but now they do, but my guess is that you switched checkpoint and this is more anime oriented. The basemodel does nudity but not as easy as the checkpoints on civitai.

 

[Mr-Fox]

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:

View attachment 2993499


It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:

View attachment 2993512


So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

Anime checkpoints are very specific to anime style. There are others that can do a little of everything. Such as Experience, Never ending dream, Dreamshaper, there are many. Yes you need to switch checkpoints depending on the style and the result you want. To get the pubic hair you can add weight to the this token (more than one word describing the same thing) like this:
pubic hair:1.2 or with brackets like this (pubic hair:1.2). Remember that brackets ads 0.1 on it's own so (pubic hair:1.2) is actually = 1.3 . A reason for it not showing could be because it's under a layer of clothing and SD doesn't understand you want the hair to be visible through the clothes. In that case you need to specify that the clothes are "sheer," or "see through" or "transparant".

 

[Mr-Fox]

Yes you can choose to not upscale the image now, you can use SD Upscale script in img2img instead. This is superior to "normal" upscaling in the extra tab. In the extra tab the upscaling is only enlarging the image and not adding more detail. The SD Upscale script is part of the generative process and thus creates new pixels wich ads more detail. I usually don't use hiresfix at first when I'm "searching" for a good image. When I "find " my image I simply re-use the seed from that image by pressing the green recycling button and then activate the hiresfix and then generates again. This is the most time effective way and it gives the absolute best result most of the time. Hiresfix is part of the generative process just like SD Upscale script, wich is an extension btw that you need to install. Go to the extension tab and available and press "load from". Then find SD Upscale and press "install", then press "apply and restart" in installed tab. Now you can find it in the scripts menu in img2img tab.



Now that SD restarted the prompts and settings are empty. Simply press the blue diagonal arrow button under "generate" to reset it or get back everything from before the restart.

 

[me3]

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:

View attachment 2993499


It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:

View attachment 2993512


So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

MJ does A LOT of stuff for you in the background, things you now need to do yourself in one way or another.
You're also comparing MJs massive data set that's been continuously tweaked, finetuned and curated to a SD base data set. (Both the sd15 base model and "anylora" is rather clean "untuned" models)
So in truth you're expecting a 1 year old to paint high quality art of concepts it's never heard of, seen or have any idea exist.

MJ wins on image "quality", simplicity in prompting (just throw words and it'll sort it out for you), but assuming you pick "the right tools for the job" SD will be more flexible, give you better control and not run off with all your money. With SDXL the quality gap is going away too.

 

[Fuchsschweif]

pubic hair:1.2 or with brackets like this (pubic hair:1.2). Remember that brackets ads 0.1 on it's own so (pubic hair:1.2) is actually = 1.3 . A reason for it not showing could be because it's under a layer of clothing and SD doesn't understand you want the hair to be visible through the clothes. In that case you need to specify that the clothes are "sheer," or "see through" or "transparant".

Ahh good to know the weight feature exists here too! What's the difference between using braces and using no braces (except that 0.1 value?).

But it still does ignore my input largely. See this example, I specified that she's supposed to lay on a bed, rear view angle, no underwear, even gave that weight (1.5):




My settings:




This was now without the upscaler which is why it is a bit faded and unsharp again. The VAE doesn't work yet it say not found, maybe I need to reload the web UI batch first.

 

[Fuchsschweif]

MJ wins on image "quality", simplicity in prompting (just throw words and it'll sort it out for you), but assuming you pick "the right tools for the job" SD will be more flexible, give you better control and not run off with all your money. With SDXL the quality gap is going away too.

Yes I realize this as we speak

What is SDXL?

And little side question: Are there already easy ways to add a bit of motion to still images with SD?

 

[Fuchsschweif]

Yes you can choose to not upscale the image now, you can use SD Upscale script in img2img instead.

So I pick "none" at upscaler right now? Or is one basic upscaler important for the base quality?

 

[me3]

Yes I realize this as we speak

What is SDXL?

And little side question: Are there already easy ways to add a bit of motion to still images with SD?

SDXL is the "next evolution" for SD, it's base image dimension goes up to 1024x1024, which i believe is still what MJ is at as well.
With your 1070 you should be able to use those models, but a lot "add ons" don't really work with it yet so you'll loose out on a lot of options for a while yet.
Main reason to stick with the older version is that there's more things that support it, it requires less in terms of hardware and as such is faster to use.

There are some "animator" scripts, not sure how well the work and what sort of requirements to have. Generally they work on the principle of a controlled series of images that gets stitched together like with old school animation. Not sure how much control you have, but i'm assuming it'll depend on the specific script and complexity of what you want to make.
Probably best to stick with the basics for a while, but future goals and all that

 

[Fuchsschweif]

Hiresfix is part of the generative process just like SD Upscale script, wich is an extension btw that you need to install. Go to the extension tab and available and press "load from". Then find SD Upscale and press "install", then press "apply and restart" in installed tab. Now you can find it in the scripts menu in img2img tab.

So when do I use SD Upscale? You said you like a picture, re-use the seed and then remake it with Hiresfix. But then there is no need for SD upscale anymore, or do you add that on top after that too?

Edit: Ah, is SD upscale just an available option now in the hiresfix dropdown menu? Didn't check it before installing SD upscale - I thought you'd apply SD upscale somewhere else because you said in the img2img tab.

SDXL is the "next evolution" for SD, it's base image dimension goes up to 1024x1024, which i believe is still what MJ is at as well.

Does that matter though if we can upscale as much as we want and if that upscaling is anyways a part of the generative process? What's the difference?

 

[me3]

Does that matter though if we can upscale as much as we want and if that upscaling is anyways a part of the generative process? What's the difference?

It's a "amount of detail" thing. Base image size is what was used to train on, so when it's 512x512 you can tell the person has skin and that it's a certain "color", etc, and yes you can upscale that to 2k or 4k it won't look stretched etc like with a texture but you'd still be limited to the "detail" from the original size.
I guess you can think of it as 2 squares of the same size, but one of them fits 512 pixels in each direction, the other fits 1024, one will have much smother and more details than the other.
If you want a RL example i guess printers work, you're limited to the same A4 sheet of paper, but the smaller each "dot" is the nicer and crisper the text or image you print gets.

The more i think about it the worse these analogies sounds and more bad ones i think of...

 

[Fuchsschweif]

I guess you can think of it as 2 squares of the same size, but one of them fits 512 pixels in each direction, the other fits 1024, one will have much smother and more details than the other.
If you want a RL example i guess printers work, you're limited to the same A4 sheet of paper, but the smaller each "dot" is the nicer and crisper the text or image you print gets.

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

 

[me3]

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

You can't add what you don't know about. It's not in the output this comes into play it's in the training. the images you train on have finer details.
More bad analogies. They both know you have skin, but one of them knows you have pores. They both know you have blue eyes, but one of them know there's multiple shades of blue and even some other colors mixed in.

 

[Jimwalrus]

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

Exactly - I'd recommend using HiResFix as part of the generative process, even if you only want a small image, rather than trying to just upscale later.
It can greatly improve the image, not just in terms of sharpness, but it's almost like a lot of extra generative steps, fine-tuning the result.
My recommendation is to get the aspect ratio you want by setting the smallest side as 512, then multiply that by the ratio yo get the other side's length (so W4xH5 would be 512x640). Then set HiResFix upscale level to whatever your GPU's vRAM can handle, select the upscaler (recommend you start with ESRGANx4 or similar as a 1st go, experiment with others later), set denoising strength to between 0.2 and 0.4, HiResFix steps to at least 150% of the generative steps and select Generate!
Experiment from there, you'll find the sweet spot for what you want to create.
Oh, and never bother with the SD1.5 original checkpoint for generating anything, always use a specialised one - far better. My preference is Cyberrealistic 3.2 for photorealism, Anything 4.5 for Western comics. I don't do anime, so can't guide you there.

 

[Jimwalrus]

One great advantage of doing generstive AI locally is you can run crazy numbers of tests - the only cost is running your PC. No tickets, tokens or subs.
If you look at the first posts in this thread there are links to posts about how to use features of SD.
A grest one to use for experimenting is X/Y/Z plot, lets you do side-by-side comparisons automatically.
We're all learning here as well, SD is only about 15 months old atm!

 

[Jimwalrus]

Further tip - grab one of the images from here*, drop it into PNGInfo and 'Send to Img2Img'. That will allow you to recreate it. You'll need to select the same Checkpoint and have the LoRAs /Textual Inversions installed but those should be available from Civitai.

*except those who generated using ComfyUI which isn't directly compatible with WebUI

 

[Fuchsschweif]

Then set HiResFix upscale level to whatever your GPU's vRAM can handle, select the upscaler (recommend you start with ESRGANx4 or similar as a 1st go, experiment with others later), set denoising strength to between 0.2 and 0.4, HiResFix steps to at least 150% of the generative steps and select Generate!

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

 

[Jimwalrus]

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

The upscaling is the most demanding part - if you look at the CMD window that's running in the background it will tell you what phase you're at and how long each iteration is taking. Gen steps normally show as being iterations per second, upscaling is usually seconds per iteration!
Try upscaling just a little less (e.g. x1.65 instead of x1.75).
If that doesn't work, it sounds like a motherboard problem or overheating SSD.

N.B. the generative elements of the HiResFix upscaling step work almost as well with x1.1 as they do with x2.2
You can then run your nice, clean creation through the standalone upscalers in the Extras tab to actually make it bigger.

 

[Sepheyer]

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

Before you grow too much into Automatic1111, do try ComfyUI - do what's called "portable install" by clicking "direct link to download":

You must be registered to see the links

.

The biggest selling point is that upscaling can be done as: latent -> latent -> latent rather than what A1111 offers of latent -> image -> latent -> image. I think this alone is ComfyUI's killer feature.

On top of it, every image created with CUI already contains the workflow that was used to create it. So, you can load this inside the CUI and immediately get the workflow that created the image: