[Stable Diffusion] Prompt Sharing and Learning Thread

[me3]

Yes I realize this as we speak

What is SDXL?

And little side question: Are there already easy ways to add a bit of motion to still images with SD?

SDXL is the "next evolution" for SD, it's base image dimension goes up to 1024x1024, which i believe is still what MJ is at as well.
With your 1070 you should be able to use those models, but a lot "add ons" don't really work with it yet so you'll loose out on a lot of options for a while yet.
Main reason to stick with the older version is that there's more things that support it, it requires less in terms of hardware and as such is faster to use.

There are some "animator" scripts, not sure how well the work and what sort of requirements to have. Generally they work on the principle of a controlled series of images that gets stitched together like with old school animation. Not sure how much control you have, but i'm assuming it'll depend on the specific script and complexity of what you want to make.
Probably best to stick with the basics for a while, but future goals and all that

 

[Fuchsschweif]

Hiresfix is part of the generative process just like SD Upscale script, wich is an extension btw that you need to install. Go to the extension tab and available and press "load from". Then find SD Upscale and press "install", then press "apply and restart" in installed tab. Now you can find it in the scripts menu in img2img tab.

So when do I use SD Upscale? You said you like a picture, re-use the seed and then remake it with Hiresfix. But then there is no need for SD upscale anymore, or do you add that on top after that too?

Edit: Ah, is SD upscale just an available option now in the hiresfix dropdown menu? Didn't check it before installing SD upscale - I thought you'd apply SD upscale somewhere else because you said in the img2img tab.

SDXL is the "next evolution" for SD, it's base image dimension goes up to 1024x1024, which i believe is still what MJ is at as well.

Does that matter though if we can upscale as much as we want and if that upscaling is anyways a part of the generative process? What's the difference?

 

[me3]

Does that matter though if we can upscale as much as we want and if that upscaling is anyways a part of the generative process? What's the difference?

It's a "amount of detail" thing. Base image size is what was used to train on, so when it's 512x512 you can tell the person has skin and that it's a certain "color", etc, and yes you can upscale that to 2k or 4k it won't look stretched etc like with a texture but you'd still be limited to the "detail" from the original size.
I guess you can think of it as 2 squares of the same size, but one of them fits 512 pixels in each direction, the other fits 1024, one will have much smother and more details than the other.
If you want a RL example i guess printers work, you're limited to the same A4 sheet of paper, but the smaller each "dot" is the nicer and crisper the text or image you print gets.

The more i think about it the worse these analogies sounds and more bad ones i think of...

 

[Fuchsschweif]

I guess you can think of it as 2 squares of the same size, but one of them fits 512 pixels in each direction, the other fits 1024, one will have much smother and more details than the other.
If you want a RL example i guess printers work, you're limited to the same A4 sheet of paper, but the smaller each "dot" is the nicer and crisper the text or image you print gets.

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

 

[me3]

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

You can't add what you don't know about. It's not in the output this comes into play it's in the training. the images you train on have finer details.
More bad analogies. They both know you have skin, but one of them knows you have pores. They both know you have blue eyes, but one of them know there's multiple shades of blue and even some other colors mixed in.

 

[Jimwalrus]

That would be true for simple resizing (stretching) of the source material, but the upscalers do generative upscaling which means they add new details and pixels as they do, don't they?

Exactly - I'd recommend using HiResFix as part of the generative process, even if you only want a small image, rather than trying to just upscale later.
It can greatly improve the image, not just in terms of sharpness, but it's almost like a lot of extra generative steps, fine-tuning the result.
My recommendation is to get the aspect ratio you want by setting the smallest side as 512, then multiply that by the ratio yo get the other side's length (so W4xH5 would be 512x640). Then set HiResFix upscale level to whatever your GPU's vRAM can handle, select the upscaler (recommend you start with ESRGANx4 or similar as a 1st go, experiment with others later), set denoising strength to between 0.2 and 0.4, HiResFix steps to at least 150% of the generative steps and select Generate!
Experiment from there, you'll find the sweet spot for what you want to create.
Oh, and never bother with the SD1.5 original checkpoint for generating anything, always use a specialised one - far better. My preference is Cyberrealistic 3.2 for photorealism, Anything 4.5 for Western comics. I don't do anime, so can't guide you there.

 

[Jimwalrus]

One great advantage of doing generstive AI locally is you can run crazy numbers of tests - the only cost is running your PC. No tickets, tokens or subs.
If you look at the first posts in this thread there are links to posts about how to use features of SD.
A grest one to use for experimenting is X/Y/Z plot, lets you do side-by-side comparisons automatically.
We're all learning here as well, SD is only about 15 months old atm!

 

[Jimwalrus]

Further tip - grab one of the images from here*, drop it into PNGInfo and 'Send to Img2Img'. That will allow you to recreate it. You'll need to select the same Checkpoint and have the LoRAs /Textual Inversions installed but those should be available from Civitai.

*except those who generated using ComfyUI which isn't directly compatible with WebUI

 

[Fuchsschweif]

Then set HiResFix upscale level to whatever your GPU's vRAM can handle, select the upscaler (recommend you start with ESRGANx4 or similar as a 1st go, experiment with others later), set denoising strength to between 0.2 and 0.4, HiResFix steps to at least 150% of the generative steps and select Generate!

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

 

[Jimwalrus]

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

The upscaling is the most demanding part - if you look at the CMD window that's running in the background it will tell you what phase you're at and how long each iteration is taking. Gen steps normally show as being iterations per second, upscaling is usually seconds per iteration!
Try upscaling just a little less (e.g. x1.65 instead of x1.75).
If that doesn't work, it sounds like a motherboard problem or overheating SSD.

N.B. the generative elements of the HiResFix upscaling step work almost as well with x1.1 as they do with x2.2
You can then run your nice, clean creation through the standalone upscalers in the Extras tab to actually make it bigger.

 

[Sepheyer]

My computer did shut off two times yesterday after using SD for a longer time, it both times happened while the "ultra sharp 4x" upscaling IIRC. I checked my GPU temperature meanwhile and it was fine (65°C), CPU usage was very low - it's really weird. I had to wait a couple of minutes both times until it was able to reboot, for the first minutes it was completely shut, as if it overheated.

Before you grow too much into Automatic1111, do try ComfyUI - do what's called "portable install" by clicking "direct link to download":

You must be registered to see the links

.

The biggest selling point is that upscaling can be done as: latent -> latent -> latent rather than what A1111 offers of latent -> image -> latent -> image. I think this alone is ComfyUI's killer feature.

On top of it, every image created with CUI already contains the workflow that was used to create it. So, you can load this inside the CUI and immediately get the workflow that created the image:

 

[Fuchsschweif]

Before you grow too much into Automatic1111, do try ComfyUI - do what's called "portable install" by clicking "direct link to download":

You must be registered to see the links

.

The biggest selling point is that upscaling can be done as: latent -> latent -> latent rather than what A1111 offers of latent -> image -> latent -> image. I think this alone is ComfyUI's killer feature.

On top of it, every image created with CUI already contains the workflow that was used to create it. So, you can load this inside the CUI and immediately get the workflow that created the image:

View attachment 2994900

Interesting, and what would be the disadvantages towards A1111? It sounded to me that A1111 is the better deal..

 

[Sepheyer]

Interesting, and what would be the disadvantages towards A1111? It sounded to me that A1111 is the better deal..

Only you eventually know if you prefer apples or oranges, despite one crowd saying apples are it and another crowd saying oranges are it. Hence you kinda won't know until you try both.

 

[Fuchsschweif]

Only you eventually know if you prefer apples or oranges, despite one crowd saying apples are it and another crowd saying oranges are it. Hence you kinda won't know until you try both.

Yeah but objectively: What can A1111 do what comfyUI can't? (What would I be missing out on?)
I must know that before I can make a decision

 

[Fuchsschweif]

Guys, I have this picture and I wanted to change the private part with inpaint and add public hair. But SD goes totally nuts and does weird stuff:



This is what it generated.. I just added "public hair".



Is it something with my settings?

 

[Fuchsschweif]

Also, how can I use any picture and just convert it into an AI picture? Let's say I find a real picture I like and want SD to replicate it as close as possible just in the style of the chosen checkpoint or even lora, how do I do this?

image2image seems not to work, I always get weird results no matter the denoising setup. It's always adding stuff.

 

[Mr-Fox]

I understand that you are new to SD and excited to try all kinds of things. If MJ is windows and Dal e is Apple, then SD is Linux. It will take some time for you to figure out everything so try to focus on one thing until you have a good grasp on it before you explore new workflows and concepts etc. The token merging you saw in the override settings is something you inherited from my setup. It's for speeding up the generation iterations. You can set this in settings/optimization. The VAE needs to be placed in the folder I told you, models/VAE.
Then you can select the VAE in settings. Yes it's likely that you need to "apply restart" if something isn't working correctly.
You activate hiresfix and deactive it simply by pressing the drop down arrow of the hiresfix window. I find that most people doesn't really understand upscaling and how it works in SD. Hiresfix is more than simply upscaling it ads "hires steps", thus it creates new pixels that gives more detail. The same is true for the SD Upscale Extension in img2img tab. "Normal" upscaling in img2img and extra tab is without the SD Upscale extention. It only upscales and doesn't add new pixels, it only enlarge what is already there. The result is a larger image but with loss of sharpness and detail. My preferred workflow is that I work in the prompt and use random seed (-1) and when my prompt is done I generates images until I find a good seed with the result of a good image, then I set this seed as my static seed by pressing the green recycle button. Now it will display the specific seed. I will then activate hiresfix by pressing the drop down arrow and then generates my image again but this time with hiresfix activated. This is all I do most of the time. If you want to explore going further then you can upscale it further in img2img tab with SD Upscale extension. More is not always better. Even using SD Upscale extension can result in loss of detail if you already used hiresfix. It is diminishing returns. I would recommend to instead use photoshop to "upscale" the second time by going to image size and double it and make sure resampling is activated. This will result in a larger image with the composition and details intact. You will not gain details as with hiresfix or SD Upscale Extension but you will not lose any detail or composition, that can happen when you upscale a second time with SD. These are rule of thumb not cut in stone, it depends on case scenario

Extra tip.
When you wish to share prompt or generation data in this thread it's better to simply go to the output folders and upload the png file with intact generation data instead of using screenshots. If you wish to refer to a specific setting, yes then ofc it's good with a screenshot. With your png file I can simply load the data with PNG Info tab and help you much faster that way if the topic is trouble shooting prompts or settings.

 

[Jimwalrus]

Guys, I have this picture and I wanted to change the private part with inpaint and add public hair. But SD goes totally nuts and does weird stuff:

View attachment 2995125

This is what it generated.. I just added "public hair".

View attachment 2995127

Is it something with my settings?

View attachment 2995129

Hang on, are you actually typing "public hair" - 'cos there's your problem!

If you are typing "pubic", try modifiers such as "(pubic hair:1.4)" and/or "pubes:1.5"

That being said, bushy pubes are a perennial blindspot with SD.
Try using MSPaint (or drawing program of your choice) to add a block of the desired colour in the approx size and position before then going to Inpaint and telling it to draw pubes where that block of colour is.

 

[Fuchsschweif]

Hang on, are you actually typing "public hair" - 'cos there's your problem!

If you are typing "pubic", try modifiers such as "(pubic hair:1.4)" and/or "pubes:1.5"

Woops! I never noticed it's spelled differently actually. But I also tried "hairy pussy" nevertheless, SD really seems to struggle with that.

Why would adding a block of color help the program more than painting the suggested area with inpaint?

 

[Mr-Fox]

Tbh I don't bother with inpaint most of the time, I try to get it right from the start instead. But if you wish to use inpaint the seed is important, the denoising is important and the inpaint method, meaning if you use "fill" or original etc. Original uses the same latent noise as the original image and fill adds new. Most of the time it's best to use a random seed for the result but sometimes you want to use the same seed. Don't use cfg 30, it will not give a better result. Set it to 6-9, it's enough most of the time only rarely or specific scenarios is it any use to go higher and then I would recommend to not go higher than 16 ish. If you still don't get the result from the prompt then something else is at fault. Uplodad the png file with intact generation data and it will be easier for us to help you further with inpainting.