[Stable Diffusion] Prompt Sharing and Learning Thread

[Fuchsschweif]

Should check the temp on it just to be sure as it's fairly high load on is known to cause a fair bit of temp increase

Already did that, it never goes over 65°C. CPU is also verrry bored. I didn't look at the VRAM usage specifically yet. But..

If you are limited to 16Gb of ram I will perhaps suggest the obvious thing here. As soon as you can afford it, invest in 2 pair more sticks. I did and it has been tremendous.

... since I ever saw my main ram usage yet while running SD at 5 out of 16gb, I doubt that it's due to overloaded ram that the VRAM handled over.

Anyways.. if I will run into the same issues with comfyUI it will most likely be something in my setup. But I doubt it, since all other programs run very well, even those with high demands, especially in terms of RAM. I work with post processing programs in other fields that sometimes consume my ram up to 90% without any crashes.

 

[hkennereth]

Does ComfyUI run on a web version too or can I run that completely locally?

It's a local running software, but you open it on your browser. Just like A1111.

 

[Mr-Fox]

Hiresfix is very demanding on the video memory and it's very easy to run out and get oom error. I have done it a bunch until I learned to not push it and instead use other options. Let's suppose you can generate a portrait ratio image of 640x960, then uspcale 2x with hiresfix. You now have 1280x1920. Send it over to img2img. Instead of using the 1280x1920 when upscaling, set it to 640x960 because this slider is the tile size when using SD Upscale. Set the upscale multiplier to 3 or 4 if you so wish.
In the end you will have 3840x5760 because the upscaler will multiplie the image 3 times not the tile. Hope it makes sense. SD Upscale script will calculate how many tiles and batches that is required. The sample steps also will increase the detail and sharpness so use at least double the amount of steps to what the txt2img sample steps were. If yo used 30 then use 60 in img2img.

 

[Fuchsschweif]

In the end you will have 3840x5760 because the upscaler will multiplie the image 3 times not the tile. Hope it makes sense. SD Upscale script will calculate how many tiles and batches that is required. The sample steps also will increase the detail and sharpness so use at least double the amount of steps to what the txt2img sample steps was. If yo used 30 then use 60 in img2img.

Is it quality wise the same output just done more efficient, or is the SD script a bit inferior compared to hiresfix?

 

[Sepheyer]

Is it quality wise the same output just done more efficient, or is the SD script a bit inferior compared to hiresfix?

Do give ComfyUI a try, seeing how long you've been struggling with it with a1111.

 

[Jimwalrus]

Apropos the discussion around Upscalers, I've run some comparisons, using 3 Upscalers I've used before and one that has always just given terrible results*.
There are 20 images in total - 4 just using each Upscaler in HiRes fix (45 sampling steps, 60 Hires steps, denoising 0.33), then I ran each of those four images through 3x Upscaling in Extras for each of the four Upscalers.

N.B. These images are intended to be as 'photo-realistic' as possible, other "hyper-realism" or cartoon styles may give very different results.
I had ADetailer set for both face and hands, there's also kkw-skindet in the embeddings too.

*ESRGAN_4x, 4x-UltraSharp, 4xNMKDSuperscale & R-ESRGAN 4x+. The latter is terrible for photo-realism.

Looking at them, my personal preference seems to be either ESRGAN_4x for HiRes fix then 4xNMKDSuperscale for upscaling in Extras or 4xNMKDSuperscale for both.

The finished images are too large to upload direct, so

You must be registered to see the links

a RAR.

The initial Hires fixed images are below:


N.B. For some reason, 4xNMKDSuperscale shows in my A1111 as "4xNMKDSuperscale_4xNMKDSuperscale". I've no idea why, but I've kept it in the filenames for completeness

 

[Jimwalrus]

Do give ComfyUI a try, seeing how long you've been struggling with it with a1111.

 

[me3]

...so close



yet, sooooo far

 

[Sepheyer]

...so close
View attachment 3012625
View attachment 3012624

yet, sooooo far
View attachment 3012623

Did you look into IPAdapter? This is just something that I learned about 2 seconds ago, yet I am strongly convinced this is the way forward rather than LORAs. Sorry I don't have anything to backup my words with other than my enormous enthusiasm for IPA after the few videos that I saw. I am slowly making my way towards amending the workflow to incorporate IPA, and should be there "soon" but not at this moment.

Among multiple videos about IPA here is the one that kinda sold me:

You must be registered to see the links

on the idea. Note, that CUI's IPA has a unique implementation where you can add noise to the image prompt. Whoa! So noise! Much image!

 

[me3]

Did you look into IPAdapter? This is just something that I learned about 2 seconds ago, yet I am strongly convinced this is the way forward rather than LORAs. Sorry I don't have anything to backup my words with other than my enormous enthusiasm for IPA after the few videos that I saw. I am slowly making my way towards amending the workflow to incorporate IPA, and should be there "soon" but not at this moment.

Among multiple videos about IPA here is the one that kinda sold me:

You must be registered to see the links

on the idea. Note, that CUI's IPA has a unique implementation where you can add noise to the image prompt. Whoa! So noise! Much image!

according to their own words it won't work well with facial details. It will work for it but it'll apparently not handle "fine details" so that kind defeats much of the purpose and as such it'll fail as a replacement.
It also seems to be dependent on controlnet which makes me wonder about memory needs and compatibility and you actually need to train it "in a different way" which for me raises question about actually having the system for it.

Edit:
It also seems to purely be "replacement" which in no way will replace the point of loras and there's already a selection of those things

 

[Sepheyer]

according to their own words it won't work well with facial details. It will work for it but it'll apparently not handle "fine details" so that kind defeats much of the purpose and as such it'll fail as a replacement.
It also seems to be dependent on controlnet which makes me wonder about memory needs and compatibility and you actually need to train it "in a different way" which for me raises question about actually having the system for it.

Edit:
It also seems to purely be "replacement" which in no way will replace the point of loras and there's already a selection of those things

It is a "latent replacement". More like "latent injection", which is really pretty cool. And in one form or another this will eventually provide some augmentation to the textual prompting. On the other hand the "plus" model handles 16 tokens (or was it eight?) so that makes me go hmm wondering if this is enough. Like, IDK how much a token represents, but I'd think to represent a face one needs a few thousand.

 

[Sepheyer]

So, after a first few trials of the IP Adapter I am realizing the thing ain't quite what it was described as in the videos. Namely videos handled abstract scenarios, where applying the same tool to the real life problem of say generating full-height character is not there yet. So, imma be trying a bit more, aight. The goal is really to transfer Lara Croft's face to a full body person. In more scientific terms, I am hoping to inject LC face latent into the model stream, just like Open Pose control net injects its pose into the model stream. Naturally, the next few days are gonna be: tests,tests, tests. Meanwhile, if anyone is into this, here is the source image and the resulting images (they are CUI workflows as well):

Spoiler: Illustration

You don't have permission to view the spoiler content. Log in or register now.

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

Like, this is not horrible, this is pretty good. For imaginary uses, like character aging over time, this is fantastic. I.e. the facial feature shapes are identical from picture to picture. I would completely believe that this is one and the same person at different phases of her life.

It is just - liekly due to me just starting the tests and being a moron on top of it - completely useless for my purposes.

Thinking aloud: I probably need to try the other workflows along the img2img pipeline. Yea, I'll head there once I snatch my motivation from the jaws of procrastination.

 

[Sepheyer]

Tho... Great tool for bringing your DAZ/HS2/VAM waifus to life:

Source: Honey Select 2 render	Sampler: Euler Ancestral Woman, Teenager	Sampler: Euler Ancestral Woman, Milf
Sampler: Heun Woman, Supermodel	Sampler: Heun Woman	Sampler: Heun Woman, MILF
Sampler: Heun Various embellishments	Sampler: Heun Different mixing proportions	Sampler: Heun Different mixing proportions
Sampler: Heun Different mixing proportions	Sampler: Heun Different mixing proportions	Sampler: Heun Different mixing proportions

 

[me3]

So, after a first few trials of the IP Adapter I am realizing the thing ain't quite what it was described as in the videos. Namely videos handled abstract scenarios, where applying the same tool to the real life problem of say generating full-height character is not there yet. So, imma be trying a bit more, aight. The goal is really to transfer Lara Croft's face to a full body person. In more scientific terms, I am hoping to inject LC face latent into the model stream, just like Open Pose control net injects its pose into the model stream. Naturally, the next few days are gonna be: tests,tests, tests. Meanwhile, if anyone is into this, here is the source image and the resulting images (they are CUI workflows as well):
View attachment 3012988

This reminded me of something which happened during all my attempts at training that.
When captioning the images, in this case with blip2, all of the images got captioned in the way you'd expect, woman clothing/description setting ....except one of them. All it gave as caption was "lara croft sexy png" (no it wasn't an actual png file). When i included an instruction to describe the "look" of the person in the picture, all i got was "sexy"...

 

[Mr-Fox]

Is it quality wise the same output just done more efficient, or is the SD script a bit inferior compared to hiresfix?

I think the difference is somewhat small but hiresfix gives clearly superior results. It is however very time inefficient. It is very necessary though since the bigger the resolution you get in txt2img the more detail and sharpness you will have in the end. The advantage it has is that it is part of the first generative process and has more impact on the image. Later upscaling doesn't have nearly as large effect and you quickly run into diminishing returns. Unless you have specific resolution target I think that most of the time it's enough with hiresfix. You can tease out the rest of the details with photoshop. Since you are looking for clarity I highly recommend that you learn to be very familiar with the camera raw filter in photoshop. It even has a specific slider named clarity. I have done a small guide or rather an overview of photoshop filters before but I might do a more detailed guide on "camera raw filter" and "smart sharpness" .

My small overview of a few filters and what not in photoshop
https://f95zone.to/threads/stable-diffusion-prompt-sharing-and-learning-thread.146036/post-10174688

Small overview of nerual filters in photoshop.
https://f95zone.to/threads/stable-diffusion-prompt-sharing-and-learning-thread.146036/post-10175271

 

[Mr-Fox]

Apropos the discussion around Upscalers, I've run some comparisons, using 3 Upscalers I've used before and one that has always just given terrible results*.
There are 20 images in total - 4 just using each Upscaler in HiRes fix (45 sampling steps, 60 Hires steps, denoising 0.33), then I ran each of those four images through 3x Upscaling in Extras for each of the four Upscalers.

N.B. These images are intended to be as 'photo-realistic' as possible, other "hyper-realism" or cartoon styles may give very different results.
I had ADetailer set for both face and hands, there's also kkw-skindet in the embeddings too.

*ESRGAN_4x, 4x-UltraSharp, 4xNMKDSuperscale & R-ESRGAN 4x+. The latter is terrible for photo-realism.

Looking at them, my personal preference seems to be either ESRGAN_4x for HiRes fix then 4xNMKDSuperscale for upscaling in Extras or 4xNMKDSuperscale for both.

The finished images are too large to upload direct, so

You must be registered to see the links

a RAR.

The initial Hires fixed images are below:
View attachment 3012589 View attachment 3012590 View attachment 3012591 View attachment 3012592

N.B. For some reason, 4xNMKDSuperscale shows in my A1111 as "4xNMKDSuperscale_4xNMKDSuperscale". I've no idea why, but I've kept it in the filenames for completeness

You should do a comparison with only the denoising strength settings as variable. I have found that it has a very big effect on the result. If you set this wrong you could get a very poor result and think that it's the upscaler's fault when in actuality you had wrong denoising strength. The better comparison would have been either with hiresfix or with SD Upscale script. The normal upscaling in either img2img or extras tab doesn't add more detail or sharpness since it's not part of the generative process it only enlargens the image, it lacks the hires steps that both hiresfix and the SD Uspcale extension has.

 

[Jimwalrus]

You should do a comparison with only the denoising strength settings as variable. I have found that it has a very big effect on the result. If you set this wrong you could get a very poor result and think that it's the upscaler's fault when in actuality you had wrong denoising strength. The better comparison would have been either with hiresfix or with SD Upscale script. The normal upscaling in either img2img or extras tab doesn't add more detail or sharpness since it's not part of the generative process it only enlargens the image, it lacks the hires steps that both hiresfix and the SD Uspcale extension has.

I did want to add the denoising strength as an option but it would've been excessively time-consuming. I think I might do it tomorrow, but without the additional step of Extras upscaling.
It's very likely true that different denoising strengths suit different upscalers better. 0.33 is possibly very high for some.
As I'll only be doing the three decent ones from the last test (not going to waste my time and leccy on R-ESRGAN-4x+ any more), does anyone have a request for a fourth upscaler to take its place?
The plan will be each upscaler at 0.15, 0.2, 0.25, 0.3 & 0.4, other conditions as before to allow comparison and to allow the previous 0.33 to stand in the lineup.

The reason I included the Extras upscaling was to address exactly your point - I think I've proved us correct!

 

[Mr-Fox]

I did want to add the denoising strength as an option but it would've been excessively time-consuming. I think I might do it tomorrow, but without the additional step of Extras upscaling.
It's very likely true that different denoising strengths suit different upscalers better. 0.33 is possibly very high for some.
As I'll only be doing the three decent ones from the last test (not going to waste my time and leccy on R-ESRGAN-4x+ any more), does anyone have a request for a fourth upscaler to take its place?
The plan will be each upscaler at 0.15, 0.2, 0.25, 0.3 & 0.4, other conditions as before to allow comparison and to allow the previous 0.33 to stand in the lineup.

The reason I included the Extras upscaling was to address exactly your point - I think I've proved us correct!

I use NMKD-Face very often. Even though I have talked about it many times here I don't see many others use it. Since it's common that people wants detailed faces I think this is a very good candidate. The problem with NMKD is though that they have many versions of the same upscaler.

 

[Jimwalrus]

I use NMKD-Face very often. Even though I have talked about it many times here I don't see many others use it. Since it's common that people wants detailed faces I think this is a very good candidate. The problem with NMKD is though that they have many versions of the same upscaler.

View attachment 3013616

So it would be SP_178000_G?
Also, one of the reasons I like to test on topless photos (apart from the obvious eye candy) is that the nipples and areolae are a distinct area of skin that's different to elsewhere but without a dedicated function to improve it.

Unless anyone knows of an ADetailer equivalent for nipples?

 

[Mr-Fox]

So it would be SP_178000_G?
Also, one of the reasons I like to test on topless photos (apart from the obvious eye candy) is that the nipples and areolae are a distinct area of skin that's different to elsewhere but without a dedicated function to improve it.

Unless anyone knows of an ADetailer equivalent for nipples?

No 8x_NMKDFaceExtended_100000_G the other one just happened to be selected.

Yes there are other models for After Detailer.


You can download the extras from my gofile. Unzip in Stable-Diffusion\stable-diffusion-webui\models.

You must be registered to see the links

Becides ADetailer there are plenty of loras and embedings.

You must be registered to see the links

Here's a selection:

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links