[Stable Diffusion] Prompt Sharing and Learning Thread

[hkennereth]

Alright, you can notice that some AI images have something odd about them. Congrats, but so does everybody.

I'll just focus on this one paragraph below because... honestly, you wrote a lot but said very little, and this is honestly my last attempt at addressing this particular subject.

So regarding the images you claim i had issues with, the person posting them found them good enough that they past their selection for posting and when i see things about them that i like, it means that what ever the "wrong" i might see is something else, so it's "comforting" (for a lack of a better term).

Buddy, I didn't claim you had issues with some images. You can tell us that within your heart of hearts you loved them all along but you, using your own words, described in detail the issues you had with those images, which is what we have been focusing all along. It's all right there on your first reply:

This seems to have the same "issues" as other "convert to real" methods. It converts the pose, background is fairly alike and the character is wearing relatively the same clothing, hair etc. However the face isn't really the all that close.
The face your render character has isn't in any way a "unrealistic anime" shape or features, yet when you look at the "real" version it hasn't even kept the basic shape of the face. The face is more rounded and "shorter", chin is different, eyes, lips, some of this could be prompting related sure, but AI is mean to be good at reading faces (scanners/cameras etc), but for things like this is doesn't seem to keep even the proportions "correct", which is exactly what is used for comparing faces.

However in the end you seem to be the one missing the point: the images that Sepheyer posted weren't shared because to show off their amazing skills, or because they thought or claimed they were perfect and flawless; they were shared because this is, and I quote, a "prompt sharing and learning thread". It was to share a technique that worked at some level for them.

Were the results perfect? There's no such thing. Were they bragging? Nope. Did they claim those specific images passed some specific quality criteria? No, they were just decent demonstrations of the specific technique. Are they end-all solutions for the problem they tackle? Again, no such thing exist. The whole point of this entire exercise, from their original post to every reply we offered was because we were hoping to help you perhaps learn something about image creation with AI, because most of the issues you described can be solved or at least minimized, when addressing them is one's focus. Again, they were clearly not the focus of the original images, hence the replies you received pointing out that the "issues" you listed were irrelevant.

Now, instead of describing your super powers of visual perception, you could have saved us time all along had you just said that you didn't really want to learn anything, you were just offering unconstructive criticism, that way we would have not bothered trying to explain why they were not really relevant on that instance. I'm assuming that's what you meant with your entire post, but maybe I'm once again missing the subtext behind whatever it was you said there. I do that a lot, it seems, but then again I'm more of a "say what you mean and mean what you say" kind of guy...

 

[Jimwalrus]

Awesome work Jim. Yes the name really does trip on the tongue rather than roll.. Since NMKD has many different versions of the same "model" of upscaler I thought it best I included the full name to avoid confusion. The difference in the test is subtle but it becomes more clear with higher resolution and at the later stage if one choose to upscale further in img2img with SD Upscale. I find that the fine details in the eyes iris and similar details gets lost with too high denoising strength. For instance skin texture or hair strands. What I have learned so far is that you want to get as much fine detail as early in the process as possible and then you need to preserve them in the steps after, as much as possible. A bad upscale can completely ruin a good image while a good one can take it to the next level.
At some point diminishing returns will become a factor and eventually there is a limit to the current technology. It's a lot of fun to push the envelope though and see how far we can stretch those limits.

I have noticed however that 8xNMKDFacesExtended takes about 0.6s longer per iteration the ESRGAN_4x on my 3060, adding a little to the generation time (~36s for 60 HiRes steps).
I don't think though that enough time would be saved across normal batch quantities to consider using a quicker Upscaler for the initial batch, then re-running the best with a better one. Batch size would have to be in excess of 40 or so if you include the time to reset before the rerun.

 

[hkennereth]

Awesome work Jim. Yes the name really does trip on the tongue rather than roll.. Since NMKD has many different versions of the same "model" of upscaler I thought it best I included the full name to avoid confusion. The difference in the test is subtle but it becomes more clear with higher resolution and at the later stage if one choose to upscale further in img2img with SD Upscale. I find that the fine details in the eyes iris and similar details gets lost with too high denoising strength. For instance skin texture or hair strands. What I have learned so far is that you want to get as much fine detail as early in the process as possible and then you need to preserve them in the steps after, as much as possible. A bad upscale can completely ruin a good image while a good one can take it to the next level.
At some point diminishing returns will become a factor and eventually there is a limit to the current technology. It's a lot of fun to push the envelope though and see how far we can stretch those limits.

I completely agree. NMKD's upscalers are really great for some things, but sometimes can fail at some different scenarios. I haven't tried this particular one yet, but so far the only one I was really happy with as a good all-arounder was Siax. But even that I noticed can introduce some noise in some edge cases, so I always keep 4x-UltraSharp close by because I find that it always gives decent results no matter the case.

 

[me3]

Awesome work Jim. Yes the name really does trip on the tongue rather than roll.. Since NMKD has many different versions of the same "model" of upscaler I thought it best I included the full name to avoid confusion. The difference in the test is subtle but it becomes more clear with higher resolution and at the later stage if one choose to upscale further in img2img with SD Upscale. I find that the fine details in the eyes iris and similar details gets lost with too high denoising strength. For instance skin texture or hair strands. What I have learned so far is that you want to get as much fine detail as early in the process as possible and then you need to preserve them in the steps after, as much as possible. A bad upscale can completely ruin a good image while a good one can take it to the next level.
At some point diminishing returns will become a factor and eventually there is a limit to the current technology. It's a lot of fun to push the envelope though and see how far we can stretch those limits.

I have noticed however that 8xNMKDFacesExtended takes about 0.6s longer per iteration the ESRGAN_4x on my 3060, adding a little to the generation time (~36s for 60 HiRes steps).
I don't think though that enough time would be saved across normal batch quantities to consider using a quicker Upscaler for the initial batch, then re-running the best with a better one. Batch size would have to be in excess of 40 or so if you include the time to reset before the rerun.

Given the 2x, 4x and 8x naming pattern used, is this in reference to a recommended upscaling amount?
In a 2x upscaling, would a 4x version would be better suited than 8x?

I can't run large highres upscalings to test myself so i've not done much with it

 

[hkennereth]

Given the 2x, 4x and 8x naming pattern used, is this in reference to a recommended upscaling amount?
In a 2x upscaling, would a 4x version would be better suited than 8x?

I can't run large highres upscalings to test myself so i've not done much with it

That's correct. If you run the upscaler without sizing parameters (I'm not sure that's possible on A1111, but it's the default behavior on ComfyUI), that value is how much the image will be upscaled.

 

[me3]

Since there was some new upscalers mentioned i wanted to test and it gave a nice reminder that highres testing is not for me
2x upscaling a 512x512 image took just 3min, a 768x768 took 13min, and it overflowed into shared memory by 11gb on the final step/percentage generating.
3x upscaling on 512x512 took 12,5 min...so yea, not really sure i'm gonna bother with anything higher.
Seems it was a long wait for more than just me, even the woman in the image went old and grey

On a more serious note, look a her hair about a bit below her chin, at first i thought something was wrong with the upscaling, but i've included a non upscaled version i generated afterwards and it's there too.
Third image is the 2x upscale, different seed and the same type of "framing" is there too, on the left side.



Fault in model data?

(Edit: added a enlarged section incase it's difficult to see on the forum)

 

[hkennereth]

Since there was some new upscalers mentioned i wanted to test and it gave a nice reminder that highres testing is not for me
2x upscaling a 512x512 image took just 3min, a 768x768 took 13min, and it overflowed into shared memory by 11gb on the final step/percentage generating.
3x upscaling on 512x512 took 12,5 min...so yea, not really sure i'm gonna bother with anything higher.
Seems it was a long wait for more than just me, even the woman in the image went old and grey
View attachment 3016726
On a more serious note, look a her hair about a bit below her chin, at first i thought something was wrong with the upscaling, but i've included a non upscaled version i generated afterwards and it's there too.
Third image is the 2x upscale, different seed and the same type of "framing" is there too, on the left side.

View attachment 3016750 View attachment 3016751

Fault in model data?

(Edit: added a enlarged section incase it's difficult to see on the forum)
View attachment 3016947

How exactly are you upscaling that? I believe you use A1111, correct? Are you using the Extras tab, or is this using something like the SD Upscale or SD Ultimate Upscale scripts in the img2img tab?

I ask because this, in addition to your render times, looks like a img2img render with one of those scripts that got interrupted mid process. A simple upscale using one of those upscalers in the Extras tab should not take more than a few seconds to complete even on a low-to-mid range GPU.

 

[Mr-Fox]

Since there was some new upscalers mentioned i wanted to test and it gave a nice reminder that highres testing is not for me
2x upscaling a 512x512 image took just 3min, a 768x768 took 13min, and it overflowed into shared memory by 11gb on the final step/percentage generating.
3x upscaling on 512x512 took 12,5 min...so yea, not really sure i'm gonna bother with anything higher.
Seems it was a long wait for more than just me, even the woman in the image went old and grey
View attachment 3016726
On a more serious note, look a her hair about a bit below her chin, at first i thought something was wrong with the upscaling, but i've included a non upscaled version i generated afterwards and it's there too.
Third image is the 2x upscale, different seed and the same type of "framing" is there too, on the left side.

View attachment 3016750 View attachment 3016751

Fault in model data?

(Edit: added a enlarged section incase it's difficult to see on the forum)
View attachment 3016947

Yes hiresfix is notorious for being slow, it's the hires steps that takes time. This script or extension can also be at fault for memory leaks etc.
The more steps the better to a point but it also will take longer. Most of the time though it works well enough for a mere peasant pleb like me with a 1070 8Gb card. I typically use 640x960 for potrait ratio images and then upscale by 2x with hiresfix since my rig cant go any further. This gives me 1280x1920 and it's plenty. The rest is up to the ckp and prompt in combination with anything else such as controlnet loras and all other fun toys.
If this is not enough than the problem is not with SD. Then you can ofc go further and use SD Upscale or go straight to photoshop for those final touches. An alternative route is ofc to skip hiresfix and only using SD Upscale. I would treat the sample steps in img2img as the hires steps and set them 2x to the sample steps in txt2img. The denoising strength is something that is probably specific for each use case and needs testing to find the right value. Depending on all variables such as style choice, ckp, prompt and loras etc. Something also to consider is that the denoising strength might be specific for each upscaler. I have not thought of this before, it just occurred to me. Something to test for sure.

I have no idea what is the cause of that blurry line or area in your image, sry. It's very late here so I can't attempt any trouble shooting until tomorrow.

 

[me3]

How exactly are you upscaling that? I believe you use A1111, correct? Are you using the Extras tab, or is this using something like the SD Upscale or SD Ultimate Upscale scripts in the img2img tab?

I ask because this, in addition to your render times, looks like a img2img render with one of those scripts that got interrupted mid process. A simple upscale using one of those upscalers in the Extras tab should not take more than a few seconds to complete even on a low-to-mid range GPU.

highres function in a1111 text2img, and as Mr-Fox mentions in his post, it's slow and really likes eating memory,
If it'd had been just through img2img etc i'd suspected it was due some kind of tiling, but that wouldn't account for the none upscaled image, that has no upscaling nor highres fix, i even generated it again without vae to see if it was that but still there.

 

[hkennereth]

highres function in a1111 text2img, and as Mr-Fox mentions in his post, it's slow and really likes eating memory,
If it'd had been just through img2img etc i'd suspected it was due some kind of tiling, but that wouldn't account for the none upscaled image, that has no upscaling nor highres fix, i even generated it again without vae to see if it was that but still there.

I find that strange because... yeah, A1111 is always pretty slow, and I don't know exactly what hardware you're running it in, but on my RTX 3060 with 8GB VRAM, which is okay but not amazing for SD, generating images with highres fix would take about 2 minutes to render, which a lot longer compared to what I get on ComfyUI (that's more in the 30 seconds range), but definitely not the 10-15 minutes range you're describing.

I know I must sound like a broken record, but you guys really should look into using ComfyUI. A1111 is just too poorly optimized to use comfortably with low-end hardware. Hell, even

You must be registered to see the links

would give you a better experience, even if a more limited one.

 

[Jimwalrus]

I find that strange because... yeah, A1111 is always pretty slow, and I don't know exactly what hardware you're running it in, but on my RTX 3060 with 8GB VRAM, which is okay but not amazing for SD, generating images with highres fix would take about 2 minutes to render, which a lot longer compared to what I get on ComfyUI (that's more in the 30 seconds range), but definitely not the 10-15 minutes range you're describing.

I know I must sound like a broken record, but you guys really should look into using ComfyUI. A1111 is just too poorly optimized to use comfortably with low-end hardware. Hell, even

You must be registered to see the links

would give you a better experience, even if a more limited one.

I wasn't aware there was a substantial performance improvement - Sepheyer was too busy telling us how easy it is to use (if you enjoy gazing at plates of spaghetti!)

I'm experiencing a lot of issues with A1111 atm, including all kinds of errors on start-up. Also, generating an image takes about 3mins with 60 HiRes steps.

May well install it and have a go, when I get time.

 

[me3]

I find that strange because... yeah, A1111 is always pretty slow, and I don't know exactly what hardware you're running it in, but on my RTX 3060 with 8GB VRAM, which is okay but not amazing for SD, generating images with highres fix would take about 2 minutes to render, which a lot longer compared to what I get on ComfyUI (that's more in the 30 seconds range), but definitely not the 10-15 minutes range you're describing.

I know I must sound like a broken record, but you guys really should look into using ComfyUI. A1111 is just too poorly optimized to use comfortably with low-end hardware. Hell, even

You must be registered to see the links

would give you a better experience, even if a more limited one.

Ah the dream of having something that fast
Running a 1060 6gb, a 20 step 512x512 image takes about 15sec. Haven't really used Comfy in a while so i can't remember what its speed was for the same. I was mainly using it to test SDXL as it could actually run it unlike a1111 which still struggle with, which is really poor considering how long comfy has been able to. Haven't looked at SD.next in a while though, which is a fork of a1111, that is claimed to be an improvement in many things.

Not really a fan of the node setup in Comfy, i get the idea for it and the more options/control it provide, but unfamiliar things often end up suffering because you more easily fall back on what's familiar. What i REALLY don't like with comfy though is the way it generates random seed. Not sure if it's a workflow setup thing, but for me the random seed shown at the end of generation isn't what was used for the generation. It's the "next one", so you have to find the previous in some way as i've not seen a easy button for it.

I wasn't aware there was a substantial performance improvement - Sepheyer was too busy telling us how easy it is to use (if you enjoy gazing at plates of spaghetti!)

I'm experiencing a lot of issues with A1111 atm, including all kinds of errors on start-up. Also, generating an image takes about 3mins with 60 HiRes steps.

May well install it and have a go, when I get time.

Assuming those "errors" are the same as the ones i'm getting it's technically not errors, it's deprecation warnings. Which is something about a1111 that annoy me, they know those warnings are there, but instead of updating the code, ppl just get told to turn the warnings off...
They have to update the code eventually and it might actually get better too, but i guess it's why there's so many things that could be optimized as well.

 

[Sepheyer]

I wasn't aware there was a substantial performance improvement - Sepheyer was too busy telling us how easy it is to use (if you enjoy gazing at plates of spaghetti!)

I'm experiencing a lot of issues with A1111 atm, including all kinds of errors on start-up. Also, generating an image takes about 3mins with 60 HiRes steps.

May well install it and have a go, when I get time.

There are solid attempts to declutter the space in CUI. Just watch these pieces for 20 seconds each, even without sound, to get the feeling how the workspace with the new custom nodes differs from old approaches:

You must be registered to see the links

You must be registered to see the links

Clearly the community recognizes the clutter as an issue and attempts to find a solution for it.

Rando chick to reinforce the point:

 

[hkennereth]

Not really a fan of the node setup in Comfy, i get the idea for it and the more options/control it provide, but unfamiliar things often end up suffering because you more easily fall back on what's familiar.

I can surely understand that, and there is definitely a steeper learning curve to understand how it works at first, and sometimes trying some slightly different process may require spending some time messing around with your workflow instead of just pushing some buttons. I personally do find that rewarding because it allows you to better understand what is actually happening behind the scenes, which in turn allows you to better fine tune the process. But it's a fair concern.

What i REALLY don't like with comfy though is the way it generates random seed. Not sure if it's a workflow setup thing, but for me the random seed shown at the end of generation isn't what was used for the generation. It's the "next one", so you have to find the previous in some way as i've not seen a easy button for it.

THAT I am 100% behind, and I wish there was a way to change this behavior. Indeed the way Comfy handles seeds is that when you set it to change the value by either randomization or addition is to change the value AFTER you queue the image for generation, instead of BEFORE generating like on every other UI out there.

The only reason this doesn't bother me more is because... it's actually pretty trivial to find an image's seed and change settings for adjustments: if you drag any PNG generated with Comfy into its window, it will load the entire flow used to build it with all the original settings. So if I make a bunch of images and want to adjust one of them, I drag them back to Comfy from the file manager, change the seed generation values to Fixed where it's relevant, alter any settings I want, and make the image again.

 

[hkennereth]

I wasn't aware there was a substantial performance improvement - Sepheyer was too busy telling us how easy it is to use (if you enjoy gazing at plates of spaghetti!)

Oh yeah, it's A LOT faster. I was always a reluctant A1111 user because I always found it too unstable and its settings very confusing, so I would usually prefer to use EasyDiffusion when making simple images, but on both of those making standard 512 x 512 px images with SD1.5 models could take about 1 minute (at first, ED got a lot faster on later versions), without using any enhancements like highres fix. My average render time rendering the same basic 512px² image with ComfyUI is... let's check...

 

[Sepheyer]

I personally do find that rewarding because it allows you to better understand what is actually happening behind the scenes, which in turn allows you to better fine tune the process.

100%. Realizing that with SD you actually have a stream where you are moving latents thru transformers is priceless and I think an exclusively CUI experience:

Spoiler: Illustration

You don't have permission to view the spoiler content. Log in or register now.

 

[Jimwalrus]

I can surely understand that, and there is definitely a steeper learning curve to understand how it works at first, and sometimes trying some slightly different process may require spending some time messing around with your workflow instead of just pushing some buttons. I personally do find that rewarding because it allows you to better understand what is actually happening behind the scenes, which in turn allows you to better fine tune the process. But it's a fair concern.


THAT I am 100% behind, and I wish there was a way to change this behavior. Indeed the way Comfy handles seeds is that when you set it to change the value by either randomization or addition is to change the value AFTER you queue the image for generation, instead of BEFORE generating like on every other UI out there.

The only reason this doesn't bother me more is because... it's actually pretty trivial to find an image's seed and change settings for adjustments: if you drag any PNG generated with Comfy into its window, it will load the entire flow used to build it with all the original settings. So if I make a bunch of images and want to adjust one of them, I drag them back to Comfy from the file manager, change the seed generation values to Fixed where it's relevant, alter any settings I want, and make the image again.

I may well use some of yours or Sepheyer's to give me a grounding - I always hate starting from a blank sheet.

So, if my celeb flight attendants start looking a little more... ...'buxom', you know whose I've copied!

 

[Sepheyer]

I may well use some of yours or Sepheyer's to give me a grounding - I always hate starting from a blank sheet.

So, if my celeb flight attendants start looking a little more... ...'buxom', you know whose I've copied!

I just can't get enough of these. Literally as if I won a billion dollar jackpot and now can snort cocaine all day long without (finally!) being afraid of getting fired:

 

[hkennereth]

Here is that IPAdapter worklow. Purple are samplers that bash the latents together either which way. That's how you can realize you can have fifty of these lined up, each doing either promp, adding, subtracting, generally acting like Photoshop layers and you go - FML, now I get where this comes from. If y'all wanna try out the actual workflow - they are here:
View attachment 3017817

BTW, thanks for not making actual "spaghetti" flows that try too hard to create interfaces but you can't understand what's happening, and instead having clear and organized flows where you can see how things are connected and how data is passing through. THAT is what makes Comfy good to use!

 

[me3]

i don't know how much this affects, but having to upgrade comfyui i came across some dependency problems.
A simple install of "torch" installs version 2.1 for cuda 11.8, but it seems that xformers that gets installed along with it is for cuda 12.
Considering a1111, kohya_ss and comfy all use this it might be worth keeping in mind if planing to update.
When i updated xformers some days ago to, to some dev version of 0.0.23, it didn't have a cuda 12 dependency. Now it seems to be the case for most recent version of 0.0.22 and dev of 0.0.23