[Stable Diffusion] Prompt Sharing and Learning Thread

[Jimwalrus]

Shouldn't that be clarified in the official documentation?

The magic word there being "Should"

If it's there, and written in a way that it's understandable. And it hasn't been superseded in the last 12 months or so.
This is all cutting edge and mostly open source, so don't expect documentation to be good!

 

[me3]

Example was requested so i guess this might work.
I added () to the prompt, first just for the closeup, then for the face lighting as well.


This does seem to suggest that the weighting isn't applied when it's not wrapped in (), or at the very least, not applied correctly.
Granted, the prompt gets "altered" by adding this but i don't think the changes are random enough as they seem to follow the intention in the prompt

 

[Mr-Fox]

Example was requested so i guess this might work.
I added () to the prompt, first just for the closeup, then for the face lighting as well.
View attachment 3003580 View attachment 3003581 View attachment 3003582

This does seem to suggest that the weighting isn't applied when it's not wrapped in (), or at the very least, not applied correctly.
Granted, the prompt gets "altered" by adding this but i don't think the changes are random enough as they seem to follow the intention in the prompt

 

[me3]

So i went through the parser code and found the weight handling, then ran the prompt through with and without the ()

Without the () the whole prompt is assigned a weight of 1.0, meaning the :1.5 and :1.4 is ignored as weighting and more considered "part of the text" along with the word infront, so "closeup:1.5" is seemingly treated as one belonging together.

[['1 girl in a black matrix coat, standing on a rooftop, upper body shot closeup:1.5, cinematic shot, golden sidelight on her face:1.4, foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

With () the weighting is done more as expected and those two sections get their 1.5 and 1.4 weighting, rest is assigned 1.0

[['1 girl in a black matrix coat, standing on a rooftop, ', 1.0], ['upper body shot closeup', 1.5], [', cinematic shot, ', 1.0], ['golden sidelight on her face', 1.4], [', foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

Just to test how things are handled i ran this as well:
(This) (is:1.1) ((a)) ((test:1.1)) (prompt:1.2)

[
    ['This', 1.1],
    [' ', 1.0],
    ['is', 1.1],
    [' ', 1.0],
    ['a', 1.2100000000000002],
    [' ', 1.0],
    ['test', 1.2100000000000002],
    [' ', 1.0],
    ['prompt', 1.2]
]

 

[Mr-Fox]

So i went through the parser code and found the weight handling, then ran the prompt through with and without the ()

Without the () the whole prompt is assigned a weight of 1.0, meaning the :1.5 and :1.4 is ignored as weighting and more considered "part of the text" along with the word infront, so "closeup:1.5" is seemingly treated as one belonging together.

[['1 girl in a black matrix coat, standing on a rooftop, upper body shot closeup:1.5, cinematic shot, golden sidelight on her face:1.4, foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

With () the weighting is done more as expected and those two sections get their 1.5 and 1.4 weighting, rest is assigned 1.0

[['1 girl in a black matrix coat, standing on a rooftop, ', 1.0], ['upper body shot closeup', 1.5], [', cinematic shot, ', 1.0], ['golden sidelight on her face', 1.4], [', foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

Just to test how things are handled i ran this as well:
(This) (is:1.1) ((a)) ((test:1.1)) (prompt:1.2)

[
    ['This', 1.1],
    [' ', 1.0],
    ['is', 1.1],
    [' ', 1.0],
    ['a', 1.2100000000000002],
    [' ', 1.0],
    ['test', 1.2100000000000002],
    [' ', 1.0],
    ['prompt', 1.2]
]

Awesome stuff.

 

[Mr-Fox]

Another way to keep the pose and composition is to use openposer in controlnet. I thought I should just mention this.
You need to install controlnet ofc. Load the original image in controlnet and select openpose, choose "open pose full " and then press "preprocessor" (small explosion cymbol). This will detect the image and create a pose, you can adjust it in the editor if you need. Then press "send pose to controlnet" . Now you can generate image after image and change the background, the clothes or what ever you want while the pose reamains the same.

The image I generated with this pose:



Or something different..


Link for the editor:

You must be registered to see the links

 

[Fuchsschweif]

This does seem to suggest that the weighting isn't applied when it's not wrapped in (), or at the very least, not applied correctly.
Granted, the prompt gets "altered" by adding this but i don't think the changes are random enough as they seem to follow the intention in the prompt

Interesting, thanks!

It makes sense in terms that it makes it easier to isolate desired parts of a prompt. So one could write:

A city with a park (and tall skyscrapers:1.5), and since the whole part is put in parantheses, SD doesn't have to take the whole prompt until the comma into consideration.

I found a website stating that adding + or - (also multiple times possible, like "forest++++") would add or subtract importance to a word, but a little simple test with "trees++++, buildings--" and vice versa didn't prove that.

 

[Jimwalrus]

So i went through the parser code and found the weight handling, then ran the prompt through with and without the ()

Without the () the whole prompt is assigned a weight of 1.0, meaning the :1.5 and :1.4 is ignored as weighting and more considered "part of the text" along with the word infront, so "closeup:1.5" is seemingly treated as one belonging together.

[['1 girl in a black matrix coat, standing on a rooftop, upper body shot closeup:1.5, cinematic shot, golden sidelight on her face:1.4, foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

With () the weighting is done more as expected and those two sections get their 1.5 and 1.4 weighting, rest is assigned 1.0

[['1 girl in a black matrix coat, standing on a rooftop, ', 1.0], ['upper body shot closeup', 1.5], [', cinematic shot, ', 1.0], ['golden sidelight on her face', 1.4], [', foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

Just to test how things are handled i ran this as well:
(This) (is:1.1) ((a)) ((test:1.1)) (prompt:1.2)

[
    ['This', 1.1],
    [' ', 1.0],
    ['is', 1.1],
    [' ', 1.0],
    ['a', 1.2100000000000002],
    [' ', 1.0],
    ['test', 1.2100000000000002],
    [' ', 1.0],
    ['prompt', 1.2]
]

This is gold dust, thank you so much.
I wondered why sometimes my single word prompts with decimal weighting didn't work.

 

[Sepheyer]

So i went through the parser code and found the weight handling, then ran the prompt through with and without the ()

Without the () the whole prompt is assigned a weight of 1.0, meaning the :1.5 and :1.4 is ignored as weighting and more considered "part of the text" along with the word infront, so "closeup:1.5" is seemingly treated as one belonging together.

[['1 girl in a black matrix coat, standing on a rooftop, upper body shot closeup:1.5, cinematic shot, golden sidelight on her face:1.4, foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

With () the weighting is done more as expected and those two sections get their 1.5 and 1.4 weighting, rest is assigned 1.0

[['1 girl in a black matrix coat, standing on a rooftop, ', 1.0], ['upper body shot closeup', 1.5], [', cinematic shot, ', 1.0], ['golden sidelight on her face', 1.4], [', foggy atmosphere, neon glowing in the back, rainy day, cloudy, cyberpunk cityscape in the back, blade runner style, cyberpunk style, serious look, rough and moody atmosphere, gritty style, photoshooting', 1.0]]

Just to test how things are handled i ran this as well:
(This) (is:1.1) ((a)) ((test:1.1)) (prompt:1.2)

[
    ['This', 1.1],
    [' ', 1.0],
    ['is', 1.1],
    [' ', 1.0],
    ['a', 1.2100000000000002],
    [' ', 1.0],
    ['test', 1.2100000000000002],
    [' ', 1.0],
    ['prompt', 1.2]
]

Would you know why it doesn't drop the blank tokens altogether, i.e. the spaces? I am surprised those are even added to the list. Also, if you have the code still up, can you please let us know how this gets converted: (((test:1.3))).

 

[me3]

Would you know why it doesn't drop the blank tokens altogether, i.e. the spaces? I am surprised those are even added to the list. Also, if you have the code still up, can you please let us know how this gets converted: (((test:1.3))).

This is the output from just the weight assignment, so there might be functions that drops "empty" array elements later. My guess is that it keeps everything for "completeness" so that you could technically just merge the [x][0] array elements back together if needed. I honestly haven't looked that the whole processing code for it, it's just a idea of possibility that popped in as "something it wouldn't surprise me if a coder had done".
Also, there's such as thing as token padding where i believe both pos and neg token gets padded to equal length, i don't know why this gets done nor what it affects. Not that unlikely the padding is just spaces so depending on where in the parsing that happens so not stripping spaces could be to maintain something related to that. Again, just a random thought.


As for your test, it comes to 1.573. every additional () after the first adds a multiplier of 1.1 to the specified weight, so since there's 2 additional () you get 1.3 * 1.1 * 1.1. Due to computer math here's a whole bunch of 0s and a number at the end but that's not really that important in this case.

 

[Sepheyer]

This is the output from just the weight assignment, so there might be functions that drops "empty" array elements later. My guess is that it keeps everything for "completeness" so that you could technically just merge the [x][0] array elements back together if needed. I honestly haven't looked that the whole processing code for it, it's just a idea of possibility that popped in as "something it wouldn't surprise me if a coder had done".
Also, there's such as thing as token padding where i believe both pos and neg token gets padded to equal length, i don't know why this gets done nor what it affects. Not that unlikely the padding is just spaces so depending on where in the parsing that happens so not stripping spaces could be to maintain something related to that. Again, just a random thought.


As for your test, it comes to 1.573. every additional () after the first adds a multiplier of 1.1 to the specified weight, so since there's 2 additional () you get 1.3 * 1.1 * 1.1. Due to computer math here's a whole bunch of 0s and a number at the end but that's not really that important in this case.

Thanks, I was hoping for the actual code intermediary results dump if you still had that snippet up.

 

[me3]

Thanks, I was hoping for the actual code intermediary results dump if you still had that snippet up.

rather limited output since it's just one word

[['test', 1.5730000000000004]]

Can

You must be registered to see the links

if interested, fairly uncomplicated. Just put the prompt in the print at the bottom and hit run, link should be valid for a month.

 

[Mr-Fox]

Synalon pointed out to me that the editor I showed in this post yesterday is something you might need to install. I thought it was included with controlnet. I apologize for the oversight. I have updated the post.

You must be registered to see the links

 

[Sepheyer]

Picking up on an idea left for later in this post: link, namely taking a slender woman-only picture and outpainting it out.

The goal is to save on the machine cycles. Namely you render and save "just the girl" and while the outpaint is rendering you get enough time to decide if you want to go ahead with the outpaint: can cancel the outpaint at any moment.

Here's the workflow file for CUI:

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

Why? Cause the actual girl takes about 1/3rd of the time required to render the entire image. Kinda sucks rendering the whole thing just to find out something turned out bad. I.e. here I realized I don't want the purple square and cancelled the render:

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

 

[Sepheyer]

In a day or so I'll be dropping a workflow where the outpaint option is disabled although present inside the workflow. You just don't enable it at the start. You can render as many smaller images as you want, then go back to the one you like, enable the outpaint and then commit machine cycles there.

 

[Sepheyer]

So this guy (

You must be registered to see the links

) keeps experimenting with the ComfyUI workflows and he posted a method of making consistent face and an outfit and a pose:

You must be registered to see the links

Here is the link to his workflow that's meant to do all of that:

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

I poped this workflow in and it turns out I am missing quite a good bit of nodes:

On his page NR suggests a CUI plugin to automatically download the missing nodes - finally a motivation to try out some of those bells and whistles.

It will take me some time to get my CUI installation to where it can run that workflow, partially due to procrastination. Still, this looks very interesting. If this works as advertised, this might become a great addition or even alternative to LORAs.

 

[Sepheyer]

How does that old joke go about being rational? The NASA spent a few million dollars for their space pen, the Russians just used an old pencil.

Depending on your machine this workflow will very quickly generate a girl against a white background and then will spend close to 10 minutes outpainting that white background 250 pixels left and 250 pixels right. And then of course, there are nuances:

The image above is a CUI workflow.

The image below is a manual edit, where one layer was the AI-generated girl and another layer was ~white background, then both layers were blended:

Now, don't let me do you a disservice thru ridiculing the CUI's outpaint workflow. The result was subpar because I am a moron; the workflow approach itself is very clearly the future. I am clowning somewhat, don't make a mistake of drawing the wrong conclusion here.

 

[Fuchsschweif]

What are the best upscalers for very sharp and good results? I got some of the NMKD ones but they all suck so far. (Or it's my skill issue)

 

[hkennereth]

What are the best upscalers for very sharp and good results? I got some of the NMKD ones but they all suck so far. (Or it's my skill issue)

I personally use two primarily: a good and reliable all-arounder (4x-Ultrasharp), and one better for high-precision and detailed upscaling but can be tricky in some cases (4x_NMKD-Siax_200k). I use those extensively and I stand by their quality. I also had issues with other NMKD upscalers like the Superscale-SP one, so it's not just you. In general I use Siax for everything unless it shows some artifacting, in which case I go back to Ultrasharp.

 

[Sepheyer]

I personally use two primarily: a good and reliable all-arounder (4x-Ultrasharp), and one better for high-precision and detailed upscaling but can be tricky in some cases (4x_NMKD-Siax_200k). I use those extensively and I stand by their quality. I also had issues with other NMKD upscalers like the Superscale-SP one, so it's not just you. In general I use Siax for everything unless it shows some artifacting, in which case I go back to Ultrasharp.

Would you post examples of what those look like on ~1200x2400 renders?