[Stable Diffusion] Prompt Sharing and Learning Thread

[Fuchsschweif]

The LoRA you downloaded seem to be to create one character with different "outfits", so it seems to be very specific in what it wants to do.
You might be able to trigger just the character if you remove the outfit parts from the prompt, but i doubt it'll work fully.
There might be other LoRAs for the same character which is less restricted

But I am not getting anything even remotely close. Not even with the full suggested copied prompt from the page. It looks rather like the SD base model than the downloaded model.

This is all I have to do, right?



(Taken from here:

You must be registered to see the links

)

By the way I don't seem to have any "show extra networks" button under the Generate button:

 

[Mr-Fox]

I somehow thought SD is Dalle2 and since they charge premium that there is no free version. Or is Dalle just another big model that gives you external rendering, making use of SD, therefore charge you?

I've got a GTX 1070, does it make sense to generate pictures with that or will it take an eternity to get pictures generated?

Thanks for all the info

I have a GTX 1070. You can check out my posts by searching my name to see what you can do with it. This card is enough for most ai generating, it's only if you go very advanced or want to train a SDXL Lora or something that this card will limit you.

 

[me3]

But I am not getting anything even remotely close. Not even with the full suggested copied prompt from the page. It looks rather like the SD base model than the downloaded model.

This is all I have to do, right?

View attachment 2992932

(Taken from here:

You must be registered to see the links

)

By the way I don't seem to have any "show extra networks" button under the Generate button:

View attachment 2992934

Yes you put loras into that directory, HOWEVER you don't select that in the dropdown for models at the top left of the page.
That's models, the lora you are trying to use shouldn't even show up in that list unless you've put the file in the wrong place.
In that drop down select the 1-5 model which you previously had selected there.
Loras (rather small files from 20-120mb) are NOT models, they are (simply put) a very small part of a model, they do not work on their own, the add to models (large files generally 2-6GB)

Also that "guide" is wrong in at least one point. :1 should usually NOT be too high, if that needs to be lowered in the case of things like characters, that lora has been overtrained/overfitted, and as such is badly made.

 

[Mr-Fox]

I would suggest that you start out much more simple to get familiar with using SD. Meaning only a checkpoint model and prompt. The main "skill" to learn is namely prompting. You need to get proficient in creating prompts to be able to create your images. First after you have become more familiar with SD and prompting, then it's time to explore Loras etc.

 

[Fuchsschweif]

I would suggest that you start out much more simple to get familiar with using SD. Meaning only a checkpoint model and prompt. The main "skill" to learn is namely prompting. You need to get proficient in creating prompts to be able to create your images. First after you have become more familiar with SD and prompting, then it's time to explore Loras etc.

I am good at prompting, I've been using MJ since it came out and get pretty high quality results with a lot of control over details. But that's only the prompting, my main issue now is getting SD to output somewhat usable pictures in specific styles.

The default SD style is pretty ugly, that's why I wanted to implement a specific model that aesthetically matches what I envision. But I can't get those models to run, no matter what I try to implement, I get only default SD results.

 

[Fuchsschweif]

Look, I downloaded this one:

You must be registered to see the links

Put it into the Lora folder. Then in Stable Diffusion I pick it from the Lora subtab, it puts "<lora:hari:1>" into the prompt tab.

I add the whole trigger word thing (just for sake of demonstration):

"1girl, solo, brown hair, medium hair large breast, brown eyes,"

let's add: in front of a window, at night, interior.



Now that's what I get out of the generation:




Where's the whole style of the downloaded model? Isn't it supposed to look like this? It's not even night!

 

[Mr-Fox]

Look, I downloaded this one:

You must be registered to see the links

Put it into the Lora folder. Then in Stable Diffusion I pick it from the Lora subtab, it puts "<lora:hari:1>" into the prompt tab.

I add the whole trigger word thing (just for sake of demonstration):

"1girl, solo, brown hair, medium hair large breast, brown eyes,"

let's add: in front of a window, at night, interior.

View attachment 2993078

Now that's what I get out of the generation:

View attachment 2993079


Where's the whole style of the downloaded model? Isn't it supposed to look like this? It's not even night!

View attachment 2993080

If you take a look at the generation data you can see that they use this checkpoint: AnyLoRA_noVae_fp16-pruned

You must be registered to see the links

They have also used a negative Lora easynegative and negative embeding deepnegative:

You must be registered to see the links

You must be registered to see the links

Embedings is placed in Stable-Diffusion\stable-diffusion-webui\embeddings.

They also generated this image with hiresfix and the upscaler 4x-Ultrasharp.

You must be registered to see the links

Place it in Stable-Diffusion\stable-diffusion-webui\models\ESRGAN
They are using clipskip 2, you can find this in settings.

Spoiler: Generation Data

You don't have permission to view the spoiler content. Log in or register now.

Enlarge the images in your browser and download. Go to the PNG Info tab in SD and load an image and press "send to txt2img". This will add all the data for you because the png files contain the generation data (prompt etc). Now you only need to make sure that you have all the models in the right folders then you can start generating. Don't forget to set the seed to -1 if you don't want the exact same image as me.

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

 

[Fuchsschweif]

If you take a look at the generation data you can see that they use this checkpoint: AnyLoRA_noVae_fp16-pruned

You must be registered to see the links

They have also used a negative Lora easynegative and negative embeding deepnegative:

You must be registered to see the links

You must be registered to see the links

Thanks, I am right now downloading the checkpoint and then proceed with your manual!

Can you explain to me what the difference is between a checkpoint and a model? Checkpoints seems to be more like a basic database, and models are very specific modules?

And the negative embedding are just for excluding / specifing what one dpesn't want? In MJ one would only add a "-" before that desired prompt, like "-hands" would make MJ try to avoid showing hands.

Why are there two different embeddings used for negatives?

And I assume with the merge tool I can combine two checkpoints or models and get both to contribute to the generations?

Edit:

Did everything now except this:

Enlarge the images in your browser and download. Go to the PNG Info tab in SD and load an image and press "send to txt2img". This will add all the data for you because the png files contain the generation data (prompt etc). Now you only need to make sure that you have all the models in the right folders then you can start generating. Don't forget to set the seed to -1 if you don't want the exact same image as me.

I just took the generation data from the civitai site (the prompts, negative prompts, and brought the sampling steps up to 30. It's way better now but the style still looks different, any idea why? (I also activated the restore faces function as suggested on the 1st page of this thread).




This is what it looks like on my side:

 

[Mr-Fox]

Thanks, I am right now downloading the checkpoint and then proceed with your manual!

Can you explain to me what the difference is between a checkpoint and a model? Checkpoints seems to be more like a basic database, and models are very specific modules?

And the negative embedding are just for excluding / specifing what one dpesn't want? In MJ one would only add a "-" before that desired prompt, like "-hands" would make MJ try to avoid showing hands.

Why are there two different embeddings used for negatives?

And I assume with the merge tool I can combine two checkpoints or models and get both to contribute to the generations?

Stable Diffusion is the Ai platform. Stabe Dilffusion 1.5 is the basemodel or generation (as in "version"). The checkpoint models are custom models that has been trained on top of SD1.5 . You can think of it as a checkpoint in a race, SD 1.5 have been custom trained with a certain type of dataset material so it will have certain style or characters or other quality. Why it's called a checkpoint is because this is the state this particular SD 1.5 model has been trained to, it has reached this checkpoint. It's like a screenshot of the state the model is in, in it's current training.
So when you download Super Duper Anime Checkpoint, it is the state of a SD1.5 model that you are downloading.
This is the convention behind the name checkpoint. I hope I'm being clear. It's an unnecessarily complicated name convention and naturally very confusing to most. The Lora is an added "mini model" that guides SD towards a specific result. It can be a concept, a character, a style. Embedings are similair to Loras it is just a different implementation. They are essentially added input models on top of the checkpoint model.

 

[Fuchsschweif]

This is the convention behind the name checkpoint. I hope I'm being clear. It's an unnecessarily complicated name convention and naturally very confusing to most. The Lora is an added "mini model" that guides SD towards a specific result. It can be a concept, a character, a style. Embedings are similair to Loras it is just a different implementation. They are essentially added input models on top of the checkpoint model.

So the checkpoints are basically addons, and the loras are something like "prompt collections"?

I think your reply did overlap with my edit on my post above yours. It tried it first with only carrying the prompts over from civitai, but did get different results. Now with the data I extracted from your PNG the result is way closer (and takes way longer though).

How did you know what to add? I see you had the upscaler, denoising, refiner and all of that in your settings.
And this:
Or, to put my question different: How would I gather these information from the original model page on civitai? Just the same way, downloading the picture and extracting the settings?

Funny is, in my own version (as above) when I added "nipples" they didn't show. Now with the extracted setting from the original it does. Any idea why?

I am asking because I of course want to understand the mechanisms behind it so that I can create more freely and not just copy and paste prompts all the time. So I have to understand for instance what changed with the extracted settings that made SD listen more closely to my input.

Aaaaand a last question: I can basically turn of the upscaler and only upscale pictures I like later on the extra page in order to safe time?

As I am typing this, the generation with the extracted settings from your PNG is done. It's quite close now, but the colors are still faded!

 

[Mr-Fox]

Thanks, I am right now downloading the checkpoint and then proceed with your manual!

Can you explain to me what the difference is between a checkpoint and a model? Checkpoints seems to be more like a basic database, and models are very specific modules?

And the negative embedding are just for excluding / specifing what one dpesn't want? In MJ one would only add a "-" before that desired prompt, like "-hands" would make MJ try to avoid showing hands.

Why are there two different embeddings used for negatives?

And I assume with the merge tool I can combine two checkpoints or models and get both to contribute to the generations?

Edit:

Did everything now except this:



I just took the generation data from the civitai site (the prompts, negative prompts, and brought the sampling steps up to 30. It's way better now but the style still looks different, any idea why? (I also activated the restore faces function as suggested on the 1st page of this thread).


View attachment 2993297

This is what it looks like on my side:

View attachment 2993314

They used a negative Lora and a negative embeding. Both are negative inputs, meaning things you don't want. The reason for using both is because they don't do the same thing I assume. Most of the time I don't use these negative models personally since I don't have direct control of them. Don't worry about merging checkpoints yet, before you get really familiar and proficient with SD. Instead download others merged checkpoints on civitai for now. The difference between a trained checkpoint and merged checkpoint is that a merged checkpoint is combining 2 or more checkpoints into 1. This can result in either very flexible and good checkpoints if you know what you are doing or it can create a completely useless mess. Unfortunately some people think that the more you throw at the wall the more will stick.. So many merged checkpoints on civitai is simply not worth your time. We can only find out if it's good by testing ourself and/or listening to the recommendations of people who's judgment we trust.

 

[Fuchsschweif]

I added "sticking tongue out" and "public hair" to see if the model could do that, the tongue (kinda) worked on the first one, public hair wasn't considered at all. How can I make SD not ignore prompts?

 

[Fuchsschweif]

Don't worry about merging checkpoints yet, before you get really familiar and proficient with SD. Instead download others merged checkpoints on civitai for now.

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:




It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:




So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

 

[Mr-Fox]

So the checkpoints are basically addons, and the loras are something like "prompt collections"?

I think your reply did overlap with my edit on my post above yours. It tried it first with only carrying the prompts over from civitai, but did get different results. Now with the data I extracted from your PNG the result is way closer (and takes way longer though).

How did you know what to add? I see you had the upscaler, denoising, refiner and all of that in your settings. Or, to put my question different: How would I gather these information from the original model page on civitai? Just the same way, downloading the picture and extracting the settings?

Funny is, in my own version (as above) when I added "nipples" they didn't show. Now with the extracted setting from the original it does. Any idea why?

I am asking because I of course want to understand the mechanisms behind it so that I can create more freely and not just copy and paste prompts all the time. So I have to understand for instance what changed with the extracted settings that made SD listen more closely to my input.

Aaaaand a last question: I can basically turn of the upscaler and only upscale pictures I like later on the extra page in order to safe time?

As I am typing this, the generation with the extracted settings from your PNG is done. It's quite close now, but the colors are still faded!

View attachment 2993328

You need this vae also. Forgot to tell you, sorry.

You must be registered to see the links

Place it inside Stable-Diffusion\stable-diffusion-webui\models\VAE .
It is for improving the face and eyes but it also can affect light and color.
On civitai you can find the generation data by enlarging the image and simply look on your right.

If you scroll all the way down you can find a button to copy all the generation data or you can choose to only copy the positive prompt for example by pressing the copy button to the right of the positive prompt. On civit the image doesn't contain the data unfortunately so you have to paste it in a txt document and then copy all the settings manually.
I have no idea why the nipples didn't show before but now they do, but my guess is that you switched checkpoint and this is more anime oriented. The basemodel does nudity but not as easy as the checkpoints on civitai.

 

[Mr-Fox]

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:

View attachment 2993499


It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:

View attachment 2993512


So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

Anime checkpoints are very specific to anime style. There are others that can do a little of everything. Such as Experience, Never ending dream, Dreamshaper, there are many. Yes you need to switch checkpoints depending on the style and the result you want. To get the pubic hair you can add weight to the this token (more than one word describing the same thing) like this:
pubic hair:1.2 or with brackets like this (pubic hair:1.2). Remember that brackets ads 0.1 on it's own so (pubic hair:1.2) is actually = 1.3 . A reason for it not showing could be because it's under a layer of clothing and SD doesn't understand you want the hair to be visible through the clothes. In that case you need to specify that the clothes are "sheer," or "see through" or "transparant".

 

[Mr-Fox]

Yes you can choose to not upscale the image now, you can use SD Upscale script in img2img instead. This is superior to "normal" upscaling in the extra tab. In the extra tab the upscaling is only enlarging the image and not adding more detail. The SD Upscale script is part of the generative process and thus creates new pixels wich ads more detail. I usually don't use hiresfix at first when I'm "searching" for a good image. When I "find " my image I simply re-use the seed from that image by pressing the green recycling button and then activate the hiresfix and then generates again. This is the most time effective way and it gives the absolute best result most of the time. Hiresfix is part of the generative process just like SD Upscale script, wich is an extension btw that you need to install. Go to the extension tab and available and press "load from". Then find SD Upscale and press "install", then press "apply and restart" in installed tab. Now you can find it in the scripts menu in img2img tab.



Now that SD restarted the prompts and settings are empty. Simply press the blue diagonal arrow button under "generate" to reset it or get back everything from before the restart.

 

[me3]

I'd like to have a more flexible way of creating that's why I am asking. With MJ I can just imagine whatever I want, specify it with 40 prompts and get very precise results, from anime to photo realistic in no time. Now here with that downloaded checkpoint, all generations look pretty close to this original girl.

If I would now type in something like "Two women with blue hair, wearing fantasy armor, on the moon, mars and space in the background, rough cyberpunk style, serious setting" I get that result with the style of the current setup:

View attachment 2993499


It looks pretty cool but whatever I create is done in this particular style. If I switch however back to the stable diffusion default checkpoint, I'll get this:

View attachment 2993512


So apparently the basic SD model is, well.. pretty inferior The checkpoints however are very specific. Is this just how SD works, that we always have to switch between different checkpoints for different purposes?

MJ does A LOT of stuff for you in the background, things you now need to do yourself in one way or another.
You're also comparing MJs massive data set that's been continuously tweaked, finetuned and curated to a SD base data set. (Both the sd15 base model and "anylora" is rather clean "untuned" models)
So in truth you're expecting a 1 year old to paint high quality art of concepts it's never heard of, seen or have any idea exist.

MJ wins on image "quality", simplicity in prompting (just throw words and it'll sort it out for you), but assuming you pick "the right tools for the job" SD will be more flexible, give you better control and not run off with all your money. With SDXL the quality gap is going away too.

 

[Fuchsschweif]

pubic hair:1.2 or with brackets like this (pubic hair:1.2). Remember that brackets ads 0.1 on it's own so (pubic hair:1.2) is actually = 1.3 . A reason for it not showing could be because it's under a layer of clothing and SD doesn't understand you want the hair to be visible through the clothes. In that case you need to specify that the clothes are "sheer," or "see through" or "transparant".

Ahh good to know the weight feature exists here too! What's the difference between using braces and using no braces (except that 0.1 value?).

But it still does ignore my input largely. See this example, I specified that she's supposed to lay on a bed, rear view angle, no underwear, even gave that weight (1.5):




My settings:




This was now without the upscaler which is why it is a bit faded and unsharp again. The VAE doesn't work yet it say not found, maybe I need to reload the web UI batch first.

 

[Fuchsschweif]

MJ wins on image "quality", simplicity in prompting (just throw words and it'll sort it out for you), but assuming you pick "the right tools for the job" SD will be more flexible, give you better control and not run off with all your money. With SDXL the quality gap is going away too.

Yes I realize this as we speak

What is SDXL?

And little side question: Are there already easy ways to add a bit of motion to still images with SD?

 

[Fuchsschweif]

Yes you can choose to not upscale the image now, you can use SD Upscale script in img2img instead.

So I pick "none" at upscaler right now? Or is one basic upscaler important for the base quality?