[Stable Diffusion] Prompt Sharing and Learning Thread

[Sepheyer]

Sure -- it's an image from the game Lust Theory I'm currently experimenting with:
View attachment 3138244

Any chance you can attach the file as an image rather than keeping it as thumbnail (or pastebin the image please)? I keep getting the tiny-tiny thumbnail for some reasons, even though for a second it opens up fullscreen normal resolution. Prolly I messed up my browser somehow with recent updates and dreading the idea I'll have to reinstall it.

Meanwhile, here is the source cartoon that I used to convert into those photos I posted above, so in theory this WF should handle DAZ.

Also, for establishing context - does this image look realistic to you? Because this is honestly what I call photo / realistic. If you prefer more, than sorry, I got nothing for the time being.

 

[theMickey_]

Any chance you can attach the file as an image rather than keeping it as thumbnail

Huh? You should be able to click on the thumbnail to get the full picture, and then you should be able to download it... But nevermind, I'll edited my previous post to attach the image instead of the thumbnail.

I can't get face swap to work - wf attached.

Here's a very basic faceswap workflow which works for me:

Spoiler

You don't have permission to view the spoiler content. Log in or register now.

 

[theMickey_]

Also, for establishing context - does this image look realistic to you?

Yes, it does. But maybe this will help as well: someone posted pictures in the The Genesis Order thread, here are just two examples (first the original picture from the game, and then the "real" version of the same picture):

Spoiler

You don't have permission to view the spoiler content. Log in or register now.

As you can see, the main picture composition (like the background, pose, lightning etc.) is still pretty much unchanged, it just changed it to a more "realistic" picture. This is pretty much what I'm trying to achieve

 

[Sepheyer]

Thanks again for your workflow -- I've tried it, but it's not doing what I want it to do: change a DAZ render into a "photo".

I've tried tuning the strength of the controlnet nodes, but it's doing all kinda stuff, because (I assume) you still have to add a lot of prompts manually on what you see in the original picture to get a somehow decent result.

Here's a quick idea on how I do want my workflow to look like:
View attachment 3138236

I might need to add more nodes to the workflow to downscale/upscale/inpaint/outpaint the images etc., but you might get the basic idea of what I'm trying to achieve. The issue is the red node, which I know exists, but I don't know what it's called and which custom nodes I'll have to install to get that specific node.

Here's the workflow and the image it converts DAZ to.

The lotsa things can be changed with the prompt - mifiness, clothes, etc. BTW, the WF looks a tad different - results of today's efforts. Finally, enable the "styles" group - for lols if nothin else.

I'll run the WF for a few more iterations and will add more images to this post if I come across any decent ones.

Spoiler: More

You don't have permission to view the spoiler content. Log in or register now.

 

[theMickey_]

Here's the workflow and the image it converts DAZ to.

That's pretty decent, I like it! But looking at your workflow, you did change your prompts quite a bit compared to your previous workflow, and I'm trying to avoid that. I really, really need to find that node that auto-generates prompts from any given image -- I guess I'll have to go through my YT and browser history trying to find that thing...

 

[me3]

Thanks again for your workflow -- I've tried it, but it's not doing what I want it to do: change a DAZ render into a "photo".

I've tried tuning the strength of the controlnet nodes, but it's doing all kinda stuff, because (I assume) you still have to add a lot of prompts manually on what you see in the original picture to get a somehow decent result.

Here's a quick idea on how I do want my workflow to look like:
View attachment 3138236

I might need to add more nodes to the workflow to downscale/upscale/inpaint/outpaint the images etc., but you might get the basic idea of what I'm trying to achieve. The issue is the red node, which I know exists, but I don't know what it's called and which custom nodes I'll have to install to get that specific node.

You're looking for something like a interogation node. That'll analyze the image for you and create something you can use as prompt. Double click on the background in Comfyui and search for wd14. You might need to install a nodepack called Comfyui WD 1.4 Tagger, should be easy to find in the manager.

 

[Sepheyer]

You're looking for something like a interogation node. That'll analyze the image for you and create something you can use as prompt. Double click on the background in Comfyui and search for wd14. You might need to install a nodepack called Comfyui WD 1.4 Tagger, should be easy to find in the manager.

-WD14, bro, what do you see?

- masterpiece, 1girl, 4k

- Thanks bro.

 

[theMickey_]

u might need to install a nodepack called Comfyui WD 1.4 Tagger, should be easy to find in the manager.

Thank you SO MUCH for this -- this really helped a lot! After installing this pack, I did a quick test, and this is the "before" and "after":

​

And if I then just enter "red lipstick" into the condition to emphasis that, this is what I get:

​

(You can get the messy workflow from those "after" images...)

That's awesome, and pretty much what I was looking for! Based on that workflow I now can try to improve those images...

Again, thanks a lot for recommending that node!

 

[theMickey_]

-WD14, bro, what do you see?

Hm. I got some decent result for the example I posted above:

 

[theMickey_]

Quick question: Is there a way to unload stuff (like models, loras etc.) from your running queue so it doesn't consume memory anymore? Or is that something that happens automatically?

I'm trying to replace a face in a video (still have to find a way to replace the hair as well, but that's something I'm still working on...). I'm loading an image of a face, and using a photorealistic model/checkpoint + conditions + a sampler to make it more appealing and matching to the video. When done, I'm using the resulting face as input for "ReActor" to replace a face in the video (about ~1:25 minutes, about 2.1k frames at 25 fps). But I'm getting an out-of-memory error mid-way through the queue, and I was wondering if the model I loaded to redefine the face is still loaded and could be somehow "unloaded" to free up some memory (as it's not needed anymore to remodel the face for every single frame -- it's a once-per-run step).

I'm not sure if that would solve the problem, and while typing this, I could probably test it by doing this in two separate workflows: first create the face I want to use as a replacement, and then in another workflow use that (saved) face to replace the face in the video... But anyways, the question stands: can I somehow free up memory during the process, does this happen automatically, or do I need to split my single workflow in multiple workflows?

I've shorten the video to ~5 seconds / ~125 frames to test it, and my "single workflow" works perfectly fine for that. It's just longer videos that give me that out-of-memory error.


// EDIT: Btw -- what GPUs are you guys using, and how much memory do you have?

I'm currently using a GTX 2080Ti (12 gig of VRAM) on my main PC with 32 gig memory, and I'm still trying to figure out if/how I can use my 2nd PC (GTX 1080 with 8 gig of VRAM and 32 gig of system memory) over the network...

 

[me3]

I had a go at replicating "render to real" that i had in a1111, and tbh i think i got something very wrong with a bunch of controlnet nodes...anyway
This is done just with prompt from wd14, i added a negative prompt to "disallow" renders/cgi etc, but that it. Rest is just AI doing a bunch of stuff, i'd hoped to make it a simple "drop an image and hit go", but unfortunately it seems things are far more touchy than hoped. Strengths seem to be very impacted by model so atm it's gonna need quite a bit of tweaking. Different models put different things on the shirt, but they all agree it has to be green for some annoying reason and at one point the AI decided that grass was a bad thing and turned it all to concrete
Seems there's a long road to go still...

 

[Mr-Fox]

If you go to civitai and search the image page you can find some amazing animations. Sometimes they have said something about how they created it in the comments. This was made with the help of ebsynth. It's a standalone software but there is apparently some form of extension for SD as well. You can find it in the internal extension database.
Ebsynth or the extension has the ability to mask the background so you can make it static and achieve a more stable background according to op of the example video.


(only a sample gif, follow the link for the full video.)

" you can mask background using ebsynth. Inside ebsynth utility, configuration / etc tab, mask mode set to normal. This is default setting in ebsynth.

Install ebsynth utility from extension tab. "

source:

You must be registered to see the links

You must be registered to see the links

------------------------------------------------------------------------------------------------------------------------------------------

Here's a video2video guide on civitai I forgot to include in my last post about creating animations and videos with SD

You must be registered to see the links

 

[Mr-Fox]

I had a go at replicating "render to real" that i had in a1111, and tbh i think i got something very wrong with a bunch of controlnet nodes...anyway
This is done just with prompt from wd14, i added a negative prompt to "disallow" renders/cgi etc, but that it. Rest is just AI doing a bunch of stuff, i'd hoped to make it a simple "drop an image and hit go", but unfortunately it seems things are far more touchy than hoped. Strengths seem to be very impacted by model so atm it's gonna need quite a bit of tweaking. Different models put different things on the shirt, but they all agree it has to be green for some annoying reason and at one point the AI decided that grass was a bad thing and turned it all to concrete
Seems there's a long road to go still...

View attachment 3140685

You can potentially use inpainting to fix the tops print. Sebastian kamph has a video tutorial about controlling text with the help of controlnet inpainting. I don't see why it could not be used for a "graphic design" or a print.

You must be registered to see the links

If all else fails you can edit it afterwards with photoshop or photopea etc. I know it's not as satisfying as having SD do it for you.

 

[theMickey_]

Update about replacing hair: at least I found a workflow to "restyle" hair, I guess that's a start...



(Use those PNG files to get the workflow. Original source

You must be registered to see the links

.)

 

[me3]

If you go to civitai and search the image page you can find some amazing animations. Sometimes they have said something about how they created it in the comments. This was made with the help of ebsynth. It's a standalone software but there is apparently some form of extension for SD as well. You can find it in the internal extension database.
Ebsynth or the extension has the ability to mask the background so you can make it static and achieve a more stable background according to op of the example video.

View attachment 3141774
(only a sample gif, follow the link for the full video.)

" you can mask background using ebsynth. Inside ebsynth utility, configuration / etc tab, mask mode set to normal. This is default setting in ebsynth.

Install ebsynth utility from extension tab. "

source:

You must be registered to see the links

You must be registered to see the links

------------------------------------------------------------------------------------------------------------------------------------------

Here's a video2video guide on civitai I forgot to include in my last post about creating animations and videos with SD

You must be registered to see the links

The "problem" i've found so far is that the animating stuff requires you to do most/all of the frames in a single batch. So for me there's a problem when the smallest batching you can do is 16, it needs a minimum of 8gb VRAM to work "well". So even if it's just a tiny overflow it drastically increases processing time. In some cases it hits >700 s/it...so it'll take you hours just todo one batch which can then give you 1-2 sec clip, if you're lucky.
You can't even work on your prompt with single images, because of how the things work it won't generate anything unless it's at a batch minimum (16), you just get barely altered noise. I was perfectly aware i'd be fighting an uphill battle before starting, but that shouldn't stop ppl from doing things. Time and digging will tell, i'm sure there are or will be solutions that work better eventually

 

[Mr-Fox]

The "problem" i've found so far is that the animating stuff requires you to do most/all of the frames in a single batch. So for me there's a problem when the smallest batching you can do is 16, it needs a minimum of 8gb VRAM to work "well". So even if it's just a tiny overflow it drastically increases processing time. In some cases it hits >700 s/it...so it'll take you hours just todo one batch which can then give you 1-2 sec clip, if you're lucky.
You can't even work on your prompt with single images, because of how the things work it won't generate anything unless it's at a batch minimum (16), you just get barely altered noise. I was perfectly aware i'd be fighting an uphill battle before starting, but that shouldn't stop ppl from doing things. Time and digging will tell, i'm sure there are or will be solutions that work better eventually

I agree.

Things are moving very fast though in terms pf progress. Just look how far we have come in such short time in regards to simple text2img. I think it's a good and sobering exercise to go back to the first pages in this thread and see the amazing progress. I'm excited for what the next year will bring.

 

[Sepheyer]

You're looking for something like a interogation node. That'll analyze the image for you and create something you can use as prompt. Double click on the background in Comfyui and search for wd14. You might need to install a nodepack called Comfyui WD 1.4 Tagger, should be easy to find in the manager.

Great call, this WD14 bro is a major time saver.

 

[Vanaduke]

Hi, I've worked with DAZ to generate 3D images of the mother from Dual Family by Gumdrops (see signature). I'm now interested in generating AI art of the same model.

Basically, BloomingPrince inspired me to one day generate a nude version of his creation:



Does this application require heavy setup? I only have my old laptop (Intel core) with me which I used for generating 3D images of said model. Thanks.

 

[Sepheyer]

Hi, I've worked with DAZ to generate 3D images of the mother from Dual Family by Gumdrops (see signature). I'm now interested in generating AI art of the same model.

Basically, BloomingPrince inspired me to one day generate a nude version of his creation:

View attachment 3145710

Does this application require heavy setup? I only have my old laptop (Intel core) with me which I used for generating 3D images of said model. Thanks.

A warning -- running ComfyUI or its alternative on a CPU will be slow and will probably limit you to 512x512 renders. I haven't tested this, speaking as someone who has an ~1650 nvidia with 6gb that even such potatoe card is taking prohibitively long.

Now, you absolutely can run the ComfyUI on a CPU. The installation and the models will require about 10gb.

Here is the link to get you started:

You must be registered to see the links

Note the line: "Works even if you don't have a GPU with: --cpu (slow)"

 

[theMickey_]

Quick question: 2-3 days ago, when I started with ComfyUI, I also played a bit with "sd_xl_turbo", the model that can render AI images almost in "real time" (in case you haven't heard of it yet, check

You must be registered to see the links

). It was generating new pictures literally while I was typing. Today I wanted to do the same, and using the exact same workflow, it now takes ~3-4 seconds to render a single image. Anyone noticed this as well? And what could have changed its behavior?

Again, I'm using the same workflow and settings shown in the video. But now the "CLIP Text encode (Prompt)" stays active for about 1 second after changing the text, and the "SampleCustom" then takes another 2-3 seconds to actually sample the image (I've replaced the "SamplerCustom" with a simple KSampler, still the same). I'm so confused...

I did install a couple custom_nodes (like IPAdapter, ControlNet etc., just some "basic" stuff), could that have an impact on how things work? Those nodes are not part of my workflow though. There also was an update for my NVIDIA driver, so I'm wondering if that could have any impact.

I'd love to figure that out, because I assume that if something simple as the sd_xl_turbo workflow got like 10-20 times worse, something (anything?) else might be affected as well.

Any advice would be highly appreciated!


P.S.: I might install a "naked" ComfyUI with just the ComyUI-Manager in a second folder and try that Turbo workflow without any additional custom nodes to see if that still works.

// EDIT: a fresh installation (even without the Manager) did fix the issue. FML, now I'm going to need to find out which of the "custom nodes" broke my other installation