Native Fine tuning would be the closest thing to training a checkpoint from the ground up.
Dreambooth training is close to native fine tuning only your creating a lora rather then training the entire checkpoint.
Ada factor, lion and prodigy are considered inferior to Adam however the resources needed are also less.
You can native fintune a SD 1.5 model with Lion with only 8GB of VRAM Adam quadruples that.
When fine tuning you also use the EMA weights vs the inference non ema weights checkpoint
(The SD 1.5 EMA is almost 8GB)
So an XL checkpoint like
You must be registered to see the links
ends up around 13GB something most people can't even load let alone train with.
(Non EMA Juggernaut X is only 7.1GB)
For XL models both those options are out of reach for most people so Ill focus on the basic Lora.
Civitai can train a basic lora for PONY up to 10,000 steps (50,000 to 80,000 repeats depending on batch size limit).
For pony it has a batch size of 5 so that allows for lora's with an image count of 400-800 to train to convergence.
My rig would be 30/secs per it for a 5 batch size on 1024x1024, it will not train at 2048x2048
(Thus all but 2 of my XL trainings have been done by Civitai)
You can train a high rank lora then adjust it using Kohya, I never merge at full wieght with a lora it is usally .1-.2.
You can rank a lora down if it is over fitting. (You can rank up but it is not advised)
So in short with Civati allowing 2k training and around the number of steps needed to refine a checkpoint (100k or so) you can merge a high quaility lora into a checkpoint.
Extensive testing should be done before hand to make sure your not causing catastrophic loss to the model. I would never merge my tanlines model as I could not find enough images to make a model that doesnt create a beach scene when you ask for a cityscape.
