Oversaturated result with low cfg.

[crapthings]

a1111

cfg4

cfg2

diffusers 0.30.2

cfg4

cfg2

i think the diffusers is better, but the issue is it looks oversaturated.
if i enable controlnet with diffusers, the result looks worst.

[asomoza]

Hi, what model is this? it looks like SD 1.5 or a really plastic SDXL.

I can't really help that much with that if it is SD 1.5, but as a reminder, diffusers is not a 1:1 conversion from auto1111, so the CFG and the other parameters don't have the same equivalent, and also, you can't reproduce the same results between them.

Still this seems and issue with the vae if it is SD 1.5, also the controlnet in auto1111 has some auto guidance scaling that we don't have in diffusers so you have to implement that if you want the similar results or adjust the parameters accordingly.

I stopped using auto1111 a long time ago, but if it followed the steps of the other webuis, probably its doing something else under the hood.

My best advice is that don't compare them, if you want to use diffusers, just play with it until you get the results you want, you can achieve the same and probably better results but for that, you will have to stop expecting to get the same results with the same parameters like the CFG or the controlnet scales.

[crapthings]

i don't use a1111
i'm only using diffusers for custom my own workflow
but i test model in a1111 sometimes after finetune
i don't need result same as a1111 but how to prevent the plastic or oversaturated?

the controlnet is trained from scratch by diffusers/example/controlnet/trainscript with specify base model
but if the base model retrained or changed other model, the controlnet result got worst
so i wonder if i retrain base model make it look good then train controlnet again will be better?

maybe overtrained base model or controlnet?

this is good sample example

[asomoza]

That's why I wrote that probably I won't be able to help here, I also like more the diffusers results and I don't think they're over saturated, they have more color saturation but to me eyes they are in no way over saturated, also this is something you can change in post production (less saturated or more saturated), as in real photography, the more color information you have the better to work with it in post and this is the only way to get more color information with AI generations until someone makes the models produce RAWs.

Also the other reason is that I'm too used to good SDXL models so I find all the results you showed plastic and with fake skin, like a doll, and there's no model in SD 1.5 that IMO fixes this, this is different if you're using SDXL, with SDXL you just need to train with a mix of real non professional photos instead of just professional/photoshoped ones with perfect skin.

For the controlnet model, it's weird that it degrades if you change the model if you train with the base SD model, if you don't, you will get this, the same as if you train a lora with a fine tuned model versus the base one, specially if the fine tuned model differs a lot from the base model.

For better answers you will need to give more information though:

• model you're using
• what are you training
• prompts and params of inference
• training params

Without this, it's hard to give a more accurate opinion, also this will have better answers if you ask in some of the good controlnets repos, the model owners of those have a lot of real experience with controlnets.

[crapthings]

@asomoza

I'm facing two issues:

• The face cannot be locked onto a specific person.
• How to make the generated scene similar to the training images (a specific scene).

controlnet and IP adapter seem to degrade the generation quality and not summon the training dataset(face, background),I'm not sure why.

i can summon the face without controlnet, finetune or lora both works fine.
but add controlnet or ip adapter can't get the face.

[asomoza]

If controlnets and IP adapters degrade the generation is because you're using a high scale with them and probably a high CFG with a restrictive prompt. Too many restrictions on the generation will make the results bad.

Also if the base model is overtrained it will produce bad results if what you're trying to do goes against it, for example, trying to do photorealism with an anime model as an extreme case.

The face cannot be locked onto a specific person.

You want to generate a person with a specific face? People (including me) use IP Face Adapters or variations (PulID, InstantID, PhotoMaker, etc) for this, and it works good, you can see the results everywhere.

How to make the generated scene similar to the training images (a specific scene).

With training it will be hard, the whole motivation of training a model is for it to learn how to be flexible with what it learns, you can try to describe the scene and if it has a good training it will be similar but not perfect if it has a lot of details, so it all depend on how similar you want it to be.

Also you probably don't need this, you can just use img2img for this or a masked IP Adapter or a combination of both to get good results, also if you use SD 1.5 you can probably use LayerDiffuse.

[crapthings]

You want to generate a person with a specific face? People (including me) use IP Face Adapters or variations (PulID, InstantID, PhotoMaker, etc) for this, and it works good, you can see the results everywhere.

is this working with controlnet?
i've tested ip face adapters with current workflow, but result is very bad.

without controlnet its okay.
my workflow is
this works good for now

1. stable diffusion pag controlnet
2. custom controlnet pose+rgba mask
3. asdff(adetailer) from_pretrained(step1's same model)

but i m trying to add new face and new background to it.
ip adapters(or face) doesn't work will with controlnet, if i lower the controlnet stregnth, the control is not working.
i test lora add both 1,3 the face doesn't same like lora, don't know why

i think maybe the base model is overtrained.
should i retrain the controlnet when base model retrained?
is the face should learned by controlnet too?
is it possible to make lora work with face or background, if the base model and the controlnet is properly trained?

---

can i resume trainning with like epicrealism or other photo realistic model?
or should i resume from base 1.5 model, but how many image works?
i have 30000 specify for fasihon 200000(generalize, include fashion)
is this enough for finetune base 1.5 model?

---

update:

all i got is distortion image