AE+CLIP+SSL Loss Balancing Coefficients #8
Description
Hello @JingfengYao thank you for the great work. I am trying to reproducing the VTP tokenizer pre-training with all AE+CLIP+SSL objectives. In your updated technical report, I can see:
For weighting, we set 𝜆_rec = 0.1, while 𝜆_clip and 𝜆_ssl are set to either 0 or 1.
However, I guess that's for the AE+CLIP or AE+SSL version.
May I know how do you set the loss weighting coefficients when all the three objectives are present? I am also curious if you apply some tricks regarding this version during training.
I would appreciate any help on this.