Model diff with Wavtokenizer

[erogol]

I think wavtokenizer uses non causal encodec encoder but simvq causal

[youngsheen]

I follow the configuration of EnCodec, which utilizes causal=True to support streaming encoding. Based on empirical evidence, causal performs worse than non-causal. Consequently, if you opt to set causal=False, the model may achieve better performance.

[erogol]

Another difference is that the released wavtokenizer checkpoints are not trained with EMA. I sent the PR for EMA, but I don't think they used it.

Dont take me wrong just pointing out differences for help

[erogol]

yet another note, I think in the GAN literature EMA is only being applied to the generator but I see SimVQ uses EMA for discriminators too. It might improve your results.

[hertz-pj]

In wavtokenizer, normalization is done as follows:

    #gain = np.random.uniform(-1, -6) if self.NAME == "train" else -3
    #waveform, _ = torchaudio.sox_effects.apply_effects_tensor(waveform, self.sample_rate, [["norm", f"{gain:.2f}"]])

However, in this work, it changed to:

if self.audio_normalize:
    mono = x.mean(dim=1, keepdim=True)
    volume = mono.pow(2).mean(dim=2, keepdim=True).sqrt()
    scale = 1e-8 + volume
    x = x / scale
    scale = scale.view(-1, 1)
else:
    scale = None

Could you explain the considerations behind this change? Has there been a comparison to determine which method is better?