New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Help]: Inquiry About Padding Handling in VALL-E AR Loss Calculation #381

Open

jubang0219 opened this issue Jan 8, 2025 · 0 comments

jubang0219 commented Jan 8, 2025

Hi there,

I have a question regarding the handling of padding during loss calculation in the AR training process of the VALL-E model.

Relevant code link: https://github.com/open-mmlab/Amphion/blob/main/models/tts/valle/valle.py#L374

In the script Amphion/models/tts/valle/valle.py, is the padding not specifically handled during AR stage training?

ar_loss = F.cross_entropy(logits, targets, reduction=reduction)

For the NAR stage, I observed that the padding is processed using the ignore option.

nar_loss = F.cross_entropy(
logits,
targets,
ignore_index=self.audio_token_num,
reduction=reduction,
) * (total_length / (total_length - prefix_len * x.shape[0]))

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment