librilight encoded data

[kevmo314]

hi there, I spent some time transcribing and encoding the libri-light small and medium datasets. since it was a nontrivial amount of work, I'd like to share it more publicly if it is useful to anyone. for each audio file in the small and medium datasets, I've:

1. split the file with voice activity detection (the script in libri-light itself)
2. encoded the audio with encodec
3. transcribed with whisper medium.en

In total, this amounts to ~6000 hours of audio data. you can find all the data here: https://storage.googleapis.com/speech-synthesis-datasets.

I hope this is useful in helping to reproduce the original VALL-E paper, as it does the same with the full 60k hours. I would like to encode and transcribe the large dataset, but I don't have the GPU resources to do so.

I will be using the above dataset to try to get closer to the VALL-E paper with this repo, thanks for your work so far!

[aditisundar]

hey - that link takes me to an XML page like this, do you know why that might be happening?

[kevmo314]

hey - that link takes me to an XML page like this, do you know why that might be happening?

The link is the manifest which tells you which files are available. It's not one single file to download because the flac files may not be needed and take up most of the space.

[aditisundar]

thank you!

[emesoohc2022]

Thanks for your work! May I ask how did you deal with the sampling rate of libri-light datasets? In the original VALL-E paper, 24 kHz Encodec model is used and the sample audios on demo website (web) are of 24 kHz.
However, it seems the audios from Libri-light are 16 kHz. Did you simply upsample the original 16 kHz audios or do some kind of super-resolution for upsampling from 16k to 24k?

[kevmo314]

They were upsampled with ffmpeg, nothing fancy.

[emesoohc2022]

Got it, thanks

[alexgain]

@kevmo314 Do you have code or a repo for how you did this? I'm looking to do this for the full 60k dataset. Happy to share when done.
Thanks!

[xiabo2011]

@kevmo314 Thanks for sharing the librilight data, seems it only has 1 speaker's voice. Speaker id is 100, wondering whether there is 6000 hours