Some optimization ideas

[jdwang125]

1. Ensure that each convolution is followed by an activation function to enhance the network's fitting ability:

• Add LeakyReLU after the First conv
• Add LeakyReLU after each EncBlock

2. Fully utilize the dataset:
   Currently, the code only uses the first 2 seconds of each corpus for training. The mls_french dataset has a duration of approximately 1000 hours, but only around 140 hours are actually utilized. Can we use the full length of each corpus? This can help improve fitting accuracy and generalization ability.

[jhauret]

Thank you for your suggestions but they are already integrated.

1. The code uses the non-linearity between each encoder_block and after the first_conv thanks to self.nl(x)
2. You can use a random selection of audio by setting the deterministic flag of TemporalTransforms to False to generate a random start index.

[jdwang125]

Hi jhauret:
Sorry, I didn't look carefully at the code and thought these two places were missing self.nl.
Regarding the second point, do you think it's better to use the entire length of the corpus or randomly selected 2-second segments from each piece of the corpus?

[jhauret]

No problem!

Now to the second point. From a mathematical point of view, it does not make much difference whether you optimize the generator and discriminators on a 4-second sample or on two 2-second samples, since the receptive fields of the networks are smaller than 2 seconds. In fact, the total gradient is the same if the 2-second samples correspond to the first and second halves of the 4-second sample (edge effects are neglected). However, from a computational point of view, more parallelization can be achieved by increasing the batch size and reducing the time length. Also, if you want to use the full length of each sample, you would have to do padding for batch size>1, as some are longer than others.