You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, fiesta only has an MLP neural network implemented which uses the ReLU activation function, and the learning rate is fixed. It would be to generalize the training code to make it more flexible so that users can play around with this.
Some ideas:
Add a learning rate scheduler from optax. This is useful since training now plateaus very quickly and we are losing a lot of potential by not varying the learning rate over time.
Add other activation functions, make sure they can be saved and loaded without errors
Add other neural network architectures. Currently, there is only the MLP, of which the layer sizes can be chosen by the user. More advanced architectures could improve the performance.
The text was updated successfully, but these errors were encountered:
ThibeauWouters
changed the title
Add support for several architectures and activation functions
More flexible neural network training
Aug 13, 2024
Currently, fiesta only has an MLP neural network implemented which uses the ReLU activation function, and the learning rate is fixed. It would be to generalize the training code to make it more flexible so that users can play around with this.
Some ideas:
The text was updated successfully, but these errors were encountered: