Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More flexible neural network training #1

Open
3 tasks
ThibeauWouters opened this issue Aug 1, 2024 · 0 comments
Open
3 tasks

More flexible neural network training #1

ThibeauWouters opened this issue Aug 1, 2024 · 0 comments

Comments

@ThibeauWouters
Copy link
Owner

ThibeauWouters commented Aug 1, 2024

Currently, fiesta only has an MLP neural network implemented which uses the ReLU activation function, and the learning rate is fixed. It would be to generalize the training code to make it more flexible so that users can play around with this.

Some ideas:

  • Add a learning rate scheduler from optax. This is useful since training now plateaus very quickly and we are losing a lot of potential by not varying the learning rate over time.
  • Add other activation functions, make sure they can be saved and loaded without errors
  • Add other neural network architectures. Currently, there is only the MLP, of which the layer sizes can be chosen by the user. More advanced architectures could improve the performance.
@ThibeauWouters ThibeauWouters changed the title Add support for several architectures and activation functions More flexible neural network training Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant