Neural Network optimizer implementation

From scratch implementation of Neural Network optimization on multi-class classification problem. The Neural Network consists of only one hidden layer for simplicity.

Data

For testing purposes, Swiss roll dataset is used.

Implementation

The implementation includes function for:

• feed-forward and backpropagation pass,
• softmax and softplus,
• loss,
• initialization of parameters and
• plotting.

The optimization is done implementing steepest descent algoritm, with optional Armijo rule for selecting step size.