Windwalkers

Arraymancer v0.6.0 - Windwalkers

This release is named after "Windwalkers" (2004, French "La Horde du Contrevent"), by Alain Damasio.

Changes:

• The symeig proc to compute eigenvectors of a symmetric matrix
  now accepts an "uplo" char parameter. This allows to fill only the Upper or Lower
  part of the matrix, the other half is not used in computation.
• Added svd_randomized, a fast and accurate SVD approximation via random sampling.
  This is the standard driver for large scale SVD applications as SVD on large matrices is very slow.
• pca now uses the randomized SVD instead of computing the covariance matrix.
  It can now efficiently deal with large scale problems.
  It now accepts a center, n_oversamples and n_power_iters arguments.
  Note that pca without centering is equivalent to a truncated SVD.
• LU decomposition has been added
• QR decomposition has been added
• hilbert has been introduced. It creates the famous ill-conditioned Hilbert matrix.
  The matrix is suitable to stress test decompositions.
• The arange procedure has been introduced. It creates evenly spaced value within a specified range
  and step
• The ordering of arguments to error functions has been converted to
  (y_pred, y_target) (from (y_target, y_pred)), enabling the syntax y_pred.accuracy_score(y).
  All existing error functions in Arraymancer were commutative w.r.t. to arguments
  so existing code will keep working.
• A solve procedure has been added to solve linear system of equations represented as matrices.
• A softmax layer has been added to the autograd and neural networks
  complementing the SoftmaxCrossEntropy layer which fused softmax + Negative-loglikelihood.
• The stochastic gradient descent now has a version with Momentum

Bug fixes:

• gemm could crash when the result was column major.
• The automatic fusion of matrix multiplication with matrix addition (A * X) + b could update the b matrix.
• Complex converters do not pollute the global namespace and do not
  prevent string conversion via $ of number types due to ambiguous call.
• in-place division has been fixed, a typo made it into substraction.
• A conflict between NVIDIA "nanosecond" and Nim times module "nanosecond"
  preventing CUDA compilation has been fixed

Breaking:

• In symeig, the eigenvectors argument is now called return_eigenvectors.
• In symeig with slice, the new uplo precedes the slice argument.
• pca input "nb_components" has been renamed "n_components".
• pca output tuple used the names (results, components). It has been renamed to (projected, components).
• A pca overload that projected a data matrix on already existing principal axes
  was removed. Simply multiply the mean-centered data matrix with the components instead.
• Complex converters were removed. This prevents hard to debug and workaround implicit conversion bug in downstream library.
  If necessary, users can re-implement converters themselves.
  This also provides a 20% boost in Arraymancer compilation times

Deprecation:

• The syntax gemm(A, B, C) is now deprecated.
  Use explicit "gemm(1.0, A, B, 0.0, C)" instead.
  Arguably not zero-ing C could also be a reasonable default.
• The dot in broadcasting and elementwise operators has changed place
  Use +., *., /., -., ^., +.=, *.=, /.=, -.=, ^.=
  instead of the previous order .+ and .+=.
  This allows the broadcasting operators to have the same precedence as the
  natural operators.
  This also align Arraymancer with other Nim packages: Manu and NumericalNim

Thanks to @dynalagreen for the SGD with Momentum, @xcokazaki for spotting the in-place division typo,
@Vindaar for fixing the automatic matrix multiplication and addition fusion,
@Imperator26 for the Softmax layer, @brentp for reviewing and augmenting the SVD and PCA API,
@auxym for the linear equation solver and @berquist for the reordering all error functions to the new API.
Thanks @B3liever for suggesting the dot change to solve the precedence issue in broadcasting and elementwise operators.