[docs] improve manual for NN extensions (#152)

docs/src/manual/AbstractGPs.md

@@ -1,34 +1,23 @@
-# AbstractGPs
+# AbstractGPs.jl
 
 [AbstractGPs.jl](https://github.com/JuliaGaussianProcesses/AbstractGPs.jl) is a
 library for fittinng Gaussian Processes in Julia.
 
-Here is an example:
-
-```jldoctest
-julia> using JuMP, MathOptAI, AbstractGPs
-
-julia> x_data = 2π .* (0.0:0.1:1.0);
-
-julia> y_data = sin.(x_data);
-
-julia> fx = AbstractGPs.GP(AbstractGPs.Matern32Kernel())(x_data, 0.1);
-
-julia> p_fx = AbstractGPs.posterior(fx, y_data);
+## Basic example
 
-julia> model = Model();
-
-julia> @variable(model, 1 <= x[1:1] <= 6, start = 3);
-
-julia> predictor = MathOptAI.Quantile(p_fx, [0.1, 0.9]);
-
-julia> y, formulation = MathOptAI.add_predictor(model, predictor, x);
-
-julia> y
-2-element Vector{VariableRef}:
- moai_quantile[1]
- moai_quantile[2]
+Here is an example:
 
-julia> @objective(model, Max, y[2] - y[1])
-moai_quantile[2] - moai_quantile[1]
+```@repl
+using JuMP, MathOptAI, AbstractGPs
+x_data = 2π .* (0.0:0.1:1.0);
+y_data = sin.(x_data);
+fx = AbstractGPs.GP(AbstractGPs.Matern32Kernel())(x_data, 0.1);
+p_fx = AbstractGPs.posterior(fx, y_data);
+model = Model();
+@variable(model, 1 <= x[1:1] <= 6, start = 3);
+predictor = MathOptAI.Quantile(p_fx, [0.1, 0.9]);
+y, formulation = MathOptAI.add_predictor(model, predictor, x);
+y
+formulation
+@objective(model, Max, y[2] - y[1])
 ```

docs/src/manual/DecisionTree.md

@@ -1,35 +1,22 @@
-# DecisionTree
+# DecisionTree.jl
 
 [DecisionTree.jl](https://github.com/JuliaAI/DecisionTree.jl) is a library for
 fitting decision trees in Julia.
 
-Here is an example:
-
-```jldoctest
-julia> using JuMP, MathOptAI, DecisionTree
-
-julia> truth(x::Vector) = x[1] <= 0.5 ? -2 : (x[2] <= 0.3 ? 3 : 4)
-truth (generic function with 1 method)
-
-julia> features = abs.(sin.((1:10) .* (3:4)'));
-
-julia> size(features)
-(10, 2)
+## Basic example
 
-julia> labels = truth.(Vector.(eachrow(features)));
-
-julia> ml_model = DecisionTree.build_tree(labels, features)
-Decision Tree
-Leaves: 3
-Depth:  2
-
-julia> model = Model();
-
-julia> @variable(model, 0 <= x[1:2] <= 1);
-
-julia> y, formulation = MathOptAI.add_predictor(model, ml_model, x);
+Here is an example:
 
-julia> y
-1-element Vector{VariableRef}:
- moai_BinaryDecisionTree_value
+```@repl
+using JuMP, MathOptAI, DecisionTree
+truth(x::Vector) = x[1] <= 0.5 ? -2 : (x[2] <= 0.3 ? 3 : 4)
+features = abs.(sin.((1:10) .* (3:4)'));
+size(features)
+labels = truth.(Vector.(eachrow(features)));
+predictor = DecisionTree.build_tree(labels, features)
+model = Model();
+@variable(model, 0 <= x[1:2] <= 1);
+y, formulation = MathOptAI.add_predictor(model, predictor, x);
+y
+formulation
 ```

docs/src/manual/Flux.md

@@ -1,37 +1,85 @@
-# Flux
+# Flux.jl
 
 [Flux.jl](https://github.com/FluxML/Flux.jl) is a library for machine learning
 in Julia.
 
-MathOptAI supports embedding Flux models in JuMP if they are a chain composed
-of:
+The upstream documentation is available at
+[https://fluxml.ai/Flux.jl/stable/](https://fluxml.ai/Flux.jl/stable/).
 
- * `Flux.Dense`
- * `Flux.softmax`
- * `Flux.relu`
- * `Flux.sigmoid`
- * `Flux.softplus`
- * `Flux.tanh`
+## Supported layers
 
-Here is an example:
+MathOptAI supports embedding a Flux model into JuMP if it is a
+[`Flux.Chain`](https://fluxml.ai/Flux.jl/stable/reference/models/layers/#Flux.Chain)
+composed of:
 
-```jldoctest
-julia> using JuMP, Flux, MathOptAI
+  * [`Flux.Dense`](https://fluxml.ai/Flux.jl/stable/reference/models/layers/#Flux.Dense)
+  * [`Flux.Scale`](https://fluxml.ai/Flux.jl/stable/reference/models/layers/#Flux.Scale)
+  * [`Flux.relu`](https://fluxml.ai/Flux.jl/stable/reference/models/activation/#NNlib.relu)
+  * [`Flux.sigmoid`](https://fluxml.ai/Flux.jl/stable/reference/models/activation/#NNlib.sigmoid)
+  * [`Flux.softmax`](https://fluxml.ai/Flux.jl/stable/reference/models/nnlib/#NNlib.softmax)
+  * [`Flux.softplus`](https://fluxml.ai/Flux.jl/stable/reference/models/activation/#NNlib.softplus)
+  * `Flux.tanh`
 
-julia> chain = Flux.Chain(Flux.Dense(1 => 16, Flux.relu), Flux.Dense(16 => 1));
+## Basic example
 
-julia> model = Model();
+Use [`MathOptAI.add_predictor`](@ref) to embed a `Flux.Chain` into a JuMP model:
 
-julia> @variable(model, x[1:1]);
+```@repl
+using JuMP, Flux, MathOptAI
+predictor = Flux.Chain(Flux.Dense(1 => 2, Flux.relu), Flux.Dense(2 => 1));
+model = Model();
+@variable(model, x[1:1]);
+y, formulation = MathOptAI.add_predictor(model, predictor, x);
+y
+formulation
+```
+
+## Reduced-space
+
+Use the `reduced_space = true` keyword to formulate a reduced-space model:
+
+```@repl
+using JuMP, Flux, MathOptAI
+predictor = Flux.Chain(Flux.Dense(1 => 2, Flux.relu), Flux.Dense(2 => 1));
+model = Model();
+@variable(model, x[1:1]);
+y, formulation =
+    MathOptAI.add_predictor(model, predictor, x; reduced_space = true);
+y
+formulation
+```
+
+## Gray-box
+
+Use the `gray_box = true` keyword to embed the network as a nonlinear operator:
+
+```@repl
+using JuMP, Flux, MathOptAI
+predictor = Flux.Chain(Flux.Dense(1 => 2, Flux.relu), Flux.Dense(2 => 1));
+model = Model();
+@variable(model, x[1:1]);
+y, formulation =
+    MathOptAI.add_predictor(model, predictor, x; gray_box = true);
+y
+formulation
+```
+
+## Change how layers are formulated
 
-julia> y, formulation = MathOptAI.add_predictor(
-           model,
-           chain,
-           x;
-           config = Dict(Flux.relu => MathOptAI.ReLU()),
-       );
+Pass a dictionary to the `config` keyword that maps Flux activation functions to
+a MathOptAI predictor:
 
-julia> y
-1-element Vector{VariableRef}:
- moai_Affine[1]
+```@repl
+using JuMP, Flux, MathOptAI
+predictor = Flux.Chain(Flux.Dense(1 => 2, Flux.relu), Flux.Dense(2 => 1));
+model = Model();
+@variable(model, x[1:1]);
+y, formulation = MathOptAI.add_predictor(
+    model,
+    predictor,
+    x;
+    config = Dict(Flux.relu => MathOptAI.ReLUSOS1()),
+);
+y
+formulation
 ```

docs/src/manual/GLM.md

@@ -1,4 +1,4 @@
-# GLM
+# GLM.jl
 
 [GLM.jl](https://github.com/JuliaStats/GLM.jl) is a library for fitting
 generalized linear models in Julia.
@@ -14,22 +14,15 @@ The input `x` to [`add_predictor`](@ref) must be a vector with the same number
 of elements as columns in the training matrix. The return is a vector of JuMP
 variables with a single element.
 
-```jldoctest
-julia> using GLM, JuMP, MathOptAI
-
-julia> X, Y = rand(10, 2), rand(10);
-
-julia> model_glm = GLM.lm(X, Y);
-
-julia> model = Model();
-
-julia> @variable(model, x[1:2]);
-
-julia> y, formulation = MathOptAI.add_predictor(model, model_glm, x);
-
-julia> y
-1-element Vector{VariableRef}:
- moai_Affine[1]
+```@repl
+using GLM, JuMP, MathOptAI
+X, Y = rand(10, 2), rand(10);
+predictor = GLM.lm(X, Y);
+model = Model();
+@variable(model, x[1:2]);
+y, formulation = MathOptAI.add_predictor(model, predictor, x);
+y
+formulation
 ```
 
 ## Logistic regression
@@ -38,22 +31,15 @@ The input `x` to [`add_predictor`](@ref) must be a vector with the same number
 of elements as columns in the training matrix. The return is a vector of JuMP
 variables with a single element.
 
-```jldoctest
-julia> using GLM, JuMP, MathOptAI
-
-julia> X, Y = rand(10, 2), rand(Bool, 10);
-
-julia> model_glm = GLM.glm(X, Y, GLM.Bernoulli());
-
-julia> model = Model();
-
-julia> @variable(model, x[1:2]);
-
-julia> y, formulation = MathOptAI.add_predictor(model, model_glm, x);
-
-julia> y
-1-element Vector{VariableRef}:
- moai_Sigmoid[1]
+```@repl
+using GLM, JuMP, MathOptAI
+X, Y = rand(10, 2), rand(Bool, 10);
+predictor = GLM.glm(X, Y, GLM.Bernoulli());
+model = Model();
+@variable(model, x[1:2]);
+y, formulation = MathOptAI.add_predictor(model, predictor, x);
+y
+formulation
 ```
 
 ## DataFrames
@@ -65,31 +51,17 @@ The input `x` to [`add_predictor`](@ref) must be a DataFrame with the same
 feature columns as the training DataFrame. The return is a vector of JuMP
 variables, with one element for each row in the DataFrame.
 
-```jldoctest
-julia> using DataFrames, GLM, JuMP, MathOptAI
-
-julia> train_df = DataFrames.DataFrame(x1 = rand(10), x2 = rand(10));
-
-julia> train_df.y = 1.0 .* train_df.x1 + 2.0 .* train_df.x2 .+ rand(10);
-
-julia> predictor = GLM.lm(GLM.@formula(y ~ x1 + x2), train_df);
-
-julia> model = Model();
-
-julia> test_df = DataFrames.DataFrame(
-           x1 = rand(6),
-           x2 = @variable(model, [1:6]),
-       );
-
-julia> test_df.y, _ = MathOptAI.add_predictor(model, predictor, test_df);
-
-julia> test_df.y
-6-element Vector{VariableRef}:
- moai_Affine[1]
- moai_Affine[1]
- moai_Affine[1]
- moai_Affine[1]
- moai_Affine[1]
- moai_Affine[1]
+```@repl
+using DataFrames, GLM, JuMP, MathOptAI
+train_df = DataFrames.DataFrame(x1 = rand(10), x2 = rand(10));
+train_df.y = 1.0 .* train_df.x1 + 2.0 .* train_df.x2 .+ rand(10);
+predictor = GLM.lm(GLM.@formula(y ~ x1 + x2), train_df);
+model = Model();
+test_df = DataFrames.DataFrame(
+    x1 = rand(6),
+    x2 = @variable(model, [1:6]),
+);
+test_df.y, _ = MathOptAI.add_predictor(model, predictor, test_df);
+test_df.y
 ```