API Reference

Graph(root_node::T) where T

Create an empty graph struture with the root node root_node.

add_node(graph::Graph{T}, node::T) where T

Add a node to the graph graph.

add_node(graph::Graph{T}, node::T) where T

Add an edge to the graph graph.

LinearGraph(stages::Int)

MarkovianGraph(transition_matrices::Vector{Matrix{Float64}})

MarkovianGraph(; stages::Int,
               transition_matrix::Matrix{Float64},
               root_node_transition::Vector{Float64})

Construct a Markovian graph object.

LinearPolicyGraph(builder::Function; stages::Int, kwargs...)

Create a linear policy graph with stages number of stages.

See PolicyGraph for the other keyword arguments.

MarkovianPolicyGraph(builder::Function;
    transition_matrices::Vector{Array{Float64, 2}}, kwargs...)

Create a Markovian policy graph based on the transition matrices given in transition_matrices.

See PolicyGraph for the other keyword arguments.

PolicyGraph(builder::Function, graph::Graph{T};
            bellman_function = AverageCut,
            optimizer = nothing,
            direct_mode = true) where T

Construct a a policy graph based on the graph structure of graph. (See Graph for details.)

Example

function builder(subproblem::JuMP.Model, index)
    # ... subproblem definition ...
end
model = PolicyGraph(builder, graph;
                    bellman_function = AverageCut,
                    optimizer = with_optimizer(GLPK.Optimizer),
                    direct_mode = false)

Or, using the Julia do ... end syntax:

model = PolicyGraph(graph;
                    bellman_function = AverageCut,
                    optimizer = with_optimizer(GLPK.Optimizer),
                    direct_mode = true) do subproblem, index
    # ... subproblem definitions ...
end

@stageobjective(subproblem, expr)

Set the stage-objective of subproblem to expr.

Example

@stageobjective(subproblem, 2x + y)

parameterize(modify::Function,
             subproblem::JuMP.Model,
             realizations::Vector{T},
             probability::Vector{Float64} = fill(1.0 / length(realizations))
                 ) where T

Add a parameterization function modify to subproblem. The modify function takes one argument and modifies subproblem based on the realization of the noise sampled from realizations with corresponding probabilities probability.

In order to conduct an out-of-sample simulation, modify should accept arguments that are not in realizations (but still of type T).

Example

Kokako.parameterize(subproblem, [1, 2, 3], [0.4, 0.3, 0.3]) do ω
    JuMP.set_upper_bound(x, ω)
end

add_objective_state(update::Function, subproblem::JuMP.Model; kwargs...)

Add an objective state variable to subproblem.

Required kwargs are:

• initial_value: The initial value of the objective state variable at the root node.
• lipschitz: The lipschitz constant of the objective state variable.

Setting a tight value for the lipschitz constant can significantly improve the speed of convergence.

Optional kwargs are:

• lower_bound: A valid lower bound for the objective state variable. Can be -Inf.
• upper_bound: A valid upper bound for the objective state variable. Can be +Inf.

Setting tight values for these optional variables can significantly improve the speed of convergence.

If the objective state is N-dimensional, each keyword argument must be an NTuple{N, Float64}. For example, initial_value = (0.0, 1.0).

objective_state(subproblem::JuMP.Model)

numerical_stability_report([io::IO=stdout,] model::PolicyGraph,
                           by_node::Bool=false, print=true, warn::Bool=true)

Print a report identifying possible numeric stability issues.

• If by_node, print a report for each node in the graph.
• If print, print to io.
• If warn, warn if the coefficients may cause numerical issues.

Kokako.train(model::PolicyGraph; kwargs...)

Train the policy of the model. Keyword arguments are

• iteration_limit: number of iterations to conduct before termination
• time_limit: number of seconds to train before termination
• print_level: control the level of printing to the screen
• log_file: filepath at which to write a log of the training progress
• runnumericalstability_report: generate a numerical stability report prior to solve
• risk_measure
• stoping_rules
• sampling_scheme: a sampling scheme to use on the forward pass of the algorithm. Defaults to InSampleMonteCarlo().
• refineatsimilar_nodes

There is also a special option for infinite horizon problems

• cyclediscretizationdelta: the maximum distance between states allowed on the forward pass.

termination_status(model::PolicyGraph)

Query the reason why the training stopped.

simulate(model::PolicyGraph,
         number_replications::Int = 1,
         variables::Vector{Symbol} = Symbol[];
         sampling_scheme::AbstractSamplingScheme =
             InSampleMonteCarlo(),
         custom_recorders = Dict{Symbol, Function}()
 )::Vector{Vector{Dict{Symbol, Any}}}

Perform a simulation of the policy model with number_replications replications using the sampling scheme sampling_scheme.

Returns a vector with one element for each replication. Each element is a vector with one-element for each node in the scenario that was sampled. Each element in that vector is a dictionary containing information about the subproblem that was solved.

In that dictionary there are four special keys:

• :node_index, which records the index of the sampled node in the policy model
• :noise_term, which records the noise observed at the node
• :stage_objective, which records the stage-objective of the subproblem
• :bellman_term, which records the cost/value-to-go of the node.

The sum of :stageobjective + :bellmanterm will equal the objective value of the solved subproblem.

In addition to the special keys, the dictionary will contain the result of JuMP.value(subproblem[key]) for each key in variables. This is useful to obtain the primal value of the state and control variables.

For more complicated data, the custom_recorders keyword arguement can be used.

data = Dict{Symbol, Any}()
for (key, recorder) in custom_recorders
    data[key] = foo(subproblem)
end

For example, to record the dual of a constraint named my_constraint, pass the following:

simulation_results = simulate(model, number_replications=2;
    custom_recorders = Dict(
        :constraint_dual = (sp) -> JuMP.dual(sp[:my_constraint])
    )
)

The value of the dual in the first stage of the second replication can be accessed as:

simulation_results[2][1][:constraint_dual]

Kokako.calculate_bound(model::PolicyGraph, state::Dict{Symbol, Float64},
                       risk_measure=Expectation())

Calculate the lower bound (if minimizing, otherwise upper bound) of the problem model at the point state, assuming the risk measure at the root node is risk_measure.

Historical(scenarios::Vector{Vector{Tuple{T, S}}},
           probability::Vector{Float64})

A sampling scheme that samples a scenario from the vector of scenarios scenarios according to probability. If probability omitted, defaults to uniform probability.

Example

Historical(
    [
        [(1, 0.5), (2, 1.0), (3, 0.5)],
        [(1, 0.5), (2, 0.0), (3, 1.0)],
        [(1, 1.0), (2, 0.0), (3, 0.0)]
    ],
    [0.2, 0.5, 0.3]
)

Historical(scenario::Vector{Tuple{T, S}})

A deterministic sampling scheme that always samples scenario with probability 1.

Example

Historical([(1, 0.5), (2, 1.5), (3, 0.75)])

Kokako.SpaghettiPlot(; stages, scenarios)

Initialize a new SpaghettiPlot with stages stages and scenarios number of replications.

Kokako.add_spaghetti(data_function::Function, plt::SpaghettiPlot; kwargs...)

Description

Add a new figure to the SpaghettiPlot plt, where the y-value of the scenarioth line when x = stage is given by data_function(plt.simulations[scenario][stage]).

Keyword arguments

• xlabel: set the xaxis label
• ylabel: set the yaxis label
• title: set the title of the plot
• ymin: set the minimum y value
• ymax: set the maximum y value
• cumulative: plot the additive accumulation of the value across the stages
• interpolate: interpolation method for lines between stages.

Defaults to "linear" see the d3 docs for all options.

Examples

simulations = simulate(model, 10)
plt = Kokako.spaghetti_plot(simulations)
Kokako.add_spaghetti(plt; title = "Stage objective") do data
	return data[:stage_objective]
end

Kokako.publication_plot(
    data_function, simulations;
    quantile = [0.0, 0.1, 0.25, 0.5, 0.75, 0.9, 1.0],
    kwargs...)

Create a Plots.jl recipe plot of the simulations.

See Plots.jl for the list of keyword arguments.

Example

Kokako.publication_plot(simulations; title = "My title") do data
    return data[:stage_objective]
end