Intermediate I: risk

Risk measures

To illustrate the risk-measures included in SDDP.jl, we consider a discrete random variable with four outcomes.

The random variable is supported on the values 1, 2, 3, and 4:

julia> noise_supports = [1, 2, 3, 4]
4-element Array{Int64,1}:
 1
 2
 3
 4

The associated probability of each outcome is as follows:

julia> nominal_probability = [0.1, 0.2, 0.3, 0.4]
4-element Array{Float64,1}:
 0.1
 0.2
 0.3
 0.4

With each outcome ω, the agent observes a cost Z(ω):

julia> cost_realizations = [5.0, 4.0, 6.0, 2.0]
4-element Array{Float64,1}:
 5.0
 4.0
 6.0
 2.0

We assume that we are minimizing:

julia> is_minimization = true
true

Finally, we create a vector that will be used to store the risk-adjusted probabilities:

julia> risk_adjusted_probability = zeros(4)
4-element Array{Float64,1}:
 0.0
 0.0
 0.0
 0.0

Expectation

Expectation()

Kokako.adjust_probability(
    Kokako.Expectation(),
    risk_adjusted_probability,
    nominal_probability,
    noise_supports,
    cost_realizations,
    is_minimization
)

risk_adjusted_probability

# output

4-element Array{Float64,1}:
 0.1
 0.2
 0.3
 0.4

Kokako.Expectation is the default risk measure in SDDP.jl.

Worst-case

WorstCase()

Kokako.adjust_probability(
    Kokako.WorstCase(),
    risk_adjusted_probability,
    nominal_probability,
    noise_supports,
    cost_realizations,
    is_minimization
)

risk_adjusted_probability

# output

4-element Array{Float64,1}:
 0.0
 0.0
 1.0
 0.0

Average value at risk (AV@R)

AVaR(β)

Kokako.adjust_probability(
    Kokako.AVaR(0.5),
    risk_adjusted_probability,
    nominal_probability,
    noise_supports,
    cost_realizations,
    is_minimization
)

round.(risk_adjusted_probability, digits = 1)

# output

4-element Array{Float64,1}:
 0.2
 0.2
 0.6
 0.0

Convex combination of risk measures

Using the axioms of coherent risk measures, it is easy to show that any convex combination of coherent risk measures is also a coherent risk measure. Convex combinations of risk measures can be created directly:

julia> cvx_comb_measure = 0.5 * Kokako.Expectation() + 0.5 * Kokako.WorstCase()
A convex combination of 0.5 * Kokako.Expectation() + 0.5 * Kokako.WorstCase()

Kokako.adjust_probability(
    cvx_comb_measure,
    risk_adjusted_probability,
    nominal_probability,
    noise_supports,
    cost_realizations,
    is_minimization
)

risk_adjusted_probability

# output

4-element Array{Float64,1}:
 0.05
 0.1
 0.65
 0.2

As a special case, the Kokako.EAVaR risk-measure is a convex combination of Kokako.Expectation and Kokako.AVaR:

julia> risk_measure = Kokako.EAVaR(beta=0.25, lambda=0.4)
A convex combination of 0.4 * Kokako.Expectation() + 0.6 * Kokako.AVaR(0.25)

EAVaR(;lambda=1.0, beta=1.0)

    λ * E[x] + (1 - λ) * AV@R(1-β)[x]

Distributionally robust

SDDP.jl supports two types of distrbutionally robust risk measures: the modified Χ² method of Philpott et al. (2018), and a method based on the Wasserstein distance metric.

Modified Chi-squard

ModifiedChiSquared(radius::Float64)

Kokako.adjust_probability(
    Kokako.ModifiedChiSquared(0.5),
    risk_adjusted_probability,
    [0.25, 0.25, 0.25, 0.25],
    noise_supports,
    cost_realizations,
    is_minimization
)

round.(risk_adjusted_probability, digits = 4)

# output

4-element Array{Float64,1}:
 0.3333
 0.0447
 0.622
 0.0

Wasserstein

Wasserstein(norm::Function, solver_factory; alpha::Float64)

risk_measure = Kokako.Wasserstein(
        with_optimizer(GLPK.Optimizer); alpha=0.5) do x, y
   return abs(x - y)
end

Kokako.adjust_probability(
    risk_measure,
    risk_adjusted_probability,
    nominal_probability,
    noise_supports,
    cost_realizations,
    is_minimization
)

round.(risk_adjusted_probability, digits = 1)

# output

4-element Array{Float64,1}:
 0.1
 0.1
 0.8
 0.0

Training a risk-averse model

Now that we know what risk measures SDDP.jl supports, lets see how to train a policy using them. There are three possible ways.

If the same risk measure is used at every node in the policy graph, we can just pass an instance of one of the risk measures to the risk_measure keyword argument of the Kokako.train function.

Kokako.train(
    model,
    risk_measure = Kokako.WorstCase(),
    iteration_limit = 10
)

However, if you want different risk measures at different nodes, there are two options. First, you can pass risk_measure a dictionary of risk measures, with one entry for each node. The keys of the dictionary are the indices of the nodes.

Kokako.train(
    model,
    risk_measure = Dict(
        1 => Kokako.Expectation(),
        2 => Kokako.WorstCase()
    ),
    iteration_limit = 10
)

An alternative method is to pass risk_measure a function that takes one argument, the index of a node, and returns an instance of a risk measure:

Kokako.train(
    model,
    risk_measure = (node_index) -> begin
        if node_index == 1
            return Kokako.Expectation()
        else
            return Kokako.WorstCase()
        end
    end,
    iteration_limit = 10
)

This concludes our first intermediate tutorial. In the next tutorial, Intermediate II: stopping rules, we discuss different ways that the training can be terminated.