Add a risk measure

Training a risk-averse model

SDDP.jl supports a variety of risk measures. Two common ones are SDDP.Expectation and SDDP.WorstCase. Let's see how to train a policy using them. There are three possible ways.

If the same risk measure is used at every node in the policy graph, we can just pass an instance of one of the risk measures to the risk_measure keyword argument of the SDDP.train function.

SDDP.train(
    model,
    risk_measure = SDDP.WorstCase(),
    iteration_limit = 10
)

However, if you want different risk measures at different nodes, there are two options. First, you can pass risk_measure a dictionary of risk measures, with one entry for each node. The keys of the dictionary are the indices of the nodes.

SDDP.train(
    model,
    risk_measure = Dict(
        1 => SDDP.Expectation(),
        2 => SDDP.WorstCase()
    ),
    iteration_limit = 10
)

An alternative method is to pass risk_measure a function that takes one argument, the index of a node, and returns an instance of a risk measure:

SDDP.train(
    model,
    risk_measure = (node_index) -> begin
        if node_index == 1
            return SDDP.Expectation()
        else
            return SDDP.WorstCase()
        end
    end,
    iteration_limit = 10
)

Note

If you simulate the policy, the simulated value is the risk-neutral value of the policy.

Risk measures

To illustrate the risk-measures included in SDDP.jl, we consider a discrete random variable with four outcomes.

The random variable is supported on the values 1, 2, 3, and 4:

julia> noise_supports = [1, 2, 3, 4]4-element Vector{Int64}:
 1
 2
 3
 4

The associated probability of each outcome is as follows:

julia> nominal_probability = [0.1, 0.2, 0.3, 0.4]4-element Vector{Float64}:
 0.1
 0.2
 0.3
 0.4

With each outcome ω, the agent observes a cost Z(ω):

julia> cost_realizations = [5.0, 4.0, 6.0, 2.0]4-element Vector{Float64}:
 5.0
 4.0
 6.0
 2.0

We assume that we are minimizing:

julia> is_minimization = truetrue

Finally, we create a vector that will be used to store the risk-adjusted probabilities:

julia> risk_adjusted_probability = zeros(4)4-element Vector{Float64}:
 0.0
 0.0
 0.0
 0.0

Expectation

SDDP.Expectation — Type

Expectation()

The Expectation risk measure. Identical to taking the expectation with respect to the nominal distribution.

source

julia> using SDDP
julia> SDDP.adjust_probability(
           SDDP.Expectation(),
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.1
 0.2
 0.3
 0.4

SDDP.Expectation is the default risk measure in SDDP.jl.

Worst-case

SDDP.WorstCase — Type

WorstCase()

The worst-case risk measure. Places all of the probability weight on the worst outcome.

source

julia> SDDP.adjust_probability(
           SDDP.WorstCase(),
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.0
 0.0
 1.0
 0.0

Average value at risk (AV@R)

SDDP.AVaR — Type

AVaR(β)

The average value at risk (AV@R) risk measure.

Computes the expectation of the β fraction of worst outcomes. β must be in [0, 1]. When β=1, this is equivalent to the Expectation risk measure. When β=0, this is equivalent to the WorstCase risk measure.

AV@R is also known as the conditional value at risk (CV@R) or expected shortfall.

source

julia> SDDP.adjust_probability(
           SDDP.AVaR(0.5),
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.2
 0.19999999999999996
 0.6
 0.0

Convex combination of risk measures

Using the axioms of coherent risk measures, it is easy to show that any convex combination of coherent risk measures is also a coherent risk measure. Convex combinations of risk measures can be created directly:

julia> cvx_comb_measure = 0.5 * SDDP.Expectation() + 0.5 * SDDP.WorstCase()A convex combination of 0.5 * SDDP.Expectation() + 0.5 * SDDP.WorstCase()
julia> SDDP.adjust_probability(
           cvx_comb_measure,
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.05
 0.1
 0.65
 0.2

As a special case, the SDDP.EAVaR risk-measure is a convex combination of SDDP.Expectation and SDDP.AVaR:

julia> SDDP.EAVaR(beta=0.25, lambda=0.4)A convex combination of 0.4 * SDDP.Expectation() + 0.6 * SDDP.AVaR(0.25)

SDDP.EAVaR — Function

EAVaR(;lambda=1.0, beta=1.0)

A risk measure that is a convex combination of Expectation and Average Value @ Risk (also called Conditional Value @ Risk).

    λ * E[x] + (1 - λ) * AV@R(β)[x]

Keyword Arguments

lambda: Convex weight on the expectation ((1-lambda) weight is put on the AV@R component. Inreasing values of lambda are less risk averse (more weight on expectation).
beta: The quantile at which to calculate the Average Value @ Risk. Increasing values of beta are less risk averse. If beta=0, then the AV@R component is the worst case risk measure.

source

Distributionally robust

SDDP.jl supports two types of distributionally robust risk measures: the modified Χ² method of Philpott et al. (2018), and a method based on the Wasserstein distance metric.

Modified Chi-squard

SDDP.ModifiedChiSquared — Type

ModifiedChiSquared(radius::Float64; minimum_std=1e-5)

The distributionally robust SDDP risk measure of Philpott, A., de Matos, V., Kapelevich, L. Distributionally robust SDDP. Computational Management Science (2018) 165:431-454.

Explanation

In a Distributionally Robust Optimization (DRO) approach, we modify the probabilities we associate with all future scenarios so that the resulting probability distribution is the "worst case" probability distribution, in some sense.

In each backward pass we will compute a worst case probability distribution vector p. We compute p so that:

p ∈ argmax p'z
      s.t. [r; p - a] in SecondOrderCone()
           sum(p) == 1
           p >= 0

where

z is a vector of future costs. We assume that our aim is to minimize future cost p'z. If we maximize reward, we would have p ∈ argmin{p'z}.
a is the uniform distribution
r is a user specified radius - the larger the radius, the more conservative the policy.

Notes

The largest radius that will work with S scenarios is sqrt((S-1)/S).

If the uncorrected standard deviation of the objecive realizations is less than minimum_std, then the risk-measure will default to Expectation().

This code was contributed by Lea Kapelevich.

source

julia> SDDP.adjust_probability(
           SDDP.ModifiedChiSquared(0.5),
           risk_adjusted_probability,
           [0.25, 0.25, 0.25, 0.25],
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.3333333333333333
 0.044658198738520394
 0.6220084679281462
 0.0

Wasserstein

SDDP.Wasserstein — Type

Wasserstein(norm::Function, solver_factory; alpha::Float64)

A distributionally-robust risk measure based on the Wasserstein distance.

As alpha increases, the measure becomes more risk-averse. When alpha=0, the measure is equivalent to the expectation operator. As alpha increases, the measure approaches the Worst-case risk measure.

source

julia> import HiGHS
julia> SDDP.adjust_probability(
           SDDP.Wasserstein(HiGHS.Optimizer; alpha=0.5) do x, y
               return abs(x - y)
           end,
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )0.0
julia> risk_adjusted_probability4-element Vector{Float64}:
  0.1
  0.10000000000000003
  0.7999999999999999
 -0.0

Entropic

SDDP.Entropic — Type

Entropic(γ::Float64)

The entropic risk measure as described by:

Dowson, O., Morton, D.P. & Pagnoncelli, B.K. Incorporating convex risk
measures into multistage stochastic programming algorithms. Annals of
Operations Research (2022). [doi](https://doi.org/10.1007/s10479-022-04977-w).

As γ increases, the measure becomes more risk-averse.

source

julia> SDDP.adjust_probability(
           SDDP.Entropic(0.1),
           risk_adjusted_probability,
           nominal_probability,
           noise_supports,
           cost_realizations,
           is_minimization
       )-0.14333892665462006
julia> risk_adjusted_probability4-element Vector{Float64}:
 0.1100296362588547
 0.19911786395979578
 0.3648046623591841
 0.3260478374221655