Cost Functions (Cost)

\[\]

This section contains cost functions classes which all derive from the abstract class Cost.

CostConst

class Cost.CostConst

Bases: Cost

CostConst: Constant cost whatever the input. Can be used to generate a null cost in particular situations.

All attributes of parent class Cost are inherited.

Parameters:

sizein – size of the input vector
const – the output constant value (default 0)

(default true)

Example C=CostConst(sz, const)

Example C=CostMod2(sz)

See also Map, Cost, LinOp

Method Summary

apply_(this, ~): Reimplemented from parent class Cost.

applyGrad_(this, ~): Reimplemented from parent class Cost.

applyProx_(~, x, ~): Reimplemented from parent class Cost.

CostGoodRoughness

class Cost.CostGoodRoughness

Bases: Cost

CostGoodRoughness: see [1] $$C(\mathrm{x}) := \sum_k \frac{\vert (\nabla \mathrm{x})_{.,k} \vert^2}{\sqrt{\vert \mathrm{x} \vert^2 + \beta}} $$ with $ \vert (\nabla \mathrm{x})_{.,k} \vert^2 $ the gradient magnitude at pixel k.

Parameters:

G – LinOpGrad object
bet – smoothing parameter (default 1e-1)

All attributes of parent class Cost are inherited.

Example GR=CostGoodRoughness(G)

Reference [1] Verveer, P. J., Gemkow, M. J., & Jovin, T. M. (1999). A comparison of image restoration approaches applied to three?dimensional confocal and wide?field fluorescence microscopy. Journal of microscopy, 193(1), 50-61.

CostHyperBolic

class Cost.CostHyperBolic

Bases: Cost

CostHyperBolic: Weighted Hyperbolic cost function $$C(\mathrm{x}) := \sum_{k=1}^K w_k \sqrt{\sum_{l=1}^L (\mathrm{x}-y)_{k,l}^2 + \varepsilon_k^2} - \sum_{k=1}^K\varepsilon_k $$

Parameters:

index – dimensions along which the l2-norm will be applied (inner sum over l)
epsilon – $\in \mathbb{R}^K_+$ smoothing parameter (default $10^{-3}$)
W – weighting LinOp object or scalar (default 1)

All attributes of parent class Cost are inherited.

Example C=CostHyperBolic(sz,epsilon,index,y,W)

See also Map, Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyGrad_(this, x): Reimplemented from parent class Cost.

CostKullLeib

class Cost.CostKullLeib

Bases: Cost

CostKullLeib: KullbackLeibler divergence $$ C(\mathrm{x}) :=\sum_n D_{KL}(\mathrm{x}_n)$$ where $$ D_{KL}(\mathrm{z}_n) := \left\lbrace \begin{array}[ll] \mathrm{z}_n - \mathrm{y}_n \log(\mathrm{z}_n + \beta) & \text{ if } \mathrm{z}_n + \beta >0 \newline + \infty & \text{otherwise}. \end{array} \right.$$

All attributes of parent class Cost are inherited.

Parameters:: bet – smoothing parameter $\beta$ (default 0)

Example C=CostKullLeib(sz,y,bet)

See also Map Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyGrad_(this, x): Reimplemented from parent class Cost.

applyProx_(this, x, alpha): Reimplemented from parent class Cost.

CostLinear

class Cost.CostLinear

Bases: Cost

CostLinear: Linear cost function $$C(\mathrm{x}) := \mathrm{x}^T\mathrm{y}$$

All attributes of parent class Cost are inherited.

Parameters:: y – data vector (default 0)

Example C=CostLinear(sz,y)

CostL1

class Cost.CostL1

Bases: Cost

CostL1: L1 norm cost function $$C(x) := \|\mathrm{x} - \mathrm{y}\|_1 $$

If nonneg is set to true, it adds positivity constraint on x: $$C(x) := \|\mathrm{x} - \mathrm{y}\|_1 + i(\mathrm{x} - \mathrm{y}) $$ where i() is the indicator function of $\mathbb{R}^{+}$

All attributes of parent class Cost are inherited.

Example C=CostL1(sz,y,nonneg)

See also Map Cost, LinOp

Method Summary

applyProx_(this, x, alpha): Reimplemented from parent class Cost. $$ \mathrm{prox}_{\alpha C}(\mathrm{x}) = \mathrm{sign(x-y)} \mathrm{max}(\vert x-y \vert- \alpha,0)+ \mathrm{y} $$

apply_(this, x): Reimplemented from parent class Cost.

applyGrad_(this, x): Reimplemented from parent class Cost. Subgradient of CostL1

CostL2

class Cost.CostL2

Bases: Cost

CostL2: Weighted L2 norm cost function $$C(\mathrm{x}) := \frac12\|\mathrm{x} - \mathrm{y}\|^2_W = \frac12 (\mathrm{x} - \mathrm{y})^T W (\mathrm{x} - \mathrm{y}) $$

All attributes of parent class Cost are inherited.

Parameters:

y – data vector (default 0)
W – weighting LinOp object or scalar (default 1)

Example C=CostL2(sz,y,W)

See also Map, Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyGrad_(this, x): Reimplemented from parent class Cost. $$ \nabla C(\mathrm{x}) = \mathrm{W (x - y)} $$ It is L-Lipschitz continuous with $ L \leq \|\mathrm{W}\|$.

applyProx_(this, x, alpha): Reimplemented from parent class Cost $$ \mathrm{prox}_{\alpha C}(\mathrm{x}) = \frac{\mathrm{x} + \alpha \mathrm{Wy}}{1 + \alpha \mathrm{W}} $$ where the division is component-wise.

makeComposition_(this, G): Reimplemented from parent class Cost. Instantiates a CostL2Composition.

CostL2Composition

class Cost.CostL2Composition

Bases: Abstract.OperationsOnMaps.CostComposition

CostL2Composition: Composition of a CostL2 with a Map $$C(\mathrm{x}) := \frac12\|\mathrm{Hx} - \mathrm{y}\|^2_W = \frac12 (\mathrm{Hx} - \mathrm{y})^T W (\mathrm{Hx} - \mathrm{y}) $$

Parameters:

H1 – CostL2 object
H2 – Map object

All attributes of parent class CostL2 and CostComposition are inherited.

Example C=CostL2Composition(H1,H2)

See also Map, Cost, CostL2, CostComposition, LinOp

Method Summary

apply_(this, x)

Reimplemented from parent class CostComposition. $$ C(\mathrm{x}) = \frac12\|\mathrm{Hx} - \mathrm{y}\|^2_W $$

If doPrecomputation is true, $\mathrm{W}$ is a scaled identity and $\mathrm{H}$ is a LinOp, then $\mathrm{H^* Wy} $ and $\| \mathrm{y} \|^2_{\mathrm{W}}$ are precomputed and $C(\mathrm{x}) $ is evaluated using the applyHtH() method, i.e. $$ C(\mathrm{x}) = \frac12 \langle \mathrm{W H^{\star}Hx,x} \rangle - \langle \mathrm{x}, \mathrm{H^* Wy} \rangle + \frac12\| \mathrm{y} \|^2_{\mathrm{W}}$$

applyGrad_(this, x)

Reimplemented from parent class CostComposition. $$ \nabla C(\mathrm{x}) = \mathrm{J_{H}^* W (Hx - y)} $$

If doPrecomputation is true, $\mathrm{W}$ is a scaled identity and $\mathrm{H}$ is a LinOp, then $\mathrm{H^* Wy} $ is precomputed and the gradient is evaluated using the applyHtH() method, i.e. $$ \nabla C(\mathrm{x}) = \mathrm{W H^{\star}Hx} - \mathrm{H^* Wy} $$

applyProx_(this, x, alpha)

Reimplemented from parent class CostComposition.

Implemented

if the operator $\alpha\mathrm{H^{\star}WH + I} $ is invertible: $$ \mathrm{y} = (\alpha\mathrm{H^{\star}WH + I} )^{-1} (\alpha \mathrm{H^TWy +x})$$
if $\alpha\mathrm{I + HH}^{\star}$ is invertible. In this case the prox is implemented using the Woodbury formulae [2]
if $\mathrm{H}$ is a LinOpComposition composing a LinOpDownsample with a LinOpConv. The implementation follows [1,2].
if $\mathrm{H}$ is a LinOpComposition composing a LinOpSum with a LinOpConv. The implementation follows [2]

Note If doPrecomputation is true, then $\mathrm{H^TWy}$ is stored.

References

[1] Zhao Ningning et al. “Fast Single Image Super-Resolution Using a New Analytical Solution for l2-l2 Problems”. IEEE Transactions on Image Processing, 25(8), 3683-3697 (2016).

[2] Emmanuel Soubies and Michael Unser. “Computational Super-Sectioning for Single-Slice Structured-Illumination Microscopy” (2018)

makeComposition_(this, G): Reimplemented from Cost. Instantiates a new CostL2Composition with the updated composed Map.

CostMixNorm21

class Cost.CostMixNorm21

Bases: Cost

CostMixNorm21: Mixed norm 2-1 cost function $$C(\mathrm{x}) := \sum_{k=1}^K \sqrt{\sum_{l=1}^L (\mathrm{x}-y)_{k,l}^2}= \sum_{k=1}^K \Vert (\mathrm{x-y})_{k\cdot} \Vert_2$$

Parameters:: index – dimensions along which the l2-norm will be applied (inner sum over l)

All attributes of parent class Cost are inherited.

Example C=CostMixNorm21(sz,index,y)

See also Map Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyProx_(this, x, alpha): Reimplemented from parent class Cost $$ \mathrm{prox}_{\alpha C}(\mathrm{x}_{k\cdot} ) = \left\lbrace \begin{array}{ll} \left( \mathrm{x}_{k\cdot} - y_{k\cdot} \right) \left(1-\frac{\alpha}{\Vert(\mathrm{x}-y)_{k\cdot}\Vert_2} \right) + y_{k\cdot} & \; \mathrm{if } \; \Vert (\mathrm{x-y})_{k\cdot}\Vert_2 > \alpha, \newline 0 & \; \mathrm{otherwise}, \end{array}\right. \; \forall \, k $$ where the division is component-wise.

makeComposition_(this, G): Reimplemented from parent class Cost. Instantiates a CostL2Composition.

CostMixNorm21NonNeg

class Cost.CostMixNorm21NonNeg

Bases: Cost

CostMixNorm21: Mixed norm 2-1 with non negativity constraints cost function $$C(\mathrm{x}) := \left\lbrace \begin{array}{ll} \sum_{k=1}^K \sqrt{\sum_{l=1}^L (\mathrm{x}-y)_{k,l}^2} & \text{ if } \mathrm{x-y} \geq 0 \newline + \infty & \text{ otherwise.} \end{array} \right. $$

Parameters:: index – dimensions along which the l2-norm will be applied (inner sum over l)

All attributes of parent class Cost are inherited.

Example C=CostMixNorm21(sz,index,y)

See also Map Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyProx_(this, x, alpha): Reimplemented from parent class Cost $$ \mathrm{prox}_{\alpha C}(\mathrm{x}_{k\cdot} ) = \left\lbrace \begin{array}{ll} \max(\mathrm{x}_{k\cdot} - y_{k\cdot} ,0) \left(1-\frac{\alpha}{\Vert(\mathrm{x}-y)_{k\cdot}\Vert_2} \right) +y_{k\cdot} cd .. & \; \mathrm{if } \; \Vert (\mathrm{x-y})_{k\cdot}\Vert_2 > \alpha, \newline 0 & \; \mathrm{otherwise}, \end{array}\right. \; \forall \, k $$ where the division is component-wise.

CostMixNormSchatt1

class Cost.CostMixNormSchatt1

Bases: Cost

CostMixNormSchatt1 Mixed Schatten-l1 Norm [1] $$C(\mathrm{x}) := \sum_n \| \mathrm{x}_{n\cdot} \|_{Sp}, $$ for $p \in [1,+\infty]$. Here, $\|\cdot\|_{Sp}$ denotes the p-order Shatten norm defined by $$ \|\mathrm{X}\|_{Sp} = \left[\sum_k (\sigma_k(\mathrm{X}))^p\right]^{1/p},$$ where $\sigma_k(\mathrm{X})$ is the k-th singular value of $\mathrm{X}$. In other words it is the lp-norm of the signular values of $\mathrm{X}$.

Parameters:: p – order of the Shatten norm (default 1)

All attributes of parent class Cost are inherited.

Note The actual implementation works for size (sz) having one of the two following forms:

(NxMx3) such that the Sp norm will be applied on each symetric 2x2 $$ \begin{bmatrix} \mathrm{x}_{n m 1} & \mathrm{x}_{n m 2} \newline \mathrm{x}_{n m 2} & \mathrm{x}_{n m 3} \end{bmatrix}$$ and then the $\ell_1$ norm on the two other dimensions.

(NxMxKx6) such that the Sp norm will be applied on each symetric 3x3 $$ \begin{bmatrix} \mathrm{x}_{n m k 1} & \mathrm{x}_{n m k 2} & \mathrm{x}_{n m k 3} \newline \mathrm{x}_{n m k 2} & \mathrm{x}_{n m k 4} & \mathrm{x}_{n m k 5} \newline \mathrm{x}_{n m k 3} & \mathrm{x}_{n m k 5} & \mathrm{x}_{n m k 6} \newline \end{bmatrix}$$ and then the $\ell_1$ norm on the three other dimensions.

References [1] Lefkimmiatis, S., Ward, J. P., & Unser, M. (2013). Hessian Schatten-norm regularization for linear inverse problems. IEEE transactions on image processing, 22(5), 1873-1888.

Example C=CostMixNormSchatt1(sz,p,y)

See also Map, Cost, LinOp

Method Summary

apply_(this, x): Reimplemented from parent class Cost.

applyProx_(this, x, alpha): Reimplemented from parent class Cost.

CostRobustPenalization

class Cost.CostRobustPenalization

Bases: Cost

CostRobustPenalization: Robust penalization cost function $$C(\mathrm{x}) := sum rho((A.x-y)/s)$$

All attributes of parent class Cost are inherited.

Parameters:

M – MAP to apply on the input (default Identity)
y – data vector (default 0)
options – structure containing the different options of the robust penalization
options.method – objective function method to compute the cost function - ‘Andrews’ - ‘Beaton-Tukey’ - ‘Cauchy’ (default) - ‘Fair’ - ‘Huber’ - ‘Logistic’ - ‘Talwar-Hinich’ - ‘Welsch-Dennis’
options.mu – parameters to tune the method. Default: - ‘Andrews’ -> 1.339 - ‘Beaton-Tukey’ -> 4.685 - ‘Cauchy’ -> 2.385 - ‘Fair’ -> 1.400 - ‘Huber’ -> 1.345 - ‘Logistic’ -> 1.205 - ‘Talwar-Hinich’ -> 2.795 - ‘Welsch-Dennis’ -> 2.985
options.flag_s – method to scale the residue before applying the objective function (default: none) - ‘none’ -> no scaling applied - ‘MAD’ -> median absolute deviation - ‘STD’ -> standard deviation - numeric -> scaling (can be a matrix to have a different scaling for each variables)
options.noise_model – model of the noise to scale the residues according to the input variable x - ‘Poisson’ -> scaling_factor = sqrt(x+var_0) - ‘none’ -> no scaling according to a noise model
options.var_0 – value of the variance of the data at null flux (default 0)
options.eta – ratio to scale the model in the corresponding unit for the Poisson noise
options.flag_memoizeRes – memoize the computation of the residues? (default: true)

Example C=CostRobustPenalization(M, y)

Example C=CostRobustPenalization(M, y, options)

Example C=CostRobustPenalization(M, y, [])

CostTV

class Cost.CostTV

Bases: Abstract.OperationsOnMaps.CostComposition

CostTV : Composition of a CostTV with a Map $$C(\mathrm{x}) := \sum_{k=1}^K \sqrt{\sum_{l=1}^L (\mathrm{H_2 x}-y)_{k,l}^2}= \sum_{k=1}^K \Vert (\mathrm{H_2 x-y})_{k\cdot} \Vert_2$$

Parameters:

H1 – CostMixNorm21 object
H2 – LinOpGrad object

All attributes of parent CostComposition are inherited.

Example C=CostTV(sz)

Example C=CostTV(H1,H2); with H1=CostMixNorm21(sz,index,y); and H2=LinOpGrad(sz);

See also Map, Cost, CostL2, CostComposition, LinOp

Method Summary

setProxAlgo(this, bounds, maxiter, xtol, Outop): Set the parameters of OptiFGP used to compute the proximity operator.

applyProx_(this, x, alpha): Reimplemented from parent class CostComposition. Computed using the iterative OptiFGP

IndicatorFunctions

CostIndicator

class Cost.IndicatorFunctions.CostIndicator

Bases: Cost

CostIndicator: Indicator cost function $$ C(\mathrm{x}) = \left\lbrace \begin{array}[l] \text{0~if } \mathrm{x -y} \in \mathrm{C}, \newline + \infty \text{ otherwise.} \end{array} \right. $$ where $\mathrm{C} \subset \mathrm{X} $ is a constraint set.

All attributes of parent class Cost are inherited

Note CostIndicator is an generic class for all indicator cost functions

CostRectangle

class Cost.IndicatorFunctions.CostRectangle

Bases: Cost.IndicatorFunctions.CostIndicator

CostRectangle: Rectangle Indicator function $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \mathrm{real(xmin)} \leq \mathrm{imag(x-y)} \leq \mathrm{real(xmax)} \text{ and } \mathrm{imag(xmin)} \leq \mathrm{imag(x-y)} \leq \mathrm{imag(xmax)}, \newline + \infty \text{ otherwise.} \end{array} \right. $$

Parameters:

xmin – minimum value (default -inf + 0i)
xmax – maximum value (default +inf + 0i)

All attributes of parent class CostIndicator are inherited

Example C=CostRectangle(sz,xmin,xmax,y)

CostReals

class Cost.IndicatorFunctions.CostReals

Bases: Cost.IndicatorFunctions.CostRectangle

CostReals: Reals Indicator function $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \mathrm{xmin} \leq \mathrm{x-y} \leq \mathrm{xmax} \newline + \infty \text{ otherwise.} \end{array} \right. $$

Parameters:

xmin – minimum value (default -inf)
xmax – maximum value (default +inf)

All attributes of parent class CostRectangle are inherited

Example C=CostReals(sz,xmin,xmax,y)

See also Map, Cost, CostIndicator, CostRectangle

CostNonNeg

class Cost.IndicatorFunctions.CostNonNeg

Bases: Cost.IndicatorFunctions.CostReals

CostNonNeg : Non negativity indicator $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \mathrm{x-y} \geq 0 \newline + \infty \text{ otherwise.} \end{array} \right. $$

All attributes of parent class CostRectangle are inherited

Example C=CostNonNeg(sz,y)

See also Map, Cost, CostIndicator, CostRectangle CostReals

CostComplexRing

class Cost.IndicatorFunctions.CostComplexRing

Bases: Cost.IndicatorFunctions.CostIndicator

CostComplexRing: Implements the indicator function operator on the complex ring $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \mathrm{inner} \leq \vert x-y\vert \leq \mathrm{outer} \newline + \infty \text{ otherwise.} \end{array} \right. $$

Parameters:

inner – inner radius of the ring (default 0)
outer – outer radius of the ring (default 1)

All attributes of parent class CostIndicator are inherited

Example CostComplexRing(inner, outer, sz,y)

CostComplexCircle

class Cost.IndicatorFunctions.CostComplexCircle

Bases: Cost.IndicatorFunctions.CostComplexRing

CostComplexCircle: Implements the indicator on the complex circle of radius c $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \vert \mathrm{x-y}\vert =c \newline + \infty \text{ otherwise.} \end{array} \right. $$

All attributes of parent class CostComplexRing are inherited

Parameters:: radius – radius of the circle (default 1)

Example C=CostComplexCircle(radius,sz,y)

See also Map, Cost, CostIndicator, CostComplexRing

CostComplexDisk

class Cost.IndicatorFunctions.CostComplexDisk

Bases: Cost.IndicatorFunctions.CostComplexRing

CostComplexDisk: Implements the indocator function on the complex disk of radius c $$ C(x) = \left\lbrace \begin{array}[l] \text{0~if } \vert \mathrm{x-y}\vert \leq c \newline + \infty \text{ otherwise.} \end{array} \right. $$

All attributes of parent class CostComplexRing are inherited

Parameters:: radius – radius of the disk (default 1)

Example C=CostComplexDisk(sz,radius, y)

See also Map, Cost, CostIndicator, CostComplexRing