Gradient Descent

Gradient Descent

steepestDescent(M, F, ∇F, x)

perform a steepestDescent $x_{k+1} = \mathrm{retr}_{x_k} s_k\nabla f(x_k)$ with different choices of $s_k$ available (see stepsize option below).

Input

  • M – a manifold $\mathcal M$
  • F – a cost function $F\colon\mathcal M\to\mathbb R$ to minimize
  • ∇F: the gradient $\nabla F\colon\mathcal M\to T\mathcal M$ of F
  • x – an initial value $x\in\mathcal M$

Optional

  • stepsize – (ConstantStepsize(1.)) specify a Stepsize functor.
  • retraction – (exp) a retraction(M,x,ξ) to use.
  • stoppingCriterion – ([stopWhenAny](@ref)([stopAfterIteration](@ref)(200), [stopWhenGradientNormLess](@ref)(10.0^-8))) a functor inheriting from [StoppingCriterion`](@ref) indicating when to stop.

and the ones that are passed to decorateOptions for decorators.

Output

  • xOpt – the resulting (approximately critical) point of gradientDescent
  • record - if activated (using the record key, see RecordOptions an array containing the recorded values.
source

Options

GradientDescentOptions{P,T} <: Options where {P <: MPoint, T <: TVector}

Describes a Gradient based descent algorithm, with

Fields

a default value is given in brackets if a parameter can be left out in initialization.

  • x0 : an MPoint as starting point
  • stoppingCriterion : a function s,r = @(o,iter,ξ,x,xnew) returning a stop indicator and a reason based on an iteration number, the gradient and the last and current iterates
  • retraction : (exp) the rectraction to use
  • stepsize : a Function to compute the next step size)

Constructor

GradientDescentOptions(x, stop, s [, retr=exp])

construct a Gradient Descent Option with the fields and defaults as above

See also

steepestDescent, GradientProblem

source

Debug Actions

DebugGradient <: DebugAction

debug for the gradient evaluated at the current iterate

Constructors

DebugGradient([long=false,p=print])

display the short (false) or long (true) default text for the gradient.

DebugGradient(prefix[, p=print])

display the a prefix in front of the gradient.

source
DebugGradientNorm <: DebugAction

debug for gradient evaluated at the current iterate.

Constructors

DebugGradientNorm([long=false,p=print])

display the short (false) or long (true) default text for the gradient norm.

DebugGradientNorm(prefix[, p=print])

display the a prefix in front of the gradientnorm.

source
DebugStepsize <: DebugAction

debug for the current step size.

Constructors

DebugStepsize([long=false,p=print])

display the short (false) or long (true) default text for the step size.

DebugStepsize(prefix[, p=print])

display the a prefix in front of the step size.

source

Record Actions

RecordGradient <: RecordAction

record the gradient evaluated at the current iterate

Constructors

RecordGradient(ξ)

initialize the RecordAction to the corresponding type of the TVector.

source
RecordGradientNorm <: RecordAction

record the norm of the current gradient

source
RecordStepsize <: RecordAction

record the step size

source