|
WebCab Optimization v2.6 (J2SE Edition) |
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Object | +--webcab.lib.math.optimization.multidimensional.MultiDimensionalSolver
The MultiDimensionalSolver class offers methods for finding the location of the extremum (i.e. minimum or maximum) of a function of many real variables (or equivalently, a function defined on a multi dimensional real space) with or without linear constraints.
Each of the procedures provided within this class proceeds by first being passed the object function of the optimization problem for which the location of the extremum is to be found. We then call one of the optimization procedures giving various parameters used to define procedure specific information such as the tolerance used and in the case of constrained optimization the constraints which must be satisfied. In most instances we are also required to provide an initial point from where the given optimization procedure will start its iterative search of the (local or global) extremum.
The aim of the theory of optimization is to find the location of the extremum (i.e. maximum or minimum) of a (possibly constrained) function defined over one or more (real) dimensions. An extremum point can be either local or global. A local extremum is a point such that the object function is known to obtain its highest (in the case of maximization) or lowest (in the case of minimization) value within a given sub-region of the entire domain over which the optimization problem is considered. A global extremum is the point at which the (object) function takes its maximum or minimum value over the entire domain for which the optimization problem is considered. Here we offer procedures which are designed for finding the local and global extremum.
You should select an optimization procedure to use in accordance with the properties of the optimization problem you are considering. In particular, you should take into account the following criteria when selecting the particular optimization procedure to use:
By using this information and taking into account your particular applications needs you will able to select the most appropriate procedure to use.
Depending on the nature of the object function at hand will determine which of the algorithms firstly can be used and secondly which algorithm(s) are most appropriate. Below we give the algorithms which you should use in the case of optimization problems with a general or differentiable (multidimensional) object function:
nelderMead(webcab.lib.math.optimization.ExtremumTypes, double[], double) - Uses the downhill simplex method of Nelder and
Mead to find a local minimum for the generic multidimensional function.
powell(webcab.lib.math.optimization.ExtremumTypes, double[], double[][], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Seeks a local minimum of a generic multidimensional
function using Powell's direction set method.
globalAnnealing(webcab.lib.math.optimization.ExtremumTypes, double[], int, double, double, double, double, int) - Finds the Global extremum of a general
multidimensional function.
derivSteepestDescent(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Applies the method of
Steepest Descent to a differentiable function.
derivFletcherPowell(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Applies the method of Fletcher-Powell to a differentiable
function.
derivBFGS(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Applies the Broyden-Fletcher-Goldfarb-Shanno algorithm
to a differentiable function.
derivPolakRiviere(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Applies the Polak-Riviere algorithm
to a differentiable function. The Polak-Riviere algorithm is generally much more efficient
than the Steepest Descent algorithm (i.e. derivSteepestDescent(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double)).
derivFletcherReeves(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double) - Applies the Fletcher-Reeves algorithm
to a differentiable function. The Fletcher-Reeves algorithm is generally much more efficient than
the Steepest Descent algorithm (i.e. derivSteepestDescent(webcab.lib.math.optimization.ExtremumTypes, double[], webcab.lib.math.optimization.unidimensional.BracketingAlgorithm, webcab.lib.math.optimization.unidimensional.LocateAlgorithm, double, double, double, int, double)).
linearConstraintRosen(webcab.lib.math.optimization.ExtremumTypes, double[], double[][], double[], int[], double[], double) - Rosen's gradient projection
algorithm which seeks a local minimum for the differentiable multidimensional real function
under a set of linear constraints. Note that, this algorithm should only be used if your
optimization problem is constrained.
Remark: If your object function is Linear then you should use the Simplex Algorithm
found in the LinearProgramming class.
At present we only offer algorithms which can treat problems with linear constraints. In instances where you have non-linear boundaries (i.e. boundaries which are not flat hyperplanes) you will need to linearize the boundaries in order to apply these algorithms. For optimization problems with linear constraints we offer:
linearConstraintRosen(webcab.lib.math.optimization.ExtremumTypes, double[], double[][], double[], int[], double[], double) for non-linear differentiable object
function - Rosen's gradient projection.
algorithm which seeks a local minimum for the differentiable multidimensional real function
under a set of linear constraints.
LinearProgramming.multiLinearSimplex(ExtremumTypes, double[], double[][], double[][]),
from the LinearProgramming class should be applied.
All the algorithms listed within this class will yield a local extremum for any compatible optimization problem. However, in practice it is often the global extremum which is sort and often a (local) extremum found will not turn out to be a global extremum for the given optimization problem at hand. In order to address this need the technique known as Simulated Annealing which can be applied `on top' of any of the optimization algorithms for finding the local extremum. This will greatly increase the chance of the global extremum. At present we offer the following Simulating Annealing enable (local) optimization procedures:
globalAnnealing(webcab.lib.math.optimization.ExtremumTypes, double[], int, double, double, double, double, int) - Nelder and Mead's downhill simplex
algorithm is used together with a simple annealing schedule.
Remark: If the object function is linear then the Simplex algorithm should
be used from the LinearProgramming class which will
always yield a global extremum.
An Optimization problem may also have constraints, such problems are referred to as
constrained optimization. In the case of Linear Programming (see
LinearProgramming) where the object function and constraints
are linear there exists efficient and refined algorithms. However, for general multi-dimensional
constrained optimization problems there are no known efficient general algorithms.
Therefore in practice specific algorithms are developed to address particular instances of
the general problem. We have implemented one such algorithm, known as Rosen's algorithm
(see linearConstraintRosen) which deals with the case when
the constraints are linear and the object function is differentiable.
As mentioned above the algorithms implemented within this class work most efficiently when the functions are `well behaved'. Generally speaking what we mean by `well behaved' is that the smoother the function more `well behaved' the function is. The more `well behaved the function the more efficient and stable the optimization algorithms contained within the class will be when applied to the given function at hand. This principle is true whether we are dealing within general or differentiable functions. The reason for this is that the `greedy type' algorithms implemented here (excluding simulated annealing) have as there basis that at each iterative step we move to the extremum point within a given neighborhood lying in one of a number of chosen directions. Such `greedy type' or calculus inspired arguments work more effectively as the function which it is applied to gets smoother.
Remark: In practice, the requirement that the majority of the procedures within this class require that the object function is continuous or even differentiable is not restrictive since in most practical applications. The reason being that processes generated from real world systems such as economic systems are in nearly all cases continuous and even differentiable.
It is also important to point out that though we may consider non-continuous functions
with some of the algorithms these functions must never return a NaN (Not a
number). In order to guarantee this we must ensure that for all points within the domain
over which the optimization problem is considered the object function does not return
such as value. If such an value is returned, the algorithms will exit and throw an
InvalidMultiDimensionalFunctionException
exception. If this exception is thrown then you will be able to call methods within the
Exception class which provide the point where the function evaluates to Double.NaN.
By modifying your given function at points where it returns a NaN you should be able to
modify the optimization problem so that the essential features of the underlying problem
are preserved but the optimization algorithm can be successfully applied.
As mentioned before the local extremum (i.e. minimum or maximum) is just the extremum in a neighborhood of a point, where as the global extremum is the extremum on the whole domain over which the optimization problem is considered. In practice it is generally the global extremum which is desired but this problem in general is very difficult. The difficulty lies in the fact that `greedy type' optimization methods used here are an abstraction of methods based on differential calculus and hence use the local differentiable information about function in order to move closer to the extremum. The problem with such local methods is that even if they find a (local) extremum they have no way of knowing whether that (local) extremum is in fact the global extremum.
In the case of uni-dimensional optimization there are effective algorithms for finding
the global extremum; for example UniDimensionalSolver.globalExtremeDeriv,
UniDimensionalSolver#globalExtreme. However, in the
multi-dimensional case we are required to use one of the three approaches:
globalAnnealing.
The first two approaches listed above are heuristic in nature and should be applied
with particular insight into the problem at hand. The third approach simulated annealing
is a general technique which can be applied to any of the standard local multi-dimensional
optimization algorithms in order to transform them from an algorithm design to find local
extremum to an algorithm which is much more likely to find a global extremum (see
globalAnnealing or the PDF documentation for more details).
The MultiDimensionalException is thrown if a function
is not set in the case of application of methods which require (general) functions, or
if the differentiable function is not set, that is you do not also supply the derivative.
Within this class we offer eight algorithms aimed at unconstrained optimization with object functions will differing qualitative properties and one algorithm for constrained optimization.
nelderMead - Uses the downhill simplex method of Nelder and Mead to
find a local minimum for the generic multidimensional function.
powell - Seeks a local minimum of a generic multidimensional function
using Powell's direction set method.
derivSteepestDescent - Requires differentiable function.
derivFletcherPowell - Requires differentiable function.
derivBFGS - Uses Broyden-Fletcher-Goldfarb-Shanno algorithm which
requires differentiable function.
derivPolakRiviere - Uses Polak-Riviere algorithm which is much
more efficient than Steepest Descent.
derivFletcherReeves - Uses Fletcher-Reeves algorithm which is
much more efficient than Steepest Descent.
globalAnnealing - This is the only global multidimensional algorithm
which can be applied to optimization problems where the object function is non-differentiable.
linearConstraintRosen - Rosen's gradient
projection algorithm which seeks a local minimum for the differentiable multidimensional real function
under linear a set of linear constraints.
| Constructor Summary | |
MultiDimensionalSolver()
Creates an MultiDimensional instance without registering a function. |
|
MultiDimensionalSolver(MultiDimensionalFunction instanceOfMultiDimensionalFunction)
Creates an MultiDimensional instance and assigns to it the specified function. |
|
| Method Summary | |
double[] |
derivBFGS(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Finds the location of a local extremum (i.e. minimum or maximum) for a differentiable multidimensional function using the Broyden-Fletcher-Goldfarb-Shanno algorithm. |
double[] |
derivFletcherPowell(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Finds a local extremum (i.e. minimum or maximum) for a differentiable multidimensional function set using using the Fletcher-Powell algorithm. |
double[] |
derivFletcherReeves(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Finds the location of a local extremum (i.e. minimum or maximum) of a differentiable multidimensional function using the Fletcher-Reeves algorithm. |
double[] |
derivPolakRiviere(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Find the location of a local extremum (i.e. minimum or maximum) of a differentiable multidimensional function using the Polak-Riviere algorithm. |
double[] |
derivSteepestDescent(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Finds the location of a local extremum (i.e. minimum or maximum) of a differentiable multidimensional function using the method of Steepest Descent. |
double[] |
globalAnnealing(ExtremumTypes extremumType,
double[] initialPoint,
int annealingSteps,
double initialTemperature,
double alpha,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance,
AnnealingAlgorithmTypes algorithmUsed)
Seeks a global extremum (minimum or maximum) of a differentiable multidimensional object function using simulated annealing applied in conjunction with one of the following algorithms for finding the local extremum of differentiable object functions: Steepest-Descent (i.e. |
double[] |
globalAnnealing(ExtremumTypes extremumType,
double[] initialPoint,
int annealingSteps,
double initialTemperature,
double alpha,
double radius,
double fractionalTolerance,
int stepIterations)
Finds the location of the global extremum (i.e. minimum or maximum) of a general multidimensional function using the technique known as simulated annealing applied to a modified version of the Needler and Mead's downhill simplex algorithm. |
double[] |
linearConstraintRosen(ExtremumTypes extremumType,
double[] initialPoint,
double[][] coeffGeneral,
double[] constGeneral,
int[] simpleIndex,
double[] lowerBounds,
double projectionTolerance)
Implementation of Rosen's gradient projection algorithm which seeks a (local) extremum (i.e. |
double[] |
nelderMead(ExtremumTypes extremumType,
double[] initialPoint,
double tolerance)
Uses the downhill simplex method of Nelder and Mead to find a local extremum (i.e. minimum or maximum) of a generic multidimensional function. |
double[] |
powell(ExtremumTypes extremumType,
double[] initialPoint,
double[][] initialDirections,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
Seeks a local extremum of a general multi-dimensional function using Powell's direction set method. |
void |
setFunction(MultiDimensionalFunction instanceOfMultiDimensionalFunction)
Submits a new multi-dimensional function for which the location of the extremum is to be found. |
| Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
public MultiDimensionalSolver()
public MultiDimensionalSolver(MultiDimensionalFunction instanceOfMultiDimensionalFunction)
| Method Detail |
public void setFunction(MultiDimensionalFunction instanceOfMultiDimensionalFunction)
In order to provide a function which is to be used within the algorithms contained within this class you are required to implement one of the following interfaces:
MultiDimensionalFunction - For multi-dimensional
functions for which the gradient either does not exist or for which the gradient is not known.
Derivative - For multi-dimensional functions for which the gradient
exists and the gradient is known. Note that the Derivative interface inherits
from the MultiDimensionalFunction interface.
For further explicit details concerning how these interfaces are implemented (including source code)
please see the documentation accompanying the interfaces MultiDimensionalFunction
and Gradient.
public double[] nelderMead(ExtremumTypes extremumType,
double[] initialPoint,
double tolerance)
throws TooManyMultiDimensionalIterationsException,
MultiDimensionalException,
InvalidMultiDimensionalFunctionException
When addressing general multi-dimensional problems you have the choice of selecting this
method or Powell's direction set method (see Powell's method). If your aim is to
`get something working quickly' then you should use this simplex method. The simplex method
make no requirements on the function in fact it is possible to consider even non-continuous
functions. However, partly because of its generality it is not particularly efficient in terms
of the number of function evaluations required in order to achieve a given level of accuracy.
It is also important to point out that though this method can be very slow and is in a sense
simple, it can also be very robust.
Before you call this method you are required to set the object function of the optimization
problem using setFunction. Please see the documentation of
setFunction or the programmers guide within the PDF documentation for
more details on exactly how to set the function. Once the object function has been set all
that means to be done to to select an initial points from which the iterative search for the
extremum will begin and the tolerance which will determine the exit condition for the search
algorithm.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.tolerance - An internal algorithm parameter known as a fractional convergence tolerance which is used within the termination condition. The smaller the tolerance given the more iteratives of algorithm will be performed, which can lead to the exception TooManyMultiDimensionalIterationsException being thrown. If this happens you should increase the value of the tolerance and re-run the algorithm. A reasonable value to take from this parameters is 0.01.
N-dimensional vector) where the object function has a local
extremum.
TooManyMultiDimensionalIterationsException - thrown if the user function returns invalid values
like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement ,
or an interface which extends MultiDimensionalFunction.
InvalidMultiDimensionalFunctionException
public double[] powell(ExtremumTypes extremumType,
double[] initialPoint,
double[][] initialDirections,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
When addressing general multi-dimensional problems you have the choice of selecting this
method or the Simplex method of Nelder and Mead (see Nelder and Mead).
The advantage of this approach is that it has the ability to choose good initial directions
in which to search for an extremum. This ability to select good directions means that problems
such as getting stuck in long valleys of the object function are mitigated. It also means that
this algorithm is almost surely faster than the Simplex method used in Nelder and Mead.
However, it should be pointed out that this approach in general is more difficult and time
consuming to set up and get running.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.initialDirections - a matrix whose columns contain the initial set of directions (usually the N unit vectors).bracketingAlgorithm - the bracketing algorithm used within this method. For generic functions for which the derivative is not known you will need to use one of the two bracketing algorithms ParabolicBracketing or AccelBracketing. When deciding which of these two algorithms is more appropriate please see the remarks given above or the PDF documentation.locateAlgorithm - the locate algorithm used within this method. For generic functions for which the derivative is not known you will need to use one of the following locate algorithms: LinearLocate, BrentLocate, ParabolicIterativeLocate, and ParabolicLocate. Please see the remarks given above or the PDF documentation for assistance in the selection of the most appropriate algorithm for your given problem.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than
maxIterations is necessary in order to obtain the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values
like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement MultiDimensionalFunction,
or an interface which extends MultiDimensionalFunction.
public double[] derivSteepestDescent(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
This algorithm proceeds by taking a series of steps along the steepest downhill
direction which we are able to evaluate because we are able to evaluate the functions
gradient. After each steepest downhill step we imply a unidimensional minimization
algorithm, which is determined by the bracketing algorithm (i.e. BracketingAlgorithm)
and Locate Algorithm (i.e. LocateAlgorithm) chosen.
Remark: Since the steepest downhill steps are in a sense ideal, the locate algorithm used does not need to be particularly precise. Moreover, the precision of the Locate algorithm will only influence the speed at which the extremum is found and not the accuracy of the final result.
It is recommended that you use AccelDerivBracketing and CubicDerivLocate.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.bracketingAlgorithm - the bracketing algorithm used within this method. We recommend in most cases that the AccelDerivBracketing bracketing algorithm is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Bracketing Algorithm.locateAlgorithm - the locate algorithm used within this method. We recommend in most cases that the CubicDerivLocate locate algorithms is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Locate Algorithm.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than maxIterations
is necessary in order to obtain the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values
like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement
Gradient or an interface which extends Gradient.
public double[] derivFletcherPowell(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
This is a variable metric method which is generally much more efficient than
Steepest Descent (see derivSteepestDescent) or
conjugate gradient method. This algorithm is also said to be quadratically convergent
which means that if the object function is a quadratic form then in exactly
N steps, where N is the dimension of the problem;
the extrema is found.
This algorithm proceeds to find an extremum by applying at each step a
unidimensional minimization algorithm, which is determined by the selection of
a Bracketing (see BracketingAlgorithm) and Locate
(see LocateAlgorithm) Algorithms. The accuracy of the
Locate algorithm chosen will not effect the precision of the final results
but only the speed with which it is found.
It is recommended that you use AccelDerivBracketing and CubicDerivLocate.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.bracketingAlgorithm - the bracketing algorithm used within this method. We recommend in most cases that the AccelDerivBracketing bracketing algorithm is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Bracketing Algorithm.locateAlgorithm - the locate algorithm used within this method. We recommend in most cases that the CubicDerivLocate locate algorithms is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Locate Algorithm.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than
maxIterations is necessary in order to obtain the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values
like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement
Gradient or an interface which extends Gradient.
public double[] derivBFGS(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
This is a variable metric method which is generally much more efficient than
Steepest Descent (see derivSteepestDescent) or
conjugate gradient method. This algorithm is also said to be quadratically convergent
which means that if the object function is a quadratic form then in exactly
N steps, where N is the dimension of the problem;
the extrema is found.
This algorithm proceeds to find an extremum by applying at each step a
unidimensional minimization algorithm, which is determined by the selection of
a Bracketing (see BracketingAlgorithm) and Locate
(see LocateAlgorithm) Algorithms. The accuracy of the
Locate algorithm chosen will not effect the precision of the final results
but only the speed with which it is found.
It is recommended that you use AccelDerivBracketing and CubicDerivLocate.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.bracketingAlgorithm - the bracketing algorithm used within this method. We recommend in most cases that the AccelDerivBracketing bracketing algorithm is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Bracketing Algorithm.locateAlgorithm - the locate algorithm used within this method. We recommend in most cases that the CubicDerivLocate locate algorithms is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Locate Algorithm.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than maxIterations
is necessary in order to obtain the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values like infinity
or NaN.
MultiDimensionalException - thrown if the delivered function does not implement Gradient or an interface
which extends Gradient.
public double[] derivPolakRiviere(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
This is a variable metric method which is generally much more efficient than
the Steepest Descent (see derivSteepestDescent) method.
This algorithm proceeds to find an extremum by applying at each step a
unidimensional minimization algorithm, which is determined by the selection of
a Bracketing (see BracketingAlgorithm) and Locate
(see LocateAlgorithm) Algorithms. The accuracy of the
Locate algorithm chosen will not effect the precision of the final results
but only the speed with which it is found.
It is recommended that you use AccelDerivBracketing and CubicDerivLocate.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.bracketingAlgorithm - the bracketing algorithm used within this method. We recommend in most cases that the AccelDerivBracketing bracketing algorithm is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Bracketing Algorithm.locateAlgorithm - the locate algorithm used within this method. We recommend in most cases that the CubicDerivLocate locate algorithms is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Locate Algorithm.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than maxIterations is necessary in order to obtain
the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement Gradient or an interface
which extends Gradient.
public double[] derivFletcherReeves(ExtremumTypes extremumType,
double[] initialPoint,
BracketingAlgorithm bracketingAlgorithm,
LocateAlgorithm locateAlgorithm,
double minDistance,
double bracketingParameter,
double tolerance,
int maxIterations,
double fractionalTolerance)
throws TooManyMultiDimensionalIterationsException,
InvalidMultiDimensionalFunctionException,
MultiDimensionalException
This is a conjugate gradient method which is generally much more efficient than
the Steepest Descent (see derivSteepestDescent) method.
This algorithm proceeds to find an extremum by applying at each step a
unidimensional minimization algorithm, which is determined by the selection of
a Bracketing (see BracketingAlgorithm) and Locate
(see LocateAlgorithm) Algorithms. The accuracy of the
Locate algorithm chosen will not effect the precision of the final results
but only the speed with which it is found.
It is recommended that you use AccelDerivBracketing and CubicDerivLocate.
After this algorithm has be run it will exit when one of the following two conditions has been satisfied:
maxIterations is exceeded,
then TooManyMultiDimensionalIterationsException is thrown resulting in the algorithm
exiting.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value.bracketingAlgorithm - the bracketing algorithm used within this method. We recommend in most cases that the AccelDerivBracketing bracketing algorithm is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Bracketing Algorithm.locateAlgorithm - the locate algorithm used within this method. We recommend in most cases that the CubicDerivLocate locate algorithms is used. See the remarks given above, the PDF documentation or the examples for more information on selecting the most appropriate Locate Algorithm.minDistance - this parameter is required by all bracketing algorithms, and usually signifies the minimal distance between the end points of any bracketed interval in which the extremum lies. Therefore, the minDistance parameter should be approximately the same magnitude as tolerance parameter.bracketingParameter - this parameter is required by some bracketing algorithms. For a detailed explanation as to its meaning and use for particular bracketing algorithms selected please see the documentation of the BracketingAlgorithm selected or the Mathematical Documentation chapter of the accompanying PDF documentation.tolerance - the tolerance used when the exit condition of this iterative algorithm is checked. The smaller the tolerance used the more stable the convergence to the extremum within the iterative algorithm must be before the exit condition is trigger. Hence the smaller the tolerance generally the more accuracy the result, however it also has the draw back of making the algorithm more computationally demanding. A reasonable value to use for the tolerance is 0.1.maxIterations - the maximum number of iterative steps taken before the algorithm exits and a TooManyMultiDimensionalIterationsException is thrown. A reasonable number to use for this parameter is 300.fractionalTolerance - when the difference between the function values in two consecutive steps is less than fractionalTolerance, the point in the last step is considered a local extreme and returned. In most cases a reasonable value to take for this parameter is 0.0001, however if the object function in very oscillatory in a neighborhood of its extremum then a correspondingly larger value should be used.
N-dimensional vector) where the function has a local extremum.
TooManyMultiDimensionalIterationsException - thrown if a number of steps greater than maxIterations
is necessary in order to obtain the desired tolerance.
InvalidMultiDimensionalFunctionException - thrown if the user function returns invalid values like infinity or NaN.
MultiDimensionalException - thrown if the delivered function does not implement Gradient or an interface
which extends Gradient.
public double[] linearConstraintRosen(ExtremumTypes extremumType,
double[] initialPoint,
double[][] coeffGeneral,
double[] constGeneral,
int[] simpleIndex,
double[] lowerBounds,
double projectionTolerance)
throws MultiDimensionalException
Rosen algorithms basic approach is to iteratively shift in the direction of the extremum (i.e.. maximum or minimum), where the constraints (i.e. boundaries) are incorporated into the algorithm by allowing the shift to `bounce off' the boundary if a shift moves the point against the boundary. This basic nature of the algorithm raises issues with regard to how it should be applied. In particular, since the point is iteratively shifted within a region to ensure that the algorithm returns a point which satisfies the constraints the region in which it is defined by the constraints should be closed. Moreover, to allow the initial point to `bounce off' the boundary the initial point should lie strictly within the constraints and not on the boundary.
There is no simple sufficiency test by which given a set of arbitrary constraints you are able to say whether or not the resulting region is closed. However, a clear necessary condition is that each variable has an absolute upper and lower bound implied from the constraints. Ideally the initial point selected (strictly within the region) should be implied from domain knowledge of problem at hand. However, if no such knowledge exists then a suitable initial point can be found by applying the following abstract mathematical fact. The center of gravity of a (closed) simplex is an interior point of that simplex. Therefore, we can find the vector corresponding to an interior point by adding the vectors of the vertices's of the simplex defined by the linear constraints forming the closed regions and then dividing that vector by the number of vertices.
Though the above described two conditions are advisable, strictly speaking they are not necessary in order to allow the algorithm to run and return the correct result. However if they are not followed then you may encounter the following errors. If the region is not closed then the iterative sequence of points may either diverge to infinity or even move around onto the other side of the constraint(s). If the initial point given lies on a boundary then the first shift may shift the initial point onto the wrong side of the constraint(s) leading to errors.
The role played by having a formally closed (or almost closed) region is to ensure that the sequence of iterative points stays within the neighborhood of interest. Unless you have detailed knowledge of the nature of the object function in which case it may be possible to safely use a non-closed region we suggest that you construct a collection of constraints such that the domain over which the optimization problem is defined is formally closed in order to ensure that the iterative sequence of points does not escape the region of interest.
Ideally the problem you are considering will naturally suggest a set of constraints which define
a closed region. However if this is not the case then you will need to extend your set of constraints
in order to define a closed region. One approach to obtain a closed region is to place simple constraints
on the variables which do not have upper and lower bounds implied from the initial set of constraints.
Note that if all the variables are bounded above and below then the region is closed. Therefore, in order
to provide a closed region we only need to provide bounds for those variables which are not bounded by
the original set of constraints. In order to bound a variable say x_k, we select an appropriate
upper and lower bound, after which we add the following constraints to the existing set of constraints:
x_k >= lowerBound ===> -x_k <= - lowerBound
x_k <= upperBound
When applying Rosen's algorithm you should supply a point strictly within the interior of the domain defined by the constraints. That is, the initial point should satisfy all the constraints but not lie on the boundary defined by any one of them. If a global extremum of the function of the domain is required then care should be taken in the selection of the initial point since Rosen's algorithm being of greedy (downhill) type is only certain to converge to a local extremum. However, by using a range of initial points and re-running the algorithm will increase your likely-hood of finding the global extremum.
For complex sets of constraints it may not be any obvious initial points from which to select. In such cases we suggest that you use an initial point which lies on one of the constraints (which clear lies in the domain in the boundary of the constraints lie in the domain). If you which to re-run the algorithm then we suggest that you use a initial point on another constraints.
Assuming that the region defined by the constraints is closed it is only necessary that the object function f(x1, x2, ... , xN), and its gradient is given over the closed region, outside of this region the function may not even exist.
Providing the function
There are two procedures in which you are able to provided the function to Rosen's algorithm, namely:
Function1 function1 = new Function1()optimization = createOptimizationInstance(function1)setFunction method in order to set the function which you will need
to implement within a class. See the notes for the method setFunction for more
details.
Which ever one of these two methods are used you will be required to implement the
real function with a class with a method which returns the value of the function at
a point and in the case of differential functions also contains a method which returns the
value of the derivative at a point. We provide more details with the documentation for the
setFunction method.
Remark: Setting the functions during the instantiation of the class is the
safer of the two approaches. Since two clients (threads) could use the same instance of the
optimization class where the function has not been set and where each client wishes
to analyze a different function. If one of the clients is analyzing a given function then
other client may inadvertently change the function to the function he wishes to analyze and
so causes the first clients analysis to fail. That is, the construct approach is thread safe
whereas the setFunction approach is not.
The constraints which can be used within this algorithm must be linear. Since we
are working within multidimensional space in geometric language we would say that the
optimization problem of constrained to lie within a domain bound by N-dimensional
hyperplanes. What this means in practice is that the solution of the optimization problem
must satisfy a number of inequality constraints of the following form:
Such constraints which involve many terms involving many xi's, we will refer to as general constraints to indicate their full generality. Below we describe how these full constraints which can be used to represent all possible constraints can be provided to the method by the use of arrays. Since in many practical applications you will only require to put an upper bound on the coordinates, that is constraints of the following form:
Such constraints are referred to as simple constraints and we provide another mechanism (explained below) which allows such constraints to be set in order to simplify the application and in fact increase to efficiency of this algorithm.
Remark: If your optimization problem contains a mixture of general and simple constraints then you may use both mechanism within the same problem. However, we advise that you use the simple only mechanism or simple constraints rather than the general mechanism since it is simpler and more importantly the optimization algorithm will be more efficient.
For a given optimization problem you will need to provide the general constraints as parameters within the method call. Here we will need to convert the constraints into an array of dimension 2, A, containing the coefficients of the coordinates and an array B, containing the constant terms within each of the constraints. This is done as follows, say of the ith constraint of the given problem is as follows:
where c1, c2, ..., cN + 1 are some real numbers and x1, ..., xN are the coordinates of the space over which the problem is considered. This particular inequality is encoded as an array {c1, ..., cN} and a double value for cN + 1, where the array is the ith element of the two-dimensional array A, and the double is the ith term of the array of doubles B. By completing this process for each of the constraints within the given problem you are able to populate the arrays A and B.
If your given problem contains a constraint of the following for:
That is, ... ≤ cN + 1, rather than ... ≥ cN + 1, then you are may reverse the order of the inequality by multiplying everything through by -1, to give:
which you can then encoded and add to the arrays A and B, following the procedure as described above. Similar remarks apply to reversing the order of the inequality for simple constraints.
Though you are able to provide all possible linear constraints by using the procedure outlined above using the array of dimension 2, A, and the array B. We provide a simplified procedure which allows just the lower bounds on the coordinates to be given. By providing the lower bounds on the coordinates by this means you actually allow the algorithm to perform the algorithm more efficiently in terms of execution time since the algorithm with use specialized procedures to deal with these particular constraint type.
The lower bounds on the solution are given by the use of two array parameters: simpleIndex,
lowerBounds; containing integers and doubles respectively. Each of these arrays
have the same length equal to the dimensional of the space over which the optimization problem
is considered. Therefore, we are able to provide a lower bound for each of the coordinates.
The lower bounds for each coordinate is provide by a term of simpleIndex, and
lowerBounds in the following way. The k-th term of the array simpleIndex
is the index of coordinate for which the k-th term of lowerBounds is a lower
bound. For example, if simpleIndex[3] = 0, and lowerBounds[2] = 2.5,
then in the notion used above, x[0] >= 2.5.
extremumType - indicates the type of extreme searched: in the case of a minimum, extremumType == ExtremumTypes.MINIMUM; and in the case of a maximum, extremumType == ExtremumTypes.MAXIMUM.initialPoint - The initial point about which the iterative optimization procedure starts. The initial point is given as an array where the n-th term of the array corresponds to the value of the n coordinate value (i.e. an N-dimensional vector). This point must be chosen so as to satisfy all the inequalities (that is, to be "inside" the constraints). There is no other way to specify the sense of the inequalities. The constraints are boundaries which cannot be crossed. The only way you can say which side of the hyperplane is "good" and which is "bad" is by correctly choosing the initial pointcoeffGeneral - an array of dimension 2 of length equal to the number of general constraints used where the kth term of the array, which itself is an array of length equal to the number of coordinates over which the problem is defined, corresponding to the coefficients of the kth general constraint. That is, the values of the kth term, constGeneral, which represents the constant terms of the general constraints.constGeneral - an array of length equal to the number of general constraints used where the kth term corresponds to the value of the constant term bk within the kth general constraint: simpleIndex - an array of integers where the kth term of this array signifies the index (starting from 0) of the coordinate for which the corresponding kth term of the array lowerBounds is a lower bound. For example, if the first term of the array is 5 then it indicates that the first term of array lowerBounds, is the lower bound of the 6th coordinate value (the indexing starts from zero).lowerBounds - an array of doubles where the kth term is the lower bound of the coordinate signified by the kth terms of the simpleIndex parameters. For example, if the kth term of the array lowerBounds is 10, and the kth term of simpleIndex is 3, then the value of the 4th coordinate of the solution on the domain over which the problem expressed must be greater than or equal to 10.projectionTolerance - if the gradient projection is less than this tolerance (in absolute terms), it is considered to be zero and hence the Kuhn-Tucker conditions are satisfied which implies that the extremum has been found. That is, by setting the projectionTolerance we determine the tolerance to which the Kuhn-Tucker conditions are measured. Once we have satisfied these conditions to lie within this tolerance the points found will be returned. Therefore, in principle the smaller the projectionTolerance is set the more accuracy the result but also the longer the algorithm will take. A reasonable value for the projectionTolerance is 1e-5 (a double value equal to 0.00001, or 10-5).
MultiDimensionalException - thrown if the delivered function does not implement Gradient or
an interface which extends Gradient.
public double[] globalAnnealing(ExtremumTypes extremumType,
double[] initialPoint,
int annealingSteps,
double initialTemperature,
double alpha,
double radius,
double fractionalTolerance,
int stepIterations)
throws MultiDimensionalException
The general idea used here is that after each step within the selected algorithm for
finding the local extremum, we perturb the point by a random jump corresponding to "temperature of
the surface". The temperature of the surface starts from an initial value initialTemperature
and is decreases by a constant ratio after each iteration (that is, the surface is allowed to cool).
The rationale for this approach is that by "heating" the surface we vibrate it so that we are
knocked out by the shaking of local extr