Java Program to Implement Steepest Descent Method and Compute Local Optima
Last Updated :
30 Dec, 2022
The steepest descent method is an optimization algorithm that can be used to find the local minimum of a differentiable function. It is an iterative method that starts with an initial guess for the solution, and then repeatedly takes steps in the direction of the steepest descent (the direction of the greatest negative slope) until it reaches a local minimum. To implement the steepest descent method in Java, you would first need to define a differentiable function that you want to minimize. This function should take as input some set of parameters and return a scalar value that represents the cost or error of the current parameter values.
Next, you would need to define an initial guess for the solution, which would typically be a set of starting values for the parameters of the function. This initial guess would be used as the starting point for the optimization algorithm. Then, you would need to implement the main loop of the steepest descent algorithm. This loop would iterate until some stopping criterion is met, such as reaching a pre-defined number of iterations or a certain level of precision in the solution. At each iteration, the algorithm would compute the gradient of the function at the current parameter values, and then take a step in the direction of the negative gradient. This would typically involve updating the parameter values by subtracting a fraction of the gradient vector from the current values.
After the main loop of the algorithm has finished, the final parameter values would represent the local minimum of the function, according to the steepest descent method. You could then use these values to make predictions or evaluate the performance of the model. It is important to note that the steepest descent method only guarantees to find a local minimum, not a global minimum. This means that the solution obtained using this algorithm may not be the optimal solution for the problem, especially if the function has multiple local minima. To improve the chances of finding the global minimum, you could try using different initial guesses for the solution or modifying the step size of the algorithm.
Here are examples of a Java program that implements the steepest descent method to compute the local optima of a function.
Example 1:
Java
import java.util.function.Function;
public class SteepestDescent {
public static double minimize(Function< double [], Double> f, double [] x0, double alpha, double eps) {
int n = x0.length;
double [] x = x0.clone();
double [] grad = new double [n];
while ( true ) {
double fx = f.apply(x);
for ( int i = 0 ; i < n; i++) {
double old = x[i];
x[i] = old + eps;
grad[i] = (f.apply(x) - fx) / eps;
x[i] = old;
}
for ( int i = 0 ; i < n; i++) {
x[i] = x[i] - alpha * grad[i];
}
boolean converged = true ;
for ( int i = 0 ; i < n; i++) {
if (Math.abs(grad[i]) > eps) {
converged = false ;
break ;
}
}
if (converged) {
break ;
}
}
return f.apply(x);
}
public static void main(String[] args) {
double [] x0 = { 1 , 2 , 3 };
double alpha = 0.1 ;
double eps = 0.001 ;
Function< double [], Double> f = x -> x[ 0 ] * x[ 0 ] + x[ 1 ] * x[ 1 ] + x[ 2 ] * x[ 2 ];
double localOptima = minimize(f, x0, alpha, eps);
System.out.println( "Local Optima: " + localOptima);
}
}
|
Output
Local Optima: 1.9972843147380194E-7
Example 2:
Below program uses the steepest descent method to minimize the function f(x, y) = x2 + 2y2 – 2xy + 3x + 5y.
Java
import java.util.Arrays;
public class SteepestDescent {
public static double f( double x, double y) {
return x*x + 2 *y*y - 2 *x*y + 3 *x + 5 *y;
}
public static double [] grad( double x, double y) {
return new double [] { 2 *x - 2 *y + 3 , 4 *y - 2 *x + 5 };
}
public static double alpha = 0.1 ;
public static int maxIter = 100 ;
public static double epsilon = 1e- 6 ;
public static void main(String[] args) {
double [] x0 = { 1 , 1 };
double [] x = x0;
double [] xPrev = new double [x0.length];
int k = 0 ;
while (k < maxIter && euclideanNorm(x, xPrev) > epsilon) {
xPrev = Arrays.copyOf(x, x.length);
double [] g = grad(x[ 0 ], x[ 1 ]);
x[ 0 ] = x[ 0 ] - alpha * g[ 0 ];
x[ 1 ] = x[ 1 ] - alpha * g[ 1 ];
k++;
}
System.out.println( "Local optima: " + Arrays.toString(x));
}
public static double euclideanNorm( double [] x, double [] y) {
double sum = 0 ;
for ( int i = 0 ; i < x.length; i++) {
sum += (x[i] - y[i]) * (x[i] - y[i]);
}
return Math.sqrt(sum);
}
}
|
Output
Local optima: [-5.497545059289484, -3.9984827632005353]
Share your thoughts in the comments
Please Login to comment...