Java Program to Implement Steepest Descent Method and Compute Local Optima

The steepest descent method is an optimization algorithm that can be used to find the local minimum of a differentiable function. It is an iterative method that starts with an initial guess for the solution, and then repeatedly takes steps in the direction of the steepest descent (the direction of the greatest negative slope) until it reaches a local minimum. To implement the steepest descent method in Java, you would first need to define a differentiable function that you want to minimize. This function should take as input some set of parameters and return a scalar value that represents the cost or error of the current parameter values.

Next, you would need to define an initial guess for the solution, which would typically be a set of starting values for the parameters of the function. This initial guess would be used as the starting point for the optimization algorithm. Then, you would need to implement the main loop of the steepest descent algorithm. This loop would iterate until some stopping criterion is met, such as reaching a pre-defined number of iterations or a certain level of precision in the solution. At each iteration, the algorithm would compute the gradient of the function at the current parameter values, and then take a step in the direction of the negative gradient. This would typically involve updating the parameter values by subtracting a fraction of the gradient vector from the current values.

After the main loop of the algorithm has finished, the final parameter values would represent the local minimum of the function, according to the steepest descent method. You could then use these values to make predictions or evaluate the performance of the model. It is important to note that the steepest descent method only guarantees to find a local minimum, not a global minimum. This means that the solution obtained using this algorithm may not be the optimal solution for the problem, especially if the function has multiple local minima. To improve the chances of finding the global minimum, you could try using different initial guesses for the solution or modifying the step size of the algorithm.

Here are examples of a Java program that implements the steepest descent method to compute the local optima of a function.

Example 1:

Java

/*Java Program to Implement Steepest Descent Method and Compute Local Optima*/

import java.util.function.Function;

public class SteepestDescent {

  public static double minimize(Function<double[], Double> f, double[] x0, double alpha, double eps) {
    int n = x0.length;
    double[] x = x0.clone();
    double[] grad = new double[n];

    while (true) {
      double fx = f.apply(x);

      // Compute the gradient of f at x
      for (int i = 0; i < n; i++) {
        double old = x[i];
        x[i] = old + eps;
        grad[i] = (f.apply(x) - fx) / eps;
        x[i] = old;
      }

      // Update x in the direction of the gradient
      for (int i = 0; i < n; i++) {
        x[i] = x[i] - alpha * grad[i];
      }

      // Check for convergence
      boolean converged = true;
      for (int i = 0; i < n; i++) {
        if (Math.abs(grad[i]) > eps) {
          converged = false;
          break;
        }
      }

      if (converged) {
        break;
      }
    }

    return f.apply(x);
  }

  public static void main(String[] args) {
    // Test the minimization function
    double[] x0 = {1, 2, 3};
    double alpha = 0.1;
    double eps = 0.001;
    Function<double[], Double> f = x -> x[0] * x[0] + x[1] * x[1] + x[2] * x[2];
    double localOptima = minimize(f, x0, alpha, eps);
    System.out.println("Local Optima: " + localOptima);
  }

}

Output

Local Optima: 1.9972843147380194E-7

Example 2:

Below program uses the steepest descent method to minimize the function f(x, y) = x² + 2y² – 2xy + 3x + 5y.

Java

import java.util.Arrays;

public class SteepestDescent {

    // Function to be minimized
    public static double f(double x, double y) {
        return x*x + 2*y*y - 2*x*y + 3*x + 5*y;
    }

    // Gradient of the function
    public static double[] grad(double x, double y) {
        return new double[] { 2*x - 2*y + 3, 4*y - 2*x + 5 };
    }

    // Step size for the gradient descent
    public static double alpha = 0.1;

    // Maximum number of iterations
    public static int maxIter = 100;

    // Stopping criteria
    public static double epsilon = 1e-6;

    public static void main(String[] args) {

        // Starting point
        double[] x0 = { 1, 1 };

        // Current point
        double[] x = x0;

        // Previous point
        double[] xPrev = new double[x0.length];

        // Iteration counter
        int k = 0;

        // Gradient descent loop
        while (k < maxIter && euclideanNorm(x, xPrev) > epsilon) {

            // Update the previous point
            xPrev = Arrays.copyOf(x, x.length);

            // Compute the gradient at the current point
            double[] g = grad(x[0], x[1]);

            // Update the current point
            x[0] = x[0] - alpha * g[0];
            x[1] = x[1] - alpha * g[1];

            // Increase the iteration counter
            k++;
        }

        // Print the result
        System.out.println("Local optima: " + Arrays.toString(x));
    }

    // Euclidean norm
    public static double euclideanNorm(double[] x, double[] y) {
        double sum = 0;
        for (int i = 0; i < x.length; i++) {
            sum += (x[i] - y[i]) * (x[i] - y[i]);
        }
        return Math.sqrt(sum);
    }
}

Output

Local optima: [-5.497545059289484, -3.9984827632005353]