Algorithms Design Techniques

What is an algorithm?
An Algorithm is a procedure to solve a particular problem in a finite number of steps for a finite-sized input. 
The algorithms can be classified in various ways. They are: 

1. Implementation Method
2. Design Method
3. Design Approaches
4. Other Classifications

In this article, the different algorithms in each classification method are discussed. 

The classification of algorithms is important for several reasons:

Organization: Algorithms can be very complex and by classifying them, it becomes easier to organize, understand, and compare different algorithms.

Problem Solving: Different problems require different algorithms, and by having a classification, it can help identify the best algorithm for a particular problem.

Performance Comparison: By classifying algorithms, it is possible to compare their performance in terms of time and space complexity, making it easier to choose the best algorithm for a particular use case.

Reusability: By classifying algorithms, it becomes easier to re-use existing algorithms for similar problems, thereby reducing development time and improving efficiency.

Research: Classifying algorithms is essential for research and development in computer science, as it helps to identify new algorithms and improve existing ones.

Overall, the classification of algorithms plays a crucial role in computer science and helps to improve the efficiency and effectiveness of solving problems.
Classification by Implementation Method: There are primarily three main categories into which an algorithm can be named in this type of classification. They are: 

1. Recursion or Iteration: A recursive algorithm is an algorithm which calls itself again and again until a base condition is achieved whereas iterative algorithms use loops and/or data structures like stacks, queues to solve any problem. Every recursive solution can be implemented as an iterative solution and vice versa. 
   Example: The Tower of Hanoi is implemented in a recursive fashion while Stock Span problem is implemented iteratively.
2. Exact or Approximate: Algorithms that are capable of finding an optimal solution for any problem are known as the exact algorithm. For all those problems, where it is not possible to find the most optimized solution, an approximation algorithm is used. Approximate algorithms are the type of algorithms that find the result as an average outcome of sub outcomes to a problem. 
   Example: For NP-Hard Problems, approximation algorithms are used. Sorting algorithms are the exact algorithms.
3. Serial or Parallel or Distributed Algorithms: In serial algorithms, one instruction is executed at a time while parallel algorithms are those in which we divide the problem into subproblems and execute them on different processors. If parallel algorithms are distributed on different machines, then they are known as distributed algorithms.

Classification by Design Method: There are primarily three main categories into which an algorithm can be named in this type of classification. They are: 

1. Greedy Method: In the greedy method, at each step, a decision is made to choose the local optimum, without thinking about the future consequences. 
   Example: Fractional Knapsack, Activity Selection.
2. Divide and Conquer: The Divide and Conquer strategy involves dividing the problem into sub-problem, recursively solving them, and then recombining them for the final answer. 
   Example: Merge sort, Quicksort.
3. Dynamic Programming: The approach of Dynamic programming is similar to divide and conquer. The difference is that whenever we have recursive function calls with the same result, instead of calling them again we try to store the result in a data structure in the form of a table and retrieve the results from the table. Thus, the overall time complexity is reduced. “Dynamic” means we dynamically decide, whether to call a function or retrieve values from the table. 
   Example: 0-1 Knapsack, subset-sum problem.
4. Linear Programming: In Linear Programming, there are inequalities in terms of inputs and maximizing or minimizing some linear functions of inputs. 
   Example: Maximum flow of Directed Graph
5. Reduction(Transform and Conquer): In this method, we solve a difficult problem by transforming it into a known problem for which we have an optimal solution. Basically, the goal is to find a reducing algorithm whose complexity is not dominated by the resulting reduced algorithms. 
   Example: Selection algorithm for finding the median in a list involves first sorting the list and then finding out the middle element in the sorted list. These techniques are also called transform and conquer.
6. Backtracking: This technique is very useful in solving combinatorial problems that have a single unique solution. Where we have to find the correct combination of steps that lead to fulfillment of the task.  Such problems have multiple stages and there are multiple options at each stage. This approach is based on exploring each available option at every stage one-by-one. While exploring an option if a point is reached that doesn’t seem to lead to the solution, the program control backtracks one step, and starts exploring the next option. In this way, the program explores all possible course of actions and finds the route that leads to the solution.  
   Example: N-queen problem, maize problem.
7. Branch and Bound: This technique is very useful in solving combinatorial optimization problem that have multiple solutions and we are interested in find the most optimum solution. In this approach, the entire solution space is represented in the form of a state space tree. As the program progresses each state combination is explored, and the previous solution is replaced by new one if it is not the optimal than the current solution. 
   Example: Job sequencing, Travelling salesman problem.

Classification by Design Approaches : There are two approaches for designing an algorithm. these approaches include 

1. Top-Down Approach :
2. Bottom-up approach

• Top-Down Approach: In the top-down approach, a large problem is divided into small sub-problem. and keep                repeating the process of decomposing problems until the complex problem is solved.
• Bottom-up approach: The bottom-up approach is also known as the reverse of top-down approaches.
  In approach different, part of a complex program is solved using a programming language and then this is combined into a complete program.

Top-Down Approach:

Breaking down a complex problem into smaller, more manageable sub-problems and solving each sub-problem individually.
Designing a system starting from the highest level of abstraction and moving towards the lower levels.
Bottom-Up Approach:

Building a system by starting with the individual components and gradually integrating them to form a larger system.
Solving sub-problems first and then using the solutions to build up to a solution of a larger problem.
Note: Both approaches have their own advantages and disadvantages and the choice between them often depends on the specific problem being solved.

Here are examples of the Top-Down and Bottom-Up approaches in code:

Top-Down Approach:

C++

// Nikunj Sonigara

#include <iostream>
#include <vector>

using namespace std;

// Function to break down a complex problem
vector<string> break_down_complex_problem(const string& problem) {
    // Implementation for breaking down a complex problem
    // Replace this with actual implementation if needed
    vector<string> sub_problems;
    // ...
    return sub_problems;
}

// Function to combine sub-solutions
string combine_sub_solutions(const vector<string>& sub_solutions) {
    // Implementation for combining sub-solutions
    // Replace this with actual implementation if needed
    string combined_solution = "";
    // ...
    return combined_solution;
}

// Function to solve the problem
string solve_problem(const string& problem) {
    if (problem == "simple") {
        return "solved";
    } else if (problem == "complex") {
        vector<string> sub_problems = break_down_complex_problem(problem);
        vector<string> sub_solutions;

        for (const string& sub : sub_problems) {
            sub_solutions.push_back(solve_problem(sub));
        }

        return combine_sub_solutions(sub_solutions);
    } else {
        // Handle other cases if needed
        return "unknown";
    }
}

int main() {
    // Example usage
    cout << solve_problem("simple") << endl;
    cout << solve_problem("complex") << endl;

    return 0;
}

Java

import java.util.ArrayList;
import java.util.List;

public class Main {

    // Function to break down a complex problem
    public static List<String> breakDownComplexProblem(String problem) {
        // Implementation for breaking down a complex problem
        // Replace this with the actual implementation if needed
        List<String> subProblems = new ArrayList<>();
        // ...
        return subProblems;
    }

    // Function to combine sub-solutions
    public static String combineSubSolutions(List<String> subSolutions) {
        // Implementation for combining sub-solutions
        // Replace this with the actual implementation if needed
        StringBuilder combinedSolution = new StringBuilder();
        // ...
        return combinedSolution.toString();
    }

    // Function to solve the problem
    public static String solveProblem(String problem) {
        if ("simple".equals(problem)) {
            return "solved";
        } else if ("complex".equals(problem)) {
            List<String> subProblems = breakDownComplexProblem(problem);
            List<String> subSolutions = new ArrayList<>();

            for (String sub : subProblems) {
                subSolutions.add(solveProblem(sub));
            }

            return combineSubSolutions(subSolutions);
        } else {
            // Handle other cases if needed
            return "unknown";
        }
    }

    public static void main(String[] args) {
        // Example usage
        System.out.println(solveProblem("simple"));
        System.out.println(solveProblem("complex"));
    }
}

Python

def solve_problem(problem):
    if problem == "simple":
        return "solved"
    elif problem == "complex":
        sub_problems = break_down_complex_problem(problem)
        sub_solutions = [solve_problem(sub) for sub in sub_problems]
        return combine_sub_solutions(sub_solutions)

C#

using System;
using System.Collections.Generic;

class Program {
    // Function to break down a complex problem
    static List<string>
    BreakDownComplexProblem(string problem)
    {
        // Implementation for breaking down a complex
        // problem Replace this with actual implementation
        // if needed
        List<string> subProblems = new List<string>();
        // ...
        return subProblems;
    }

    // Function to combine sub-solutions
    static string
    CombineSubSolutions(List<string> subSolutions)
    {
        // Implementation for combining sub-solutions
        // Replace this with actual implementation if needed
        string combinedSolution = "";
        // ...
        return combinedSolution;
    }

    // Function to solve the problem
    static string SolveProblem(string problem)
    {
        if (problem == "simple") {
            return "solved";
        }
        else if (problem == "complex") {
            List<string> subProblems
                = BreakDownComplexProblem(problem);
            List<string> subSolutions = new List<string>();

            foreach(string sub in subProblems)
            {
                subSolutions.Add(SolveProblem(sub));
            }

            return CombineSubSolutions(subSolutions);
        }
        else {
            // Handle other cases if needed
            return "unknown";
        }
    }

    // Main method
    static void Main()
    {
        // Example usage
        Console.WriteLine(SolveProblem("simple"));
        Console.WriteLine(SolveProblem("complex"));
    }
}

Javascript

// Function to break down a complex problem
function break_down_complex_problem(problem) {
    // Implementation for breaking down a complex problem
    // Replace this with actual implementation if needed
    const sub_problems = [];
    // ...
    return sub_problems;
}

// Function to combine sub-solutions
function combine_sub_solutions(sub_solutions) {
    // Implementation for combining sub-solutions
    // Replace this with actual implementation if needed
    let combined_solution = "";
    // ...
    return combined_solution;
}

// Function to solve the problem
function solve_problem(problem) {
    if (problem === "simple") {
        return "solved";
    } else if (problem === "complex") {
        const sub_problems = break_down_complex_problem(problem);
        const sub_solutions = [];

        for (const sub of sub_problems) {
            sub_solutions.push(solve_problem(sub));
        }

        return combine_sub_solutions(sub_solutions);
    } else {
        // Handle other cases if needed
        return "unknown";
    }
}

// Example usage
console.log(solve_problem("simple"));
console.log(solve_problem("complex"));

// This code is contributed by Yash Agarwal(yashagarwal2852002)

Bottom-Up Approach (in Python):

Python

def solve_sub_problems(sub_problems):
    return [solve_sub_problem(sub) for sub in sub_problems]

def combine_sub_solutions(sub_solutions):
    # implementation to combine sub-solutions to solve the larger problem

def solve_problem(problem):
    if problem == "simple":
        return "solved"
    elif problem == "complex":
        sub_problems = break_down_complex_problem(problem)
        sub_solutions = solve_sub_problems(sub_problems)
        return combine_sub_solutions(sub_solutions)

Other Classifications: Apart from classifying the algorithms into the above broad categories, the algorithm can be classified into other broad categories like: 

1. Randomized Algorithms: Algorithms that make random choices for faster solutions are known as randomized algorithms. 
   Example: Randomized Quicksort Algorithm.
2. Classification by complexity: Algorithms that are classified on the basis of time taken to get a solution to any problem for input size. This analysis is known as time complexity analysis. 
   Example: Some algorithms take O(n), while some take exponential time.
3. Classification by Research Area: In CS each field has its own problems and needs efficient algorithms. 
   Example: Sorting Algorithm, Searching Algorithm, Machine Learning etc.
4. Branch and Bound Enumeration and Backtracking: These are mostly used in Artificial Intelligence.