Greedy Algorithms (General Structure and Applications)

The general structure of a greedy algorithm can be summarized in the following steps:

1. Identify the problem as an optimization problem where we need to find the best solution among a set of possible solutions.
2. Determine the set of feasible solutions for the problem.
3. Identify the optimal substructure of the problem, meaning that the optimal solution to the problem can be constructed from the optimal solutions of its subproblems.
4. Develop a greedy strategy to construct a feasible solution step by step, making the locally optimal choice at each step.
   Prove the correctness of the algorithm by showing that the locally optimal choices at each step lead to a globally optimal solution.

Some common applications of greedy algorithms include:

1. Coin change problem: Given a set of coins with different denominations, find the minimum number of coins required to make a given amount of change.
   Fractional knapsack problem: Given a set of items with weights and values, fill a knapsack with a maximum weight capacity with the most valuable items, allowing fractional amounts of items to be included.
   Huffman coding: Given a set of characters and their frequencies in a message, construct a binary code with minimum average length for the characters.
   Shortest path algorithms: Given a weighted graph, find the shortest path between two nodes.
   Minimum spanning tree: Given a weighted graph, find a tree that spans all nodes with the minimum total weight.
   Greedy algorithms can be very efficient and provide fast solutions for many problems. However, it is important to keep in mind that they may not always provide the optimal solution and to analyze the problem carefully to ensure the correctness of the algorithm.
2. Greedy Algorithms work step-by-step, and always choose the steps which provide immediate profit/benefit. It chooses the “locally optimal solution”, without thinking about future consequences. Greedy algorithms may not always lead to the optimal global solution, because it does not consider the entire data. The choice made by the greedy approach does not consider future data and choices. In some cases making a decision that looks right at that moment gives the best solution (Greedy), but in other cases, it doesn’t. The greedy technique is used for optimization problems (where we have to find the maximum or minimum of something). The Greedy technique is best suited for looking at the immediate situation.

All greedy algorithms follow a basic structure:

1. declare an empty result = 0.
2. We make a greedy choice to select, If the choice is feasible add it to the final result.
3. return the result.

Why choose Greedy Approach:

The greedy approach has a few tradeoffs, which may make it suitable for optimization. One prominent reason is to achieve the most feasible solution immediately. In the activity selection problem (Explained below), if more activities can be done before finishing the current activity, these activities can be performed within the same time.  Another reason is to divide a problem recursively based on a condition, with no need to combine all the solutions. In the activity selection problem, the “recursive division” step is achieved by scanning a list of items only once and considering certain activities.

Greedy choice property: 

This property says that the globally optimal solution can be obtained by making a locally optimal solution (Greedy). The choice made by a Greedy algorithm may depend on earlier choices but not on the future. It iteratively makes one Greedy choice after another and reduces the given problem to a smaller one.

Optimal substructure:

A problem exhibits optimal substructure if an optimal solution to the problem contains optimal solutions to the subproblems. That means we can solve subproblems and build up the solutions to solve larger problems.

Note: Making locally optimal choices does not always work. Hence, Greedy algorithms will not always give the best solutions.

Characteristics of Greedy approach:

• There is an ordered list of resources(profit, cost, value, etc.) 
• Maximum of all the resources(max profit, max value, etc.) are taken. 
• For example, in the fractional knapsack problem, the maximum value/weight is taken first according to available capacity. 

Characteristic components of greedy algorithm:

1. The feasible solution: A subset of given inputs that satisfies all specified constraints of a problem is known as a “feasible solution”.
2. Optimal solution: The feasible solution that achieves the desired extremum is called an “optimal solution”. In other words, the feasible solution that either minimizes or maximizes the objective function specified in a problem is known as an “optimal solution”.
3. Feasibility check: It investigates whether the selected input fulfils all constraints mentioned in a problem or not. If it fulfils all the constraints then it is added to a set of feasible solutions; otherwise, it is rejected.
4. Optimality check: It investigates whether a selected input produces either a minimum or maximum value of the objective function by fulfilling all the specified constraints. If an element in a solution set produces the desired extremum, then it is added to a sel of optimal solutions.
5. Optimal substructure property: The globally optimal solution to a problem includes the optimal sub solutions within it.
6. Greedy choice property: The globally optimal solution is assembled by selecting locally optimal choices. The greedy approach applies some locally optimal criteria to obtain a partial solution that seems to be the best at that moment and then find out the solution for the remaining sub-problem.

 The local decisions (or choices) must possess three characteristics as mentioned below: 

1. Feasibility: The selected choice must fulfil local constraints.
2. Optimality: The selected choice must be the best at that stage (locally optimal choice).
3. Irrevocability: The selected choice cannot be changed once it is made.

Applications of Greedy Algorithms:

• Finding an optimal solution (Activity selection, Fractional Knapsack, Job Sequencing, Huffman Coding). 
• Finding close to the optimal solution for NP-Hard problems like TSP. 
• Network design: Greedy algorithms can be used to design efficient networks, such as minimum spanning trees, shortest paths, and maximum flow networks. These algorithms can be applied to a wide range of network design problems, such as routing, resource allocation, and capacity planning.
• Machine learning: Greedy algorithms can be used in machine learning applications, such as feature selection, clustering, and classification. In feature selection, greedy algorithms are used to select a subset of features that are most relevant to a given problem. In clustering and classification, greedy algorithms can be used to optimize the selection of clusters or classes.
• Image processing: Greedy algorithms can be used to solve a wide range of image processing problems, such as image compression, denoising, and segmentation. For example, Huffman coding is a greedy algorithm that can be used to compress digital images by efficiently encoding the most frequent pixels.
• Combinatorial optimization: Greedy algorithms can be used to solve combinatorial optimization problems, such as the traveling salesman problem, graph coloring, and scheduling. Although these problems are typically NP-hard, greedy algorithms can often provide close-to-optimal solutions that are practical and efficient.
• Game theory: Greedy algorithms can be used in game theory applications, such as finding the optimal strategy for games like chess or poker. In these applications, greedy algorithms can be used to identify the most promising moves or actions at each turn, based on the current state of the game.
• Financial optimization: Greedy algorithms can be used in financial applications, such as portfolio optimization and risk management. In portfolio optimization, greedy algorithms can be used to select a subset of assets that are most likely to provide the best return on investment, based on historical data and current market trends.

Applications of Greedy Approach:

Greedy algorithms are used to find an optimal or near optimal solution to many real-life problems. Few of them are listed below:

(1) Make a change problem

(2) Knapsack problem

(3) Minimum spanning tree

 (4) Single source shortest path

(5) Activity selection problem 

(6) Job sequencing problem

(7) Huffman code generation.

(8) Dijkstra’s algorithm

(9) Greedy coloring

(10) Minimum cost spanning tree

(11) Job scheduling

(12) Interval scheduling

(13) Greedy set cover

(14) Knapsack with fractions

Advantages of the Greedy Approach:

• The greedy approach is easy to implement.
• Typically have less time complexity.
• Greedy algorithms can be used for optimization purposes or finding close to optimization in case of Hard problems.
• Greedy algorithms can produce efficient solutions in many cases, especially when the problem has a substructure that exhibits the greedy choice property.
• Greedy algorithms are often faster than other optimization algorithms, such as dynamic programming or branch and bound, because they require less computation and memory.
• The greedy approach is often used as a heuristic or approximation algorithm when an exact solution is not feasible or when finding an exact solution would be too time-consuming.
• The greedy approach can be applied to a wide range of problems, including problems in computer science, operations research, economics, and other fields.
• The greedy approach can be used to solve problems in real-time, such as scheduling problems or resource allocation problems, because it does not require the solution to be computed in advance.
• Greedy algorithms are often used as a first step in solving optimization problems, because they provide a good starting point for more complex optimization algorithms.
• Greedy algorithms can be used in conjunction with other optimization algorithms, such as local search or simulated annealing, to improve the quality of the solution.

Disadvantages of the Greedy Approach:

• The local optimal solution may not always be globally optimal.
• Greedy algorithms do not always guarantee to find the optimal solution, and may produce suboptimal solutions in some cases.
• The greedy approach relies heavily on the problem structure and the choice of criteria used to make the local optimal choice. If the criteria are not chosen carefully, the solution produced may be far from optimal.
• Greedy algorithms may require a lot of preprocessing to transform the problem into a form that can be solved by the greedy approach.
• Greedy algorithms may not be applicable to problems where the optimal solution depends on the order in which the inputs are processed.
• Greedy algorithms may not be suitable for problems where the optimal solution depends on the size or composition of the input, such as the bin packing problem.
• Greedy algorithms may not be able to handle constraints on the solution space, such as constraints on the total weight or capacity of the solution.
• Greedy algorithms may be sensitive to small changes in the input, which can result in large changes in the output. This can make the algorithm unstable and unpredictable in some cases.

Standard Greedy Algorithms :

• Prim’s Algorithm
• Kruskal’s Algorithm
• Dijkstra’s Algorithm