Greedy Algorithms (General Structure and Applications)

• Difficulty Level : Easy
• Last Updated : 13 Mar, 2023

The general structure of a greedy algorithm can be summarized in the following steps:

1. Identify the problem as an optimization problem where we need to find the best solution among a set of possible solutions.
2. Determine the set of feasible solutions for the problem.
3. Identify the optimal substructure of the problem, meaning that the optimal solution to the problem can be constructed from the optimal solutions of its subproblems.
4. Develop a greedy strategy to construct a feasible solution step by step, making the locally optimal choice at each step.
Prove the correctness of the algorithm by showing that the locally optimal choices at each step lead to a globally optimal solution.

Some common applications of greedy algorithms include:

1. Coin change problem: Given a set of coins with different denominations, find the minimum number of coins required to make a given amount of change.
Fractional knapsack problem: Given a set of items with weights and values, fill a knapsack with a maximum weight capacity with the most valuable items, allowing fractional amounts of items to be included.
Huffman coding: Given a set of characters and their frequencies in a message, construct a binary code with minimum average length for the characters.
Shortest path algorithms: Given a weighted graph, find the shortest path between two nodes.
Minimum spanning tree: Given a weighted graph, find a tree that spans all nodes with the minimum total weight.
Greedy algorithms can be very efficient and provide fast solutions for many problems. However, it is important to keep in mind that they may not always provide the optimal solution and to analyze the problem carefully to ensure the correctness of the algorithm.
2. Greedy Algorithms work step-by-step, and always choose the steps which provide immediate profit/benefit. It chooses the “locally optimal solution”, without thinking about future consequences. Greedy algorithms may not always lead to the optimal global solution, because it does not consider the entire data. The choice made by the greedy approach does not consider future data and choices. In some cases making a decision that looks right at that moment gives the best solution (Greedy), but in other cases, it doesn’t. The greedy technique is used for optimization problems (where we have to find the maximum or minimum of something). The Greedy technique is best suited for looking at the immediate situation.

All greedy algorithms follow a basic structure:

1. declare an empty result = 0.
2. We make a greedy choice to select, If the choice is feasible add it to the final result.
3. return the result.

Why choose Greedy Approach:

The greedy approach has a few tradeoffs, which may make it suitable for optimization. One prominent reason is to achieve the most feasible solution immediately. In the activity selection problem (Explained below), if more activities can be done before finishing the current activity, these activities can be performed within the same time.  Another reason is to divide a problem recursively based on a condition, with no need to combine all the solutions. In the activity selection problem, the “recursive division” step is achieved by scanning a list of items only once and considering certain activities.

Greedy choice property:

This property says that the globally optimal solution can be obtained by making a locally optimal solution (Greedy). The choice made by a Greedy algorithm may depend on earlier choices but not on the future. It iteratively makes one Greedy choice after another and reduces the given problem to a smaller one.

Optimal substructure:

A problem exhibits optimal substructure if an optimal solution to the problem contains optimal solutions to the subproblems. That means we can solve subproblems and build up the solutions to solve larger problems.

Note: Making locally optimal choices does not always work. Hence, Greedy algorithms will not always give the best solutions.

Characteristics of Greedy approach:

• There is an ordered list of resources(profit, cost, value, etc.)
• Maximum of all the resources(max profit, max value, etc.) are taken.
• For example, in the fractional knapsack problem, the maximum value/weight is taken first according to available capacity.

Characteristic components of greedy algorithm:

1. The feasible solution: A subset of given inputs that satisfies all specified constraints of a problem is known as a “feasible solution”.
2. Optimal solution: The feasible solution that achieves the desired extremum is called an “optimal solution”. In other words, the feasible solution that either minimizes or maximizes the objective function specified in a problem is known as an “optimal solution”.
3. Feasibility check: It investigates whether the selected input fulfils all constraints mentioned in a problem or not. If it fulfils all the constraints then it is added to a set of feasible solutions; otherwise, it is rejected.
4. Optimality check: It investigates whether a selected input produces either a minimum or maximum value of the objective function by fulfilling all the specified constraints. If an element in a solution set produces the desired extremum, then it is added to a sel of optimal solutions.
5. Optimal substructure property: The globally optimal solution to a problem includes the optimal sub solutions within it.
6. Greedy choice property: The globally optimal solution is assembled by selecting locally optimal choices. The greedy approach applies some locally optimal criteria to obtain a partial solution that seems to be the best at that moment and then find out the solution for the remaining sub-problem.

The local decisions (or choices) must possess three characteristics as mentioned below:

1.  Feasibility: The selected choice must fulfil local constraints.
2.  Optimality: The selected choice must be the best at that stage (locally optimal choice).
3. Irrevocability: The selected choice cannot be changed once it is made.

Applications of Greedy Approach:

Greedy algorithms are used to find an optimal or near optimal solution to many real-life problems. Few of them are listed below:

(1) Make a change problem

(2) Knapsack problem

(3) Minimum spanning tree

(4) Single source shortest path

(5) Activity selection problem

(6) Job sequencing problem

(7) Huffman code generation.

• The greedy approach is easy to implement.
• Typically have less time complexity.
• Greedy algorithms can be used for optimization purposes or finding close to optimization in case of Hard problems.

• The local optimal solution may not always be globally optimal.

Standard Greedy Algorithms :

• Prim’s Algorithm
• Kruskal’s Algorithm
• Dijkstra’s Algorithm

My Personal Notes arrow_drop_up