Dynamic Programming is an algorithmic paradigm that solves a given complex problem by breaking it into subproblems using recursion and storing the results of subproblems to avoid computing the same results again. Following are the two main properties of a problem that suggests that the given problem can be solved using Dynamic programming.
In this post, we will discuss the first property (Overlapping Subproblems) in detail. The second property of Dynamic programming is discussed in the next post i.e. Set 2. 1) Overlapping Subproblems 2) Optimal Substructure
1) Overlapping Subproblems: Like Divide and Conquer, Dynamic Programming combines solutions to sub-problems. Dynamic Programming is mainly used when solutions to the same subproblems are needed again and again. In dynamic programming, computed solutions to subproblems are stored in a table so that these don’t have to be recomputed. So Dynamic Programming is not useful when there are no common (overlapping) subproblems because there is no point in storing the solutions if they are not needed again. For example, Binary Search doesn’t have common subproblems. If we take the example of following a recursive program for Fibonacci Numbers, there are many subproblems that are solved again and again.
C++
#include <iostream>
usingnamespacestd;
/* a simple recursive program for Fibonacci numbers */
intfib(intn)
{
if(n <= 1)
returnn;
returnfib(n - 1) + fib(n - 2);
}
intmain() {
cout << fib(7);
return0;
}
// This code is contributed by sanjoy_62.
C
/* a simple recursive program for Fibonacci numbers */
intfib(intn)
{
if(n <= 1)
returnn;
returnfib(n - 1) + fib(n - 2);
}
Java
/*package whatever //do not write package name here */
/* a simple recursive program for Fibonacci numbers */
staticintfib(intn)
{
if(n <= 1)
returnn;
returnfib(n - 1) + fib(n - 2);
}
// This code is contributed by umadevi9616
Python
# a simple recursive program for Fibonacci numbers
deffib(n):
ifn <=1:
returnn
returnfib(n -1) +fib(n -2)
C#
/* a simple recursive program for Fibonacci numbers */
staticintfib(intn)
{
if(n <= 1)
returnn;
returnfib(n - 1) + fib(n - 2);
}
// This code contributed by umadevi9616
Javascript
<script>
/*package whatever //do not write package name here */
/* a simple recursive program for Fibonacci numbers */
functionfib(n)
{
if(n <= 1)
returnn;
returnfib(n - 1) + fib(n - 2);
}
// This code is contributed by gauravrajput1
</script>
Output
13
Time Complexity: O(2^n), As at every step we need to make 2 calls. Auxiliary Space: O(1), As constant extra space is used.
We can see that the function fib(3) is being called 2 times. If we would have stored the value of fib(3), then instead of computing it again, we could have reused the old stored value. There are following two different ways to store the values so that these values can be reused: a) Memoization (Top Down) b) Tabulation (Bottom Up)
a) Memoization (Top Down): The memoized program for a problem is similar to the recursive version with a small modification that looks into a lookup table before computing solutions. We initialize a lookup array with all initial values as NIL. Whenever we need the solution to a subproblem, we first look into the lookup table. If the precomputed value is there then we return that value, otherwise, we calculate the value and put the result in the lookup table so that it can be reused later.
Following is the memoized version for the nth Fibonacci Number.
C++
/* C++ program for Memoized version
for nth Fibonacci number */
#include <bits/stdc++.h>
usingnamespacestd;
#define NIL -1
#define MAX 100
intlookup[MAX];
/* Function to initialize NIL
values in lookup table */
void_initialize()
{
inti;
for(i = 0; i < MAX; i++)
lookup[i] = NIL;
}
/* function for nth Fibonacci number */
intfib(intn)
{
if(lookup[n] == NIL) {
if(n <= 1)
lookup[n] = n;
else
lookup[n] = fib(n - 1) + fib(n - 2);
}
returnlookup[n];
}
// Driver code
intmain()
{
intn = 40;
_initialize();
cout << "Fibonacci number is "<< fib(n);
return0;
}
// This is code is contributed by rathbhupendra
C
/* C program for Memoized version for nth Fibonacci number
*/
#include <stdio.h>
#define NIL -1
#define MAX 100
intlookup[MAX];
/* Function to initialize NIL values in lookup table */
void_initialize()
{
inti;
for(i = 0; i < MAX; i++)
lookup[i] = NIL;
}
/* function for nth Fibonacci number */
intfib(intn)
{
if(lookup[n] == NIL) {
if(n <= 1)
lookup[n] = n;
else
lookup[n] = fib(n - 1) + fib(n - 2);
}
returnlookup[n];
}
intmain()
{
intn = 40;
_initialize();
printf("Fibonacci number is %d ", fib(n));
return0;
}
Java
/* Java program for Memoized version */
publicclassFibonacci {
finalintMAX = 100;
finalintNIL = -1;
intlookup[] = newint[MAX];
/* Function to initialize NIL values in lookup table */
void_initialize()
{
for(inti = 0; i < MAX; i++)
lookup[i] = NIL;
}
/* function for nth Fibonacci number */
intfib(intn)
{
if(lookup[n] == NIL) {
if(n <= 1)
lookup[n] = n;
else
lookup[n] = fib(n - 1) + fib(n - 2);
}
returnlookup[n];
}
publicstaticvoidmain(String[] args)
{
Fibonacci f = newFibonacci();
intn = 40;
f._initialize();
System.out.println("Fibonacci number is"
+ " "+ f.fib(n));
}
}
// This Code is Contributed by Saket Kumar
Python
# a program for Memoized version of nth Fibonacci number
# function to calculate nth Fibonacci number
deffib(n, lookup):
# base case
ifn <=1:
lookup[n] =n
# if the value is not calculated previously then calculate it
iflookup[n] isNone:
lookup[n] =fib(n-1, lookup) +fib(n-2, lookup)
# return the value corresponding to that value of n
returnlookup[n]
# end of function
# Driver program to test the above function
defmain():
n =34
# Declaration of lookup table
# Handles till n = 100
lookup =[None] *101
print"Fibonacci Number is ", fib(n, lookup)
if__name__ =="__main__":
main()
# This code is contributed by Nikhil Kumar Singh(nickzuck_007)
C#
// C# program for Memoized versionof nth Fibonacci number
usingSystem;
classGFG {
staticintMAX = 100;
staticintNIL = -1;
staticint[] lookup = newint[MAX];
/* Function to initialize NIL
values in lookup table */
staticvoidinitialize()
{
for(inti = 0; i < MAX; i++)
lookup[i] = NIL;
}
/* function for nth Fibonacci number */
staticintfib(intn)
{
if(lookup[n] == NIL) {
if(n <= 1)
lookup[n] = n;
else
lookup[n] = fib(n - 1) + fib(n - 2);
}
returnlookup[n];
}
// Driver code
publicstaticvoidMain()
{
intn = 40;
initialize();
Console.Write("Fibonacci number is"
+ " "+ fib(n));
}
}
// This Code is Contributed by Sam007
Javascript
<script>
let MAX = 100;
let NIL = -1;
let lookup = newArray(MAX);
function_initialize()
{
for(let i = 0; i < MAX; i++)
lookup[i] = NIL;
}
functionfib(n)
{
if(lookup[n] == NIL)
{
if(n <= 1)
lookup[n] = n;
else
lookup[n] = fib(n-1) + fib(n-2);
}
returnlookup[n];
}
let n = 40;
_initialize();
document.write("Fibonacci number is"+ " "+ fib(n)+"<br>");
// This code is contributed by avanitrachhadiya2155
</script>
Output
Fibonacci number is 102334155
b) Tabulation (Bottom Up): The tabulated program for a given problem builds a table in a bottom-up fashion and returns the last entry from the table. For example, for the same Fibonacci number, we first calculate fib(0) then fib(1) then fib(2) then fib(3), and so on. So literally, we are building the solutions to subproblems bottom-up.
Following is the tabulated version for the nth Fibonacci Number.
C
/* C program for Tabulated version */
#include <stdio.h>
intfib(intn)
{
intf[n + 1];
inti;
f[0] = 0;
f[1] = 1;
for(i = 2; i <= n; i++)
f[i] = f[i - 1] + f[i - 2];
returnf[n];
}
intmain()
{
intn = 9;
printf("Fibonacci number is %d ", fib(n));
return0;
}
Java
/* Java program for Tabulated version */
publicclassFibonacci {
intfib(intn)
{
intf[] = newint[n + 1];
f[0] = 0;
f[1] = 1;
for(inti = 2; i <= n; i++)
f[i] = f[i - 1] + f[i - 2];
returnf[n];
}
publicstaticvoidmain(String[] args)
{
Fibonacci f = newFibonacci();
intn = 9;
System.out.println("Fibonacci number is"
+ " "+ f.fib(n));
}
}
// This Code is Contributed by Saket Kumar
Python
# Python program Tabulated (bottom up) version
deffib(n):
# array declaration
f =[0] *(n +1)
# base case assignment
f[1] =1
# calculating the fibonacci and storing the values
fori inxrange(2, n +1):
f[i] =f[i -1] +f[i -2]
returnf[n]
# Driver program to test the above function
defmain():
n =9
print"Fibonacci number is ", fib(n)
if__name__ =="__main__":
main()
# This code is contributed by Nikhil Kumar Singh (nickzuck_007)
C#
// C# program for Tabulated version
usingSystem;
classGFG {
staticintfib(intn)
{
int[] f = newint[n + 1];
f[0] = 0;
f[1] = 1;
for(inti = 2; i <= n; i++)
f[i] = f[i - 1] + f[i - 2];
returnf[n];
}
publicstaticvoidMain()
{
intn = 9;
Console.Write("Fibonacci number is"
+ " "+ fib(n));
}
}
// This Code is Contributed by Sam007
Javascript
<script>
// Javascript program for Tabulated version
functionfib(n)
{
varf = newArray(n + 1);
vari;
f[0] = 0;
f[1] = 1;
for(i = 2; i <= n; i++)
f[i] = f[i - 1] + f[i - 2];
returnf[n];
}
// Driver code
varn = 9;
document.write("Fibonacci number is "+ fib(n));
// This code is contributed by akshitsaxenaa09
</script>
PHP
<?php
// PHP program for Tabulated version
functionfib($n)
{
$f[$n+ 1]=0;
$i;
$f[0] = 0;
$f[1] = 1;
for($i= 2; $i<= $n; $i++)
$f[$i] = $f[$i- 1] +
$f[$i- 2];
return$f[$n];
}
// Driver Code
$n= 9;
echo("Fibonacci number is ");
echo(fib($n));
// This code is contributed by nitin mittal.
?>
C++
/* C++ program for Tabulated version */
#include <iostream>
usingnamespacestd;
intfib(intn)
{
intf[n + 1];
inti;
f[0] = 0;
f[1] = 1;
for(i = 2; i <= n; i++)
f[i] = f[i - 1] + f[i - 2];
returnf[n];
}
intmain()
{
intn = 9;
printf("Fibonacci number is %d ", fib(n));
return0;
}
Output
Fibonacci number is 34
Time Complexity: O(n), As we are traversing linearly. Auxiliary Space: O(n), The extra space is used in tabulation array.
Both Tabulated and Memoized store the solutions to subproblems. In Memoized version, the table is filled on demand while in the Tabulated version, starting from the first entry, all entries are filled one by one. Unlike the Tabulated version, all entries of the lookup table are not necessarily filled in Memoized version. For example, Memoized solution to the LCS problem doesn’t necessarily fill all entries.
To see the optimization achieved by Memoized and Tabulated solutions over the basic Recursive solution, see the time taken by following runs for calculating the 40th Fibonacci number: Recursive solution Memoized solution Tabulated solution Time taken by the Recursion method is much more than the two Dynamic Programming techniques mentioned above – Memoization and Tabulation!
Also, see method 2 of the Ugly Number post for one more simple example where we have overlapping subproblems and we store the results of subproblems.
We will be covering Optimal Substructure Property and some more example problems in future posts on Dynamic Programming.
Try the following questions as an exercise of this post. 1) Write a Memoized solution for LCS problem. Note that the Tabular solution is given in the CLRS book. 2) How would you choose between Memoization and Tabulation?
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above. References: http://www.youtube.com/watch?v=V5hZoJ6uK-s
We use cookies to ensure you have the best browsing experience on our website. By using our site, you
acknowledge that you have read and understood our
Cookie Policy &
Privacy Policy