Memoization (1D, 2D and 3D)

Most of the Dynamic Programming problems are solved in two ways:

  1. Tabulation: Bottom Up
  2. Memoization: Top Down

One of the easier approaches to solve most of the problems in DP is to write the recursive code at first and then write the Bottom-up Tabulation Method or Top-down Memoization of the recursive function. The steps to write the DP solution of Top-down approach to any problem is to:

  1. Write the recursive code
  2. Memoize the return value and use it to reduce recursive calls.

1-D Memoization

The first step will be to write the recursive code. In the program below, a program related to recursion where only one parameter changes its value has been shown. Since only one parameter is non-constant, this method is known as 1-D memoization. E.g., the Fibonacci series problem to find the N-th term in the Fibonacci series. The recursive approach has been discussed over here.

Given below is the recursive code to find the N-th term:



C++

// C++ program to find the Nth term
// of Fibonacci series
#include <bits/stdc++.h>
using namespace std;
  
// Fibonacci Series using Recursion
int fib(int n)
{
  
    // Base case
    if (n <= 1)
        return n;
  
    // recursive calls
    return fib(n - 1) + fib(n - 2);
}
  
// Driver Code
int main()
{
    int n = 6;
    printf("%d", fib(n));
    return 0;
}

Java

// Java program to find the 
// Nth term of Fibonacci series
import java.io.*;
  
class GFG
{
      
// Fibonacci Series 
// using Recursion
static int fib(int n)
{
  
    // Base case
    if (n <= 1)
        return n;
  
    // recursive calls
    return fib(n - 1) +
           fib(n - 2);
}
  
// Driver Code
public static void main (String[] args) 
{
    int n = 6;
    System.out.println(fib(n));
}
}
  
// This code is contributed
// by ajit

Python3

# Python3 program to find the Nth term
# of Fibonacci series
  
# Fibonacci Series using Recursion
def fib(n):
  
  
    # Base case
    if (n <= 1):
        return n
  
    # recursive calls
    return fib(n - 1) + fib(n - 2)
  
# Driver Code
if __name__=='__main__':
    n = 6
    print (fib(n))
   
# This code is contributed by 
# Shivi_Aggarwal

C#

// C# program to find 
// the Nth term of 
// Fibonacci series
using System;
  
class GFG
{
      
// Fibonacci Series 
// using Recursion
static int fib(int n)
{
    // Base case
    if (n <= 1)
        return n;
  
    // recursive calls
    return fib(n - 1) +
           fib(n - 2);
}
// Driver Code
static public void Main ()
{
    int n = 6;
    Console.WriteLine(fib(n));
}
}
  
// This code is contributed
// by akt_mit

PHP

<?php
// PHP program to find 
// the Nth term of 
// Fibonacci series
// using Recursion
  
function fib($n)
{
  
    // Base case
    if ($n <= 1)
        return $n;
  
    // recursive calls
    return fib($n - 1) +
           fib($n - 2);
}
  
// Driver Code
$n = 6;
echo fib($n);
  
// This code is contributed 
// by ajit
?>

Output:

8

A common observation is that this implementation does a lot of repeated work (see the following recursion tree). So this will consume a lot of time for finding the N-th Fibonacci number if done.

                            fib(5)   
                     /                 \        
               fib(4)                  fib(3)   
             /      \                /       \
         fib(3)      fib(2)         fib(2)    fib(1)
        /   \          /    \       /      \ 
  fib(2)   fib(1)  fib(1) fib(0) fib(1) fib(0)
  /    \ 
fib(1) fib(0) 

In the above tree fib(3), fib(2), fib(1), fib(0) all are called more then once. 

The following problem has been solved using Tabulation method.

In the program below, the steps to write a Top-Down approach program has been explained. Some modifications in the recursive program will reduce the complexity of the program and give the desired result. If fib(x) has not occurred previously, then we store the value of fib(x) in an array term at index x and return term[x]. By memoizing the return value of fib(x) at index x of an array, reduce the number of recursive calls at the next step when fib(x) has already been called. So without doing further recursive calls to compute the value of fib(x), return term[x] when fib(x) has already been computed previously to avoid a lot of repeated work as shown in the tree.

Given below is the memoized recursive code to find the N-th term.

C++

// CPP program to find the Nth term
// of Fibonacci series
#include <bits/stdc++.h>
using namespace std;
int term[1000];
// Fibonacci Series using memoized Recursion
int fib(int n)
{
    // base case
    if (n <= 1)
        return n;
  
    // if fib(n) has already been computed
    // we do not do further recursive calls
    // and hence reduce the number of repeated
    // work
    if (term[n] != 0)
        return term[n];
  
    else {
  
        // store the computed value of fib(n)
        // in an array term at index n to
        // so that it does not needs to be
        // precomputed again
        term[n] = fib(n - 1) + fib(n - 2);
  
        return term[n];
    }
}
  
// Driver Code
int main()
{
    int n = 6;
    printf("%d", fib(n));
    return 0;
}

Java

// Java program to find 
// the Nth term of 
// Fibonacci series
import java.io.*;
  
class GFG 
{
      
// Fibonacci Series using
// memoized Recursion
static int fib(int n)
{
    int []term = new int [1000];
      
    // base case
    if (n <= 1)
        return n;
  
    // if fib(n) has already 
    // been computed we do not
    // do further recursive 
    // calls and hence reduce 
    // the number of repeated
    // work
    if (term[n] != 0)
        return term[n];
  
    else 
    {
  
        // store the computed value
        // of fib(n) in an array 
        // term at index n to so that 
        // it does not needs to be 
        // precomputed again
        term[n] = fib(n - 1) + 
                  fib(n - 2);
  
        return term[n];
    }
}
  
// Driver Code
public static void main (String[] args) 
{
    int n = 6;
    System.out.println(fib(n));
}
}
  
// This code is contributed by ajit

Output:

8

If the recursive code has been written once, then memoization is just modifying the recursive program and storing the return values to avoid repetitive calls of functions which have been computed previously.

2-D Memoization

In the above program, the recursive function had only one argument whose value was not constant after every function call. Below, an implementation where the recursive program has two non-constant arguments has been shown.

For e.g., Program to solve the standard Dynamic Problem LCS problem when two strings are given. The general recursive solution of the problem is to generate all subsequences of both given sequences and find the longest matching subsequence. Total possible combinations will be 2n. Hence recursive solution will take O(2n). The approach to write the recursive solution has been discussed here.

Given below is the recursive solution to the LCS problem:

C++

// A Naive recursive implementation of LCS problem
#include <bits/stdc++.h>
  
int max(int a, int b);
  
// Returns length of LCS for X[0..m-1], Y[0..n-1]
int lcs(char* X, char* Y, int m, int n)
{
    if (m == 0 || n == 0)
        return 0;
    if (X[m - 1] == Y[n - 1])
        return 1 + lcs(X, Y, m - 1, n - 1);
    else
        return max(lcs(X, Y, m, n - 1),
                  lcs(X, Y, m - 1, n));
}
  
// Utility function to get max of 2 integers
int max(int a, int b)
{
    return (a > b) ? a : b;
}
  
// Driver Code
int main()
{
    char X[] = "AGGTAB";
    char Y[] = "GXTXAYB";
  
    int m = strlen(X);
    int n = strlen(Y);
  
    printf("Length of LCS is %dn", lcs(X, Y, m, n));
  
    return 0;
}

Output:

Length of LCS is 4n

Output:

Length of LCS is 4

Considering the above implementation, the following is a partial recursion tree for input strings “AXYT” and “AYZX”

                  lcs("AXYT", "AYZX")
                       /                 \
         lcs("AXY", "AYZX")            lcs("AXYT", "AYZ")
         /           \                   /               \
lcs("AX", "AYZX") lcs("AXY", "AYZ")   lcs("AXY", "AYZ") lcs("AXYT", "AY")

In the above partial recursion tree, lcs(“AXY”, “AYZ”) is being solved twice. On drawing the complete recursion tree, it has been observed that there are many subproblems which are solved again and again. So this problem has Overlapping Substructure property and recomputation of same subproblems can be avoided by either using Memoization or Tabulation. The tabulation method has been discussed here.

A common point of observation to use memoization in the recursive code will be the two non-constant arguments M and N in every function call. The function has 4 arguments, but 2 arguments are constant which do not affect the Memoization. The repetitive calls occur for N and M which have been called previously. So use a 2-D array to store the computed lcs(m, n) value at arr[m-1][n-1] as the string index starts from 0. Whenever the function with the same argument m and n are called again, we do not perform any further recursive call and return arr[m-1][n-1] as the previous computation of the lcs(m, n) has already been stored in arr[m-1][n-1], hence reducing the recursive calls that happen more then once.

Below is the implementation of the Memoization approach of the recursive code.

C++

// C++ program to memoize
// recursive implementation of LCS problem
#include <bits/stdc++.h>
int arr[1000][1000];
int max(int a, int b);
  
// Returns length of LCS for X[0..m-1], Y[0..n-1] */
// memoization applied in recursive solution
int lcs(char* X, char* Y, int m, int n)
{
    // base case
    if (m == 0 || n == 0)
        return 0;
  
    // if the same state has already been
    // computed
    if (arr[m - 1][n - 1] != -1)
        return arr[m - 1][n - 1];
  
    // if equal, then we store the value of the
    // function call
    if (X[m - 1] == Y[n - 1]) {
  
        // store it in arr to avoid further repetitive 
        // work in future function calls
        arr[m - 1][n - 1] = 1 + lcs(X, Y, m - 1, n - 1);
        return arr[m - 1][n - 1];
    }
    else {
        // store it in arr to avoid further repetitive 
        // work in future function calls
        arr[m - 1][n - 1] = max(lcs(X, Y, m, n - 1),
                                lcs(X, Y, m - 1, n));
        return arr[m - 1][n - 1];
    }
}
  
// Utility function to get max of 2 integers
int max(int a, int b)
{
    return (a > b) ? a : b;
}
  
// Driver Code
int main()
{
    memset(arr, -1, sizeof(arr));
    char X[] = "AGGTAB";
    char Y[] = "GXTXAYB";
  
    int m = strlen(X);
    int n = strlen(Y);
  
    printf("Length of LCS is %d", lcs(X, Y, m, n));
  
    return 0;
}

Output:

Length of LCS is 4

3-D Memoization

In the above program, the recursive function had only two arguments whose value were not constant after every function call. Below, an implementation where the recursive program has three non-constant arguments is done.
For e.g., Program to solve the standard Dynamic Problem LCS problem for three strings. The general recursive solution of the problem is to generate all subsequences of both given sequences and find the longest matching subsequence. Total possible combinations will be 3n. Hence recursive solution will take O(3n).

Given below is the recursive solution to the LCS problem:

C++

// A recursive implementation of LCS problem
// of three strings
#include <bits/stdc++.h>
int max(int a, int b);
  
// Returns length of LCS for X[0..m-1], Y[0..n-1]
int lcs(char* X, char* Y, char* Z, int m, int n, int o)
{
    // base case
    if (m == 0 || n == 0 || o == 0)
        return 0;
  
    // if equal, then check for next combination
    if (X[m - 1] == Y[n - 1] and Y[n - 1] == Z[o - 1]) {
  
        // recursive call
        return 1 + lcs(X, Y, Z, m - 1, n - 1, o - 1);
    }
    else {
  
        // return the maximum of the three other
        // possible states in recursion
        return max(lcs(X, Y, Z, m, n - 1, o),
                   max(lcs(X, Y, Z, m - 1, n, o),
                       lcs(X, Y, Z, m, n, o - 1)));
    }
}
  
// Utility function to get max of 2 integers
int max(int a, int b)
{
    return (a > b) ? a : b;
}
  
// Driver Code
int main()
{
  
    char X[] = "geeks";
    char Y[] = "geeksfor";
    char Z[] = "geeksforge";
    int m = strlen(X);
    int n = strlen(Y);
    int o = strlen(Z);
    printf("Length of LCS is %d", lcs(X, Y, Z, m, n, o));
  
    return 0;
}

Output:

Length of LCS is 5

The tabulation method has been shown here. On drawing the recursion tree completely, it has been noticed that there are many overlapping sub-problems which are been calculated multiple times. Since the function parameter has three non-constant parameters, hence a 3-D array will be used to memoize the value that was returned when lcs(x, y, z, m, n, o) for any value of m, n and o was called so that if lcs(x, y, z, m, n, o) is again called for the same value of m, n and o then the function will return the already stored value as it has been computed previously in the recursive call. arr[m][n][o] stores the value returned by the lcs(x, y, z, m, n, o) function call. The only modification that needs to be done in the recursive program is to store the return value of (m, n, o) state of the recursive function. The rest remains the same in the above recursive program.

Below is the implementation of the Memoization approach of the recursive code:

C++

// A memoize recursive implementation of LCS problem
#include <bits/stdc++.h>
int arr[100][100][100];
int max(int a, int b);
  
// Returns length of LCS for X[0..m-1], Y[0..n-1] */
// memoization applied in recursive solution
int lcs(char* X, char* Y, char* Z, int m, int n, int o)
{
    // base case
    if (m == 0 || n == 0 || o == 0)
        return 0;
  
    // if the same state has already been
    // computed
    if (arr[m - 1][n - 1][o - 1] != -1)
        return arr[m - 1][n - 1][o - 1];
  
    // if equal, then we store the value of the
    // fucntion call
    if (X[m - 1] == Y[n - 1] and Y[n - 1] == Z[o - 1]) {
  
        // store it in arr to avoid further repetitive work
        // in future function calls
        arr[m - 1][n - 1][o - 1] = 1 + lcs(X, Y, Z, m - 1,
                                            n - 1, o - 1);
        return arr[m - 1][n - 1][o - 1];
    }
    else {
  
        // store it in arr to avoid further repetitive work
        // in future function calls
        arr[m - 1][n - 1][o - 1] =
                               max(lcs(X, Y, Z, m, n - 1, o),
                                 max(lcs(X, Y, Z, m - 1, n, o),
                                    lcs(X, Y, Z, m, n, o - 1)));
        return arr[m - 1][n - 1][o - 1];
    }
}
  
// Utility function to get max of 2 integers
int max(int a, int b)
{
    return (a > b) ? a : b;
}
  
// Driver Code
int main()
{
    memset(arr, -1, sizeof(arr));
    char X[] = "geeks";
    char Y[] = "geeksfor";
    char Z[] = "geeksforgeeks";
    int m = strlen(X);
    int n = strlen(Y);
    int o = strlen(Z);
    printf("Length of LCS is %d", lcs(X, Y, Z, m, n, o));
  
    return 0;
}

Output:

Length of LCS is 5

Note: The array used to Memoize is initialized to some value (say -1) before the function call to mark if the function with the same parameters has been previously called or not.



My Personal Notes arrow_drop_up

Striver(underscore)79 at Codechef and codeforces D

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.



Improved By : jit_t, Shivi_Aggarwal