Longest Increasing Subsequence | DP-3

We have already discussed Overlapping Subproblems and Optimal Substructure properties.

Now, let us discuss the Longest Increasing Subsequence (LIS) problem as an example problem that can be solved using Dynamic Programming.
The Longest Increasing Subsequence (LIS) problem is to find the length of the longest subsequence of a given sequence such that all elements of the subsequence are sorted in increasing order. For example, the length of LIS for {10, 22, 9, 33, 21, 50, 41, 60, 80} is 6 and LIS is {10, 22, 33, 50, 60, 80}.
longest-increasing-subsequence

Examples:

Input: arr[] = {3, 10, 2, 1, 20}
Output: Length of LIS = 3
The longest increasing subsequence is 3, 10, 20

Input: arr[] = {3, 2}
Output: Length of LIS = 1
The longest increasing subsequences are {3} and {2}

Input: arr[] = {50, 3, 10, 7, 40, 80}
Output: Length of LIS = 4
The longest increasing subsequence is {3, 7, 40, 80}


Method 1: Recursion.

Optimal Substructure: Let arr[0..n-1] be the input array and L(i) be the length of the LIS ending at index i such that arr[i] is the last element of the LIS.



Then, L(i) can be recursively written as:

L(i) = 1 + max( L(j) ) where 0 < j < i and arr[j] < arr[i]; or
L(i) = 1, if no such j exists.

To find the LIS for a given array, we need to return max(L(i)) where 0 < i < n.

Formally, the length of the longest increasing subsequence ending at index i, will be 1 greater than the maximum of lengths of all longest increasing subsequences ending at indices before i, where arr[j] < arr[i] (j < i). Thus, we see the LIS problem satisfies the optimal substructure property as the main problem can be solved using solutions to subproblems. The recursive tree given below will make the approach clearer:

Input  : arr[] = {3, 10, 2, 11}
f(i): Denotes LIS of subarray ending at index ‘i’

(LIS(1)=1)

      f(4)  {f(4) = 1 + max(f(1), f(2), f(3))}
  /    |    \
f(1)  f(2)  f(3) {f(3) = 1, f(2) and f(1) are > f(3)}
       |      |  \
      f(1)  f(2)  f(1) {f(2) = 1 + max(f(1)}
              |
            f(1) {f(1) = 1}

Below is the implementation of the recursive approach:

C/C++

filter_none

edit
close

play_arrow

link
brightness_4
code

/* A Naive C/C++ recursive implementation 
   of LIS problem */
#include<stdio.h>
#include<stdlib.h>
  
/* To make use of recursive calls, this 
   function must return two things:
   1) Length of LIS ending with element arr[n-1]. 
      We use max_ending_here for this purpose
   2) Overall maximum as the LIS may end with 
      an element before arr[n-1] max_ref is 
      used this purpose.
   The value of LIS of full array of size n 
   is stored in *max_ref which is our final result 
*/
int _lis( int arr[], int n, int *max_ref)
{
    /* Base case */
    if (n == 1)
        return 1;
  
    // 'max_ending_here' is length of LIS 
    // ending with arr[n-1]
    int res, max_ending_here = 1; 
  
    /* Recursively get all LIS ending with arr[0], 
       arr[1] ... arr[n-2]. If arr[i-1] is smaller 
       than arr[n-1], and max ending with arr[n-1] 
       needs to be updated, then update it */
    for (int i = 1; i < n; i++)
    {
        res = _lis(arr, i, max_ref);
        if (arr[i-1] < arr[n-1] && res + 1 > max_ending_here)
            max_ending_here = res + 1;
    }
  
    // Compare max_ending_here with the overall 
    // max. And update the overall max if needed
    if (*max_ref < max_ending_here)
       *max_ref = max_ending_here;
  
    // Return length of LIS ending with arr[n-1]
    return max_ending_here;
}
  
// The wrapper function for _lis()
int lis(int arr[], int n)
{
    // The max variable holds the result
    int max = 1;
  
    // The function _lis() stores its result in max
    _lis( arr, n, &max );
  
    // returns max
    return max;
}
  
/* Driver program to test above function */
int main()
{
    int arr[] = { 10, 22, 9, 33, 21, 50, 41, 60 };
    int n = sizeof(arr)/sizeof(arr[0]);
    printf("Length of lis is %dn",
           lis( arr, n ));
    return 0;
}

chevron_right


Java

filter_none

edit
close

play_arrow

link
brightness_4
code

/* A Naive Java Program for LIS Implementation */
class LIS
{
   static int max_ref; // stores the LIS
  
   /* To make use of recursive calls, this function must return
   two things:
   1) Length of LIS ending with element arr[n-1]. We use
      max_ending_here for this purpose
   2) Overall maximum as the LIS may end with an element
      before arr[n-1] max_ref is used this purpose.
   The value of LIS of full array of size n is stored in
   *max_ref which is our final result */
   static int _lis(int arr[], int n)
   {
       // base case
       if (n == 1)
           return 1;
  
       // 'max_ending_here' is length of LIS ending with arr[n-1]
       int res, max_ending_here = 1;
  
        /* Recursively get all LIS ending with arr[0], arr[1] ...
           arr[n-2]. If   arr[i-1] is smaller than arr[n-1], and
           max ending with arr[n-1] needs to be updated, then
           update it */
        for (int i = 1; i < n; i++)
        {
            res = _lis(arr, i);
            if (arr[i-1] < arr[n-1] && res + 1 > max_ending_here)
                max_ending_here = res + 1;
        }
  
        // Compare max_ending_here with the overall max. And
        // update the overall max if needed
        if (max_ref < max_ending_here)
           max_ref = max_ending_here;
  
        // Return length of LIS ending with arr[n-1]
        return max_ending_here;
   }
  
    // The wrapper function for _lis()
    static int lis(int arr[], int n)
    {
        // The max variable holds the result
         max_ref = 1;
  
        // The function _lis() stores its result in max
        _lis( arr, n);
  
        // returns max
        return max_ref;
    }
  
    // driver program to test above functions
    public static void main(String args[])
    {
        int arr[] = { 10, 22, 9, 33, 21, 50, 41, 60 };
        int n = arr.length;
        System.out.println("Length of lis is "
                           + lis(arr, n) + "\n");
    }
 }
/*This code is contributed by Rajat Mishra*/

chevron_right


Python

filter_none

edit
close

play_arrow

link
brightness_4
code

# A naive Python implementation of LIS problem
  
""" To make use of recursive calls, this function must return
 two things:
 1) Length of LIS ending with element arr[n-1]. We use
 max_ending_here for this purpose
 2) Overall maximum as the LIS may end with an element
 before arr[n-1] max_ref is used this purpose.
 The value of LIS of full array of size n is stored in
 *max_ref which is our final result """
  
# global variable to store the maximum
global maximum
  
def _lis(arr , n ):
  
    # to allow the access of global variable
    global maximum
  
    # Base Case
    if n == 1 :
        return 1
  
    # maxEndingHere is the length of LIS ending with arr[n-1]
    maxEndingHere = 1
  
    """Recursively get all LIS ending with arr[0], arr[1]..arr[n-2]
       IF arr[n-1] is maller than arr[n-1], and max ending with
       arr[n-1] needs to be updated, then update it"""
    for i in xrange(1, n):
        res = _lis(arr , i)
        if arr[i-1] < arr[n-1] and res+1 > maxEndingHere:
            maxEndingHere = res +1
  
    # Compare maxEndingHere with overall maximum. And
    # update the overall maximum if needed
    maximum = max(maximum , maxEndingHere)
  
    return maxEndingHere
  
def lis(arr):
  
    # to allow the access of global variable
    global maximum
  
    # lenght of arr
    n = len(arr)
  
    # maximum variable holds the result
    maximum = 1
  
    # The function _lis() stores its result in maximum
    _lis(arr , n)
  
    return maximum
  
# Driver program to test the above function
arr = [10 , 22 , 9 , 33 , 21 , 50 , 41 , 60]
n = len(arr)
print "Length of lis is ", lis(arr)
  
# This code is contributed by NIKHIL KUMAR SINGH

chevron_right


C#

filter_none

edit
close

play_arrow

link
brightness_4
code

using System;
  
/* A Naive C# Program for LIS Implementation */
class LIS
{
   static int max_ref; // stores the LIS
   
   /* To make use of recursive calls, this function must return
   two things:
   1) Length of LIS ending with element arr[n-1]. We use
      max_ending_here for this purpose
   2) Overall maximum as the LIS may end with an element
      before arr[n-1] max_ref is used this purpose.
   The value of LIS of full array of size n is stored in
   *max_ref which is our final result */
   static int _lis(int[] arr, int n)
   {
       // base case
       if (n == 1)
           return 1;
   
       // 'max_ending_here' is length of LIS ending with arr[n-1]
       int res, max_ending_here = 1;
   
        /* Recursively get all LIS ending with arr[0], arr[1] ...
           arr[n-2]. If   arr[i-1] is smaller than arr[n-1], and
           max ending with arr[n-1] needs to be updated, then
           update it */
        for (int i = 1; i < n; i++)
        {
            res = _lis(arr, i);
            if (arr[i-1] < arr[n-1] && res + 1 > max_ending_here)
                max_ending_here = res + 1;
        }
   
        // Compare max_ending_here with the overall max. And
        // update the overall max if needed
        if (max_ref < max_ending_here)
           max_ref = max_ending_here;
   
        // Return length of LIS ending with arr[n-1]
        return max_ending_here;
   }
   
    // The wrapper function for _lis()
    static int lis(int[] arr, int n)
    {
        // The max variable holds the result
         max_ref = 1;
   
        // The function _lis() stores its result in max
        _lis( arr, n);
   
        // returns max
        return max_ref;
    }
   
    // driver program to test above functions
    public static void Main()
    {
        int[] arr = { 10, 22, 9, 33, 21, 50, 41, 60 };
        int n = arr.Length;
        Console.Write("Length of lis is "
                           + lis(arr, n) + "\n");
    }
 }

chevron_right



Output:

Length of lis is 5

Complexity Analysis:

  • Time Complexity: The time complexity of this recursive approach is exponential as there is a case of overlapping subproblems as explained in the recursive tree diagram above.
  • Auxiliary Space: O(1). No external space used for storing values apart from the internal stack space.

Method 2: Dynamic Programming.

We can see that there are many subproblems in the above recursive solution which are solved again and again. So this problem has Overlapping Substructure property and recomputation of same subproblems can be avoided by either using Memoization or Tabulation.

The simulation of approach will make things clear:



Input  : arr[] = {3, 10, 2, 11}
LIS[] = {1, 1, 1, 1} (initially)

Iteration-wise simulation :

  1. arr[2] > arr[1] {LIS[2] = max(LIS [2], LIS[1]+1)=2}
  2. arr[3] < arr[1] {No change}
  3. arr[3] < arr[2] {No change}
  4. arr[4] > arr[1] {LIS[4] = max(LIS [4], LIS[1]+1)=2}
  5. arr[4] > arr[2] {LIS[4] = max(LIS [4], LIS[2]+1)=3}
  6. arr[4] > arr[3] {LIS[4] = max(LIS [4], LIS[3]+1)=3}

We can avoid recomputation of subproblems by using tabulation as shown in the below code:
Below is the implementation of the above approach:

C++

filter_none

edit
close

play_arrow

link
brightness_4
code

/* Dynamic Programming C++ implementation 
   of LIS problem */
#include<bits/stdc++.h> 
using namespace std;
    
/* lis() returns the length of the longest  
  increasing subsequence in arr[] of size n */
int lis( int arr[], int n ) 
    int lis[n];
   
    lis[0] = 1;   
  
    /* Compute optimized LIS values in 
       bottom up manner */
    for (int i = 1; i < n; i++ ) 
    {
        lis[i] = 1;
        for (int j = 0; j < i; j++ )  
            if ( arr[i] > arr[j] && lis[i] < lis[j] + 1) 
                lis[i] = lis[j] + 1; 
    }
  
    // Return maximum value in lis[]
    return *max_element(lis, lis+n);
    
/* Driver program to test above function */
int main() 
    int arr[] = { 10, 22, 9, 33, 21, 50, 41, 60 }; 
    int n = sizeof(arr)/sizeof(arr[0]); 
    printf("Length of lis is %d\n", lis( arr, n ) ); 
  
    return 0; 
}

chevron_right


Java

filter_none

edit
close

play_arrow

link
brightness_4
code

/* Dynamic Programming Java implementation 
   of LIS problem */
  
class LIS
{
    /* lis() returns the length of the longest 
       increasing subsequence in arr[] of size n */
    static int lis(int arr[],int n)
    {
          int lis[] = new int[n];
          int i,j,max = 0;
  
          /* Initialize LIS values for all indexes */
           for ( i = 0; i < n; i++ )
              lis[i] = 1;
  
           /* Compute optimized LIS values in 
              bottom up manner */
           for ( i = 1; i < n; i++ )
              for ( j = 0; j < i; j++ ) 
                         if ( arr[i] > arr[j] && 
                                  lis[i] < lis[j] + 1)
                    lis[i] = lis[j] + 1;
  
           /* Pick maximum of all LIS values */
           for ( i = 0; i < n; i++ )
              if ( max < lis[i] )
                 max = lis[i];
  
            return max;
    }
  
    public static void main(String args[])
    {
        int arr[] = { 10, 22, 9, 33, 21, 50, 41, 60 };
            int n = arr.length;
            System.out.println("Length of lis is " 
                              + lis( arr, n ) + "\n" );
    }
}
/*This code is contributed by Rajat Mishra*/

chevron_right


Python

filter_none

edit
close

play_arrow

link
brightness_4
code

# Dynamic programming Python implementation
# of LIS problem
  
# lis returns length of the longest 
# increasing subsequence in arr of size n
def lis(arr):
    n = len(arr)
  
    # Declare the list (array) for LIS and 
    # initialize LIS values for all indexes
    lis = [1]*n
  
    # Compute optimized LIS values in bottom up manner
    for i in range (1 , n):
        for j in range(0 , i):
            if arr[i] > arr[j] and lis[i]< lis[j] + 1 :
                lis[i] = lis[j]+1
  
    # Initialize maximum to 0 to get 
    # the maximum of all LIS
    maximum = 0
  
    # Pick maximum of all LIS values
    for i in range(n):
        maximum = max(maximum , lis[i])
  
    return maximum
# end of lis function
  
# Driver program to test above function
arr = [10, 22, 9, 33, 21, 50, 41, 60]
print "Length of lis is", lis(arr)
# This code is contributed by Nikhil Kumar Singh

chevron_right


C#

filter_none

edit
close

play_arrow

link
brightness_4
code

/* Dynamic Programming C# implementation of LIS problem */
  
using System ;
class LIS 
    /* lis() returns the length of the longest increasing 
    subsequence in arr[] of size n */
    static int lis(int []arr,int n) 
    
        int []lis = new int[n]; 
        int i,j,max = 0; 
  
        /* Initialize LIS values for all indexes */
        for ( i = 0; i < n; i++ ) 
            lis[i] = 1; 
  
        /* Compute optimized LIS values in bottom up manner */
        for ( i = 1; i < n; i++ ) 
            for ( j = 0; j < i; j++ ) 
                        if ( arr[i] > arr[j] && lis[i] < lis[j] + 1) 
                    lis[i] = lis[j] + 1; 
  
        /* Pick maximum of all LIS values */
        for ( i = 0; i < n; i++ ) 
            if ( max < lis[i] ) 
                max = lis[i]; 
  
            return max; 
    
  
    public static void Main() 
    
        int []arr = { 10, 22, 9, 33, 21, 50, 41, 60 }; 
            int n = arr.Length; 
            Console.WriteLine("Length of lis is " + lis( arr, n ) + "\n" ); 
    
  
    // This code is contributed by Ryuga
}

chevron_right



Output:

Length of lis is 5

Complexity Analysis:

  • Time Complexity: O(n2).
    As nested loop is used.
  • Auxiliary Space: O(n).
    Use of any array to store LIS values at each index.

Note: The time complexity of the above Dynamic Programming (DP) solution is O(n^2) and there is a O(N log N) solution for the LIS problem. We have not discussed the O(N log N) solution here as the purpose of this post is to explain Dynamic Programming with a simple example. See below post for O(N log N) solution.
Longest Increasing Subsequence Size (N log N)

Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.

Don’t stop now and take your learning to the next level. Learn all the important concepts of Data Structures and Algorithms with the help of the most trusted course: DSA Self Paced. Become industry ready at a student-friendly price.




My Personal Notes arrow_drop_up