Find the probability of a state at a given time in a Markov chain | Set 1

Given a Markov chain G, we have the find the probability of reaching the state F at time t = T if we start from state S at time t = 0.

A Markov chain is a random process consisting of various states and the probabilities of moving from one state to another. We can represent it using a directed graph where the nodes represent the states and the edges represent the probability of going from one node to another. It takes unit time to move from one node to another. The sum of the associated probabilities of the outgoing edges is one for every node.

Consider the given Markov Chain( G ) as shown in below image:

Examples:

Input : S = 1, F = 2, T = 1
Output : 0.23
We start at state 1 at t = 0, 
so there is a probability of 0.23 
that we reach state 2 at t = 1.

Input : S = 4, F = 2, T = 100
Output : 0.284992


We can use dynamic programming and depth-first search (DFS) to solve this problem, by taking the state and the time as the two DP variables. We can easily observe that the probability of going from state A to state B at time t is equal to the product of the probability of being at A at time t - 1 and the probability associated with the edge connecting A and B. Therefore the probability of being at B at time t is the sum of this quantity for all A adjacent to B.

Below is the implementation of the above approach:

filter_none

edit
close

play_arrow

link
brightness_4
code

// C++ implementation of the above approach
#include <bits/stdc++.h>
using namespace std;
  
// Macro for vector of pair to store
// each node with edge
#define vp vector<pair<int, float> >
  
// Function to calculate the
// probability of reaching F
// at time T after starting
// from S
float findProbability(vector<vp>& G, int N,
                      int F, int S, int T)
{
    // Declaring the DP table
    vector<vector<float> > P(N + 1, vector<float>(T + 1, 0));
  
    // Probability of being at S
    // at t = 0 is 1.0
    P[S][0] = 1.0;
  
    // Filling the DP table
    // in bottom-up manner
    for (int i = 1; i <= T; ++i)
        for (int j = 1; j <= N; ++j)
            for (auto k : G[j])
                P[j][i] += k.second * P[k.first][i - 1];
  
    return P[F][T];
}
  
// Driver code
int main()
{
    // Adjacency list
    vector<vp> G(7);
  
    // Building the graph
    // The edges have been stored in the row
    // corresponding to their end-point
    G[1] = vp({ { 2, 0.09 } });
    G[2] = vp({ { 1, 0.23 }, { 6, 0.62 } });
    G[3] = vp({ { 2, 0.06 } });
    G[4] = vp({ { 1, 0.77 }, { 3, 0.63 } });
    G[5] = vp({ { 4, 0.65 }, { 6, 0.38 } });
    G[6] = vp({ { 2, 0.85 }, { 3, 0.37 }, { 4, 0.35 }, { 5, 1.0 } });
  
    // N is the number of states
    int N = 6;
  
    int S = 4, F = 2, T = 100;
  
    cout << "The probability of reaching " << F
         << " at time " << T << " \nafter starting from "
         << S << " is " << findProbability(G, N, F, S, T);
  
    return 0;
}

chevron_right


Output:

The probability of reaching 2 at time 100 
after starting from 4 is 0.284992

Time complexity: O(N2 * T)
Space complexity: O(N * T)



My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.