Given a Markov chain G, we have the find the probability of reaching the state F at time t = T if we start from state S at time t = 0.

A Markov chain is a random process consisting of various states and the probabilities of moving from one state to another. We can represent it using a directed graph where the nodes represent the states and the edges represent the probability of going from one node to another. It takes unit time to move from one node to another. The sum of the associated probabilities of the outgoing edges is one for every node.

Consider the given Markov Chain( G ) as shown in below image:

**Examples**:

Input: S = 1, F = 2, T = 1 Output : 0.23 We start at state 1 at t = 0, so there is a probability of 0.23 that we reach state 2 at t = 1. Input : S = 4, F = 2, T = 100 Output : 0.284992

We can use **dynamic programming** and **depth-first search (DFS)** to solve this problem, by taking the state and the time as the two DP variables. We can easily observe that the probability of going from state A to state B at time is equal to the product of the probability of being at A at time and the probability associated with the edge connecting A and B. Therefore the probability of being at B at time is the sum of this quantity for all A adjacent to B.

Below is the implementation of the above approach:

## C++

`// C++ implementation of the above approach ` `#include <bits/stdc++.h> ` `using` `namespace` `std; ` ` ` `// Macro for vector of pair to store ` `// each node with edge ` `#define vp vector<pair<int, float> > ` ` ` `// Function to calculate the ` `// probability of reaching F ` `// at time T after starting ` `// from S ` `float` `findProbability(vector<vp>& G, ` `int` `N, ` ` ` `int` `F, ` `int` `S, ` `int` `T) ` `{ ` ` ` `// Declaring the DP table ` ` ` `vector<vector<` `float` `> > P(N + 1, vector<` `float` `>(T + 1, 0)); ` ` ` ` ` `// Probability of being at S ` ` ` `// at t = 0 is 1.0 ` ` ` `P[S][0] = 1.0; ` ` ` ` ` `// Filling the DP table ` ` ` `// in bottom-up manner ` ` ` `for` `(` `int` `i = 1; i <= T; ++i) ` ` ` `for` `(` `int` `j = 1; j <= N; ++j) ` ` ` `for` `(` `auto` `k : G[j]) ` ` ` `P[j][i] += k.second * P[k.first][i - 1]; ` ` ` ` ` `return` `P[F][T]; ` `} ` ` ` `// Driver code ` `int` `main() ` `{ ` ` ` `// Adjacency list ` ` ` `vector<vp> G(7); ` ` ` ` ` `// Building the graph ` ` ` `// The edges have been stored in the row ` ` ` `// corresponding to their end-point ` ` ` `G[1] = vp({ { 2, 0.09 } }); ` ` ` `G[2] = vp({ { 1, 0.23 }, { 6, 0.62 } }); ` ` ` `G[3] = vp({ { 2, 0.06 } }); ` ` ` `G[4] = vp({ { 1, 0.77 }, { 3, 0.63 } }); ` ` ` `G[5] = vp({ { 4, 0.65 }, { 6, 0.38 } }); ` ` ` `G[6] = vp({ { 2, 0.85 }, { 3, 0.37 }, { 4, 0.35 }, { 5, 1.0 } }); ` ` ` ` ` `// N is the number of states ` ` ` `int` `N = 6; ` ` ` ` ` `int` `S = 4, F = 2, T = 100; ` ` ` ` ` `cout << ` `"The probability of reaching "` `<< F ` ` ` `<< ` `" at time "` `<< T << ` `" \nafter starting from "` ` ` `<< S << ` `" is "` `<< findProbability(G, N, F, S, T); ` ` ` ` ` `return` `0; ` `} ` |

*chevron_right*

*filter_none*

## Java

`// Java implementation of the above approach ` `import` `java.util.*; ` ` ` `class` `GFG ` `{ ` ` ` `static` `class` `pair ` ` ` `{ ` ` ` `int` `first; ` ` ` `double` `second; ` ` ` ` ` `public` `pair(` `int` `first, ` `double` `second) ` ` ` `{ ` ` ` `this` `.first = first; ` ` ` `this` `.second = second; ` ` ` `} ` ` ` `} ` ` ` ` ` `// Function to calculate the ` ` ` `// probability of reaching F ` ` ` `// at time T after starting ` ` ` `// from S ` ` ` `static` `float` `findProbability(Vector<pair>[] G, ` ` ` `int` `N, ` `int` `F, ` `int` `S, ` `int` `T) ` ` ` `{ ` ` ` `// Declaring the DP table ` ` ` `float` `[][] P = ` `new` `float` `[N + ` `1` `][T + ` `1` `]; ` ` ` ` ` `// Probability of being at S ` ` ` `// at t = 0 is 1.0 ` ` ` `P[S][` `0` `] = (` `float` `) ` `1.0` `; ` ` ` ` ` `// Filling the DP table ` ` ` `// in bottom-up manner ` ` ` `for` `(` `int` `i = ` `1` `; i <= T; ++i) ` ` ` `for` `(` `int` `j = ` `1` `; j <= N; ++j) ` ` ` `for` `(pair k : G[j]) ` ` ` `P[j][i] += k.second * P[k.first][i - ` `1` `]; ` ` ` ` ` `return` `P[F][T]; ` ` ` `} ` ` ` ` ` `// Driver code ` ` ` `public` `static` `void` `main(String[] args) ` ` ` `{ ` ` ` `// Adjacency list ` ` ` `Vector<pair>[] G = ` `new` `Vector[` `7` `]; ` ` ` `for` `(` `int` `i = ` `0` `; i < ` `7` `; i++) ` ` ` `{ ` ` ` `G[i] = ` `new` `Vector<pair>(); ` ` ` `} ` ` ` ` ` `// Building the graph ` ` ` `// The edges have been stored in the row ` ` ` `// corresponding to their end-point ` ` ` `G[` `1` `].add(` `new` `pair(` `2` `, ` `0.09` `)); ` ` ` `G[` `2` `].add(` `new` `pair(` `1` `, ` `0.23` `)); ` ` ` `G[` `2` `].add(` `new` `pair(` `6` `, ` `0.62` `)); ` ` ` `G[` `3` `].add(` `new` `pair(` `2` `, ` `0.06` `)); ` ` ` `G[` `4` `].add(` `new` `pair(` `1` `, ` `0.77` `)); ` ` ` `G[` `4` `].add(` `new` `pair(` `3` `, ` `0.63` `)); ` ` ` `G[` `5` `].add(` `new` `pair(` `4` `, ` `0.65` `)); ` ` ` `G[` `5` `].add(` `new` `pair(` `6` `, ` `0.38` `)); ` ` ` `G[` `6` `].add(` `new` `pair(` `2` `, ` `0.85` `)); ` ` ` `G[` `6` `].add(` `new` `pair(` `3` `, ` `0.37` `)); ` ` ` `G[` `6` `].add(` `new` `pair(` `4` `, ` `0.35` `)); ` ` ` `G[` `6` `].add(` `new` `pair(` `5` `, ` `1.0` `)); ` ` ` ` ` `// N is the number of states ` ` ` `int` `N = ` `6` `; ` ` ` ` ` `int` `S = ` `4` `, F = ` `2` `, T = ` `100` `; ` ` ` ` ` `System.out.print(` `"The probability of reaching "` `+ F + ` ` ` `" at time "` `+ T + ` `" \nafter starting from "` `+ ` ` ` `S + ` `" is "` ` ` `+ findProbability(G, N, F, S, T)); ` ` ` `} ` `} ` ` ` `// This code is contributed by Rajput-Ji ` |

*chevron_right*

*filter_none*

## C#

`// C# implementation of the above approach ` `using` `System; ` `using` `System.Collections.Generic; ` ` ` `class` `GFG ` `{ ` ` ` `class` `pair ` ` ` `{ ` ` ` `public` `int` `first; ` ` ` `public` `double` `second; ` ` ` ` ` `public` `pair(` `int` `first, ` `double` `second) ` ` ` `{ ` ` ` `this` `.first = first; ` ` ` `this` `.second = second; ` ` ` `} ` ` ` `} ` ` ` ` ` `// Function to calculate the ` ` ` `// probability of reaching F ` ` ` `// at time T after starting ` ` ` `// from S ` ` ` `static` `float` `findProbability(List<pair>[] G, ` ` ` `int` `N, ` `int` `F, ` `int` `S, ` `int` `T) ` ` ` `{ ` ` ` `// Declaring the DP table ` ` ` `float` `[,] P = ` `new` `float` `[N + 1, T + 1]; ` ` ` ` ` `// Probability of being at S ` ` ` `// at t = 0 is 1.0 ` ` ` `P[S, 0] = (` `float` `) 1.0; ` ` ` ` ` `// Filling the DP table ` ` ` `// in bottom-up manner ` ` ` `for` `(` `int` `i = 1; i <= T; ++i) ` ` ` `for` `(` `int` `j = 1; j <= N; ++j) ` ` ` `foreach` `(pair k ` `in` `G[j]) ` ` ` `P[j, i] += (` `float` `)k.second * ` ` ` `P[k.first, i - 1]; ` ` ` ` ` `return` `P[F, T]; ` ` ` `} ` ` ` ` ` `// Driver code ` ` ` `public` `static` `void` `Main(String[] args) ` ` ` `{ ` ` ` `// Adjacency list ` ` ` `List<pair>[] G = ` `new` `List<pair>[7]; ` ` ` `for` `(` `int` `i = 0; i < 7; i++) ` ` ` `{ ` ` ` `G[i] = ` `new` `List<pair>(); ` ` ` `} ` ` ` ` ` `// Building the graph ` ` ` `// The edges have been stored in the row ` ` ` `// corresponding to their end-point ` ` ` `G[1].Add(` `new` `pair(2, 0.09)); ` ` ` `G[2].Add(` `new` `pair(1, 0.23)); ` ` ` `G[2].Add(` `new` `pair(6, 0.62)); ` ` ` `G[3].Add(` `new` `pair(2, 0.06)); ` ` ` `G[4].Add(` `new` `pair(1, 0.77)); ` ` ` `G[4].Add(` `new` `pair(3, 0.63)); ` ` ` `G[5].Add(` `new` `pair(4, 0.65)); ` ` ` `G[5].Add(` `new` `pair(6, 0.38)); ` ` ` `G[6].Add(` `new` `pair(2, 0.85)); ` ` ` `G[6].Add(` `new` `pair(3, 0.37)); ` ` ` `G[6].Add(` `new` `pair(4, 0.35)); ` ` ` `G[6].Add(` `new` `pair(5, 1.0)); ` ` ` ` ` `// N is the number of states ` ` ` `int` `N = 6; ` ` ` ` ` `int` `S = 4, F = 2, T = 100; ` ` ` ` ` `Console.Write(` `"The probability of reaching "` `+ F + ` ` ` `" at time "` `+ T + ` `" \nafter starting from "` `+ ` ` ` `S + ` `" is "` ` ` `+ findProbability(G, N, F, S, T)); ` ` ` `} ` `} ` ` ` `// This code is contributed by 29AjayKumar ` |

*chevron_right*

*filter_none*

**Output:**

The probability of reaching 2 at time 100 after starting from 4 is 0.284992

**Time complexity**: O(N^{2} * T)

**Space complexity**: O(N * T)

Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the **DSA Self Paced Course** at a student-friendly price and become industry ready.

## Recommended Posts:

- Finding the probability of a state at a given time in a Markov chain | Set 2
- Minimum time to return array to its original state after given modifications
- Word Ladder (Length of shortest chain to reach a target word)
- Difference between Distance vector routing and Link State routing
- Three-State Bus Buffers
- A matrix probability question
- Maximizing Probability of one type from N containers
- Find minimum value of y for the given x values in Q queries from all the given set of lines
- Find time taken for signal to reach all positions in a string
- Find time taken to execute the tasks in A based on the order of execution in B
- MakeMyTrip Interview Experience | Set 13 (On-Campus for Full Time)
- Minimum time taken by each job to be completed given by a Directed Acyclic Graph
- Disjoint Set (Or Union-Find) | Set 1 (Detect Cycle in an Undirected Graph)
- Find the number of Islands | Set 2 (Using Disjoint Set)
- Find all reachable nodes from every node present in a given set
- Find if an undirected graph contains an independent set of a given size
- Find integral points with minimum distance from given set of integers using BFS
- Minimum value possible of a given function from the given set
- Check if the given array can be constructed uniquely from the given set of subsequences
- Minimum time required to rot all oranges

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.