# Bayes’ Theorem

**Bayes’ Theorem or Bayes’ Rule** is named after Reverend **Thomas Bayes**. It describes the probability of an event, based on prior knowledge of conditions that might be related to that event. It can also be considered for conditional probability examples. **For example:** There are 3 bags, each containing some white marbles and some black marbles in each bag. If a white marble is drawn at random. With probability to find that this white marble is from the first bag. In cases like such, we use the Bayes’ Theorem. It is used where the probability of occurrence of a particular event is calculated based on other conditions which are also called conditional probability. So before jumping into detail let’s have a brief discussion on **The theorem of Total Probability**.** **

**Theorem of Total Probability**

Let E_{1}, E_{2},…………..E_{n} is mutually exclusive and exhaustive events associated with a random experiment and lets E be an event that occurs with some E_{i}. Then, prove that

P(E) =^{n}∑_{i=1}P(E/E_{i}) . P(E_{j})

**Proof:**

Let S be the sample space. Then,

S = E

_{1}∪ E_{2}∪ E_{3}∪………………… ∪ En and E_{i}∩ E_{j}= ∅ for i ≠ j.Therefore, E = E∩S = E ∩ (E

_{1}∪ E_{2}∪ E_{3}∪………………… ∪ E_{n})= (E ∩ E

_{1}) ∪ (E ∩ E_{2}) ∪ ……∪ (E ∩ E_{n})=> P(E) = P{(E ∩ E

_{1}) ∪ (E ∩ E_{2})∪……∪(E ∩ E_{n})}= P(E ∩ E

_{1}) + P(E ∩ E_{2}) + …… + P(E ∩ E_{n})= {Therefore, (E ∩ E

_{1}), (E ∩ E_{2}),………….,(E ∩ E_{n})} are pairwise disjoint}= P(E/E

_{1}) . P(E_{1}) + P(E/E_{2}) . P(E_{2}) +……………………+ P(E/E_{n}) . P(E_{n}) [by multiplication theorem]=

^{n}∑_{i=1}P(E/E_{i}) . P(E_{i})

### Examples

**Example 1: A person has undertaken a job. The probabilities of completion of the job on time with and without rain are 0.44 and 0.95 respectively. If the probability that it will rain is 0.45, then determine the probability that the job will be completed on time?**

**Solution:**

Let E1 be the event that the mining job will be completed on time and E2 be the event that it rains. We have,

P(A) = 0.45,

P(no rain) = P(B) = 1 − P(A) = 1 − 0.45 = 0.55

By multiplication law of probability,

P(E1) = 0.44

P(E2) = 0.95

Since, events A and B form partitions of the sample space S, by total probability theorem, we have

P(E) = P(A) P(E1) + P(B) P(E2)

= 0.45 × 0.44 + 0.55 × 0.95

= 0.198 + 0.5225 = 0.7205

So, the probability that the job will be completed on time is 0.684.

**Example 2: There are three urns containing 3 white and 2 black balls; 2 white and 3 black balls; 1 black and 4 white balls respectively. There is an equal probability of each urn being chosen. One ball is equal probability chosen at random. what is the probability that a white ball is drawn?**

**Solution:**

Let E1, E2, and E3 be the events of choosing the first, second, and third urn respectively. Then,

P(E1) = P(E2) = P(E3) =1/3

Let E be the event that a white ball is drawn. Then,

P(E/E1) = 3/5, P(E/E2) = 2/5, P(E/E3) = 4/5

By theorem of total probability, we have

P(E) = P(E/E1) . P(E1) + P(E/E2) . P(E2) + P(E/E3) . P(E3)

= (3/5 * 1/3) + (2/5 * 1/3) + (4/5 * 1/3)

= 9/15 = 3/5

**Bayes’ Theorem Statement**

Let E_{1}, E_{2}, E_{3},………, E_{n} be mutually exclusive and exhaustive events associated with a random experiment, and let E be an event that occurs with some E_{i}. Then,

P(E|E_{i}) = P(E|E_{i}) . P(E_{i})/^{n}∑_{i=1}P(E|E_{i}) . P(E_{i})

Where i = 1, 2, 3, 4…….., n

**Proof:**

By the theorem of total probability, we have

P(E) =

^{n}∑_{i=1}P(E|E_{i}) . P(E_{i}) …………………………….(i)Therefore, P(E|E

_{i}) = P(E ∩ E_{i})/P(E) [by multiplication theorem]= P(E|E

_{i}) . P(E_{i})/P(E) [Therefore, P(E/E_{i}) = P(E ∩ E_{i})/P(E_{i})]= P(E|E

_{i}) . P(E_{i})/^{n}∑_{i=1}P(E|E_{i}) . P(E_{i}) [Using (i)]Hence, P(E|E

_{i}) = P(E|E_{i}) . P(E_{i})/^{n}∑_{i=1}P(E|E_{i}) . P(E_{i})

**Bayes’ Theorem Formula**

Let’s consider E and Ei as two events so the formula for Bayes’ theorem is:

P(E_{i}|E) = P(E ∩ E_{i})/P(E)Here,

P(Eis in conditional probability when event E_{i}|E)_{i }occurs before event E.

P(E ∩ Eis the probability of event E and event E_{i})_{i}.

P(E)as the Probability of E.

**Bayes’ Theorem Derivation**

As we know Bayes Theorem can be derived from events and random variables separately with the help of conditional probability and density. As per conditional probability, we assume that there are two events T and Q associated with the same rab = ndom experiment. Then, the probability of occurrence of T under the condition that Q has already occurred and P(Q)≠0 is called conditional probability, denoted by P(T/Q). So we ultimately define it as

P(T|Q) = P(T ⋂ Q)/P(Q), where P(Q) ≠ 0

P(Q|T) = P(Q ⋂ T)/P(T), where P(T) ≠ 0

Likewise, If the conditional distribution of J given G is in a continuous distribution, then its probability of density function is known as the conditional density function. Bayes theorem can derive these two continuous random variables namely F and G as given below:

ƒ J|G = g(j) = ƒ J,G

_{(j,g)}/ƒ G(g)ƒ J|G = j(g) = ƒ J,G

_{(j,g)}/ƒ J(j)ƒ J|G = g(j) = ƒ

_{G|J=j(g)}ƒJ(j) ƒG(g)

**Bayes’ Theorem Examples**

**Example 1: A card from a pack of 52 cards is lost. From the remaining cards of the pack, two cards are drawn and are found to be both hearts. find the probability of the lost card being a heart?**

**Solution:**

Let E

_{1}, E_{2}, E_{3,}and E_{4}be the events of losing a card of hearts, clubs, spades, and diamonds respectively.Then P(E

_{1}) = P(E_{2}) = P(E_{3}) = P(E_{4}) = 13/52 = 1/4.Let E be the event of drawing 2 hearts from the remaining 51 cards. Then,

P(E|E

_{1}) = probability of drawing 2 hearts, given that a card of hearts is missing=

^{12}C_{2}/^{51}C_{2}= (12 * 11)/2! * 2!/(51 * 50) = 22/425P(E|E

_{2}) = probability of drawing 2 clubs ,given that a card of clubs is missing=

^{13}C_{2}/^{51}C_{2}= (13 * 12)/2! * 2!/(51 * 50) = 26/425P(E|E

_{3}) = probability of drawing 2 spades ,given that a card of hearts is missing=

^{13}C_{2}/^{51}C_{2}= 26/425P(E|E

_{4}) = probability of drawing 2 diamonds ,given that a card of diamonds is missing=

^{13}C_{2}/^{51}C_{2}= 26/425Therefore,

P(E

_{1}|E) = probability of the lost card is being a heart, given the 2 hearts are drawn from the remaining 51 cards= P(E

_{1}) . P(E|E_{1})/P(E_{1}) . P(E|E_{1}) + P(E_{2}) . P(E|E_{2}) + P(E_{3}) . P(E|E_{3}) + P(E_{4}) . P(E|E_{4})= (1/4 * 22/425) / {(1/4 * 22/425) + (1/4 * 26/425) + (1/4 * 26/425) + (1/4 * 26/425)}

= 22/100 = 0.22

Hence, The required probability is 0.22.

**Example 2: Suppose 15 men out of 300 men and 25 women out of 1000 are good orators. An orator is chosen at random. Find the probability that a male person is selected. Assume that there are equal numbers of men and women?**

**Solution:**

Let there be 1000 men and 1000 women.

Let E

_{1}and E_{2}be the events of choosing a man and a woman respectively. Then,P(E

_{1}) = 1000/2000 = 1/2 , and P(E_{2}) = 1000/2000 = 1/2Let E be the event of choosing an orator. Then,

P(E|E

_{1}) = 50/1000 = 1/20, and P(E|E_{2}) = 25/1000 = 1/40Probability of selecting a male person ,given that the person selected is a good orator

P(E

_{1}/E) = P(E|E_{1}) * P(E_{1})/ P(E|E_{1}) * P(E_{1}) + P(E|E_{2}) * P(E_{2})= (1/2 * 1/20) /{(1/2 * 1/20) + (1/2 * 1/40)}

= 2/3

Hence the required probability is 2/3.

**Example 3: A man is known to speak the lies 1 out of 4 times. He throws a die and reports that it is a six. Find the probability that is actually a six?**

**Solution: **

In a throw of a die, let

E

_{1 }= event of getting a six,E

_{2}= event of not getting a six andE = event that the man reports that it is a six.

Then, P(E

_{1}) = 1/6, and P(E_{2}) = (1 – 1/6) = 5/6P(E|E

_{1}) = probability that the man reports that six occurs when six has actually occurred= probability that the man speaks the truth

= 3/4

P(E|E

_{2}) = probability that the man reports that six occurs when six has not actually occurred= probability that the man does not speak the truth

= (1 – 3/4) = 1/4

Probability of getting a six ,given that the man reports it to be six

P(E

_{1}|E) = P(E|E_{1}) * P(E_{1})/P(E|E_{1}) * P(E_{1}) + P(E|E_{2}) * P(E_{2}) [by Bayes’ theorem]= (3/4 * 1/6)/{(3/4 * 1/6) + (1/4 * 5/6)}

= (1/8 * 3) = 3/8

Hence the probability required is 3/8.