Open In App

Bayes’ Theorem

Improve
Improve
Improve
Like Article
Like
Save Article
Save
Share
Report issue
Report

Bayes’ Theorem is used to determine the conditional probability of an event. It was named after an English statistician, Thomas Bayes who discovered this formula in 1763.

Bayes Theorem is a very important theorem in mathematics, that laid the foundation of a unique statistical inference approach called the Bayes’ inference.

It is used to find the probability of an event, based on prior knowledge of conditions that might be related to that event. For example, if we want to find the probability that a white marble drawn at random came from the first bag, given that a white marble has already been drawn, and there are three bags each containing some white and black marbles, then we can use Bayes’ Theorem.

This article will explore the Bayes theorem. We will present the statement, proof, derivation, and formula of the theorem, as well as illustrate its applications with various examples.

What is Bayes’ Theorem?

Bayes theorem (also known as the Bayes Rule or Bayes Law) is used to determine the conditional probability of event A when event B has already occurred.

The general statement of Bayes’ theorem is “The conditional probability of an event A, given the occurrence of another event B, is equal to the product of the event of B, given A and the probability of A divided by the probability of event B.” i.e.

P(A|B) = P(B|A)P(A) / P(B)

where,

  • P(A) and P(B) are the probabilities of events A and B
  • P(A|B) is the probability of event A when event B happens
  • P(B|A) is the probability of event B when A happens

Bayes Theorem Statement

Bayes’ Theorem for n set of events is defined as,

Let E1, E2,…, En be a set of events associated with the sample space S, in which all the events E1, E2,…, En have a non-zero probability of occurrence. All the events E1, E2,…, E form a partition of S. Let A be an event from space S for which we have to find probability, then according to Bayes’ theorem,

P(Ei|A) = P(Ei)P(A|Ei) / ∑ P(Ek)P(A|Ek)

for k = 1, 2, 3, …., n

Bayes Theorem Formula

For any two events A and B, then the formula for the Bayes theorem is given by: (the image given below gives the Bayes’ theorem formula)

bayes theorem formula

Bayes’ Theorem Formula

where,

  • P(A) and P(B) are the probabilities of events A and B also P(B) is never equal to zero.
  • P(A|B) is the probability of event A when event B happens
  • P(B|A) is the probability of event B when A happens

Bayes Theorem Derivation

The proof of Bayes’ Theorem is given as, according to the conditional probability formula,

P(Ei|A) = P(Ei∩A) / P(A)…..(i)

Then, by using the multiplication rule of probability, we get

P(Ei∩A) = P(Ei)P(A|Ei)……(ii)

Now, by the total probability theorem,

P(A) = ∑ P(Ek)P(A|Ek)…..(iii)

Substituting the value of P(Ei∩A) and P(A) from eq (ii) and eq(iii) in eq(i) we get,

P(Ei|A) = P(Ei)P(A|Ei) / ∑ P(Ek)P(A|Ek)

Bayes’ theorem is also known as the formula for the probability of “causes”. As we know, the Ei‘s are a partition of the sample space S, and at any given time only one of the events Ei occurs. Thus we conclude that the Bayes’ theorem formula gives the probability of a particular Ei, given the event A has occurred.

Various terms used in the Bayes theorem are explained below in this article.

After learning about Bayes theorem in detail, let us understand some important terms related to the concepts we covered in formula and derivation.

Hypotheses

Events happening in the sample space E1, E2,… En is called the hypotheses

Priori Probability 

Priori Probability is the initial probability of an event occurring before any new data is taken into account. P(Ei) is the priori probability of hypothesis Ei.

Posterior Probability 

Posterior Probability is the updated probability of an event after considering new information. Probability P(Ei|A) is considered as the posterior probability of hypothesis Ei

Conditional Probability

The probability of an event A based on the occurrence of another event B is termed conditional Probability. It is denoted as P(A|B) and represents the probability of A when event B has already happened.

Joint Probability

When the probability of two more events occurring together and at the same time is measured it is marked as Joint Probability. For two events A and B, it is denoted by joint probability is denoted as, P(A∩B).

Random Variables

Real-valued variables whose possible values are determined by random experiments are called random variables. The probability of finding such variables is the experimental probability.

Applications of Bayes’ Theorem

Bayesian inference is very important and has found application in various activities, including medicine, science, philosophy, engineering, sports, law, etc., and Bayesian inference is directly derived from Bayes’ theorem.

Example: Bayes’ theorem defines the accuracy of the medical test by taking into account how likely a person is to have a disease and what is the overall accuracy of the test.

Difference Between Conditional Probability and Bayes Theorem

The difference between Conditional Probability and Bayes Theorem can be understood with the help of the table given below,

Bayes’ Theorem Conditional Probability
Bayes’ Theorem is derived using the definition of conditional probability. It is used to find the reverse probability. Conditional Probability is the probability of event A when event B has already occurred.
Formula: P(A|B) = [P(B|A)P(A)] / P(B) Formula: P(A|B) = P(A∩B) / P(B)

Theorem of Total Probability

Let E1, E2,…………..En is mutually exclusive and exhaustive events associated with a random experiment and lets E be an event that occurs with some Ei. Then, prove that 

P(E) = ni=1P(E/Ei) . P(Ej)

Proof:

Let S be the sample space. Then,

S = E1 ∪ E2 ∪ E3  ∪………………… ∪ En and Ei ∩ Ej = ∅ for i ≠ j.

E = E∩S 

= E ∩ (E1 ∪ E2 ∪ E3  ∪………………… ∪ En)

= (E ∩ E1) ∪ (E ∩ E2) ∪ ……∪ (E ∩ En)

P(E) = P{(E ∩ E1) ∪ (E ∩ E2)∪……∪(E ∩ En)}

= P(E ∩ E1) + P(E ∩ E2) + …… + P(E ∩ En)

= {Therefore, (E ∩ E1), (E ∩ E2),………….,(E ∩ En)} are pairwise disjoint}

= P(E/E1) . P(E1) + P(E/E2) . P(E2) +……………………+ P(E/En) . P(En)  [by multiplication theorem]

= ni=1P(E/Ei) . P(Ei)

Numerical Examples of Bayes Theorem

Example 1: A person has undertaken a job. The probabilities of completion of the job on time with and without rain are 0.44 and 0.95 respectively. If the probability that it will rain is 0.45, then determine the probability that the job will be completed on time.

Solution:

Let E1 be the event that the mining job will be completed on time and E2 be the event that it rains. We have,

P(A) = 0.45,

P(no rain) = P(B) = 1 − P(A) = 1 − 0.45 = 0.55

By multiplication law of probability,

P(E1) = 0.44

P(E2) = 0.95

Since, events A and B form partitions of the sample space S, by total probability theorem, we have

P(E) = P(A) P(E1) + P(B) P(E2)

= 0.45 × 0.44 + 0.55 × 0.95

= 0.198 + 0.5225 = 0.7205

So, the probability that the job will be completed on time is 0.684.

Example 2: There are three urns containing 3 white and 2 black balls; 2 white and 3 black balls; 1 black and 4 white balls respectively. There is an equal probability of each urn being chosen. One ball is equal probability chosen at random. what is the probability that a white ball is drawn?

Solution:

Let E1, E2, and E3 be the events of choosing the first, second, and third urn respectively. Then,

P(E1) = P(E2) = P(E3) =1/3

Let E be the event that a white ball is drawn. Then,

P(E/E1) = 3/5, P(E/E2) = 2/5, P(E/E3) = 4/5

By theorem of total probability, we have

P(E) = P(E/E1) . P(E1) + P(E/E2) . P(E2) + P(E/E3) . P(E3)

       = (3/5 × 1/3) + (2/5 × 1/3) + (4/5 × 1/3)

       = 9/15 = 3/5

Example 3: A card from a pack of 52 cards is lost. From the remaining cards of the pack, two cards are drawn and are found to be both hearts. find the probability of the lost card being a heart.

Solution:

Let E1, E2, E3, and E4 be the events of losing a card of hearts, clubs, spades, and diamonds respectively.

Then P(E1) = P(E2) = P(E3) = P(E4) = 13/52 = 1/4.

Let E be the event of drawing 2 hearts from the remaining 51 cards. Then,

P(E|E1) = probability of drawing 2 hearts, given that a card of hearts is missing

           = 12C2 / 51C2 = (12 × 11)/2! × 2!/(51 × 50) = 22/425

P(E|E2) = probability of drawing 2 clubs ,given that a card of clubs is missing

           = 13C2 / 51C2 = (13 × 12)/2! × 2!/(51 × 50) = 26/425

P(E|E3) = probability of drawing 2 spades ,given that a card of hearts is missing

           = 13C2 / 51C2 = 26/425

P(E|E4) = probability of drawing 2 diamonds ,given that a card of diamonds is missing

           = 13C2 / 51C2 = 26/425

Therefore,

P(E1|E) = probability of the lost card is being a heart, given the 2 hearts are drawn from the remaining 51 cards

           = P(E1) . P(E|E1)/P(E1) . P(E|E1) + P(E2) . P(E|E2) + P(E3) . P(E|E3) + P(E4) . P(E|E4)

           = (1/4 × 22/425) / {(1/4 × 22/425) + (1/4 × 26/425) + (1/4 × 26/425) + (1/4 × 26/425)}

           = 22/100 = 0.22

Hence, The required probability is 0.22.

Example 4: Suppose 15 men out of 300 men and 25 women out of 1000 are good orators. An orator is chosen at random. Find the probability that a male person is selected. Assume that there are equal numbers of men and women.

Solution:

Let there be 1000 men and 1000 women.

Let E1 and E2 be the events of choosing a man and a woman respectively. Then,

P(E1) = 1000/2000 = 1/2 , and P(E2) = 1000/2000 = 1/2

Let E be the event of choosing an orator. Then,

P(E|E1) = 50/1000 = 1/20, and P(E|E2) = 25/1000 = 1/40

Probability of selecting a male person ,given that the person selected is a good orator 

P(E1/E) = P(E|E1) × P(E1)/ P(E|E1) × P(E1) + P(E|E2) × P(E2)

            = (1/2 × 1/20) /{(1/2 × 1/20) + (1/2 × 1/40)}

            = 2/3

Hence the required probability is 2/3.

Example 5: A man is known to speak the lies 1 out of 4 times. He throws a die and reports that it is a six. Find the probability that is actually a six.

Solution: 

In a throw of a die, let

E1 = event of getting a six,

E2 = event of not getting a six and

E = event that the man reports that it is a six.

Then, P(E1) = 1/6, and P(E2) = (1 – 1/6) = 5/6

P(E|E1) = probability that the man reports that six occurs when six has actually occurred 

           = probability that the man speaks the truth

           = 3/4

P(E|E2) = probability that the man reports that six occurs when six has not actually occurred

           = probability that the man does not speak the truth

           = (1 – 3/4) = 1/4

Probability of getting a six ,given that the man reports it to be six 

P(E1|E) = P(E|E1) × P(E1)/P(E|E1) × P(E1) + P(E|E2) × P(E2)     [by Bayes’ theorem]

           = (3/4 × 1/6)/{(3/4 × 1/6) + (1/4 × 5/6)}

           = (1/8 × 3) = 3/8

Hence the probability required is 3/8.

FAQs on Bayes’ Theorem

What is Bayes’ theorem?

Bayes, theorem as the name suggest is a mathematical theorem which is used to find the conditionality probability of an event. Conditional probability is the probability of the event which will occur in future. It is calculated based on the previous outcomes of the events.

What is the Bayes rule used for?

Bayes rule lets you calculate the posterior (or updated) probability. It is used to calculate conditional probability of events.

When to use Bayes’ theorem?

Bayes’ theorem is applicable when the conditional probability of an event is given, it is used to find the reverse probability of the event.

How is Bayes’ theorem different from conditional probability?

Bayes’ theorem is used to define the probability of an event based on the previous conditions of the event. Whereas, Bayes’ Theorem uses conditional probability to find the reverse probability of the event.

What is the formula for Bayes’ theorem?

Bayes theorem formula is explained below,

P(A|B) = [P(B|A) P(A)] / P(B)



Last Updated : 08 Mar, 2024
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads