# Axiomatic Approach to Probability

• Last Updated : 26 May, 2021

Hearing the word probability brings up nebulous concepts related to uncertainty or randomness. The concept of probability is hard to describe formally, the closest intuition is that it helps us analyze the likelihood or chances that a certain event will happen. This analysis helps us to describe a lot of phenomena we see in real life. Even the most randomly seeming processes or phenomena can be described using the probability models and can be predicted up to a certain extent. That’s why probability makes the foundation for artificial intelligence algorithms that we encounter in real life. Before formally describing probability laws, let’s look at basic terminology.

### Events and Sample Space

Suppose an experiment that involves tossing a coin. Now, there are only two outcomes of a coin toss – Heads or Tails. The interest is to study and calculate the chances of getting a tail as a result of a coin toss. This is called a random experiment and all the possible outcomes of this experiment constitute sample space. For example, let’s say a coin is tossed 2 times. What are the possible outcomes?

TH, HH, HT, TT

All these outcomes constitute sample space.

Random Experiment: A random experiment is an experiment in which outcomes are random and thus cannot be predicted with certainty.

Sample Space: Sample space is the set of all possible outcomes associated with a random experiment. It is denoted using the symbol S.

Let’s measure the probability of getting two heads in the above experiment. Then the probability of this outcome is defined as,

P =

For this case, favorable outcome is HH and the total number of possible outcomes are four.

### Different Probability Approaches

The previous formula for calculating the probabilities assumes that all the outcomes are equally likely. For example, in the tossing of a fair coin. The outcomes head and tails are equally likely. So this cannot be generalized to every experiment. Initially, there were basically two schools of thought in probability:

1. Classical Probability
2. Frequentist Probability

Classical Probability

This approach assumes that all the outcomes are equally likely. If our event can happen in “n” ways out of a total of “N” ways. Then probability can be given by,

Frequentist Probability

This is a more general approach to calculating the probability. It does not make the assumption that all the outcomes are equally likely. When outcomes are not equally likely, we repeat the experiment many times let’s say M. Then, observe how many times that particular event occurred let’ say m. Then, calculate the empirical estimate of the probability. So, use the relation,

Both of these approaches fail to generalize well and stand up to mathematical rigor.

The axiomatic approach to probability takes on the approach of considering probability as a function associated with any event.

### Axiomatic Approach to Probability

Perform a random experiment whose sample space is S and P is the probability of occurrence of any random event. This model assumes that P should be a real-valued function with a range between 0 and 1. The domain of this function is defined to be a power set of sample space. If all these conditions are satisfied then, the function should satisfy the following axioms:

Axiom 1: For any given event X, the probability of that event must be greater than or equal to 0. Thus,

0 ≤ P(X)

Axiom 2: We know that the sample space S of the experiment is the set of all the outcomes. This means that the probability of any one outcome happening is 100 percent i.e P(S) = 1. Intuitively this means that whenever this experiment is performed, the probability of getting some outcome is 100 percent.

P(S) = 1

Axiom 3: For the experiments where we have two outcomes A and B. If A and B are mutually exclusive,

P(A ∪ B) = P(A) + P(B)

Here, ∪ stands for union. This can be understood as if saying “If A and B are mutually exclusive outcomes, that probability that either one of these events will happen is probability of A happening plus the probability of B happening”.

These axioms are also called Kolmogorov’s three axioms. The third axiom can also be extended to a number of outcomes given all are mutually exclusive.

Let’s say the experiment has A1, A2, A3, and … An. All these events are mutually exclusive. In this case, the three axioms become:

Axiom 1: 0 ≤ P(Ai) ≤ 1 for all i = 1,2,3,… n.

Axiom 2: P(A1) + P(A2) + P(A3) +…. = 1

Axiom 3: P(A1 ∪ A2∪ A3 ….) = P(A1) + P(A2) + P(A3) ….

Let’s look at some sample problems based on these concepts.

### Sample Problems

Question 1: Find out the sample space “S” for a random experiment involving the tossing of three coins.

Solution.

We know that tossing a coin gives us either Heads or Tails. Tossing three coins will give us either triplets of either heads or tails. So, the possible outcomes can be,

HHH, HHT, HTH, HTT, ….

All these outcomes will constitute the sample space.

S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

Question 2: Find out the probability of getting a number 3 when a die is tossed.

We know that possible outcomes when a die is tossed are,

{1, 2, 3, 4, 5 and 6}

We want to calculate the probability for getting a number 3.

Number of favorable outcomes = 1

Total Number of outcomes = 6.

So, the probability of getting a number 3, P(3) =

P(3) =

Question 3: Let’s say a class is choosing their class captain through a random draw. The class has 30% Indian students, 50% American students, and 20% Chinese students. Calculate the probability that the chosen captain will be an Indian.

Let’s define an event A: Chosen captain is Indian. We know that there are only 30% Indian students in class.

Measure of favorable outcome = 0.3

Total Number of outcomes = 1

So, P(A) =

P(A) =

P(A) = 0.3

So, there is a 30% probability that an Indian student will be chosen as class captain.

Question 4: Find out the probability of getting an even number when a die is tossed.

We know that possible outcomes when a die is tossed are,

{1, 2, 3, 4, 5 and 6}

We want to calculate the probability for getting an even number. Even number are {2,4,6}

Number of favorable outcomes = 3

Total Number of outcomes = 6.

So, the probability of getting an even number, P(Even) =

P(Even) =

⇒ P(Even) =

Question 5: Let’s say we have an urn with 5 red balls and 3 black balls. We want to draw balls from this bag. Find out the probability of picking a red ball.

Let’s define the experiment as “Drawing a ball from the bag”. Now it is required to calculate the probability for getting a red ball.

Number of favorable outcomes = 5

Total Number of outcomes = 8.

So, P(red) =

Question 6: For the above experiment, verify that the probability of getting a red ball and the probability of getting a black ball follow the axioms of probability mentioned above.

Let’s define two events,

R = Red ball is picked

B = Black Ball is picked

Calculate the probability for getting a red ball in the previous example,

P(R) =

Similarly,

P(B) =

Now notice that both P(R) and P(B) lie between 0 and 1. So they satisfy axiom 1. Let’s verify it for second axiom.

P(R) + P(B)

⇒P(R) + P(B) = +

⇒ P(R) + P(B) = 1

Thus second axiom is also satisfied.

We know that both of these events are mutually exclusive.

So, P(R ∪ B) = P(Getting either a Red Ball or Black Ball)

⇒ P(R ∪ B) = P(R) + P(B)

⇒ P(R ∪ B) = +

⇒ P(R ∪ B) = 1

Thus, all three of these axioms are satisfied. Thus, above experiment follows the axioms of the probability.

My Personal Notes arrow_drop_up