Visualizing the Bivariate Gaussian Distribution in Python
The Gaussian distribution(or normal distribution) is one of the most fundamental probability distributions in nature. From its occurrence in daily life to its applications in statistical learning techniques, it is one of the most profound mathematical discoveries ever made. This article will ahead towards the multi-dimensional distribution and get an intuitive understanding of the bivariate normal distribution.
The benefit of covering the bivariate distribution is that we can visually see and understand using appropriate geometric plots. Moreover, the same concepts learned through the bivariate distribution can be extended to any number of dimensions. We’ll first briefly cover the theoretical aspects of the distribution and do an exhaustive analysis of the various aspects of it, like the covariance matrix and the density function in Python!
Probability Density Function(or density function or PDF) of a Bivariate Gaussian distribution
The density function describes the relative likelihood of a random variable at a given sample. If the value is high around a given sample, that means that the random variable will most probably take on that value when sampled at random. Responsible for its characteristic “bell shape”, the density function of a given bivariate Gaussian random variable is mathematically defined as:
Whereis any input vector while the symbols and have their usual meaning.
The main function used in this article is the scipy.stats.multivariate_normal function from the Scipy utility for a multivariate normal random variable.
Syntax: scipy.stats.multivariate_normal(mean=None, cov=1)
- mean: A Numpy array specifyinh the mean of the distribution
- cov: A Numpy array specifying a positive definite covariance matrix
- seed: A random seed for generating reproducible results
Returns: A multivariate normal random variable object scipy.stats._multivariate.multivariate_normal_gen object. Some of the methods of the returned object which are useful for this article are as follows:
- pdf(x): Returns the density function value at the value ‘x’
- rvs(size): Draws ‘size’ number of samples from the generated multivariate Gaussian distribution
A “visual” view of the covariance matrix
The covariance matrix is perhaps one of the most resourceful components of a bivariate Gaussian distribution. Each element of the covariance matrix defines the covariance between each subsequent pair of random variables. The covariance between two random variables and is mathematically defined as where denotes the expected value of a given random variable . Intuitively speaking, by observing the diagonal elements of the covariance matrix we can easily imagine the contour drawn out by the two Gaussian random variables in 2D. Here’s how:
The values present in the right diagonal represent the joint covariance between two components of the corresponding random variables. If the value is +ve, that means there is positive covariance between the two random variables which means that if we go in a direction where increases then will increase in that direction also and vice versa. Similarly, if the value is negative that means will decrease in the direction of an increase in .
Below is the implementation of the covariance matrix:
In the following code snippets we’ll be generating 3 different Gaussian bivariate distributions with same mean but different covariance matrices:
- Covariance matrix with -ve covariance =
- Covariance matrix with 0 covariance =
- Covariance matrix with +ve covariance =
We can see that the code’s output has successfully met our theoretical proofs! Note that the value 0.8 was taken just for convenience purposes. The reader can play around with different magnitudes of covariance and expect consistent results.
3D view of the probability density function:
Now we can move over to one of the most interesting and characteristic aspects of the bivariate Gaussian distribution, the density function! The density function is responsible for the characteristic bell shape of the distribution.
1) Plot of the density function
2) Plot of contours
As we can see, the density function’s contours exactly match the samples drawn by us in the previous section. Note that the 3 sigma boundary(concluded from the 68-95-99.7 rule) ensures maximum sample coverage for the defined distribution. As mentioned earlier, the reader can play around with different boundaries and expect consistent results.
We understood the various intricacies behind the Gaussian bivariate distribution through a series of plots and verified the theoretical results with the practical findings using Python. The reader is encouraged to play around with the code snippets for gaining a much more profound intuition about this magical distribution!