Parzen Windows density estimation technique

Parzen Window is a non-parametric density estimation technique. Density estimation in Pattern Recognition can be achieved by using the approach of the Parzen Windows. Parzen window density estimation technique is a kind of generalization of the histogram technique.

It is used to derive a density function, ${ f(x)$ .
${ f(x)$ is used to implement a Bayes Classifier. When we have a new sample feature $x$ and when there is a need to compute the value of the class conditional densities, ${ f(x)$ is used.
${ f(x)$ takes sample input data value and returns the density estimate of the given data sample.

An n-dimensional hypercube is considered which is assumed to possess k-data samples.
The length of the edge of the hypercube is assumed to be h_n.

Hence the volume of the hypercube is: V_n = h_n^d

We define a hypercube window function, φ(u) which is an indicator function of the unit hypercube which is centered at origin.:
φ(u) = 1 if |u_i| <= 0.5
φ(u) = 0 otherwise
Here, u is a vector, u = (u₁, u₂, …, u_d)^T.
φ(u) should satisfy the following:

$\varphi\((u) >= 0 ; \forall u$
$\int_{R^{d}}^{} \varphi\((u).du = 1$

Let $V = \int_{R^{d}}^{} \varphi$ \frac{u}{h} $)du = \int_{R^{d}}^{} \varphi\ (\frac{u-u_{0}}{h} \))du$

Since, φ(u) is centered at the origin, it is symmetric.
φ(u) = φ(-u)

$\varphi$\frac{(u-u_{0})}{h}$$ is a hypercube of size h cenetered at u₀
Let D = {x₁, x₂, …, x_n} be the data samples.
For any $x, \varphi$\frac{(x-x_{i})}{h}$$ would be 1 only if $x_{i}$ falls in a hypercube of side $h$ centered at $x$ .
Hence the number of data points falling in a hypercube of side h centered at x is $k =$\sum_{i=1}^{n}\varphi\(\frac{(x-x_{i})}{h}$$

Hence the estimated density function is : ${\LARGE f(x) = $ \frac{1}{n} $ $\sum_{i=1}^{n}$ $ \frac{1}{h^{d}} $ \varphi$\frac{(x-x_{i})}{h}$ }$

Also Since, V_n = h_n^d, Density Function becomes :
${\LARGE f(x) = $ \frac{1}{n} $ $\sum_{i=1}^{n}$ $ \frac{1}{V} $ \varphi$\frac{(x-x_{i})}{h}$ }$

$f(x)$ would satisfy the following conditions:

$f(x) >= 0 ; \forall x$
$\int_{}^{} f(x).dx = 1$

GeeksforGeeks has prepared a complete interview preparation course with premium videos, theory, practice problems, TA support and many more features. Please refer Placement 100 for details

My Personal Notes

Recommended Posts: