What is Saliency Map?

Saliency Map is an important concept of deep learning and Computer vision. While training images of birds how does CNN knows to focus on bird-related pixels and ignore the leaves and the other background things in the image? By using the concept of Saliency Map. 

Saliency Map

Saliency Map is an image in which the brightness of a pixel represents how salient the pixel is i.e brightness of a pixel is directly proportional to its saliency. It is generally a grayscale image. Saliency maps are also called as a heat map where hotness refers to those regions of the image which have a big impact on predicting the class which the object belongs to. 
The purpose of the saliency map is to find the regions which are prominent or noticeable at every location in the visual field and to guide the selection of attended locations, based on the spatial distribution of saliency. 
It is used in various Visual Attention models. “ITTI and Koch” Computational Model of Visual Attention is based on the concept of saliency Map. 

Examples:

Here is an example, the picture shown in the right is the saliency map of the left one which shows the regions which are more attentive part to CNN. 

Why we use Saliency Map

In General, we take an image as input and we use the whole image to predict the output. So if we have an image of a bird and we predict bird but not the whole input is actually important and not the whole input contributes equally to predict the output. So if we have a really big image where only a few pixels the class we want to predict so computing the whole input is not a good idea i.e why we use a saliency map to highlight the important regions of the image and processed only the highlighted parts. It will actually help to relieve the computational burden.  

How to create Saliency Map?

It is created by using the following Steps. 

• We have an image and the basic features like colour, orientation, the intensity is extracted from the image. 
  
• These processed images are used to create Gaussian pyramids to create features Map. 
  
• Saliency map is created by taking the mean of all the feature maps. 

Application of Saliency Map in various fields

• region-of-interest extraction: Saliency Map is used to extract the regions of most potential interest. 
  
• Image cropping: It selects the region of interest and then we can crop the unnoticeable region. 
  
• Image Captioning: Used in image captioning to recognise the objects. 
  
• Medical Imaging: Used in medical image to process MRI images. 
  
• Robot active vision: Robotics is a very large domain of application with various needs. It is used in image registration, landmarks extraction, object recognition and robot action guidance. 
  
• Audio surveillance: Saliency models are used to spot unusual sounds in classical contextual sounds like a gunshot in the middle of a metro station audio ambience.