Image Inpainting using OpenCV

Image inpainting is the process of removing damage, such as noises, strokes or text, on images. It is particularly useful in the restoration of old photographs which might have scratched edges or ink spots on them. These can be digitally removed through this method. Image inpainting works by replacing the damaged pixels with pixels similar to the neighboring ones, therefore, making them inconspicuous and helping them blend well with the background. Consider the image below. The image has some marks to the right. To inpaint this image, we require a mask, which is essentially a black image with white marks on it to indicate the regions which need to be corrected. In this case, the mask is created manually on GIMP. 

But don’t worry. You don’t need to learn how to use GIMP tool. Even if you know how to use OpenCV, you can easily create the following mask.

Creating the mask manually in OpenCV

We will follow the following steps to create the mask using OpenCV python:

1. Reading the damaged image
2. Getting the shape (height and width) of the image
3. Converting all the pixels in the damaged image to our desired output i.e., making all the black pixels white and other remaining pixels except black into black
4. We’ll save the mask in .jpg format or other format (As per your requirements)

Python3

import cv2
import numpy as np

# reading the damaged image
damaged_img = cv2.imread(filename=r"cat_damaged.png")

# get the shape of the image
height, width = damaged_img.shape[0], damaged_img.shape[1]

# Converting all pixels greater than zero to black while black becomes white
for i in range(height):
    for j in range(width):
        if damaged_img[i, j].sum() > 0:
            damaged_img[i, j] = 0
        else:
            damaged_img[i, j] = [255, 255, 255]

# saving the mask
mask = damaged_img
cv2.imwrite('mask.jpg', mask)

# displaying mask
cv2.imshow("damaged image mask", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

If you run the above code block, you can see the following result (mask created using the above cat_damaged image using OpenCV):

mask created using the above cat_damaged image using OpenCV

Now if you use the above mask and apply the following inpainting algorithm then you would get same results. 

Inpainting Algorithms – OpenCV implements two inpainting algorithms:

1. “An Image Inpainting Technique Based on the Fast Marching Method”, Alexandru Telea, 2004: This is based on Fast Marching Method (FMM). Looking at the region to be inpainted, the algorithm first starts with the boundary pixels and then goes to the pixels inside the boundary. It replaces each pixel to be inpainted with a weighted sum of the pixels in the background, with more weight given to nearer pixels and boundary pixels.
2. “Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting”, Bertalmio, Marcelo, Andrea L. Bertozzi, and Guillermo Sapiro, 2001: This algorithm is inspired by partial differential equations. Starting from the edges (known regions) towards the unknown regions, it propagates isophote lines (lines that join same-intensity points). Finally, variance in an area is minimized to fill colors.

FMM can be invoked by using cv2.INPAINT_TELEA, while Navier-Stokes can be invoked using cv2.INPAINT_NS. The Python code below inpaints the image of the cat using Navier-Stokes. 

Python

import numpy as np
import cv2

# Open the image.
img = cv2.imread('cat_damaged.png')

# Load the mask.
mask = cv2.imread('cat_mask.png', 0)

# Inpaint.
dst = cv2.inpaint(img, mask, 3, cv2.INPAINT_NS)

# Write the output.
cv2.imwrite('cat_inpainted.png', dst)

Output: