Tensorflow.js tf.image.nonMaxSuppressionWithScore() Function

Tensorflow.js is an open-source library developed by Google for running machine learning models as well as deep learning neural networks in the browser or node environment.

The .image.nonMaxSuppressionWithScore() function is used to execute the non maximum suppression of the limiting boxes on the basis of iou i.e. intersection over union. Moreover, this operation also favors a Soft-NMS mode where boxes decrease the stated score of different intersecting boxes, thus supporting various areas of the image beside high scores. In order to enable aforementioned Soft-NMS mode, we need to set the softNmsSigma parameter to be greater than zero.

Syntax:

tf.image.nonMaxSuppressionWithScore(boxes, scores, maxOutputSize, 
iouThreshold?, scoreThreshold?, softNmsSigma?)

Parameters:  

• boxes: The stated 2d tensor, which is of configuration [numBoxes, 4]. And every access is [y1, x1, y2, x2], allowing that (y1, x1) and (y2, x2) are the edges of the restricting box. It can be of type tf.Tensor2D, TypedArray, or Array.
• scores: The stated 1d tensor, provided that the box scores is of configuration [numBoxes]. It is of type tf.Tensor2D, TypedArray, or Array.
• maxOutputSize: It is the maximum count of the stated boxes that is to be picked. It is of type number.
• iouThreshold: It is the stated float signifying the threshold in order to decide if the stated boxes intersect too much with reference to the IOU. It should be in the midst of [0, 1]. The by default value is 0.5 i.e. 50 percent of the box intersects. It is optional and is of type number.
• scoreThreshold: It is the stated threshold in order to decide at which time boxes are to be removed on the basis of the stated score. The by default value is -inf, i.e. every single score is allowed. It is optional and is of type number.
• softNmsSigma: It is an optional parameter of type number. It is the stated float signifying the sigma parameter in favor of Soft NMS. Moreover, if sigma is zero, then it comes back to the nonMaxSuppression.

Return Value: It returns {[name: string]: tf.Tensor}.

Example 1: In this example, we will be going to use a 2d tensor, scores, and maxOutputSize parameters.

Javascript

// Importing the tensorflow.js library
import * as tf from "@tensorflow/tfjs"

// Calling image.nonMaxSuppressionWithScore() method
const output = tf.image.nonMaxSuppressionWithScore(
    tf.tensor2d([1, 2, 3, 4, 2, 4, 6, 7],
    [2, 4]), [1, 1], 4
);

// Printing output
console.log(output);

Output:

{
  "selectedIndices": {
    "kept": false,
    "isDisposedInternal": false,
    "shape": [
      2
    ],
    "dtype": "int32",
    "size": 2,
    "strides": [],
    "dataId": {
      "id": 74
    },
    "id": 74,
    "rankType": "1",
    "scopeId": 35
  },
  "selectedScores": {
    "kept": false,
    "isDisposedInternal": false,
    "shape": [
      2
    ],
    "dtype": "float32",
    "size": 2,
    "strides": [],
    "dataId": {
      "id": 75
    },
    "id": 75,
    "rankType": "1",
    "scopeId": 35
  }
}

Example 2: In this example, we will be going to use an array of floats, iouThreshold, scoreThreshold, as well as softNmsSigma.

Javascript

// Importing the tensorflow.js library
import * as tf from "@tensorflow/tfjs"

// Defining an array of floats
const arr = [[11.1, 2.3, 7.3, 6.4], [3, 6]]

// Calling image.nonMaxSuppressionWithScore() method
const res = tf.image.nonMaxSuppressionWithScore(
    arr, [2.1, 0], 100, 0.5, 1, 0.5);

// Printing output
console.log(res);

Output:

{
  "selectedIndices": {
    "kept": false,
    "isDisposedInternal": false,
    "shape": [
      1
    ],
    "dtype": "int32",
    "size": 1,
    "strides": [],
    "dataId": {
      "id": 84
    },
    "id": 84,
    "rankType": "1",
    "scopeId": 42
  },
  "selectedScores": {
    "kept": false,
    "isDisposedInternal": false,
    "shape": [
      1
    ],
    "dtype": "float32",
    "size": 1,
    "strides": [],
    "dataId": {
      "id": 85
    },
    "id": 85,
    "rankType": "1",
    "scopeId": 42
  }
}

Reference: https://js.tensorflow.org/api/latest/#image.nonMaxSuppressionWithScore