Tensorflow.js tf.conv3d() Function

Tensorflow.js is javascript library developed by google to run and train machine learning model in the browser or in Node.js.

The tf.conv3d() function is used to compute 3d convolutions over given inputs. The inputs are mainly 3D image data like CT or MRI imaging or any video. To extract features from these data we use a convolution layer which creates convolution kernels and outputs a tensor of 4D or 5D.

Syntax:

tf.conv3d (x, filter, strides, pad, dataFormat?, dilations?) 

Parameters:

• x : Tensor rank of 4 and 5 is fed into the convolution. Tensor can also be a typed array with  shape [batch, depth, height, width, channels].
• filter : Filter is of rank 5 and same datatype as x with shape [filterDepth, filterHeight, filterWidth, inChannels, outChannels]. The inchannels of filter and input must match.
• stride:  Strides are used to move the filter over the input tensor. For this a list is passed of length>=5.
• pad : Padding algorithm are of ‘same’ and ‘valid’ types. ‘Same’ will create an output of same size as input. Whereas ‘valid’ output will be of smaller size than input.
• dataFormat  : Out of two strings it can be either: “NHWC”, “NCHW”. With the default format “NHWC”, the data is stored in the order of: [batch, depth, height, width, channels]. Data format of the input and output data need to be specified. This is optional.
• dilations : The dilation rate is two tuples of integers that check the rate of the convolution.[dilationDepth,dilationHeight ,dilationWidth]is passed as parameters. This is optional.

Example 1: In this example we will create a input and kernel tensor manually and perform convolution operation. We will print out the shape of the tensors to note the channels number and padding algorithm. 

Javascript

// Importing libraries
import * as from "@tensorflow/tfjs"

// Input tensor
const X=tf.tensor5d([[[[[0.],[2.]],
                       [[3.],[1.]],
                       [[4.],[2.]]],
                      [[[1.],[0.]],
                       [[2.],[1.]],
                       [[4.],[2.]]]]]);
console.log('Shape of the input:' ,X.shape);

// Kernel vector has been set
const kernel=tf.tensor5d([[[[[0.3,1.2]]],[[[0.6,1.8]]],[[[0.8,1.5]]]]]);
console.log('Shape of the kernel:' ,kernel.shape);

// Output tensor after convolution
let output=tf.conv3d(X,kernel,strides=[1,1,1,1,1],'same');
output.print();
console.log('Shape of the output tensor:',output.shape);

Output:

Shape of the input: 1,2,3,2,1
Shape of the kernel: 1,3,1,1,2
Tensor
    [[[ [[2        , 5.0999999],],

        [[2.8000002, 7.1999998],],

        [[1.5      , 4.8000002],]],


      [ [[0.8      , 1.5      ],],

        [[2.2      , 4.8000002],],

        [[1.5      , 4.8000002],]]]]
Shape of the output tensor: 1,2,3,1,2

Example 2: In the below code an error will occur as the inchannels number i.e. 5th index of input tensor and 4th index of kernel tensor will not match.

Javascript

import * as from "@tensorflow/tfjs"

// Input tensor
const X=tf.tensor5d([0.,2.,3.,1.,4.,2.,1.,0.,2.,1.,4.,2.],[1,2,3,2,1]);


// Kernel vector has been set
const kernel=tf.tensor5d([0.3,1.2,0.6,1.8,0.8,1.5],[1,1,1,2,3]);


// Output tensor after convolution
let output=tf.conv3d(X,kernel,strides=[1,1,1,2,1],'same');
output.print();
console.log('Shape of the output tensor:',output.shape);

Output:

An error occurred on line: 10
Error in conv3d: depth of input (1) must match input depth for filter 2.

Example 3: In this example we will perform two convolution operations. In first we will create a tensor of values randomly picked from a normal distribution and after convolution print the output tensor. After that we take a tensor of bigger size(which can be of 3D image or video) and perform convolution which will again generate a big tensor. We will print the output shape of that tensor.

Javascript

// Importing libraries
import * as from "@tensorflow/tfjs"

// A tensor with values selected from a
// normal distribution
const x=tf.randomNormal([1,2,4,1,3]);

const kernel=tf.tensor5d([0.3,1.2,0.6,1.8,0.8,1.5],[1,1,2,3,1])

// Output tensor after convolution
let out=tf.conv3d(x,kernel,strides=[1,1,1,1,1],'same')

// Printing the output tensor
console.log('First output tensor is:')
out.print()

// Input tensor of bigger shape is taken
const y=tf.randomNormal([3,25,25,25,1]);

const kernel2=tf.randomNormal([1,3,3,1,1])

// Convolution is performed
let output2=tf.conv3d(y,kernel2,strides=[1,1,1,1,1],'same')

// Output2.print()
// Printing out the bigger output shape
console.log('\n The shape of the output tensor',output2.shape)

Output:

First output tensor is:
Tensor
    [[[ [[-0.702378 ],],

        [[1.9029824 ],],

        [[-0.1904218],],

        [[-2.2287691],]],


      [ [[-1.9580686],],

        [[-2.8335922],],

        [[0.0155853 ],],

        [[-3.6478395],]]]]

 The shape of the bigger output tensor 3,25,25,25,1

Reference: https://js.tensorflow.org/api/1.0.0/#conv3d