Tensorflow.js tf.layers.layerNormalization() Function

Tensorflow.js is a Google-developed open-source toolkit for executing machine learning models and deep learning neural networks in the browser or on the node platform. It also enables developers to create machine learning models in JavaScript and utilize them directly in the browser or with Node.js.

The tf.layers.layerNormalization() function is used to apply the layer normalization operation on data.

Syntax:

tf.layers.layerNormalization(args?)

Input Shape: Arbitrary. When utilizing this layer as the initial layer in a model, use the inputShape configuration.

Output Shape: The output has the same shape as the input.

Parameters: It accepts the args object which can have the following properties:

• axis (number): The integer axis that should be normalized (typically the features axis). -1 is the default value.
• momentum (number): The moving average’s momentum. The default value is 0.99.
• epsilon (number): The small float is added to the variance to avoid division by zero. Defaults to 1e-3.
• center (boolean): If this is true, add the offset of beta to the normalized tensor. If false, beta isn’t taken into account. The value is set to true by default.
• scale (boolean): If this is true, multiplied by gamma. Gamma is not utilized if false. True is the default value.
• betaInitializer: This is the beta weight’s initializer. ‘zeroes’ is the default value.
• gammaInitializer: This is the gamma weight’s initializer. ‘ones’ is the default value.
• movingMeanInitializer: This is the moving mean’s initializer. ‘zeroes’ is the default value.
• movingVarianceInitializer: This is the moving variance’s initializer. ‘ones’ is the default value.
• betaConstraint: The constraint for the beta weight.
• gammaConstraint: The constraint for the gamma weight.
• betaRegularizer: The regularizer for the beta weight.
• gammaRegularizer: The regularizer for the beta weight.
• inputShape: If this property is set, it will be utilized to construct an input layer that will be inserted before this layer. 
• batchInputShape: If this property is set, an input layer will be created and inserted before this layer. 
• batchSize: If batchInputShape isn’t supplied and inputShape is, batchSize is utilized to build the batchInputShape.
• dtype: It is the kind of data type for this layer. float32 is the default value. This parameter applies exclusively to input layers.
• name: This is the layer’s name and is of string type.
• trainable: If the weights of this layer may be changed by fit. True is the default value.
• weights: The layer’s initial weight values.

Returns: It returns an object (LayerNormalization).

Example 1:

Javascript

import * as tf from "@tensorflow/tfjs";

const layerNormalizationLayer = tf.layers.layerNormalization();

const x = tf.tensor([1.12, -0.8, 1.9, 0.12, 0.25, -3.4], [2, 3]);

layerNormalizationLayer.apply(x).print();

Output:

Tensor
   [[0.334805 , -1.3568414, 1.0220363 ],
    [0.6681985, 0.7450709 , -1.4132695]]

Example 2:

Javascript

import * as tf from "@tensorflow/tfjs";

const layerNormalizationLayer = tf.layers.layerNormalization();

const x = tf.tensor([12, 3.2, 4.8, 9, 10, 2.5, 8,
    11, 9.4, 25, 24.9, 98.7], [2, 3, 2]);

layerNormalizationLayer.apply(x).print();

Output:

Tensor
   [[[0.9999743 , -0.9999742],
     [-0.9998867, 0.9998867 ],
     [0.9999645 , -0.9999645]],
    [[-0.9997779, 0.9997779 ],
     [-0.9999919, 0.9999917 ],
     [-0.9999996, 0.9999996 ]]]
 

Reference: https://js.tensorflow.org/api/latest/#layers.layerNormalization