Tensorflow.js tf.train.rmsprop() Function

Tensorflow.js is an open-source library developed by Google for running machine learning models and deep learning neural networks in the browser or node environment.

The tf.train.rmsprop() function is used to create a tf.RMSPropOptimizer that uses RMSProp gradient decent algorithm. The implementation of RMSProp optimizer is not the centered version of RMSProp and it uses plain momentum. 

Syntax:

tf.train.rmsprop(learningRate, decay, momentum, epsilon, centered)

Parameters:

• learningRate (number): It specifies the learning rate which will be used by adadelta gradient descent algorithm.
• decay (number): It specifies the decay rate of each gradient.
• momentum (number): It specifies the momentum which will be used by rmsprop gradient descent algorithm.
• epsilon:  It specifies a constant small value which is used to avoid zero denominator.
• centered (boolean): It specifies whether the gradients are normalised by the estimated gradient variance or not.

Return value: It returns a tf.RMSPropOptimizer

Example 1: Fit a function f=(a*x+y) using RMSProp optimizer, by learning coefficients a and b.

Javascript

// Importing tensorflow
import * as tf from "@tensorflow/tfjs"

const xs = tf.tensor1d([0, 1, 2]);
const ys = tf.tensor1d([1.1, 5.9, 16.8]);

// Choosing random coefficients.
const a = tf.scalar(Math.random()).variable();
const b = tf.scalar(Math.random()).variable();

// Defining function f = (a*x + b). We will use
// optimizer to fit f
const f = x => a.mul(x).add(b);
const loss = (pred, label) => pred.sub(label).square().mean();

// Define rate which will be used by rmsprop algorithm
const learningRate = 0.01;

// Create optimizer
const optimizer = tf.train.rmsprop(learningRate);

// Train the model.
for (let i = 0; i < 8; i++) {
   optimizer.minimize(() => loss(f(xs), ys));
}

// Make predictions.
console.log(
     `a: ${a.dataSync()}, b: ${b.dataSync()}}`);
const preds = f(xs).dataSync();
preds.forEach((pred, i) => {
   console.log(`x: ${i}, pred: ${pred}`);
});

Output:

a:0.9164762496948242, b: 1.0887205600738525}
x: 0, pred: 1.0887205600738525
x: 1, pred: 2.0051968097686768
x: 2, pred: 2.921673059463501

Example 2:  Fit a quadratic equation using RMSProp optimizer, by learning coefficients a, b and c. Optimizer will have following configuration:

• learningRate = 0.01
• decay = 0.1
• momentum = 1
• epsilon = 0.5
• centered = true

Javascript

// Importing tensorflow
import * as tf from "@tensorflow/tfjs"

const xs = tf.tensor1d([0, 1, 2, 3]);
const ys = tf.tensor1d([1.1, 5.9, 16.8, 33.9]);

// Choosing random coefficients
const a = tf.scalar(Math.random()).variable();
const b = tf.scalar(Math.random()).variable();
const c = tf.scalar(Math.random()).variable();

// Defining function f = (a*x^2 + b*x + c)
const f = x => a.mul(x.square()).add(b.mul(x)).add(c);
const loss = (pred, label) => pred.sub(label).square().mean();

// Setting configurations for our optimizer
const learningRate = 0.01;
const decay = 0.1;
const momentum = 1;
const epsilon = 0.5;
const centered = true;

// Create the optimizer
const optimizer = tf.train.rmsprop(learningRate,
        decay, momentum, epsilon, centered);

// Train the model.
for (let i = 0; i < 8; i++) {
   optimizer.minimize(() => loss(f(xs), ys));
}

// Make predictions.
console.log(`a: ${a.dataSync()},
    b: ${b.dataSync()}, c: ${c.dataSync()}`);
const preds = f(xs).dataSync();
preds.forEach((pred, i) => {
   console.log(`x: ${i}, pred: ${pred}`);
});

Output:

a: 3.918823003768921, b: 3.333444833755493, c: 6.297145843505859
x: 0, pred: 6.297145843505859
x: 1, pred: 13.549413681030273
x: 2, pred: 28.639328002929688
x: 3, pred: 51.56688690185547

Reference: https://js.tensorflow.org/api/1.0.0/#train.rmsprop