Tensorflow.js tf.train.adadelta() Function

Tensorflow.js is an open-source library developed by Google for running machine learning models and deep learning neural networks in the browser or node environment.

The tf.train.adadelta() function us used to create a tf.AdadeltaOptimizer that uses adadelta algorithm. The adadelta algorithm is a extension of gradient decent optimization algorithm. It is used to optimise neural networks.

Syntax:

tf.train.adadelta(learningRate)

Parameters:

• learningRate: It specifies the learning rate which will be used by adadelta gradient descent algorithm.
• rho: It specifies the learning rate decay over each update.
• epsilon:  It specifies a constant epsilon which is used to improve grad update’s condition. Optional

Return value: It returns a tf.adadeltaOptimizer

Example 1: Fit a function f=(a*x+y) using adadelta optimizer, by learning coefficients a and b.

Javascript

// Importing tensorflow
import * as tf from "@tensorflow/tfjs"

const xs = tf.tensor1d([0, 1, 2, 3]);
const ys = tf.tensor1d([1.1, 5.9, 16.8, 33.9]);

// Choosing variable coefficients
const a = tf.scalar(Math.random()).variable();
const b = tf.scalar(Math.random()).variable();

// Defining function f = (a*x + b)
const f = x => a.mul(x).add(b);
const loss = (pred, label) => pred.sub(label).square().mean();

const learningRate = 0.01;

// Creating optimizer
const optimizer = tf.train.adadelta(learningRate);

// Train the model.
for (let i = 0; i < 10; i++) {
   optimizer.minimize(() => loss(f(xs), ys));
}

// Make predictions.
console.log(
     `a: ${a.dataSync()}, b: ${b.dataSync()}}`);
const preds = f(xs).dataSync();
preds.forEach((pred, i) => {
   console.log(`x: ${i}, pred: ${pred}`);
});

Output:

a: 5.39164924621582, b: 1.8858184814453125}
x: 0, pred: 1.8858184814453125
x: 1, pred: 7.277467727661133
x: 2, pred: 12.669116973876953
x: 3, pred: 18.060766220092773

Example 2: Fit a quadratic equation using adadelta optimizer, by learning coefficients a, b and c. Optimizer configuration is as follows:

• learningRate = 0.01
• rho = 0.2
• epsilon = 0.5

Javascript

// Importing tensorflow
import * as tf from "@tensorflow/tfjs"

const xs = tf.tensor1d([0, 1, 2, 3]);
const ys = tf.tensor1d([1.1, 5.9, 16.8, 33.9]);

const a = tf.scalar(Math.random()).variable();
const b = tf.scalar(Math.random()).variable();
const c = tf.scalar(Math.random()).variable();

const f = x => a.mul(x.square()).add(b.mul(x)).add(c);
const loss = (pred, label) => pred.sub(label).square().mean();

// Setting configurations for our optimizer
const learningRate = 0.01;
const rho = 0.2;
const epsilon = 0.5;

// Creating the optimizer
const optimizer = tf.train.adadelta(learningRate, rho, epsilon);

// Train the model.
for (let i = 0; i < 10; i++) {
   optimizer.minimize(() => loss(f(xs), ys));
}

// Make predictions.
console.log(
     `a: ${a.dataSync()}, b: ${b.dataSync()}, c: ${c.dataSync()}`);
const preds = f(xs).dataSync();
preds.forEach((pred, i) => {
   console.log(`x: ${i}, pred: ${pred}`);
});

Output:

a: 3.1871466636657715, b: 1.5096971988677979, c:0.8317463397979736
x: 0, pred: 0.8317463397979736
x: 1, pred: 5.528590202331543
x: 2, pred: 16.599727630615234
x: 3, pred: 34.04515838623047

Reference: https://js.tensorflow.org/api/1.0.0/#train.adadelta