How to Create a Custom Loss Function in Keras

Creating a custom loss function in Keras is crucial for optimizing deep learning models. The article aims to learn how to create a custom loss function.

Need to create Custom Loss Functions

Loss function is considered as a fundamental component of deep learning as it is helpful in error minimization. Loss is computed by comparing predicted values and actual values for a given set of inputs. Loss functions vary depending on the task. The need to create custom loss functions is discussed below:

• The loss functions vary depending on the machine learning task, there might be some cases where the standard loss functions provided by Keras might not be suitable for a given assignment. This issue can be addressed by introducing custom loss functions.
• These custom loss functions are helpful not only when standard loss functions are inadequate but also when data is imbalanced. Class imbalance can be addressed by employing a custom loss function when the dataset is extremely imbalanced (one class is significantly more abundant than others).
• By assigning minority classes greater weight, custom loss functions can avoid bias in the model’s favour of the dominant class. These custom loss functions can be implemented with Keras.

Creating a Custom Loss Function in Keras

Step 1: Import the necessary libraries

In this step, we import TensorFlow and Keras libraries along with NumPy for numerical operations. We also import necessary modules like Sequential for creating the model, Dense for defining layers, and K from keras.backend for backend operations.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from keras import backend as K
import numpy as np
from tensorflow.python.ops import math_ops

Step 2: Define Custom Loss Function

Here I have defined a custom loss function which calculates mean of absolute difference of squares of true value and the predicted value.

def customloss(Y_actual, Y_predicted):
 absolute_diff = abs((Y_predicted*Y_predicted) - (Y_actual*Y_actual))  #squared difference
 final_loss =(K.mean(absolute_diff, axis=-1))/100 #mean over last dimension
 return final_loss

Step 3: Create the model using Keras.

Using sequential model from the Keras API model is created using Rectified Linear Unit as activation function with two dense layers.

model = keras.Sequential([
                   keras.layers.Dense(10, activation='relu', input_shape=(1,)),
                   keras.layers.Dense(1)
])

Step 4: Compiling the model with custom loss.

Created model is compiled for custom loss function and the optimizer used in ‘adam’.

model.compile(loss=customloss, optimizer='adam')

Step 5: Fitting the Model

Input data (X_train and Y_train) is created in list format and then converted to NumPy arrays. The model is trained using the fit() function for 5 epochs with a batch size of 3.

a=[[12.3],[20.0], [17.6],[15.0],[20.0],[7.5],[5.9], [20.0]]
b=[6.0, 2, 18,24,30, 3,6, 12]
X_train = np.array(a)
Y_train = np.array(b) #dummy data

model.fit(X_train, Y_train, batch_size=3, epochs=5)

Complete Code to create a Custom Loss Function in Keras

Python

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from keras import backend as K
import numpy as np
from tensorflow.python.ops import math_ops

def customloss(Y_actual, Y_predicted):
 absolute_diff = abs((Y_predicted*Y_predicted) - (Y_actual*Y_actual))  #squared difference
 final_loss =(K.mean(absolute_diff, axis=-1))/100 #mean over last dimension
 return final_loss

model = keras.Sequential([
                   keras.layers.Dense(10, activation='relu', input_shape=(1,)),
                   keras.layers.Dense(1)
])
model.compile(loss=customloss, optimizer='adam')

a=[[12.3],[20.0], [17.6],[15.0],[20.0],[7.5],[5.9], [20.0]]
b=[6.0, 2, 18,24,30, 3,6, 12]
X_train = np.array(a)
Y_train = np.array(b) #dummy data

model.fit(X_train, Y_train, batch_size=3, epochs=5)

Output:

Epoch 1/5
3/3 [==============================] - 1s 23ms/step - loss: 2.4541
Epoch 2/5
3/3 [==============================] - 0s 13ms/step - loss: 2.4497
Epoch 3/5
3/3 [==============================] - 0s 8ms/step - loss: 2.4474
Epoch 4/5
3/3 [==============================] - 0s 10ms/step - loss: 2.4445
Epoch 5/5
3/3 [==============================] - 0s 10ms/step - loss: 2.4402

The output displayed shows the loss calculated in each epoch using the custom loss function defined. Each epoch’s loss is printed, indicating how well the model is optimizing towards the desired outcome.