How to optimize networking using optimization algorithms in PyBrain

In this article, we are going to see how to optimize networking using optimization algorithms in PyBrain using Python.

In the field of machine learning, Optimization algorithms are specifically used to reduce certain functions known as loss function/error function. By loss function, the optimization algorithm may result in reducing the difference between the actual and predicted output. Eventually, building the model more accurate for the task. This article focuses on optimizing networking using optimization algorithms in PyBrain. PyBrain provides the support of GA optimization algorithm in order to optimize a network.

GA optimization algorithm

Step 1: Construct a classification dataset.

Let us firstly create a classification dataset. In this example, we have taken OR dataset.

Python3

# Python program to create a classification dataset

# importing library
from pybrain.datasets.classification import ClassificationDataSet

# Creating  OR dataset
orDataset = ClassificationDataSet(2)

# Inserting sample to orDataset
orDataset.addSample([0., 0.], [0.])
orDataset.addSample([0., 1.], [1.])
orDataset.addSample([1., 0.], [1.])
orDataset.addSample([1., 1.], [1.])

# Set the target field
orDataset.setField('class', [[0.],[1.],[1.],[1.]])

Step 2: Creating a network.

To create a network, PyBrain provides us with pybrain.tools.shortcuts. We can import buildNetwork shortcuts from it.

Python3

# Python program to create a network

from pybrain.tools.shortcuts import buildNetwork

# Building a network
# The network consists of two input layers,
# four hidden layers and one output layer
myNetwork = buildNetwork(2, 4, 1)

Step 3:  Applying GA optimization algorithm.

The GA has the following syntax:

GA(dataset, network, minimize = True / False)

Here,

• dataset: A dataset
• network: The created network
• minimize = “True”: For reducing error function

Python3

from pybrain.optimization.populationbased.ga import GA

# GA optimization algorithm
gaOptimization = GA(orDataset.evaluateModuleMSE,
                    myNetwork, minimize=True)

Step 4: Applying learn operation.

After that, we need to iterate using a loop and optimize the created gaOptimization using learn(0) operation.

Python3

# 100 iterations for learning
for i in range(100):
    myNetwork = gaOptimization.learn(0)[0]

# Giving input to activate the network
print(myNetwork.activate([0, 0]))
print(myNetwork.activate([1, 0]))
print(myNetwork.activate([0, 1]))
print(myNetwork.activate([1, 1]))

Below is the complete implementation:

Python3

# Python program to demonstrate how to
# optimize a network using Optimization
# algorithms in PyBrain

# Importing library
from pybrain.datasets.classification import ClassificationDataSet
from pybrain.tools.shortcuts import buildNetwork
from pybrain.optimization.populationbased.ga import GA

# Creating  OR dataset
orDataset = ClassificationDataSet(2)

# Inserting sample to orDataset
orDataset.addSample([0., 0.], [0.])
orDataset.addSample([0., 1.], [1.])
orDataset.addSample([1., 0.], [1.])
orDataset.addSample([1., 1.], [1.])

# Set the target field
orDataset.setField('class', [[0.], [1.], [1.], [1.]])


# Building a network
# The network consists of two input layers,
# four hidden layers and one output layer
myNetwork = buildNetwork(2, 4, 1)

# GA optimization algorithm
gaOptimization = GA(orDataset.evaluateModuleMSE,
                    myNetwork, minimize=True)

# 100 iterations for learning
for i in range(100):
    myNetwork = gaOptimization.learn(0)[0]

# By passing input optimize the network
print(myNetwork.activate([0, 0]))
print(myNetwork.activate([1, 0]))
print(myNetwork.activate([0, 1]))
print(myNetwork.activate([1, 1]))

Output: