Implementation of Particle Swarm Optimization
Previous article Particle Swarm Optimization – An Overview talked about inspiration of particle swarm optimization (PSO) , it’s mathematical modelling and algorithm. In this article we will implement particle swarm optimization (PSO) for two fitness functions 1) Rastrigin function 2) Sphere function. The algorithm will run for a predefined number of maximum iterations and will try to find the minimum value of these fitness functions.
Rastrigin function is a non-convex function and is often used as a performance test problem for optimization algorithms.
Attention reader! Don’t stop learning now. Get hold of all the important Machine Learning Concepts with the Machine Learning Foundation Course at a student-friendly price and become industry ready.
For an optimization algorithm, rastrigin function is a very challenging one. Its complex behavior cause optimization algorithms to often stuck at local minima. Having a lot of cosine oscillations on the plane introduces the complex behavior to this function.
2) Sphere function
Sphere function is a standard function for evaluating the performance of an optimization algorithm.
Choice of hyper-parameters
Parameters of problem:
- Number of dimensions (d) = 3
- Lower bound (minx) = -10.0
- Upper bound (maxx) = 10.0
Hyperparameters of the algorithm:
- Number of particles (N) = 50
- Maximum number of iterations (max_iter) = 100
- inertia coefficient (w) = 0.729
- cognitive coefficient (c1) = 1.49445
- social coefficient (c2) = 1.49445
- Fitness function
- Problem parameters ( mentioned above)
- Population size (N) and Maximum number of iterations (max_iter)
- Algorithm Specific hyper parameters ( w, c1, c2)
The pseudocode of the particle swarm optimization is already described in the previous article. Data structures to store Swarm population, as well as a data structure to store data specific to individual particle, were also discussed.
Begin particle swarm optimization on rastrigin function Goal is to minimize Rastrigin's function in 3 variables Function has known min = 0.0 at (0, 0, 0) Setting num_particles = 50 Setting max_iter = 100 Starting PSO algorithm Iter = 10 best fitness = 8.463 Iter = 20 best fitness = 4.792 Iter = 30 best fitness = 2.223 Iter = 40 best fitness = 0.251 Iter = 50 best fitness = 0.251 Iter = 60 best fitness = 0.061 Iter = 70 best fitness = 0.007 Iter = 80 best fitness = 0.005 Iter = 90 best fitness = 0.000 PSO completed Best solution found: ['0.000618', '0.000013', '0.000616'] fitness of best solution = 0.000151 End particle swarm for rastrigin function Begin particle swarm optimization on sphere function Goal is to minimize sphere function in 3 variables Function has known min = 0.0 at (0, 0, 0) Setting num_particles = 50 Setting max_iter = 100 Starting PSO algorithm Iter = 10 best fitness = 0.189 Iter = 20 best fitness = 0.012 Iter = 30 best fitness = 0.001 Iter = 40 best fitness = 0.000 Iter = 50 best fitness = 0.000 Iter = 60 best fitness = 0.000 Iter = 70 best fitness = 0.000 Iter = 80 best fitness = 0.000 Iter = 90 best fitness = 0.000 PSO completed Best solution found: ['0.000004', '-0.000001', '0.000007'] fitness of best solution = 0.000000 End particle swarm for sphere function
Research paper citation: Kennedy, J. and Eberhart, R., 1995, November. Particle swarm optimization. In Proceedings of ICNN’95-international conference on neural networks (Vol. 4, pp. 1942-1948). IEEE.
Inspiration of the implementation: https://fr.mathworks.com/matlabcentral/fileexchange/67429-a-simple-implementation-of-particle-swarm-optimization-pso-algorithm