Continuous Integration and Continuous Development(CI/CD)
Continuous Integration and Continuous Development(CI/CD) is a new methodology, which is gaining huge space in the software industry and got popularity in less time due to its features. However, there are many companies that already have started the software development process using this methodology and others are on this way. There are many advantages of CI/CD, which attracted developers and the software industry towards it and some are below:
- Speed: Continuous integration is meant to be rapid with instant feedback
- Accuracy: Applying automation to the deployment process is a great starting point
- Reliability: Having a reliable CI/CD pipeline improves the speed of building and deploying new commits significantly
- Smaller Code Changes: Using continuous testing, these small pieces can be tested as soon as they are integrated into the code repository
- Faster Release Rate: Failures are detected faster and as such, can be repaired faster, leading to increased release rates
Stages of CI/CD
There are no defined stages of CI/CD. However, CI/CD is a very flexible methodology and developers can set these stages according to their choice. There are below some standard choice of CI/CD stages and these are all in the pipeline if previous stages fail it will not go to the next stage.
- Version Control: It is used to manage the changes in code, documentation, etc. For example, GitLab, GitHub, TFS, etc.
- Build: This stage build the solution of the software product and store the artifacts
- Unit Test: After successfully building the solution in stage 2, it will go for unit testing if it has any.
- Deploy: After successfully building and Unit testing stages, the solution will be deployed in a temporary environment.
- Automated Test: This stage will run all the automated tests to verify that product is able to fulfill all the requirements before it deploys to production.
- Deploy To Production: After successfully running all the test cases and verification, now this stage will deploy the product in a production environment.
There are many software tools available which are providing an implementation of CI/CD, and some are below and these can be used by developers or companies’ choice and each one has its own characteristics:
GitLab CI/CD Requirements
There can be any software tool to implement CI/CD as mentioned above. However, GitLab providing CI/CD implementation in a flexible way, and without much knowledge or other additional software, anyone who has some basic knowledge about software development, can easily setup CI/CD configuration with GitLab. Below are some requirements:
- Version Control: GitLab is itself a version control to implement CI/CD
- GitLab Runner: It is an important tool to implement CI/CD and need to install it before any configuration of CI/CD. It picks up and execute jobs for GitLab when anyone commits the changes. There are two types of Runner – Shared Runners and Specific Runner.
- Shared Runner: These Runners execute code of different projects on the same Runner and use shared runners when you have multiple jobs with similar requirements.
- Specific Runner: It is an individual runner and each project will be executed by a separate runner and managed by the only developer who is working on this specific project.
- GitLab Runner Executors: GitLab Runner implements a number of executors that can be used to run builds in different scenarios. There are several executors available in GitLab, which can be implemented according to the requirements of the project or the availability of the software tools. Some executors are below:
- SSH: The SSH executor supports only scripts generated in Bash and the caching feature is currently not supported. This is a simple executor that allows you to execute builds on a remote machine by executing commands over SSH.
- Shell: Shell is the simplest executor to configure. All required dependencies for your builds need to be installed manually on the same machine that the Runner is installed on.
- Parallels: This executor configuration is used with Virtual Box.
- VirtualBox: This type of executor allows to use an already created virtual machine, which is cloned and used to run your build.
- Docker: A great option is to use Docker as it allows a clean build environment, with easy dependency management (all dependencies for building the project can be put in the Docker image).
- Docker Machine (auto-scaling): The Docker Machine is a special version of the Docker executor with support for auto-scaling. It works like the normal Docker executor but with the build, hosts created on demand by Docker Machine.
- Kubernetes executor: The Kubernetes executor allows to use an existing Kubernetes cluster for your builds.
- Custom executor: The Custom executor allows to specify your own execution environments.
- Yml File: Yml file is needed to run the CI/CD and it will need to store in a source file of the project, and it includes configuration of each technology and proceeds method.
GitLab Runner Windows
Follow the below steps, download GitLab Runner on Windows, and go through commands to verify and register it.
1. Download the GitLab runner from https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-386.exe 2. Create a folder in your local directory (for ex. C:\GitLab-Runner) and rename the downloaded file to gitlab-runner.exe, store it in this folder.
3. Type below command, to verify and register it.
GitLab Runner Commands
There are many commands of GitLab Runner if already installed in the system, For example, start GitLab Runner, Stop GitLab Runner, Register, etc.
|gitlab-runner.exe register||Register the project with CI/CD|
|gitlab-runner.exe start||Start the runner|
|gitlab-runner.exe stop||Stop the runner|
|gitlab-runner.exe status||To know the status of gitlab-runner|
|gitlab-runner unregister –name test-runner||Unregister Runner of a project and replace test-runner with your runner name and this name can be found inside the config.toml file (where your gitlab.exe) available.|
|gitlab-runner unregister –url http://gitlab.example.com/ –token t0k3n||Remove Runner by URL and token|
|gitlab-runner unregister –all-runners||Unregister All Runners|
|gitlab-runner restart||This command stops and then starts the GitLab Runner service|
|gitlab-runner uninstall||This command stops and uninstalls the GitLab Runner from being run as a service|
|gitlab-runner exec||To see a list of available executors, run|
|gitlab-runner –help||Check a recent list of commands by executing|
|gitlab-runner run –help||can see the name of the environment variable|
|gitlab-runner –debug||To run a command in debug mode|
|gitlab-runner exec shell||To see a list of all available options for the shell executor, run|
GitLab Runner Linux
Follow the below steps to download GitLab Runner on Linux Environment and go through the below commands to verify and register a project.
- sudo curl -L –output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64 [Simply download one of the binaries for your system]
- sudo chmod +x /usr/local/bin/gitlab-runner [Give it permissions to execute]
- sudo useradd –comment ‘GitLab Runner’ –create-home gitlab-runner –shell /bin/bash [Create a GitLab CI user]
- sudo gitlab-runner install –user=gitlab-runner –working-directory=/home/gitlab-runner [Install and run as service]
- sudo gitlab-runner start
- After doing the successful installation now GitLab Runner started and can perform to do the registration of the repository.
|sudo gitlab-runner register||Register the project with CI/CD|
|sudo gitlab-runner start||Start the runner|
|sudo gitlab-runner stop||Stop the runner|
|sudo gitlab-runner status||To know the status of gitlab-runner|
|sudo gitlab-runner unregister –name test-runner||Unregister Runner of a project and replace test-runner with your runner name and this name can be found inside the config.toml file (where your gitlab.exe) available|
|sudo gitlab-runner unregister –url http://gitlab.example.com/ –token||Remove Runner by Url and token|
|sudo gitlab-runner unregister –all-runners||Unregister All Runners|
|sudo gitlab-runner restart||This command stops and then starts the GitLab Runner service|
|sudo gitlab-runner uninstall||This command stops and uninstalls the GitLab Runner from being run as a service|
|sudo gitlab-runner exec||To see a list of available executors, run|
|sudo gitlab-runner –help||Check a recent list of commands by executing|
|sudo gitlab-runner run –help||Can see the name of the environment variable|
|sudo gitlab-runner –debug||To run a command in debug mode|
|sudo gitlab-runner exec shell||To see a list of all available options for the shell executor, run|
CI/CD Windows Project Configuration
Follow the below steps to configure CI/CD with Windows Environment projects:
- Go to the downloaded gitlab-runner .exe path, where you have stored it, and open the command prompt. (follow above windows command)
- Check the gitlab-runner status, stop it if it is already running before registering any new project with CI/CD
- Now type, gitlab-runner.exe register and fill the below details
- Enter your GitLab instance URL: Go to GitLab Account -> Select Project which you want to implement CI/CD -> Settings -> CI/CD
- Enter the gitlab-ci token for this runner: Go to GitLab Account -> Select Project which you want to implement CI/CD -> Settings -> CI/CD.
- Enter the gitlab-ci description for this runner: put any name (for ex: MyRunner1) and this just to remember which runner’ is running.
- Enter the gitlab-ci tags for this runner: This is tag to start the CI / CD can put any name or no name it’s not required (for ex: my-tag, another-tag)
- Enter the executor: need to select one executor from the above-given list of executors and every executor has their own characteristics.
- Now the project is registered successfully and start the GitLab runner.
CI/CD Linux Project Configuration
Follow the below steps to configure CI/CD with Linux Project.
- Go to your Linux machine, where you have installed the GitLab Runner and Git
- Check the GitLab runner status, if it is already running stop it before registering any new project
- Type sudo gitlab-runner register -> to register the new project with CI/CD
- Now follow all the below steps, same as Windows, and fill in the required information
Add Yml File in Source Code
This is an important step to include yml file to your source code directory, if you do not include it CI/CD will not work. This file includes a configuration that how this project will run.
For Example, .Net project Yml File format
MSBUILD_PATH: ‘C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\MSBuild.exe’
– ‘& “$env:NUGET_PATH” restore’
– ‘& “$env:MSBUILD_PATH” /p:Configuration=Release /clp:ErrorsOnly’
– ‘& “$env:MSBUILD_PATH” src\CiCdExample\CiCdExample.csproj /p:DeployOnBuild=true /p:Configuration=Release /P:PublishProfile=FolderProfile.pubxml’
expire_in: 20 days
– ‘& “$env:XUNIT_PATH\xunit.console.exe” “$env:UNITTEST_FOLDER\CiCdExample.Tests.dll”‘
– ‘xcopy /y /s “.\src\CiCdExample\bin\Release\Publish\*.*” “C:\Tools\inetpub\wwwroot\ci-cd-example”‘
C/C++ Yml Project format
– cd sourcecode
– export CXX=/usr/bin/g++
– export CC=/usr/bin/gcc
– chmod -R 777 *
expire_in: 365 days
– ls -lrt sourcecode/testdata_1.0
– cd testDir
– chmod -R 777 *