In modern software development, continuous integration (CI) and continuous delivery (CD) are essential factors to be considered. Software moves very fast and demands more from developers than ever. CI/CD helps developers ships higher quality software faster but what does successfully CI/CD implementation look like and how do you know you are on the right track? if these problems hit a little too close to home, stay tuned.
This piece has two main goals:
- To be a standalone introductory guide to CI/CD pipeline with GitLab
- To form part of a larger series of articles where I'm going to dive deep into:
- Fundamental concept of CI/CD workflow
- Gitlab CI fundamental
- More Advance concept of CI/CD workflow
- And some specific DevOps topics
You don't need to know anything about GitLab I'll walk you through it. However, before you begin this tutorial you will need the following:
- Nodejs and NPM installed on your system
- Some basic version control skills
- Basic experience with docker will be helpful
- Basic experience with Linux, Linux commands, and using the terminal
- Basic understanding of yml file format
GitLab is a web-based DevOps lifecycle tool that provides a Git-repository manager providing wiki, issue-tracking, and continuous integration and deployment pipeline features, using an open-source license, developed by GitLab Inc.
Gitlab has what is called the GitLab server which allows you to create a repository and manage all your projects and everything you do will be saved in their database.
As soon as you create a pipeline that pipeline will be managed by the GitLab server and delegate to the GitLab runner the GitLab server ensures that the GitLab runner is picking up the jobs, and the outputs are saved and so on. This architecture allows for scalability because you can add as many runners as possible depending on your project needs.
Why Gitlab CI
Maybe you are wondering why you should use GitLab among other software like Travis CI, Circle CI, Jenkins e.t.c or what exactly is Gitlab. Gitlab is a modern tool and is continuously being updated and improved and the market share is growing rapidly. Gitlab has a :
- Simple scalable architecture
- Docker first approach
- pipeline as code
- merge request with CI support
There are a lot of features built into GitLab which works really great.
What Is A Pipeline
Let's make a quick analogy to an automobile assembly line. In other to build an automobile you need to follow a couple of steps, for example, you need an automobile body, trunk, then you install the engine, and then the wheels so on. As you can see, there are few characteristics of an automobile assembly line which are a series of steps that need to be done in a certain order. So you cannot start adding the wheels until you have the body and the engine so we can say the result from the previous output is the input for the next step and some steps can be done in parallel, for example, you can fix the side mirror at the same time. Before the car can leave the factory, it has to be tested to ensure that everything works as expected so the end goal is to get the product out of the factory in good shape.
Now you will notice that building software is quite similar to building a car and this is what GitLab is supporting us to do with our products. Now let's try to build something with GitLab CI
Setting Up A Gitlab Pipeline
From the analogy, we have two stages the build and the test stages so the first step to get started is to register with GitLab. Head over to Gitlab
You need to sign in if this is your first time after that click on "create a project"
I call it "Toyota assembly plant" then click on "create project". Next, let's create a pipeline, and the way we do this is by creating a new file in the project we just created so click on the "new file"
And the name of the file is always .gitlab-ci.yml (yml is a special file format that we define our pipeline in).
We need the .gitlab-ci.yml file in the root of the repo which defines the structure and determines what to be executed using the GitLab runner.
Next, we need to add jobs to our .gitlab-ci.yml file (job serves as a fundamental layer of the .gitlab-ci.yml jobs are picked by runners and executed within the environment of the runners.
job 1: script: "execute script for job1" job 2: script: "execute script for job2"
So let's create create our first job in .gitlab-ci.yml file
Here, "build Toyota car" is our first job and the script specify the steps needed to produce a Toyota car in our own case:
- make a directory called build
- Change our directory to build
- Then we create a file called toyota.txt and we use echo command (echo command is used to display line of text/string)
- And we append the word "body" and "engine" to our toyota.txt file (with the >> operator the output from the echo command will get appended to the toyota.txt file.
So let's save by committing our changes click on the commit button and write a commit message and commit your changes. As soon as we commit the file, GitLab CI will detect our file. You can navigate to CI/CD select Pipelines and you can see your job either running or has finished running you can click on that specific job to see details
The tool that is executing this script is called the GitLab runner. So let's add another job which is to test our car so navigate back to .gitlab-ci.yml file and click on edit so let's create a new job under the first job in my case I will call it "test Toyota car"
- the test command is used to verify that the file toyota.txt was created and the -f check if the file is a regular file.
- next we want to check if the words that we append in the toyota.txt file in the first job exist and we do this using the grep command (grep command is used to search for a string of characters in a specified file)
We need to specify in which order these jobs need to be executed because by default GitLab runner will run then in parallel but in our own case the second job depends on the first job.
So in other to specify the order in which the jobs should be executed we need to define stages and the other of the stages are important.
stages: - build - test
Next, we specify the job belonging to each stage by defining stage and the name of the stage in each job
stages: - build - test job 1: stage: build script: "execute job 1" job 2: stage: test script: "execute job 2"
Let's commit our file and check on the running jobs you can navigate to CI/CD select Pipelines and check on the jobs. You will notice that our second job failed.
It seems our build directory is not present in the second job this is quite normal because what happens is that the jobs we are running are independent of each other and they don't exchange any data if not told to do that, so after the first job terminate all files are immediately removed and the environment where the first job was created was immediately destroyed.
So let's get that fixed in other to tell the first job what to save we need to define an artifacts (artifacts are a list of files and directories created by a job once it finishes) and specify where it is located using paths and we can save the entire build directory. These build folder will now be saved after the first job's environment is destroyed so that any jobs can make use of it.
Finally our two jobs passed
Our build directory is created in the pipeline and is save as artifacts that are independent of the git project so that is why the folder is not committed in your repository but is available for download. This is how you use GitLab to create a simple pipeline with two stages.
So far, you have learned how to create a CI/CD Pipeline using Gitlab CI. The intention of DevOps is to create better-quality software more quickly and with more reliability while inviting greater communication and collaboration between teams. Subsequent articles will focus on real-world CI/CD from development to production.