CI/CD Python project with Gitlab

DevOps is the new trend, and developers must embrace or .... well you know.

This is the kind of article I just read this morning, and I must admit that it is indeed clearly a trend. As a Python developer, I try to do my best to be TDD as much as possible. However one thing was still missing until I switched to Gitlab : Continuous Integration and Automated Testing. Gitlab offers a very convenient pipeline for your projects and quite easy to setup. I then created a dummy project. All you need to do, is adding a YML file to your project and you're done !

Setup

Imagine you have a running project with tests associated and you want to run tests automatically after each commit on the master branch. The project structure looks like this

Code

For this dummy project, we'll use very useless code just to illustrate our point. Source code can be found here. The app.py  and test_app.py  code file can look as follow:

src/app.py

tests/test.py

Gitlab config file

Now, let the magic begins. All you need, is adding a .gitlab-ci.yml  file to the project root. For more informations on how to write the YML configuration file, please refer to the documentation.
For our simple case, this file is quite short:

Let's now describe the sections of the file

image

Pipelines are built on a Docker image. You can then specify any image you want, from public ones to private custom ones. And that's really cool as Gitlab offers a Docker registry for your projects ! For our dummy example, we simply use a light Python 3.6 image.

before_script

This defines a script (bash) that must be executed before all jobs. In our case, we just print the Python version for debugging purpose.

stages

That's where we define the order the different jobs have to be executed. All jobs associated to a stage are run in parallel, and each stage is executed sequentially. If a job fails, subsequent stages/jobs are not executed. In our case, we just defined a single stage test , but in real life you might probably want multiple stages, for example test, staging , deploy  etc.

test_job

Here you can provide any name for your job. I choose test_job . The rest of the file is then the list of every jobs your CI can run. A job must have at least a script  clause. You can refer to the documentation for more details on the clauses you can use. In our case, we first specify the stage. As we only have one stage, it is straightforward. Then the core of the file is the script  clause. It is a shell script and can contain multiple lines. Then script of our project prints Running tests  on screen (useless, but just for the demo), then runs the unittests we defined in the tests  folder. We allow here Python to discover the tests to run.

Final word

And that's all ! After that, every commit will trigger the pipeline. Of course, you can customize to run only certain jobs on commit of a given branch of your project. You can find details on the documentation page.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: