Package build deployment with GitLab CI/CD – Part 1
This post is written by Tobias Friede and Christopher Köhler from the Fraunhofer Institute for Wood Research – Wilhelm-Klauditz-Institut WKI in Braunschweig, Germany.
About us
Wood is a traditional raw material with a future. Wood products have outstanding technical properties and exemplary life cycle assessments. In six scientific specialist departments, the Fraunhofer Institute for Wood Research, Wilhelm-Klauditz-Institut WKI in Braunschweig addresses current and future-oriented tasks concerning the use of wood and other renewable resources.
The Institute, founded in 1946 by Dr. Wilhelm Klauditz, is located in Braunschweig, Germany. In 1972 the Institute, which counts among the most significant research institutions for applied wood research in Europe, joined the Fraunhofer-Gesellschaft.
The Fraunhofer WKI works as closely and as application-oriented with the companies of the wood and furniture industries and the supplier industry as it does with the construction industry, the chemical industry and the automotive industry. Virtually all procedures and materials which result from the research activities of the Institute are used industrially.
The local IT department of six employees is responsible for the IT infrastructure and clients on the joint campus of the Fraunhofer WKI and the Fraunhofer IST in Braunschweig. We manage around 600 clients with opsi, which we have been using since 2012.
Intro
In recent years, we neglected our opsi server and package deployment. The upcoming end-of-support for Microsoft Windows 7 and the "new" Windows 10 came with a big workload for us. We had to create new netboot products, test the opsi UEFI module (which is working pretty well) and rework our existing packages. Until mid-2017, we only wrote the scripts locally on our own devices and copied the files to the workbench where we build the packages manually. To improve the complete process of creating and maintaining product packages, we took a look at GitLab. With GitLab, respectively git, we plan to structure our development process. Just a simple branch and let’s go. Now we are able to continue the development of packages which were built by another colleague. With the commit messages we are able to review the recent edits.
How to automate the build process
Later, we took our time in testing a little with the GitLab CI/CD. Our current process is pretty simple:
We are still scripting on our own devices with the preferred text editor. After editing, we push the
changes to our GitLab server. The GitLab CI/CD then differentiates between two branches: dev and
master. Each commit to dev-branch then triggers a shared runner on the opsi-testing server which
clones the project and pulls the latest changes from the branch. Afterwards, it automatically creates
an *.opsi package. This package is copied to the GitLab server which saves this file as an artifact for
two days. In the next step, the shared runner pulls this artifact again and installs it on the test server.
Now we are able to test the install and uninstall routines on some client test VMs.
If the tests are successful, you can create a merge request to merge the changes from the dev branch to master branch. In our environment this will NOT trigger anything! We created a rule that the changes are only deployed to our opsi-stable server when a new tag is created in master branch. The tag has to be in a specific syntax. We use PRODUCTVERSION-PACKAGEVERSION to automatically change the control file and then build and install the package on our opsi-stable server. If this job is successful, we can deploy the new product or new version for the clients.
How to automate this process with a package
For each product, we create a new repository in our GitLab. All products are retained in the group
OPSI. So the repository name will look like OPSI/productname. A specific file with commands for
the process is needed in each repository. This file is named .gitlab-ci.yml. We "enhanced" it a
bit. When you want to improve or change the process, you don’t want to change the file in each
repository. We created an own repository OPSI/gitlab-ci.yml which contains three files:
gitlab-ci.yml (NO leading dot)
OPSI-gitlab-ci.yml
README.md
The gitlab-ci.yml file is the template which needs to be copied into each new repository. With the content of this file, we include the OPSI/OPSI-gitlab-ci.yml where the important commands will be applied.
listings/package-build-deployment-with-gitlab-ci-cd/gitlab-ci.yml (Source)
Currently, the OPSI-gitlab-ci.yml file is split into four stages: * build-dev * deploy-dev * build-release * deploy-release
In the build stages, we clone or update the local repository on the respective shared runners. After pulling the latest commits, it builds a new package and installs it.
listings/package-build-deployment-with-gitlab-ci-cd/OPSI-gitlab-ci.yml (Source)
Future improvements
Currently, we still have to test the products on our test VM manually. We are working on an improved script which will automatically install and uninstall the products on a specific test VM. The runner will also collect any log files and attach them to the GitLab Job. Any error state should end in a failing GitLab job. In the end, you can see a successful installation and uninstallation procedure directly in GitLab without connecting to opsi-testing and testing the package manually.
We plan on following up with another post when we have finished these improvements.