opsi-[linux-|mac-]client-agent 4.2x

Over the last months we have worked intensively on a new generation of opsi-client-agents. With the release on 05.08.2021 we have published them under "testing". This blogpost is intended to highlight their new features, as well as changes in their structure and handling.

  • The opsi-client-agents 4.2.x are compatible with opsi 4.1, but some of the new features, like providing the installer, only work with a current opsi 4.2 config server.

  • All Windows versions starting from NT6.0 are supported by opsi-client-agent 4.2.x, older Windows versions are no longer supported.

  • The installation procedures of the different modes (OS installation / service_setup / opsi-deploy-client-agent/ update via service) have been unified and simplified. Installations of the opsi-client-agent now always take place in service context. For OS installation, opsi-deploy-client-agent and service_setup, a connection to the opsi service is now established first, so that all configurations, product properties, etc. can be retrieved directly from the opsi service. This eliminates the need to use config files (e.g. config.ini) and duplicate installation as part of the OS installation.

  • Additionally there is now a comfortable installer to install the opsi-client-agent without the need to connect to the opsi depot share first. This installer is provided by the opsi config server and can be downloaded from https://<config-server>:4447/public/opsi-client-agent without authentication.

  • The installer and the service_setup now use an installation helper. This finds available opsi 4.2 config servers in the LAN via Zeroconf automatically.

  • The installation and update of the opsi-client-agents usually do not need a reboot anymore.

  • The action processor opsi-script (successor of opsi-winst) is now included (also for windows). To enable updates of the action processor, the opsi package "opsi-script" should be installed on all depots.

  • The opsiclientd included in the opsi-client-agent now uses a server certificate issued by the configserver via the "opsi CA". Clients that trust the "opsi CA" also trust the certificate of the opsiclient (client port 4441).

  • File permissions under windows are set via icacls, which makes the process now much more performant.

  • opsi-client-agent, opsi-linux-client-agent and opsi-mac-client-agent have been aligned so that their structure is now (largely) consistent. For windows and linux an extensible postinst mechanism is available to install the respective opsi-client-agent after netboot installation and to make adjustments if necessary.

  • The structure of the client-agents is now so that under CLIENT_DATA directly the setup.opsiscript is located, which regulates the installation procedure. In addition opsi-deploy-client-agent (and opsi-deploy-client-agent41 for compatibility with very old server operating systems under opsi4.1) and the oca-installation-helper are located there. The latter is used by service_setup (, silent_setup) and the installer and assists in setting the configuration for a new client. custom files, which should overwrite the defaults, can be stored under CLIENT_DATA/files/custom.

  • On Linux, opsi-script and opsi-client-systray are now compactly located under /opt so that they can be updated efficiently - the update_action_processor thus also extends to the opsi-script libraries, default skin and locales.

  • The opsiclientdguard which previously existed to restart the opsiclientd service in case of a crash is no longer necessary and has been removed from the opsi-client-agent (on windows).

These changes were made with the goal of making the client-agents simpler, more efficient and easier to maintain. We hope that you like the new features and we always appreciate your feedback and suggestions.

opsi packages that support Python 2 and 3

The work on opsi 4.2 is ongoing and the switch of Python versions means that we also have to look for the helper scripts that come with our opsi packages. There is a simple pattern I follow to make sure that we can use a package with opsi 4.1 and 4.2 I will show here.

First I make sure that the script runs fine with Python 2 and 3. As the standard library of Python changed you can use a tool like 2to3 that will aid you in changing the code the be working with Python 3. I won't go into detail about this here because there are a lot of resources on the internet on how to deal with these changes - python3porting is one I like a lot. Scripts that make use of the OPSI package (provided through python-opsi) are usually without any need to change imports or usage.

When we have a script that can be run with Python 2 and 3 we are changing the used interpreter. These helper scripts are executable and specify the used program through the shebang line - this is the first line that usually starts with #!. I will adjust this so our script is running with Python 3 by using the following: #!/usr/bin/python3

This makes the script running with Python 3 by default. For Python 2 we will modify the file to run with the older version instead.

To achieve this I add the following to the OPSI/postinst of the package:

# Patching scripts to work with opsi 4.1 and opsi 4.2
set +e
python3 -c "import OPSI"
ret=$?
set -e
if [ $ret -eq 0 ]; then
        # Python 3 - opsi 4.2 or later
        echo "Running on opsi 4.2 or later. Nothing to do."
else
        echo "Running on opsi 4.1. Patching scripts..."
        # Python 2 - opsi 4.1
        sed --in-place "s_/usr/bin/python3_/usr/bin/python_" "$CLIENT_DATA_DIR/my_script.py"
fi

Once support for Python 2 is completely dropped we will be able to simply remove this part from the postinst and call it a day.

Public opsi 4.2 experimental repos

It's not a secret that we are working on getting a new version of opsi out there.

Previous versions of opsi have been mostly developed internally and were only released after they have been declared ready for a public release. With opsi 4.2 we are going a different route and decided to make experimental packages available earlier. Since nearly two weeks we have public repos for the OS packages on OBS. Be aware that these are still experimental and bugs will occur.

At uib we use these repos for our internals tests and since we have them set up more and more people started testing their components against opsi 4.2 which already helped me a tremendously amount in order to discover bugs or problems we need to tackle. We aim at providing an easy way to migrate an up-to-date opsi 4.1 to opsi 4.2 with only very few manual steps.

Right now we only support Debian 10 and Ubuntu 18.04 so we can focus on getting opsi 4.2 out of the door. Additional distributions will be added later. On the technical side it is important that they bring Python 3.6 or newer. Lastly our users demand for new distributions is important for us to be able to priorize what distro to add next.

HTTP headers are case-insensitive

During tests of the new webservice with some more real-life workloads I noticed that whenever a JSONRPCBackend is used to connect to the service each request generates a new session. But it should re-use the session ID provided by the service instead so that the session gets reused and methods like backend_setOptions work as intended for the currently used session. Since I've successfully used the session ID with requests and curl before the error had to be somewhere else.

Turns out that the custom HTTP response class we currently use in python-opsi does handle headers case-sensitive. While the old service returned the field as set-cookie the new one capitalized it into Set-Cookie.

The solution to this is to follow RFC 7230 and make the headers in our custom class case-insensitive. This should be coming with python-opsi 4.1.1.73.

Moving towards gzip

If you are following the development closely you might have noticed that gzip popped up in a few places.

One of the reasons we are moving more towards gzip is that Tornado offers easy support for gzip-compressed data transfer both to and from the server. This makes the move to Tornado faster for us because we do not have to implement deflate compression features for Tornado right away.

The opsiconfd from opsi 4 does also support gzip and with the latest versions it preferrably returns gzip even if a client would accept other headers aswell.

For an easier transfer towards gzip the JSONRPCBackend now is able to use gzip aswell. The parameter deflate and the method setDeflate are now both obsolete. Instead you should make use of the compression parameter or setCompression method. Why not have a look at the documentation?

If you are using the JSONRPCBackend please remember that you can switch to the new methods once you have updated to python-opsi 4.1.1.68 or newer!

As the (de)compression is based around HTTP headers it is simple to make your application work as desired by sending the appropriate headers. To get a gzip-compressed response from the server make sure to set your Accept-Encoding HTTP header to gzip. A response that is compressed will have the Content-Encoding header set to gzip.

Package build deployment with GitLab CI/CD – Part 1

This post is written by Tobias Friede and Christopher Köhler from the Fraunhofer Institute for Wood Research – Wilhelm-Klauditz-Institut WKI in Braunschweig, Germany.

About us

Wood is a traditional raw material with a future. Wood products have outstanding technical properties and exemplary life cycle assessments. In six scientific specialist departments, the Fraunhofer Institute for Wood Research, Wilhelm-Klauditz-Institut WKI in Braunschweig addresses current and future-oriented tasks concerning the use of wood and other renewable resources.

The Institute, founded in 1946 by Dr. Wilhelm Klauditz, is located in Braunschweig, Germany. In 1972 the Institute, which counts among the most significant research institutions for applied wood research in Europe, joined the Fraunhofer-Gesellschaft.

The Fraunhofer WKI works as closely and as application-oriented with the companies of the wood and furniture industries and the supplier industry as it does with the construction industry, the chemical industry and the automotive industry. Virtually all procedures and materials which result from the research activities of the Institute are used industrially.

The local IT department of six employees is responsible for the IT infrastructure and clients on the joint campus of the Fraunhofer WKI and the Fraunhofer IST in Braunschweig. We manage around 600 clients with opsi, which we have been using since 2012.

Intro

In recent years, we neglected our opsi server and package deployment. The upcoming end-of-support for Microsoft Windows 7 and the "new" Windows 10 came with a big workload for us. We had to create new netboot products, test the opsi UEFI module (which is working pretty well) and rework our existing packages. Until mid-2017, we only wrote the scripts locally on our own devices and copied the files to the workbench where we build the packages manually. To improve the complete process of creating and maintaining product packages, we took a look at GitLab. With GitLab, respectively git, we plan to structure our development process. Just a simple branch and let’s go. Now we are able to continue the development of packages which were built by another colleague. With the commit messages we are able to review the recent edits.

Read more…

The current state of opsi and Secure Boot

Customers regularly ask for the possibility to install their UEFI Secure Boot clients via opsi. Until now this was not possible. We started an investigation in the second half ot the last year. There are various ways to implement a Secure Boot installation via opsi. The first and not justifiable way is to deploy a so called machine owner key or MOK to every Secure Boot machine by hand. There is currently no way to deploy a MOK to a client in an automated way, which is understandable as an unattended added key might result in harmful signed malware. The second and selected way is to use a Microsoft signed shim which includes an uib public certificate. With this choice we have not the problem we would have with the previously mentioned MOK way. The customers do not have to deploy a key to their system by hand. With a Microsoft signed shim the system verifies the authenticity of a shim file based on the official signature supplied by Microsoft. Every chainloaded binary is then verified against the embedded uib public key inside the shim. So therefore we can sign our own binaries and they get accepted by the UEFI Secure Boot mechanism. In the end we olny have to get the shim signed by Microsoft and we can sign everything chainloaded by ourselves and don't have to rely on other instances. It is obvious why we decided to use the second possibility. The second way adds a lot of simplicity to deploying a netboot product to a Secure Boot enabled machine.

The way to get a Microsoft signed shim is long and not well documented. Microsoft hosts a semi automatic platform to sign uploaded files. However if one wants to get a shim signed by Microsoft additional steps are needed. RedHat hosts a shim-review repository on GitHub where the to be signed shim is reviewed by authorized people along with our specific changes. Microsoft will sign the submitted shim if the SHA256 checksums in the file and the shim-review match and the files gets signed by Microsoft. While it requires only a few simple steps it took months (in our case) to be completed because approval is done by hand Finally we have a Microsoft signed shim binary we can deploy on UEFI netboot clients without the need of deploying a MOK on every client.

Stay tuned to get more information on opsi and Secure Boot, as we will keep you updated.