API: new method setupWhereFailed

Sometimes the small things are the ones that make the greatest difference.

So with python-opsi comes a new method setupWhereFailed that takes an product ID as parameter and then will set that product to setup on all clients where that product has failed as action result.

I used this during some development tests and this made things very straight forward.

Here is how you can use it through opsi-admin with the product testproduct:

opsi-admin -d method setupWhereFailed testproduct

Quick tip: show certificate data

A small tip for anyone who needs to check what values their certificate uses. For the certificate renewal in opsi-setup --renew-opsiconfd-cert we want to reuse the values existing in the current certificate and have some code in python-opsi we can leverage now.

With the following command we are able to load the data from the certificate and print it to our shell:

python -c "import OPSI.Util.Task.Certificate as c; from pprint import pprint; pprint(c.loadConfigurationFromCertificate('/etc/opsi/opsiconfd.pem'))"

On my test system I get an output like the following:

{'commonName': u'nikosserver.test.local',
 'country': u'DE',
 'emailAddress': u'niko@foo.bar',
 'locality': u'Mainz',
 'organization': u'Ev1l C0rp',
 'organizationalUnit': u'Katzen',
 'serialNumber': 12345678901595886952L,
 'state': u'RP'}

Improving tests with any()

There are many gems inside the standard library of Python and I think the slogan batteries included is a great choice. But not many are that known and sometimes features get re-implemented because the function from the stdlib is not known.

While refactoring some tests I found code like the following:

found = False
for line in result:
    if line.strip():
        if 'admin users' in line:
            found = True

self.assertTrue(found, 'Missing Admin Users in Share opsi_depot')

That is a lot of code to check if a line with some required text is inside the iterable result.

This brings us to todays gem any()! If any item of the iterable passed to any is truth-y then the function returns True. If the iterable is empty or no truthy value is found we get a False.

With this knowledge we can shorten the test code to the following:

self.assertTrue(any('admin users' in line for line in result if line.strip()), 'Missing Admin Users in Share opsi_depot')

I have been even going so far as to throw out the self.assertTrue coming from unittest.TestCase as this makes a conversion to a different test framework easier and the test is still nice to read.

We can also remove the check if the line is filled (line.split()) because there is no chance of a false positive introduced by this.

Now we end up with this:

assert any('admin users' in line for line in result), 'Missing Admin Users in Share opsi_depot'

And if you ever want to check if all values in an iterable are truth-y you should check out the function all!

A Jenkins- and opsi-based test environment

opsi currently runs on 15 different Linux distributions. For a long time we didn't have automated tests. Most were tested manually. Usually Ubuntu and Debian were tested more intense, because we use these distributions internally.

We then started to build a more complex test environment including functional tests. This test environment uses Jenkins, opsi and small scripts. Jenkins introduced the pipeline plugin with v2.0. With this plugin it is fairly easy to build complex tests with a good overview. opsi is higly integrated into theses tests.

Each test consists of the following build steps:

  • install linux distribution
  • install opsi on this fresh machine
  • perform basic tests (logrotate, opsi-convert, etc.)
  • install a Windows netboot product
  • add a client to the new opsi server and install Windows (7, 8.1 and 10)
  • test for basic installed products on new Windows machine

The above image shows the graphical output of each stage of the jenkins pipeline. Whenever an error occours the step turns red and the pipeline aborts.

To simplify the steps of installing opsi, test it and then run the Windows installation we have seperate products. l-opsi-server installs opsi on a linux machine. It detects the distribution and installs the opsi packages for the distribution. The test product tests this machine for opsi commands like opsi-convert, opsi-backup, opsi-set-rights etc.. The client product adds a client and installs Windows netboot products. Additionally it fills the installfiles directory and sets this product on setup. A script then starts the virtual client and waits until the netboot product is installed, along with the opsi-client-agent. This step performs three times with Microsoft Windows versions (7, 8.1 and 10).

These tests are orchestrated by an opsi-server. We have three different stability levels. Therefore three opsi-servers with different package versions, matching the current stable, testing and experimental versions. The opsi-servers also just install package versions from their stability level on their client machines. This way we can check if our tested products can be released to the next stability level or not.

Please note that our test environment is currently work in progress and not all released packages are tested with it. We plan to integrate more and more products in the near future.

Checking the webservice with curl

One of the cool things of using open protocols and interfaces is that you are not limited to use special tooling and can resort to tools you are familiar with. No need to use some unknown programming language or to throw huge amounts of money at some company to get access to the data in your system. This post will explain how you can access the API of your opsi server with curl.

Read more…

Exit-Codes for opsi-admin

Since opsi-utils opsi-admin will fail with appropriate exit codes when used non-interactive to make writing scripts and reacting to failure easier.

The old behaviour was that every call in non-interactive mode - usually by supplying -d - returned an exit-code of 0. This made it very hard to check if something went wrong during such a call.

The new opsi-admin will return an appropriate exit-code when things fail:

  • Exit code 1 will be returned whenever the call itself is problematic - unknown method or wrong/missing parameters.
  • Exit code 2 will be returned when the call to the API was correct but the executed method did return an error.

If you rely on the old behaviour where everything returned exit code 0 you now need to supply --exit-zero to your call of opsi-admin.

Hello World

For many Hello World is the first thing they type when trying a new system.

Having a blog centered around opsi for sure is something new. Therefore this is our hello world!

Opsi - as most of you probably know - is a reliable and flexible open source client management tool. We, the opsi developers, want to use this blog as a platform to allow for a look into the development, for sharing tips and insights into the various facets we experience every day.