opsi webservice with Tornado progress

Today has been quite pleasant as I've been able to make some important progress with the new webservice.

I've implemented parameter passing for RPC calls as done in opsiconfd. Now I can hook up an Python 2 JSONRPCBackend to the service running with Python 3 and everything works as expected. This means the core functionality we need from the webservice is working properly.

We still want the new webservice to replace the old one without clients having to alter their behaviour. Another step in this direction is the possibility to parse opsiconfd.conf and use the given settings. opsiconfd has some options for limiting access that haven't yet been implemented.

Prior to this I've made some changes in python-opsi that fixed authenticate through PAM which I sadly broke before. Along this way I learned that from the multiple Python PAM libraries you can find pam3 on PyPI does not have any releases. Bummer. The world of Python module appears as a jungle to me once you go away from PyPI and try to find the macthing package for your Linux distro. There are quite a few similar sounding packages but finding out what version is available with what name isn't a pleasant task.

However having the service this far means that it is time for me to create a proper OS package. Other dependencies of opsi-server have already been ported to Python 3 and are already packaged with the only missing package being the webservice. Keep your fingers crossed that all dependencies are now matching the ones downloaded from PyPI during development!

Introducing opsirc

For many opsi user opsi-admin provides the go-to tool for scripting and even regular work on their system. Usually this makes use of the -d argument in order to not get asked for a password. But what if you want to go through the webservice instead - you don't want to use -d.

One reason for this might be that you want to have logs about what happened. You still can configure opsi-admin to write logs to a location of your choice but you have to think about this every time.

One thing that probably bug you is that you want to go through the webservice but the requirement to type in a password could be an annoyance. The options you have is that you use --password but this means that the password may be visible in the process list. Various other means of passing the password to the prompt may help but this would also mean that you may have to edit a whole lot of scripts if you ever change the password.

For sure you could whip out your favourite programming language and create scripts to do all your tasks but this might end up as not being as flexible and sometimes even overkill if one simple call is all you need.

To make your - and our - live a little bit easier there is now the possibility of providing credentials in a specific file we call opsirc. This file may contain username, password and address in order to have an easy way to benefit from going through the webservice while scripting - or just making use of the webservice by default.

An example might look like this:

username = grace
password = sundown
address = https://opsi.casa.hanson:4447/rpc

Save the contents at ~/.opsi.org/opsirc and opsi-admin (4.1.1.30 or newer) will pick the information up and use it for an connection. You can now run an command like opsi-admin method backend_info without the need to enter a password.

The opsirc file is kept simple so that other tools have the possibility to make use of this aswell. None of the lines are mandatory. A minimum usable example would be only providing a password in the file. And it is possible to save the password in a separate file by using the key password file and the path to the file as value. This makes it possible to show the file to others without presenting your password in plain view.

I hope this makes everyones live a little bit easier!

Looking back to 2018

The new year is now a few days old and I'd like to take to opportunity to look back at what happened in my opsi world in 2018.

The year begun with getting opsi 4.1 ready for release. Most of the development work was already done but for a proper release you need some more. Documentation is one of the things. Not only did we need to change the documentation in some places, we also worked hard on making a reliable and easy to follow migration guide. The migration guide is quite large, covering many distributions and mentioning all kind of changes that could be done, but the things you need to do come done to a few steps.

The release has also led to writing some tools that are in use at our public repositories. Behind the scenes we have tools that help us making the release of new opsi packages an easier task by automatically collecting the corresponding files and knowing where the files need to be placed (localboot/netboot; linux, windows or both). Since a lot of the packages are runnable under opsi 4.0 and opsi 4.1 we wanted to have these packages available in both repositories. We solve this by having a tool watching the opsi 4.0 repository directories and once there is a new file this will be linked to the corresponding repository for opsi 4.1. This is accompied by another tool that watches all our repositories and alters the access rights of new files so that retrieving won't fail because of insufficent rights. This is one of the simpler and still nicer things I'd like to integrate into opsi in the future because it removes a potential error source when handling packages.

From my view the release of opsi 4.1 went nicely. Problems seemed to be rare and seemed to come mostly from diverging from the migration steps. The order was important this time because we changed the way we are tracking what migrations have been run. This will save us some headaches in the future since we know what migrations have been run and therefore what migrations are missing.

I enjoy that we are able to rely on systemd and Python 2.7 with opsi 4.1. This makes a lot of things easier for developing and maintaining things.

The release of opsi 4.1 happened shortly before the great opsiconf. The conference itself was a great joy to me. I got to see new and old faces and talk to many people - and I've talked to even more if time would have allowed it! One thing that I'd like to improve with the next conference is the diversity. I think this is something where we really could do better but I'm unsure of how to achieve it. I know that we have a very diverse user base and I'd love this to be represented a lot more at the next conference!

With release and conference behind me it was again time to look towards the future. One of the major points we started working to is the migration to Python 3. Since I've been constantly blogging about this topic I won't go into detail here. Being able to focus on this wouldn't have been possible if there weren't my co-workers giving the time and taking care of other tasks I'd normally do. I'm very thankful for this because this allowed for making good progress. Lately some more things have come up that I personally had to take care of and this has shown me even more how valuable the time has been.

Another big change was the overhaul of data collection for opsipxeconfd. It took us quite some time to get to this solution but I am happy how this turned out. The duration it took to get there is one thing not bringing a smile to my face but if you are stepping into a new terrain you won't always know what waits for you there.

In between all this there has been the occasional bugfix that lead to touching opi 4.0. This is where the migration from subversion to git really payed off. Maintaining multiple versions and bringing changes from one into the other has become mostly a no-brainer. Because we are already working on a new larger release for opsi this means that most repositories are having at least three different version branches at the same time. Before we went all in on git we tried some workflows and finally came up with our one which has been heavily inspired by git flow.

The winter of 2018 has seen some work on the monitoring extension I quite enjoyed because there was a constant stream of feedback and even though most changes were rather small I had the feeling that these changes made a rather large improvement.

And then things slowed down before the year came to an end. And so did the support span for opsi 4.0.

New things are waiting for us in 2019, new things will be coming in 2019. I am looking forward to this and hope you do aswell!

Quick tip: Easy weekly backups with systemd

Creating regular backups does not have to bee a cumbersome job but can be easily automated. Instead of using cron I have settled to use systemd for such tasks. In my eyes the benefit is that I can still see any job output after it has been run by using either systemctl status or journalctl without the need to configure any logging whatsoever. And I have easy access to things like execution time or exit code.

Read more…

First service endpoint running with Python 3

I am really happy with the progress we are making with supporting Python 3. Our internal build server already serves the first packages that are running on Python 3 only. There are still errors popping up where we need to port parts but this is what I expected earlier.

The previous weeks saw some feedback-based improvements to the opsi monitoring connector in opsi 4.1. Because I was already working in the codebase and it is comparably small I decided the first endpoint I want to port to Python 3 will be /monitoring.

Read more…

New appliance based on Ubuntu 18.04

For a very long time we have been providing a preconfigured opsi server (often referred as opsivm) in the form of a virtual machine, which can be integrated under various virtualisation systems including Virtualbox and VMWare. After the server has been started for the first time, a script is executed to setup the last configuration settings. These are, among other things name, domain, IP of the server, the standard gateway, the DNS server and also the passwords for the already created users. The opsi server is then ready for use and only needs to be filled up with the opsi standard packages.

We always try to keep the opsi server up to date, both in terms of the opsi version and the Linux distribution used. The latest version has been changed from being based on Ubuntu 16.04 to Ubuntu 18.04.

Network configuration

Since Ubuntu 18.04, netplan has been used instead of ifconfig as the default network configuration. After a first attempt to disable netplan and keep the configuration procedure using ifconfig as before, this attempt has been discarded because this solution would require installing additional programs as well as making more adjustments. So the decision was made to use the netplan configuration.

The configuration for netplan can be found in /etc/netplan/config.yaml:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0:
      addresses:
        - 10.10.10.2/24
      gateway4: 10.10.10.1
      nameservers:
          search: [mydomain, otherdomain]
          addresses: [10.10.10.1, 1.1.1.1]

By making adjustments to our start script we are able to use a template based on the default configuration. This template is patched with the values as retrieved during the first startup.

network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0:
      addresses:
        - @ip@/@cidrnetmask@
      gateway4: @gateway@
      nameservers:
          search: [@domain@]
          addresses: [@dns@]

A particular challenge was the network mask, which is required to be in format 255.255.255.0, but in config.yaml must be given as in CIDR notation including the network address as 10.10.10.1/24.

Once the template is filled the following call activates the network settings from config.yaml:

netplan apply

Deploying a root shell

Another change was the way we made the use of a terminal with root privileges available via a desktop icon.

Most of the tasks on an opsi server can be performed by a dedicated user called adminuser. If system administrator rights are necessary, there is a desktop call to open a shell with root rights. So far this has been done using the gksu tool which is considered obsolete and is therefore no longer supported in the latest versions of Debian and Ubuntu.

For this reason we now start our root shell with the following call:

lxterminal --title "root shell" --command "sudo -s"

Testing the new opsi-appliance

Are you curious? Download the latest version here and then follow the Getting Started.

Have fun

Making parts of opsi-package-updater re-usable

This weeks release of opsi-utils 4.1.1.26 consists mostly of internal changes. We sometimes do this to keep up with current developments but this time it is a little bit different. This change lead to shrinking the size of opsi-package-updater by a huge margin.

How did this come you might ask. Well, it all started a while ago when a customer got in contact through the support because he wanted to use some functionality from opsi-package-updater with his own script. Discussing the options it came clear that the best way to achieve this was to make sure that the functionality of opsi-package-updater becomes available for importing in another Python module. Luckily the customer was up for this route even though this would not be the fastest option but the one that provided the best long-term support for such an solution.

To achieve this we added the module OPSI.Util.Task.UpdatePackages to python-opsi. This now includes the majority of code that is opsi-package-updater. Besides becoming reusable we now also can provider proper unittests for the different components. Working on the module I have split some components up to achieve looser coupling. I'd like to refactor some internal workings of this in the future but right now my focus is on opsi supporting Python 3.

This brings us to the other changes that happend in opsi-utils. The small changes that have been made are to achieve better compatibility with Python 3. For most tools it will now be enough to change the shebang line to use Python 3 to work with python3-opsi. Some more work will have to be done there but we can already cross more things of our todo list.

opsi-package-updater: Failing on locked products

With an software as old as opsi it is sometimes inevitable to stumble over something and then be left scratching your head.

One of this cases I recently discovered in opsi-package-updater. Whenever an package is installed it is made in a way that ignores the lock of the product on the depot.

The lock is in place to avoid problems where multiple tools would (un)install the same product. This could result in problems which leave the system in an undefined state regarding the product installation - something that should be avoided!

With opsi-utils 4.1.1.24 we do not ignore the lock anymore. If a locked product is detected no installation will be made but we will abort with an error instead. This may lead to problems coming to the surface which may have been hidden before, especially when having a heavily automated setup, but we think not knowing about the problem is far worse. The problems probably have been there before but were usually not in sight.

If you run into such an problem now you should take a look at the logs and find out what is causing your problem. Once you figured this out and resolved it you can use opsi-package-manager --install --force /var/lib/opsi/repository/<packagefile>.opsi to make install the package in way that ignores the locked state of the product. If everything works you can run opsi-package-updater again to finish what was started.

Improved backup handling

Small things do make a difference. This weeks release brought some small changes to opsi-backup that should make the handling of opsi backups easier in day-to-day life.

The first change improved automatic detection of restorable data. There is no need anymore to pass what backends should be restored as the new mechanism will by default restore all backend data that is found in your backup file. This boils a restore for most cases down to opsi-backup restore <backupfile>.

The second change is that if any combination of options is given that would lead to not performing a restore the process will be aborted which is much easier to spot.

And the last change is that it is now possible to get a short listing to see what data (backendtype and if there is configuration data) is in a backup with the command opsi-backup list <filenames>

Go give it a try and send us some feedback! All you need is opsi-utils 4.1.1.22 or newer which can currently be found in the testing branch.