Some faded old paper logs

Feeding log files to the systemd journal

Not everything integrates with the systemd journal. Here is how you can force-feed log files from programs that don’t work with systemd into the journal.

Lets assume you want to import the /var/log/awesome.log written by the awesomed example service process to the journal. awesomed, like most services, is managed by the systemd init system and such has a service unit file.

You’ll need to modify the systemd service file by adding a single line. However, you shouldn’t modify the system provided service files as these may be overwritten during system updates. To preserve your changes during updates, make a copy to your local /etc/ directory. Locate and copy over the service file using the below command, and make sure the target and destination file names are exactly the same.

cp /usr/lib/systemd/system/awesome.service \
   /etc/systemd/system/awesome.service

Then open up the copied file at /etc/systemd/system/awesome.service and add the following line under the [Service] section:

ExecStartPost=/bin/sh -c \
    'tail -n 15 "/var/log/awesome.log" | \
     systemd-cat -t "awesomed" '

Proceed by running the systemctl daemon-reload command so the system know to look for the new and modified service file, and finally run systemctl restart awesome to update the running service.

You can now use your regular tools for interacting with the journal to read from the log. E.g. journalctl -t "awesomed".

Note that since log forwarding only begins after the main process ExecStart has started successfully, you may loose some start-up logging messages. You can increase the number in the -n argument to catch more backlog at startup, but this may lead to message duplication. It might seem tempting to change the log forwarder pipe to use ExecStartPre instead, but that would block ExecStart from ever executing.

You can manipulate the ExecStartPre pipe in many ways. Some immediately useful ways include filtering the output between tail and systemd-cat through grep, or assigning a different log message priority using the -p argument on systemd-cat.

As a closing note, I’d like to remind you that maybe it isn’t the best idea in the world to feed messages from extremely chatty programs into the systemd journal. Then again, it isn’t really a bright idea to begin with to store super-chatty logs in files the first place.

Sources

Feature image based on a photo by © 2016 Michael Gaida.