Package services


Overview

Your packages consist of Lua code that controls what to display. Often you need to fetch external information for that. For example if you want to show a weather forecast you need to fetch weather information and make this data available to the display code. Services allow you to do that.

Every directory in a package might contain an optional file named service. The sychronization process on a device will automatically ensure that every service file is made executable and is started.

Permissions

Package services are sandboxed by default and can't (for example) access any remote network service. If you want to grant a service additional priviledges you have to declare them in the permissions value inside the node.json file.

Running a service

Each service file should be something that can be executed on a normal armv6 system. The following technologies are recommended and can run on any device out of the box.

Shell script

The hosted OS provides the busybox shell, so basic shell operations should work.

Python script

An executable python script. So a normal python script with the

#!/usr/bin/python2.7

as the first line. The hosted OS provides Python 2.7 in addition to the following libraries:

  • PIL, for image manipulation
  • pytz, for timezone manipulation
  • requests, for easy http/https requests

You probably want to use our prepared hosted.py module from the info-beamer package SDK. It makes it easier to starting building a package service in Python. If you want to use another programming language or version, have a look at the custom overlay section below for possible solutions.

Precompiled binaries

You might ship precompiled binaries, but be aware that you have to compile the code for the armv6h architecture, so it runs on all Raspberry Pi versions. You also have to make sure to include all dependencies, so it's probably a good idea to statically link the binary.

Using the Go programming language might be a good idea, since it satisfies these requirements.

Runtime environment

Each service will operate as a separate randomly assigned user without root permissions. Its current work directory is its node directory, so writing to files will be picked up by, for example, the util.file_watch or util.json_watch functions of the locally running info-beamer code.

The service cannot overwrite existing package files or directories. It can however create new files or directories in either its own directory or any other node directory. Additionally each package service can create files and directories in a SCRATCH directory.

While a service can freely create new files, it should limit the amount of data it writes in total to around 100MB - 200MB: info-beamer OS tries to ensure that 500MB are always free on the storage device by deleting older downloaded content, but other package services or info-beamer OS tasks might require some free space as well. So you should not build a your own synchronization process for huge files.

If the service script is updated (so at least a single byte is different) by the synchronization process the syncer will:

  • Remove all files or directories created by the service. The only exception are files and directories created in the SCRATCH directory.
  • Terminate the running version with SIGKILL (see below on why SIGKILL).
  • Restart the new version of the service script.
  • Turn on any attached screen if they have been turned of previously

The service will be automatically restarted if it terminates for any reason. You should try to make a services robust, so they don't randomly exit in case of an error as that would require a full service restart. It's recommended to handle errors inside the service.

Access to the network and other features is prevented by default, unless the permission is set in the node.json file. Note that changing the node.json file will not automatically restart a package service and thus permissions are not granted immediately. The service will only be restarted and given updated permissions if the service file itself is updated.

Custom Overlay

Sometimes you might want to include additional complex software or language runtimes in your package. You could include required files directly in your package. Often this might not work, depending on the software you're trying to include as, for examples, files are not at their expected location. It's also going to slow deployment to a device down if you include a large number of files and is not recommended for this reason.

To help with that, info-beamer OS allows you to include a file named overlay.squashfs next to any of the package's service files. The overlay.squashfs file must be a SquashFS filesystem. It is a compressed filesystem image and can include a large number of files and directories. Its content is mounted on-top of the normal filesystem at the top level directory (/) right before the package service is started. If you want to inspect the result for debugging purposes, have a look at the debugging section and its run shell command.

You can include complete language runtimes this way: Extract the required files into the directory structure required and use mksquashfs to bundle them up into a single overlay.squashfs file. Include that next to your service script and you can use other scripting languages like Ruby or Python 3.

Lifecycle

Similarly it should be able to handle being started at any time. It might happen that the service is started before the system can fetch a proper system time. This can happen under normal circumstances if the NTP is non-responsive, but can of course also happen if the device is completely offline while restarting. Be careful when relying on (for example) the unix timestamp as it might make a rather large jump once a correct system time is retrieved. Be sure to handle this in your code. If you need time deltas (instead of absolute timestamps), consider basing your calculations on the system uptime for example.

Same with external dependencies. You cannot rely on other package services to be fully running already: They might start slowly for some reason and the order in which package services get started is not guaranteed in any way. You also cannot rely on the info-beamer process running: If the screen is shut down by, for example, the power saver package the system terminates the info-beamer process as well.

The service will not automatically restart if the config.json file is changed as the result of configuration changes make on the dashboard or using the API. If you want to react to config changes, you have to monitor config.json. Using inotify is probably a good idea to limit polling. If your package service is written in Python, you can use the hosted.py file from the info-beamer package SDK. It makes developing Python based package services a bit easier.

Note that the config.json file might change automatically at any time, even if no configuration changes are made to the setup's configuration itself: Metadata values are updated at least once every 24 hours and a forced sychronization might be triggered by the info-beamer backend at any time.

A package service is not terminated if a different setup using the same package is assigned to a device. It is also not restarted if you assign a completely different package that happens to use the exactly the same service file. If your service file depends on imported files, like library code, changes to that code will also not result in a package service restart. You can either make your code monitor for those changes as well or make minor changes to your service file to force a restart.

A package service must always handle the case of abrupt termination. This can always happen if the device owner just removes power from the device or the device is rebooted. An info-beamer device can't and doesn't have to be cleanly shut down and the OS is explicitly designed to handle random power loss. Your package service must also handle this. This is also the reason why the package service is killed with SIGKILL instead of sending a SIGTERM signal first: Your service should always be prepared to die at any time.

Network environment

By default a package service cannot access outside network resources. You'll have to add the network permission if you want to connect or talk to external services. Internal communication to localhost/127.0.0.1 is always possible though.

For your convenience, external access to port 21 and 80 is internally redirected to port 2121 and 8080. This allows you to run an externally accessible FTP or web server on user-bindable ports.

Scratch directory

If the device reboots or a service script is updated, all files created by a package service inside any node directory will be lost. If you need a more permanent storage, you might use the SCRATCH directory. Or you might store data outside the device itself in some web service you provide. Be sure to grant your node the network permission in that case.

If you use the local SCRATCH directory, by default files will be scoped by the instance_id and path of the node. In other words: Your package service will only see files stored previously if you run the same setup with the same package at the same path. If you create a new setup or detach and reattach the same package to the same setup, the package service will see a new (and therefore empty) SCRATCH directory.

There is an alternative way to scope the scratch directory. This setting is controlled by the scratch_scope option within the node.json file for the package service. Setting this value to "package" will instead scope the SCRATCH directory based on the package_id and node's local path within the package. As a result, when the same package is reused across setups, the SCRATCH directory will not change. This can be useful if want to store settings on the device that remain "attached" to the package, even if it's used across multiple setups. Beware: Using this scope setting and attaching the same package twice to a single setup results in undefined behaviour with one of the package services not being able to access its scratch directory due to permission issues.

You should always be prepared to encounter an empty scratch directory or even a directory with some of the files missing. Never rely on any persistence as info-beamer OS can't guarantee that. All files should be treated as disposable. Files might even be corrupt due to sudden power loss.

The info-beamer OS is optimized to minimize the number of SD card writes. Writes to the SD card can be delayed for a long time before they end up being persisted on the SD card. You don't have to do anything special if the data you store is not too important and can be easily restored or recreated if lost due to a sudden reboot. If you want to increase the chance of the data ending up on the SD, use fsync.

info-beamer OS might decide to remove old or big scratch directories or even individual files within them. Removal never happens while a package service is running. So you can rely on consistency while the service is active.

You you want, you can instruct info-beamer OS to automatically create a symlink from the SCRATCH directory to a name of your choice within the node directory. Use the scratch_mount option in node.json for that. This is essentially the same as if you run ln -s $SCRATCH $scratch_mount in the package service, expect the OS does it for you before the service is started. Combined with the node.make_nested feature you can access the content of the SCRATCH directory directly from info-beamer Lua code.

Environment variables

The following environment variables will be set for each package service process:

PATH So that a variety of shell commands are available.
SHELL Set to /bin/sh
SERIAL The device/Raspberry Pi's serial number
NODE The complete node name of the node associated with the service. The root node is always named root. Child nodes will look like root/<child> and so on. You can use this value to talk to your node code running in info-beamer by connecting to it using UDP or TCP.
USER The username of the account running the service script. It will consist of "svc" followed by a number.
SCRATCH Points to a directory which might survive reboots. Your service can read and write files there. See the description above.
TMPDIR A memory-backed temporary directory. Don't store big files there, as the Pi might run out of memory. The data is lost on service restart.
SERVICE_DATA_PORT Added with OS release 14: A UDP port the package service might bind to. The service can then receive messages sent using the device service command API.

Debugging a service

You can manually run a package service if you have activated SSH and connected to a device. Each service has a run script located in /service/service.root*. The top level service's run script is located in /service/service.root/run.

You can manually stop a service like this:

$ sv d /service/service.root

or start it again with:

$ sv u /service/service.root

If you just want to restart a service, use

$ sv i /service/service.root

If you stopped a service you can manually invoke it from the command line with:

$ /service/service.root/run

This allows you to directly see the complete service output and any error message. If you want to examine the service script runtime environment you can run a shell inside its sandbox:

$ /service/service.root/run shell
$ ls -l # this is running inside the sandbox
$ env # print all service environment variables

You're given a shell in the environment your service normally runs in. This allows you to check permissions and examine the runtime environment of the service. If you want to start the service inside that environment, just run

$ ./service

Common issues

service: No such file or directory

If you encounter the following error message:

Jun XXX info-beamer-YYYYY user.notice service.root: execvp ./service: No such file or directory

This might seem odd, especially if the file service exists and the file is a script that has a first line that might look like this:

#!/usr/bin/python
...

The problem here is most likely that you've been using a Windows based editor and use Windows-style line endings. Windows usually uses Carriage Return followed by Newline (\r\n) to separate lines. info-beamer is a Linux based system and line endings are expected to be Newline only (\n). The result is that the first line within your script is #!/usr/bin/python\r. The \r at the end of the line then becomes part of the filename and python\r does not exist.

The solution is to switch Line encodings to Newline or Unix-style within your editor.