Unix filesystem semantics or 'How to foolproof your syncing process'

Sometimes you want play a sequence of videos or images. Making sure they are not deleted while playing them might be tricky. It might add complexity to your process of syncing media files to info-beamer. info-beamer 0.9 and later help you make this a lot easier.

The Problem: playing a sequences of files

Say you have a complicated playlist where videos and images are not used independantly from each other. So you cannot just play one media file after the other. Instead you might have a number of sequences that each consist of multiple media files that must be played one after the other. An example for that might be an intro video followed by a bunch of images.

To make this complicated you also want to be able to update your playlist at any time.

Those two requirements pose a problem: Say you have a sequence this:

video1, image1, image2, image3, video2

The easiest way to build a playlist player in info-beamer is just to open the file on demand. So you load video1, wait for it to complete and then load image1. Repeat the same process for all media files. This will work as long as you never delete any media file.

The easiest way to synchronize content to a device running info-beamer is probably just using something like rsync.

If you add or remove media files while info-beamer is running you have to be careful not to delete any of the files in the currently running sequence. Otherwise info-beamer won't be able to open those files when required.

To solve this problem you have to be really careful about how you sync files to your device: You have to make sure that you are not deleting files that might still get used in the currently running sequence. Implementing this in a realiable way is a nightmare.

Another futile attempt to solve this might be loading all media resources before using them. Doing that might not always be possible since images and videos require a lot of memory.

Unix filesystem semantics to the rescue

info-beamer 0.9 offers a way to solve this problem while keeping the synchronization easy and memory usage minimal: You can preopen files and only later make use of them. So how does this help?

Each time a new sequence is about to be scheduled you try to preopen all files that are required during that sequence. This includes image files and videos files. If you have been successful you know that you can load these files later, even is a synchronization process deletes them in the meantime.

You might wonder how this works: On Linux (and other unices), a file is not immediately delete from the disk if there is still a process running that has an open file handle for that file. While the file is no longer visible in the filesystem (ls won't list it) its data is still there and can be used by any process that has such a file handle. Only when the last process closes the file handle is the file really gone and its space reclaimed.

info-beamer adds a new object type called openfile which you can create using resource.open_file. It represents an open (but otherwise unused) file. You can then use this opened file for calls to resource.load_video or resource.load_image instead of providing a filename:

-- Create an openfile object of 'image.jpg'
local imgfile = resource.open_file "image.jpg"

-- Some time later (image.jpg might be delete at this point)
-- you can load this image.
local image = resource.load_image(imgfile)

Tips & Tricks

You can only use an openfile once. After calling load_image in the example the imgfile object cannot be used again to load the same image. If you try to repeat the load_image call you'll get a error: operation on disposed openfile.

If you (for some reason) want to load the same openfile more than once you can use :copy():

-- Create a copy of imgfile and use it to load the image.
-- imgfile can use used again later.
local image = resource.load_image(imgfile:copy())

Every openfile object wraps an open file handle. By default a Linux process can only have 1024 file handles open at the same time. So like with other resources you should help info-beamer release them once you don't need them any more. To to that just call :dispose() on your openfile object when you no longer need it.

The underlaying file handle is already taken care of once you used an openfile object to load a resource. So you don't have to call :dispose() if you used your openfile object in (for example) load_image.

You can use the new helper function util.open_files to open more than a single file without leaking resources should one of them fail to open.

local filenames = {"image1.jpg", "image2.jpg"}
local ok, files = util.open_files(filenames)
if ok then
    -- files is a list of all openfile objects in argument order
else
    -- second return value contains the error message
    error(files)
end

Easier media synchronization

This new capability make it a lot easier to sychronize a playlist and associated media files to an info-beamer directory. You can now just delete and replace files without fearing that the currently running playback will break because a file has been deleted.

Each time you schedule a new sequence of files you can now just preopen all of them at once using util.open_files. If the call succeeds you know you can use all those file later even if they get deleted on disk. If it fails you should schedule the next sequence.

Give the new version a try. I'd love to get feedback, so get in contact.