• Intro
• Downloads
• Community
• Development
• News
• TALK

Documentation/Maemo 5 Developer Guide/DBus/D-Bus Server Design Issues

Latest revision as of 12:22, 6 September 2010

[edit] Definition of Server

When talking about software, a server is commonly understood to mean some kind of software component that provides a service to its clients. In Linux, servers are usually implemented as daemons, which is a technical term for a process that has detached from the terminal session, and performed other preparatory actions, so that it stays running on the background until it terminates (or is terminated).

Sometimes people might refer to servers as engines, but it is a more generic term, and normally is not related directly to the way a service is implemented (as a separate process, or as part of some library, directly used from within a client process). Broadly defined, an engine is the part of application that implements the functionality, but not the interface, of an application. In Model-View-Controller, the engine is the Model.

The servers in these examples have so far been running without daemonization, in order to display debugging messages on the terminal/screen more easily. Often a server can be started with a --stay-on-foreground option (or -f or something similar), which means that they do not daemonize. This is a useful feature to have, because it allows the use of simpler outputting primitives, when testing the software.

By default, when a server daemonizes, its output and input files are closed, so reading user input (from the terminal session, not GUI) fails, as does each output write (including printf and g_print).

[edit] Daemonization

The objective of turning a process into a daemon is to detach it from its parent process and create a separate session for it. This is necessary, so that parent termination does not automatically cause the termination of the server as well. There is a library call that performs most of the daemonization work, called daemon, but seeing what has to be done (and usually is done) when implementing the functionality can also be helpful:

• Fork the process, so that the original process can be terminated and this causes the child process to move under the system init process.
• Create a new session for the child process with setsid.
• Possibly switch working directory to root (/), so that the daemon does not keep file systems from being unmounted.
• Set up a restricted umask, so that directories and files that are created by the daemon (or its child processes) do not create publicly accessible objects in the filesystem. This does not actually apply in Maemo compatible devices, because the devices only have one user.
• Close all standard I/O file descriptors (and preferably also files), so that if the terminal device closes (user logs out), it does not cause SIGPIPE signals to the daemon the next time it accesses the file descriptors (by mistake or intentionally because of g_print/printf). Reopening the file descriptors is also possible, so that they are connected to a device, which just ignore all operations (like /dev/null that is used with daemon).

The daemon function allows to select whether or not a change of the directory is wanted and to close the open file descriptors. This utilizes in the servers of this example in the following way: glib-dbus-sync/server.c

#ifndef NO_DAEMON
 
  /* This attempts to daemonize this process. It switches this
     process working directory to / (chdir) and then reopen stdin,
     stdout and stderr to /dev/null. Which means that all printouts
     that occur after this are lost. Obviously the
     daemonization also detaches the process from the controlling
     terminal as well. */
  if (daemon(0, 0) != 0) {
    g_error(PROGNAME ": Failed to daemonize.\n");
  }
#else
  g_print(PROGNAME
          ": Not daemonizing (built with NO_DAEMON-build define)\n");
#endif

This definition is available to the user inside the Makefile: glib-dbus-sync/Makefile

# -DNO_DAEMON : do not daemonize the server (on a separate line so can
#               be disabled just by commenting the line)
ADD_CFLAGS += -DNO_DAEMON
# Combine flags
CFLAGS  := $(PKG_CFLAGS) $(ADD_CFLAGS) $(CFLAGS)

Combining the options so that CFLAGS is appended to the Makefile provided defaults allows the user to override the define as well:

[sbox-DIABLO_X86: ~/glib-dbus-sync] > CFLAGS='-UNO_DAEMON' make server
dbus-binding-tool --prefix=value_object --mode=glib-server \
  value-dbus-interface.xml > value-server-stub.h
cc -I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/include
   -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include
   -g -Wall -DG_DISABLE_DEPRECATED -DNO_DAEMON -UNO_DAEMON
   -DPROGNAME=\"server\" -c server.c -o server.o
cc server.o -o server -ldbus-glib-1 -ldbus-1 -lgobject-2.0 -lglib-2.0

Because all -D and -U options are processed from left to right by gcc, this allows the -UNO_DAEMON to undefine the symbol that is preset in the Makefile. If the user does not know this technique, the Makefile can also be edited directly. Grouping all additional flags that the user might be interested in to the top of the Makefile makes this simpler (for the user).

Running the server with daemonization support is performed as before, but this time the & (do not wait for child exit) token for the shell is left out:

[sbox-DIABLO_X86: ~/glib-dbus-sync] > run-standalone.sh ./server
server:main Connecting to the Session D-Bus.
server:main Registering the well-known name (org.maemo.Platdev_ex)
server:main RequestName returned 1.
server:main Creating one Value object.
server:main Registering it on the D-Bus.
server:main Ready to serve requests (daemonizing).
[sbox-DIABLO_X86: ~/glib-dbus-sync] >

Because server messages are not visible any more, some other mechanism is needed to determine whether or not the server is still running:

[sbox-DIABLO_X86: ~/glib-dbus-sync] > ps aux | grep "\./server" | grep -v pts
user 8982  0.0  0.1 2780 664 ? Ss 00:14 0:00 ./server

The slightly convoluted way of using grep was necessary to list only those lines of the ps report, which have ./server in them, and to remove the lines which do not have pts in them (so that seeing the processes which have no controlling terminals is possible).

The client could have been used to test whether the server responds, but the above technique is slightly more general. If the pstree tool is available, it could be run it with -pu options to see how the processes relate to each other and that the daemonized server is running directly as a child of init (which was the objective of the fork).

[edit] Event Loops and Power Consumption

Most modern CPUs (even for desktops and servers) allow multiple levels of power savings to be selected. Each of the levels is progressively more power-conservative, but there is always a price involved. The deeper the power saving level required, the more time it normally takes to achieve it, and the more time it also takes to come out of it. In some CPUs, it also requires special sequences of instructions to run, hence taking extra power itself.

All this means that changing the power state of the CPU should be avoided whenever possible. Obviously, this is in contrast to the requirement to conserve battery life, so in effect, what is needed is to require the attention of the CPU as rarely as possible.

One way at looking the problem field is contrasting event-based and polling-based programming. Code that continuously checks for some status and only occasionally performs useful work is clearly keeping the CPU from powering down properly. This model should be avoided at all cost, or at least its use should be restricted to a bare minimum, if no other solution is possible.

In contrast, event-based programming is usually based on the execution of callback functions when something happens, without requiring a separate polling loop. This then leaves the question of how to trigger the callbacks, so that they are issued when something happens. Using timer callbacks can seem like a simple solution, so that it continuously (once per second or more often) checks for status, and then possibly reacts to the change in status. This model is undesirable as well, because the CPU is not able to enter into deep sleep modes, but fluctuate between full power and high-power states.

Most operating system kernels provide a mechanism (or multiple mechanisms) by which a process can be woken up when data is available and otherwise kept off the running queue of the scheduler. The most common mechanism in Linux is based around the select/poll system calls, which are useful when waiting for a change in status for a set of file descriptors. Because most of the interesting things in Linux can be represented as a "file" (an object supporting read and write system calls), using select and poll is quite common. However, when writing software that uses GLib (implicitly like in GTK+ or explicitly like in the non-GUI examples in this document), the GMainLoop structure is used instead. Internally, it uses the event mechanism available on the platform (select/poll/others), but the program needs to register callbacks, start the main loop execution and then just execute the callbacks as they come.

If there are some file descriptors (network sockets, open files, etc), they can be integrated into the GMainLoop using GIOChannels (see the GLib API reference on this).

This still leaves the question of using timers and callbacks that are triggered by timers. They should be avoided when:

• The timer is used at high frequencies (greater than 1 Hz) for long periods of time (greater than 5 seconds).
• There is a mechanism that triggers a callback when something happens, instead of forcing a manual status poll or re-executing a timer callback that does the checking.

As an example, the LibOSSO program (FlashLight) has to use timers in order to keep the backlight active. However, the timer is very slow (only once every 45 seconds), so this is not a big issue. Also, in FlashLight's defense, the backlight is on all the time, so having a slow timer does not hurt battery life very much anyway.

Another example could be a long-lasting download operation, which proceeds slowly but steadily. It is advisable to consider whether updating a progress bar after each small bit of data is received makes sense (normally it does not). Instead, it is better to keep track of when was the last time when the progress bar was updated, and if enough time has passed since the last time, update the GUI. In some cases, this allows the CPU to be left in a somewhat lower power state than full-power, and allows it to fall back to sleep more quickly.

Having multiple separate programs running, each having their own timers, presents another interesting problem. Because the timer callback is not precise, at some time the system wakes at a very high frequency, handling each timer separately (the frequency and the number of timers executing in the system is something that cannot be controlled from a single program, but instead is a system-wide issue).

If planning a GUI program, it is fairly easy to avoid contributing to this problem, because it is possible to get a callback from LibOSSO, which tells when the program is "on top" and when it is not visible. When not visible, the GUI does not need to be updated, especially with timer-based progress indicators and such.

Because servers do not have a GUI (and their visibility is not controlled by the window manager), such mechanism does not exist. One possible solution in this case is avoiding the use of timers (or any resources for that matter) when the server does not have any active clients. Resources should only be used when there is a client connection or when there is a need to actually do something. As soon as it becomes likely that the server is not used again, the resources should be released (timer callbacks removed, etc.).

If possible, try and utilize the D-Bus signals available on the system bus (or even the hardware state structure via LibOSSO) to throttle down activity based on the conditions in the environment. Even if making a non-GUI server, the system shutdown signals should be listened to because they tell the process to shutdown gracefully.

All in all, designing for a dynamic low-powered environment is not always simple. Four simple but important rules hold true for most cases:

• Avoid doing extra work when possible.
• Do it as fast as possible (while trying to minimize resource usage).
• Do it as rarely as possible.
• Keep only those resources allocated that you need to get the work done.

For GUI programs, the "graphical side" of things has to be taken into account as well. Making a GUI that is very conservative in its power usage is, most of the time, very simple, but it provides little excitement to the user and can even look quite ugly. The priorities for the programmer can lie in a different direction.

[edit] Supporting Parallel Requests

The value object server with delays has one major deficiency: it can only handle one request at a time, while blocking the progress of all the other requests. This is a problem if multiple clients use the same server at the same time.

Normally support for parallel requests is added by using some kind of multiplexing mechanism right on top of the message delivery mechanism (in this case, libdbus).

The possible solutions can be grouped around three models:

• Launching a separate thread to handle each request. This can seem like an easy way out of the problem, but coordinating access to shared resources (object states in this case) between multiple threads is prone to cause synchronization problems, and makes debugging much harder. Also, performance of such an approach depends on efficient synchronization primitives in the platform (which might not always be available), as well as lightweight thread creation and tear-down capabilities of the platform.
• Using an event-driven model that supports multiple event sources simultaneously and "wakes up" only when there is an event on any of the event sources. The select and poll (and epoll on Linux) are very often used in these cases. Using them normally requires an application design that is driven by the requirements of the system calls (which means that retrofitting them into existing "linear" designs is difficult). However, the event-based approach normally outperforms the thread approach, because there is no need for synchronization (when implemented correctly), and there is only one context to switch from the kernel and back (there are extra contexts with threads). GLib provides a high-level abstraction on top of the low-level event programming model, in the form of GMainLoop. GIOChannel objects are used to represent each event source and register callbacks that are triggered on the events.
• Using fork to create a copy of the server process, so that the new copy handles one request and then terminates (or returns to the pool of "servers"). The problem here is the process creation overhead, and the lack of implicit sharing of resources between the processes. A separate mechanism for synchronization and data sharing between the processes (using shared memory and proper synchronization primitives) have to be arranged. In some cases, resource sharing is not actually required, or it happens at some lower level (accessing files), so this model does not need to be automatically ruled out, even if it seems quite heavy at first. Many static content web servers use this model because of its simplicity (and they do not need to share data between themselves).

However, the problem for the slow server lies elsewhere: the GLib/D-Bus wrappers do not support parallel requests directly. Even using the fork model is problematic because there are multiple processes accessing the same D-Bus connection. Furthermore, this problem is not only specific to the slow server. The same issues are encountered when using other high-level frameworks (such as GTK+) whenever they cannot complete something immediately, because not all data is present in the application. In the latter case, using the GMainLoop/GIOChannel approach in parallel with GTK+ (because it uses GMainLoop internally anyway) is normally sufficient, but with GLib/D-Bus there is no mechanism which could be used to integrate own multiplexing code (no suitable API exists).

In this case, the solution is picking one of the above-mentioned models and using the libdbus functions directly. In effect, this requires a complete rewrite of the server, disregarding the GType implementation and possibly creating a light-weight wrapper for integrating libdbus functions into the GLib GMainLoop mechanism (but dropping support for GType).

Dropping the support for GType and stub code means that the introspection support needs to be implemented manually and that it is dependent on the possible API changes in libdbus in the future.

Another possible solution is to "fake" the completion of client method calls, so that the RPC method completes immediately but the server continues processing the request (using GIOChannel integration), until it actually completes. The problem in this solution is that it is very difficult to know which client actually issued the original method call and how to communicate the final result (or errors) of the method call to the client once it completes. One possible model here is using signals to broadcast the end result of the method call, so that the client gets the result at some point (assuming the client is still attached to the message bus). Needless to say, this is quite inelegant and difficult to implement correctly, especially because sending signals causes unnecessary load by waking up all the clients on the bus (even if they are not interested in that particular signal).

In short, there is no simple solution that works properly when GLib/D-Bus wrappers are used.

[edit] Debugging

The simplest way to debug servers is the intelligent use of print-out of events in the code sections that are relevant. Tracing everything that goes on rarely makes sense, but having a reliable and working infrastructure (in code level) helps. One such mechanism is utilizing various built-in tricks that gcc and cpp provide. In the server example, a macro called dbg is used, which expands to g_print when the server is built as a non-daemonizing version. If the server becomes a daemon, the macro expands to "nothing", meaning that no code is generated to format the parameters or to even access them. Extending this idea to support multiple levels of debugging is advisable, as is using different "subsystem" identifiers, so that a single subsystem can be switched on or off, depending on the object of the debugging procedure.

The dbg macro utilizes the __func__ symbol, which expands to the function name where the macro is expanded. This is quite useful because the function name does not need to be explicitly added: glib-dbus-sync/server.c

/* A small macro that wraps g_print and expands to empty when
   a server daemonizes. We use this to add debugging info on
   the server side, but if the server is daemonized, it does not
   make sense to even compile the code in.
 
   The macro is quite "hairy", but very convenient. */
#ifdef NO_DAEMON
#define dbg(fmtstr, args...) \
  (g_print(PROGNAME ":%s: " fmtstr "\n", __func__, ##args))
#else
#define dbg(dummy...)
#endif

Using the macro is quite simple, as it looks and acts like a regular printf-formatting function (g_print included): glib-dbus-sync/server.c

  dbg("Called (internal value2 is %.3f)", obj->value2);

The only small difference here is that adding the trailing newline (\n) explicitly into each call is not necessary because it is added automatically.

Assuming NO_DAEMON is defined, the macro expands to the following output when the server is run:

server:value_object_getvalue2: Called (internal value2 is 42.000)

For larger projects, it is advisable to combine __file__, so that tracing multifile programs will become easier.

Coupled with proper test cases (which are using the client code and possibly also dbus-send in D-Bus related programs), this is a very powerful technique, and often much easier than single stepping through the code with a debugger (gdb), or setting evaluation breakpoints. Using Valgrind to help detecting memory leaks (and some other errors) can also be of interest. More information on these topics and examples is available in the Maemo Debugging Guide.