Migrating from Xenomai 2.x to 3.x

In such a case, the application code using any API based on the Copperplate layer, shall call the copperplate_init(int *argcp, char *const **argvp) routine manually, as part of its initialization process, before any real-time service is invoked.

This routine takes the addresses of the argument count and vector passed to the main() routine respectively. copperplate_init() handles the Xenomai options present in the argument vector, stripping them out, leaving only unprocessed options on return in the vector, updating the count accordingly.

xeno-config enables the Copperplate auto-init feature by default.

The Xenomai system state is now fully exported via a FUSE-based filesystem. The hierarchy of the Xenomai registry is organized as follows:

Each leaf entry under a session hierarchy is normally viewable, for retrieving the information attached to the corresponding object, such as its state, and/or value. There can be multiple sessions hosted under a single registry mount point. Session-less application processes are grouped under the special "anon" hierarchy.

The /system hierarchy provides information about the current state of the Xenomai core, aggregating data from all processes which belong to the parent session. Typically, the status of all threads and heaps created by the session can be retrieved.

The registry daemon is a companion tool managing exactly one registry mount point, which is specified by the --root option on the command line. This daemon is automatically spawned by the registry support code as required. There is normally no action required from users for managing it.

The /proc/xenomai interface is still available when running over the Cobalt core, mainly for pure POSIX-based applications. The following changes took place:

{sched, stat, acct} → sched/{threads, stat, acct}

There is no kernel-based time base management anymore with Xenomai 3.0-rc3. Functionally speaking, only the former master time base remains, periodic timing is now controlled locally from the Xenomai libraries in user-space.

Xenomai 3.0-rc3 defines a built-in clock named coreclk, which has the same properties than the former master time base available with Xenomai 2.x (i.e. tickless with nanosecond resolution).

The settings of existing clocks can be read from entries under the new clock/ hierarchy. Active timers for each clock can be read from entries under the new timer/ hierarchy.

As a consequence of these changes:

The clock gravity management departs from Xenomai 2.x as follows:

The following command reports the current gravity triplet for the target system, along with the setup information for the core timer:

Conversely, writing to this file manually changes the gravity values of the Xenomai core clock:

The new --enable-dlopen-libs configuration switch must be turned on to allow Xenomai libaries to be dynamically loaded via dlopen(3).

This replaces the former --enable-dlopen-skins switch. Unlike the latter, --enable-dlopen-libs does not implicitly disable support for thread local storage, but rather selects a suitable TLS model (i.e. global-dynamic).

The former --with-__thread configuration switch was renamed --enable-tls.

As mentioned earlier, TLS is now available to dynamically loaded Xenomai libraries, e.g. --enable-tls --enable-dlopen-libs on a configuration line is valid. This would select the global-dynamic TLS model instead of initial-exec, to make sure all thread-local variables may be accessed from any code module.

By default, any application linking against libcobalt has its main thread attached to the real-time system automatically, this process is called auto-shadowing.

This behavior may be disabled at runtime, by setting the XENO_NOSHADOW variable in the application process environment, before the libcobalt library constructors are executed.

This replaces the former static mechanism available with Xenomai 2.x, based on turning on --enable-dlopen-skins when configuring. Starting with Xenomai 3.x, applications should set the XENO_NOSHADOW variable using putenv(3), before loading libcolbalt using dlopen(3).

When auto-shadowing is enabled, global memory locking is also performed, and remains in effect afterwards (i.e. mlockall(MCL_CURRENT|MCL_FUTURE)).

Xenomai’s libcobalt installs a handler for the SIGWINCH (aka SIGSHADOW) signal. This signal may be sent by the Cobalt core to any real-time application, for handling internal duties.

Applications are allowed to interpose on the SIGSHADOW handler, provided they first forward all signal notifications to this routine, then eventually handle all events the Xenomai handler won’t process.

This handler was renamed from xeno_sigwinch_handler() (Xenomai 2.x) to cobalt_sigshadow_handler() in Xenomai 3.x. The function prototype did not change though, i.e.:

A non-zero value is returned whenever the event was handled internally by the Xenomai system.

Xenomai’s libcobalt installs a handler for the SIGXCPU (aka SIGDEBUG) signal. This signal may be sent by the Cobalt core to any real-time application, for notifying various debug events.

Applications are allowed to interpose on the SIGDEBUG handler, provided they eventually forward all signal notifications they won’t process to the Xenomai handler.

This handler was renamed from xeno_handle_mlock_alert() (Xenomai 2.x) to cobalt_sigdebug_handler() in Xenomai 3.x. The function prototype did not change though, i.e.:

void cobalt_sigdebug_handler(int sig, siginfo_t *si, void *ctxt)

Copperplate is a library layer which mediates between the real-time core services available on the platform, and the API exposed to the application. It provides typical programming abstractions for emulating real-time APIs. All non-POSIX APIs are based on Copperplate services (e.g. alchemy, psos, vxworks).

When Copperplate is built for running over the Cobalt core, it sits on top of the libcobalt library. Conversely, it is directly stacked on top of the glibc or uClibc when built for running over the Mercury core.

Normally, Copperplate should initialize from a call issued by the main() application routine. To make this process transparent for the user, the xeno-config script emits link flags which temporarily overrides the main() routine with a Copperplate-based replacement, running the proper initialization code as required, before branching back to the user-defined application entry point.

This behavior may be disabled by passing the --no-auto-init option.

rtdm/rtdm_driver.h → rtdm/driver.h

rtdm/rtcan.h → rtdm/can.h

rtdm/rtserial.h → rtdm/serial.h

rtdm/rttesting.h → rtdm/testing.h

rtdm/rtipc.h → rtdm/ipc.h

Unlike with Xenomai 2.x, named RTDM device nodes in Xenomai 3 are visible from the Linux device namespace. These nodes are automatically created by the hotplug kernel facility. Application must open these device nodes for interacting with RTDM drivers, as they would do with any regular chrdev driver.

All RTDM device nodes are created under the rtdm/ sub-root from the standard /dev hierarchy, to eliminate potential name clashes with standard drivers.

The legacy (and redundant) rt_dev_*() API for calling the I/O services exposed by a RTDM driver from another driver was dropped, in favour of a direct use of the existing rtdm_*() API in kernel space. For instance, calls to rt_dev_open() should be converted to rtdm_open(), rt_dev_socket() to rtdm_socket() and so on.

analogy/analogy_driver.h → rtdm/analogy/driver.h

analogy/driver.h → rtdm/analogy/driver.h

analogy/analogy.h → rtdm/analogy.h

analogy/types.h analogy/command.h analogy/device.h analogy/subdevice.h analogy/instruction.h analogy/ioctl.h → all files merged into rtdm/analogy.h

As a consequence of these changes, the former include/analogy/ file tree has been entirely removed.

Failing this, the low-level interrupt service code in user-space would be sensitive to external thread management actions, such as being stopped because of GDB/ptrace(2) interaction. Unfortunately, preventing the device acknowledge code from running upon interrupt request may cause unfixable breakage to happen (e.g. IRQ storm typically).

Since the application should provide proper top-half code in a dedicated RTDM driver for synchronizing on IRQ receipt, the RTDM API available in user-space is sufficient.

Removing the pthread_intr API should be considered as a strong hint for keeping driver code in kernel space, where it naturally belongs to.

Conversely, a thread which calls pthread_setschedparam() does know unambiguously whether the current calling context is safe for the incurred migration.

Switching a thread to the SCHED_WEAK class can be done by negating the priority level in the scheduling parameters sent to the Cobalt core. For instance, SCHED_FIFO, prio=-7 would be scheduled as SCHED_WEAK, prio=7 by the Cobalt core.

SCHED_OTHER for a Xenomai-enabled thread is scheduled as SCHED_WEAK,0 by the Cobalt core. When the SCHED_WEAK support is disabled in the kernel configuration, only SCHED_OTHER is available for weak scheduling of threads by the Cobalt core.

Cobalt replacements for sigwait(), sigwaitinfo(), sigtimedwait(), sigqueue() and kill() are available. pthread_kill() was changed to send thread-directed Xenomai signals (instead of regular Linux signals).

Cobalt-based signals are stricly real-time. Both the sender and receiver sides work exclusively from the primary domain. However, only synchronous handling is available, with a thread waiting explicitly for a set of signals, using one of the sigwait calls. There is no support for asynchronous delivery of signals to handlers. For this reason, there is no provision in the Cobalt API for masking signals, as Cobalt signals are implicitly blocked for a thread until the latter invokes one of the sigwait calls.

Signals from SIGRTMIN..SIGRTMAX are queued.

COBALT_DELAYMAX is defined as the maximum number of overruns which can be reported by the Cobalt core in the siginfo.si_overrun field, for any signal.

Cobalt’s implementation of kill() behaves identically to the regular system call for non thread-directed signals (i.e. pid ⇐ 0). In this case, the caller switches to secondary mode.

Otherwise, Cobalt first attempts to deliver a thread-directed signal to the thread whose kernel TID matches the given process id. If this thread is not waiting for signals at the time of the call, kill() then attempts to deliver the signal to a thread from the same process, which currently waits for a signal.

When Cobalt’s replacement for pthread_kill() is invoked, a Xenomai-enabled caller is automatically switched to primary mode on its way to sending the signal, under the control of the real-time co-kernel. Otherwise, the caller keeps running under the control of the regular Linux kernel.

This behavior also applies to the new Cobalt-based replacement for the kill() system call.

By default, the Alchemy API sets the clock resolution for the new process to one nanosecond (i.e. tickless, highest resolution).

However, this work around won’t work if the caller is not the target task of rt_task_set_periodic(), which is fortunately unusual for most applications.

rt_task_set_periodic() still switches to primary as previously over Cobalt. However, it does not return -EWOULDBLOCK anymore.

The owner field of a RT_MUTEX_INFO structure now reports the owner’s task handle, instead of its name. When the mutex is unlocked, a NULL handle is returned, which has the same meaning as a zero value in the former locked field.

Instead, the application decides about the server priority instead of the real-time core applying implicit dynamic boosts.

Applications may override the default priority normalization/denormalization handlers, by implementing the following routines.

Over Cobalt, the POSIX scale is extended to 257 levels, which allows to map pSOS over the POSIX scale 1:1, leaving normalization/denormalization handlers as no-ops by default.

As a consequence of this change, any reference to a user-visible TCB should be refreshed by calling taskTcb() anew, each time reading the status field is required.

Applications may override the default priority normalization/denormalization handlers, by implementing the following routines.