Juju Migration

From IEEE 1394 FireWire Wiki
(Difference between revisions)
Jump to: navigation, search
(Compatibility and stability: better text structure, FCP issue will probably be solved in 2.6.33, assessment of controller issues)
(Compatibility and stability: summary of the summary, tape decks)
Line 10: Line 10:
  
 
At the time of this writing (December 2009, kernel 2.6.32.2/ 2.6.33-rc2), the new FireWire kernel driver stack (alias Juju) is nearing feature-completion.  Here is a summary compared to the old stack:
 
At the time of this writing (December 2009, kernel 2.6.32.2/ 2.6.33-rc2), the new FireWire kernel driver stack (alias Juju) is nearing feature-completion.  Here is a summary compared to the old stack:
 +
 +
* Storage devices:  The new stack works better.
 +
* Video devices:  The new stack works better or as well as the old one.
 +
* Audio devices:  Caveats apply.
 +
* Controllers:  All more or less widespread controllers are known to work.  There are known issues with some rare controllers.
  
 
=== New stack working better than the old stack ===
 
=== New stack working better than the old stack ===
Line 28: Line 33:
 
=== New stack not yet working as the old stack does ===
 
=== New stack not yet working as the old stack does ===
  
* Controller support:
+
* Controllers:
 
:* ALi M525x controllers don't work yet.  These controllers are not very widespread though.  They were available as mid-range priced PCI cards (possibly also CardBus cards) and could be swapped for any cheap FireWire PCI card or CardBus card as a stop-gap solution.
 
:* ALi M525x controllers don't work yet.  These controllers are not very widespread though.  They were available as mid-range priced PCI cards (possibly also CardBus cards) and could be swapped for any cheap FireWire PCI card or CardBus card as a stop-gap solution.
 
:* There seem to be problems with the Apple UniNorth rev.1 controller found in PowerBook G3 Pismo and G4 Titanium.  These controllers are very rare nowadays.  As a stop-gap solution, it is possible to use a CardBus FireWire controller in affected machines.
 
:* There seem to be problems with the Apple UniNorth rev.1 controller found in PowerBook G3 Pismo and G4 Titanium.  These controllers are very rare nowadays.  As a stop-gap solution, it is possible to use a CardBus FireWire controller in affected machines.
 
:* NVIDIA NForce2, which may or may not work with the old drivers, do not yet work with the new drivers.  These controllers are very rare.  It is possible to use any cheap FireWire PCI controller in affected machines.
 
:* NVIDIA NForce2, which may or may not work with the old drivers, do not yet work with the new drivers.  These controllers are very rare.  It is possible to use any cheap FireWire PCI controller in affected machines.
 
:* No support for PCILynx controllers.  These do not implement the OHCI-1394 standard and are nowadays very rare, hence it is unlikely that support for them will ever be written for the new driver stack.  Note that their support in the old stack is extremely limited as well (no isochronous I/O which means no video, no audio, not even IP over 1394; slow and possibly very unstable SBP-2 support i.e. storage support).  For all practical purposes, PCILynx controllers are not actually supported in the old stack either.
 
:* No support for PCILynx controllers.  These do not implement the OHCI-1394 standard and are nowadays very rare, hence it is unlikely that support for them will ever be written for the new driver stack.  Note that their support in the old stack is extremely limited as well (no isochronous I/O which means no video, no audio, not even IP over 1394; slow and possibly very unstable SBP-2 support i.e. storage support).  For all practical purposes, PCILynx controllers are not actually supported in the old stack either.
 +
* A professional Panasonic DV tape deck is reportedly not yet recognized by the new drivers.
  
 
=== Old drivers installed as fallback if the new ones are insufficient ===
 
=== Old drivers installed as fallback if the new ones are insufficient ===

Revision as of 13:15, 26 December 2009

Migration to the new FireWire driver stack

Watch this page for future updates on migration requirements. (Or better yet, add here what you know about it.)


Contents


Compatibility and stability

At the time of this writing (December 2009, kernel 2.6.32.2/ 2.6.33-rc2), the new FireWire kernel driver stack (alias Juju) is nearing feature-completion. Here is a summary compared to the old stack:

  • Storage devices: The new stack works better.
  • Video devices: The new stack works better or as well as the old one.
  • Audio devices: Caveats apply.
  • Controllers: All more or less widespread controllers are known to work. There are known issues with some rare controllers.

New stack working better than the old stack

  • With storage devices (SBP-2), the new drivers perform better and are more secure and more compatible.
  • The new drivers are superior to the old ones for use with industrial cameras (IIDC/ DCAM). But the latest libdc1394 release is required to make full use of the latest kernel enhancements (at the time of this writing, libdc1394 v2.1.2).
  • With consumer video equipment, i.e. DV and HDV camcorders, cable TV set top boxes etc., the new drivers are as functional and stable as the old drivers, but more secure. Libraw1394 v2 is required to use the new drivers with applications such as Kino, dvgrab, MythTV etc., preferably libraw1394 v2.0.2 or later.
  • Compatibility with FireWire controller cards is as complete and stable as (but more secure than) the old drivers, with a few known exceptions listed further below.

New stack working almost as well as the old stack

  • Audio devices:
  • FireWire studio audio devices, driven by the Linux FireWire audio framework FFADO, can only be used with the new drivers if you upgrade to kernel 2.6.32 or later and to latest libraw1394 from the libraw1394 git repo. These libraw1394 updates will probably be released in libraw1394 v2.0.5.
  • Even with kernel 2.6.32, it is perhaps not possible to use more than one audio device at once. Usage of two or more FireWire audio devices together has not been tried with the new drivers yet and may fail due to a remaining problem in firewire-core related to FCP registers. This problem is likely to be fixed in an upcoming -rc of kernel 2.6.33.
  • The firedtv kernel driver for FireWire DVB devices has been adapted to the new driver stack only recently in 2.6.33-rc1, i.e. users of older kernels still need the old stack in order to make use of FireDTV boxes and FloppyDTV cards.
  • IP over 1394 support has been added to the new stack in kernel 2.6.31. IP over 1394 is not yet fully stable neither in the old stack no in the new stack, but the implementation in the old stack has naturally been in use much longer already than the one in the new stack.

New stack not yet working as the old stack does

  • Controllers:
  • ALi M525x controllers don't work yet. These controllers are not very widespread though. They were available as mid-range priced PCI cards (possibly also CardBus cards) and could be swapped for any cheap FireWire PCI card or CardBus card as a stop-gap solution.
  • There seem to be problems with the Apple UniNorth rev.1 controller found in PowerBook G3 Pismo and G4 Titanium. These controllers are very rare nowadays. As a stop-gap solution, it is possible to use a CardBus FireWire controller in affected machines.
  • NVIDIA NForce2, which may or may not work with the old drivers, do not yet work with the new drivers. These controllers are very rare. It is possible to use any cheap FireWire PCI controller in affected machines.
  • No support for PCILynx controllers. These do not implement the OHCI-1394 standard and are nowadays very rare, hence it is unlikely that support for them will ever be written for the new driver stack. Note that their support in the old stack is extremely limited as well (no isochronous I/O which means no video, no audio, not even IP over 1394; slow and possibly very unstable SBP-2 support i.e. storage support). For all practical purposes, PCILynx controllers are not actually supported in the old stack either.
  • A professional Panasonic DV tape deck is reportedly not yet recognized by the new drivers.

Old drivers installed as fallback if the new ones are insufficient

To provide a seemless migration from the old to the new drivers, it is possible to build and install the new and the old drivers together. However, care needs to be taken to keep in control which drivers are loaded. I.e. create proper blacklist entries in /etc/modprobe.conf as explained below to avoid auto-loading of the wrong drivers. Also, you need to upgrade your userland to libraw1394 v2 if you want to switch to the new drivers (or freely between old and new drivers).

If you are having trouble with the new drivers, do not hesitate to get in touch via the Mailinglists.

Note, don't use the new drivers in Linux kernel versions older than 2.6.27.5. They are too buggy.


Why the migration?

The code base of the new stack is smaller, cleaner, and closer follows modern Linux kernel programming paradigms. There are already some features in the new stack which the old one lacks, notably bus manager capability and so-called gap count optimization. The latter provides a noticeable speedup of SBP-2 and other asynchronous protocols. Instead of 3 or 5 userspace interfaces in the old stack, there is a single universal interface in the new stack. Furthermore, there are some fundamental bugs and security considerations with the old stack which to a large part motivated the rewrite of the driver stack.

FIXME: Add a Wiki page on FireWire security.


Module auto-loading

How to get auto-loading

The drivers firewire-ohci and firewire-sbp2 contain the module aliases "pci:v*d*sv*sd*bc0Csc00i10*" and "ieee1394:ven*mo*sp0000609Ever00010483*" which should suffice with hotplug scripts and recent coldplug scripts to automatically load these two drivers when respective hardware is detected. The dependence of both drivers on firewire-core is of course recognized by modprobe.

If the kernel was configured without ohci1394 and sbp2 as modules, then firewire-core and firewire-sbp2 also contain the module aliases ohci1394 and sbp2 respectively. That is, "modprobe ohci1394" will load firewire-ohci (and firewire-core) instead of ohci1394 if ohci1394 was excluded from the kernel build.

How to suppress auto-loading

To avoid confusion, it is recommended that either the old or the new driver stack is built, but not both together. It is nevertheless possible to build and install both stacks together. If you chose to do so, you may want to add lines like

# blacklist firewire-ohci
# blacklist firewire-sbp2
# blacklist firewire-net

blacklist ohci1394
blacklist sbp2
blacklist eth1394
blacklist dv1394
blacklist raw1394
blacklist video1394

to /etc/modprobe.d/file_of_your_choice or /etc/modprobe.conf to suppress auto-loading of one of the stacks.

Extremely old modutils which do not support the blacklist keyword can be instructed by configuration entries like "install module /bin/true" to suppress loading of a particular module.


Character device files, block device files

Basic operation

Out of the box, udevd and udev scripts automatically create and remove

  • /dev/fw* devices exposed by firewire-core (for use by libraries like libraw1394 and libdc1394, provided the libraries are updated),
  • /dev/{sd,sg,sr,st}* devices exposed by SCSI command set drivers (sd_mod, sg, sr_mod, st) with firewire-sbp2 underneath.

Permissions and ownership for /dev/fw*

Without any configuration file, the device files will be only accessible to root. This is fine and intended for SCSI block device files. But it is usually desirable to access the /dev/fw* character device files as non-root user. The example udev rules shown below allow any user in group "video" to access /dev/fw* with programs such as dvgrab, kino, or coriander.

By evaluating information in firewire-core's sysfs files, it is possible to automatically adapt file permissions and ownership of /def/fw* files according to device types:

  • Files pertaining to local nodes (the controllers) and SBP-2 nodes (storage devices, scanners etc.) should get restrictive file permissions and ownership. Actually, the default should be restrictive on most systems.
  • Files pertaining to audio and video devices (AV/C or IIDC compliant nodes) can get liberal permissions so that they are accessible to application programs without root privilege.
  • Old libraw1394 versions and some special-purpose libraw1394 clients (e.g. gscanbus) currently require also access to the local node. Its permissions need to be liberal in those cases too. To the author's knowledge, this does not have any actual security impact on systems on which user access to AV/C and IIDC devices is already allowed, but distributors typically won't want to open this up per default.

The following rules implement this device type dependent control over permissions, using the example "video" group. These rules work with Linux kernel 2.6.31 and later. These rules have been merged into /lib/udev/rules.d/50-udev-default.rules of udev v144.

# /etc/udev/rules.d/40-example-firewire.rules

# IIDC devices: industrial cameras and some webcams
SUBSYSTEM=="firewire", ATTR{units}=="*0x00a02d:0x00010*", GROUP="video"

# AV/C devices: camcorders, set-top boxes, TV sets, various audio devices, and more
SUBSYSTEM=="firewire", ATTR{units}=="*0x00a02d:0x010001*", GROUP="video"

To do the same on kernel 2.6.30 and earlier, a trick is required: Useful attributes are only found in children devices, not in the fw* device itself, hence the program chgrp needs to be used to change group ownership of the children's parent in retrospect:

SUBSYSTEM=="firewire", ATTR{specifier_id}=="0x00a02d", ATTR{version}=="0x010001",\
  PROGRAM="/bin/chgrp video /dev/%P"
SUBSYSTEM=="firewire", ATTR{specifier_id}=="0x00a02d", ATTR{version}=="0x00010?",\
  PROGRAM="/bin/chgrp video /dev/%P"

Or if your application needs access to all nodes, simply use:

SUBSYSTEM=="firewire", GROUP="video"

There are also schemes which are based on access control lists instead of UNIX file permissions. For example, the Fedora Linux distribution currently contains a mechanism to add read and write permission for the locally logged in user to the ACLs of fw* files of some FireWire device types.

Symlinks to SCSI block device files

Reasonably recent udev scripts create symbolic links in /dev/disk/by-id/*, pointing to the block devices of FireWire harddisks. These links are convenient for example to use them in static fstab entries: Their names are always the same because they are based on persistent and unique device identifiers, while the actual device files have arbitrary names that change all the time when disks are plugged in and out.

The names of the by-id links look per default a little bit different with firewire-sbp2 compared to sbp2. To make them look exactly the same, add the following to /etc/modprobe.d/file_of_your_choice or /etc/modprobe.conf:

options sbp2 long_ieee1394_id=y

This option has been added to sbp2 in Linux 2.6.22.

You only need this if you plan to switch between sbp2 and firewire-sbp2 and if you are using the /dev/disk/by-id symlinks.


Hald support

It has been reported that firewire-sbp2 driven disks don't show up on desktops, even though they are recognized by the drivers and can be manually mounted just fine.

FIXME: Is special support by hald required?


Initrd

Scripts which generate initrd (an initial RAM disk used during boot) may need to be updated to deal with the new kernel module names firewire-core, firewire-ohci, firewire-sbp2. Scripts within initrd may already work with the new drivers, see #Module auto-loading.


Libraries

libraw1394

At the time of this writing (June 2009), libraw1394 v2.0.4 is the current release and also the recommended stable release for use with the new firewire drivers.

Compatibility with the new drivers is available since libraw1394 v2.0.0, released in July 2008. This version is able to transparently switch between old and new stack, depending on which drivers you have loaded. v2.0.1, released in January 2009, brought a number of important bug fixes.

Either get the latest libraw1394 2.x release, or build libraw1394 from a fresh git checkout:

$ git clone git://dennedy.org/libraw1394
$ cd libraw1394/
$ autoreconf -fi
$ ./configure
$ make
$ sudo make install

Note the following requirements:

You need headers from a kernel with Juju support (at least 2.6.22 headers, better 2.6.30 headers). It may be necessary to install a recent kernel-headers package. Or you can point libraw1394's configure to a directory with these headers with "./configure --with-fw-dir=<dir>".

The Juju backend of libraw1394 requires the inotify kernel interface. If you have got an old libc which does not contain support for inotify, you can have a look at the following patches for libraw1394: patch 1, patch 2. They don't apply to recent versions of libraw1394 anymore though.

Libraw1394's Juju backend furthermore expects that the kernel was configured and built with the following options:

CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_EPOLL=y

The inotify options are located in the "File systems" section of the Kconfig dialogs. CONFIG_EPOLL is already enabled in usual kernel configurations and not even visible in the Kconfig dialogs. But if the kernel was configured for embedded systems it may be necessary to switch CONFIG_EPOLL on explicitly; it is then found in the "General setup" section of the kernel configuration.

libdc1394

At the time of this writing (June 2009), libdc1394 v2.1.2 is the current release and also the recommended stable release for use with the new firewire drivers.

There is compatibility with libdc1394 v2, released in January 2008. Some deployed libdc1394 applications still use the older libdc1394 v1 though which has no support for the Juju drivers. Since v2.1.0, libdc1394 transparently supports the old and the new drivers, though with different extent of functionality. Since libdc1394 v2.1.2 and kernel 2.6.30, libdc1394's Juju backend fully supports multiple cameras on the same bus and precise packet timestamps/ framerate measurement, i.e. offers same or better functionality with the new drivers compared to the old ones.

If life on the bleeding edge appeals to you, you can check libdc1394 v2 out from the project's repository and build it with support for both the new and the old driver stack by the following steps:

$ git clone git://libdc1394.git.sourceforge.net/gitroot/libdc1394/libdc1394
$ cd libdc1394/libdc1394/
$ autoreconf -fi
$ ./configure
$ make
$ sudo make install

(Note, in libdc1394 until v2.0.3 inclusive, a configure switch called "--with-juju-dir" is necessary to support the new driver ABI, e.g. "./configure --with-juju-dir=/usr/src/linux/include". But this also disables support for the old ieee1394 stack.)

Libdc1394 is able to work with the new drivers even if you have only a libraw1394 v1 installed alongside libdc1394. This is because libdc1394 requires libraw1394 only in order to use the old drivers while it accesses the new drivers without assistance of libraw1394.

No special kernel headers package is required to build libdc1394.

FFmpeg: libavformat

Alas the dv1394 module of FFmpeg's libavformat has not been ported to firewire-core yet. This particularly affects players like MPlayer and Xine. However, you can use the capture utility dvgrab to provide input to players:

$ dvgrab - | xine stdin:/

pwlib: AV/C and IIDC plugins

The AV/C video input module of pwlib a.k.a. ptlib, used in Opal, Ekiga etc. uses the old raw1394_start_iso_rcv API which is no longer implemented in the kernel since 2.6.23.

The IIDC video input module (a.k.a. DC module) of pwlib/ ptlib uses libdc1394 v1 to which the new drivers are not compatible.

Best would be if we had a V4L2 interface to FireWire cameras; then these applications would work out of the box without special 1394 backends and libraries.

Personal tools