LKML Summary Podcast

Audio: http://media.libsyn.com/medi/jcm/linux_kernel_podcast_20090615.mp3

Correction: Oops. I meant “Kernel Mode Setting, and not “Kernel Memory System”. I must have been smoking something at the time!

For Monday, June 15th 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: the continuing 2.6.31 merge window, kernel oops reports, and disabling the staging driver tree.

The Continuing 2.6.31 merge window.

Poisonous hardware. Fengguang Wu posted version 5 of the HWPOISON patchset. This version includes new “uevent” code designed to send events to userspace on memory corruption event detection. While many cleanups have indeed taken place and some are happy for the code to merge (because it does very little until an error occurs, and then, it’s probably not much worse to tend to panic on handling the error than on simply ignoring the error condition), others have expressed concern that this needs more bake-time. They include Alan Cox (who feels it needs at least another cycle or two and some buy-in from other architectures – for example, the PowerPC folks), Nick Piggin (who echoes similar concerns about the level of change), and so forth.

Early SLAB. Pekka Enberg, Hugh Dickins, and Nick Piggin continue to discuss Pekka’s “early SLAB” work. This work aims to remove the use of bootmem and replace it (mostly) with a much earlier initialization of SLAB allocation. Some controvasy has arisen over just how much early memory users should be aware of their situation. While Pekka argues that it is appropriate to implement and use a GFP_BOOT option to kmalloc, others believe that users of kmalloc should not have to be aware of low-level machine state (interrupts enabled, and the like). The counter argument is that users at this point (prior to running the initcalls list) should really know what they’re doing, already are using bootmem, and so can live with an extra GFP flag.

Scheduler. Fabio Checconi posted an 8 part patch series implementing an Earliest Deadline First (EDF) group level scheduler to support generic period assignments for Real Time tasks. Quoting Fabio, “With this patch, the rt_runtime and rt_period parameters can be used to specify arbitrary CPU reservations for RT tasks”. As an early RFC, this probably warrents further investigation by anyone with an interest. Separately, Lennart Poettering posted several versions of a patch series intended to extend the POSIX mandated sched_setscheduler call with an optional additional policy parameter of SCHED_RESET_ON_FORK, which, as it implies, resets any elevated priority on subsequent task fork. No doubt this is useful to Real Time tasks like the PulseAudio sound daemon Lennart works on. I am similarly interested in revising an optional extension to mlockall for inheriting MCL_ flags.

Networking. David Miller posted an updated net tree for the 2.6.31 merge. There were many great highlights, amongst them ongoing work on the 802.15.4 protocol stac  (ZigBee support anyone?), conversion to net_device_ops finalization leading to a removal of legacy comptibility code, fixes, and a lot more SKB list abstraction work so that SKB’s can ultimately use list_heads rather than, as David puts it, “our custom crap”. There is also an RFKILL rewrite in the tree, which John Linville’s wireless-next tree uses also.

Kbuild. Sam Ravnborg posted a few minor kbuild updates, including preparation for the vmlinux linker script cleanups that haven’t taken hold yet. Separately, Pekka J Enberg pointed out that a number of distributions are now enabling -Wformat-security for gcc by default, which causes a number of unnecessary warnings to be generated for the kernel, quoting “sometimes in cases where fixing the warning is not likely to actually fix a bug”. Pekka posted a patch to force disable building the kernel with these warnings.

KMS. Dave Airlie posted a git tree containing updates to the Kernel Mode Setting (KMS). The latest tree introduces support for ATI radeon and includes the initial TTM memory manager. I confess to being a little out of touch with the latest KMS work and hope that someone will elect to enlighten us with an article.

Time. Thomas Gleixner posted updates to the ntp, timers (including migration support for power saving), clockevents (including a new register_device exported function smybol), etc.

Architecture updates include: powerpc and the new S+Core architecture.

The S+Core architecture. Liqin Chen posted initial support for the S+core CPU. This is a low power embedded 32-bit processor architecture created by Sunplus Core Technology (S+Core) that uses hybrid 16/32 bit instruction modes and parallel conditional code execution. The Sunplus Core Technology company was created in 2007 and the website currently has little in the way of additional documentation on the ISA or the value-add available in this “multimedia application processor” architecture. Since he did not have access to a git repository of his own, the patches are contained within a branch of Arnd Bergmann’s “asm-generic” git tree (Arnd had signed off on the patches) – having such patches as a branch on an unrelated tree is highly unusual, but there’s no technical reason why they can’t be there.

Documentation. Luis R. Rodriguez noted a need to adequately document the rc-series and kernel merge window, so he posted an RFC patch adding such documentation into the kernel tree.

Arjan van de Ven posted a kerneloops.org report for the week of June 14 2009. In it, he thanked the kernel.org team for their newly provided hosting (replacing a previous virtual machine instance), and drew attention to new oops sightings amongst the XFI driver, and the i915 gem and DMAR code. For this week’s report, a total of 4026 oopses and warnings were reported for kernels 2.6.29 and later (earlier kernels were ommitted from the report). Separately, Arjan proposed adding a “\n” to the existing kernel “WARN()” macro, saying that “many (most) users of WARN() don’t have a \n at the end of their string; as is understandable from the API usage point of view. But this means that the backend needs to add this \n or the warning message gets corrupted (as is seen by kerneloops.org).”

Finally today, there’s a little controvasy over a patch to the RT tree that disables all “staging” drivers. These are drivers in the “staging” area of the kernel source (formerly a separate tree entirely) that are not ready for prime time but are included early in order to drum up support and encourage developers to get them ready for wider use. Many of these drivers barely work on regular Linux kernels, let alone the RT kernel, but Greg Kroah Hartman (the original creator of -staging) objects to the idea of purposely keeping them out of the hands of RT users. Quoting Greg, “This seems like a patch to ensure that the staging drivers never get a chance to be fixed for any potential -rt issues. How about just sending me bug reports instead?”.

In today’s announcements: ndiswrapper 1.54-2.6.30. Jeff Merkey announced a new release of ndiswrapper for the 2.6.30 kernel. He separately took the opportunity to bash both Xen and KVM with, well, you can imagine. Thomas Gleixner posted to say he has released 2.6.29.4-rt19, which includes a couple of trivial fixes. Meanwhile, RT moves on to post-30 work.

The latest kernel release is 2.6.30, which was released by Linus last Tuesday.

Greg Kroah-Hartman announced the release of the 2.6.29.5 kernel and encouraged users to upgrade to the latest update.

Stephen Rothwell posted a linux-next tree for June 15th. Since Friday, he removed the rr-latest-cpumask tree (it having served its purpose), undropped the kvm tree (since the conflicts have been resolved), and the tree continues to fail to build in an allyesconfig build configuration for powerpc. A large number of other conflicts and build problems were accounted. The total number of trees is now down to 129, Rusty’s latest experiment having run its course.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20090614.mp3

For the weekend of June 14th, 2009 I’m Jon Masters with a summary of the weekend’s LKML traffic.

In today’s issue: 2.6.31 merge window, shipping userspace (sub)packages, large kernel images, and matching disks to boot order.

The 2.6.31 merge window. Here we go again, folks, with a pile of updates. Here is a sample of the major updates.

Per-BDI writeback flusher threads. Jens Axboe posted version 10 of his per-BDI writeback flusher threads that have been covered a number of times by this podcast already. Since the previous version, a number of (relatively minor) fixes have been made, and it is really looking ready to be merged.

Early boot SLAB. Pekka Enberg continues the migration away from early bootmem to use of earlier SLAB initialization and usage. He posted a round of fixes for 2.6.31 that included one from Yinghai Lu affecting genirq slab allocation for irq_affinity that had apparently caused various lockdep and lock up problems for some of the assembled kernel hackers.

lguest. Rusty Russell’s latest lguest tree now includes support for PAE. As you might recall, lguest is a minimal 32-bit x86 hypervisor for Linux kernel developers intended to allow experimentation with virtualization without being tied to a specific ABI or the complexities of Xen or KVM. Rusty also posted a number of other updates, including some minor updates to virtio.

VFS. Al Viro posted part 1 of the proposed VFS patches for 2.6.31. As he noted, there were more people than usual involved. Patches included superblock method santization, more pushing of locking into filesystems, and a speedup of mnt_want_write from Nick Piggin.

CUSE (Character device in userspace). Miklos Szeredi requested that support for CUSE be merged in 2.6.31. CUSE (Character Device in UserSpace) enables one to write chatever device drivers in userspace in much the same way that filesystems can be implemented in userspace today, using FUSE. Indeed, it borrows some of the existing FUSE infrastructure. The patch series comes with an example implementation of legacy OSS (Open Sound System) emulation called OSSP, that uses PulseAudio on its userspace side for the sound backend.

Crypto. Herbert Xu posted a series of crypto updates including 64-bit support for VIA crypto/RNG, more operation modes for Intel AES, and a slight change to the compress API for squashfs.

oprofile. Robert Richter posted a series of 23 patches implementing updates to oprofile for 2.6.31, mainly targeting x86. Most of the patches replace macro definitions by function calls and implement internal 64 bit usage for MSR (Machine Status Register) access. struct op_x86_model_spec has also been extended to better describe the x86 model, as a first step toward merging similar model handling code implemented elsewhere in the kernel.

Kmemcheck. Vegard Nossum posted a series of patches implementing kmemcheck support for the 2.6.31 merge window. Kmemcheck is a feature which can (at run-time) detect some uses of uninitialized memory within the kernel, similar to the mechanism offered in Valgrind to do memcheck and used in userspace. It is self-contained in that it has no effect when CONFIG_KMEMCHECK=n.

VDSO (Virtual Dynamic Shared Object). Modern Linux systems use VDSO as a means for the kernel to provide fast-linking of certain commonly used functions into code provided by the kernel and mapped into each process’ address space. This can really speed up services such as gtod (gettimeofday). Petr Tesarik posted a number of clean up patches to the vdso-layout linker script in an effort to fix the “pretty haphazard” way the sections were being handled.

IDE. Bartlomiej Zolnierkiewicz posted a second round of IDE updates for the 2.6.31 merge window. These included the contentious HPA (Host Protected Area) patches that Linus had been unhappy to take late in 30-rc8.

SCSI. James Bottomley posted a first round of SCSI updates for the 2.6.31 merge window (incorrectly refering to “2.6.30″ in the subject line). These include a “large slew of driver updates and miscellaneous changes”. Amongst the changes is an expansion into bits of the net tree courtesy of the FCoE (Fibre Channel over Ethernet) patches, for which apparently David Miller has given all pieces his ACK.

IEEE1394 (Firewire). Stefan Richter posted a teaser email concerning linux1394-2.6.git. Apparently there are two pending batches of updates, that include a number of simplifications to the sysfs ABI which should make the corresponding udev rulesets in distributions simpler – look out for that, distro maintainers, you’ll probably need to clean up your udev rules.

Power Management. Rafael J. Wysocki posted a series of Power Management updates for 2.6.31. These include more 32/64 bit x86 unification, re-arrangements of the code under kernel/power, and bug fixes.

Documentation. Jonathan Corbet posted a series of documentation updates.

Architectural merge requests. Requests came in for merging the latest fixes for AVR32, Blackfin (separately, ongoing work continues to migrate Blackfin to asm-generic), Microblaze (implementing MMU support), and s390.

Of the things that didn’t make 2.6.31, Izik Eidus asked Hugh Dickins about the best way forward for KSM. Quoting, ‘Because we are skipping 2.6.31, It is ok to you to tell me something like: “Shut up and let me see what I can get with this madvise”‘.

Shipping userspace (sub)packages as part of the kernel. Linus Torvalds, Alan Cox, and Al Viro engaged in a discussion surrounding the recent Performance Counters patches, which include a “perf” userspace utility that needs to be shipped alongside the kernel. Al Viro was concerned about ABI breakage, especially if we were to now have a utility that must be kept in perfect sync with the running kernel. Linus suggested the situation was not that bad since the “perf” tool is benchmarking specific (and so not required for booting or similar – though he appreciated the generalization of the dialogue into other scenarios) and the original authors have already agreed not to break the ABI between their kernelspace and userspace “perf” components. The dialogue brought up the old issue of the kernel generally not shipping userspace tools, in some kind of aversion to dealing with these issues.

Large kernel images. Michael Tokarev noted how LILO loads initrd images below 15MB by default. This is normally fine, but on some kernel configurations, the kernel image can encroach upon the initrd and cause corruption – for which LILO eventually gained a “large-memory” option. As Michael notes, most users are likely running GRUB these days with vendor kernels and so probably won’t hit this. It’s this author’s opinion that some of this may have also been caused as a side affect by recent boot image changes in 2.6.30 from Peter Anvin’s patches, although I don’t know this to be the case. On a related note, Ingo’s latest x86-fixes-for-linus tree includes a “handle initrd that extends into unusable memory” patch from Yinghai Lu.

Matching disks to boot order. Harvey Chapman brought up the old issue of matching a disk to the BIOS boot order. This is something Linux has struggled to do historically since the bootloader would have to notify the kernel of the device from which it was booted, and so forth. But recent systems fortunately include support for EDD 3.0, as Alan Cox pointed out in his reply, which does allow the Operating System to find out from which disk it was booted.

Finally today, Robert P. J. Day wondered what the “right time” was to run his “super-duper, kernel cleanup scripts”. He is considering waiting until rc1 has been released and the merge craziness calms down a little.

In today’s announcements: SystemTap release 0.9.8 was announced by Josh Stone. The latest version includes module signing support, debuginfo package suggestions, system call dwarfless probing, new tapset functions, support for 2.6.30 kernels, and many bug fixes and miscellaneous speed improvements. Thomas Gleixner posted version 2.6.29.4-rt18 of the Real Time kernel patchset, including my hwlat hardware latency detector patches and a futex bug fix.

Greg Kroah-Hartman announced the release of Linux 2.6.27.25, which he strongly encourages all users of the 2.6.27 kernel to upgrade to. There are many fixes in this release, including at least one with security implications.

The latest kernel release is 2.6.30, which was released by Linus last Tuesday.

Stephen Rothwell posted a linux-next tree for June 12. Since Thursday, he added a new rr-latest-cpumask tree (which Rusty Russell has posted a number of updates against – expect to see all arches migrate to set_cpu_online, etc. rather than touching bits in the legacy masks, and variable-length cpumasks in struct mm also), dropped the kvm tree (due to too many merge window conflicts), and the tree continues to fail to build in an allyesconfig powerpc build configuration. There were a number of other issues, which might be best summed up as Stephen put it, “Isn’t the merge window fun”. The tree is now back up to a total of 130 sub-trees, due to Rusty’s new one.

Andrew Morton posted an mm-of-the-moment for 2009-06-12-12-20. He is still actively working on removing patches from the tree and merging them.

That’s a summary of today’s LKML traffic. For further information visit kernel.org. I’m Jon Masters.