LKML Summary Podcast

Audio: COMING SOON

For the weekend of September 27th, 2009, I’m Jon Masters with a summary of the weekend’s LKML traffic.

In today’s issue: CFS vs. BFS, Performance Events, (a) Shortage of flags, and Tree Scanning.

CFS vs. BFS. Takeuchi Satoru posted a comparitve analysis of the BFS and CFS algorithms, using a benchmark known as “massive_intr” (intended to measure a scheduler’s fairness and overall throughput). The kernels used were a vanilla 2.6.31, 2.6.31-bfs211 (BFS) and 2.6.31-edf (the latest Linus tree with the new EDF patch applied). The tests found that both CFS and BFS “look good”. CFS had better fairness (which isn’t surprising), while BFS had better throughput (which is perhaps not too surprising either). EDF seemed a little immature for producing real benchmarks. Takeuchi notes that the slight difference in kernel means that this isn’t entirely an Apples to Apples comparison. The results are fairly interesting, providing a good case for BFS on the kinds of hardware that this test was run on (a laptop machine with a dual core x86_64 CPU). What remains to be seen is how well BFS performs on a much more scaled system.

Performance Events. Arjan van de Ven posted an RFC patch implementing an kernel-internal interface to performance counters, citing that there “are reasons for kernel code to ask for, and use, performance counters. For example, in CPU freq governors this tends to be a good idea, but there are other examples possible as well of course”. The patch adds the necessary bits, which an experimental cpufreq driver that Arjan is working on is currently making use of (this being the reason for the patch existing in the first instance).

Shortage of flags. Nigel Cunningham had previously noted that the vm_flags field within VMA structures had run out of space for an additional flag that would be required by the TuxOnIce alternative suspend/resume code. He had made various suggestions, which had culminated previously in Kamezawa Hiroyuki suggesting that an additional vm_flags2 be created. Kamezawa added that little used flags in vm_flags could be migrated over to vm_flags2 in order to avoid any real performance hit in making such changes, to which Nigel added his hearty concurrence with the idea.

Tree scanning. Robert P. J. Day posted seeking advice concerning his tree scanning scripts that he uses (post merge-window closure) to detect unused and incorrectly referenced Kconfig CONFIG variables in the wild. He would like to know where to put the output, and the best form in which to share results, which are currently on a separate website (the idea being that he pokes subsystem maintainers individually to fix their own problems currently). Separately, Robert announced that he had updated a wiki page of “unused(?)” CONFIG variables that he would like people to take a look at.

In the weekend’s pull requests: some powerpc bits for 2.6.32 from Ben Herrenschmidt (including some performance “nits” from Anton Blanchard), part 2 of some sh updates for 2.6.32-rc1 from Paul Mundt, some networking updates from David Miller (including fixing CONFIG_NET=n on some platforms, and a large number of other fixes besides), some watchdog patches from Wim Van Sebroeck, some fatfs-2.6 patches from Ogawa Hirofumi, a round of writeback updates for 2.6.31-rc1 from Jens Axboe, a second batch of kbuild updates for 2.6.32 from Sam Ravnborg, some fixes for Alpha and MIPS from Sam Ravnborg, some SPARC and networking updates for 2.6.32 from David Miller, a single writeback fixup from Jens Axboe, some x86, tracing, perf events, and futex fixes for 2.6.32 from Ingo Molnar, some backlight and LED updates from Richard Purdie, part 3 of some ACPI patches for 2.6.32 from Len Brown, and some i2c updates for 2.6.32 from Jean Delvare.

In the weekend’s miscellaneous items: some KVM hotplug patches from Zachary Amsden (supporting CPU hotplug – his 5th of the 5 patches is amusingly entitled “Math is hard; let’s do some cooking”, and who wouldn’t agree?), a question concerning how git really handles merge conflict resolution from Joe Perches (who was trying to figure out a problem merging MAINTAINERS fixes), a fix for the case of CONFIG_KALLSYMS being disabled from Paul Mundt, a fix to sscanf format handling from Andy Spencer, a patch correcting ARCH=x86 from Peter Anvin (which he said he “could have sworn” he had tested previously before it started breaking, something he said might be a toolchain issue), ongoing discussion of the eventual unification of the various in-kernel RAID efforts, ongoing discussion of “immediate values” and jumps, ongoing discussion of the relative merits of various IO bandwidth controlling patches and their impact upon rotational media, some tracing/kprobes updates from Masami Hiramatsu, an RFC patch series implementing a new “perf kprobe” command allowing one to control kprobes using the “performance events” “perf” utility from Masami Hiramatsu, a fix to undo massive breakage of “the vast majority of ARM systems in the wild which are still pre ARMv6″ from Nicolas Pitre, a patch from Fengguang Wu removing the “unsafe” use of __set_page_locked in the HWPOISON code, a reported issues with e1000e jumbo frames no longer working in 2.6.31 from “Nix”, some patches intended to make copy_from_user to a stack slot provably right from Arjan van de Ven (taking advantage of a GCC feature), a patch removing USE_ELF_CORE_DUMP (which is defined everywhere but on the Microblaze architecture, for which he deems Microblaze to be at fault) from Christoph Hellwig, a patch removing some overhead from /proc/net/tcp from Yakov Lerner, and a question about section mismatches from Russell King.

Finally today, have you ever stopped to really look at the mail clients in use by folks posting to the LKML? I do. Surprisingly, a large number of patches are sent by Thunderbird running on Windows systems. An interesting fact. There are even still a few Microsoft LookOut users floating around posting bits.

In the weekend’s announcements: Linux 2.6.32-rc1. To great fanfare, Linus Torvalds announced the release of 2.6.32-rc1 of the Linux kernel on Sunday afternoon at 15:34 Best Coast Time (PDT). He cited LinuxCon and Plumbers amongst the reasons why the merge window had been somewhat extended. The latest kernel contains 67% driver updates, for a change doesn’t include a new filesystem, and does have the usual range of “interesting changes” (including the fact that “ZERO_PAGE is back!”). Linus encourages everyone to “Go wild, test it out, and let us know about any regressions you find”.

Sysprof 1.1.2. Soeren Sandmann announced release 1.1.2 of his “sysprof” system-wide CPU profiler for Linux. This version is based upon the “performance counters” interface that was introduced in kernel 2.6.31. As he notes, 1.1.2 is a development release and will not work with kernels prior to 2.6.31 due to the switch to performance events.

Linux-RT for Debian. Pengutronix announced that they have started providing the Realtime Preemption Linux kernel patches in the form of Debian packages. They timed the announcement to co-incide with the “RT” workshop in Dresden [which LWN already has an excellent writeup of at this point in time]. For further information, you can refer to the Pengutronix website announcement: www.pengutronix.de/software/linux-rt/debian_en.html.

The latest kernel release was 2.6.31.

Andrew Morton released an mm-of-the-moment for 2009-09-25-14-35.

Peter Volkov posted a couple of patches that he believes had been missed from the 2.6.31.1 stable kernel. Tejun Heo reported a race bug in get_device_parent which he said was being caused by sysfs creating several devices in the “cuse” class concurrently and with CONFIG_SYSFS_DEPRECATED disabled. Zdenek Kabelac reported an NFS problem with the 2.6.31 git tree running as a qemu guest. Philippe De Muyter reported some issues reading particular Simon and Garfunkel music [let's face it, this is a pretty serious regression compared with being unable to read Britney Spears or Lady Gaga] files using the Linux vfat implementation when the file ended in more than one “dot” (US: “period”) – he attached a patch. Aneurin Price noted that kernels since 2.6.26 had been “unusably slow” for him and that he tracked it down to an MTRR patch. Rafael J. Wysocki noted that a 2.6.31 git commit to the tty code had broken resume from hibernation on his MSI Wind U100 system. Ted T’so bisected a regression affecting his T400 system in which the second suspend/resume operation after the first was bound to fail – he listed the offending merge commit. Con Kolivas reported some stack corruption on 2.6.31 using KVM.

Stephen Rothwell posted a linux-next tree for September 25th. In announcing today’s compose, Stephen enclosed a “diffstat” that shows 886 files changed, 114306 insertions, and 17547 deletions, which he suggested implied that “we are nearly done for -rc1″. Since Thursday, his fixes tree contains a build fix for powerpc/kvm, the cifs tree gained a couple of build failures and the tip tree lost its conflict. Stephen notes that we continue to see issues bouncing from one tree to another as Linus continues his merging in earnest. The total subtree count remains at 139 trees in the Friday linux-next compose.

Stephen Rothwell also posted (unusually) a linux-next tree for Saturday. Since Friday, his fixes tree still contains the build fix for powerpc/kvm, the sparc-current, and acpi trees gained issues, while the arm-current, net-current, and cifs trees lost their issues. Conflicts continue to bounce from one (sub-) tree to another and the total tree count remains at 139 trees in the Saturday linux-next compose.

Stephen Rothwell also posted (extremely unusually!) a linux-next tree for Sunday. Since Saturday, his fixes tree still contains a build fix for powerpc/kvm, the sparc-current, acpi, and backlight trees lost their issues, while he added a patch to remove includes of autoconf.h that had been causing some build failures. As before, Stephen notes that conflicts continue to bounce between trees as Linus continues with his ongoing merging. The sub-tree count remained steady at 139 trees in the latest compose.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Thursday, September 24th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Checkpoint and restart, ksplice, and RAID.

Checkpoint and restart. Oren Laadan posted 80(!) patches implementing the latest “checkpoint and restart” code for the Linux kernel. The first 17 patches are cleanups, patch 18 adds the new system calls, while the remainder contain the actual “checkpoint and restart” (also refered to as “c/r”) code. Patch 32 contains an implementation for “deferqueue”, which is similar to workqueues but allows a process to defer work that will run in the original context of the task that scheduled it. So far, the code can checkpoint and restart interactive sessions of “screen” across a kernel reboot, which is useful for testing, but a far cry from a full implementation useful for the majority of users.

Ksplice. Tim Abbott (ksplice author extrodinaire) and Christoph Hellwig had a back and forth concerning the (currently) out-of-tree ksplice use of the each_symbol export, which was apparently originally exported for use by ksplice. A recent patch (part 3 of a 4 part patch series) from Christoph had removed that export (it was entitled “module: unexport each_symbol()) and Tim would really have prefered that it stick around. In response, Christoph sent a one line reply: “We don’t keep symbols for out of tree junk around.” Rusty Russell (the in-kernel module loader infrastructure maintainer) stated that he had expected ksplice to go in earlier and had originally applied the patch to make life easier for the eventual merge. But, in his words, “it’s a big hunk of code” [so it might not get merged in the 2.6.32 timeframe, either]. Separately, Tim sent some more linker script cleanups for ksplice preparation.

RAID. Discussion continued (in the thread entitled: “DRBD for 2.6.32″) concerning the eventual removal of the older “md” code once unification between the various in-kernel (md/dm) RAID implementations is complete. Neil Brown thought this process ought to take about 3-5 years, given the average time frame for “enterprise” releases of the kernel. Neil also suggested that the unification effort hadn’t really gotten under way yet, that there was little notion of what shape such an implementation might take, and that it might make more sense to spend time discussing that than the relative merits of merging DRBD into the 2.6.32 kernel. Fujita Tomonori pointed out some previous comments (from Christoph Hellwig) on what a new userspace visible API for RAID management might look like, on the DRBD development list.

In today’s pull requests: some bugfixes for the security subsystem from James Morris, round 2 of some hwmon updates for 2.6.32 from Jean Delvare, some md updates for 2.6.32 from Neil Brown (essentiall a ping reminder to pull), some microblaze fixes for 2.6.32 from Michal Simek, an update to the cputime tree maintained by Martin Schwidefsky (correcting the format of /proc/uptime), some module and parameter updates for 2.6.32 from Rusty Russell, some btrfs updates for 2.6.32 from Chris Mason (including various work on snapshot and subvolume deletion from Yan Zheng – and the btrfs worker threads are now more dynamic as they will die in order to be later respawned if they are not used for a while), some drm-intel updates for 2.6.32 from Eric Anholt, some MAINTAINERS cleanups from Joe Perches, some infiniband updates from Roland Dreier, version 2 of tracepoints for uses of the BKL (Big Kernel Lock) useful in detecting and removing such uses as the BKL is slowly being removed from Frederic Weisbecker (to which Ingo Molnar later noted some build problems), some cris updates for 2.6.32 from Jesper Nilsson, and some eCryptfs fixes for 2.6.32 from Tyler Hicks.

In today’s miscellaneous items: a patch reducing the overly excessive (one per CPU) “TSC is reliable” and “PAT enabled” messages on boot from Rolan Dreier, an assertion that a previous module loader fix in the usermodehelper code from Neil Horman had been incorrect and was causing another backtrace from Sebastian Andrzej Siewior (he included a replacement patch, which Neil agreed was actually the correct fix), some documentation updates for cgroups tasks and procs files from Paul Menage, some cpumask conversions and obsolescence removal patches from Rusty Russell, a suggestion from Kamezawa Hiroyuki that a new vma->vm_flags2 might be required for the overflow from vm_flags becoming full (in reply to the desire fro, Nigel Cunningham to have some room for TuxOnIce to make use of an additional bit for a status flag), some KVM patches from Zachary Amsden (moving timer initialization into an independent function), a patch from Peter Williams (following up to the previous day’s discussions with Peter Zijlstra on the topic) setting the correct normal_prio and prio values in sched_fork() (fixing some issues where some paths through sched_fork() would ignore this requirement and removing the need for a call to effective_prio() in wake_up_new_task()), a page writeback patch from Shaohua Li that effectively reconciles inode commits from the same filesystem so that they will be written together and reduce rotational media head movement (which was deemed a “Good idea!” by Fengguang Wu and immediately signed off on), an optimization removing an unnecessary call to wbinvd on Intel MP CPUs that support a C3 share cache when entering an C3 ACPI power state, an RFC patch series updating autofs4 to deal with some VFS locking changes from Ian Kent, some AMD64 EDAC fixes from Borislav Petkov, some percpu patches making allocation failure warnings more verbose from Tejun Heo, a patch fixing a buffer allocation problem in the “perf” tools from Eric Dumazet, some “immediate values” patches from Mathieu Desnoyers intending to take advantage of data-cache optimization (to which Jason Baron replied that his alternative “jump label” infrastructure had already been proposed, although Jason seemed to favor various aspects of Mathieu’s patch), a patch allowing “perf timechart” to output only power mode information from Arjan van de Ven, version 7 of the new “clone2″ system call intended to better support checkpoint and restart from Sukadev Bhattiprolu, some memory barrier commentary in the futex_wait_queue_me code from Darren Hart, version 10 of the “IO scheduler based IO controller” patches from Vivek Goyal, another round of “jump label” patches from Jason Baron (obviously prompted by Matheiu Desnoyers’ earlier posting of a – related – “immedate values” implementation), and a note from David Miller that “what’s really sad is that you [refering to Ingo Molnar] don’t attend enough conferences so that you can meet face to face with people and you (and others) would as a result avoid many unnecessary heated discussions as a result” [in the spirit of not becoming the Us Weekly of kernel traffic, I'll leave the rest of the discussion up to you to read].

Finally today, Steven Rostedt (and various others) had a somewhat lengthy discussion about the correct English (grammar) for refering to a mailing list intended for the discussion of user issues pertaining to tracing. Steven’s comment becomes the quote of the day: “Oh God, Linux is now tracing its users, but I don’t want to be traced!”.

In today’s announcements: Greg Kroah-Hartman announced the release of stable kernels 2.6.27.35, 2.6.30.8, and 2.6.31.1. He strongly encourages users of these stable kernels to upgrade.

The latest kernel release was 2.6.31.

Ingo Molnar reported a crash in the -tip tree in ibm_find_acpi_device. Daniel J. Blueman reported an NFS suspend to RAM hang when using an NFS home directory (including the full oops report) on recent kernels. Ed Tomlinson reported a “strange” ATA related freeze with 2.6.31.1-rc1.

Stephen Rothwell posted a linux-next tree for September 24th (a little later than usual). Since Wednesday, the drbd tree has been undropped, the tip tree gained a conflict, the ocfs2, jdelvare-hwmon, block, drbd, battery, and usb lost their conflicts, and Stephen applied a patch for a build failure in Linus’ git tree. The total subtree count is reported as falling to 139 in the latest compose (although that again seems a little out-of-sync). Stephen reminds everyone not to push for 2.6.33 until 2.6.32-rc1 is out.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.