LKML Summary Podcast

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20091018.mp3

From London, England, for the weekend of October 18th, 2009, I’m Jon Masters with a summary of the weekend’s LKML traffic.

NOTICE: We got quite behind for a while. Rather than keep being two weeks behind, and now that the merge window is closed for 2.6.32, I am going to jump forward to the present. I will fill in the two week gap through additional back episodes – and if that fails, I’ll do a “summary” show of that period (and mention all the cool things from URCU to the latest Git, stable kernels, 2.6.32-rc3, 2.6.32-rc4, a clone3(!) system call proposal, a trace types registry proposal, and Grant Likely’s awesome work on flattened device trees finally getting properly recognized in the MAINTAINERS file).

Remember, I do this in my spare time and without any help from others. I wanted to make sure the merge window was covered, which is why we lagged, and as Linus says, things have been calm enough over the past two weeks. You could always drop me a line and help me form a group of podcasters. I am interested in hacking up a TurboGears front end to a special LKML site that fellow podcasters could use to easily prep the show – maybe when I’m traveling over the holidays I will spend some time poking at that.

In today’s issue: EDF, ext4, fast symbol resolution, M68K, and the staging tree.

EDF. Raistlin posted the latest RFC version of the EDF (Earliest Deadline First) scheduler patches for wider kernel community consideration, including links to various papers, talks, and news coverage, and also thanking the community for feedback at the recent RTLWS (Real Time Workshop) in Dresden. The patches are available via various git repositories covering users of mainline, sched-devel, and also the preempt-rt patches.

ext4. Parag Warudkar posted a story involving various attempts to use ext3, XFS, and ext4 on his laptop as a root filesystem, and in particular the handling after an unclean forced shutdown due to a failed resume from sleep. His experience anecdotally suggests that ext4 has become more intollerant to unclean shutdowns and so he asks, “is this to be expected or it’s just sheer coincidence?”. Ted T’so followed up, referencing a longstanding bug on kernel.org that he has mentioned before. He says “it’s been frustrating because I have not been ble to replicate it myself; I’ve been very much looking for someone who is (a) willing to work with me on this… and (b) who can reliably reproduce this prolem”. Maybe Parag can help.

Fast symbol resolution. Carmelo Amoroso posted to let everyone know about his “Fast LKM symbol resolution” patches. These add a SysV ELF hash table to speed up module symbol resolution at load time. I was at the Embedded Linux Conference as this year’s keynote speaker. As I expected, Alan Jenkins was also interested in taking a look at this as he has also been looking at ways to speed up symbol resolution through using a binary search. Clearly, as Alan notes, only one of the two solutions is going to work out – so the two of them can now help to figure out which one that is going to be Greg Kroah-Hartman added that he is happy to see the work being done, as obviously most distributions are “forced” to ship very modular kernels.

M68K. Steven King posted a script and a patch that enables merging m68knommu and regular m68k into a single tree, at the inspiration of Sam Ravnborg’s recent efforts to merge the include files. This is a big win because it reduces the amount of code duplication in having two “architecture”s.

Staging. Various discussion has been taking place concerning the impact of effectively removing a driver via the staging tree. This is the case of what to do when an improved or next generational driver is being worked on via the staging tree and will replace a driver that has been removed from mainline. Questions included how should users be made aware of this (given that they are likely using a distribution kernel and thus will only notice many months after the removal occurs), and what onus should be place upon vendors.

In today’s pull requests: some libata fixes from Jeff Garzik, some vbus-enet and vbus fixes from Gregory Haskins (fixing an “illegal” use of a GFP_KERNEL kmalloc within a DEVADD, detected via lockdep and not really seen in the wild), some AMD64 EDAC fixes for 2.6.32-rc6 from Borislab Petkov, some device mapper updates for 2.6.32-rc6 from Alasdair Kergon, some KVM updates against 2.6.32-rc5 from Marcelo Tosatti, some input updates from Dmitry Torokhov, and some inotify/dnotify/fsnotify updates from Eric Paris.

In today’s miscellaneous items: ongoing debate as to the best way to do TSC emulation within Xen (and other virtualized guests in general), a question as to why a software RAID device undergoing reconstruction would cause large numbers of processes to get stuck in a “D” state from Holger Kiehl, a patch adding const qualifiers to various users of quota_format_ops from Alexy Dobriyan, an x86 patch from Andreas Herrmann making use of a new MSR that convieniently includes NodeID and number of nodes per processor meta-data, some thermal patches from Roel Kluin, some Kconfig cleanup patches for powerpc from Kumar Gala, version v0.30 of checkpatch (including a fix for the perl warnings that Andrew Morton had managed to trigger previously), version 3 of some ACPI docking support cleanup patches from Alex Chiang, concerns about a hang on boot when using kgdb from Peter Teoh, a note that the rt2×00 wireless project’s mailing list is actually moderated (although the MAINTAINERS file did not list this fact previously) from Bartlomiej Zolnierkiewicz, a rant about rfkill userspace visible interface changes between 2.6.30.2 and 2.6.31.4 from Olivier Galibert, some miscellaneous MAINTAINERS file cleanups from Joe Perches, and a couple of BKL removal patches from John Kacur (thanks for that, John!).

In today’s announcements: BFS v0.304 stable release. Con Kolivas announced the first officially stable release of his “Brain Fuck Scheduler”. Since the patch is quite large, he posted an URL to download it. Citing the usual warnings about development code, he says it is “known to be quite stable”, though it is apparently relatively easy to trigger a well known keyboard+Xorg failure that has recently been discussed and deemed not to be a BFS issue specifically. He also includes a link to the latest version of the BFS FAQ.

Git version 1.6.5.1. Junio C Humano announced version 1.6.5.1 of the Git SCM (Software Configuration Management) tool as used in development and maintainership of the Linux kernel. The latest release fixes an infinite loop bug when processing corrupted packs, addition of MiB/s download speed listing for fast links, and various other fixes also.

Sparse 0.4.2. Christopher Li announced version 0.4.2 of the sparse kernel source code checker tool as originally written by Linus Torvalds. He is the new maintainer, as previously mentioned on the sparse mailing list, and he thanks Josh Triplett for previously maintaining the project. He also took the opportunity to announce a new kernel.org wiki for the sparse project.

The latest kernel release is 2.6.32-rc5, which was released by Linus on Thursday evening at 18:11:49 Best Coast Time (PDT). As Linus has said several times, this is a “short week” release since he will be at the annual Kernel Summit in Japan and doesn’t want to be doing horribly jetlagged releases. By far most of the changes (90%) since -rc4 are in drivers, and Linus includes a handy git command that you can use to visualize the size of them. Linus hopes that no new regressions were added, noting “like that ever happens”.

Greg Kroah-Hartman announced review patches for the 2.6.31.5 stable kernel.

Stephen Rothwell posted a linux-next tree for September 16th. Since Thursday, there was a new “devicetree” tree (thanks to the awesomeness of that work), the linux-next “fixes” tree still contained a build fix for powerpc/kvm, the kbuild tree still had a build failure that required Stephen to remove include/asm/asm-offsets.h from his object tree, and the tty tree lost its build failure. The total tree count increased to 144 trees.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Thursday, October 1st, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: concurrent workqueues, DRBD, VFAT, and writeable overlays.

Concurrent workqueues. Tejun Heo posted an RFC patch series implementing concurrency managed workqueues. The basic premise is that having such an implementation in the kernel keeps the individual workers from having to do it for themselves. The implementation adds a single shared pool of workers per cpu and will attempt to keep the CPU loaded up with as much deadlock-free work as is possible. The code is quite intrusive, illimating RT support from workqueues and touching a lot of code (including the scheduler) but Tejun thinks that, overall complexity will decrease and other code could be removed. David Howells was interested in how this might replace slow-work, and he posted some followup questions for Tejun.

DRBD. “Roland” (devzero) mailed concerning recent comments surrounding DRBD, the distributed replicating block device. He was concerned because a number of people have expressed an interest in these patches not being merged, while DRBD has already been out of tree for around 8 years, and isn’t in staging. He would like to see some more satisfactory resolution for “brilliant things like these” than have them perenially sit out of tree while folks figure out the best way to effect a dm/md merge and a timeframe thereof.

VFAT. Philippe De Muter followed up surrounding his “Simon and Garfunkel” issues (that mp3 files with two tailing dots before the extension were not being properly handled) on VFAT filesystems to say that a recent Windows box wasn’t handling this all too well either. He considers this a bug and isn’t sure that Linux should remain compatible with it, so withdraws his request.

Writeable overlays. Val Aurora posted the latest version of her union mounts and writeable overlays design document, complete with a bunch of patches that she has rebased to kernel 2.6.31, and accompanying tools patches to e2fsprogs and util-linux-ng. Apparently, there will be some review patches soon, though that isn’t an excuse not to start poking. The patches are up at http://valerieaurora.org/union/.

In today’s pull requests: some scheduler fixes from Ingo Molnar (freshly back from a trip, and now believing that it’s “good in all tests”), some networking updates from David Miller, some m68knommu updates from Greg Ungerer, some wireless updates from John Linville, and some btrfs updates from Chris Mason.

In today’s miscellaneous items: a question as to whether IA64 should use a global register for storing per-cpu pointers from Tony Luck, some netfilter patches from Joe Perches, ongoing discussion of alternatives support for cmpxchng64 (silent failure on a cmpxchg of unsupported size annoys Linus, who also provides a commentary on Windows NT’s cmpxcnhg implementation), some autofs4 patches from Ian Kent, version 5 of a fix for too big f_pos handling from Kamezawa Hiroyuki, a suggestion that there might be a buggy implementation in ftrace_profile_enable_event from Paul Mackerras, a series of patches intending to correct usage of __exit_p and __devexit_p from Uwe Kleine-Konig, version 20 of the swap over NFS patches originally worked on by Peter Zijlstra (who is short on time) and now being persued by Suresh Jayaraman, a small update to the optimization flags for the AMD Geode from Matteo Croce, a question about connector and PROC_EVENTS behavior from Kevin Fox, some Kconfig comments cleanups from Michael Roth, and some wonderings from David Miller about the status of mvalloc_user and “perf” mmap patches needed for SPARC to make use of performance events utilities properly.

Finally today, Arjan van de Ven and Andrew Morton continued to discuss the state of the Linux floppy driver, in particular that fact that GCC complains that floppy.c’s ioctl has insufficient bound checks. In response, Andrew stated: ‘gad. You said “floppy” and “ioctl” in the same sentence. Where angels fear to tread.” Separately, Andrew sent an additional error handling patch.

In today’s announcements: The Linux Foundation Technical Advisory Board. James Bottomley posted to let everyone know that there will be elections for the board of the Linux Foundation Technical Advisory Board (TAB) immediately following the forthcoming events in Japan (2009 Kernel Summit and Japan Linux Symposium). Anyone can stand for election by emailing as advised.

Clownix-spy. Vincent Perrier posted to announce a utility at clownix.net that can be used to plot any kernel variable changing over time through the use of a periodic kernel thread that wakes up to sample it and deliver the results to a userspace gtk-based plotting tool. The initial example is for plotting qdisc enqueus, dequeues, and drops.

URCU version 0.2. Mathieu Desnoyers posted version 0.2 of the userspace RCU library he has been working on. It contains some clarifications for three function usages.

The latest kernel release was 2.6.32-rc1|rc2 (both the same).

Greg Kroah-Hartman posted a series of review patches for the 2.6.27.36 stable kernel, and 136 review patches for the 2.6.31.2 stable kernel.

Rafael J. Wysocki posted a summary of regressions since the 2.6.31 kernel, based upon bug filings on the kernel.org bugzilla. As he notes, there aren’t too many new regressions since 2.6.31, but there are still “quite a number” since 2.6.30 and it’s been that way for quite some time.

Stephen Rothwell posted a linux-next tree for October 1st. Since Wednesday, the linux-next fixes tree still has a fix for powerpc/kvm, there is still a reverted SCSI commit, the sound tree gained a build failure (so the previous day’s version of that tree was used), the block tree lost its conflicts but gained a failure for which a commit was reverted, and the drm tree lost its conflict. The total subtree count remained steady at 139 trees.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20090930.mp3

From the Embedded Linux Conference in Grenoble, France, for September 30th, 2009, I’m Jon Masters with a summary of the day’s LKML traffic.

In today’s issue: dm-ioband, robust lists, and page writeback.

dm-ioband. Vivek Goyal posted anothe round of benchmarks of the dm-ioband patches, again noting some problems with the implementation. In the latest tests, he created two ioband devices (ioband1 and ioband2) of weight 100 each on two disk partitions. On one (ioband1), he had a buffered writer do writeup and on the other he had one priority 0 reader and an increasing number of priority 4 readers, to see how bandwidth distribution worked. With vanilla CFQ the results were roughly as expected, but the results for the dm-ioband patches had violently wild swings in bandwith and were quite clearly not correctly preserving any kind of fairness whatsoever. Ryo Tsuruta promised to look into it some more.

Robust Lists. The Linux kernel uses a “robust list” pointer in the task struct task representation structure in order to keep track of userspace futex locks – providing a flexible (and also extensible) way to keep track of locks that userspace might want to play with, and also an atomic means for it to do so through system calls that internally result in list ops. The kernel needs to specially handle the case of a new address space through execve, and Anirban Sinha was concerned that the code within exit_robust_list was inefficient.

Writeback. Fengguang Wu noted that WRITE_SYNC_PLUG and priotization of bio writeback had been implemented about 5 months ago due to complaints from Linus (so it’s no longer true that all requests get ultimately treated equally no matter their sync/async status). Fengguang also posted a patch increasing the MAX_WRITEBACK_PAGES size and adjusting the writeback call stack to support larger writeback chunks. The reason for a limit is to prevent holding I_SYNC against an inode for “enormous amounts of time”.

In today’s pull requests: some ext4 patches for 2.6.32 from Ted T’so, some nilfs2 fixes from Ryusuke Konishi, and some block updates for 2.6.32-rc from Jens Axboe (mostly driver fixes, and especialy to cciss) which included DRBD. Christoph Hellwig considered including DRBD to be ill advised at this stage.

In today’s miscellaneous items: a patch adjusting percpu initialization on IA64 such that the head.S provided __cpu0_per_cpu special CPU percpu area is copied over to a generic location in the linear mapping during memory initialization from Tejun Heo, a request from Amerigo Wang that Barry Song add a signed-off-by to his Y2K38 time patch, a connector bugfix from Christian Borntraeger, ongoing discussion of Intel’s TXT (Trusted eXecution Technology) and in particular Pavel Machek’s views on removal/modification of RAM chips at runtime to usurp any protections, a note from Frederic Weisbecker that patches against 2.6.29 (in this case against at the time experimental “perf” patches for ARM) are useless at this point as too much has changed and patches need to be against 2.6.32, a note from Jens Axboe that find_busiest_group uses a lot of CPU (multiple SSD testcases), a note from Florian Weimer that the new O_NODE open flag implementation does allow one to bypass permission checks on open files within directories whose permissions change while the file descriptor is open (and so “the whole thing is a bit worrisome because it may turn file descriptor information leaks into something worse”), a virtio_ids patch from Christian Borntraeger that makes Rusty’s previous cleanups (moving all device IDs into a single file) once again compatible with userspace users of the header files by moving some includes around, a note from Berthold Gunreben (who had previously posted about ATA bus errors on resume that Tejun Heo though were due to the PSU briefly dropping power to the disk – for which Tejun provided some detailed advice on burning aforementioned PSU) that he had moved to another filesystem (JFS) and could no longer reliably reproduce what might still exist as an underlying error, an RFC patch from Kamezawa Hiroyuki adding percpu array counter support (as used e.g. in vmstat) and new array_counter_add, and array_counter_read functions, a patch from Arjan van de Ven taking advantage of GCC’s ability to determine at compile time whether certain copy_from_user buffers are correctly sized to produce a compile-time (and not runtime) warning in the case that they are not, version 2 of some CFS hard limit patches from Bharata B Rao, a note that bluetooth “is very ill in -next” from Alan Cox, an update to Linus Torvald’s alternatives based cmpxchg64 with some fixes based on some actual testing from Arjan van de Ven, a patch ensuring we always return from cpu_idle with interrupts enabled from Kevin Hilman, and a note from Russell King that Linus imposes a “one pull request per week” limit on arch maintainers like himself and so this can explain why the ARM tree has been broken recently.

In today’s security items: An x86_64 patch from Jan Beulich removing a register leak situation in which a 32-bit process could temporarily switch itself into 64-bit mode in order to get access to additional 64-bit register entries that are not normally cleared on return to 32-bit userspace.

Finally today, Pavel Machek complained about Daniel Walker’s ongoing round of checkpatch warning emails, suggesting that they were “unwelcome” in the case that patches already had many other known problems to resolve.

The latest kernel release was 2.6.32-rc1/rc2 (both the same).

Stephen Rothwell posted a linux-next tree for September 30th. Since Tuesday, his “fixes” tree contains a build fix for powerpc/kvm, the usb.current tree lost all of its conflicts, the scsi tree commit that was causing boot failures was still reverted, the drm tree gained a conflict against Linus’ tree and the usb tree lost all of its conflicts. The total subtree count remained steady at 139 trees in the latest linux-next compose.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.