LKML Summary Podcast

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20100131.mp3

This podcast is brought to you by the power of Al Viro’s ima_file_free fix, saving in-progress crashed podcast recordings since February 2010, and now powering the all new 2010 2.6.33 series Linux kernel with all wheel drive.

For January 31st, 2010, I’m Jon Masters with a summary of the week’s LKML traffic.

In this week’s issue: Linux 2.6.33-rc6, ide2libata, kFIFO, lock types, netfilter connection tracking, netperf regressions, sparse, and USB storage.

Linux 2.6.33-rc6. Linus Torvalds announced Linux 2.6.33-rc6 on Friday January 29th 2010 at 2:20pm (14:20) Best Coast Time (PST), again describing it as containing “nothing earch-shattering”. About 50% of the changes were architecture updates, and 40% were drivers, with the remaining being mostly filesystem and networking updates. He called for people seeing regressions to begin making “loug noises”, since ‘things mostly should “just work”‘.

ide2libata. Bartlomiej Zolnierkiewicz posted a 68 part patch series entitled “ide2libata” that does roughly what it sounds like – it facilitates a conversion of sorts such that legacy IDE driver code can use a small “translation” layer to share source with the libata codebase. It doesn’t remove IDE but it does (allegedly) make it far easier to maintain both until IDE finally does go away. Alan Cox and others weren’t convinced. Alan thought that, “it will be a nightmare for maintenance with all the includes and the like plus the ifdefs making it very hard to read the drivers and maintain them”. He saw value in the effort, but more as a means to find subtle differences between drivers, and thought IDE was “drifting” a little too much to truly be described as in “maintainance mode” at this point.

kFIFO. Stefani Seibold posted an “enhanced reimplementation of the kfifo API”, which is apparently the last in the series of RFC patches intended to rework the kFIFO implementation (to be generic) without changing the existing API. Stefani included some analysis of the impact of the patch upon text section usage and found that it wasn’t much larger, but that the “hand optimized” inline code was substantially faster than the previous implementation.

lock types. Mitake Hitoshi posted an RFC patch (most for the review of Peter Zijlstra) that adds lock type information to the output of lockdep, as used by tools such as perf. As he points out, “Of course, as you told, type of lock dealing with is clear for human. But it is not clear for programs like perf lock”. On a related note, Frederic Weisbecker stated that he really liked the perf lock report layout, but would love to see a tree view that “can tell you which lock is delaying another one”. He gave varous examples of how this might be visualized as well as describing the benefits.

netfilter connection tracking. I discovered that one of my test systems was falling over on all recent 2.6 series kernels, when using KVM. I wasn’t alone (as I would find out later, looking at Fedora bug reports). The backtrace was variable, but typically involved some kind of IPv6 packet. After mailing the netfilter guys (”PROBLEM: reproducible crash KVM+nf_conntrack all recent 2.6″) and getting some general advice, I spent the entire weekend solid debugging the issue with the aid of Jason Wessel’s kgdb-next tree. The problem was that libvirt (the KVM server management daemon) would attempt to create a second network namespace (netns) on startup – just to see if it would be possible to also support containers – and autostart KVM guests started at that moment would crash because conntrack was missing various chunks of support code for dealing with multiple namespaces. This resulted in hash corruption, kmem caches that would get corrupted, and eventual panics.

netperf regressions in 2.6.33-rc1. Lin Ming performed a bisect analysis and determined that a “sched: Rate-limit newidle” commit had once again introduced a loopback regression (on the order of 50%) in the netperf benchmark, when run on an Intel Nehalem system. Lin assumed that this was due a large amount of rescheduling IPI (inter-processor interrupt) traffic, as evidenced by the perf top data, and /proc/interrupts output. Others could not reproduce this issue.

sparse. Tejun Heo posted a series of percpu patches intended to instrument modular use of percpu data, for the benefit of the sparse source checker utility recognizing that such data lives in a separate data section. Tejun included various descriptions within the individual patches, which only affect building when using the sparse checking tool.

USB mass storage. Catalin Marinas posted a message (mostly aimed at Matthew Dharm) concerning cache coherency of the kernel’s USB mass storage driver. In the case of Harvard Architecture (split I/D caches) ARM processor cores, when using PIO based USB host controllers, root mounted filesystems generating a page fault will only fault the requested page into the data cache, but the USB storage driver fails to call flush_dcache_page to ensure I-cache visibility and results in incoherency between the two. Catalin asked Matthew if he might add support for explicit flushes when doing PIO rather than DMA for IO. Oliver Neukum thought that this belonged in the HCD driver rather than USB storage, due to the wide range of possible underlying layers beneath USB storage, and Matthew Dharm agreed, “Given that an HCD can choose, on the fly, it it’s using DMA or PIO, the HCD driver is the only place to reasonably put any cache-synchronization code. That said, what do other SCSI HCDs do?”.

In today’s miscellaneous items: Chinang Ma posted a comparitive performance analysis between RHEL5.4 kernel 2.6.18 and upstream 2.6.33-rc4 in which he found a 0.8% OLTP performance regression, Simon Kagstrom send a “provoke crash” mail in which he described a module to force crashes for testing, Mark Lord wondered why he was seeing a large number of “page allocation failure” messages on upgrade from 2.6.31.5 to 2.6.32.5, a continuation of previous style discussions concerning 80 character line length “limits” in the kernel, a question from Andi Kleen as to whether the PnP probe code (for PS/2 mice in this particular instance) is racy as he experiences variable probe behavior, Christoph Lameter posted version 15 of “one of these year long projects to address fundamental issues in the Linux VM”, aka “SLAB fragmentation reduction”,Alex Chiang posted a patch to increase the maximum number of Infiniband HCAs per system from 32 to 64 in a “backwards-compatible manner” (hence only raising the limit to 64), and Al Viro posted an informative message entitled “Open Intents, lookup_instantiate_filp() And All That Shit(tm)” on his plans for handling atomic file open+possible create for NFS in the grand future.

In today’s announcements: Greg Kroah-Hartman announced the release of the 2.6.32.7 kernel (having previously announced the 2.6.32.6 earlier in the week and posting a series of review patches for 2.6.32.7). He also announced the 2.6.27.45 “long term release” kernel.

Clark Williams announced the latest version 0.63 of the rt-tests package is now available. This includes various utilities used to verify and experiment with the RT patchset that Thomas Gleixner and others maintain.

Mathieu Desnoyers announced the release of version 0.4.0 of his Userspace RCU library, which includes a few “minor API changes” as previously described. urcu is available for download at http://lttng.org/urcu.

Junio C Hamano announced version 1.7.0-rc1 of the Git SCM. The forthcoming release has a number of items in the draft release notes, including some behavior changes to “git push”, “git send-email” (no deep threads by default), “git status”, “git diff”, and various other goodies.

The latest kernel release was 2.6.33-rc6.

Andrew Morton posted an mm-of-the-moment (mmotm) for 2010-01-28-01-36.

Willy Tarreau announced version 2.4.37.8 of the 2.4 series kernel. It mainly includes fixes for a recentl discovered vulnerability in the e1000 network driver that could allow a carefully crafted frame to skip over filtering.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20100124.mp3

For the weekend of January 24th, 2010, I’m Jon Masters with a summary of the week’s LKML traffic.

Linux 2.6.33-rc5. Linus Torvalds announced the release of the 2.6.33-rc5 kernel, noting that he didn’t “think there is anything earth-shaking here”. Mostly, the only new stuff was in the i915 and (new) DVB “Mantis” driver. Rafael J. Wysocki followed up with his usual list of regressions since the release of 2.6.32, for which there were no know fixes yet in Linus’ tree. The number has fallen a little, but there were still 23 unresolved.

devtmpfs. The devtmpfs filesystem is a shared memory filesystem used to mount /dev nodes that are needed even before udev starts on modern Linux systems (or for those systems that do not use udev, to provide a minimum environment). The suggestion had been made to remove the EXPERIMENTAL flag on its configuration option and enable it by default. The latter received complaints as a change in behavior that would be visible to users, even if many of them would need to have devtmpfs enabled for the most recent Linux distributions.

Interruptions. Steven Rostedt, and Peter Zijlstra did some analysis of the kernel source tree, looking for inappropriate setting of TASK_*INTERRUPTIBLE (which should never be done explicitly, and in general one should always use the set_current_state macro). They found a fairly large number of incorrect code paths and posted a list of “examples of likely bugs”. David Daney replied, asking what kind of barrier should be implied in using set_current_state, as pertains to the visibility of this assignment by other CPUs.

IO error semantics. Nick Piggin started a thread entitled “IO error semantics”, in which he raised the ugly issue of kernel IO error handling behavior once again, as he said he had done during Andi Kleen’s posting of HWPOISON patches. Nick sought to clearly define specific anticipated behaviors in response to “read IOs”, “write IOs”, and so forth – how many retries? etc. He also made the point that write IO errors should not invalidate the data before an IO error is returned to “somebody” (fsync or synchronous write syscall).

NOIO. Rafael J. Wysocki posted an initial PM patch implementing forced GFP_NOIO during suspend operations (preventing the kernel from attempting to allocate memory by going to e.g. disk to offload some existing unused pages), this was largely in reaction to specific issues with the Nvidia closed source binary driver, but was something that had apparently been on the cards for some time. The problem with the patch was that it changed the VM according to the state of the system, rather than relying upon drivers to do the right thing in using explicit GFP_NOIO allocations during suspend and resume routines.

In the week’s miscellaneous items: Tejun Heo posted version 3 of his concurrency managed workqueue patches, Peter Anvin proposed the rapid removal of CONFIG_X86_CPU_DEBUG (since all such information is already exposed elsewhere), the addition of “nopat” boot option documentation to Documentation/kernel-paramters by Jiri Kosina, ongoing discussion of generalization of certain PCI functions in the wake of and intention to merge various Xilinx PCI support bits, a cache coherency problem with mmaped writes on ARM systems posted by Anfei Zhou, a patch correcting priority inheritance deboosting in the RT kernel patchset to be POSIX compliant, Dimitry Golubovsky inquired as to the current state of UML (User Mode Linux, not the silly and pointless modelling technique) development, some Restricted Access Register (Intel MID platform) patches from Mark Allyn, and a large number of floppy (yes, floppy) cleanups from Joe Perches.

In the week’s announcements: Linux 2.6.31.12 and 2.6.32.5 (proceeded by the 2.6.32.4 kernel earlier in the week) were released by Greg Kroah-Hartman. Greg stated that he no longer intended to update the .31 stable kernel short of “something really odd happening”. Greg repeated his previous assertions that the .27 kernel would live on as a “long term” stable release (but probably only for 6 more months of viability), and that the .32 kernel would also be a “long term release” because a number of distributions were apparently basing their distributions around it. His efforts depend upon engineers working on those distributions to help.

Len Brown announced that the Linux Power Management Mini-Summit would be held in Boston on Monday, August 9th 2010, the day before the LinuxCon 2010. For further information, refer to http://events.linuxfoundation.org/.

Mathieu Desnoyers (whose excellent PhD thesis was published recently and covered by LWN) announced an updated LTTng 0.187 for the 2.6.32.4 kernel.

Junio C Hamano announced Git 1.6.6.1 is now available from the kernel.org site at http://www.kernel.org/pub/software/scm/git/. The latest version contains fixes for issues such as “git blame” not working when a commit lacked an author name, “git count-objects” not handling packfiles larger than 4G on platforms with a 32-bit off_t, “git rebase -i” not aborting cleaning if it failed to start the user’s EDITOR, some issues with
the GIT_WORK_TREE environment variable, and more besides.

Thomas Gleixner announced the release of 2.6.31.12-rt20 RT patchset. This was a forward port to 2.6.31.12, which included a number of RCU assumption fixes, the aforementioned PI POSIX compliance fix, and so forth. Thomas noted the delay in releasing a new version of the patch, but noted that various locking infrastructure changes had gone upstream (advancing the cause of mainlining various bits of RT). There will be no 2.6.32-rt, but will skip directly over to 2.6.33. He also let us know about a new “housemate” of his: http://tglx.de/~tglx/housemate.png.

Sorry for the delay in getting this episode released.

Audio: http://media.libsyn.com/media/jcm/linux_kernel_podcast_20100117.mp3

For the weekend of January 17th, 2010, I’m Jon Masters with a summary of the past week’s (and some holiday) LKML traffic.

Yep, we’re back, and ready for 2010.

In this weekend’s issue: A new format (version 3.0), async page faults, async suspend and resume, feature removal, mod_timer_msec, page allocator, raw_spinlocks, and Linus’ birthday.

A new format. Ever since I started doing these podcasts last May, it’s been a constant struggle to find the time each day to prepare and produce a show. It often takes upwards of an hour to prepare the material and produce the show. I have asked for volunteers, but clearly this will remain a one man effort for the moment. I’ve been sick for the past week (just working on day job stuff) and had a lot of hassles over the holidays that prevented me from pushing updates. It got me thinking about how I can make this show better – easier to produce, and more reliable. I have decided that in 2010 I will try to produce a daily show, but I will only “commit” to a weekly show covering the events of the past week (which will be longer format and produced on Sundays). Days that don’t have a show will be rolled into the next rather than producing multiple episodes that “catchup” the record (so I won’t bother covering a linux-next tree that’s already a week out of date). I would love to do a show every single day, but I need volunteers to help make that a reality (I need time to write on other projects and read a crazy backlog of books). What really matters is that there’s some kind of reference tracking what happened each day. So you’ll just have to make do with some episodes covering multiple days at a go. Unless you want to volunteer with production…

Async page fault. There was some discussion concerning asynchronous page faults. Hirouki Kamezawa originally posted a patch over the holidays that was now in its third iteration. “asynchronous page fault” support is intended to aid highly threaded applications deal with page faults without needing to contend for the per-process (containing multiple threads) mm->mmap_sem. This is achieved through a certain amount of speculative vma handling (knowing that it might be modified or unmapped without the protection of a lock) and falling back to taking the lock if the VMA RB tree is modified while walking it. There were some issues surrounding atomicity with the patches that Hiroyuki posted, while others (including benh) were concerned that some arches make various assumptions about mmap_sem being held. Peter Zijlstra followed up several days later with version 3 of the “speculative pagefault” patch series, making use of RCU for freeing vm_area_struct (VMAs). On yet another VM note, Gleb Natapov posted an updated round of “Add host swap event notifications for PV guest” patches (that allow guests to know about host page faults), and Mathieu Desnoyers posted version 2 of a patch series implementating a new system call that is named sys_membarrier (forces a process-wide barrier).

Async suspend and resume. Rafael J. Wysocki posted some updated benchmarks for his asynchrnous suspend and resume patches, for which he replaced the disk containing rotational media with a solid state device, and enabled KMS. His results suggest that asynchronous resume is 50% faster at suspending (300-350 ms vs. 600-700 ms) and much faster at resuming (1.1-1.2 seconds vs. 4 seconds). That might given Apple a run for their money.

Feature removal. Robert P. J. Day asked about Documentation/feature-removal.txt and whether it was going to be updated (since some items referred to 2005) soon.

mod_timer_msec. Arjan van de Ven posted a patch (intended for drivers) implementating mod_timer_msec, which can be used to set or change a timer for a relative amount of milliseconds. This allows Arjan (and also others too) to remove the need for certain drivers to work directly with jiffies and HZ.

Page allocator. Mel Gorman posted a patch from Corrado Zoccolo that divides freed pages into two classifications – those that have a high probability of being merged with their next-highest buddy, and those that do not. Those pages not likely to be merged in the near future are preferred on the freelist to those that might be mergeable in order to make higher order allocations available using the newly merged buddy pages, and reduce fragmentation.

raw_spinlocks. John Kacur, noting that Thomas Gleixner’s recent work had freed up the “raw_spinlock” name within the kernel for re-use, posted a number of patches converting existing spinlocks over to raw_spinlocks. This is required in the RT tree, wherein all spinlocks are be default converted to a sleeping variety, but for which certain locks must explicitly remain non-sleeping. Locks that must always be real true spinlocks should use raw_spinlock.

Finally today, Linus’ birthday. It was noted by a few posters that Linus’ birthday is December 28th. Linux Journal had an article, and as these things are want to do, Linus’ reply took the conversation off to a tangent about 387 co-processors, to which Avi Kivity added some remarks on the original design.

In today’s miscellaneous items: a new version of kFIFO (Stefani Seibold – and an even newer version on January 14th that is re-implemented and apparently does not require any changes to existing code users of the old kFIFO API), an XFS status update from Christoph Hellwig (who notes that ongoing work includes support for new event tracing code and mkfs.xfs default support for “lazy superblock counters”), a refresh of the jump labeling patches (v4) from Jason Baron, a patch from Dave Jones removing his name from checkpatch.pl (so people stop asking him about it), initial PCI support for Xilinx Microblaze (and a discussion about generic PCI support file locations), a modpost patch implementing support for ELF objects with greater than 64K sections – recall that this has to be handled specially in ELF – for use when compiling a kernel with -ffunction-sections and having e.g. an allysconfig, and a mini-rant from Stefan Richter concerning “Changelog quality” that is worth reading. Finally, Dan williams (the Intel one) is taking over maintainership of I/OAT.

In today’s announcements: LTTng version 0.186o for 2.6.32-rc8 and Userspace RCU 0.3.3 were both released by Mathieu Desnoyers on January 4th. He posted an updated Userspace RCU 0.3.4 on January 10th, that had some additional fixes.

LTP. Subrata Modak announced the December 2009 Linux Test Project release. It included a number of build system fixes for various distributions.

rt-tests version 0.60. Clark Williams announced the latest version of rt-tests on December 29th. It includes a new ‘pip’ (Priority Inheritance stress test) from John Kacur and adds an unbuffered output option to cyclictest for those parsing the output at runtime. The source is available on git.kernel.org.

smatch 1.54. Dan Carpenter announced version 1.54 of his “smatch” static source code checker tool for C programs such as the Linux kernel. His intention is for “smatch” to become “a smarter version of checkpatch.pl”. It includes cool things like a check for DMA use on the kernel stack.

SystemTap version 1.1. David Smith announced the 1.1 release of SystemTap. This includes better support for gcc 4.5 “richer” debuginfo, amongst other fixes. It’s available from http://sourceware.org/systemtap. Frank Ch. Eigler followed up to note that this release also includes a fix for CVE-2009-4273.

util-linux-ng v2.17. Karel Zak announced the latest 2.17 release of util-linux-ng, which contains a number of new features (an fallocate command, unshare command, wipefs command), and updates to libblkid, blockdev, fdisk, and other fixes.

The latest kernel release is 2.6.33-rc4. There was an rc3 too, but that was pretty small “due to the holidays”. 2.6.33-rc4 had few updates, but 40% of those were in DRM (nouveau and radeon in the staging tree, i915 updated too), and Linus called that out as being “unusual”. There were also some bootloader issues on non-x86 systems, but Linus figured “we’ll sort it out”.

Andrew Morton posted an mm-of-the-moment (mmotm) for 2010-01-15-15-34.

Greg Kroah-Hartman posted some stable review patches for 2.6.32.4, 2.6.31.12 (there had previously been a 2.6.31.10 that was replaced with a build fix in 2.6.31.11), and 2.6.27.44.

Stephen Rothwell posted a linux-next tree for January 14th (and announced that there would be none for January 15th). Since Wednesday, there was a new mtd-current tree, Linus’ tree still had a build failure for which he reverted a comment, the net-current tree lost its conflict, and the tip-tree gained a conflict against the kgdb tree. The total sub-tree count increased to 156.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Thursday, December 24th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: ELF, and NFS.

ELF. Hatayama Daisuke posted the latest version of his ELF coredump patches supporting “extended numbering”. As covered previously, this is required when a large number of mmap()’d VMAs causes the number of sections in a dump to exceed the regular 64K limit. The “extended numbering” is similar to that implemented for other ELF based systems.

NFS. Quentin Barnes experienced some frustration with how Linux implements O_DIRECT file semantics over NFS, in which both readahead is disabled and the VFS re-fetches file attributes frequently rather than cacheing them. In order to avoid this, Quentin opens files without O_DIRECT and instead uses a POSIX fadvise call to instruct the kernel to only disable the readahead semantics. But this has the further unexpected problem of causing the kernel to switch to single page reads at a time! He posted a patch that changes the switch to single page reads in the case of disabling readahead but has “no idea” if this is the best approach, and is seeking some further comments.

In today’s announcements: Linux 2.6.33-rc2. Linus Torvalds, wishing everyone a “Merry Christmas..or [whatever] you’re celebrating today/tomorrow”, posted an updated 2.6.33-rc2 release. Whether you celebrate the holiday or not, Linus has a solution for you that involves testing and playing with the new kernel: “Because while your pants may not fit you for the next month, the new kernel might just fit your computer perfectly.” The latest kernel was released at 14:00 Best Coast Time (PST).

The latest kernel release was 2.6.33-rc2.

Stephen Rothwell posted a linux-next tree for December 24th. The main issues with today’s tree were interactions with the kfifo patches which went into Linus’ tree on the previous day and the kgdb patches. Thanks to Stephen for including a summary of changes – if you do so in future, I will use that. This Christmas tree had all the normal build testing at the end but has not had the usual build testing between merges, likely due to the holiday. The total sub-tree count remained steady at 155 trees in the latest compose.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Wednesday, December 23rd, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, get_maintainer.pl, and staging.

AlacrityVM. Gregory Haskins replied to Avi Kivity’s latest comments, in which Avi had said “[t]here was no attempt by Gregory to improve virtio-net”, with a rebuttal of this, noting that he had attempted on several occasions to work with the KVM folks but that they had essentially answered with “sorry, we are doing our own thing instead”. The thread continued along previous lines.

get_maintainer.pl. Joe Perches posted a patch to allow reading a patch from the standard input.

Staging. Greg Kroah-Hartman posted a series of staging patches for 2.6.33. These included “two big things”. Firstly, the “dst” driver was removed because there are no users and development will not be continued. Secondly, a framebuffer driver was added that had been posted before the merge window closed but was held up waiting on other patches landing in Linus’ tree.

In today’s announcements: Git version 1.6.6. Junio C Hamano announced the latest git SCM release as used for kernel development. This release includes a behavior change wherein “git fsck” defaults to “git fsck –full”. Junio also mentions a number of forthcoming compatibility issues in terms of behavior in 1.7.0, such as “git push” refusing to touch a non-checked out tree, and “git send-email” not making deep threads by default.

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Tuesday, December 22nd, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, and xtime.

AlacrityVM. Ingo Molnar repeated his previous comments concerning “forking” kernel subsystems, agreeing with Anthoy Liguori, who had said that there is a need for a clear understanding of the difference (via “bullet points”) between vbus and vhost-net (one provided by KVM, the other Alacrity), which could be used to consider whether to replace or change the existing kernel code. As he has said before, “to the extent there’s room for improvement here it should be done by shaping KVM, not by forking and rebranding it”. Once again, Gregory provided a calm and reasoned response, noting that he felt various confusion continued to exist surrounding his work, and pull requests. The conversation then took a detour into older review comments.

Xtime. John Stultz posted a request that the removal of xtime_cache be reverted, because xtime now has an overflow issue on rounding that was smoothed and covered by the use of a cache. Apparently, the “fix” is to include sub-nanosecond timekeeping accessor functions in order to avoid the need for any rounding up during accumulation, which can’t be done until three remaining architectures are converted over to GENERIC_TIME. John later followed up with some per-architecture RFC patches just for proof of concept, that had not been tested or even compiled, because he was off on vacation until the New Year and wanted to leave time for suggestions.

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Monday, December 21st, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, asynchronous suspend and resume, kernel threads, and sysctls.

AlacrityVM. Gregory Haskins and Ingo Molnar continued to debate the benefits of various AlacrityVM (a modified KVM-based hypervisor) bits Gregory had posted for inclusion. Greg explained that the bits he had posted were not the core modifications to KVM (which are controversial to some) but simply a “pull request…for drivers to support running a Linux kernel as a guest in this environment, so it actually doesn’t affect KVM in any way”. Avi Kivity countered that this was true, but “these drivers are fairly pointless for virtualization without the host side support” provided by AlacrityVM.

Asynchronous suspend and resume. Rafael J. Wysocki posted some test results of his asynchronous suspend and resume patches, as applied to two different test systems. The results show a clear benefit in time taken for async suspend and resume, although not as dramatic as might be expected (10-20% faster overall). There is a noticeable difference between asynchronous suspend and “upfront” suspend, and the same for resume. “upfront” means that some of the async suspend threads run before the main suspend loop kicks off.

Kernel threads. There has been some discussion over the past few days (in a thread entitled “workqueue thing”) concerning the benefits of concurrent workqueues and the impact of creating many kernel threads in general. Arjan van de Ven has pointed out that kernel thread creation is a lightweight process (yes, pun intended), and that the size of a thread is minimal, but then others in the past have argued for reducing the number of kernel threads overall. If you’re interested, take a look at the (mail) thread.

Sysctl. Andi Kleen posted some patches implementing RCU for reading of string sysctls, in order to avoid races during the read.

In today’s announcements: Git version 1.6.6.rc4. Junio C Hamano announced the latest release of the Git SCM as used for kernel development. The latest version (1.6.6.rc4) comes with several new features (thanks to Junio for pointing out that fixes are released in maintenance releases, and so it is wrong to say “comes with many fixes”, without a qualifier).

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For the weekend of December 20th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: KFIFO, PCI asynchronous power management concerns, and Transcendent memory.

kfifo. One of the several revived patchsets posted today was an updated version .08 of the “new kqueue API”, posted by Stefani Seibold. The patches provide generic macros providing for kernel FIFO management, in order to have only one full FIFO implementation in the kernel tree. The latest version retains the existing API but also implements in-place FIFOs where the data space is part of the FIFO structure itself.

PCI asynchronous power management concerns. Rafael J. Wysocki posted an RFD inquiring as to general consensus on his ongoing effort to provide asynchronous power management for PCI devices (by suspending and resuming unused devices dynamically when they are not in use) and whether this was really as “dangerous”, in general, as Linus had implied. He specifically wanted to know what provides could exist and what could be done about those problems in order to implement full support for this concept.

Transcendent memory. Dan Magenheimer posted an updated version of his July 2009 “Transcendent memory” patchset, rebased to 2.6.32. This patchset provides support for resizeable memory regions that are opportunistically provided by the kernel to support certain applications and virtual machines, but which may be recalled at any time if memory pressure becomes tight. The basic idea is to provide for cacheing needs of database apps, virtual machines, and the like, while not committing large chunks of memory that cannot be reused. In the latest version, Dan includes fixes, support for btrfs and ext4, and has added some performance measurements that were presented at OLS this year. On another VM note, Hiroyuki Kamezawa posted some speculative pagefault patches that were based upon the previous “mm accessor updates” patches.

In today’s announcements: Linux 2.6.33-rc1. Linus Torvalds, in announcing that the merge window for 2.6.33 is now closed, posted the first 2.6.33 (rc1) kernel on Thursday evening at 18:05 Best Coast Time (PST). Linus grumbled that developers left their pull requests pretty late this time around, saying, ‘The two-week merge window is _not_ supposed to be “one day merge window after thirteen days of silence”. He says he may try shortening the merge window next time just to catch these people off guard and “unceremoniously” bump their bits to 2.6.35 instead. There were a few last minute pull requests for s390x, wireless, and so forth.

LTTng 0.183/0.183o. Mathieu Desnoyers posted an updated version of the LTTng patches, addressing a minor issue with the OMAP timer code.

The latest kernel release is 2.6.33-rc1.

Greg Kroah-Hartman announced the release of stable kernels 2.6.27.42, 2.6.31.9, and 2.6.32.2.

Torsten Kaiser noted that he again experienced issues with the new support for MSI (Message Signalled Interrupts) in certain SATA drivers (sata_sil24 is new in 2.6.33, but sata_nv was not working either with 2.6.33).

Stephen Rothwell posted a linux-next tree for December 18th. Since Thursday, the m68knommu tree lost its conflict, the ocfs2 tree gained a conflict against Linus’ tree, the build tree lost its merge fix, the fsnotify tree still had a build failure for which Stephen applied a merge fixup patch, and the tip tree lost its conflict. The total subtree count remained steady at 155 trees, and Stephen let us know that he’s taking a break over the holidays, “most likely, Dec 29″ will be the next linux-next release. No sooner had he sent that than he noticed Linus’ 2.6.33-rc1 posting and did one more rebase for December 19, “so it will be based in -rc1″, albeit with slightly less testing time.

Finally today, Joe Perches posted an updated patch to checkpatch, increasing the warn limit on long lines of program source code to 105 from 80. It seems the VT100 is finally dead to even the last die-hard enthusiasts amongst us.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Thursday, December 17th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: 80 character limits, CFQ group scheduling, DRM, eager NFS writeback, and kbuild.

80 character limits. There has been some discussion of late concerning the historical 80 character limitation imposed upon the length of lines of kernel code, for the benefit of readibility (especially on 80 line VT100 style terminals). As Linus points out, few people use these any more (and most of those are more annoyed by the 24 vertical line limitation). He says he is “driven wild” when strings must be split up and are not easily grepped across lines, and doesn’t feel the 80 character limit is so important any more. He suggests various alternative higher limits.

CFQ group scheduling. Vivek Goyal posted an RFC concerning the structure of CFQ scheduling groups. Specifically, he wants to address the fact that RT tasks are not treated system wide so an RT task within one CFQ cgroup is treated only as having RT priority within that group, but not with respect to other groups that may exist on the system. He listed a variety of proposals involving various hierarchies of task groups that could be implemented to address this issue and asked for some feedback.

DRM. Arnd Bergmann posted a patch entitled “drm: allow unlocked ioctls in drivers”, in which he adds a new flag to ioctls for which he thinks the BKL (Big Kernel Lock) is not required. He “blindly” applied this to “all the most commonly used ioctl commands without checking if they actually rely on the lock”, and was able to reduce BKL use by “90%” on a system. Clearly an RFC patch, but one that is worth some further investigation.

Eager NFS writeback. Steve Rago posted a patch intended to improve the performance of large sequential (client) writes over NFS by implementing eager writeback support. Existing systems utilize a “lazy” writeback approach in which the client will dirty pages faster than they can be written back to the server, and can cause excessive memory pressure for large workloads. Using this patch, and a sysctl called nfs_max_woutstanding, one can now configure how many dirty pages will be allowed before an application will be blocked from dirtying any more until further writes have reached the server. The patch is “almost entirely” based upon a previous patch from Peter Staubach.

Kbuild. The discussion surrounding excessive kernel build dependencies (such as perl, awk, and others also) continued with Peter Anvin weighing in to argue that a dependency on perl might be better than a number of others that would replace it (though he wasn’t endorsing perl as the best posible solution). He also suggested it might be time for the kernel to have its own scripting language, bundled with it, to solve the mess. Rob Landley mentioned that POSIX exists for a reason in implying that the dependency upon non-standard GNU extensions to tools such as awk is worse than a simple dependency upon awk itself. The handling of unicode by tools such as awk also came under discussion.

In today’s announcements: GIT version 1.6.5.7. Junio C Hamano announced the latest version of the GIT SCM is now available on the kernel.org site at http://www.kernel.org/pub/software/scm/git/. The latest version includes a small number of miscellaneous fixes. Junio also announced that the next release candidate for 1.6.6 (rc3) is now available, and this includes a few more updates, but is still a relatively small update.

The latest kernel release was 2.6.32.

Greg Kroah-Hartman posted a number of 2.6.27 stable series kernel review patches (2.6.27.42), a larger number of 2.6.31 stable series review patches (2.6.31.9), and an even larger number of 2.6.32 stable series review patches (2.6.32.2). These were intended to be finalized by the 19th, which has passed as of this podcast being provided.

Stephen Rothwell posted a linux-next tree for December 17th. Since Wednesday, the mips tree inherited conflicts from the mtd tree, the 52xx-and-virtex tree lost its conflict, the kbuild tree inherited a merge fixup from the rr tree, the acpi tree lost its conflicts and build failure, the mtd tree lost its conflicts, the rr tree gained a conflict against Linus’ tree, the voltage tree lost its build failure, the fsnotify tree gained a build failure for which Stephen applied a merge fixup patch, and the hwpoison tree lost its conflicts. The total subtree count remained steady at 155 trees, and Stephen repeated his usual “call for calm” in not merging patches intended for 2.6.34 until after 2.6.33-rc1 has been released.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Wednesday, December 16th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Devtmpfs, kernel.h, mm, and power management.

Devtmpfs. Al Viro came forth with a couple of remarks concerning devtmpfs. He feels that there is a race waiting to happen with the code using a kstrdup (that might sleep) while holding an rwlock. This situation could arise in the case that a call is made to device_add, for example. Separately, Al posted part one of his VFS updates for the 2.6.33 kernel.

Kernel.h. Joe Perches posted a number of cleanups against the kernel.h header file, which he described as a “chaotic jumble collected over time”. Since, apparently, nobody specifically owns this, he sent the patches to Linus.

MM. Hiroyuki Kamezawa posted an updated eleven part patch series based upon Christoph Lameter’s mm_accessor patch that had been posted on November 5th. These patches replace all accesses to mm->mmap_sem with accessor functions that can be used to optimize such accesses in the future, for example by not taking mmap_sem in certain situations. Hiroyuki believes that the patch size is very large and so for eventual merging, it will be necessary to implement it in stages, with architectures later.

Power Management. Ingo Molnar sent his usual round of updates for the 2.6.33 merge window, including some updated Power Management patches. These contain, amongst other things, a patch from Arjan van de Ven that can be used to chart suspend and resume times for devices (especially when doing async suspend and resume, but also otherwise). Arjan continues to facilitate charting of boot and other latencies in getting devices up and running under Linux. On the subject of asynchronous suspend and resume, Linus Torvalds weighed in with some comments on the relative difficulty of asynchronous suspend and resume as applied to Cardbus bridges with many attached devices.

In today’s announcements: LTTng 0.182. Mathieu Desnoyers announced the release of LTTng 0.182 for the 2.6.32 series kernel. The latest release fixes builds on ppc440, and adds full support for ARM omap3 to the trace clock code (power management support, DVFS). Separately, Mathieu announced LTTV 0.12.25, the corresponding view tool that includes support for reading traces taken from an ARM omap3-based board.

The latest kernel release was 2.6.32.

Stephen Rothwell posted a linux-next tree for December 16th. Since Tuesday, the origin tree lost its build failure, the avr32 tree lost its conflict, the microblaze tree lost its conflict, the mips tree lost 4 of its conflicts, the acpi tree still had a build failure so the version from Friday was used (it also gianed a conflict against Linus’ tree), the hwpoison tree gained a conflict against Linus’ tree and also a build failure for which Stephen applied a merge fixup patch. The total subtree count remained steady at 155 trees and Stephen repeated his usual “call for calm” in not merging patches intended for 2.6.34 until after 2.6.33-rc1 has been released.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.