LKML Summary Podcast

Audio: COMING SOON

For Thursday, December 24th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: ELF, and NFS.

ELF. Hatayama Daisuke posted the latest version of his ELF coredump patches supporting “extended numbering”. As covered previously, this is required when a large number of mmap()’d VMAs causes the number of sections in a dump to exceed the regular 64K limit. The “extended numbering” is similar to that implemented for other ELF based systems.

NFS. Quentin Barnes experienced some frustration with how Linux implements O_DIRECT file semantics over NFS, in which both readahead is disabled and the VFS re-fetches file attributes frequently rather than cacheing them. In order to avoid this, Quentin opens files without O_DIRECT and instead uses a POSIX fadvise call to instruct the kernel to only disable the readahead semantics. But this has the further unexpected problem of causing the kernel to switch to single page reads at a time! He posted a patch that changes the switch to single page reads in the case of disabling readahead but has “no idea” if this is the best approach, and is seeking some further comments.

In today’s announcements: Linux 2.6.33-rc2. Linus Torvalds, wishing everyone a “Merry Christmas..or [whatever] you’re celebrating today/tomorrow”, posted an updated 2.6.33-rc2 release. Whether you celebrate the holiday or not, Linus has a solution for you that involves testing and playing with the new kernel: “Because while your pants may not fit you for the next month, the new kernel might just fit your computer perfectly.” The latest kernel was released at 14:00 Best Coast Time (PST).

The latest kernel release was 2.6.33-rc2.

Stephen Rothwell posted a linux-next tree for December 24th. The main issues with today’s tree were interactions with the kfifo patches which went into Linus’ tree on the previous day and the kgdb patches. Thanks to Stephen for including a summary of changes – if you do so in future, I will use that. This Christmas tree had all the normal build testing at the end but has not had the usual build testing between merges, likely due to the holiday. The total sub-tree count remained steady at 155 trees in the latest compose.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Folks,

I’m currently in Paris recovering from a cold and trying to have a little downtime too. So I’m just writing briefer summaries for the holidays and playing a little catchup each day. I will be heading home to the States by way of the UK, and will try to get some recordings caught up before the New Year.

Someone mentioned the audio is currently “zero bytes long”. Actually, that’s not true. The only “official” audio feed (the one that says “Podcast Feed” next to a picture of headphones) is the one listed on kernelpodcast.org on the main page, not the one attached in the RSS feed of each podcast (and the official one hasn’t been updated while I’ve been stuck in blizzards and travelling). That is generated by wordpress when I make a posting with an mp3 reference in it. Since I won’t have audio for a few days, if you’re pulling the wordpress feed then you will still continue to get warnings as there is no audio attached.

Happy (secular) holidays from me, and a happy New Year!

Jon.

Audio: COMING SOON

For Wednesday, December 23rd, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, get_maintainer.pl, and staging.

AlacrityVM. Gregory Haskins replied to Avi Kivity’s latest comments, in which Avi had said “[t]here was no attempt by Gregory to improve virtio-net”, with a rebuttal of this, noting that he had attempted on several occasions to work with the KVM folks but that they had essentially answered with “sorry, we are doing our own thing instead”. The thread continued along previous lines.

get_maintainer.pl. Joe Perches posted a patch to allow reading a patch from the standard input.

Staging. Greg Kroah-Hartman posted a series of staging patches for 2.6.33. These included “two big things”. Firstly, the “dst” driver was removed because there are no users and development will not be continued. Secondly, a framebuffer driver was added that had been posted before the merge window closed but was held up waiting on other patches landing in Linus’ tree.

In today’s announcements: Git version 1.6.6. Junio C Hamano announced the latest git SCM release as used for kernel development. This release includes a behavior change wherein “git fsck” defaults to “git fsck –full”. Junio also mentions a number of forthcoming compatibility issues in terms of behavior in 1.7.0, such as “git push” refusing to touch a non-checked out tree, and “git send-email” not making deep threads by default.

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Tuesday, December 22nd, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, and xtime.

AlacrityVM. Ingo Molnar repeated his previous comments concerning “forking” kernel subsystems, agreeing with Anthoy Liguori, who had said that there is a need for a clear understanding of the difference (via “bullet points”) between vbus and vhost-net (one provided by KVM, the other Alacrity), which could be used to consider whether to replace or change the existing kernel code. As he has said before, “to the extent there’s room for improvement here it should be done by shaping KVM, not by forking and rebranding it”. Once again, Gregory provided a calm and reasoned response, noting that he felt various confusion continued to exist surrounding his work, and pull requests. The conversation then took a detour into older review comments.

Xtime. John Stultz posted a request that the removal of xtime_cache be reverted, because xtime now has an overflow issue on rounding that was smoothed and covered by the use of a cache. Apparently, the “fix” is to include sub-nanosecond timekeeping accessor functions in order to avoid the need for any rounding up during accumulation, which can’t be done until three remaining architectures are converted over to GENERIC_TIME. John later followed up with some per-architecture RFC patches just for proof of concept, that had not been tested or even compiled, because he was off on vacation until the New Year and wanted to leave time for suggestions.

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Monday, December 21st, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: AlacrityVM, asynchronous suspend and resume, kernel threads, and sysctls.

AlacrityVM. Gregory Haskins and Ingo Molnar continued to debate the benefits of various AlacrityVM (a modified KVM-based hypervisor) bits Gregory had posted for inclusion. Greg explained that the bits he had posted were not the core modifications to KVM (which are controversial to some) but simply a “pull request…for drivers to support running a Linux kernel as a guest in this environment, so it actually doesn’t affect KVM in any way”. Avi Kivity countered that this was true, but “these drivers are fairly pointless for virtualization without the host side support” provided by AlacrityVM.

Asynchronous suspend and resume. Rafael J. Wysocki posted some test results of his asynchronous suspend and resume patches, as applied to two different test systems. The results show a clear benefit in time taken for async suspend and resume, although not as dramatic as might be expected (10-20% faster overall). There is a noticeable difference between asynchronous suspend and “upfront” suspend, and the same for resume. “upfront” means that some of the async suspend threads run before the main suspend loop kicks off.

Kernel threads. There has been some discussion over the past few days (in a thread entitled “workqueue thing”) concerning the benefits of concurrent workqueues and the impact of creating many kernel threads in general. Arjan van de Ven has pointed out that kernel thread creation is a lightweight process (yes, pun intended), and that the size of a thread is minimal, but then others in the past have argued for reducing the number of kernel threads overall. If you’re interested, take a look at the (mail) thread.

Sysctl. Andi Kleen posted some patches implementing RCU for reading of string sysctls, in order to avoid races during the read.

In today’s announcements: Git version 1.6.6.rc4. Junio C Hamano announced the latest release of the Git SCM as used for kernel development. The latest version (1.6.6.rc4) comes with several new features (thanks to Junio for pointing out that fixes are released in maintenance releases, and so it is wrong to say “comes with many fixes”, without a qualifier).

The latest kernel release was 2.6.33-rc1.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For the weekend of December 20th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: KFIFO, PCI asynchronous power management concerns, and Transcendent memory.

kfifo. One of the several revived patchsets posted today was an updated version .08 of the “new kqueue API”, posted by Stefani Seibold. The patches provide generic macros providing for kernel FIFO management, in order to have only one full FIFO implementation in the kernel tree. The latest version retains the existing API but also implements in-place FIFOs where the data space is part of the FIFO structure itself.

PCI asynchronous power management concerns. Rafael J. Wysocki posted an RFD inquiring as to general consensus on his ongoing effort to provide asynchronous power management for PCI devices (by suspending and resuming unused devices dynamically when they are not in use) and whether this was really as “dangerous”, in general, as Linus had implied. He specifically wanted to know what provides could exist and what could be done about those problems in order to implement full support for this concept.

Transcendent memory. Dan Magenheimer posted an updated version of his July 2009 “Transcendent memory” patchset, rebased to 2.6.32. This patchset provides support for resizeable memory regions that are opportunistically provided by the kernel to support certain applications and virtual machines, but which may be recalled at any time if memory pressure becomes tight. The basic idea is to provide for cacheing needs of database apps, virtual machines, and the like, while not committing large chunks of memory that cannot be reused. In the latest version, Dan includes fixes, support for btrfs and ext4, and has added some performance measurements that were presented at OLS this year. On another VM note, Hiroyuki Kamezawa posted some speculative pagefault patches that were based upon the previous “mm accessor updates” patches.

In today’s announcements: Linux 2.6.33-rc1. Linus Torvalds, in announcing that the merge window for 2.6.33 is now closed, posted the first 2.6.33 (rc1) kernel on Thursday evening at 18:05 Best Coast Time (PST). Linus grumbled that developers left their pull requests pretty late this time around, saying, ‘The two-week merge window is _not_ supposed to be “one day merge window after thirteen days of silence”. He says he may try shortening the merge window next time just to catch these people off guard and “unceremoniously” bump their bits to 2.6.35 instead. There were a few last minute pull requests for s390x, wireless, and so forth.

LTTng 0.183/0.183o. Mathieu Desnoyers posted an updated version of the LTTng patches, addressing a minor issue with the OMAP timer code.

The latest kernel release is 2.6.33-rc1.

Greg Kroah-Hartman announced the release of stable kernels 2.6.27.42, 2.6.31.9, and 2.6.32.2.

Torsten Kaiser noted that he again experienced issues with the new support for MSI (Message Signalled Interrupts) in certain SATA drivers (sata_sil24 is new in 2.6.33, but sata_nv was not working either with 2.6.33).

Stephen Rothwell posted a linux-next tree for December 18th. Since Thursday, the m68knommu tree lost its conflict, the ocfs2 tree gained a conflict against Linus’ tree, the build tree lost its merge fix, the fsnotify tree still had a build failure for which Stephen applied a merge fixup patch, and the tip tree lost its conflict. The total subtree count remained steady at 155 trees, and Stephen let us know that he’s taking a break over the holidays, “most likely, Dec 29″ will be the next linux-next release. No sooner had he sent that than he noticed Linus’ 2.6.33-rc1 posting and did one more rebase for December 19, “so it will be based in -rc1″, albeit with slightly less testing time.

Finally today, Joe Perches posted an updated patch to checkpatch, increasing the warn limit on long lines of program source code to 105 from 80. It seems the VT100 is finally dead to even the last die-hard enthusiasts amongst us.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Thursday, December 17th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: 80 character limits, CFQ group scheduling, DRM, eager NFS writeback, and kbuild.

80 character limits. There has been some discussion of late concerning the historical 80 character limitation imposed upon the length of lines of kernel code, for the benefit of readibility (especially on 80 line VT100 style terminals). As Linus points out, few people use these any more (and most of those are more annoyed by the 24 vertical line limitation). He says he is “driven wild” when strings must be split up and are not easily grepped across lines, and doesn’t feel the 80 character limit is so important any more. He suggests various alternative higher limits.

CFQ group scheduling. Vivek Goyal posted an RFC concerning the structure of CFQ scheduling groups. Specifically, he wants to address the fact that RT tasks are not treated system wide so an RT task within one CFQ cgroup is treated only as having RT priority within that group, but not with respect to other groups that may exist on the system. He listed a variety of proposals involving various hierarchies of task groups that could be implemented to address this issue and asked for some feedback.

DRM. Arnd Bergmann posted a patch entitled “drm: allow unlocked ioctls in drivers”, in which he adds a new flag to ioctls for which he thinks the BKL (Big Kernel Lock) is not required. He “blindly” applied this to “all the most commonly used ioctl commands without checking if they actually rely on the lock”, and was able to reduce BKL use by “90%” on a system. Clearly an RFC patch, but one that is worth some further investigation.

Eager NFS writeback. Steve Rago posted a patch intended to improve the performance of large sequential (client) writes over NFS by implementing eager writeback support. Existing systems utilize a “lazy” writeback approach in which the client will dirty pages faster than they can be written back to the server, and can cause excessive memory pressure for large workloads. Using this patch, and a sysctl called nfs_max_woutstanding, one can now configure how many dirty pages will be allowed before an application will be blocked from dirtying any more until further writes have reached the server. The patch is “almost entirely” based upon a previous patch from Peter Staubach.

Kbuild. The discussion surrounding excessive kernel build dependencies (such as perl, awk, and others also) continued with Peter Anvin weighing in to argue that a dependency on perl might be better than a number of others that would replace it (though he wasn’t endorsing perl as the best posible solution). He also suggested it might be time for the kernel to have its own scripting language, bundled with it, to solve the mess. Rob Landley mentioned that POSIX exists for a reason in implying that the dependency upon non-standard GNU extensions to tools such as awk is worse than a simple dependency upon awk itself. The handling of unicode by tools such as awk also came under discussion.

In today’s announcements: GIT version 1.6.5.7. Junio C Hamano announced the latest version of the GIT SCM is now available on the kernel.org site at http://www.kernel.org/pub/software/scm/git/. The latest version includes a small number of miscellaneous fixes. Junio also announced that the next release candidate for 1.6.6 (rc3) is now available, and this includes a few more updates, but is still a relatively small update.

The latest kernel release was 2.6.32.

Greg Kroah-Hartman posted a number of 2.6.27 stable series kernel review patches (2.6.27.42), a larger number of 2.6.31 stable series review patches (2.6.31.9), and an even larger number of 2.6.32 stable series review patches (2.6.32.2). These were intended to be finalized by the 19th, which has passed as of this podcast being provided.

Stephen Rothwell posted a linux-next tree for December 17th. Since Wednesday, the mips tree inherited conflicts from the mtd tree, the 52xx-and-virtex tree lost its conflict, the kbuild tree inherited a merge fixup from the rr tree, the acpi tree lost its conflicts and build failure, the mtd tree lost its conflicts, the rr tree gained a conflict against Linus’ tree, the voltage tree lost its build failure, the fsnotify tree gained a build failure for which Stephen applied a merge fixup patch, and the hwpoison tree lost its conflicts. The total subtree count remained steady at 155 trees, and Stephen repeated his usual “call for calm” in not merging patches intended for 2.6.34 until after 2.6.33-rc1 has been released.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Audio: COMING SOON

For Wednesday, December 16th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Devtmpfs, kernel.h, mm, and power management.

Devtmpfs. Al Viro came forth with a couple of remarks concerning devtmpfs. He feels that there is a race waiting to happen with the code using a kstrdup (that might sleep) while holding an rwlock. This situation could arise in the case that a call is made to device_add, for example. Separately, Al posted part one of his VFS updates for the 2.6.33 kernel.

Kernel.h. Joe Perches posted a number of cleanups against the kernel.h header file, which he described as a “chaotic jumble collected over time”. Since, apparently, nobody specifically owns this, he sent the patches to Linus.

MM. Hiroyuki Kamezawa posted an updated eleven part patch series based upon Christoph Lameter’s mm_accessor patch that had been posted on November 5th. These patches replace all accesses to mm->mmap_sem with accessor functions that can be used to optimize such accesses in the future, for example by not taking mmap_sem in certain situations. Hiroyuki believes that the patch size is very large and so for eventual merging, it will be necessary to implement it in stages, with architectures later.

Power Management. Ingo Molnar sent his usual round of updates for the 2.6.33 merge window, including some updated Power Management patches. These contain, amongst other things, a patch from Arjan van de Ven that can be used to chart suspend and resume times for devices (especially when doing async suspend and resume, but also otherwise). Arjan continues to facilitate charting of boot and other latencies in getting devices up and running under Linux. On the subject of asynchronous suspend and resume, Linus Torvalds weighed in with some comments on the relative difficulty of asynchronous suspend and resume as applied to Cardbus bridges with many attached devices.

In today’s announcements: LTTng 0.182. Mathieu Desnoyers announced the release of LTTng 0.182 for the 2.6.32 series kernel. The latest release fixes builds on ppc440, and adds full support for ARM omap3 to the trace clock code (power management support, DVFS). Separately, Mathieu announced LTTV 0.12.25, the corresponding view tool that includes support for reading traces taken from an ARM omap3-based board.

The latest kernel release was 2.6.32.

Stephen Rothwell posted a linux-next tree for December 16th. Since Tuesday, the origin tree lost its build failure, the avr32 tree lost its conflict, the microblaze tree lost its conflict, the mips tree lost 4 of its conflicts, the acpi tree still had a build failure so the version from Friday was used (it also gianed a conflict against Linus’ tree), the hwpoison tree gained a conflict against Linus’ tree and also a build failure for which Stephen applied a merge fixup patch. The total subtree count remained steady at 155 trees and Stephen repeated his usual “call for calm” in not merging patches intended for 2.6.34 until after 2.6.33-rc1 has been released.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Folks,

We got stranded in New York during the blizzard (after spending 5 hours on a plane, at the gate) after flying down from Boston to get to London, only to have to take a train back to Boston and fly from there on the only flight left. The trip from London had few trains, no buses, and almost no taxis, and the fun isn’t over yet as we’re in Edinburgh now trying to get back to London. Once this crazy travel craziness is over, more podcasts are coming for the holidays

Jon.

Audio: COMING SOON

For Tuesday, December 15th, 2009, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: ELF, power capping, OOM, and TSC.

ELF. Hatayama Daisuke posted a four part RFC patch series implementing an unofficial ELF (Executable Linking Format) extension known as “Extended Numbering”. This is (apparently) originally a Solaris hack that extends the possible number of program headers contained within an ELF file to greater than the 64K limit currently imposed by the standard in the case that the kernel dumper is dumping out a core file that contains a great many mmaps. It’s worth bearing in mind that the ELF standard has an unhealthy relation with a certain litigious corporation and hasn’t been updated in some time [that was at least my experience when I read both the 32 and 64 bit versions of the ELF spec and inquired who was maintaining it now.]

Power capping. Salman Qazi posted to let everyone know about a project within Google refered to as “power capping”. Briefly, this is an effort to break the longstanding relation between power supply and demand in the data center by overcommitting resources and running those resources more efficiently, by using such measures as “forced idling” of CPUs within the Linux kernel. The email thread was entitled “RFC: A proposal for power capping through forced idle in the Linux kernel”, and detaile a proposal for in-kernel assistance, an “idle cycle injector”, and so forth. Similar work has been done before, but it generated a fair amount of discussion. It would seem one of the ideas here is to allow datacenters to ride out temporary reductions in available power, rather than powering down systems. Arjan van de Ven would like to see all CPUs idling in unison, while Salman is interested in this being a userspace controlled policy decision.

OOM. Hiroyuki Kamezawa posted another version of his mm rss counting patches, intended to faciltate easier statistical gathering of RSS (Resident Set Size) data per task, and without scalability impact. This can be used to improve the OOM killer, and for other purposes. This version is less “invasive”.

TSC. Zachary Amsden posted a series of preparatory patches intended to unify support for the virtualized TSC emulation between VMX and SVM implementations within Linux. There is a lot of churn currently in this space as folks attempt to present a consistent TSC experience.

In today’s announcements: rt-tests version 0.57. Clark Williams announced that version 0.57 of the rt-tests package intended to provide various test cases for the preempt-rt kernel patch series, is now available from kernel.org. On a related note, Keith Mannthey (IBM) announced that the “ibm_rtl” driver is available, a module that tells various IBM BIOSes to turn off particular gregarious SMIs that can cause all manner of system latencies. I wonder if we need a generic framework for doing this and am considering it.

The latest kernel release was 2.6.32.

Jens Axboe reported a kexec boot regression affecting kernels after 2.6.32. In order to avoid excessive boot delays, Jens test boots new kernels through an kexec call in general, and this failed recently due to a patch from Yinghai Lu, according to the git bisect (a patch that was confirmed working was posted). Matthew Garrett also reported an ACPI (non-)boot regression.

Stephen Rothwell posted a linux-next tree for December 15th. Since Monday, the origin tree gained a build failure for which Stephen applied a patch, the md-current tree lost its conflict, the mips tree gained a conflict against Linus’ tree, the 52xx-and-virtex tree lost its build failure but gained a conflict against Linus’ tree, the acpi tree still has its build failure so Stephen used the version from Friday, the voltage tree gained 2 conflicts against Linus’ tree – one requiring a fixup and the other requiring a rollback of the voltage tree to the version from November 20th, the devicetree tree lost its build failure, the limits tree lost three conflicts, the tip tree lost its conflict, and the percpu tree lost its conflicts. Stephen repeated his usual “call for calm” in not merging patches intended for 2.6.34 until after the release of 2.6.33-rc1. The total subtree count remains steady at 155 trees in the latest compose.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.