LKML Summary Podcast

2011/03/27 Linux Kernel Podcast

jcm — Sun, 27 Mar 2011 21:52:03 +0000

Audio: http://traffic.libsyn.com/jcm/linux_kernel_podcast_20110328.mp3

For the weekend of March 27th 2011, I’m Jon Masters with a summary of this week’s LKML traffic. Yup, it’s back, sorry it was away so long.

In this week’s issue: SMBIOS, Lazy patch checking, and farewell to APM.

SMBIOS vs. DMI. SMBIOS has long since replaced the DMI specification (the last update to DMI was apparently in 2003), but the kernel is still DMI-centric, having been overloaded to understand various parts of SMBIOS. This is a trend that has been going on for some time, but it finally annoyed Prarit Bhargava enough to want to do something about it – he proposes various changes to expose SMBIOS data directly in sysfs in a thread entitled “SMBIOS: Add initial code and export version via sysfs”. Alan Cox was keen to retain backward compatibility and preserve the same userspace API, even if the actual data being provided came from SMBIOS rather than legacy DMI. (disclaimer: I discussed this with Prarit as part of a separate issue).

Lazy Patch checking. Sarah Sharp hit upon a common problem when cleaning up existing code. Checkpatch would report an error on lines exceeding 81 characters in length, even though she didn’t write the original code. To make such review and improvement work less cumbersome, Sarah adds a “–lazy” flag to checkpatch.pl to change the return code on such errors. She notes she’s “not a perl hacker”, which is of course not a failing.

Farewll APM. Len Brown posted a feature-removal-schedule update that will plan the removal of “APM” power management in 2.6.40. It’s been a long time since ACPI and its kin killed off the need for APM on modern systems. Apparently, Microsoft pulled this support in 2006, so 5 years later is probably more than sufficient for Linux systems(!).

In this week’s pull requests for the 2.6.39 merge window:

* Stefan Richter requested Linus pull some firewire updates, including an ALSA bug fix.

* Jean Delvare requested Linus pull some hwmon subsystem updates, including support for two new devices (ADS1015, and SMSC SCH5627).

* Alex Elder requested Linus pull some XFS updates.

* Michal Marek requested Linus pull some kbuild updates (v2).

* Ingo Molnar requested Linus pull some x86 fixes.

* Miklos Szeredi reqested Linus pull some fuse updates.

* Pekka Enberg requested Linus pull some SLAB changes that include SLUB lockless fastpath patches from Christoph Lameter.

* Sage Weil requested Linus pull some Ceph and RBD updates, including “support for an object watch/notify framework that allows and RBD block device to get/send notifications about snapshot creation”.

* Dave Airlie requested Linus pull some drm fixes, including “One radeon, 2 core fixes, and an interface update to allow for 2 > crtcs in vblank”.

* Takashi Iwai requested Linus pull some sound fixes for VIA HD audio codes, and “a few other small fixes”.

* Samuel Ortiz requested Linus pull some MFD fixes, including MFD cell sharing support, and new drivers for TI’s TPS6105x, Maxim MAX8997, STE AB8500 GPADC, and Intel’s Tunnel Creek).

* Roland Dreier requested Linus pull some infiniband fixes.

* J. Bruce Fields posted some nfsd changes, including a new interface from Kevin Coffman to allow userspace to query for enctype support.

* Al Viro posted some vfs and procfs fixes, including fixes for “dumb leaks in path_lookupat() and a large pile of procfs fixes”.

* Phillip Lougher requested Linus pull some squashfs updates, including support for reading compression options out of the filesystem.

* Florian Tobia Schandinat posted some viafs cleanup patches.

* Colin Cross posted some ARM Tegra updates.

* Len Brown requested Linus pull some ACPI patches, and some idle patches.

* Mike Frysinger requested Linus pull some changes for blackfin.

* Mauro Carvalho Chehab requested Linux pull some drivers/media updates, which includes the addition of the “Media Controller API”, required to “adjust stream parameters on SoC media devices”.

* Alasdair Kergon requested Linus pull some device-mapper patches.

* Artem Bityutskiy stated he had found “a couple of brown-paperbag bugs”, so posted a second UBI/UBIFS pull request to take care of the matter.

* Anton Vorontsov requested Linus pull some battery fixes.

* Ted Ts’o requested Linus pull some ext4 updates.

* Trond Myklebust requested Linus pull some NFS client changesets.

* Martin Scheidefsky requested Linus pull some s390 patches.

* Ingo Molnar requested Linus pull some core kernel, IRQ, perf, and x86 fixes.

* Thomas Glexiner requested Linus pull some additional IRQ cleanups. These included code for architectures which Ingo considers “orphaned”, but Ingo had done some cross-building at least to check they still build.

* Jesse Barnes requested Linus pull some PCI fixes.

* James Bottomley requested Linus pull some “final SCSI updates”.

In this week’s miscellaneous items:

* Amerigo Wang posted a patch implementing “acpi_addr” kernel command line handling for passing the ACPI RSDP structure (pointer) into kdump kernels. Cong Wang suggested that this be renamed acpi_rsdp instead.

* Shawn Guo posted some patches to SDHI support to take driver specific bits out of sdhi-pltfm.c and “make them self registered”.

* Joseph Cihula posted a patch disabling Intel VT-d Protected Memory Regions (PMRs) when the kernel has been launched with TXT (trusted execution).

* Tony Luck posted a patch removing normal shutdown logs from the new “persistent store” (pstore), to avoid cluttering up limited storage. Seiji Aguchi considered it useful to retain normal shutdown logs for Enterprise use. Artem Bityutskiy and Americo Wang considered this policy in kernel and offered a mount option as the preferred solution to deciding logging action.

* Sarah Sharp noticed that PCI hotplug wasn’t working properly on 2.6.38 for her particular setup. Rafael J. Wysocki pointed out a stable-bound patch, which Sarah tested and soon found PCI hotplug working again.

* Dave Chinner posted some “vfs: inode lock breakup” patches, which are “derived from Nick Piggin’s vfs-scale tree”. He has been “sitting on them until the dache_lock breakup and rcu path-walk has had some time to be shaken out”. He forward ported the patches and ran some XFS/ext4 tests.

* Paul Turner posted version 5 of his CFS Bandwith Control patch series.

* Herbert Poetzl inquired about “debugging a modern laptop …”, which Andi Kleen replied to. In his reply, Andi pointed out the existance of the USB DebugPort, which is an optional feature built-in to many EHCI controllers. There’s usually only one, and it requires a $100 cable, but this is something this author has not played with and suspects many others haven’t either. An investment of $100 isn’t bad if you need to routinely debug laptop kernels. Mark Lord added that those with ExpressCard slots could pick up an “Oxford Chipset” card, which provides true serial/parallel ports, the latter of which can also be used for homebrew JTAG adapters (according to Mark).

* Kamezawa Hiroyuki posted an RFC patch entitled “A forkbomb killer and mm tracking system”, which is all new compared with older posts, and apparently includes support for dead process tracking (struct mm_struct tracking).

* Tejun Heo was cleaning up the custom locking in btrfs, migrating to standard kernel mutex adaptive spinlocks. He noted that mutex_trylock doesn’t currently do adaptive spinning, but that it probably should in his benchmarks (and that btrfs equivalent code was already doing this). Linus thought this sounded reasonable, and Tejun updated his patches accordingly.

* The Microsoft HyperV folks posted a few staging cleanups.

* Thomas Gleixner posted some final ARM genirq cleanups.

* David Miller posted some Networking, and SPARC cleanups.

* Rafael J. Wysocki requested Linus pull some more power management updates.

* Jonathan Corbet posted some documentation updates.

* Jens Axboe requested Linus pull some block driver updates.

* Tony Lindgren requested Linus pull some OMAP fixes.

* Guenter Roeck requested Linus pull some hwmon fixes.

* Liam Girdwood requested Linus pull some voltage regulator updates.

In today’s announcements:

* Christian Dietrich announced “undertaker” version 1.1. This is another static code analysis tool for C code that users preprocessor directives. It’s part of work done by the VAMOS team at the University of Erlangen.

* Scott James Remnant announced the release of version 1.2 of “Upstart”, the premier legacy init replacement daemon for Linux systems. In this release, Scott corrects a “brown paper bag” bug that broke use of pdksh.

* John Linn requested that Stephen Rothwell include the new Xilinx ARM tree in linux-next.

* Junio C. Hamano announced the releast of Git 1.7.4.2. It includes lots of fixes, even half hour GMT timezone offset support in gitweb.

The latest kernel release is 2.6.38.

Greg Kroah-Hartman announced the release of stable kernel 2.6.33.8, which is intended for use by the pre-empt RT kernel patch, which is still 2.6.33 based.

Greg Kroah-Hartman announced the release of stable kernels 2.6.32.34, 2.6.38.1, and 2.6.37.5. Later, Greg posted 2.6.32.35, which had only a build fix for those bitten with a particular build problem on 2.6.32.34.

Tony Luck requested Linus pull some IA64 changes.

Greg Kroah-Hartman requested Linus pull some USB changes for problems reported so far during the 2.6.39 merge window.

Paul Mundt requested Linus pull part two of super-H updates and some rmobile updates.

John Linville posted some further wireless fixes for 2.6.39 to Dave Miller.

David Howells requested Linus pull some NOMMU and FRV patches.

Con Kolivas announced the release of 2.6.38-ck1, which includes various patches of his intended to “improve system responsiveness”.

That’s a summary of the past week’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Finally this week, Lukasz Sokol noted that the UK branch of WalMart is selling a very tux-like product, wondering if this was a legal issue. Alan Cox responded that Larry Ewing was “sort of CC-BY”, requiring attribution if asked, so that was probably ok. But more worrying to Alan was “the unfortunate placing of the stick which means it looks like its taking a crap”.

2010/11/07 Linux Kernel Podcast

jcm — Sat, 27 Nov 2010 06:23:58 +0000

Audio: COMING SOON

For the weekend of November 7th 2010 (and uploaded from 35,000ft), I’m Jon Masters with a summary of the past week’s LKML traffic.

In today’s issue:

*). Linux 2.6.37-rc1. Linus Torvalds announced the release of Linux kernel 2.6.37-rc1 on Monday Nov. 1st 2010 at 08:07am Non-Best Coast Time (NBCT). In his announcement, which he sent from the Hyatt Hotel in Cambridge, MA used for the 2010 Kernel Summit, Linus notes that “there’s a lot of changes there – just shy of 10k commits since 2.6.36 – despite the slightly shortened merge window [due to the kernel summit]“. He called out special attention to the removal of the BKL (Big Kernel Lock) in all of the core kernel stuff (pretty much most things aside from V4L at this point) and noted that it’s easy to compile away at this point. That effort was lead by Arnd Berman and others, who deserve credit. I myself have a laptop running an RC kernel with the BKL compiled away (I can live without the webcam for the moment to get the benefit of having killed the BKL).

*). Google’s Android kernel. A thread entitled “Forked android kernel development from linux kernel mainline” discussed the state of affairs with regard to Google maintaining a separate Android kernel containing various non-upstream core features exposed to userspace, such as the wakelocks used on Android devices to prevent the device from suspending (Android devices otherwise aggressively try to suspend by default). On Android, drivers must be modified to be aware of these wakelocks. Greg Kroah-Hartman and Tes Ts’o (who works at Google but explicitly disclaimed that he was speaking for himself alone) had a difference of opinion. Both agreed that the work to get such a feature upstream was complex. Ted noted that if such code did go into Greg’s “staging” tree that it would probably be yanked after 6-9 months due to lack of forward progress. He also noted that “someone no less than Linus Torvalds has said that sometimes forks are good, and that the _freedom_ to fork is crticial”.

Greg noted that under Rusty Russell’s “rule of lkml etiquette, we are all allowed to participate, or start, one massive, no-holds-barred, ugly as mud flame war per year.” he composed such a missive, but deleted it and instead replied with “I respectfully disagree with your opinion, so we will have to just agree to disagree at the moment”. Amidst the slight hostility, Alan Cox wondered aloud with Ted Ts’o why the Android drivers couldn’t be submitted without the wakelock code and then a small patch be developed to add that back in for the Android specific kernel.

*). VM. Mandeep Singh Barnes, Kosaki Motohiro, Rik van Riel, and others discussed low-memory handling on Google’s Chromium OS. On that OS, which behaves much as other Embedded Linux Operating Systems, there is no swap volume/partition implemented, while simultaneously there is a desired to retain a certain level of performance and interactivity in displaying web pages even under low memory situations. Chromium apparently currently uses an approach in which, under low memory conditions, pages are directly reclaimed from the file list, which results in a lot of thrashing as pages are evicted and pulled back in for various applications that don’t use features such as mlock and mlockall to retain their RSS in physical RAM. The discussion did turn to why a combination of mlock and mlockall could not be used – which can be briefly summarized as being due to the very coarse nature of using mlock and mlockall – and lead onto further talk of implementing a min_filelist_kbytes tunable beyond which the system would not go (prefering to OOM in certain situations, but before reaching the point of utterly pathetic performance).

The conversation failed to address a generic solution to forcing early reclaim in such situations of the type that Rik and I have discussed in another setting and currently have on my todo. In my proposal (in which the kernel co-operates with userspace more in such situations), the kernel sends a generic notification to userspace applications – that poll a file descriptor for such notices from the kernel – that system memory resources are running below some threshold (way before OOM). The kernel asks these applications to voluntarily relinquish some of their internal caches and other memory that can be returned to the kernel. If the application complies, it may get looked upon favorably if the OOM still has to run. Thus, Firefox can receive a notice to flush its internal page (web) caches, and virtual machine host qemu processes can balloon down their resources accordingly. I’ll write up and post my proposal to LKML. I had held off because the code isn’t there yet…but now the discussion is and I would hate for us to miss an opportunity to fix this problem properly.

In today’s miscellaneous items:

*). Dave Chinner posted some locking cleanups for the VFS on top of the recent “inode lock breakup” work. The thread was entitled “inode freeing and hash lookup via RCU”.

*). A thread entitled “cgroup: Avoid a memset by using vzalloc” in which it was pointed out that there are many places within the kernel wherein a chunk of memory is vmalloc’d and then memset with zeros that might benefit from simply using vzalloc in the first place prompted Christoph Lameter to recommend a semantic patch of the form being developed by Julia Lawall under the Coccinelle project. The spatch utility even comes with direct examples of such find and replace (though using kmalloc, which is easily substituted for its virtual memory cousin). Still, Jesper Juhl said he would use his tried-and-tested approach for now to avoid taking the time hit from learning a new tool. He did seem interested in spatch however.

*). Dave Chinner and Wu Fengguang debated the “soft and dynamic dirty throttling limits” patchset tht allowed for modification of the low threshold number of dirty pages beyond which the kernel throttle IO. The patchset was causing some problems in a 1-byte test workload.

*). A thread based around a patch entitled “Implement a virtio GPU transport” debated the relative merits of implementing a completely virtualized GPU for guests, as opposed to implementing GL passthough for host rendering. Ian Moltan noted that, due to the design of OpenGL, it isn’t really reasonable to expect to be able to pass rendering hardware through to guests securely (even if each gets its own GL context) but until there is a fancy virtualized GPU available, interim solutions are needed. One of those is the virtio GPU transport that has been posted.

*). Randy Dunlap discovered a problem with the floppy driver in 2.6.37 RC kernels, which was originally confused with another problem, before Linus tracked down the failure to some work done in patch 488211844e0c.

*). Tejun Heo posted a patchset entitled “clean up bdev claim/release handling” that attempts to reconcile the various different functions used to grab and release block devices down to just blkdev_get and blkdev_put. The aim here is to reduce complexity and solve some odd problems, like non-atomic exclusive open caused by the open process being disjoint from acquiring the exclusive use of a block device.

*). Jiri Olsa found that older versions of Upstart were killed by the SAK (Secure Attention Key) after they left /dev/console open. Eric W. Biederman suggested that this was correct behavior and Jiri noted a changelog in more recent versions of Upstart that it had corrected this behavior. The patch Jiri had sent to avoid killing init was therefore not required.

*). The latest Linux architecture (Tile from Tilera) got support for its on chip network devices.

*). Asbjoern Sloth Toennesen posted a Kbuild patch to the deb-pkg target that allows kernel builds to specify ARCH even when not strictly cross-compiling. For example, this allows a 32-bit kernel package to be built on a 64-bit Debian host and vice versa.

*). Some discussion of a “SLAM” (mutable) allocator in the (rejected) paper proposal for this year’ Linux Plumber’s Conference from David Rientjes: http://www.linuxplumbersconf.org/2010/ocw/proposals/405

*). Oren Laaden, Tejun Heo, Christoph Hellwig, and others debated the relative merits of in-kernel vs. in-userspace Checkpoint and Restart in a thread entitled “checkpoint-restart: naked patch”.

*). Kees Cook inquired about “RO/NX protection for loadable kernel modules”. Separately, Dan Rosenberg posted a patch providing a new sysctl entitled “modules_restrict”. This can be used in order to prevent unprivileged users from causing the kernel to load modules indirectly by way of using a service or other kernel feature that will trigger a module load. He based his patch on the concept implemented in the grsecurity patches.

In today’s announcements:

*). Subrata Modak announced that the Linux Test Project for October 2010 has been released: http://ltp.sourceforge.net/
*). James Bottomley sent a “Final Reminder” about the Linux Foundation TAB (Technical Advisory Board) elections and nomination process. These were held at the Linux Plumbers Conference before this podcast was completed.

*). Mathieu Desnoyers announced the release of LTTng 0.239 for Linux 2.6.36. http://lttng.org/content/download

The latest kernel release was 2.6.37-rc1.

Andrew Morton released an mm-of-the-moment (mmotm) for 2010-11-02-12-17.

Michael Kerrisk announced tht man-pages version 3.30 is now available: http://www.kernel.org/pub/linux/docs/man-pages:w

That’s a summary of the week’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/10/31 Linux Kernel Podcast

jcm — Wed, 03 Nov 2010 06:46:53 +0000

Audio: http://traffic.libsyn.com/jcm/linux_kernel_podcast_20101031.mp3

“Well, there’s no difference between 16 and 36, so what would be the rationale for changing it? The only reason is that it’s unsightly and uncommon, but, if you ask me, 36 is _much_ closer to 42 and so is _much_ better. The glory days of kernel 42 are coming. Lo and behold.” — Tejun Heo, “On Linux numbering scheme”.

For Sunday, October 31st, 2010, I’m Jon Masters with a summary of the past week’s LKML traffic.

In today’s issue: ARM, Device Renaming, and Hardware Error Reporting.

Short Merge Window. Linus Torvalds posted a thread entitled “Reminder: short merge window” in which he reminded everyone that the 2.6.37 kernel will have a shorter merge window in order to get the RC1 out the door before the kernel summit. He was “tentatively planning on doing the release using the free PDX WiFi (at the airport). True to his word, Linus released 2.6.37-rc1 on Monday November 1st 2010 at 07:54:12 NBDT (Non-Best Coast Daylight Time). Also worth noting is a reply from Linus to a posting of various NFSd changes for 2.6.37 in which Linus talks about pre-merging, saying that he prefers people do not attempt to pre-merge bits to avoid conflicts with Linus’ tree. He “much prefer[s] seeing the conflicts rather than have them hidden from [him] by a pre-merge.” Linus also added that he wanted to see testing of the CONFIG_BKL disabled option for NFS now that he is planning for 2.6.37 to be BKL-free. J. Bruce Fields said he did some connectathon tests than ran BKL-free ok.

ARM. Catalin Marinas posted an 18 part patch series entitled “ARM: Add support for the Large Physical Address Extensions”. These split ARM page table handling into the classic 2-level and new 3-level LPAE version, which provides for a total of a 40-bit physical address space. The newer LPAE page table format does make a few changes (such as dropping support for domains, and “simplifying” the permission model). A link to the documentation was also provided (free registration is required to view that on the ARM website). Arnd Bergmann and Catalin discussed the patches, including the impact upon future support for the new ARM Virtualization Extensions in KVM.

Device Renaming. Nao Nishijima posted a thread entitled “Device Renaming Mechanism” in which he asked about having the kernel rename devices using some kind of persistent criteria. Greg Kroah-Hartman and Kay Sievers both had reservations about this, since as Greg noted, it’s possible to ascertain things like volume identification using many userspace tools that already form part of the Linux hotplug infrastructure. I noted the DMTF SMBIOS table extensions (type 9) that allow for system vendors to provide slot and device naming information, and asked for further comment on that.

Hardware Error reporting. Huang Ying posted a patch series entitled “ACPI, APEI, Generic Hardware Error Source POLL/IRQ/NMI notification type support”, which implements support for an ACPI error reporting extension. This extension operates in so-called “Firmware First” mode in which errors are first caught by system firmware (which might be able to ascertain more useful detail from underlying hardware registers), then reported to the kernel. The code also adds various other kernel infrastructure to support ioremap in atomic context. It is the third hardware error logging and reporting wheel that has been re-invented in the kernel space, and Ingo Molnar was not happy.

Ingo Molnar replied, NAKing the patch series, with a “WTF?” and saying “Sigh, please integrate all this into EDAC (drivers/edac/) properly, We can do better than this. EDAC is almost there: it has support for Nehalem, AMD, a couple of older chips. Guys, instead of carving out a special driver area where you can produce crap without anyone looking too much, and pretending that the EDAC code does not exist, please try to work with others who are aiming higher and who are using saner interfaces.” Andi Kleen objected, noting that Ingo had no authority to NAK these patches, and adding that there is a difference between EDAC and the ACPI spec implementation. He also noted, “Error injection is hard and one size definitely doesn’t fit all. You need quite different ones depending on what you want to test, in which context etc. For hwpoison we current have three different injectors at least and I expect that to even grow more in the future as different features get added.”

In response to Andi, Ingo didn’t let up, criticizing the potential for vendor-driven fragmentation by not having a single error reporting mechanism in the kernel for such purposes. Thomas Gleixner chimed in, adding that the ‘patch series carries a lot of other weird stuff including a new “memory allocator”, a new ioremap implementation private to the acpi code and new character device driver for hardware error reporting”. He, like Ingo, was critical of the lack of an attempt to “explain why this error reporting cannot be done via the existing interfaces”, ending with “The only explanation I have is that you are simply not willing to work with others and this is just another proof of a repeating problem”. That echoed some similar comments from Ingo in another part of the thread that had become somewhat more personal – Ingo noted various apparent commonalities in the approach Andi had taken to previous situations involving criticism. In the end, it was agreed (mostly by other parties) that this would be discussed face-to-face in Boston this week at the Plumbers Conf. There is a BoF session set aside, which promises to be interesting.

In today’s miscellaneous items:

*). David Howells noted that a previous bug fix for a problem with older versions of gas (the GNU assembler) involving the use of brackets being interpreted as immediate or absolute address references and blowing up one architecture David was working on. After some suggestions for various hacks, Linus Torvalds noted that “This all seems very wrong”, and “the whole thing was added for some stupid has bug for a very specific case”. After some chatter, it sounds as if older versions of gas (such as those in use by certain Enterprise Linux versions) may not be supported in the not too distant future, since the bug is fixed in recent gas versions.

*). David Howells also posted what seemed like a simple fix for percpu alignment on NM10300, which after some discussion first morphed into a patch from Tejun Heo to revert a previous commit that had aligned percpu area and irq stacks to THREAD_SIZE, then after Ingo noting that the revert would re-introduce a nasty boot bug into Linus suggesting that IRQ stacks should no longer be allocated in the percpu area as the “upsides are almost zero afaik, and the complexity has been ridiculous”. To add more fun, Eric Dumazet noticed that “per_cpu data on my machine (NUMA capable) all sit on a single node, if 32bit kernel used”. He ultimately posted a patch that “restore[s] irq stacks NUMA-aware allocations”. In poking at that problem, Eric found a problem that certain users of first_zones_zonelist would ignore the return value, which might be NULL, which bothered Linus and Andrew Morton for the lack of documentation and apparent bugginess. Linus ultimately also noted that he also wants “to remove all the crap that got added for the multi-page percpu support. It was ugly, and apparently never really worked. All the PER_CPU_MULTIPAGE_ALIGNED crud just needs to go away”. Ingo says he will look into it. David later posted 43 patches implementing SMP and AM34 CPU support for MN10300.

*). Tejun Heo and Alexey Zaytsev discussed using devres with network drivers.

*). Elvis Dowson asked about the “+” appended to kernel versions for certain git trees, and was given information about LOCALVERSION behavior in a reply from Stefan Richter.

*). Kamezawa Hiroyuki posted version 2 of his “bug chunk memory allocator”.

*). Timur Tabi asked some questions about tty_port use that Alan Cox answered.

*). Charles Manning asked about the best way to get YAFFS filesystem support finally accepted into mainline (after many years). YAFFS is an excellent flash filesystem, and so it will be exciting to see this merged. Charles is going to read through various documentation and advice, before deciding if there is anything “left to do” (that would necessitate first landing in “staging”) or whether he wants to push straight into upstream (likely).

*). Tommaso Cucinotta asked various questions about “Understanding Cgroups” that started a thread of interest to those trying to also figure this out.

In today’s announcements:

* Greg Kroah-Hartman announced the release of Linux 2.6.27.55, 2.6.32.25, and 2.6.35.8.

* Mathieu Desnoyers announced the release of LTTng 0.236 for kernel 2.6.36. Mathieu also posted an “LTTng Mainlining Roadmap” at http://lttng.org/roadmap http://lttng.org/content/download

* James Bottomley posted a reminder of the Linux Foundation Technical Advisory Board (TAB) nomination process, which took place on Tuesday Nov. 2nd at the joint Kernel Summit and Linux Plumbers Conference reception (voting day!).

* Steven Rostedt announced “ktest.pl”, a perl script implementing a reasonably lightweight testing framework for doing git bisects and quick test kernel builds that are tested on a second machine. It doesn’t attempt to be a replacement for autotest and other bigger projects. http://rostedt.homelinux.com/ktest/

The latest kernel release is 2.6.37-rc1.

That’s a summary of the past week’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/09/20 Linux Kernel Podcast

jcm — Mon, 20 Sep 2010 08:31:33 +0000

NOTE: Backlog episodes are in various states of completion. They will be slotted in as time permits. Let’s see if we can get back to doing this thing regularly

Audio: http://traffic.libsyn.com/jcm/linux_kernel_podcast_20100919.mp3

For the weekend of Sunday, September 19th 2010, I’m Jon Masters with a summary of the past week’s LKML traffic.

In today’s issue: Linux 2.6.36-rc4, compiler versions, media polling and detection, writeback, and much more!

* Linux 2.6.36-rc4. Linus Torvalds announced the latest 2.6.36-rc4 release of the kernel on Sunday, September 12th 2010 at 4:49pm Best Coast Summer Time (BCST). In noting it had been two weeks (rather than one) due to travel, Linus said little stood out at this point in the RC, though he was a little bothered by the amount of GPU driver churn. Linus devoted a lengthy paragraph to calling out the need for greater use of the “Reported-by: tagline in patch message bodies in order to give due credit to those who help to track down and fix bugs, adding “Sometimes the fix is trivial, and the real work was in noticing and figuring our that a problem exists in the first place, and reporting it”.

* Compiler versions. Florian Mickler, Peter Zijlstra, and Peter Anvin debated the fact that older gcc 3.3 compilers were known not to work correctly when building x86 Linux kernels (Peter Anvin noted that some “Enterprise” distros were shipping custom patches and so their “3.3″ compilers did still work). Russell King noted that it would be ok to bump the generic requirement up to GCC 3.4, and then suggested architectures could require a higher version individually as needed. Neither Russell, nor others saw any reason for the generic requirement to be higher than 3.4 at this stage, and Russell noted that ARM developers like himself were still using 3.4 quite heavily.

* Media polling and detection. Maxim Levitsky posted, drawing attention to a potential regression, in a thread entitled “cdrom driver doesn’t detect removal”. Recent work on block device claiming had seemingly changed the logic for emiting uevents from the kernel that udev would pick up and use to trigger a mount or unmount of a CD or DVD device. Except this wasn’t a problem. Kay Sievers noted that recent systems rely on a polling process in userspace that won’t be running unless there is a desktop session, and so the real problem was that this process was not running on the Ubuntu system Maxim has – re-enabling the older HAL-provided process (now replaced with udisks upstream) fixed the problem. Of course, there is a wider problem here that is that no UNIX-like system should need a user running a graphical session (this should be init-initiated).

* Writeback. Michael Rubin posted a five part patch implementing entries in /proc/vmstat that provide visibility into writeback behavior. The two entries, nr_dirtied and nr_written “allow user apps to understand writeback speed over time”. Michael proceeds to then describe why it is important to provide visibility into writeback behavior, “to know how active it is over the whole system, if it’s falling behind or to quantify its efforts”. Apparently, these patches are used at Google in order to allow their non-kernel engineers to solve performance issues.

In today’s miscellaneous items:

* Joe Perches posted a series of cleanup patches intended to remove the extraneous provision of a loglevel in the parameters of the various pr_ functions, which already encode the loglevel into their naming.

* Avi Kivity, Ingo Molnar, and Pavel Machek continued to discuss possible “bytecode” intepretors built into the kernel for defining perf events.

* Robert Richter noted what seemed to be spurious interrupts after disabling performance counters. Actually, these were deemed to likely be already in flight interrupts at the time of the counter disabling, which were then not handled by the time the handler was called, resulting in an erro. Robert posted a patch to catch and handle these “spurious” interrupts.

* Heiko Bauke noted some recent issues with Realtek network cards detecting a link, which seemed to go away when using the (GPL) drivers directly from Realtek. The problem was that two “stable” tree update patches had not been added to 2.6.32 stable kernels. David Miller said he would take care of it.

* Robert Mueller noted that the default zone_reclaim_mode on NUMA kernels was pretty disasterous for performance on his very meaty servers. Cross-node memory use is bad, but not as bad as heavy disk IO with 5GB of RAM free. He and Christoph Lameter have started a threat to discuss default options.

* Mathieu Desnoyers posted an RFC patch entitled “sched: START_NICE feature (temporarily niced forks)”, which bumps the nice level on both parent and child temporarily (for their first slice only). The goal is to be able to reduce the impact to latency-sensitive workloads that do many forks. He included some impressive stats that were “tempting” to Ingo Molnar, and is currently working on a new patch. Ingo would like Mike Galbraith to work his magic to look for bad corner cases with taking this patch. A second version was posted, without any followup at this point.

* Vladislav Bolkhovitin posted a 17 part patch series implementing “SCST”, a new SCSI target framework with device handlers and 2 target drivers.

* Arnd Bergmann posted a 7 part patch series implementing “BKL mass-conversion to mutex”, which is part of a much larger effort that he has been working on for some time. He’d like to see this (and other bits) in linux-next in time to land in the forthcoming 2.6.37 kernel release. Arnd later posted a longer thread entitled “Remaining BKL users, what to do” in which he proposed various ways to removing remaining BKL use from different drivers, filesystems, and so forth. Christoph Hellwig noted that isofs just needed its own private mutex, as had been done in other drivers, for example.

* Dave Hansen posted a patch adding a WARN_ONCE when using drop_caches, and an update to the documentation, since he says “[t]here seems to be an epidemic spreading around. People get the idea in their heads that the kernel caches are evil. They eat too much memory, and there’s no way to set a size limit on them! Stupid kernel!”. This seems to be a heavyweight solution the problem of bad advice on the interwebs, in Google searches. The WARN_ONCE was deemed to be “meddling”, the documentation was well received, and there was some discussion about possible remaining issues that could mean a drop_caches is actually useful for some workloads.

* Christopher Yeoh posted an RFC patch entitled “Cross Memory Attach” that “allow[s] MPI programs doing intra-node communication to do a single copy of the message rather than a double copy of the message via shared memory. The mechanism is to allow a destination process to do a copy from a source process memory directly, using a system call. Apparently, splicing isn’t an option at this stage (zero-copy) due to the need for both processes to work co-opertively over a pipe. Ingo Molnar was impressed with the stats, which used a modified OpenMPI to run some MPI benchmarks, showing a very hugely dramatic speedup in overall MB/s of throughput in all of the benchmarks.

* VMWare decided to rename vmware_balloon to vmw_balloon, apparently following the new convention of “vmw_”, according to Dmitry Torokhov.

* Valerie Aurora posted a 34 part patch series implementing the latest version of her “Union mount core”, for general review. She included a TODO, and a summary of the changes (including to documentation) since the last revision.

In today’s announcements:

* Greg Kroah-Hartman announced the release of stable kernel 2.6.34.7, which contains a fix for a single USB issue apparently bothering “hundreds of OpenSuSE users” at the moment.

* Michael Kerrisk announced that man-pages version 3.26 is now available.

* Junio C Hamano announced Git version 1.7.3. It includes a number of test updates, and some GUI changes, amongst other things.

* Nicholas A. Bellinger announced that TCM/LIO version 4.0.0-rc4 for 2.6.36-rc4 is now available. It includes a large number of changes.

* Phillip Lougher announced the release of squashfs version 4.1.

The latest kernel release was 2.6.36-rc4.

* Greg Kroah-Hartman posted a series of review patches for future stable kernels 2.6.27.54, 2.6.32.22, and 2.6.35.5.

* Rafael J. Wysocki posted a summary of reported regressions from 2.6.34 to 2.6.35, and from 2.6.35 to 2.6.36-rc3-git5. From 2.6.34 to 2.6.35, there are at present 25 unresolved regressions remaining, up from 10 at the start of June. From 2.6.35 to 2.6.36-rc3-git5, there are at present 15 unresolved regressions remaining, up from 13 at the end of August, but the overall number has fallen as others have been fixed. None of the regressions appear to be fantastically earth shatteringly bad ones, although kernel bug 16549 does appear to be quite familiar to this author as a longstanding issue.

Rafael also notes that Florian Mickler has joined the “regression tracking team”, in the capacity of recording and noting regressions that are fixed. It is requested that he be copied whenever such fixes are made available.

* Mathieu Desnoyers posted LTTng 0.230 for Linux kernel 2.6.35.4.

That’s a summary of the week’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

Update on the LKML Podcast

jcm — Thu, 26 Aug 2010 23:03:14 +0000

Folks,

I’ve been super-crazy-nuts busy with my dayjob, but I do have podcasts up to Aug. 1 ready to upload from several weeks ago (literally didn’t have time to post them yet). I will do some catchup but I think the best thing is to just start with the latest stuff and move on, missing a couple of slow weeks from August. Sorry, it takes a lot of work. Sometimes this thing lags, etc. I don’t plan to stop doing it, I just have to prioritize work and sanity (what’s left)

Jon.

2010/07/04 Linux Kernel Podcast

jcm — Mon, 12 Jul 2010 01:21:32 +0000

Audio: COMING SOON

For the weekend of the 4th of July 2010, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Linux 2.6.35-rc4, Btrfs, Defconfig kernel configs, GDB, Timekeeping, and the VM.

*). Linux 2.6.35-rc4. Linus Torvalds announced the release of Linux 2.6.35-rc4 on July 4th 2010 at 8:44pm Best Coast Time (PDT). Linus says he’s been back online for a week and is happy at the relatively small number of changes building up, “having been strict for -rc3″, in his absence. He obviously sees the increased rigidity in enforcing the merge window has been a success, and considers that there will likely be an on time 2.6.35 release, “despite my vacation”. Linus says his vacation was very enjoyable and was the longest time away from the kernel in many years – apparently he did take a cellphone for email, but didn’t do any compiles while he was having “a great time under water.”

*). Btrfs. Edward Shishkin posted a rather scathing technical review of btrfs internal design, criticising variable record size allocations, file system utilization, the balancing algorithms used, and even suggesting that engineers leave the algorithm design up to academics, rather than re-inventing things for their programs. Edward performed various benchmarks and published his results in a thread entitled (variously), “Unbound(?) Internal fragmentation in Btrfs”, “Btrfs: broken file system design”, and “Balancing leaves when walking from top to down”. For his part, Chris Mason was very civil in his reply on a number of occasions, saying that he didn’t see a fundamental design problem existing in Btrfs. Edward “NACKed” Btrfs anyway for enterprise use (even though it’s been in tree for a while).

*). Defconfig kernel configs. Linus Torvalds (in a thread renamed to “ARM defconfig files”) essentially conveyed his discomfort with the continued existance of many dozens (or perhaps hundreds) of “defconfig” files in the architecture directories. These are reference files which are based upon copies of “known good” configuration files. They worked well back in the day, but as Linus says, times have changed and nobody is really making these files by hand any more without using Kconfig. So he proposes replacing them – eating the pain – with single config files per machine type that use Kconfig and source in particulars for the various chip and architecture family particulars. Russell King pointed out that this is basically what already happens, but the point of the defconfig files is to also handle stuff outside of the architecture – for example, choosing not to use certain “IDE” options on particular boards or systems – as Daniel Walker also pointed out. Daniel noted that those setting up e.g. a BeagleBoard or a Nexus One don’t really want to troll through thousands of possible kernel options if a good reference set is available to begin with. Daniel also point out a previous posting for a boolean SATisfiability solver in the kernel config. Linus thought that was interesting but ‘At the same time, “SAT solver” does scream “over-engineering failure” to me’. Linus later explained that he was looking to either kill the defconfigs or replace them with some templates and a means to generate them, but otherwise prefered them to live some place outside of the kernel.

*). GDB. David Howells posted a patch implementing GDB remote protocol support for the “p” command on FRV. The “p” command is used to transfer information about a single register, as opposed to the “g” command, that transfers data on several. But when a gdb client connects, it will attempt to use “p” or “g” and will then stick with that choice without varying. For this reason, Linus wondered aloud if using single reads would actually slow down clients connecting (since they usually will request a number of registers at a time). Jason Wessel said he had actualy done some fairly detailed benchmarking and would share his findings at a later point.

*). Timekeeping. Oleg Nesterov posted a thread entitled “Q: sys_futex() && timespec_valid()”, in which he attempted to summarize some concerns that the glibc folks were having with the Linux implementation of timespec timeouts. Ulrich Drepper replied, explaining that his point was that a negative value for tv_sec in the case of an absolute timeout should not return -EINVAL, but instead -ETIMEDOUT. He contends that a negative relative time in the 1960s is not an invalid time. Linus strongly disagreed, saying, “Ulrich – you’re wrong. Go away.” and then clarified, ‘In the end, it’s quite simple: the kernel doesn’t accept invalid timevals. And negative tv_secs are invalid. It’s that simple. If somebody gives the kernel a timeout from before the epoch [January 1st 1970], that somebody is being a total idiot. We know it’s not a valid absolute timeout, since there’s no way somebody is “waiting” for something that happened in the sixties. Yeah, yeah, maybe you’re waiting for flower power and and free sec. Good for you. But if you are, don’t ask the Linux kernel to wait with you. Ok?’ This author wonders what those still waiting for Elvis will do now that this is clarified.

*). VM. Larry Woodman posted a patch entitled “Call cond_resched() at bottom of main loo[sic: s/k/p/] in balance_pgdat()”, which handles a situation on small single CPU systems wherein a task should OOM (Out Of Memory) and call the OOM-killer, but it does not because kswapd is constantly running due to at least one system RAM zone being below the high page watermark. Larry adds a single cond_resched() call that will allow the watchdog, tasks, and OOM killer to run, freeing up the affected resources. Andrew Morton didn’t like this approach – implying he prefered something more specific than a cond_resched and waiting for the OOM killer to get chance to run – but he could live with it if there were a giant FIXME and/or some documentation at least explaining the essential nature of the specific cond_resched() call as opposed to a regular point of voluntary kernel preemption.

In today’s miscellaneous items:

*). Patrick Pannuto proposed a usleep API for the kernel to augment the existing msleep one, and be used as an alternative to udelay so as to allow the CPU to go into lower power C-states. After some dialogue between Patrick and Daniel Walker, in which Walker pointed out that some stats were needed to prove that this was power beneficial for small delays, it seemed that there was a small improvement for 50us delay values.

*). Ronny Tschuter had some issues with tracing power_start events when using the cpuidle framework with a menu governor and an cpi-based driver to handle idle states. There wer no instrumentation points in the processor_idle code, so he posted a patch, but Arjan van de Ven pointed out that the ACPI STATE type is pretty much “useless random garbage” so the posted should set their system to use mwait idle.

*). Dave Jones raised a concern with crypto and device-mapper. A potential regression was introduced somewhere between 2.6.32 and now, and the details are available in Red Hat Bugzilla 610278. Nobody replied to the posting on the list, but the Bugzilla says that one should be using LUKS, and in the case of not using it the default encryption options were changed due to a vulnerability. It is possible to mount the existing device using the instructions provided.

In today’s announcements:

*). Jeff Merkey announced the latest version of his MDB “Merkey’s Kernel Debugger” x86_64 2.6.34 07-01-2010 Release 4. It’s available on googlecode.com. There has been no community discussion thereof. Jeff also posted his Open Cworthy Libraries 07-01-2010.

*). Junio C Hamano announced Git version 1.7.1.1 is now available at: http://www.kernel.org/pub/software/scm/git/ He also announced Git 1.7.2.rc1 is available for review.

*). Karel Zak announced the latest stable release of util-linux-ng 2.18 is now available: http://www.kernel.org/pub/linux/utils/util-linux-ng/

*). Subrata Modak announced that the Linux Test Project for June 2010 has been released. http://ltp.sourceforge.net/

The latest kernel release is 2.6.35-rc4.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/06/27 Linux Kernel Podcast

jcm — Sun, 11 Jul 2010 21:32:35 +0000

Audio: COMING SOON

For the weekend of June 27th 2010, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Concurrent coredumps, OpenFirmware, and Power management policy.

*). Concurrent coredumps. Edward Allcutt posted, inquiring about placing a limit on the number of concurrent process coredumps that should be allowed to take place on a system. He cited an example Apache-based webserver in which large numbers of CGI processes were crashing, each with a 150-200MB core file that needed writing to disk. He was using a custom patch that would cease dumping cores after a certain number were already concurrently taking place. Roland McGrath and Andrew Morton did not favor this approach, instead prefering either that core dumps would begin to block (but not consuming resources) after a point, or that the blkio_cgroup IO controller be used to limit the IO being consumed. Hiroyuki Kamezawa suggested that distributions like Fedora – which in that case has its own dumping tool called abrt that manages coredumps – could wire up the blkio cgroup prior to beginning the dump process.

*). OpenFirmware. Andres Salomon posted a patch implementing support for making calls into OpenFirmware on x86 OLPC XO systems. The patch works by preserving the necessary page mappings for the OpenFirmware (OFW), which remains in memory at a virtual address. Just the minimum number of mappings are retained, but this does allow calls into the firmware even after Linux has booted. It’s always been interesting to see the XO using OpenFirmware as one of the only x86-based devices doing so.

*). Power management policy. Len Brown posted an RFC patch implementing a new centralized location for userspace to express its power management vs. performance policy preferences to the kernel. In the patch, such expression occurs through the new /sys/power/policy_preference file, which contains 5 different possible levels – ranging from “max_performance”, through “balanced” (the new default), to the “max_powersave” option on the other extreme. The idea is to centralize setting scheduler, cpuidle, governor, and other options.

In today’s miscellaneous items:

*). Dave Chinner posted a 5 part patch series implementing some fixes for emergency filesystem thawing (via sysrq control).

*). Michael Kerrisk posted some man-pages text for the MADV_MERGEABLE and MADV_UNMERGEABLE flags added in 2.6.32 for use with KSM (Kernel Samepage Mapping – the kernel support for detecting duplicate pages in guest virtual machines and mapping them to a single shared page instance).

*). Paul E. McKenney concluded that it was sufficient to turn off the CONFIG_PROVE_RCU option in Fedora rawhide kernels since it’s mostly a developer tool, rather than change licensing or otherwise make it available to non-GPL modules with which it is not compatible.

*). Luis R. Rodriguez posted a script and some documentation to implement some rudimentary ASPM (a PCI extension that allows devices to go to an entirely electrically idle bus state) support. For further information: http://wireless.kernel.org/en/users/Documentation/ASPM

*). Konrad Rzeszutek Wilk posted a 19 part patch series implementing PCI pass-through for Paravirtualizaed Xen guests, using SWIOTLB support.

*). Mike McCormack wasn’t happy with the 32 (NGROUPS_SMALL) group limit on the number shown in /proc/ /status for a given process ID. He and others discussed various ways those who really want more than 32 groups assigned to a process could get the full data through various API changes.

*). Rusty Russell posted the last (hopefully) of his cpumask patches which he says now also means that everyone should be using the cpumask_functions. At least, everyone in kernel is, according to his tests on 32-bit.

In today’s announcements:

*). Mathieu Desnoyers announced that LTTng 0.218 for kernel 2.6.34 is now available. For further information: http://www.lttng.org/

*). Henrik Rydberg announced version 1.0.1 of the mtdev Multitouch Translation Library is now available (releaseed under the MIT license). mtdev does all of the necessary finger tracking pieces in userspace, and separate from the Xorg driver from which it came, as a means to further adoption. This author is still waiting for his Apple Multitouch keypad to work on a Fedora system without having to patch the kernel with a kludge. mtdev is available at: http://bitmath.org/code/mtdev/

*). Len Brown announced the Boston Linux Power Management Mini-Summit will take place concurrently with the Linux Foundation LinuxCon 2010, on the day immediately prior to the beginning of the main events, August 9th. For further information: http://events.linuxfoundation.org/

The latest kernel release was 2.6.35-rc3.

Finally today, Piotr Hosowicz wondered aloud why Linus’ git repository was not being updated, asking if it’s because he’s on vacation. As mentioned before, Linus was indeed on a (well deserved) vacation.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/06/20 Linux Kernel Podcast

jcm — Sun, 11 Jul 2010 02:59:24 +0000

Audio: COMING SOON

For the weekend of June 20th, I’m Jon Masters with a summary of today’s LKML traffic.

In today’s issue: Panic, Performance Events, Slow-work, and Timekeeping.

*). Panic. Shoichi Tamuki posted version 2 of a patch intended to fix keyboard LED blinking on panic. Existing systems will call mdelay to handle the reboot timeout post-panic, during which time the keyboard LEDs well blink. When a hypervisor is being used, those mdelay calls of 1 second or more will be implemented as spins, in order to avoid timeout accuracy slips, but the side effect is that the keyboard LEDs won’t blink properly. The patch will call panic_blink_enter() between every mdelay call, and it also fixes up the longer mdelays so that the blinking still occurs.

*). Performance Events. Nils Carlson, Andi Kleen, Eric W. Biederman, Tony Luck, and others, discussed the “Hardware Error Kernel Mini-Summit” followup in which it had been proposed to introduce a new hardware error subsystem. They pondered what (mostly) Andi saw as failings of EDAC and the need for a better way to find such things as which DIMM has failed without doing a binary search removal of individual modules (”the way of the 21st century”). Tony Luck proposed some further ideas for a generic subsystem.

*). Slow-work. Ted Ts’o reported that recent 2.6.35 kernels with an Ubuntu userspace would periodically get into a state in which large amounts of CPU time was spent in the kslowd worker threads. It turned out that this was caused by a change to the DRM/KMS code to pull polling of the display connectors into the DRM core. Reverting a specific commit fixed the issue for Nick Bowler, who had also been experiencing this problem.

*). Timekeeping. Suresh Rajashekara inquired as to what appeared to be a problem with timekeeping on his OMAP1 platform with a 2.6.29 kernel. It seemed odd that certain timers were not expiring immediately upon resume on a system that tries to spend most of its time in a suspend state (waking for 35 milliseconds every 4 seconds, apparently). Thomas Gleixner replied, saying that during such suspend operations, only the CLOCK_REALTIME based timers are kept correct (aligned to real time), whereas others won’t expire the moment the system resumes because there may otherwise be a thundering hurd problem as many timers expire at the point that the system wakes up from the suspend state.

In today’s miscellaneous items:

*). R. F. Burns inquired as to whether it was possible to “write a kernel module which, when loaded, will blow the PC speaker?”. Alan Cox replied that this wasn’t really likely, and in the absence of the root password and proper expertise, “throwing it out of the window or feeding it iron filings will work just as well.”

*). Lai Jiangshan posted a patch removing the use a default write bit with EPT page allocations under KVM virtualization. It wasn’t causing a problem now since get_user_pages is always called with write=1 at the moment.

*). Adrian Hunter posted MMC patches adding support for secure erase, trim, and secure trim – all now variants of erase in eMMC v4.4 cards.

*). Peter Zijlstra noted that the historical uses of perf_disable to prevent NMI races in the PMU code were basically now done per-arch, so he suggested that he would remove perf_disable as it did not seem to be really needed.

*). Christoph Hellwig posted the XFS status update for May 2010, in which he noted several of the important features that lands in 2.6.34 (including new inode and quota flushing code). Christoph also posted a patch (not entirely related to XFS) that removed the 4K stacks option on 32-bit x86 systems as it is deemed “too small” these days, even with now mandatory split IRQ/kernel stacks, given the depth of many kernel call chains.

*). A number of objections to the new automated addition of a “+” to the localversion for modified kernel trees, if no other is set. Mark Hills pointed out that this triggers a lengthy modpost step even when doing “casual kernel development” to test out some simple patch.

*). Dan Carpenter posted a patch that changes the output of kernel oops messages such that the previous “cut here” is replaced with a message asking for the entirity of the oops to be sent in to kernel folks.

*). Zachary Amsden (who has been working on this for some time) posted some TSC cleanup patches and documentation for KVM. This should help resolve many of the issues that have been affecting some TSC users under KVM. On that note, Hagen Paul Pfeifer sent a patch that effectively allows for deliberate speeding-up of time for certain guests for testing use.

*). Huan Ying posted a three-part “Unified NMI delayed call mechanism”, which essentially allows the deferment of certain NMI-time processing until the NMI context has been left. Ingo Molnar prefered that the solution be to re-use the existing unified NMI watchdog code. Sadly, the rest of the thread turned into a bit of a flamewar between Andi Kleen and Ingo.

In today’s announcements:

*). Jeff Merkey announced Open CWorth Libraries 06-19-2010, and ranted about wanting larger stack sizes. He also posted version 2.6.34-06-17-2010 of his “MDB” or “Merkey Debugger”. Nobody replied to any of these threads.

*). Etienne Lorrain announced version 2.8.2 of the gujin GPL bootloader. It contains several bugfixs and improvements – http://gujin.org/

*). James Morris announced the Program Schedule for the Linux Security Summit that will run in conjunction with the 2010 LinuxCon in Boston, on August 9. Further information is available at http://www.linuxfoundation.org/

*). Karel Zak announced that the second util-linux-ng 2.18 release candidate is now available. It contains lots of fixes (e.g. disable DOS mode and cylinders by default now in fdisk). Further information is available at: http://www.kernel.org/pub/linux/utils/util-linux-ng/v2.18/

*). Mathieu Desnoyers announced the release of Userspace RCU 0.4.6. The latest release includes added ARMv7l support. Further information is available at: http://www.lttng.org/urcu/

The latest kernel release was 2.6.35-rc3.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/06/13 Linux Kernel Podcast

jcm — Thu, 17 Jun 2010 06:53:36 +0000

Audio: COMING SOON

For the weekend of June 13th 2010, I’m Jon Masters with a summary of the past week’s LKML traffic.

In today’s issue: Linux 2.6.35-rc3, ext3, floppy, Kconfig, Memory Corruption, and Suspend Blockers.

Linux 2.6.35-rc3. Linus Torvalds announced the latest RC of 2.6.35 on Friday, June 11th 2010 at 8:01pm Best Coast Time (PDT). Quoting Linus, “So I’ve been hardnosed now for a week – perhaps over so – and hopefully that means that 2.6.35-rc3 will be better than -rc2 was. Not only do we have a number of regressions handled, we don’t have that silly memory corruptor that bit so many people with -rc2 and confused people with its many varied forms of bugs it seemed to take, depending on just what random memory it happened to corrupt. One effect of being strict is that this is likely the smallest -rc3 we’ve had in a long long time.” Linus has been on a bit of a crusade in recent times to reduce the churn post merge-window, which is what he means in describing himself as being “hardnosed” to take patches.

Ext3. Jeff Merkey posted a thread entitled “EXT3 File System Corruption in 2.6.34″, in which – in between his usual extraneous use of 4-letter words and outright offensiveness – he raised an issue that he was seeing on some systems running ext3 and recovering from a power failure. His workload raised an older concern that has been addressed before: file payload corruption following a power failure. The filesystem recovered its journal, but the content of the file was replaced with random other data (not either the original data, or the new data, as might have been expected). Jeff and Eric Sandeen had a discussion about the need to mount with data=ordered vs. the fact that Linus had previously changed the default such that the kernel config option CONFIG_EXT3_DEFAULTS_TO_ORDERED is no longer the default. Ted T’so noted that in lieu of explicit use of syncing within apps, everything is a tradeoff. Users could always mount with “-o sync” to guarantee things hit the disk, “but then the performance will be horrible”.

Floppy. Stephen Hemminger noted a number of issues with the very legacy floppy driver. Linus wondered why anyone really cared about floppies any more, but he agreed that oopsing was always bad. Stephen says he has a few fixes for the immediate issues that he will target for 2.6.36 (most of these were generated under virtualization without using real hardware), and both he and Linus seem quite interested in converting floppy over to threaded irqs, amongst other things. As a super special offer, after reading an online report on a website selling floppy drives (in which one purchaser noted that 3 out of 5 floppy drives had failed within 45 days) Linus even agreed to invest a cool $7.99 to pay for a real floppy disk drive for anyone twisted enough to really want to fix that code, while he himself will not “touch it…with a 10-foot pole”.

Kconfig. Vegard Nossum mailed to let everyone know that he has been accepted into this year’s Google Summer of Code (GSoC) program and will spend the summer working on integrating a “proper boolean constraint satisfiability solver into the configuration editors (menuconfig, etc.) in order to allow partial/incomplete configuration specifications. In short, this means that the user can choose to not specify a particular value for some config options, but let the system deduce their values”. There was some talk of the libzypp library used by Novell’s zypper boolean SAT enabled package manager, though James Bottomley noted that the difference here is that while sometimes package combinations result in an unsatisfiable install, kernel configuration should always be resolvable int a set of valid config options – a checker can be used to flag up broken Kconfig combinations and fix them. This conversation actually began a few weeks back, but had not previously been covered here.

Memory Corruption. It has been well known for some time that certain firmware (PC BIOS software) can be particularly prone to cause low memory (less than 1MB) corruption during suspend or other state transitions, including a mere call into a BIOS-provided System Management Mode (SMM) function. Yuhong Bao noted that the problem was also known to Microsoft, and he pointed to a Microsoft WHDC article covering the problem (and the fact that Windows 7 does not typically make use of low memory for this reason) in a posting entitled “Windows side agrees that lowmem corruption is a problem too”. Ingo Molnar noted that in one corruption case he saw, there was an on screen graphical bitmap that had been spewed all over the low memory. Peter Anvin suggested to just not make use of the first few pages (64K) of “low memory” region at all in order to avoid the increasing number of systems that corrupt it taking down the kernel and causing a system crash. He said that the rest of the lowest 1MB could be kept for ZONE_DMA use only “or something”.

Suspend Blockers. Ingo Molnar, replying to a separate message from Ted T’so, begun a lengthy thread entitled “suspend blockers & Android integration”. Google’s Android platform implements suspend blockers as a means to do what is implied in the naming: provide a means for applications to prevent the system from entering a suspend state. In the case of Android, this is apparently because it will attempt to suspend even with running applications in many cases where it has not been explicitly told not to, to save power. Google’s feature is interesting, but there have been some obstacles to it going into the upstream kernel, due it’s semi-invasive nature (and the objection of some to adding such features to the kernel). Ingo objected to Ted’s comment that “hundreds of engineering hours have been made trying to accomodate the demands of LKML — and LKML has said no to suspend blockers/wakelocks”, noting the many possible features that might have been added to the kernel if engineering time was the sole driving factor, rather than a pursuit of a more perfect option. Ingo believes “Linux is an engineering effort that has literally cost about ten thousand man years. That’s about a 85 million man hours. It takes effort to keep that kind of work valuable!”. Linux Weekly News had a lengthy article on this topic, so I’m going to suggest you subscribe and read that for more detail.

In today’s miscellaneous items:

*). Linus Walleij posted a patch implementing MTDparts style partition table specification via kernel command line parameter. The rationale here is that not all embedded systems use standard partition tables, as was already the case with MTD partitions. This extends the MTD concept to cover regular block devices on some newer embedded devices.

*). Yanmin Zhang reported a hang in btrfs related to calling sync() at the end of a test cycle. The kernel would ultimately output a hangcheck warning.

*). Some online controller reset patches for megaraid_sas from Bo Yang.

*). Jari Ruusu reported a module reference counting issue affecting block device mount and unmount that was caused by a bug introduced in a patch from Tejun Heo. Al Viro rediscovered the problem (bd_start_claiming grabs an extra reference that is never released), which Tejun had already fixed and was queued up for inclusion in a later kernel tree.

*). Patrick J. LoPresti posted a patch for a NULL pointer dereference bug he thought he had found in the device mapper multi-path code, and wondered aloud whether the Coverity folks were still running their nightly checks (he thought this was something that should have been flagged up by Coverity automatically in the course of such checks). Last time I heard, Coverity had had some issues with automated nightly runs but scan.coverity.com seems to still contain various data pertaining to linux-2.6.

*). Alan Olsen (whose name I had to extract from whois records because his email client is not configured to include it) noted a modpost segfault that Krzysztof Halasa tracked down to a faulty offset calculation in reading module license data. It was wondered why this had been ok with previous GCC releases, but in any case was urgently suggested for 2.6.35.

In today’s announcements:

*). Karel Zak announced the latest RC version of the util-linux-ng package is available at ftp://ftp.kernel.org/pub/linux/utils/util-linux-ng/v2.18/ The release candidate removes the rdev, ramsize, vidmode, and rootflags legacy utilities, adds a new libmount library for utlities such as mount (though the library API is not yet officially stable), and adds other commands cuh as findmnt, fsfreeze, swaplable, and so forth.

*). Robert P J Day announced that he has created an “online beginner’s kernel programming course” for $39, the details of which are available at http://www.crashcourse.ca Robert notes on that site that he will require a course text in the form of Robert Love’s upcoming third edition of “Linux Kernel Development”. That seems reasonable – this author already has a copy on pre-order, and it will be officially released on July 5th.

The latest kernel release is 2.6.35-rc3.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.

2010/06/06 Linux Kernel Podcast

jcm — Wed, 09 Jun 2010 06:55:14 +0000

Audio: http://traffic.libsyn.com/jcm/linux_kernel_podcast_20100606.mp3

For the weekend of June 6th 2010, I’m Jon Masters with a summary of the past week’s LKML traffic.

In today’s issue: Linux 2.6.35-rc2, Hibernation, KVM clock, Phoronix benchmarks, Threaded Interrupts, and UML.

Linux 2.6.35-rc2. Linus Torvalds announced the release of 2.6.35-rc2 on Saturday, June 6th 2010 at 9:15pm Best Coast Time (PDT). Quoting Linus, “So -rc2 is out there, and hopefully fixes way more problems than it introduces”. Linus expresses some concern at the size of the RC2, which although not as big as the same time last cycle (that was unusally big in general however), was nonetheless larger than he would have liked and less in line with his intention to have a “calmer release cycle this time”. He says he’s going to enforce making RC3 more sane because “the upcoming week is the last week of school for my kids. And when the kids get out of school, I’m going to be offline for a while. And as a result, I _really_ don’t want to pull anything even half-way scary in the next week for -rc3″. He doesn’t specificially mention the Phoronix regression covered later in this episode.

Hibernation. Nigel Cunningham (TuxOnIce) posted some ponderings on idealized algorithms for writing memory images to disk during hibernation in a thread entitled “Proposal for a new alhorithm for reading & writing a hibernation image”, in which he postulated (amongst other things) essentially setting up a new set of page tables while existing pages are being written out to disk such that future writes to memory would result in a page fault that could be used to setup a number of new tables of pages that should later be written out as having changed during the disk write operation. Current algorithms do play games with page faults, but don’t use secondary tables.

KVM clock. Orion Poplawski followed up to a posting from last week in which he noted that guest virtual machines would occasionlly experience wild jumps in time of 6-12 hours. Gleb Netapov, Alexander Graf, John Stultz, Zachary Amsden, and others asked some questions – for example whether ntp was in use or not – and Zachary noted that a theoretical problem existed with the way that some CPUs have seemingly reliable TSC increment until they enter certain power states, at which point the clocking can break. He has some patches in the works to handle this server-side for kvm-clock and he noted that newer kernels should also detect an unreliable host clock quickly on boot. I should add that I have previously experienced similar issues with kvm-clock and raised them directly with the KVM folks quite a few months ago now.

Phoronix benchmarks. Phoronix are known in certain kernel circles for posting “benchmarks” of various new features, especially those targeting the desktop, and so it was not surprising that Alex Buell would wonder aloud why Phoronix were seeing “20 times” performance loss in 2.6.35 release candidate kernels. What was sad, as Robert Hancock noted, was that they wrote an article without choosing to draw the developer’s attention to the problem (although they did note rather specifically when the problem – believed to be causing udev to get into a 100% CPU utilitization loop – was introduced at the end of May). Mike Galbraith and Ingo Molnar replied and infered that such stories were increasingly par for the course as Linux becomes more mainstream. Mike in particular said that, “If eggshells land in our omelet, they can make a buck telling people about it. Who cares? If tasty bacon bits land, they’ll make a buck on that event. Either way, we get some test coverage.” Others (including Ted T’so) were less concerned about “news” stories, and were instead more concerned that there had been a performance problem that had made it to the release candidate stage without being dealt with much sooner.

Thread interrupts. The way of the future for Linux interrupts has been the work that Thomas Gleixner has been organizing to switch over to threaded handlers (specifically, a small quiescent handler intended to quickly placate a device, and a later thread to do the “interrupt” work in thread context). Dmitry Torokhov was very interested in migrating the various input drivers over, but some of those need to do some very long lived polling of devices post interrupt. Dmitry wondered if it was ok to have a threaded handler do that work and just run for a very long time, to which Thomas said, “Sure, why not?” and noted that the only real thing to watch out for was appropriately lowering the priority of the interrupt thread such that other work will take scheduler priority in the interim. Of course the thread should probably not blindly adjust its priority without restoring it to whatever value it started it, just in case the user has specifically configured the intented priority (in the case of RT users).

UML. User Mode Linux (not the silly “modelling” language) has had a few on-and-off issues over the past year. Typically it will break for a few days and be fixed up, but there is some sense that its one-time popularity is truly on the wane. That’s sad, because UML can be useful in an educational context. Most recently, on x86, the UML breakage was caused by a callee register saving optimization trick in the arch-specific “hweight” (hamming weight) function. Borislav Petkov had posted a “bandage” (as Peter Anvin would later describe it), before Jeff Dike, and also others weighed in with various opinions. Peter’s main concern was that it was not at all obvious why UML broke.

In today’s miscellaneous items:

*). Andi Kleen posted a dm-crypt patch intended to facilitate it scaling to multiple CPUs (by means of using per-cpu workqueues), rather than the existing case of all work being bound on a single workqueue. In a separate thread, Neil Brown provided some useful insight into device mapper behavior for layered mappings, explaining why touching an underlying mapping is not a good idea when another is layered on top of it, including side effects.

*). Xudong Hao reported a possibly bridging related regression in 2.6.34 that caused the host kernel to panic when starting KVM guests. Avi Kivity kindly forwarded the mail to the netdev and briding mailing lists for followup.

*). Peter Ovtchenkov inquired as to common bit similaries in seemingly randomly generated UUID pseudofile entries within the kernel. It turns out that version 4 UUIDs specify that the “7th high half-byte” contain 4. Both Lukasz Gromanowski and Ian Campbell followed up to explain this..

*). Michal Marek posted a pull request for some kbuild changes in 2.6.35 (to include the new nconfig interface), to which Linus responded, giving some advice about the best way to do a pull request, and calling out Dave Miller’s networking pull requests as a good example of how to do it. Linus had similarly posted some good recommendations for diffstat generation on May 27th in response to some btrfs update requests.

*). Daisuke Nishimura posted some patches correcting a bad interaction between Andrea Arcangeli’s transparent hugepage patches and memory cgroups (the tail pages during a split did not have correct cgroup LRU entries).

*). Jeff Garzik posted some libata updates. Amongst them was a default disabling of Asynchronous Notification (AN), since, “Proper use is vague, and behavior of firmwares in the field do not match each other.”

*). Rusty Russell suggested to me that support for waiting module removal in modprobe be officially removed, to which I don’t have huge objections. At this point, very few people actually remove modules from production boxes. Rusty and Linus had some other conversation surrouding the in-kernel module loader and locking use therein in a thread overwise devoted to a problem with bne2 network driver initialization that went long (the thread).

*). Luis R. Rodriguez and Stephen Hemminger continued discussing an Ethernet drivers wiki page that could rival the content on compat-wireless today. Stephen wondered why Luis didn’t want this documentation in-tree, and Luis responded that he thought this stuff was better done on a wiki.

*). Dmitry Torokhov noted that “make install” over sshfs is now painfully slow since various git poking is done at the install stage. He wondered if the work implemented by Greg Thelen could be done at build time instead of at install time because it is slowing his build cycles “down to a crawl”.

In today’s announcements:

*). Jeff Merkey posted an announcement (free from any 4-letter-words this time) saying that he had written something called “Cworthy” that was intended to look like an old Netware interface. It’s on Googlecode.com. Jeff also had a few remarks on 2.6.34 that he decided to share under “general comments”.

*). Vladislav Bolkhovitin posted to let everyone know that the ISCSI-SCST project has now implemented a “full set of SCSI Persistent Reservations”. Further information is available at http://scst.sourceforge.net/.

The latest kernel release is 2.6.35-rc2.

Andrew Morton posted an mm-of-the-moment (mmotm) for 2010-06-03-16-36.

Greg Kroah-Hartman announced the release of “long term” stable series kernel 2.6.32.15, which reverted two patches that had unintentionally been included within the previous 2.6.32.14 release. He noted that those not experiencing problems had no real need to upgrade to the latest version at this time.

That’s a summary of today’s Linux Kernel Mailing List traffic, for further information visit www.kernel.org. I’m Jon Masters.