Hi all,

after many many weeks working on forum.ubuntu-it.org, we finally done!

The forum for ubuntu-it.org has been migrated from SMF 1.X (MySQL) to PhpBB3 (PSQL). The entire work has taken more than one month, considering the huge data (almost 5GB database) to migrate.

The new theme is inspired to the ubuntu-it website and ubuntu design guidelines, hope you like it.

 

Thanks to #canonical-sysadmin to help me in tuning psql, apache, php, etc… PphBB3 (default) is not so fast with big forums as I expected.

Galapinux is an annual meeting organized by FP Empresa on IES Santa Elena (Galapagar-Madrid). The first meeting was on 2006 and the main attendees are students from 6 High professional schools of Madrid. Today’s agenda had the presence of  Pablo Machón (FSF Spain), Jorge Salamero (Zentyal SL) and SOLIMADRID (council of free software company of [...]

This is going to be just a short update, as I guess most of the developers are still suffering from their UDS-Q hangover.

There was a call for testing that went out on Tuesday for PulseAudio 2.0. So far there haven’t been any big problems.

The Software Center, crashing was a problem for quite a few users, fortunately mc4man found a solution, that solved the crashing problem.

We also discussed some of the upcoming changes being proposed for this development cycle.

As always, if you have a problem with Quantal, please drop by the sub-forum as we will be able to help you solve it.

Firstly, please excuse the ugly formatting of this ultra-wide image on my blog, and I’m sure the respective planets that I’m on.

poŝto is my first “real” node project, and I’m planning to get it to a point where it makes email less painful then it otherwise is. The code’s on my github, and I’d like to thank Rick Waldron for his work in helping me, as well as Scott Wells for the name.

For those wondering, poŝto is an Esperanto word meaning (according to wiktionary):

1. a public institution (usually government-run) to deliver mail, post

   

2. mail distribution in general

3. one’s mail, collectively

The front-end’s name is “poŝtilo” — the suffix denotes “tool” such that poŝtilo means something like “mail tool” — fitting!

More to come…

As we all know, Qt5 is approaching. The “big new thing” with Qt5 is the introduction of QtQuick/QML for writing dynamic user interfaces. If you are (somehow ) unfamiliar with QtQuick and QML, you can read up about it here. The rest of this post assumes that you know something about Qt Quick.

Qt5 is imminent, with a release set for sometime within the next few months. But as of currently, there seems to be several questions that remain in regards to QML. I have encountered several questions, some directed at me, such as “Why would a desktop application want to use QML?”, “How will traditional desktop applications be able to use QML?”, “How will QML applications integrate with KDE and its existing applications?”, and, more specifically, “How will Muon Discover integrate with KDE since it is using QML?”. In this post I will attempt to answer these questions, with my opinions and thoughts on the matters.

Why would a desktop application want to use QML?

Traditionally, user interfaces for applications have been written to be composed of discreet “forms” or “pages”,  with each form containing a (mostly) static collection of controls called “widgets”. To present new forms to the user, an application would either present the new form as a separate window/dialog, or replace the form in the main window with a new one entirely, using a widget stack to contain the forms.  These interfaces were coded either in straight C++ (or with language bindings), or by compiling XML files created with a WYSIWIG-esque interface designer. A common set of widgets is shared among applications of a particular toolkit. (Be it Qt or Gtk, etc) This, along with theming, allows for very tight graphical consistency between applications written in the same toolkit.

Now, there is nothing particularly “wrong” with the way things currently work, but that does not mean things couldn’t be better. Animations are a useful tool with regards to usability. Aside from looking slick, user interfaces that morph gradually are easier for the human mind to “track”, as opposed to instantaneous change. But until the advent of QML, providing animations (and fluid interfaces in general) with the traditional static widget-driven user interface, you had to roll your own. Qt did provide several APIs within the last few releases to provide animations/effects within QWidget-based interfaces, but even then doing animations and graphical effects was not exactly trivial to “get right”. And even when using these APIs, the effects’ drawing performance is tied to the painting backend in use with Qt, which can be slow (e.g. the old X11 Qt painting plugin), or at the very least can’t take advantage of the dedicated graphics hardware of the modern computer.

In short, QML makes it very easy to write fluid user interfaces, while retaining speed. The advantages expand well beyond the mobile application sphere, and represent a new way to do GUIs.(tm) [/marketing mode]

How would a desktop application use QML?

But what, specifically, would you use QML for within a desktop application? Aside from generally animating the interface, how would QML be used to provide advantages of the current QWidget paradigm?

For one, it makes it much easier to present non-control media such as pictures in an appealing fashion. Take for example, the slideshow for featured items in the home page of Muon Discover. It is written in a mere 126 lines of QML, and provides navigation between featured items, animated item switching, fading in new items, and it automatically pans around screenshots that are too big to fit in the banner. Doing the equivalent with QWidgets or even QGraphicsItem and friends would be a lot more involved, requiring lots of custom painting an animation code, with potentially worse performance.

Aside from the presentation of things that are strictly media (though to be fair, clicking on the aforementioned slideshow will take you to the app’s page), QML can also aide in the creation of user controls that themselves can incorporate media and animations.

Take delegates for list views as an example. If you want to go beyond the simple icon-and-text that the default Qt list view delegate offers, you must subclass your own delegate and re-implement the paint function. Re-implementing the paint function requires you to do a lot of pixel math with a lot of code that, frankly, I never mastered. Whenever I’ve needed a custom delegate, such as for the Muon Software Center and Muon Package Manager, I’ve resorted to searching lxr.kde.org for delegate subclasses, searching for a delegate that kind of does what I want, and tweaking that until I get something useful. At the end of the day, even creating a delegate that paints an icon, two lines of text, and a rating widget takes around 300 lines of code. You get a whole lot of control, at the expense of a whole lot of hassle.

With QML, a similar listview delegate takes only a bit more than 100 lines of QML, without forcing the UI designer to do tons of pixel-based positional calculations within the QML script. With it being so easy to make itemview delegates in QML, Aleix was even able to create a second “icon grid view” delegate that incorporates animation within the delegate to show different information when moused over, as seen in his video here.

On the issue of integration and consistency

One of the strengths of KDE is how well its applications fit together. Most every applications share exactly the same set of controls. (Buttons, list views, line edits, scroll bars, etc, etc) Similarly, these common controls all follow a common theme. The combination of these two provide a consistent experience common to all KDE applications, which makes it easier for somebody new to KDE to pick it up. If you learn how a combobox or list view works in one application, generally, you’ve learned how they work in all KDE/Qt applications.

This level of integration is not yet available in QML, which is why we’re holding off from replacing the Muon Software Center with Muon Discover until things change. As of now, Qt does not offer a set of common components like we see with the current set of QWidgets. There is the set of common QML Elements that provide a rudimentary set of common components, but it’s nothing as extensive as what we have currently. If you need a UI control in QML, currently you must roll your own. Doing this results in un-integrated controls reimplemented by each application that potentially do not share a common theme. Muon Discover is currently using Plasma’s QML components, which while providing graphical and widget-level integration with Plasma and making Muon Discover fit in on a Plasma Active system, does not provide integration with most of the other QWidget applications on “the desktop”.

The solution to this does appear to be coming, however, in the form of the Qt Desktop Components for QML. The QML Desktop Components provide a set of UI controls analagous to the ones available with QWidgets. Additionally, they use QWidget theming, allowing graphical integration as well as behavioral integration with QWidgets. You can see them in action in KAlgebra Mobile here. Through the use of desktop components, we can make applications that utilize all the benefits of QML while retaining integration with the rest of the desktop. The only caveat is that the desktop components haven’t actually been released yet. Previews can be found at their Gitorious site, and we even have a git branch for Muon Discover that (partially) uses the desktop components. I’ve not heard a concrete release date for the QML desktop components, but seems like they will be The True Way Forward ™ for QML on the desktop.

Conclusion

I feel very strongly about consistency, so to re-iterate Muon Discover will not displace the Muon Software Center until we have a way to integrate with the rest of KDE. QML does, however, provide a very powerful way to create attractive, easy-to-use interfaces without too much effort. While we wait for a solution for the desktop to become available, we can continue to hack on Muon Discover and allow it to mature. And hey, it already integrates well with Plasma via the use of the Plasma QML components, so perhaps in the nearish future (with a bit of work to make Muon Discover touch-suitable) you could find Muon Discover on a Kubuntu Active spin near you.

What would it take to get Ubuntu CDs available for sale from a national electronic retailer in the United States? Would that increase potential for access? Would that create revenue for Canonical? Would it matter in the end?

 

After two years, Catalan LoCo Team returned to IES Nicolau Copèrnic School in Terrassa, near Barcelona, for an Ubuntu release party, this time 12.04.  Again, the organization lied basically on the school teachers and we achieved a full and varied activity schedule. With 5 simultaneous tracks, perhaps it’s the party where we offered more variety for activity picking in every moment.

Install party, two simultaneously lectures, GPG and CAcert signature party, game room and an Ubuntu Server Jam.

As I wrote two weeks ago, I consider the QA related changes in Ubuntu 12.04 a great success. But while we will continue and even extend our efforts there, this is not where the ball stops: it’s great to have the feedback cycle between “break it” and “notice the bug” reduced from potentially a few months to one day in many cases, but wouldn’t it be cool to reduce that to a few minutes, and also put the machinery right at where stuff really happens — into the upstream trunks? If for every commit to PyGObject, GTK, NetworkManager, udisks, D-BUS, telepathy, gvfs, etc. we’d immediately build and test all reverse dependencies and the committer would be told about regressions?

I have had the desire to work on automated tests in Linux Plumbing and GNOME for quite a while now. Also, after 8 years of doing distribution work of packaging and processes (tech lead, release engineering/management, stable release updates, etc.) I wanted to shift my focus towards technology development. So I’ve been looking for a new role for some time now.

It seems that time is finally there: At the recent UDS, Mark announced that we will extend our QA efforts to upstream. I am very happy that in two weeks I can now move into a role to make this happen: Developing technology to make testing easier, work with our key upstreams to set up test suites and reporting, and I also can do some general development in areas that are near and dear to my heart, like udev/systemd, udisks, pygobject, etc. This work will be following the upstream conventions for infrastructure and development policies. In particular, it is not bound by Canonical copyright license agreements.

I have a bunch of random ideas what to work on, such as:

• Making it possible/easier to write tests with fake hardware. E. g. in the upower integration tests that I wrote a while ago there is some code to create a fake sysfs tree which should really go into udev itself, available from C and introspection and be greatly extended. Also, it’s currently not possible to simulate a uevent that way, that’s something I’d like to add. Right now you can only set up /sys, start your daemon, and check the state after the coldplugging phase.
• Interview some GNOME developers what kind of bugs/regressions/code they have most trouble with and what/how they would like to test. Then write a test suite with a few working and one non-working case (bugzilla should help with finding these), discuss the structure with the maintainer again, find some ways to make the tests radically simpler by adding/enhancing the API available from gudev/glib/gtk, etc. E. g. in the tests for apport-gtk I noticed that while it’s possible to do automatic testing of GUI applications it is still way harder than it should and needs to be. I guess that’s the main reason why there are hardly any GUI tests in GNOME?
• I’ve heard from several people that it would be nice to be able to generate some mock wifi/ethernet/modem adapters to be able to automatically test NetworkManager and the like. As network devices are handled specially in Linux, not in the usual /dev and sysfs manner, they are not easy to fake. It probably needs a kernel module similar to scsi_debug, which fakes enough of the properties and behaviour of particular nmetwork card devices to be useful for testing. One could certainly provide a pipe or a regular bridge device at the other end to actually talk to the application through the fake device. (NB this is just an idea, I haven’t looked into details at all yet).
• For some GUI tests it would be much easier if there was a very simple way of providing mocks/stubs for D-BUS services like udisks or NetworkManager than having to set up the actual daemons, coerce them into some corner-case behaviour, and needing root privileges for the test due to that. There seems to be some prior art in Ruby, but I’d really like to see this in D-BUS itself (perhaps a new library there?), and/or having this in GDBus where it would even be useful for Python or JavaScript tests through gobject-introspection.
• There are nice tools like the Clang static code analyzer out there. I’d like to play with those and see how we can use it without generating a lot of false positives.
• Robustify jhbuild to be able to keep up with building everything largely unattended. Right now you need to blow away and rebuild your tree way too often, due to brittle autotools or undeclared dependencies, and if we want to run this automatically in Jenkins it needs to be able to recover by itself. It should be able to keep up with the daily development, automatically starting build/test jobs for all reverse dependencies for a module that just has changed (and for basic libraries like GLib or GTK that’s going to be a lot), and perhaps send out mail notifications when a commit breaks something else. This also needs some discussion first, about how/where to do the notifications, etc.

Other ideas will emerge, and I hope lots of you have their own ideas what we can do. So please talk to me! We’ll also look for a second person to work in that area, so that we have some capacity and also the possibility to bounce ideas and code reviews between each other.

I just uploaded Apport 2.1 to Quantal. A big change in that version is that the whole code now works with both Python 2 and 3, except for the launchpadlib crash database backend (as we do not yet have a python3-launchpadlib package).

I took some care that apport report objects get along with both strings (unicode type in Python 2) and byte arrays (str type in Python 2) in values, so most package hooks should still work. However, now is the time to check whether they also work with Python 3, to make the impending transition to Python 3 easier.

However, you need to watch out if you use projects or scripts which directly use python-apport to process reports: The open(), write(), and write_mime() methods now require the passed file descriptors to be open in binary mode. You will get an exception otherwise.

A common pattern so far has been code like

  report = apport.Report()
  report.load(open('myfile.crash'))

This needs to be changed to

  report = apport.Report()
  with open('myfile.crash', 'rb') as f:
      report.load(f)

The “with” context is not strictly required, but it takes care of timely closing the files again. This avoids ResourceWarning spew when you run this in test suites or enable warnings.

I think the Mac Performance Guide hit the nail on the head:

As I wrote back in December, Apple shows a disregard for its professional users in the way it arbitrarily changes programs like Final Cut, and when it also provides no guidance as to whether a crucial product might continue. When there is credible speculation of the Mac Pro being discontinued, Apple’s silence speaks volumes. Professionals need to know they have a path forward, any vague guidance in the affirmative would address the issue.

When we jumped into this Novacut adventure (nearly 2 years ago now), we choose Ubuntu as our primary platform because we felt that Ubuntu would provide the best long-term, low-risk ecosystem for creative professionals. The strengths and trajectory of Ubuntu were clear, and my spidey-sense was telling me that we couldn’t count on Apple to continue to make an OS and hardware suitable for professional storytellers.

Back in July 2010, my take on Apple was definitely a subtle reading of the tea-leaves, and I easily could have been wrong. But as luck would have it, we played the right cards. Today I would feel silly if we had bet on Apple.

If you’re a creative professional who depends on Apple products to do your job, I think now is the time for some soul searching, the time to think about risk-reduction. Likewise if you’re in the business of selling software or hardware that depends on the Apple pro creative ecosystem. Of course, I’m just a some guy on the internet (plus I have obvious bias), so please weigh the evidence for yourself and act in your own best interest.

But for what it’s worth, at this point I’m personally 100% convinced that Apple is a dead-end road for creative professionals, and that Ubuntu is the creative platform of the future. Apple is a publicly traded company, and its board of directors are obligated to maximize Apple’s profits. So I recommend keeping a keen eye on where Apple is making money. As much as it pains me to see artists and small companies blindsided by, say, FCPX or the mysteriously disappeared Mac Pro, the harsh reality is this is just the result of Apple executives doing their job, and doing it well.

And Novacut isn’t the only pro video editor with an eye on Ubuntu. Last month at NAB, EditShare demoed Lightworks on Ubuntu. I think it’s quite telling that EditShare is now clearly prioritizing the Linux port of Lightworks over the OSX port. EditShare seems very in-tune with their customers, and I have no doubt this flip is because there is now more demand for Lightworks on Linux than on OSX.

Momentum

If I sound extra enthusiastic, it’s because last week we were at the Ubuntu Developer Summit in Oakland, California. Although it was our fourth UDS, there was an electricity in the air that really set this one apart. For example, Canonical announced they expect Ubuntu to ship on 5% of all PCs sold next year. This is deeply important for us, because if you can’t readily buy great Ubuntu hardware, there’s not much point in Novacut running on Ubuntu.

Speaking of great Ubuntu hardware, I had the pleasure of interviewing Carl Richell, the CEO of System76. We really hit it off with the System76 folks, and they just happen to have a passion for bringing artists great hardware on which to create. If you’re looking for a laptop on which to run Novacut, I highly recommend their new Gazelle Professional, which will be released this Monday (May 21st). How does this sound: quad-core Intel i7 (ivy bridge), 16GB of RAM, and a 95% NTSC gamut 1920×1080 matte display!

We were hired to film for the week, and our main responsibility was filming the Ubuntu Cloud Summit. Not only can Ubuntu hold its own against OSX on the desktop, Ubuntu is also an absolute champ in the cloud and on the server. Ubuntu isn’t just lower risk than OSX, it’s a superior technology on which we can build a far more practical and productive solution for storytellers.

Novacut is designed for distributed rendering and storage, so that you can spread both rendering and storage across any combination of your workstations, local clusters, and the cloud. In fact, the Novacut and Dmedia “servers” that will run in the cloud are the exact same “servers” that run locally on your desktop (which saves us a huge amount of work, by the way). We can do this because we’re building on the same core Ubuntu platform all the way from the desktop to the cloud.

The same Ubuntu that will be shipping on 5% of PCs next year also has the server muscle to power Instagram and Wikipedia and the cluster that rendered Avatar. Ubuntu even runs on ARM-powered devices, on everything from the humble Vodafone Webbook sold in South Africa to this 192-core Calxeda server. No other platform currently has this kind of reach. Novacut wants to help the storytellers of the world tell their stories on the world stage, so a big reach we need indeed.

Story Arc

While at UDS, I also had the pleasure of interviewing Mark Shuttleworth, the founder of Ubuntu. Did you know that before starting Ubuntu, Mark flew to space aboard a Russian Soyuz and spent a week on the International Space Station? I reckon something about the perspective of seeing the world from space played a part in Mark deciding Ubuntu was the most important thing to do next.

Did you know I was lucky enough to be in Porto Alegre, Brazil when both Ubuntu (at DebConf4) and the Creative Commons (at FISL) were first announced? And that before my trip to Brazil, I spent a year train-hopping and hitch-hiking around the US, daydreaming about ways in which technology could help empower storytellers?

I think we have all the ingredients here for a wonderful story, where artists and rebels and dreamers struggle together and, against all odds, slay every dragon that stood between them and a more beautiful world.

As many of you are probably aware Dell launched Project Sputnik which is a short 6-month pet project in which Dell will take a crack at making a laptop for developers specifically those who develop Free Open Source Software.

While at UDS-Q in Oakland, CA I witnessed the awarding of three Dell XPS 13 Ultrabooks to some amazing Juju Charm authors and I also got to hold and have a look at one of the devices and one thing that I found interesting was Dell went ahead and left the Windows logo on the bottom panel of the devices they were giving away to developers and mind you the device runs Ubuntu 12.04.

I realize it is a great thing that Dell is investing money and time into developing a device that runs Ubuntu and is geared towards developers but in my mind Dell could have very easily replaced the panels before handing them off to people who work with Free Open Source Software instead of leaving the brand on the devices.

Anyways kudos to Dell for yet again giving Ubuntu a try again and hopefully we will see Dell expand their offerings to include Ubuntu laptops in the mainstream marketplace but until then there are nice vendors like System76 and ZaReason who are soaking up money from Ubuntu Users and I’m personally looking forward to CTL’s launch of a Ultrabook competitor product running Ubuntu in the near future.

There is currently an empty virtual seat on the IRC Council and we are now accepting nominations to fill it. If you are an Ubuntu Member with a competent technical knowledge of IRC, a passion for IRC and a willingness to help maintain effective governance of our IRC community then you could be the leader we are looking for.

The Process
You can read about the Charter and Appointment Process for the IRC Council here https://wiki.ubuntu.com/IRC/IrcCouncil
It starts with an open call for nominations (you are reading it now), applicants will make themselves known to the IRCC via an email to irc-council@lists.ubuntu.com and will fill out a wiki page outlining their work in the community and why they would be a good candidate.

The current IRCC will administer this process and at the end of the nomination period will provide a full list of applicants to the Community Council along with any feedback and comments for the CC to review.

The Community Council will set up a vote for the Ubuntu IRC Members to vote on qualified candidates

The Community Council will then finalise the appointment, which will be for a term of about 18 months, expiring concurrently with the sitting IRCC.

Timescales
Nominations are open now, until the end of May (midnight UTC to be precise, but don’t wait until then). There will be a short gap while feedback is considered and the vote is set up, then the vote will be open for 2 weeks.

Voting Details
Voting will be done by the Ubuntu IRC Members group using the Condorcet Internet Voting Service at http://www.cs.cornell.edu/andru/civs.html and we will be using the Schulze method for completion. The voting group is made up of people who have been awarded Ubuntu Membership and consider themselves stakeholders in our IRC system, if you think you should be in that group and are not then you do have time to apply for membership if you have made a significant and sustained contribution to the Ubuntu Project. If you are already an Ubuntu Member but not in that group then please contact a member of the IRCC to be added.

If you have any questions about the process, or the IRCC and what we do, then please find us on IRC

Dear Lazyweb,

is there a simple way to block some users who login with SSH to read /proc/<pid>/cmdline of processes they don’t own? Or better yet: don’t see these pids at all?

I know that there are PID namespaces, but they seem to require a special PID 1. Seems hard to get for a simple SSH login. (I wouldn’t mind changing a user’s shell. But brittle shell startup scripts wouldn’t cut it.) systemd-nspawn wants to boot a full Linux distribution in a container and even then I’d be unsure how to wire it up so that it cannot be skipped. I wouldn’t mind a read-only bind mount of the outermost Linux installation into a chroot environment, as long as the parent SSH process can get the user jailed into it securely. (No need for someone to be root in the chroot.)

I know that there are RBAC frameworks, but they’re cumbersome to use. I don’t need file labelling or path-based access control, as I do trust the Linux file permissions for this. I think SMACK wouldn’t help here, AppArmor isn’t really useable in Debian testing and TOMOYO is a tad crazy to use with its domain transitions through process invocations.

I bet that grsecurity would have something for me up its sleeve. But it’s not in a Debian kernel. Even though I know how to compile my own kernel I’d only do that if everything else fails.

Thanks in advance for your help.

UPDATE: That was quick, thanks to everyone who participated! Vasiliy Kulikov came up with a kernel patch to my problem (a hidepid mount option for procfs) that landed in 3.3. I tested it with the kernel in experimental and it works just fine and as expected. With hidepid set to 1, it will still leak the process count and their euids and egids. With hidepid set to 2, you only see your own processes, unless you’re root. For ps there’s no visible distinction between the two. So to test it you can just invoke this as root on a host running 3.3+:

mount -o remount,hidepid=1 /proc

There’s even a backport request in the Debian BTS to get the feature into the wheezy kernel (3.2).

The awesome Jason Gerard DeRose interviewed Mark Shuttleworth at the Ubuntu Developer Summit (Quantal Quetzal edition) in Oakland California on Friday.

Our mascot Quincy (the most famous Quetzal) invites you to click the video below and watch Mark talk about software instrumentation, developer ultrabooks, high density ARM servers, and his favourite part of UDS-Q.

Thanks for putting this together, Jason. And extra points for not even flinching a bit when Quincy dropped in.

–

Reminder: UDS-Q has ended and now I am free to fly! Chirp!

Introduction

The NIH Utility Library (libnih) is a small, efficient and most importantly safe library of general purpose routines. It was written by Keybuk so you can be assured that it is extremely elegant, well-designed, well-tested (includes 2863 tests currently!) and well-written. NIH is used by Upstart, the event-based init daemon which is used by:

• Ubuntu Desktop
• Ubuntu Server
• Ubuntu Cloud
• RedHats RHEL 6
• Chromium OS (and Chrome OS)

That’s a lot of deployments of Upstart and NIH around the world!! (And we’re not even including mobile device operating systems in that list).

But why not just use glib I hear you ask? Well, glib is a very large library whereas NIH is small and designed for low-level daemons and systems which may be resource-constrained. Also, lets not forget that NIH, like Upstart, comes with a very comprehensive test suite so bugs are rare.

Other reasons to use NIH:

• It handles garbage collection for you

  That’s right, you don’t need to free memory manually.

• It uses an Object-Oriented-like Design… in C!

  This is extremely powerful and elegant. It’s also quite easy to use once you understand the way the API works.

Let’s start with some basics…

Garbage Collection

/* WARNING! Contains bugs! */
int
main (int argc, char *argv[])
{
    nih_local char *string;

    if (argc > 1) {
        string = nih_strdup (NULL, "hello, world");
        nih_message ("string is set to '%s'", string);
    }
}

This code nominally is trying to display a message if the user runs this application with one or more command-line arguments specified. However, there are a couple of problems with it:

• No check is performed on the memory allocated by nih_strdup().
• If no command-line argument is specified, chances are this program will crash.

The first issue is easy to spot and easy to remedy, but what about this crash? Well, nih_local variables are garbage collected automatically when they go out of scope. The string variable will therefore be garbage collected when it goes out of scope, which is when main() exits. However, since string was never initialized, it will be pointing to a random location in memory such that when the program exits, the runtime will attempt to free that random memory address. That will probably result in a SIGSEGV caused by dereferencing an illegal pointer value. The fix is easy and you should chant this mantra whenever you use nih_local variables:

Here’s a corrected version:

/* Correct version */
int
main (int argc, char *argv[])
{
    /* XXX: *ALWAYS* set nih_local variables to NULL */
    nih_local char *string = NULL;

    if ( argc > 1) {
        string = nih_strdup (NULL, "hello, world");
        if (string)
            nih_message ("string is set to '%s'", string);
        else {
            nih_error ("failed to allocate space for string");
            exit (EXIT_FAILURE);
        }
    }
}

However, there is an even better way to code that check to ensure nih_strdup() succeeded:

/* Improved version */
int
main (int argc, char *argv[])
{
    /* XXX: *ALWAYS* set nih_local variables to NULL */
    nih_local char *string = NULL;

    if ( argc > 1) {
        string = NIH_MUST (nih_strdup (NULL, "hello, world"));
        nih_message ("string is set to '%s'", string);
    }
}

So now, if the user specifies a command-line option, the program will print “hello, world” and automatically free the variable string. If the user does not specify a command-line option, no garbage collection will be performed since the string variable will never be associated with allocated memory.

Note that the code is simpler and easier to understand as a result. Note too that we’re now using NIH_MUST(). This is a macro which will call the block you pass to it (‘nih_strdup (NULL, “hello, world”)‘ in this case) repeatedly until it succeeds. You should exercise caution using NIH_MUST()though since if there is a high likelihood of the allocation never succeeding, the code will spin forever at this point. There is similar call “NIH_SHOULD()” that will call the block passed to it repeatedly until either the result is TRUE, or an error other than ENOMEM is raised.

Parent-Pointer

Let’s take a closer look at that call to nih_strdup. The system version of strdup takes a single argument (the string to copy), so why does nih_strdup take two arguments?

nih_strdup (NULL, "hello, world");

Well that first NULL parameter is the parent pointer. Most NIH functions take a parent pointer as their first argument. Lets see these pointers in action before explaining the detail…

#include <nih/macros.h>
#include <nih/logging.h>
#include <nih/string.h>
#include <nih/alloc.h>

int
main(int argc, char *argv[])
{
    typedef struct foo {
        char *str1;
        char *str2;
    } Foo;

    nih_local Foo *foo = NIH_MUST (nih_new (NULL, Foo));

    foo->str1 = NIH_MUST (nih_strdup (foo, "first string"));
    foo->str2 = NIH_MUST (nih_strdup (foo, "second string"));

    nih_message ("foo->str1='%s'", foo->str1);
    nih_message ("foo->str2='%s'", foo->str2); 

    exit(EXIT_SUCCESS);
}

Here we see our first complete NIH program. There are a couple of important points to note:

• The call to nih_new() is like malloc()except it too takes a parent pointer. Since the foo object we’re creating doesn’t have a parent, we set the pointer to NULL.
• Note that there is no call to free the memory allocated by nih_new()because since we’re using nih_local, the object and all its children will be freed automatically when the block (in this example the main() function) ends. This is incredibly powerful: we’ve made 3 memory allocations in the example (one call to nih_new() and two calls to nih_strdup()), and all that memory will be automatically garbage collected for us because NIH knows to free the foo object when it goes out of scope, but it also knows that the str1 and str2 elements also need to be freed (since we told nih_strdup() their parent is the foo object we previously created).

Lists

typedef struct nih_list {
    struct nih_list *prev, *next;  
} NihList;

Lists are designed to be contained within other objects like this:

typedef struct bar {
    NihList   entry;
    char     *str;
} Bar;

So you don’t create a “list of Bar objects”, you create a list of list objects which provide access to their containing types.

Note that the list element is the first in the Bar structure. This allows a list pointer to be dereferenced to its containing type trivially.

Let’s look at an example of list usage by implementing echo(1):

#include <nih/macros.h>
#include <nih/logging.h>
#include <nih/string.h>
#include <nih/alloc.h>

typedef struct bar {
    NihList  entry;
    char    *str;
} Bar;

int
main(int argc, char *argv[])
{
    int i;
    NihList *args;

    args = NIH_MUST (nih_list_new (NULL));

    /* store all arguments in a list */
    for (i = 1; i < argc; ++i) {
        Bar *bar = NIH_MUST (nih_new (args, Bar));

        nih_list_init (&bar->entry);

        bar->str = NIH_MUST (nih_strdup (bar, argv[i]));

        nih_list_add (args, &bar->entry);
    }

    i = 1;

    /* display all arguments by iterating over list */
    NIH_LIST_FOREACH (args, iter) {
        Bar *bar = (Bar *)iter;

        nih_message ("argument %d='%s'", i, bar->str);

        ++i;
    }

    nih_free (args);

    return (0);
}

The new features introduced here are the calls to nih_list_init() to initialise a list, and nih_list_add(), which adds the second argument to the list specified by the first argument. Additionally, we have that rather funky NIH_LIST_FOREACH() macro which allows for easy (and fast!) list traversal. In this example we are not using nih_local so what happens when nih_free() is called? Well, all entries in the args list are freed, but before each is freed, the str string within each entry is freed. Then the list itself is freed. Neat huh?

To build our version of echo:

gcc -std=gnu99 -Wall -pedantic echo.c -o echo $(pkg-config --cflags --libs libnih)

Now let’s run it:

$ ./echo a b c "hello world" "foo bar" wibble "the end"
argument 1='a'
argument 2='b'
argument 3='c'
argument 4='hello world'
argument 5='foo bar'
argument 6='wibble'
argument 7='the end'
$

We’ve really only scratched the surface of NIHs abilities in this post. Here are some of the other facilities it provides:

• hashes
• binary trees
• string arrays
• file watches
• I/O handling
• signal handling
• timers
• reference handling
• error/exception handling
• main loop handling
• command-line option and usage handling
• child process handling
• config file handling
• logging facilities
• test facilities

References

• The libnih project page on launchpad

I was reading the first book of The Losers, where in issue/chapter 2 Jensen is stealing some information from a hard drive in an insurance firm office:

and suddenly I notice…

bash, emacs, and bsd!

I have to admit a had a little nerdgasm.

By the way, I totally recommend this book. Consistently fun over the top action within an interesting caper story.

I’m home from UDS-Q in lovely Oakland, California! Okay, I actually got home about four days ago, but I’m slow with these things. As always, it was an interesting week with lots of interesting people. I’ll try to cram it into this post…

The Ubuntu Ecosystem (with a bonus secret session)

I attended a strange session on Monday, called The Ubuntu Ecosystem Part 1 (of 2). We don’t know who scheduled it, or why, or what we were meant to talk about, but it had an alluring title and the room was packed with people. Eventually, Jono stepped in to lead the session and it actually turned into a really nice chat about how Ubuntu is doing as a platform for developers outside the project itself. The session was removed from the schedule, but I think the session’s notes are interesting anyway. It was a good lead-in: app developers were a big topic for the whole week.

There was another session on Tuesday morning: App Developer external outreach. This one was quite interesting. Many different points of view emerged. When we talk about app developers, we tend to be preaching to the choir: we’re often grabbing people who already happen to be making, or interested in making, Linux desktop apps in general. I think Stuart Langridge said it best here: if there is a Linux desktop app that is incompatible with Ubuntu, it’s because they don’t like us. He also challenged the people in the room to name a single Linux app that doesn’t run on Ubuntu already. (I think Stuart is brilliant, by the way). I know there are a few, but what’s in the way is more packaging mumbo jumbo than the software itself, so there are really many branches to this. Getting things compatible, and getting things packaged and in Software Centre, are really two or three different discussions.

Another bit of strangeness is we really don’t seem to be sure what we mean by “app developer.” Do web apps count? Games? For this discussion, at least, “app developer” seemed to be anyone who makes software, but I wonder if we should be careful saying what kinds of “apps” we are interested in.

So, most of the session was brainstorming where we can go to talk to developers who haven’t thought about building for Ubuntu, and how to get them on our side. There is a lot of software (especially games) that is almost platform independent, and in some cases making that software run on Ubuntu is as simple as the developer choosing to support Ubuntu. There were some interesting ideas about writing articles for magazines and media sites, defining target audiences (demographics?), and making a solid business case for people who like those kinds of things.

At one of the app developer sessions, somebody mentioned it would be nice if there was an Ubuntu presence at places like game development conferences, and there wasn’t much interest. The big one is that Canonical doesn’t really have the resources for it. I wish there were. In terms of third party developers making and supporting stuff for Ubuntu, I think there is one place where we can say this is really going well at the moment: indie games. No, this doesn’t involve using Qt or GTK or lenses or any of our fancy notification APIs, or even packaging (most of the time). But there are thousands and thousands of people who have paid for, downloaded and played Linux versions of games from sources like the Humble Bundle, likely using Ubuntu, and many indie developers are seriously looking to build their games for Linux so they can reach that audience. That is a heck of an opportunity and I think it needs to be recognized, cherished and nurtured or it will go away as quickly as it came.

Making life easy for app developers

So, continuing along that trend, we also had a productive session called Upstream App Developer Documentation. This is something I have been really glad to see over at developer.ubuntu.com, and it was nice to learn we are on the same page here: great progress, but there is still work to be done!

One thing the site needs is content, so if you’re lovely and talented, think about submitting a tutorial for the Resources section. It would definitely be appreciated.

We talked about making it a little clearer that the site is dynamic, so people might be encouraged to approach it a little differently. We also discussed shiny ways to present API docs. The site has a good start there — particularly with its selection of a target Ubuntu version — but it is so far a little limited in scope and not as shiny as it could be. Alberto Ruiz is working on a neat Django documentation site called GDN, so ideally we can fit these things together.

Installer slideshows

I missed a chunk of Monday — including my own session — because I was trying to sleep off a headache. So, we did the ubiquity-slideshow session, Installer Slideshow checkup and planning, on Tuesday! There weren’t a lot of people there, but I think we had an interesting chat, and it continued nicely in the hallway session later on. It was nice to meet some people who work on installer slideshows for different Ubuntu flavours, and we arrived on a few things to work on for Ubuntu 12.10.

I’m going to make a real start allowing translators to localize screenshots. This is a tough problem because of the tight schedule close to release, so it won’t be finished for 12.10, but I want to have something. Actually creating translated screenshots is part of the puzzle, and that will be similar to what the documentation team does but with some little decorations. Chances are I will be writing about it in the future, and it will involve some magical scripts by Jeremy Bicha as well as some funny manual work to make things look pretty. That bit is kind of entertaining anyway.

The next puzzle is actually getting those localized screenshots to users. They can’t live on the install CD because there just is not enough space (and even if there was, we’d be adding a lot of files that don’t do anything). In the future they might live on a web service, but for 12.10 I’m looking at localized ISOs. There was a great session about that: Localized ISO community growth, Quantal plans. This is an amazing project where loco teams build their own Ubuntu install CDs (like the Italian CD), completely ready to go with a translation of their choice, with extra goodies like custom Firefox bookmarks and Rhythmbox radio stations. I will be playing with slideshow customization in the ISO builder so translated screenshots can be added to language-specific install CDs.

Updates, sound themes and stuff

On Monday morning, there was a session titled Sound Theme, which was mostly a quick update about Ubuntu’s new, upcoming sound theme. I think that project is in good hands. It wasn’t finished last cycle because it’s actually a pretty big piece of work: this isn’t just the startup sound. Having seen how much work is going into this, I really look forward to where it goes.

This cycle, I will be working a little bit with Michael Terry to implement the Software Updates specification. My goal is to get updates presented in a tree list where packages are collapsed beneath their respective applications. So, an update for Firefox will be presented with a single list item that says “Firefox Web Browser” along with a pretty icon, instead of a list of scary package names. This is going to be exciting! I’ll keep you posted, probably with the odd Twitter update.

As always, I have decided to actually use IRC. I do this every UDS. Hold me to it, okay? I actually set Empathy to connect to Freenode along with my other accounts this time, so it’s progress! (Yeah, I know, Empathy isn’t very good at IRC, but this way it will blend nicely with the background).

Oh, while I’m talking about random things, I’d just like to add a thanks for the t-shirts at this UDS! I love the design, and they’re super comfortable, too. So, whoever was in charge of that, you’re brilliant.

Fun!

 

And my favourite plenaries

Dave Walker’s MAAS demo gave me a powerful urge to buy eight computers and a lot of network cables just to play with that stuff, even though I have no use for it at all. It’s an impressive demo.

The lightning talks! I don’t know where or if a video is posted, but there was a fun lightning talk full of trivia about the infrastructure behind UDS. Metres of network cable, distance travelled by our heroic technicians, amount of data downloaded and uploaded, the wifi setup — that sort of thing. I hope it gets posted somewhere.

Ubuntu at Google with Thomas Bushnell. All about how Google is doing with Ubuntu deployed as a desktop operating system for thousands of employees. It’s always interesting to learn about Ubuntu desktops being used in big operations, especially when the resulting software can still be recognized as Ubuntu — which, in this case, it sounds like it is.

Electronic Arts with Richard Hillman, EA’s chief creative director. I think some people are disappointed that EA hasn’t gone and announced SimCity 5 for Ubuntu or something (okay, that would be pretty great), but I was perfectly happy to learn it’s a cautious, ongoing investigation instead of someone deciding to throw an expensive product at Ubuntu and hoping it sticks. This way, it could really work well in the long term — and there is room to sort out any problems along the way. After all, Ubuntu isn’t perfect yet. The Q&A after this talk is really informative, and it connects with the sessions we had about app developer outreach. It’s worth a watch!

Uh…

Thanks for the fantastic event, everyone! As usual I learned a lot, met some cool people, felt very humbled, and am writing way too much stuff in one blog post. As usual, I should have made daily posts like all the cool people do. Oh well. See you next year! Quantal Quetzal (that’s pronounced ketzal) will be glorious.

It’s always nice to know you can get help via all sorts of ways, especially via text browsers. This would have been amazingingly useful to me back when I only had one PC and I was still learning things, but it’s nice to know sites like this exist still for the real emergencies.

column80 is basically a text-browser (read Lynx) way to search and use Stack Exchanges. Which means, we have a nice text-friendly version of Ask Ubuntu:

Pair this up with pastebinit and maybe being stuck in a terminal on some godforsaken server in some abandoned building might not be so bad.

I suspect this might be useful for people who prefer to browse in a more 1980’s view as well. Enjoy!

With the release of MythTV 0.25 users now have the ability to opt-in and submit their hardware profile (see smolt). We’ve been reviewing that data and have decided to make our first change based on it. Today we are proud to announce that the default download link for the 12.04 ISO has been changed to 64-bit! Mythbuntu is the first of the official Ubuntu flavors to make the 64-bit ISO the default download. While this isn’t a monumental change we believe it is an important one due to the data submitted by our users.

The submitted data shows that half (the difference is less than 20 people currently) are running the 64-bit version versus the 32-bit version. What is a more interesting statistic is that two-thirds of the users running the 32-bit version have 64-bit capable hardware. Combining that information with the users that are running 64-bit means there is 5 times more 64-bit than 32-bit hardware running MythTV 0.25.

What does this mean for new users? Clicking the download link on the download page will start downloading the 64-bit version of the ISO. For users that still need the 32-bit version of the ISO they can download it by expanding the “Advanced Options” link on the download page.

At the recent Ubuntu Developer Summit, I managed to convince a few people (after assurances that there would be no permanent damage) to plug a USB stick into their machines so we could watch Xorg crash and wedge their console. What was this evil thing, you ask? It was an AVR microprocessor connected to USB, acting as a USB HID Keyboard, with the product name set to “%n”.

Recently a Chrome OS developer discovered that renaming his Bluetooth Keyboard to “%n” would crash Xorg. The flaw was in the logging stack, triggering glibc to abort the process due to format string protections. At first glance, it looks like this isn’t a big deal since one would have to have already done a Bluetooth pairing with the keyboard, but it would be a problem for any input device, not just Bluetooth. I wanted to see this in action for a “normal” (USB) keyboard.

I borrowed a “Maximus” USB AVR from a friend, and then ultimately bought a Minimus. It will let you put anything you want on the USB bus.

I added a rule for it to udev:

SUBSYSTEM=="usb", ACTION=="add", ATTR{idVendor}=="03eb", ATTR{idProduct}=="*", GROUP="plugdev"

installed the AVR tools:

sudo apt-get install dfu-programmer gcc-avr avr-libc

and pulled down the excellent LUFA USB tree:

git clone git://github.com/abcminiuser/lufa-lib.git

After applying a patch to the LUFA USB keyboard demo, I had my handy USB-AVR-as-Keyboard stick ready to crash Xorg:

-       .VendorID               = 0x03EB,
-       .ProductID              = 0x2042,
+       .VendorID               = 0x045e,
+       .ProductID              = 0x000b,
...
-       .UnicodeString          = L"LUFA Keyboard Demo"
+       .UnicodeString          = L"Keyboard (%n%n%n%n)"

In fact, it was so successfully that after I got the code right and programmed it, Xorg immediately crashed on my development machine.

make dfu

After a reboot, I switched it back to programming mode by pressing and holding the “H” button, press/releasing the “R” button, and releasing “H”.

The fix to Xorg is winding its way through upstream, and should land in your distros soon. In the meantime, you can disable your external USB ports, as Marc Deslauriers demonstrated for me:

echo "0" > /sys/bus/usb/devices/usb1/authorized
echo "0" > /sys/bus/usb/devices/usb1/authorized_default

Be careful of shared internal/external ports, and having two buses on one port, etc.

© 2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

We have uploaded a new Quantal linux kernel. The most notable changes are as follows:

* Updated overlayfs patch set
* Updated patch to prevent upgrading a non-PAE CPU

The full changelog can be seen at:

https://launchpad.net/ubuntu/+source/linux/3.4.0-2.6

UPDATE: Here’s the link to the actual interview on youtube – http://youtu.be/5WiZA-b_XNg (There’s also some useful reference links in the notes there)

Later tonight, around 8pm EST/5pmPST/0000UTC I’ll be interviewing Deepak Saxena, Tech Lead for the Kernel Team at Linaro and Vicky Janicki, Program Director for Member Services at Linaro about the training sessions which will be offered to attendees during the Q2.12 Linaro Connect event being held at the Gold Coast Hotel in Hong Kong from 28 May through 1 June, 2012.  This interview will be live and both streamed and recorded using Google + Hangouts on Air.

We’ll be discussing the following ‘Into Training’ sessions, which are currently scheduled to take place on Monday 28 June 2012. More about the schedule can be found on the Linaro Connect schedule page.

Upstreaming 101 -In this training session Deepak, will cover the basic “Whys” and “Hows” of upstreaming. In the “Why” section, will quickly go over the reasons that working with upstream is beneficial and in the “How” section will discuss both code design choices and also low level commands used to generate upstream ready patches.

Introduction to Git – Matt Waddel, FAE and Support Engineer at Linaro, will teach attendees about Git-the distributed version control system used by developers to manage the Linux kernel. Matt will walk attendees though the basics of using Git find it in this session.

Introduction to the ARM SoC tree – In this session, Arnd Bergmann, the ARM SoC tree maintainer, gives attendees and overview of the ARM SoC Tree.

Introduction to Device Tree – In this session, Thomas Abraham, of the Linaro Kernel Working Group, will provide an overview of creating Device Tree bindings for new and existing platforms and SoCs.

Introduction to PinMux/Pin Control – For this session, Linus Walleij, Kernel Engineer at Linaro, will discuss how to develop a driver and give an over view of the API.

So if you want to hear more about what attendees can look forward to during these sessions, which we will try to stream live via hangouts on air during the Connect event,  but which will be recorded and made available on the Connect Website after the event, then you can view the interview via my Google+ Page or my Youtube Channel starting tonight, Wednesday, May 16, 2012 at 8pm EST/5PM PST/0000 UTC.

As many of you know, Jono Bacon, along with other people, has been developing a system called Ubuntu Accomplishments, which is very cool, indeed. You earn trophies for doing different things in the community, or locally. The system even has its own daemon!

But, the problem is: We need accomplishments writers! Although writing one seems very complicated, it’s much easier than it seems. For me, it took less than 45 minutes, including a step-by-step class on how to use bzr to upload and propose for merging by Rafal Cieślak. Then, if you’re just writing it, it would be around 15-20 minutes. I started working on a “Gain an Ubuntu Member Cloak” accomplishment. Here’s what I did:

1. Write the script. When writing the script, if the accomplishment needs to be verified by checking if you’re a LP team member, you can just grab one of the scripts and modify it writing the correct team name (these are on the scripts folder, in the trunk branch). In my case, I used the motu.py script, and replaced motu bu ubuntu-irc-cloaks. Save your script with a name (name.py).
2. Write the accomplishment file. You should use one of the accomplishments that is already created, modifying it with the correct information again (these are in the accomplishments folder, in the trunk branch again). Just fill in the correct information, and you’ve done it. Remember that, as it’ll be our official accomplishments program, it should be a squeakly clean and great documentation. Save this file with the name name.accomplishment.
3. Write the test file. One more time, you can just grab this from the test files. In this you should specify two emails. The success email, is to give the tester a positive result, it means that it should be the email of an user that has already accomplished what needed, and so on with the fail email. You should save it with the name <name> (with no <>).
4. Once you’ve done that, you should upload it to a personal branch, and ask for review.

After all this process, my accomplishment got into the collection, check it out!

Remember that all your files should be named under the same name. As you can see, this is not so difficult. I hope many of you get into this fun and helpful process. You can check http://wiki.ubuntu.com/Accomplishments/Creating/Guidelines for the guidelines. If you need any further help, just go to #ubuntu-accomplishments in freenode. Thanks to all of you who have already contributed, and to all of those who will contribute to us!

 

Today at some time around 3am UTC, the one millionth (1,000,000th) bug was filed in Launchpad:

https://bugs.launchpad.net/edubuntu/+bug/1000000 (congrats Stéphane Graber!)

This is a huge milestone for everyone that uses and contributes to Launchpad and serves as a great witness to all the achievements, trials and challenges we’ve faced over the past 7 years. Today’s post is made up of contributions from some of the people who’ve worked with Launchpad and on developing Launchpad itself, right from the very start, up until fairly recently, like myself.

We’d love to hear your thoughts and experiences too, so please add a comment at the end if you have a story to share.

Francis Lacoste – Launchpad Manager
“Launchpad is vast. The significant milestones reached could be quite varied. But to me, the most important ones are the ones that enabled a community to use Launchpad for new activities. Thus, the first milestone was in the very earliest days, when the Ubuntu community switched from Bugzilla to Launchpad for tracking Ubuntu bugs!

“Other important milestones were when bzr and Launchpad code hosting were fast enough to host the huge Launchpad source tree itself (back in 2007). Then in 2008, when Launchpad started using Launchpad for code reviews! Other significant milestones were when MySQL joined Launchpad and bzr also in 2008. This opened the door for other big communities to join Launchpad: drizzle and then OpenStack. Finally, more recent milestones of this sort were when we introduced source package branches and Ubuntu started importing all of their packages in bzr: https://code.launchpad.net/ubuntu

“Last year, we introduced derived distributions which is now being used to synchronize with Debian development versions.”

Matthew Revell – Launchpad Product Manager
“There’s so much in Launchpad that it’s almost impossible to settle on a particular highlight. However, PPAs stick out as something of a game-changer. Someone once said that the cool thing about apt isn’t so much apt, but actually the software archive behind it. I love that I can trust the Ubuntu archive to give me what I need in a reliable form.

“However, PPAs have helped bring greater  diversity to Ubuntu by allowing anyone to build and publish their own packages in their own apt repository. With the addition of private PPAs and package branches, we have probably the best combination of centralised repos and software from elsewhere that I’ve seen in any operating system.”

Dave Walker (Daviey) – Engineering Manager, Ubuntu Server Infrastructure
“The real shining star is Launchpad bugs, the features and flexibility has really enabled the server team to deliver a quality product.  It’s rich API allows ease of mashups, and easy task prioritisation.”

Graham Binns – Software Engineer, Launchpad
“Probably the most significant moment for me over the time I’ve worked on Launchpad was its open-sourcing. Suddenly, this big beast that we’d worked on for years was open to outside contributions, and that was and still is incredibly exciting to me.”

Laura Czajkowski – Launchpad Support Specialist
“I think the best thing I’ve seen in a long time on Launchpad was the set downtime and reduced downtime that happens each day.  This minimises the effect for all projects hosted on launchpad an many people never even notice it down.”

J.C.Sackett – Software Engineer, Launchpad
“When I started on launchpad, the volume of bug data was a source of constant performance problems. Our 1 millionth bug is noteworthy in that we’re handling 1 million bugs better now than we were handling 500,000 then.”

Curtis Hovey – Launchpad Squad Lead
“Launchpad’s recipes rock. They allow projects to automatically publish packages created from the latest commits to their branch. Users can test the latest fixes and features hours after a developer commits the work.”

Diogo Matsubara – QA Engineer
“Personal package archives combined with source package recipes allows any Launchpad user to easily put their software into Ubuntu and this is a pretty unique feature from Launchpad.”

Tom Ellis – Premium Service Engineer
“It’s been great to see Launchpad grow and scale. A key milestone for me was seeing Launchpad move from a system that was not scaling well to one which has been a great example of continuous development and seeing the web UI improve in usability.”

 

 

(Photo by ‘bunchofpants’ on flickr, Creative Commons license)

Hey everyone, I am riding 100 miles (161 km) for Chicago’s 2012 Tour de Cure. My goal this year is to raise $1,500, because last year I was blown out of the water by the generosity of the people at my mom’s work, the KDE community, the cycling community, and a few friends. So, if 300 of you donated the minimum $5, I would make my goal Last year I had a blast doing this ride and completed it in 6.5 hours. My other goal this year is to finish it in under 6 hours.

To donate, you can click on the Support image above, or go to my Tour de Cure page. Scroll down, and on the right hand side you will see “CLICK HERE TO SPONSOR ME”, click it. I appreciate any and all help!

Donate To My Tour de Cure 2012 Ride is a post from Richard A. Johnson‘s blog.

I’m finally home from UDS, it was good to run into so many friends and colleagues and work on what will become 12.10. Here’s a quick summary of my upcoming cycle.

We will definately have Charm Schools:

And here are all my blueprints:

• https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-juju-charm-best-practices
• https://blueprints.launchpad.net/ubuntu/+spec/community-q-irc-workshops
• https://blueprints.launchpad.net/ubuntu/+spec/community-q-juju-charm-growth
• https://blueprints.launchpad.net/ubuntu/+spec/community-q-juju-charm-workflow
• https://blueprints.launchpad.net/ubuntu/+spec/community-q-juju-conferences

The TLDR on juju is basically to streamline the review process, we’re going to go just how distro does it, with a sponsorship queue and review days instead of the ad-hoc way we do it now.

The “loose” review process served us fine for 12.04 but needs to grow up a bit; you’ll see typical governance structures added as well. Lots of Charm Schools and training events, both online and at conferences, and we’ll be experimenting more with things like Google Hangouts for tutorials, etc.

I had a CR-48 Chromebook for a while, which has recently fallen in disuse. While I have never being totally convinced about Chrome OS being a polished, well designed, interface that simplifies the “always connected” user journey that Google was envisioning, I liked the concept.

Now I am reading in ArsTechnica that Chrome OS is getting a brand new look, that is … basically.. well, not new. While I am sure there are many technical advantages of a fully hardware accelerated windows managers, my issue is with the [lack of] concept.

Google has spent much energy convincing users that they do not need to have local apps, that they can do everything in the cloud and that the portal to this experience is Chrome. Having an OS which the only application that could possibly run, and at full screen, was the browser was a controversial but bold move. More over, it really hit home the user experience they were targeting.

This new UI seems to be sending the opposite message. It seems to be saying: “OK, we were wrong.. but  maybe if we make Chrome OS look more like windows you will like it better?”. Is that really the message? Well if you give me an app launcher in a desktop, I am bound to ask for local apps. If you give me off-line sync for Google apps, I am bound to ask for local apps.

I fear Google is paving the road to [windows vista] hell with good window manager intentions. I am primary an Ubuntu user, and what I like about it is that every single release over the last few years has continue to build on a design concept. Every new release is closely wrap on a consistent user message. Take as an example the HUD introduced in 12.04: it is new and different, but somehow it feels like it always belonged in Unity.

I am bought into the Ubuntu user experience, and I am excited to see what a new release will bring. If I had bought into the Chrome OS experience, I think I will be asking for a refund.

Anyway, I am looking forward to the new Chrome OS UI being available for the CR-48. Maybe I will change my mind once I get my hands on it.

A couple of photos from the Ubuntu release fest in Tampere yesterday.

People gathering up before presentations

Tieto‘s Markus Mannio

Again, continuing on how Ubuntu is used at Tieto

A cut to the end of presentations, Trine 2 game licenses from Frozenbyte being raffled. A great game available on Linux.

Tablets running KDE Plasma, and Ubuntu for Android being demoed.

Someone else probably has photos of my generic Ubuntu 12.04 LTS presentation (what’s new, what’s next), and likewise for the other presentations (Ubuntu for Android, uTouch) held. Those will be available as slides and videos later on, although do note the whole event was in the crypto-language called Finnish.

Thanks to the organizers, sponsors and everyone I met, it was a great event with nice little dinner and wine served at the end!

Summary: If you have performance problems using the JACK Audio-Connection-Kit and the fglrx ATI grpahics card driver, switching to radeon may solve them. Unity 3D and radeon can work, but leftovers of other drivers might get in the way. Also: Proprietary, binary blobs smell bad and Ubuntu’s infrastructure around those drivers is dodgey.

On Ubuntu 11.10, I switched graphics cards and thus drivers from nvidia to fglrx without much of a problem.

I recently upgraded to Ubuntu 12.04 and was quite pleased by how smooth that went and glad for not having to reconfigure and reinstall a bunch of stuff. As with every release so far, some issues might have disappeared, but a very noticable new one arrived: focus-follows mouse combined with auto-raise does no longer work reliably. So far I failed to identify the pattern for the cases where windows are not raised, when they should be.

After a while, I wanted to get back to music production with JACK and Ardour. My system was still configured for JACK to run in realtime mode, but I got many disconnects, often right when Ardour brought up its main window. I found out this only happened with Unity 3D, not with 2D. So it seemed like either one or the combination of Unity 3D and the fglrx driver interfered with realtime mode. A fellow #lad inhabitant knowledgeable about this realtime kernel business suspects that the 3D accleration part of the fglrx driver is not preemptable.

Where does one even report bugs about that proprietary blob? And how would one diagnose what exactly goes wrong?

Now I could use Unity 2D, but I really miss window drop-shadows, dislike the look and different notification animations for the Launcher icons and hate the fact that the Dash doesn’t react to the same shortcut I configured while using the 3D version.

Initially, I thought I would need the fglrx driver for Unity 3D, but still wanted to try switching to radeon. The Additional Drivers dialog claimed that neiter of the 2 ATI options were active, but lsmod told me otherwise. I have some Wacom-related stuff in my xorg.conf, which had to be moved out of the way, to get that thing to work. After a reboot, radeon was in use, but Unity decided to drop back to 2D. The cause: Xlib: extension "GLX" missing on display ":0.0". The solution was purging any trace of fglrx and nividia(!) from my system. Also, for good measure, but I suspect it’s unnecessary: sudo apt-get install —reinstall libgl1-mesa-glx libgl1-mesa-dri xserver-xorg-core; sudo dpkg-reconfigure xserver-xorg.

Now I have a working Unity 3D, using radeon, no disconnects or xruns galore using JACK and Ardour. Only new problem so far: shaky mouse pointer on the login screen.

Filed under: Planet Ubuntu, Ubuntu After some delay, I am continuing the posting on the third day of the Ubuntu Develop Summit for Quantal Qeutzal in Oakland, California.

Getting Home

In order to maximize my weekend before returning to work, my flight leaves at 06:00 on Saturday.  Apparently I naively expected to take the BART but I have learned that the first BART is at 06:30.

Turns out the answer is a local shuttle service (thanks Elizabeth and Charles) who will pick me up at 03:30.  I might not even go to sleep Friday night :/

QA

I learned about some remarkable things today related to available QA tools which hopefully will reduce the work load on our small team.

During a ubuntu-qa-tools session, I learned about automated ways to download and start a new image in KVM from a single command.  Rock on!

Turns out there are many other tools that will lower the threshold for new testers to easily join and help with testing..  I will certainly be  exploring these tools more. Also, during this session Gema mentioned her Plenary presentation for QA.  I look forward to learning more

Improving the testing tools used by Ubuntu Studio is another important aspect for our future.  By automating the basic ISO test we should be able to devote more time to deeper testing.

Learning of the available QA tools, along with the available backports tools, should really have significant impact to Ubuntu Studio starting with this cycle

Pictures

Last UDS I only took two pictures, this year I intend to do much, much better.  Hopefully tomorrow I start taking them.

There are a huge number of extremely cool and incredible people at UDS and I really hope to document some of this experience with a Picasa photo album.

finis

As happened last year, midweek seemed to slip out of high gear as I didn’t find as many interesting sessions. But I am sure I am an outlier at UDS.

Oh, I know of one interesting session coming up on Friday; it is the a session that I will be leading for the ‘Desktop Juju (see JuJu Studio section)‘ blueprint that was approved and scheduled.

Recently I have been talking a little about building quality and precision into Ubuntu Accomplishments. Tonight I put one of the final missing pieces in place and I thought I would share in a little more detail about some of this work. Some of you might find this useful in your own projects.

Before I get started though, I just wanted to encourage you to start playing our software and for those of you that had a crash when using certain languages with the Accomplishments Information viewer, I released a 0.1.2 update earlier that fixes this.

Automated Testing

As we continue to grow the Ubuntu Community Accomplishments collection it is going to be more and more complex to ensure all of the accomplishments are working effectively every day; we are already at 28 accomplishments and growing! What’s more, the community accomplishments scripts work by checking third-party services for data (e.g. Launchpad) to assess if you have accomplished something. These external services may change their APIs, adjust how they work, add/reduce services etc, so we need to know right away when one of our accomplishments no longer works and needs updating.

To do this I wrote a tool called battery. It works by reading in a test that is available for each accomplishment that feeds the accomplishment validation data that should succeed and also data that should not validate. As an example, for the Ubuntu Member accomplishment the data that succeeds is an existing member’s email address (such as my own) and the test for failure is an email address on Launchpad that is not a member. The original script requires the user’s email address to assess this accomplishment, so battery tests simply require the same types of information, with data that can trigger success and failure.

This approach allows us to test for three outcomes:

• That the valid email address returns exit code 0 (the script ran successfully and the user is verified as being an Ubuntu Member).
• That the invalid email address returns exit code 1 (the script ran successfully but the user is not an Ubuntu Member).
• If the script has an internal issues and returns exit code 2.

The way this works is that battery includes a customized version of the general accomplishments.daemon module that we use for the backend service. In the code I override the back-end module and load a custom module. This way the original accomplishment script does not need to be modified; instead of get_extra_information() calling the back-end daemon and gathering the user’s details, the custom module that comes with battery instead has it’s own get_extra_information() that gets returns the test data so battery can run the tests.

Originally battery only output textual results, but this would require us manually running it. As such, last night I added HTML output support. I then enabled battery to run once a day and automatically update the HTML results. You can see the output here.

There are a few important features in this report other than a list of all the accomplishment test results:

• It shows the failures: this provides a simple way for us to dive into the accomplishments and fix issues where they occur.
• It shows which tests, if any, are missing. This gives us a TODO lists for tests that we need to write.

While this was useful, it still required that we would remember to visit the web page to see the results. This could result in days passing without us noticing a failure.

Tonight I fixed this by adding email output support to battery. With it I can pass an email address as a command-line switch and battery will generate an email report of the test run. I also added battery‘s default behavior to only generate an email when there are failures or tests are missing. This prevents it generating results that don’t need action.

With this feature I have set battery to send a daily “Weather Report” to the Ubuntu Accomplishments mailing list; this means that whenever we see a weather report, something needs fixing.

One final, rather nice feature, that I also added was the ability to run battery on one specific accomplishment. This is useful for when we are reviewing contributions of new accomplishments; we ask every contributor to add one of these simple tests, and using battery we can test that the script works for validation success, validation failure, and script failure with a single command. This makes reviewing contributions much easier and faster and improves our test coverage.

Graphing

Something Mark Shuttleworth discussed at UDS was the idea of us building instrumentation into projects to help us identify ways in which we can make better decisions around how we build software. This is something I have also been thinking of for a while, and to kick the tyres on this I wanted to first track popularity and usage of Ubuntu Accomplishments before exploring other ways of learning how people contribute to communities to help us build a better community.

Just before we released version 0.1 of Ubuntu Accomplishments, I created a little script that does a scan of the validation server to generate some statistics about the number of daily new users, the daily number of new trophies issued, and the totals. Importantly, I only count users and trophies, and I am only interest in publishing anonymized data, not exposing someone’s own activity.

To do this my script scans the data and generates a CSV file with the information I am interested in. I then used the rather awesome Google Charts API to take my CSV and generate the Javascript need to display the graph. Here are some examples:

While this is not exactly instrumentation, it got me thinking about the kind of data that could be interesting to explore. As an example, we could arguably explore which types of contributions in our community are of most interest in our users, how effective our documentation and resources are, which processes are working better than others, and also some client side instrumentation that explores how people use Ubuntu Accomplishments and how they find it rewarding and empowering.

Importantly, none of this instrumentation will happen without anyone’s consent; privacy always has to be key, but I think the idea of exploring patterns and interesting views of data could be a fantastic means of building better software and communities.

Ubuntu Hour is a chance to meet up for an hour and chat with other Ubuntu users. The meeting is open to anyone interested whether they use Ubuntu or not, and everyone’s welcome with no commitments or RSVPs. It’s definitely a good opportunity to bring along friends who are curious about Ubuntu.

Not only is it fun to meet local Ubuntu fans, but we can also be a valuable introduction to Ubuntu for others. Wear that cool Ubuntu or Linux shirt or bring your laptop with the Ubuntu stickers, if you have them. We’ll also follow the Ubuntu Code of Conduct while we’re there. Easily summarized as “be excellent to each other,” it means that we’ll be good examples of the wonderful Ubuntu community.

The latest information, including locations and times, is always available at http://www.nhaines.com/ubuntu/hour/

Upcoming dates

• Thursday, May 17, 2012, 6pm – 7pm
• Thursday, May 31, 2012, 6pm – 7pm

Location

Panera Bread (Yelp) (Google Maps)
23592 Rockfield Blvd.
Lake Forest, CA 92618

Panera Bread is a casual restaurant that has fresh bread, soups, and sandwiches and free wi-fi access. We’re the group with a laptop or two and some Ubuntu logos, so please feel free to come up and say hi.

Congratulations

First off, congratulations to the Launchpad.net team for reaching bug #1000000. They’ve managed to build a huge platform that scales very well. Very few bug trackers live to that milestone and it’s amazing how they have managed to keep it snappy and also keep downtime so low by doing continuous roll-out.

1 000 000 x 67

A million bugs are a lot, but even more mind-blowing: for every bug filed in Launchpad.net, 67 iPads have been sold. Educational institutions everywhere are jumping on the iPad bandwagon, and in the Edubuntu project, we believe that the tools are quickly coming together that allows us to deliver a product that can be truly competitive with the iPad in educational environments.

We’re currently re-designing the Edubuntu website and will soon have a dedicated section to this project, but in the meantime, please join us on the edubuntu-devel mailing list and introduce yourself, or on the #edubuntu IRC channel on Freenode.

As Trever blogged yesterday, the Zeigeist team has been busy with tweaking the DB and the engine. During that process tools and benchmarks have been developed to make the tweaking and testing more interesting. Trever will be blogging about that tomorrow so make sure to check his blog.

Our end goal is  trying to scale the engine to be able to handle a few billion events just as fast as it can handle a few hundred thousand. While we are not there yet we managed to have some pretty nice stable results for the first iteration. A lot of results show more than 100% speed enhancement. In other words a lot of queries from our standard benchmarks now consume more than 50% less time to execute. Here are some graphs of our benchmarks.

Green indicates the 0.9 release

Yellow indicates the new trunk

Most notable performance enhancement is querying Zeitgeist with a specified timeframe (from data x to date y).

 

Same queries with an open timeframe also improved

 

We also have a copy of the Synapse queries benchmarked

The queries here are typical queries used to extract info from Zeitgeist. So right now the team is really happy with the initial results. For Synapse on my local DB (over a year old), all my synapse queries perform under 0.08 seconds. We still can get more out of this. The trick here was improving our indexes and our sql query generator.

Next month we will be going through another iteration.

 

 

We have uploaded a new Quantal linux kernel. The most notable changes are as follows:

* perarch and indep tools builds need separate build directories
* Prevent upgrading a non-PAE CPU

The full changelog can be seen at:

https://launchpad.net/ubuntu/+source/linux/3.4.0-2.5

As many people know I am a fanatic when it comes to web optimization and if my blog is taking more than a second or two to load I’m freaking out because I know how important load times are to end-users and that a few milliseconds could mean loss of a potential reader or new connection.

But more importantly load times also play a major role in how search engines rank you in results because in turn they consider slow loading websites to be of lesser quality to their users and rank accordingly in their algorithms.

I have been a big fan of the folks at NetDNA which offer the service MaxCDN and recently launched CloudCache two services aimed at producing top performance when it comes to serving content on your site. Some of the top sites on the internet rely on the technology that NetDNA offers to make their websites run blazing fast 365 days of the year.

I told my followers I would give something away when I reached 2,500+ people on Google+ and I recently passed this mark as such I am going to giveaway of  Five CloudCache Basic Plans for an entire year totally free courtesy of NetDNA.

 

a Rafflecopter giveaway

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20120515 Meeting Agenda