Launchpad’s strategist, Jonathan Lange, has started a series of blog posts on getting started with Launchpad’s Python library, launchpadlib:

launchpadlib is the Python client-side library that talks to Launchpad’s own REST API. It turns out that customize scripted control of a bug-tracker-code-hosting-translation-distribution-building-cross-project-collaboration thing is actually quite handy.

Catch-up with the first two posts in the series:

• Get started with launchpadlib
• launchpadlib powerup.

And you can subscribe to Jonathan’s blog or follow blogs from both Jonathan and other members of the Launchpad community on Planet Launchpad.

This is awkward.   We need to talk.

March 3rd was a strange day. It was one day before User Interface freeze for the upcoming Ubuntu Lucid long term support release.   By the end of the day we were supposed to have the entire look and feel of the desktop settled on so people could start writing documentation and books.

This was the same day that Canonical finally released the new theme that had been under secretive development.   It was bold, daring, light-inspired, and perhaps most popularly, not brown.   Jono Bacon, Canonical’s community manager, broke the news.   Mark Shuttleworth followed up on his own blog, thanking three members of the design team for leading the effort.

The most important change, however, wasn’t actually talked about.   The designers don’t blog themselves, and Mark and Jono didn’t mention it directly.   It had to be found in the screenshots, or experienced firsthand by alpha testers.

This is not ok

Soon, the community learned the change was intentional: a bug about the misplaced window controls was quickly marked invalid,   and when the controls briefly reappeared on the right again the change was reverted.   What’s disturbing is that Planet Ubuntu has been rather silent on the topic.   No one’s posted a real defense of this change yet, or for that matter even claimed responsibility. It’s like there’s this collective unease about criticizing something that feels like it came directly from on high.   So, instead, people are just silent.   I’d certainly be if I worked for Canonical.   Perhaps I should be, as I still hope to work for them.

If you read between the lines, you can tell that people aren’t too happy about it.   The most flattering thing a developer’s said about the left-sided Window controls is that they “got used to them after a few days”.   We’re quick to praise the theme (it’s gorgeous), but talking about this major sudden change to the window controls feels like taboo.   That’s incredibly unhealthy for a community project.   It’s like there’s this collective unease and everyone’s worrying if we’re about to release something embarrassing.

This experiment was a failure and we need to realize it

The alpha releases are great places for usability experiments.   Sometimes, they don’t work out.   Put a new user on today’s Ubuntu Lucid and they’ll think it’s fantastic, sleek, and absolutely gorgeous right up to the point where they have to close a window.   That’s where our first impression becomes something awful.

Note: The new card backs pictured above are my doing and are now default (Mads Rosendahl drew them).

A brief summary of the complaints about  the left side window controls

Some of these I noticed myself, a few are gathered from various comment threads on forums and blogs over the past week.

• Because the window title isn’t centered, the window controls being placed directly in front of it put it in a weird indented position
• The “slightly off left” location is inconsistent with Nautilus, Firefox, Thunderbird, Pidgin, Empathy, and every other tabbed program we have, which have close buttons for their tabs on the right.
• The left position is inconsistent with Windows, previous versions of Ubuntu, and even OSX – users have to relearn decades of muscle memory.
• Users who interact with both Windows and Ubuntu machines (or migrate from Windows) will have a much harder time than they did before.
• The buttons are too close to the file and edit menus, making catastrophic misclicks much more likely.   Closing something on accident should be as rare as possible.
• Even without misclicking, a user will have to take more time to use the window control and avoid a misclick.   This is an example of Fitt’s Law.
• The close position is also inconsistent with the power button in upper right.   Currently, “close it down” is something you can always do from the upper right anywhere in the system: within a tab, within a window, and even for the whole computer.   The new window controls break that entirely.
• The new position leaves a lot of empty, wasted space in the upper right of most windows.   While strictly speaking the amount of unused space is the same, it looks much worse when it’s all clustered together.   When the controls are on the right, the extra space can function as a buffer for the potentially destructive window controls.
• Similarly, the upper left of most windows now becomes much more crowded, creating a rather unpleasing contrast to the relatively empty upper right.
• In previous Ubuntus you could close windows on the left if you really wanted, by expanding the small circle menu that’s now gone entirely.   File->Quit is also an option, which is now very close to the close box.
• Gnome upstream has them on the right, causing consistency and developmental problems when we deviate.   This is particularly jarring with the adoption of future projects like Gnome shell and Gnome 3, which will change again how we interact with window controls.
• The current implementation breaks themes not designed for the new button order (which is currently every theme we ship, so even changing the theme back doesn’t help)
• A day before User Interface freeze of a long term support release is the worst possible time to suddenly spring this on everyone without explanation.
• It is very difficult to change them back as we don’t have any UI tool for doing this (the current method is manually editing gconf keys)
• The new position doesn’t actually do anything beneficial.

That last point is the most important.   Other than “looking different” the change doesn’t do anything helpful.   It’s a huge usability loss for an awful lot of people.   Some people get used to it quickly.   Others don’t, and like me end up getting physically angry when trying to use their computer.   I can’t remember ever having my computer make me feel this way for a long time, and I’ve been running Ubuntu alphas for five years now.

Let’s admit we have a problem

Ivanka deserves credit for being the first member of the design team to  at least talk about the controls:

Are we smoking crack to think that the learning curve or getting used to a new position is ever going to be worth any real or perceived benefit of new positions?

I’ll leave that question to the reader.

I saw the Translations of package descriptions video from FOSDEM 2010. Every distribution has two translatable string for each package: a synopsis (summary, short description) and a long description. These descriptions differ from distribution to distribution. The descriptions should be shared between the distributions. This will enable us to share the translations of the descriptions.

My idea: Why not letting upstream provide the package description and the translation for it? They should have the knowledge to provide a good description and to update it if required. To encourage upstream to provide the description, we should create a freedesktop specification for it. Quick draft: The tarball should contain a file named package.info. The package.info file should contain three RFC-2822-like fields for each package: Package, Synopsis, and Description. Translation can be stored in package.info.<language> (for example package.info.de).

Example package.info:

Package: audacity
Synopsis: A fast, cross-platform audio editor
Description: Audacity is a multi-track audio editor for Linux/Unix,
 MacOS and Windows. It is designed for easy recording, playing
 and editing of digital audio. Audacity features digital effects and
 spectrum analysis tools. Editing is very fast and provides unlimited
 undo/redo.
 .
 Supported file formats include Ogg Vorbis, MP2, MP3, WAV, AIFF, and AU.

What do you think about this idea?

Edit: Alexandre Franke found an existing markup language designed for our use case: Description of a Project (DOAP). Package is name there, synopsis is shortdesc, and description is description. Here is my example in DOAP:

<Project xmlns="http://usefulinc.com/ns/doap#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:foaf="http://xmlns.com/foaf/0.1/">
  <name>Audacity</name>
  <shortdesc xml:lang="en">
    A fast, cross-platform audio editor
  </shortdesc>
  <description xml:lang="en">
    Audacity is a multi-track audio editor for Linux/Unix,
    MacOS and Windows. It is designed for easy recording, playing
    and editing of digital audio. Audacity features digital effects and
    spectrum analysis tools. Editing is very fast and provides unlimited
    undo/redo.
    Supported file formats include Ogg Vorbis, MP2, MP3, WAV, AIFF, and AU.
  </description>
</Project>

Scottie posted an entry earlier about the new Ubuntu branding. I’ve been meaning to make a very similar post, but I’ve had lots more important things to do the last two weeks.

For those who have missed it, Canonical announced the new branding and artwork on the day of the User  Interface  Freeze for the Lucid development cycle. The decisions around the new branding have been met with some  controversy, and as Scottie pointed out, some awkwardness around it. I’ll try to sum up my views about it in this post.

Purple / Aubergine

One of the major changes in the artwork is a change from the yellowy-brown colours to purple and orange. Most of the highlights are in purple, so is the terminal background. The aubergine (purple) is supposed to signify the corporate and the commercial aspects of Ubuntu, while the orange represents community.

I love the way the background is slightly transparent by default. The compositing capabilities in Ubuntu is powerful and should be showed off. The amount of transparency is also just subtle enough that it shouldn’t bother people who usually prefer a solid background. I think the colour is horrible though. I’d go as far as to say it’s offensive, I can write that off to personal taste though. The really bad thing about it is that it changes your to profile to the so-called “Ambiance” theme automatically without any warning. At least it was easy to delete and my older, much more aesthetically pleasing terminal was restored.

Button Positions

The button positions have been a bit more controversial than the actual colours. Personally, I didn’t like it at first but it only took a few hours to get used to it. It was the same when I used OSX for the first time, the positions feel weird at first but they grow on you fast. I don’t think the button positions will be a problem for users who use Ubuntu pretty much exclusively. Not all my machines are on Lucid yet, so it’s been a pain to move between machines that have the controls on the left and that have them on the right. I can imagine that having the controls on the left may be a major annoyance for someone who uses the one system at home and another at work.

Besides that, many applications are set up to have the close button at the top right, and it’s usually not configurable. Scott mentioned something similar with regards to tabs, here’s an example in an OpenOffice document:

Font / Logo

Besides the announcement on the day of UI Freeze, the new Ubuntu font is still not complete either. This means that some derivatives (Edubuntu, for one), doesn’t have a new logo yet since the “e” and “d” letters haven’t been finalized yet. We’re also waiting for a new logo from Canonical, it’s a bit painful that this couldn’t have been better communicated or planned in advance.

Scott wonders why few other people have talked about this, but it’s not the first time Canonical has made big changes on or just before a freeze. It’s probably not the last either. I don’t think people are comfortable providing feedback, especially at the risk of being labeled as a bad apple that just wants to complain about stuff and especially if it won’t change anything.

Changes

Having said what I have so far, I’m all in favour of making big, bold changes. If Canonical puts this much effort into the design of every release, and if they can find a way to involve the community as well (turn their cold purple hearts a bit more orangy), then I think Ubuntu will make major strides in  usability  and design that ultimately, everyone else would want to copy.

For the first time in several years, I had the opportunity to attend a software conference in the city where I lived at the time. I’ve benefited from many InfoQ articles in the past couple of years, and watched recordings of some excellent talks from previous QCon events, so I jumped at the opportunity to attend QCon London 2010. It is being held in the Queen Elizabeth II Conference Center, conveniently located a short walk away from Canonical’s London office.

Whenever I attend conferences, I can’t help taking note of which operating systems are in use, and this tells me something about the audience. I was surprised to notice that in addition to the expected Mac and Windows presence, there was a substantial Ubuntu contingent and some Fedora as well.

Today’s tracks included two of particular interest to me at the moment: Dev and Ops: A single team and the unfortunately gendered Software Craftsmanship.

Jason Gorman: Beyond Masters and Apprentices

A Scalable, Peer-led Model For Building Good Habits In Large & Diverse Development Teams

Jason explained the method he uses to coach software developers.
I got a bad seat on the left side of the auditorium, where it was hard to see the slides because they were blocked by the lectern, so I may have missed a few points.

He began by outlining some of the primary factors which make software more difficult to change over time:

• Readability: developers spend a lot of their time trying to understand code that they (or someone else) have written
• Complexity: as well as making code more difficult to understand, complexity increases the chance of errors. More complex code can fail in more ways.
• Duplication: when code is duplicated, it’s more difficult to change because we need to keep track of the copies and often change them all
• Dependencies and the “ripple effect”: highly interdependent code is more difficult to change, because a change in one place requires corresponding changes elsewhere
• Regression Test Assurance: I didn’t quite follow how this fit into the list, to be honest. Regression tests are supposed to make it easier to change the code, because errors can be caught more easily.

He then outlined the fundamental principles of his method:

• Focus on Learning over Teaching – a motivated learner will find their own way, so focus on enabling them to pull the lesson rather than pushing it to them (“there is a big difference between knowing how to do something and being able to do it”)
• Focus on Ability over Knowledge – learn by doing, and evaluate progress through practice as well (“how do you know when a juggler can juggle?”)

…and went on to outline the process from start to finish:

1. Orientation, where peers agree on good habits related to the subject being learned. The goal seemed to be to draw out knowledge from the group, allowing them to define their own school of thought with regard to how the work should be done. In other words, learn to do what they know, rather than trying to inject knowledge.
2. Practice programming, trying to exercise these habits and learn “the right way to do it”
3. Evaluation through peer review, where team members pair up and observe each other. Over the course of 40-60 hours, they watch each other program and check off where they are observed practicing the habits.
4. Assessment, where learners practice a time-boxed programming exercise, which is recorded. The focus is on methodical correctness, not speed of progress. Observers watch the recording (which only displays the code), and note instances where the habit was not practiced. The assessment is passed only if less than three errors are noticed.
5. Recognition, which comes through a certificate issued by the coach, but also through admission to a networking group on LinkedIn, promoting peer recognition

Jason noted that this method of assessing was good practice in itself, helping learners to practice pairing and observation in a rigorous way.

After the principal coach coaches a pilot group, the pilot group then goes on to coach others while they study the next stage of material.

To conclude, Jason gave us a live demo of the assessment technique, by launching Eclipse and writing a simple class using TDD live on the projector. The audience were provided with worksheets containing a list of the habits to observe, and instructed to note instances where he did not practice them.

Julian Simpson: Siloes are for farmers

Production deployments using all your team

After a brief introduction to the problems targeted by the devops approach, Julian offered some advice on how to do it right.

He began with the people issues, reminding us of Weinberg’s second law, which is “no matter what they tell you, it’s always a people problem”.

His people tips:

• In keeping with a recent trend, he criticized email as a severely flawed communication medium, best avoided.
• respect everyone
• have lunch with people on the other side of the wall
• discuss your problems with other groups (don’t just ask for a specific solution)
• invite everyone to stand-ups and retrospectives
• co-locate the sysadmins and developers (thomas allen)

Next, a few process suggestions:

• Avoid code ownership generally (or rather, promote joint/collective ownership)
• Pair developers with sysadmins
• It’s done when the code is in production (I would rephrase as: it’s not done until the code is in production)

and then tools:

• Teach your sysadmins to use version control
• Help your developers write performant code
• Help developers with managing their dev environment
• Run your deploy scripts via continuous integration (leading toward continuous deployment)
• Use Puppet or Chef (useful as a form of documentation as well as deployment tools, and on developer workstations as well as servers)
• Integrate monitoring and continuous integration (test monitoring in the development environment)
• Deliver code as OS packages (e.g. RPM, DEB)
• Separate binaries and configuration
• Harden systems immediately and enable logging for tuning security configuration (i.e. configure developer workstations with real security, making the development environment closer to production)
• Give developers access to production logs and data
• Re-create the developer environment often (to clear out accumulated cruft)

I agreed with a lot of what was said, objected to some, and lacked clarity on a few points. I think this kind of material is well suited to a multi-way BOF style discussion rather than a presentation format, and would have liked more opportunity for discussion.

Lars George and Fabrizio Schmidt: Social networks and the Richness of Data

Getting distributed webservices done with Nosql

Lars and Fabrizio described the general “social network problem”, and how they went about solving it. This problem space involves the processing, aggregation and dissemination of notifications for a very high volume of events, as commonly manifest in social networking websites such as Facebook and Twitter which connect people to each other to share updates. Apparently simple functionality, such as displaying the most recent updates from one’s “friends”, quickly become complex at scale.

As an example of the magnitude of the problem, he explained that they process 18 million events per day, and how in the course of storing and sharing these across the social graph, some operations peak as high as 150,000 per second. Such large and rapidly changing data sets represent a serious scaling challenge.

They originally built a monolithic, synchronous system called Phoenix, built on:

• LAMP frontends: Apache+PHP+APC (500 of them)
• Sharded MySQL multi-master databases (150 of them)
• memcache nodes with 1TB+ (60 of them)

They then added on asynchronous services alongside this, to handle things like Twitter and mobile devices, using Java (Tomcat) and RabbitMQ. The web frontend would send out AMQP messages, which would then be picked up by the asynchronous services, which would (where applicable) communicate back to Phoenix through an HTTP API call.

When the time came to re-architect their activity , they identified the following requirements:

• endless scalability
• storage- and cloud-independent
• fast
• flexible and extensible data model

This led them to an architecture based on:

• Nginx + Janitor
• Embedded Jetty + RESTeasy
• NoSQL storage backends (no fewer than three: Redis, Voldemort and Hazelcast)

They described this architecture in depth. The things which stood out for me were:

• They used different update strategies (push vs. pull) depending on the level of fan-out for the node (i.e. number of “friends”)
• They implemented a time-based activity filter which recorded a global timeline, from minutes out to days. Rather than traversing all of the user’s “friends” looking for events, they just scan the most recent events to see if their friends appear there.
• They created a distributed, scalable concurrent ID generator based on Hazelcast, which uses distributed locking to assign ranges to nodes, so that nodes can then quickly (locally) assign individual IDs
• It’s interesting how many of the off-the-shelf components had native scaling, replication, and sharding features. This sort of thing is effectively standard equipment now.

Their list of lessons learned:

• Start benchmarking and profiling your app early
• A fast and easy deployment keeps motivation high
• Configure Voldemort carefully (especially on large heap machines)
• Read the mailing lists of the NoSQL system you use
• No solution in docs? – read the sources
• At some point stop discussing and just do it

Andres Kitt: Building Skype

Learnings from almost five years as a Skype Architect

Andres began with an overview of Skype, which serves 800,000 registered users per employee (650 vs. 521 million). Their core team is based in Estonia. Their main functionality is peer-to-peer, but they do need substantial server infrastructure (PHP, C, C++, PostgreSQL) for things like peer-to-peer supporting glue, e-commerce and SIP integration. Skype uses PostgreSQL heavily in some interesting ways, in a complex multi-tiered architecture of databases and proxies.

His first lesson was that technical rules of thumb can lead us astray. It is always tempting to use patterns that have worked for us previously, in a different project, team or company, but they may not be right for another context. They can and should be used as a starting point for discussion, but not presumed to be the solution.

Second, he emphasized the importance of paying attention to functional architecture, not only technical architecture. As an example, he showed how the Skype web store, which sells only 4 products (skype in, skype out, voicemail, and subscription bundles of the previous three) became incredibly complex, because no one was responsible for this. Complex functional architecture leads to complex technical architecture, which is undesirable as he noted in his next point.

Keep it simple: minimize functionality, and minimize complexity. He gave an example of how their queuing system’s performance and scalability were greatly enhanced by removing functionality (the guarantee to deliver messages exactly once), which enabled the simplification of the system.

He also shared some organizational learnings, which I appreciated. Maybe my filters are playing tricks on me, but it seems as if more and more discussion of software engineering is focusing on organizing people. I interpret this as a sign of growing maturity in the industry, which (as Andres noted) has its roots in a somewhat asocial culture.

He noted that architecture needs to fit your organization. Design needs to be measured primarily by how well they solve business problems, rather than beauty or elegance.

He stressed the importance of communication, a term which I think is becoming so overused and diluted in organizations that it is not very useful. It’s used to refer to everything from roles and responsibilities, to personal relationships, to cultural norming, and more. In the case of Skype, what Andres learned was the importance of organizing and empowering people to facilitate alignment, information flow and understanding between different parts of the business. Skype evolved an architecture team which interfaces between (multiple) business units and (multiple) engineering teams, helping each to understand the other and taking responsibility for the overall system design.

Conclusion

Overall, I thought the day’s talks gave me new insight into how Internet applications are being developed and deployed in the real world today. They affirmed some of what I’ve been wondering about, and gave me some new things to think about as well. I’m looking forward to tomorrow.

On Monday our partner Groundwork Open Source, whom are selling a monitoring solutions that they recently certified for Ubuntu, published their statistics of the OS on which they see GWOS running on.

read more