Feb 08 2009

FOSDEM09 and my Summer of Code at Debian slides

Category: Summer of Code, debianObey Arthur Liu @ 4:50 pm

I'm going to FOSDEM, the Free and Open Source Software Developers' European Meeting

Actually, I went to FOSDEM 2009.

It was really a blast. I would thank the whole FOSDEM team and the Debian team, especially the video team who did a fascinating work doing video streaming the right way (!= simple way).

Here are the slides for my talk (Debian and Google Summer of Code 2008: wrap-up and insights) in PDF format: debian-fosdem09-gsoc08. The videos of the talks of this FOSDEM should be up soon and I will post the extended version of what I said as a blog post; as soon as I’ve finished merging the changes from the slides back onto the blog post.

Cheers and see you at the next FOSDEM.

LH: cheers, really cool talk right now.


Feb 02 2009

Debian Summer of Code ‘08 : Where are they now (part 3/3)

Category: Summer of Code, debianObey Arthur Liu @ 1:03 am

UPDATE: The slides of the FOSDEM 2009 talk about this are here. They include analysis and recommendations for the next Summer of Code.

Welcome back for the last part of the reviews. You may want to look at the previous parts :  part 1 and part 2.

Jigdo-ivory, a JavaScript Jigdo client

Presentation

“Debian CDs and DVDs take up a huge mount of space on download servers. Using jigdo to download those images can significantly reduce the amount of bandwidth and space needed on the central servers. Unfortunately, jigdo currently needs special client software to be downloaded/installed first. Adding support directly into a browser-based application could potentially make a very big difference for first-time users here.”

Jigdo was created in 2001. It allowed to create ISOs from .debs grabbed from regular mirrors. It eliminated the need to duplicate the entire contents of the package repository into ISO files for each release, or even more importantly, for weekly snapshots of testing/unstable/whatever.

You may find the complete proposal from the student here. The original idea originated from the Debian-CD people, who wanted to explore ideas about creating a light web client. The project was mentored by Steve McIntyre, who developed a new version of the Jigdo tools, jigit, which is much more efficient.

Student

Dustin Rayner was a 5th year senior undergraduate student at the Oklahoma Christian University in Oklahoma City, Oklahoma. I studied Computer Engineering for 3 years as a Computer Engineering student before deciding to pursue a Mathematics and Computer Science degree.

Result

This project was unsuccessful due to numerous issues. First, because of an inadequate technical preparation of the original proposal. The Debian-CD people were too optimistic with the possibilities of Javascript. In the end, the copying and checksumming part of the Jigdo process were implemented but the checksumming (with a Javascript implementation of md5) was so slow that it was unusable (think 50kb/s on a regular laptop at full CPU charge). The student did the right thing to investigate Java and ActiveX but it was too late unfortunately and he ultimately lacked the experience and knowledge in the relevant technologies.

If the proposal is tried again, the student would be requested to have much more experience with Java (and possibly ActiveX). Those would be much more efficient for the task, as they are the most used technologies among on-line anti-virus scanners, which have a workload somewhat similar to Jigdo.

I could not find further public involvement of Dustin Rayner within Debian.

Aptitude-gtk, usability and GTK+ GUI for the Aptitude package manager

Presentation

“A GTK+ GUI for Aptitude that will work alongside improved current ncurses and command-line interfaces. This will offer an alternative to Synaptic with an interface design geared toward usability and advanced functionality.”

Debian currently supports multiple non-command-line package managers, the most used being Synaptic and Aptitude. Synaptic uses a GTK+ interface but offers no command-line mode. Aptitude offers a command-line mode but no X interface, although it offers a ncurses interface.
Comparing the interfaces of Synaptic and Aptitude reveal many design differences. Although Synaptic may be more accessible to beginners, Aptitude offers many interface behaviors and functions that are useful to the regular to advanced users : fully hyperlinked tabbed navigation between packages and versions of packages, mostly modeless interface, interactive dependency conflict resolver…

The proposal was introduced by the student in coordination with Daniel Burrows, the mentor and developer of Aptitude.

Student

Obey Arthur Liu was a 22 year old french student of Computer Science and Applied Mathematics at Grenoble Institute of Technology – ENSIMAG, in France. Did I mention that he’s also yours truly ? If you want to know more, you might be interested in my previous post.

Result

This project was successful. The interface was mostly done and functional by the end of the summer. Daniel merged the code into the main post-lenny branch. Development is still ongoing and packages are released into Experimental. For further information, just read the rest of my blog.

I could find some further public involvements of Obey Arthur Liu within Debian. Doh!

Lintian for fuller automated setups

Presentation

“lintian, the Debian package checker, at the moment presents possible problems in three categories: errors, warnings and informational messages. This leads to several problems, most importantly that the severity and certainty of a check can’t be expressed separately. In the course of this project, the student should design and implement in lintian an improvement of the current situation, for example by using a two-letter code (one for certainty, one for severity).”

This project would make lintian errors much more fine-grained and help in maintaining pertinent quantitative analysis of package quality.

The project was mentored by Marc Brockschmidt. The project proposal was commonly introduced by the Lintian team.

Student

UPDATED: Jordà Polo Bardés has done a lot of work with translation in Catalan, his native tongue. He can usually be found on #debian-catalan. He also maintains a few packages as a DM.

Result

This project was successful. The classification was entirely done. Jordà also helped with the new lintian.debian.org website. The Lintian team was very satisfied with the revamped errors list and new website. They have an immediate impact on packages quality reporting.

Jordà is still active within Debian, helping package a few games.

Debexpo, a generic web-based package repository

Presentation

mentors.debian.net is currently a very specialized web-based repository that allows everybody to contribute software packages to Debian without the need to be a Debian Developer (or Debian Maintainer). It has successfully helped simplifying the sponsoring process in the last years. However it needs to be refactored and in the process should be turned into a generic piece of software that can be used for other Debian source/binary package repositories, too.”

Mentors is a very good initiative to recruit new packages maintainers (and needs your help!) and the software underlying it could be reused for many different purposes (think PPA).

The project was mentored by Christoph Haas. The project proposal was commonly introduced by the mentors team.

Student

Jonny Lamb was a Computer Science student in the United Kingdom. He was already quite involved within Debian, maintaining a lot of significant packages.

Result

This project was successful. The whole proposal was perfectly executed. Jonny now continues to develop debexpo, with the mailing-lists and commit logs showing interesting activity. Of course, help for debexpo is appreciated to get it into full shape.

Jonny has since become a Debian Developer (here is his AM report). Congratulations to him.

It’s nice to end on a nice note isn’t it ? Now that we’re done with the individual reports, I’m going to write down my recommendations report. Hopefully it will help with next year’s Summer of Code.


Feb 01 2009

Debian Summer of Code ‘08 : Where are they now (part 2.5/3)

Category: Summer of Code, debianObey Arthur Liu @ 5:16 am

I’ve so far been going through the list of projects that were done last year (part 1, part 2) in a somewhat dry fashion so I’m going to make a little pause here and tell where I’m going from with these posts.

As I went through the 2008 Summer of Code at Debian, I moved from being a nearly total outsider student to something more of a developer. I’ve been contributing to Aptitude since the end of the Summer of Code (well, trying to find time to contribute more, as with many people..) and will be going to FOSDEM. I can’t say I’m an insider yet: I haven’t met a lot of people, most people have no idea who I am, I’ve been active here for, like 9 months and I’m not even in new-maintainer yet (though I plan to apply when I feel I’ll have contributed significantly to Debian).

The Student point of view

See, I’m a student in Computer Science. I use free software, I’d like to participate but it’s intimidating and you never know where to start. You know the drill. Then comes the Google Summer of Code. Let’s review its stated goals, as per its FAQ:

Google Summer of Code has several goals:

  • Get more open source code created and released for the benefit of all;
  • Inspire young developers to begin participating in open source development;
  • Help open source projects identify and bring in new developers and committers;
  • Provide students in Computer Science and related fields the opportunity to do work related to their academic pursuits (think “flip bits, not burgers”);
  • Give students more exposure to real-world software development scenarios (e.g., distributed development, software licensing questions, mailing-list etiquette).

I’m going to start with these goals and provide some of my opinions in something of a candid way.

Inspire young developers to begin participating in open source development

I have been playing with the idea of making a GUI for Aptitude ever since I dropped Synaptic, about 2 months into its use. It felt like when I bought a high-school required Texas Instruments TI-83+, that I dropped for a TI-89 within a month. Since back in 2005, every time I would see someone using Synaptic, I would pitch Aptitude as a better tool. The main reason for not doing so was that Aptitude was scary-looking. See, it’s a lot of blocky text and wacky colors.

With life and cool stuff like CPGE, I never had time to really code so I left it at that. In 2008, for the first time I was free the whole summer and so, I tried to get into the Summer of Code program and into Debian.

Actually, it wasn’t the first try. One popular way to get acquainted with Debian is to go to wnpp, adopt a package (new or orphaned) and find a mentor to upload it. In January 2008, I did try to package a set of geocaching tools I used at that time. But I didn’t find a mentor to upload it. I didn’t try very hard though and the package had some minor issues anyway. I reckon that Debian-mentor is a good idea to bring in new Debian Maintainers but the whole process is still quite technical. It is true that the minimal technical level for good packaging is not trivial in itself and the process should filter out unserious people, but the technicality curve could be adjusted to be more welcoming.

I think the Debian website could be improved in this area (ok, it’s a quite long-standing bug). Holger Levsen mentioned the way Fedora and Sugar presented avenues of collaboration to prospective developers. The crux here is that it should feel much easier to identify areas to get involved into and who to contact if needed.

Help open source projects identify and bring in new developers and committers

Actually, it would rather be the other way around: help students identify and integrate into open source projects.

For the Summer of Code, I only postulated at two organizations, Debian and Freenet (the ones working on an anonymous darknet, remember ?). I got accepted at both and ultimately chose Debian (was my first choice from the beginning).

The Debian developers community is quite unique in the way it is very decentralized, independent and fluid. There are teams in some areas (Kernel, KDE, Translation, Edu, Publicity, whatever) but much of what makes up Debian is done by individual developers working on their own.

The downside of this is that for a newcomer, it’s a little off-putting. Organized teams are not the way all things are done within Debian so there are often no smaller circle of people one gets to know. Going to Debian meetings and not knowing most of the people is a little intimidating. Keeping up with all the faces is a little hard too :) .

Many other organizations participated into the Summer of Code and many would feel arguably different. Many may be more corporate-like, more hierarchical, more centralized. I preferred Debian because it was less formal in its structure. I felt that I didn’t want to get into something that looked too much like work with supervisors and the like. It is indeed how it felt, there were no one up there to decide what we had to do. We were quite independent.

I can say that the Summer of Code is quite a good way to get a feeling of how a particular organization works in the inside.

Give students more exposure to real-world software development scenarios

As said earlier, many parts of Debian are independent, which is a result of the work separation through packages. In my work on Aptitude, it is a pity that I didn’t have to interact a lot with other members of the Debian community. Aptitude talks with the rest of the Debian packaging ecosystem through mature library interfaces so there’s not much need to ask questions beyond them, and even more so because my Daniel Burrows, my mentor and developer of Aptitude participated in their development. Also, I was working on bringing a graphical interface to it, so I didn’t have to modify a lot of core code that interacted with the outside world.

Despite this, I still met a few people interested with future developments in the area of package managers. One example is Enrico Zini who pushed his work with Xapian APT Index. Over the summer, my mentor integrated packages search through Xapian which was interesting with the expanded possibilities of a graphical interface such as search as you type, drop down suggestions and so on.

Because of the short duration of the Summer of Code, most projects can’t complete a full development cycle. The proposed work in my proposal was quite imposing. In the end, I managed to produce an (probably not even) alpha quality version of a graphical interface. My branch (if I remember correctly) was merged into the main trunk at the end of the summer and a version landed in Experimental at the end of the year.

One aspect that I missed was beta-testing feedback. During summer, only a handful people popped up on the mailing-list giving feedback on the GUI I was writing, although I knew through stats on my mercurial repository that dozens of people cloned it and followed it. Debian has no “testing team”, or any kind of semi-organized group of people who try stuff, which would be very useful to have an idea of how well I was doing.

So, did the Summer of Code at Debian give me “real-world software development scenarios” ? Not really in my case but by staying longer into Debian, I think caught up a little with that.

Here is some advice for the future Summer of Code student at Debian:

  • the work of a Linux distribution might look a little nebulous and mysterious but the reality is that Debian does a very large range of development work: web development (debexpo), user application development (aptitude-gtk), hardware interaction (Debian NAS), developer infrastructure (lintian) or even quite theoretical stuff (debgraph)… there is really a lot to choose from.
  • Debian is not a nanny-project: you will not be managed, you’re free, but you’re on your own, be prepared
  • don’t hesitate to contact anybody working in areas you’re interested in: look for packages maintainers, search mailing lists and wiki, look for debconf/fosdem/wherever talks by Debian developers. Developers will be happy to redirect you to the right people.
  • choose something that can be reasonably finished by summer’s end. Also take into account release freezes.
  • be sure to contact anyone involved into the area you’re working on: they will try to make it easier to merge your work in
  • be sure to contact anyone involved into the area you’re working on: it’s ok to ask for help
  • publicize your work: the Debian Project News will be happy to take your “press releases”, write a blog and get included on the planet, go on #debian-devel and #debian-soc, go to conferences and meetings
  • you’re not alone, interact with the other Summer of Code students, try out their code, give them feedback
  • Communicate, nobody will kill you if you say something stupid
  • Communicate, don’t be afraid if you run into problems
  • Communicate, clear enough ?

To conclude, despite all the difficulties, I would say that the Summer of Code at Debian was awesome. Although Debian is quite different from many other organizations, it was a very fruitful experience. And that’s why I’m still here.

Coming next is the rest of the projects reviews and a more concise and substantive list of recommendations for the handling of the next Summer of Code at Debian. And of course, see you at FOSDEM.