Inductive Bias

When talking to fellow developers or meeting with students it happens from time to time that I get the question of why on earth I spent my freetime working on an open source project? Why do I spend weekends at developers’ conferences like FOSDEM? Why do spent afternoons organising meetups? Why is it that I am reviewing and writing code after work for free?

Usually I point people to a post by Shalin explaining some of his reasons to contribute to open source. The post quite nicely summarises most reasons that match well with why I contribute back.

On the Apache Community mailing list Grant Ingersoll asked the question about whether devs who work on or use open source are happier in their employment.

In his response Mike posted a link to a video on what motivates people that adds another piece of information to the question of why work on open source software can be perceived as very rewarding though no money is involved: With people doing cognitively challenging tasks, motivation via payment can get you only so far. There are other motivational factors that might play an equal if not larger role in getting people to perform well on their day-to-day work:

• Autonomy: If people are supposed to be engaged with their project they need time and freedom to chose how to solve their tasks. Many large engineering driven companies like Google or Atlassian have gone even further by introducing the concept of giving people a day a week to work on what they want how they want provided they share their results. These so-called 20% projects have shown to have high potential of turning into new, creative project ideas but also even into bugs or problems getting fixed.
• Mastery: Great developers strive to get better at what they do - simply because realizing that you actually learn something and get better at what you do can be very satisfying. One way of achieving that goal is to work together with peers on common projects. The larger the pool of peers to draw from, the higher the probability of you finding mentors to help you out and to point out mistakes you make.

  There is one more factor why working on open source increases your coding level that should not be underestimated. Grant Ingersoll nicely described it in the thread mentioned above: “I was just talking with a friend yesterday, and fellow committer, who said he is a much better programmer since contributing. Of course, it makes sense. If your underwear is on display for all to see, you sure better make sure it is clean!”

• Purpose: People like to work on projects for a purpose. Be it to make all information accessible to the world or to turn earth into a better place by making cheap calls available to everyone. As a counter example deploying some software only for the purpose of selling a license and not make life of your client better by recommending the best solution to help solve his problem may not be half as satisfying.

There is quite some documentation out there on what drives people who contribute to open source projects. The video shared by Mike nicely summarizes some of the motivations of people that are independent of open source work but are closely related to it.

Freetime, Hacking apprenticeship patterns, Free Software, Get Better, Mastery, Motivation, Purpose, Self Direction, software development

A few days ago I finished reading the book “Apprenticeship Patterns” - Guidance for the Aspiring Software Craftsman, by
Dave Hoover, Adewale Oshineye. The book is addressed to readers who have the goal of becoming great software devleopers.

One naive question one could ask is why there is a need for such a book at all? Students are trained in computer science at university, then enter some IT departement and simply learn from their peers. So how is software development any different than other professions? Turns out there are a few problems with that approach: At university students usually don’t get the slightest idea of what professional software development looks like. After four years of study they still have a long way to go before writing great software. When entering your average IT shop these juniors usually are put on some sort of customer project with tight deadlines. However learning implies making mistakes, it implies having time to try different routes to find the best one. Lucky are those very few who join a team that has a way for integrating and training junior developers. Last but not least at least in Germany tech carrier paths are still rare: As soon as developers excel they are offered a promotion - which usually leads straight into management before they even had a chance to become masters in their profession.

So what can people do who love writing software and want to become masters in their profession? The book provides various patterns, grouped by task:

• Emptying the cup deals with setting up an attitude that enables learning: To be able to learn new skills the trainee first has to face his ignorance and realise that what he knows already is just a tiny little fraction of what differenciates the master from the junior.
• In the second chapter “Walking the long road” the book deals with the problem of deciding whether to stick with software development or to go into management. Both paths provide their own hurdles and rewards - in the end the developer himself has to decide which one to go. Deciding for a technical carrier however might involve identifying new kinds of rewards: Instead of being promoted to senior super duper manager, this may involve benefits like getting a 20% project, setting up a company internal user group, getting support for presenting ones projects at conferences. The chapter also deals with motivational side of software development: Let’s face it, professional development usually is way different from what we’d do if we had unlimited time. It may involve deadlines that cannot be met, it may invovle customers that are hard to communicate with. One might even have to deal with unmovtivated colleagues who have lower quality standards and no intention to learn more than what is needed to accomplish the task at hand. So there is the problem of staying motivated even if times get rough. Getting in touch with other developers - external and internal - here can be a great help: Attending user groups (or organising one), being part of an open source project, meeting regularly with other developers in one’s general geografical area all may help to remember the fun things about developing software.
• The third group of patterns has been put under the headline “Accurate self-assessment” - as people get better and better it get ever harder to remember that there are techniques out there one does not yet know. Being the best in a team means that there is not more room to learn in that environment. It’s time to find another group to get in touch with others again: To be the worst in a team means there is a lot of room for learning, finding mentors helps with getting more information on which areas to explore next. Especially helpful is working on a common project with others - doing pair programming can help even with picking up just minor optimisations in their work environment.
• The fourth chapter “Perpetual learning” deals with finding opportunities to learn new technologies - either in a toy project that in contrast to professional work is allowed to break and can be used to try and test new techniques and learn new languages. Other sources for learning are the source code itself, tech publications on magazines, books (both new and classic), blogs and mailing lists. Reflecting on what you learned helps remember it later - on option to reflect may involve writing up little summaries of what you read and keeping them in a place where you can easily retrieve them (for me this blog has turned into such a resource - yeah, I guess writing this book summary is part of the exercise, even was a proposal in the book itself). Last but not least one of the best resources for reflection and continued learning is to share knowledge - though you may feel there are others out there way better then you are, you are the one who just went though all the initial loops that no master remembers anymore. You can explain concepts in easy to understand words. Sharing and teaching means quickly finding gaps in your own knowledge and fixing them as you go forward. Last but not least it is important to create feedback loops: It does not help to learn after three years of coding that what you did does not match a customers expectations. As an apprentice you need faster feedback: On a technical level this may involve automated tests, code analysis and continuous integration. On a personal level it involves finding people to review your code. It means discussing your ideas with peers.
• The last chapter on “Constructing your curriculum” finally dealt with the task of finding a way to remain up to date, e.g. by following re-known developers’ blogs. But also studying the classic literature - there are various books in computer science and software development that have been written back in the 60s and 70s but are still highly relevant.

The book does not give you a recipe to turn from junior to master in the shortest possible time. However it successfully identifies situations many a software developer has encountered in his professional life that made him quesion his current path. It provides ideas on what to do to improve one’s skills even if the current IT industry may not be best equipped with tools for training people.

My conclusion from the book was that most important is getting in touch with other developers, exchanging ideas and working on common projects. Open source get several mentions in the book, but also for me has turned out to be a great source for getting feedback, help and input from the best developers I’ve met so far.

In addition meeting people who are working on similar projects face-to-face provides a lot of important feedback as well as new ideas to try out. Talking with someone over a cup of coffee for two hours sometimes can be more productive than discussing for days over e-mail. Hacking on a common project, maybe even in the same location, usually is the most productive way not only to solve problems but also to pick up new skills.

Free Software, Hacking apprenticeship patterns, Free Software, Hacking, learning, software development

Various research projects focus on learning more on how open source communities work:

• What makes people commit themselves to such projects?

• How much involvement from various companies is there?
• Do people contribute during working hours or in their spare time?
• Who are the most active contributors in terms of individuals and in terms of companies?

When asked to fill out surveys, especially in cases where that happens for the n-th time with n being larger than say 5, software developers usually are not very likely to fill out these questionairs. However knowing some of the processes of open source software development it soon becomes obvious there are way more extensive sources for information - albeit not trivial to evaluate and prone to at least some error.

Free software tends to be developed “in the open”: Project members with various backgrounds get together to collaborate on a common subject, exchanging ideas, designs and thoughts digitally. Nearly every project with more then two members at least has mailing list archives and some sort of commit log to some version control system. Usually people also have bugtrackers that one can use as a source for information.

If we take the ASF as an example, there is a nice application to create various statistics from svn logs:

The caveats of this analysis are pretty obvious: Commit times are set according to the local of the server, however that may be far off compared to the actual timezone the developer lives in. Even when knowing each developer’s timezone there is still some in-accuracy in the estimations as people might cross timezone bounderies when going off for vacation. Still the data available from that source should already provide some input as to when people are contributing, how many projects they work on, how much effort in general goes into each project etc.

Turning the analysis the other way around and looking at mailing list contributions, one might ask whether a company indeed is involved in free software development. One trivial, naive first shot could be to simply look for mailinglist postings that originate from some corporate mail address. Again the raw numbers displayed below have to be normalised. This time company size and fraction of developers vs. non-developers in a company has to be taken into consideration when comparing graphs and numbers to each other.

Yet another caveat are mailinglists that are not archived in the mail archiving service that one may have choosen as the basis for comparison. In addition people may contribute from their employer’s machines but not use the corporate mail address (me personally I am one of these outliers, using the apache.org address for anything at some ASF project).

101tec	JTeam	Tarent
Kippdata	Lucid Imagination	Day
HP	IBM	Yahoo!
Nokia	Oracle	Sun

Easily visible even from that trivial 5min analysis however is general trending of involvement in free software projects. In addition those projects are displayed prominently projects that employees are working with and contributing to most actively - it comes as no surprise that for Yahoo! that is Hadoop. In addition if graphs go back in time far enough, one might even see the timeframe of when a company changed its open source strategy (or was founded (see the graph of Lucid), or got acquired (see Sun’s graph), or acquired a company with a different stategy (see Oracle’s graph) ).

Sort of general advise might be to first use the data that is already out there as a starting point - in contrast to other closed communities free software developers tend to generate a lot of it. And usually it is freely available online. However when doing so, know your data well and be cautious to draw premature conclusions: The effect you are seeing may well be caused by some external factor.

Free Software, Hacking Free Software, involvement, Metrics, Open Source, Professional support

Probably not too unusual for people working on free software mostly (though no longer exclusively) in their spare time, the number of items that appear in my private calendar have increased steadily in the past months and years:

• Every three months I am organising the Apache Hadoop Get Together in Berlin.
• I have been asked (and accepted the offer) to publish articles on Hadoop and Lucene in magazines.
• There are various conferences I attend - either as speaker or simply as participant: FOSDEM, Froscon, Apache Con NA, Devoxx, Chemnitzer Linuxtag - to name just a few.
• For Berlin Buzzwords I did get quite a bit of time for organisation, still some issues leaked over to what others would call free time.
• I am mentoring one of Mahout’s GSoC students which is a lot of fun.
• At least I try to spend as much time as possible on the Mahout mailing lists keeping up with what is developed and discussed there.

There are various techniques to cope with increased work load and still find enough time to relax. Some of them involve simply remembering what to do at the right time, some involve prioritization, others deal with measuring and planning what to do. In this tiny series I’ll explain the techniques I employ - or at least try to - in the hope of getting your feedback, and comments on how to improve the system. After all, the most important task is to constantly improve ones own processes.

Freetime, Hacking, Scrum Free Software, Getting things done, Scrum, Software Foundation

In Summer last year I was invited to give a presentation on Apache Mahout at TU Berlin. After the talk was over some of the research group members asked me to design and give a course on scalable machine learning with open source software during the winter semester.

The project attracted four to five students - not very many - but then again it is a course people can take voluntarily. During the first semester participants were asked to integrate Mahout to build a system that crawls web pages, assigns them to clusters and makes the content searchable with Lucene. The intention was to get students to publish any patches they have to make at Mahout. In addition the code behind the system was supposed to be published after the project was over.

This setup turned out to be sub-optimal: The participants never grew confident enough to publish not only their ideas and design on the mailinglist but also send in the access data to the SCM system that hosted the project source code.

Some similar setup was run at HPI Potsdam by Christoph Böhm: He let students implement various information retrieval and machine learning algorithms on top of Apache Hadoop. After the course was over he tried to motivate students to publish their code at Apache Mahout. So far I have seen no submissions.

Being aware of these problems next time I setup the course for the summer semester at TU I chose a slightly different model: Having only four students who do not have enough free cycles to work on the project full time I set the goal to implement an HMM - including tests, example and documentation. Being roughly aligned with GSoC I asked students to publish their timeline in JIRA. As soon as coding started I urged them to publish even incremental progress and ask the community for feedback.

Now we do have an open JIRA issue with a patch attached to it. People also got some code review feedback already. Having Berlin Buzzwords in town while the course was still running I used my chance to get students in touch with other Mahout developers. Looks like at least one of them is planning to stay with the project for a little longer. For me it would be a great success if at least one student could be turned into a longer term contributor to the project.

So far it looks like applying the general principle of releasing code early and often helps people do integrate into some project. My own lesson learned from those experiences however is to urge students early on to get in touch and release their code: It was not particularly easy to get them to send e-mails to public mailing lists. However if they had done this just once, feedback usually was very positive - and surprised by how friendly and helpful in the free software community generally are.

Mahout Free Software, Mahout, Teaching, TU Berlin, University

Last weekend FrOSCon took place in Sankt Augustin, near Cologne. FrOSCon is organized on a yearly basis at the university of applied sciences in Sankt Augustin. It is a volunteer driven event with the goal of bringing developers and users of free software projects together. This year, the conference featured 5 tracks, two examples being cloud computing and the Java track.

Unfortunately this year the conference started with a little surprise for me and my boyfriend: Being both speakers, we had booked a room in Hotel Regina via the conference committee. Yet on Friday evening we had to learn that the reservation never actually reached the hotel… So, after several minutes talking to the receptionist, calling the organizers we ended up in a room that was booked for Friday night by someone who was known to arrive no earlier than Saturday. Fortunately for us we have a few friends close by in Düsseldorf: Fnord was so very kind to let us have his guest couch for the following night.

Checkin time next morning: On the right hand side the regular registration booth. On the left hand side the entrance for VIPs only. The FSFE quickly realized it’s opportunity: They soon started distributing flyers and stickers among the waiting exhibitors and speakers.

Set aside the organizational issues, most of the talks were very interesting and well presented. The Java track featured two talks by Apache Tomcat committer Peter Roßbach, the first one on the new Servlet 3.0 API, the second one on Tomcat 7. Too sad, my talk was in parallel to his Tomcat talk, so I couldn’t attend that. I appreciate several of the ideas on cloud computing highlighted in the keynote: Cloud computing as such is not really new or innovative, it is several good ideas so far known for instance as utility computing that are now being improved and refined to make computation a commodity. At the very moment however cloud computing providers tend to sell their offers as new, innovative products. There is no standard API for cloud computing services. That makes switching from one provider to another extremely hard and leads to vendor-lockin for its users.

The afternoon was filled by my talk. This time I tried something, that so far I only have done in user groups of up to 20 people: I first gave a short introduction into who I am and than asked the audience to describe themselves in one sentence. There were about 50 people, after 10 minutes everyone had given is self-introduction. It was a nice way of getting detailed information of what knowledge to expect from people, and it was interesting to hear people from IBM and Microsoft being in the room.

After that I attended the RestMS talk by Thilo Fromm and Peter Hintjens. They showed a novel, community driven way to standards creation. RestMS is a messaging standard that is based on a restful way for communication. So far the standard itself is still in it’s very early stages, still there are some very “alpha, alpha, alpha” implementations out there that can be used for playing around. According to Peter there are actually people who already use these implementations for production servers and send back bug reports.

Sunday started with an overview of the DaVinci VM by Dalibor Topic, the author of the OpenJDK article series in the German Java Magazin. Second talk of the day was an introduction to Scala. I already know a few details of the language, but the presentation made it easy to learn more: It was organised as an open question and answer session with live coding leading through the talk.

After lunch and some rest, the last two topics of interest were on details on the campaigns of FFII against software patents and an overview of the upcoming changes in gnome3.0.

This year’s FrOSCon did have some organizational quirks but the quality of most of the talks was really good with at least one interesting topic in one of the sessions at nearly every time slot - though I must admit that that was easy in my case with Java and cloud computing being of interest to me.

Update: Videos are up online.

Free Software Free Software, FrOSCon, Germany, Mahout, Software Foundation

Grant Ingersoll has posted a very interesting interview with Ian Holsman on Solr at Relegance, now AOL. It describes the business side of the decission to switch to an open source solution, provides some inside on the size of the installation and details which technological reasons have driven the decission to switch from a proprietary implementation to Solr:

http://www.lucidimagination.com/Community/Hear-from-the-Experts/Podcasts-and-Videos/Interview-Ian-Holsman-Relegence

Hacking, Software Foundation Free Software, Solr

GSoC (Google summer of code) - one of the open source programs of Google - has started again in 2009. Students come to work for open source projects during the summer and on success are paid by Google a fair amount of money.

This program is an ideal oportunity for students to get into open source projects: You get a mentor, you have pre-defined task to work on with a goal you set yourself. And in the end there is money.

At the beginning of GSoC student ranking Ted Dunning posted a very interesting mail on his view on why students should participate in open source development:

• It is a perfect chance to work together with senior developers that are passionate about what they do.

• Usually universities teach the theoretical side of life, which is good. But if working in industry later, students need experience with current development best practices and tools. They need to be aware of test driven development, they need to know how to use source control systems, continuous integration tools, build management frameworks, bug tracking tools. Open source projects usually are a great place to try out these technologies and learn how to best apply them.
• Working on open source students need to coordinate with their peers. They need to learn that development is not only about coding, but about communication as well.
• Last but not least this is a chance to chose yourself what you are working on and achieve so much more than when starting yet another brand new single developer project.

In the end all this adds up to learning and practicing the skills needed to successfully work on software development projects with more than just a few developers.

Free Software, Hacking Free Software, students, SwEng

Since several months at neofonie we have a talk given by external or internal developers on various subjects each Tuesday. Usually these presentations are a nice way to get an overview of new emerging technologies, to get an overview of current conference topics or to gain insight into interesting internal projects.

This week we had Apache Tomcat Committer and PMC Peter Rossbach here at neofonie to talk about the Tomcat architecture and Tomcat clustering solutions. He gave two pretty in-depth presentations on the Tomcat internals, Tomcat optimization and extension points.

Some points that were especially interesting to me: The project started out in the late nineties, initiated by a bunch of developers who just wanted to see what it takes to write a web application container and that fullfills the spec. The goal basically was a reference implementation. Soon enough however users defined the resulting code as production ready and used it.

There are a few caveats from this history that are still visible. One is the lack of tests in the codebase. Sure, each release is tested agains the Sun TCK - but these tests cannot be opened to the general public. So if you as a developer make extensions or modifications to the code base there is no easy way of knowing whether you broke something or not.

For me as a developer it was interesting to see really how complex it quickly gets to cluster tomcat deployments and make them failure resistant. Some tools mentioned that help automatic with easier deployment are Puppet and FAI. One issue however that is still on the developer’s agenda is Tomcat monitoring.

To summarize: The conference room was packed with developers expecting two very interesting talks. Thanks to Peter Rossbach for coming to neofonie and explaining more on the internals of the Tomcat software, the project and the community behind.

Software Foundation Event, Free Software, Software Foundation, tomcat

Title: Cloud Camp Berlin
Link out: Click here
Date: 2009-04-30

*Camp, General Event, Free Software