Inductive Bias

“But Scrum involves so many meetings - we already have meetings like all day and don’t get to coding anything.” - “However we do need transparency and communication to build great software.” - “Actually scheduling and re-prioritising items during scrum planning takes so much time.” Does that sound familiar to you?

What if you could fit Sprint Review and Planning I for a team of three people doing three week sprints into one hour? Impossible? Well, not quite so. As always there is a very easy trick: Be prepared.

Before the review re-visit all items you have accomplished. Get the application into a state that makes it easy to demonstrate all new features to your product owner. If you are a team working on internal projects with many internal clients - get one of them to test our the new features early on (as in way before review) and get their input onto the table.

Get a list of all items up on a whiteboard. Then with the PO work you way through these items, either demonstrating them or referring to what the “real client” said about the feature. After that compute the velocity of your past sprint.

Then go over to the list of still to be done items. (You did estimate them in separate estimation meetings already, didn’t you? You also got the prioritised by your PO beforehand, didn’t you?) According to the computed velocity simply check out what you can do in the upcoming sprint.

After that team and PO are done. Guess what: Working with pen, whiteboard and paper did speed up our process considerably. After all, these are meetings of at least four people: Don’t want to waste working hours of four people by not using the fastest and most flexible planning method.

So - who’s responsible for getting all this information up? And - if using a digital planning tool, who is supposed to get it back into the tool? Trivial: It’s the scrum master’s job to provide all help and tools to the team to make it a high performing team. It’s way better to spend 2 extra hours preparing and “tidying up” the sprint planning than have four people spend 2 hours (total 6 vs. 8 hours) in a sub-optimal meeting.

Together with estimation meetings for me this means that each sprint one to two days go into organisational work. Still this is very low overhead compared to what the team is able to accomplish in that time.

Scrum Scrum

A few months ago I entered a new team of until then two software developers. Being so small and with a rather busy product owner, until then people had followed the rituals of Scrum only loosely. When starting to work on a new component the three of us decided to change a few thing several weeks ago:

• We would setup infrastructure to be somewhat similar to what we knew from Apache projects: All issues to be accomplished were to be tracked in our issue tracker. We would have a dev list that mirrors whatever goes to JIRA, a commit list to mirror whatever goes into svn and a user list.
• We would try to follow Scrum rituals more closely: Dailies were re-introduced, we setup estimation meetings and got a cooking timer to stick with 60min per meeting, review and planning was supposed to be prepared and done with paper and pens on a whiteboard. After each planning we would have a planning II as well as a retrospective to improve our processes.

After the first weeks of following these ideas we did notice several improvements that are going to be described in upcoming blog posts.

Lets start with the first (trivial) change we made: Introducing a commit mailing list. With developers sitting all together in one room, communicating openly and regularly this may seem like a huge overkill. However there are a few changes that brought to how we develop:

People would suddenly start to think about what goes into svn: Being very obviously publicly readable, commit messages became much more readable, explaining what’s going on. Changes would be checked-in only if they belong to one logical change set. In return others would review what was checked in sort of automatically - spotting problems or questionable code and configuration very early. Changes that got done were made transparent to other team members very easily.

Of course the information is available anyway. However to be honest - I never really closely check each change set that got committed after issuing an svn up. Getting the stuff pushed to my inbox hugely improved the situation while still keeping a faster development cycle than when working with a review-before-commit model. This is especially important for cases where only smaller adjustments are being made. Where larger refactoring steps were necessary we would still get the code reviewed by our colleagues before checking it in.

Scrum Scrum

For Berlin Buzzwords we concentrated pretty heavily on scalable systems and architectures: We had talks on Hadoop for scaling data analysis; HBase, Cassandra and Hypertable for scaling data storage; Lucene and Solr for scaling search.

A recurring pattern was people telling success stories involving project that either involve large amounts of data or growing user numbers. Of course the whole topic of scalability is extremely interesting for ambitious developers: Who would not be happy to solve internet-scale problems, have petabytes of data at his fingertips or tell others that their “other computer is a data center”.

There are however two aspects of scalability that people tend to forget pretty easily: First of, if you are designing a new system from scratch that implements a so far unknown business case - your problem most likely is not scalability. It’s way more likely that you have to solve marketing tasks, just getting people to use your cool new application. Only after observing what users actually do and use you have the slightest chance of spotting the real bottlenecks and optimising with clear goals in mind (e.g. reduce database load for user interaction x by 40%).

The second issue people tend to forget about scalability is that the term is about scaling systems - some developers easily mix that up with high performance. The goal is not to be able to deal with high work load, but to build a system that can deal with increasing (or decreasing) work load. Ultimately this means that not only your technology must be scalable: Any architecture can only scale to a certain load. The organisation building the system must be willing to continuously monitor the application they built - and be willing to re-visit architectural decisions if the environment changes.

Jan Lehnardt had a very interesting point in his talk on CouchDB: When talking about scalability, people usually look into the upper right corner of the application benchmark. However to be truely scalable one should also look into the lower left corner: Being scalable should not only mean to be able to scale systems up - but also to be able to scale them down. In the case of CouchDB this means that not only large installations at BBC are possible - but running the application on mobile devices should be possible without problems as well. It’s an interesting point in the current “high scalability” hype.

Free Software CouchDB, scalability

… for the next two weeks: I’ll be on vacation with strict internet interdiction. Will be a tourist exploring beaches and maybe a few hiking tracks in the next few days, so don’t expect to read anything here apart from what was scheduled already

Freetime vacation

In Summer last year I was invited to give a presentation on Apache Mahout at TU Berlin. After the talk was over some of the research group members asked me to design and give a course on scalable machine learning with open source software during the winter semester.

The project attracted four to five students - not very many - but then again it is a course people can take voluntarily. During the first semester participants were asked to integrate Mahout to build a system that crawls web pages, assigns them to clusters and makes the content searchable with Lucene. The intention was to get students to publish any patches they have to make at Mahout. In addition the code behind the system was supposed to be published after the project was over.

This setup turned out to be sub-optimal: The participants never grew confident enough to publish not only their ideas and design on the mailinglist but also send in the access data to the SCM system that hosted the project source code.

Some similar setup was run at HPI Potsdam by Christoph Böhm: He let students implement various information retrieval and machine learning algorithms on top of Apache Hadoop. After the course was over he tried to motivate students to publish their code at Apache Mahout. So far I have seen no submissions.

Being aware of these problems next time I setup the course for the summer semester at TU I chose a slightly different model: Having only four students who do not have enough free cycles to work on the project full time I set the goal to implement an HMM - including tests, example and documentation. Being roughly aligned with GSoC I asked students to publish their timeline in JIRA. As soon as coding started I urged them to publish even incremental progress and ask the community for feedback.

Now we do have an open JIRA issue with a patch attached to it. People also got some code review feedback already. Having Berlin Buzzwords in town while the course was still running I used my chance to get students in touch with other Mahout developers. Looks like at least one of them is planning to stay with the project for a little longer. For me it would be a great success if at least one student could be turned into a longer term contributor to the project.

So far it looks like applying the general principle of releasing code early and often helps people do integrate into some project. My own lesson learned from those experiences however is to urge students early on to get in touch and release their code: It was not particularly easy to get them to send e-mails to public mailing lists. However if they had done this just once, feedback usually was very positive - and surprised by how friendly and helpful in the free software community generally are.

Mahout Free Software, Mahout, Teaching, TU Berlin, University

First thing to do after getting up: Go to the kitchen and switch on the coffee machine. However, one random Sunday morning that caused the fuse for exactly this kitchen to go off. After fixing that we turned on the coffee machine again - trying to finally get a first cup. All worked well until having a closer look at what the machine produced as coffee: It was cold!

We initially got the machine from Giuseppetti - a vendor in Berlin. Though it was to late to get it fixed on warranty we still took the thing over to his shop the following week. What happened than was amazing to see for us:

The mechanics unscrewed the machine, started examining it immediately. Knowing we had bought it there, the owner gave both of us a cup of coffee. Long before I had finished mine the fixed machine was brought back to us - a fuse in it had gone off as well.

So after less than half an hour we got our working coffee machine back w/o being charged for the repair. Next morning was way better than the previous Sunday morning.

Freetime coffee, service

A few days ago, Linus send a very interesting mail on why he considers C the programming language that is most suitable for the Linux kernel. Despite the language specific arguments, the text contains quite a few insights on how the Linux kernel community works and communicates that might be interesting to non-kernel-hackers as well:

People working for free still doesn’t mean that it’s fine
to make the work take more effort - people still work for
other compensation, and not feeling excessively
frustrated about the tools (including language) and getting
productive work done is a big issue.

When attending Open Source conferences or contributing to free software projects I have made the exact same observation multiple times: Developers may not get money out of contributing to a free software project (though often they may do) there are other rewards that keep them working on any particular project: Learning from excellent peers usually is one reason. Being able to work on a topic you like at any time that suits you may be another one.

Developing free software is largely different than your usual professional work: People work voluntarily, putting as much effort into the code as is needed for them to be satisfied with the end-result. There may not be any deadlines fixed by contracts, still developers honour release cycles with the goal of providing a reliable product to their end users.

In the end it all boils down to being passionate about what you work on. To be involved in any open source project takes a huge amount of energy - but usually you get more in return than you are even able to invest. However if passion is a pre requisite to working on any free software this also means it is extremely hard to pay developers to work on any free software project: You just cannot buy passion or love for money.

But the thing is, “lines of code” isn’t even remotely close
to being a measure of productivity, or even the gating
issue. The gating issue in any large project is pretty much
all about (a) getting the top people and (b) communication.

Can only quote that - totally agree with the analysis. This applies to any software project - free or proprietary. So if you own a software development business: What is your strategy for getting the top people and facilitating communication in your company? What is your measure of productivity?

Free Software FreeSoftware, LinuxKernel, Software Foundation

After Berlin Buzzwords is over now, I thought it might be time to update a formerly published velocity graph. If you have been following this blog during the past few months you may know already, that we are using Scrum @ Home for several months already. I even published results of a Scrum Nokia test earlier this year.

Today I created another one of these nice velocity charts. What is interesting about this graph are two things. First of all: There are two spikes in the velocity graph. As our scrum board is used only for things done during our spare time, these spikes roughly correspond to German vacation days (Christmas and Easter).

More interestingly the work load shortely before Berlin Buzzwords happend was not much higher than usual. Weekend was totally free of any work - except for meeting with friends, going to town and attending the Barcamp. A sure sign of fantastic preparation and organisation with that few tasks left to do. Julia, thanks for helping with scheduling tasks such that no panicky late-night action was needed before, during or after the conference.

Scrum Berlin Buzzwords, Scrum

Last November I innocently asked Grant what it would take to make him to give a talk in Berlin. The only requirement he told me was that I’d have to pay for his flight. About eight months later we had Berlin Buzzwords - a conference all around the topics scalability, data storage and search. With Simon Willnauer, Uwe Schindler, Michael Busch, Robert Muir, Grant Ingersoll, Andrzej Bialecki and many others we had quite a few Lucene people in town.

From the NoSQL community, Peter Neubauer, Rusty Klophaus, Jan Lehnardt, Mathias Meyer, Eric Evans and many others made sure people got their fair share of NoSQL knowledge. With Aaron Kimball, Jay Booth, Doug Judd and Steve Loughran we had several Hadoop and related people at the conference.

The conference also featured two talks on Apache Mahout: An overview from Frank Scholten as well as a more in-depth talk by Sean Owen. It’s great to see the project grow - not only in terms of development community but also in terms of requests from professional Mahout users.

In addition we had a keynote by Pieter Hintjens that concentrated on messaging in general and 0MQ in particular - a scalability topic otherwise highly underrepresented at Berlin Buzzwords.

We got well over 300 attendees that filled Berlin Kosmos - a former cinema. Attendees were a good mixture of Apache and non-Apache people, developers and users. People used the breaks and bar tours after the event to get in touch, exchange ideas. It’s always good to see developers discuss design issues and architectural challenges.

Monday evening was reserved for local people taking out the speakers and interested attendees for Bar Tours to Friedrichshain. Those from Berlin took Berlin Buzzwords people to their favourite restaurants and bars - or to what they considered to be “typical Berlin”. Some spent evenings later that week drinking beer or Berliner Weisse.

The tour for keynote speakers Grant Ingersoll, Pieter Hintjens and friends was organised by Julia and myself. We went over to Kreuzberg - some went to famous Burgermeister for Burgers, the other half went to a nearby Indian restaurant. After that we spent the evening in Club der Visionäre - a club next to the water. Me personally I left at about midnight - several people of the Lucene community moved to the well known Fette Ecke later on.

When asking the audience about repeating the conference next year, all hands went up immediately. Beside lots of praise for the organisation, from the feedback form we put up we got some good ideas on how to improve the conference next year. I’d love to have you guys back here in 2011 - and I’d love to get even more attendees in. Was great fun having you here. Thanks for 5 great days:

Five instead of two days, because:

• Keynote speakers got a special treatment - that is a personal city guide for the weekend before Buzzwords.
• We had the official conference start on Sunday with a Barcamp.
• We had another Apache dinner on Wednesday with those Apache people that live in Berlin. In addition the Aaron and Sarah joined us as they were still in town for the Apache Hadoop trainings. Also Greg Stein had pizza and beer with us - he was in town for the svn conference at the end of the week.

Thanks to all who helped turn this conference into a success: Julia Gemählich for conference management, Ulf and Wetter for WiFi setup, Nils for travel management, Simon and Jan for support ranking talks and reaching out to your communities, all speakers for fantastic talks, those taking pictures of the conference and sharing them on Flickr for showing those who stayed at home how great the conference was, peoplezapping for the videos that will soon be available online, all sponsors for supporting the conference, all attendees for their participation. I’d love to have all of you (and many more) back in Berlin next year. An informal call for presentations has been set up already - submit now and be the one to set the trend instead of just following the Buzzwords!

For those who do not want to wait for another year: We will have another Apache Hadoop Get Together in September 2010 - watch this space for more information. If you’d like to give a talk their and present your Hadoop/ Solr/ Lucene etc. system - please get in touch with me.

Berlin Buzzwords, General, Get Together Berlin Buzzwords, Hadoop, Lucene, Mahout, NOSQL, Software Foundation

After Berlin Buzzwords was over yesterday - and as there is an svn conference in the city that starts tomorrow, we thought we could easily put together a smallish Apache Dinner for tonight. So Torsten mailed a few people, booked some space at Heinz Minki in Kreuzberg. We announced it at the end of the conference and invited people to join us.

So after a beautiful day out I spent the evening with a bunch of Apache related guys and girls having drinks and great pizza: We had several svn committers, Greg Stein met us there - he arrived today for the svn conference. Of couse the usual suspects were there as well: Simon Willnauer and his wife, Torsten Curdt, Erik Abele, Thomas Wöhlke, Valerie Hajdik. In addition we had guests from the Apache Lucene and Apache Hadoop communities: Sarah Sproehnle and Aaron Kimball as well as Uwe Schindler joined us.

Thanks to Torsten for putting the meetup together. See you next time in July. If you are interested in joining our meetups: Subscribe to our mailing list.

Apache Dinner Berlin Berlin Buzzwords, Dinner, Hadoop, Lucene, Software Foundation