As there were several new participants, Felix started out with a very brief summary of the very core concepts of Scrum itself: Most important to know is the basic assumption of Scrum, that is planning ahead of time in a very detailed way is impossible. Defining goals and letting those who do the acutal work take the decisions on how to reach that goal is way easier and more promising. The whole process relies on fast feedback loops enabling developers and business people to run experiments on how to improve their work in a controlled environment.

Scrum comes with three roles: The development team responsible for delivering quality software, the product owner responsible for defining development goals that maximise return on investment and the scrum master as the moderator and facilitator who takes care that the roles and rituals are not broken.

Scrum comes with three plus one rituals: The daily standup (about 15min) used by the development team to get everyone up to date on a daily basis on everyone’s status, the Scrum Review and the Scrum Planning. In addition very important each sprint includes a retrospective that serves the purpose of improving the scrum team’s processes.

Scrum comes with three artifacts: The sorted product backlog of all user stories, the sprint backlog and the burndown chart showing the team’s progress.

However Scrum is just a framework - it tells you more on the goals, but less on exactly how to reach them. It should serve as a basis to adapt one’s processes to the project’s needs.

In the usual meetup planning phase we collected potential topics for discussions and ranked them by voting on them in the end. The topics proposed were:

• Applications of Scrum for non-software-development projects.
• How to convince teams of Scrum?
• Awareness of the definition of done.
• How to integrate testers in a team, extended by discussing the values of cross-functional teams.
• How to be a tech PO.
• Adding agile/XP to Scrum.
• How to keep the team on focus.
• Decision making in self organising teams.
• Bonus HR in Scrum.

The topics rated highest were on raising the awareness for the definition of done and on decision making in self organising teams.

Definition of done

To introduce the topic, Felix repeated the goal of Scrum teams to deliver potentially quality - ahem - shipable software. The problem of the guest and his co-workers was described as follows: Teams have definitions that are inconsistant not only across teams but also within teams.

Ultimately the goal of the definition of done is to enable teams to produce shipable software. One option to make the team aware of the need for better quality software might be to make them feel the pain their releases cause. It does not help to dictate a company-wide definition of done: It’s up to the team to define it. However to learn more on what shipable means the team must be allowed to make mistakes. They will fail - but learn from that failure as soon as they feel and see what gets influenced by their mistakes. As a resulst, they will refine their definition of done.

As the person to make happy in Scrum iterations is the PO, this could mean that the PO simply does not accept features, after all he is the one to define what shipable means. One factor that is a pre-condition for teams to be able to learn is to keep them stable. Learning needs time - teams need to be allowed to evolve. If yesterday’s team does not feel the pain their mistakes caused just because the team does not exist anymore or has been reconfigured - how would people be able to learn at all?

Decision making in self organising teams

The person proposing this question has the problem that in his teams some developers turn into leaders dictating the way software gets implemented. Other team members rarely join into that discussion and close to never take decisions. The result are endless discussions w/o real results.

The first idea that came up was for the Scrum Master to act as moderator. Marion came up with the proposal to use well known mediation techniques. She promised to share the links - would be great to have them published on the Scrumtisch Berlin blog as well. Thilo mentioned there are courses on mediation and moderation that can help him play that role.

As for long discussions: Felix mentioned a few typical patterns (or anti-patterns) that tend to lead to developers discussing endlessly:

• Fear: fear of punishment for taking the wrong decision usually leads developers to avoid decisions altogether. Fix for that would be establishing and open culture that allows for failure and that enables people to learn from failure.
• Striving for the 100% solution: Developers are not used to incremental thinking and try to solve all problems at once. Fix would be to teach them they get time for refactoring and are thus not punished for adhering to the YAGNI principle.
• Personal conflicts in teams can lead to the described situation as well and can only be fixed by double-checking the team configuration, potentially changing it.

There is a very good book by Cohen on “Succeeding with agile” that has a whole chapter on what makes a good Scrum Master. Checking these properties against your chosen Scrum master might help as well.

When discussing this topic we soon discovered one problem with the team configuration as-is: Scrum masters used to be system architects or senior software developers - that is, highly respected, influencial people. Maybe simply re-configuring teams might help already.

Thanks to Marion for organising the evening - and thanks to all attendees for your questions and input on discussion topics. Looking forward to the next edition of the Scrumtisch.

Disclaimer: I usually just take notes on an old-fashioned paper-notebook, typing stuff into the blog after the meeting is over. Only reason I do it the same evening is the goal of keeping the list of draft postings as short as possible.

Would be great to meet you there for tasty food, interesting discussions on Apache in general, as well as projects like Lucene, Hadoop or Tomcat in particular. Anyone interested in either the FSFE or Apache is welcome to join us.

One personal request: Somehow, Rainer (Kersten, FSFE) talked me into preparing a talk on what the ASF is all about - would be really great to have more people around share their experience.

See you in Düsseldorf

Please talk to Marion Eickmann if you would like to attend the next meetup.

• What makes people commit themselves to such projects?

• How much involvement from various companies is there?
• Do people contribute during working hours or in their spare time?
• Who are the most active contributors in terms of individuals and in terms of companies?

When asked to fill out surveys, especially in cases where that happens for the n-th time with n being larger than say 5, software developers usually are not very likely to fill out these questionairs. However knowing some of the processes of open source software development it soon becomes obvious there are way more extensive sources for information - albeit not trivial to evaluate and prone to at least some error.

Free software tends to be developed “in the open”: Project members with various backgrounds get together to collaborate on a common subject, exchanging ideas, designs and thoughts digitally. Nearly every project with more then two members at least has mailing list archives and some sort of commit log to some version control system. Usually people also have bugtrackers that one can use as a source for information.

If we take the ASF as an example, there is a nice application to create various statistics from svn logs:

The caveats of this analysis are pretty obvious: Commit times are set according to the local of the server, however that may be far off compared to the actual timezone the developer lives in. Even when knowing each developer’s timezone there is still some in-accuracy in the estimations as people might cross timezone bounderies when going off for vacation. Still the data available from that source should already provide some input as to when people are contributing, how many projects they work on, how much effort in general goes into each project etc.

Turning the analysis the other way around and looking at mailing list contributions, one might ask whether a company indeed is involved in free software development. One trivial, naive first shot could be to simply look for mailinglist postings that originate from some corporate mail address. Again the raw numbers displayed below have to be normalised. This time company size and fraction of developers vs. non-developers in a company has to be taken into consideration when comparing graphs and numbers to each other.

Yet another caveat are mailinglists that are not archived in the mail archiving service that one may have choosen as the basis for comparison. In addition people may contribute from their employer’s machines but not use the corporate mail address (me personally I am one of these outliers, using the apache.org address for anything at some ASF project).

101tec	JTeam	Tarent
Kippdata	Lucid Imagination	Day
HP	IBM	Yahoo!
Nokia	Oracle	Sun

Easily visible even from that trivial 5min analysis however is general trending of involvement in free software projects. In addition those projects are displayed prominently projects that employees are working with and contributing to most actively - it comes as no surprise that for Yahoo! that is Hadoop. In addition if graphs go back in time far enough, one might even see the timeframe of when a company changed its open source strategy (or was founded (see the graph of Lucid), or got acquired (see Sun’s graph), or acquired a company with a different stategy (see Oracle’s graph) ).

Sort of general advise might be to first use the data that is already out there as a starting point - in contrast to other closed communities free software developers tend to generate a lot of it. And usually it is freely available online. However when doing so, know your data well and be cautious to draw premature conclusions: The effect you are seeing may well be caused by some external factor.

Indian food at Yogi Haus was great and very tasty - though we should introduce a sharing algorithm for the various dishes next time around. Speaking of next time: If you would like to be part of the dinner, subscribe to our Apache Dinner mailing list. Best way to make the location suit your needs is to simply send out the next proposal yourself.

As usual the Lucene guys are the last to leave: Currently they are on their way to X-Berg for further drinks, some food and lots of fun. Looking forward to the pictures you promised, Simon

Update: Images added. Thanks for forwarding them.

Unlike the other dinner mails this one is not a poll, it’s an announcement. Some Apache folks are in town next monday (9th of August) so we decided to have a Apache Dinner with a short term notice. If you plan to come please shoot me a quick heads-up and I count you in!

We will meet at http://www.restaurant-yogihaus-berlin.de/ on Monday 9th of August at 7:30 pm. I will reserve a table for about 14 people (the average size of the last two meetings while gstein and his gang wasn’t counted :D)

Looking forward to meet you there on Monday!!

Simon

Looking forward to seeing you on Monday evening next week. Please do not forget to give Simon a quick heads-up if you are coming: Would be nice if our estimated number of guests would at least be close to the real number this time (instead of somewhere at 50% ).

In the unlikely event that you can’t make it next Monday, please subscribe to our Apache Dinner Mailinglist to recieve further announcements. If you are not living in Berlin but are still interested in dropping in from time to time: Don’t worry we do take into account that schedules of people travelling here are tight and organise meetings accordingly.

Update: Corrected year - must have mixed that up with another conference’s kick off meeting that takes place today…

I am planning to split the article in four pieces as follows as keeping all information in one article would lead to a text longer then I could possibly expect to be read from beginning to end:

Several years ago, I had no problem with tasks like going out reading a book for hours, working on code for hours, answering mails only from time to time, thinking about one particular problem for days. As the number of projects and tasks grew, these tasks became increasingly hard to accomplish: Writing code, my mind would wander off to the mailing list; when reviewing patches my mind would start actually thinking about that one implementation that was still lingering on my hard disk.

There are a few techniques for getting back to that state of thinking about just one thing at a time. One article I found very insightful was an essay by Paul Graham. He gave a pretty good analysis of thoughts that can bind your attention and draw them away from what should actually be the thing you are thinking about. According to his analysis a pretty reliable way to discover ideas that steal your attention is to observe what thoughts your mind wanders to when you are taking a shower (I would add cycling to work here, basically anything that lets your mind free to dream and think): If it is not in line with what you would like to think about, it might be a good time to think about the need to change.

There are a few ways to force your mind to stay “on-topic”. Some very easy ones are explained in a recent blog post on attention span (Thanks to Thilo for the link):

• Organising your virtual desktops such that applications are sorted according to tasks (one for communication, one for coding project x, another one for working on project y) helps to switch off distraction that would otherwise hide in plain sight. Who wants to work on code if TweetDeck is blinking at you next to your editor? In contrast to the original author I would not go so far to switch off multiple monitors: Its great to have your editor, some terminals, documentation in the browser open all at the same time in one workspace. However I do try to keep everything that has do with communication separate from coding etc.
• Train to work for longer and longer periods of time on one task and one task only: The world does not fall apart, if people have to wait for an answer to your mail for longer than 30min - at least they’ll get used to it. You do not need to take your phone to meetings: If anything is starting to melt down there will be people who know where you are and who will drag you out of the meeting room in no time. Anything else can well wait for another 60min.
• When working with tabbed browsing: Don’t open more tabs then you can easily scan. You won’t read those interesting blog post you found four weeks ago anyway. In modern browsers it is possible to detach tabs. That way you can follow the first hint of keeping even the web pages sorted on desktops according to activity: You do not need your time tracking application next to your editor. Having only documentation and testing application open there does help.
• Keep your environment friendly and supportive. Who has ever shared an office (or a lecture at university back when I was a student) with me knows that close to my desk the probability of finding sweets, cookies, drinks and snacks approaches one. Being hungry when trying to fix a bug does not help, believe me.

One additional trick that helps staying just focussed enough for debugging complex problems is to make use of systematic debugging by Andreas Zeller (also explained in Zen and the Art of Motorcycle Maintenance). The trick is to explicitly track you thoughts on paper: Write down your hypothesis of what causes the problem. Then identify an experiment to test the hypothesis - you should know how to use your debugger, when to use print statements, which unit tests to write and when to simply take a very close look at the code and potentially make it simpler for that. Only when your experiment confirms that you have found the cause of the problem you really have identified what you need to fix.

There are a few other techniques for getting things off of your head that are just there to distract you: If you ever have read the book “Getting things done” or seen the Inbox zero presentations you may already have an idea of what I am hinting at.

By now I have a calendar application that works like a charm: It reminds me of meetings ahead of time, it warns me in case of conflicts, it accepts notes, it has an amazing life span of one year and is always available (provided I do not forget it at home):

For week to week planning we tend to use Scrum including a scrum board. However that is not only for planning as anyone using Scrum may have expected already.

For my inbox the rule is to filter any mailing list into its own folder. Second rule is to keep the number of messages in my inbox to something that fits into a window with less than 15 lines: Anything I need for further reference (conference instructions, contacts, addresses that did not yet go into my little blue book, phone numbers not yet stored in my mobile phone) goes into its own folder. Anything that needs a reply is not allowed to stay in the inbox for longer than half a week. For larger projects mail gets sorted into their own project folders. Anything else simply goes to an archive: There are search indexes available, even Linux supports desktop search, search is even integrated in most mail clients. Oh and did I mention that I managed to search for one specific mail for an hour just recently, though it was filed into its own perfectly logical folder - simply because I had forgotten which folder it was?

To get rid of things I have to do “some time in the near future but not now” I keep a list in my notebook - just so my mind knows the note is there for me to review and it knows I don’t forget about it. So to some extend my notebook is my personal swap space. One thing I learnt at Google was to not use loose paper for these kinds of notes - a bound book is way better in that it keeps all notes in one place. In addition you do not get into danger of throwing notes away too early or mis-place them.

The only thing missing is a real product backlog that keeps track of larger things to do and projects to accomplish - something like “I really do need to find a weekend to drive these >250km north to the eastbaltic sea (Thanks to Astro for pointing out the typo to me - hey, that means there is at least one guy who actually did read that blog post from beginning to end - wow!) and relax”

• Every three months I am organising the Apache Hadoop Get Together in Berlin.
• I have been asked (and accepted the offer) to publish articles on Hadoop and Lucene in magazines.
• There are various conferences I attend - either as speaker or simply as participant: FOSDEM, Froscon, Apache Con NA, Devoxx, Chemnitzer Linuxtag - to name just a few.
• For Berlin Buzzwords I did get quite a bit of time for organisation, still some issues leaked over to what others would call free time.
• I am mentoring one of Mahout’s GSoC students which is a lot of fun.
• At least I try to spend as much time as possible on the Mahout mailing lists keeping up with what is developed and discussed there.

There are various techniques to cope with increased work load and still find enough time to relax. Some of them involve simply remembering what to do at the right time, some involve prioritization, others deal with measuring and planning what to do. In this tiny series I’ll explain the techniques I employ - or at least try to - in the hope of getting your feedback, and comments on how to improve the system. After all, the most important task is to constantly improve ones own processes.

As I am also an Apache Mahout committer, closely related to the Apache Hadoop project, I always found it kind of sad that there were no Hadoop packages in the official Debian repositories. I tried multiple times to find some time to get into Debian packaging myself, I learned what “debian/rules” is all about and discovered some of the intricacies of packaging Java based software. However I have to admit that I never was able to find enough time to really finish that task.

A few weeks before this year’s FOSDEM I learned on the Apache Hadoop as well as on the Debian Java lists that a guy called Thomas Koch was working on solving bug 535861 - ITP to package Hadoop. We met at FOSDEM where I tried to raise some attention in the audience for Thomas’ plans (back then he was in need for help with a few last missing pieces). In addition I invited him for Berlin Buzzwords to get in touch with other Hadoop developers and users for further input.

I am really happy that by now Hadoop has made it into the official Debian package repositories - as soon as Debian Squeezeapt-get install [Hadoop component you need]: Debian package search.

If you want to speed up the process of Squeeze being released as stable version: Help fixing the remaining bugs in that distribution. There are various Debian Bug Squashing Parties being organised around the world. Next one in Berlin is on next Monday, the one for Munich is running this weekend. Just got the information that Fefe posted in his blog a link to the Mozilla bug bounty:

The packages are based on the upstream Apache Hadoop distribution, being comparably new they are intended for development machines at the moment. If you are using Debian and want to work with Hadoop - this is a great opportunity to help making the packages more stable by simply using them and reporting your experiences back to the Debian community.

In addition Debian now also provides packages for Zookeeper as well as HBase - though the HBase version is not yet production ready as the HDFS-append patch is still missing.

To follow the general state and progress of these packages feel free to follow the packages pages for Hadoop, HBase, Zookeeper respectively.

Thomas currently plans to work more closely with upstream e.g. to tidy up the chaos in the start-up scripts and other minor glitches. So watch out for further improvements.

In addition I just saw another interesting ITP in the Debian bugtracker: Wishlist: katta. I am sure there are quite a few others as well.