Reminder: Apache Hadoop Get Together Berlin Today

2010-10-07 09:52
Just a brief reminder: The Apache Hadoop Get Together Berlin is supposed to take place today in newthinking store, Tucholskystr. 48 at 5p.m.

The meeting features two talks on Apache Mahout: Committer Sebastian Schelter will explain how to scale recommender systems with Mahout. Contributor Max Heimel is going to give an introduction to the sequence labeling facilities available in Mahout.

As usual the group will move over to Cafe Aufsturz after the meetup is over.

A big Thanks goes to JTeam for sponsoring video taping as well as to newthinking store for providing the venue for free.

Apache Dinner October Today in Potsdam

2010-10-04 08:16
The Apache Dinner Berlin will take place today. As always, everybody is invited even if you didn't participate in the poll ( This time around the dinner was organised by Daniel Naber. Thanks!

When: Monday, 2010-10-04, 19:30
Where: Lindencafe, Rudolf-Breitscheid-Str. 47/48, 14482 Potsdam-Babelsberg, This is directly next to the S-Bahn (S7, Babelsberg).

Looking forward to meeting you there!

Apache Dinner this evening

2010-09-21 08:00
This evening the September Apache Dinner takes place in Jamerica Schöneberg. I have booked a table for ten to fifteen people - we'll see whether that is sufficient this time :)

Looking forward to see you there at 7p.m.

Berlin Scrumtisch - open discussion

2010-08-25 21:29
This evening the Berlin Scrumtisch took place in Friedrichshain. More than thirty participants followed Marion's invitation for discussions on Scrum, wine and pizza at Vecchia Trattoria.

As there were several new participants, Felix started out with a very brief summary of the very core concepts of Scrum itself: Most important to know is the basic assumption of Scrum, that is planning ahead of time in a very detailed way is impossible. Defining goals and letting those who do the acutal work take the decisions on how to reach that goal is way easier and more promising. The whole process relies on fast feedback loops enabling developers and business people to run experiments on how to improve their work in a controlled environment.

Scrum comes with three roles: The development team responsible for delivering quality software, the product owner responsible for defining development goals that maximise return on investment and the scrum master as the moderator and facilitator who takes care that the roles and rituals are not broken.

Scrum comes with three plus one rituals: The daily standup (about 15min) used by the development team to get everyone up to date on a daily basis on everyone's status, the Scrum Review and the Scrum Planning. In addition very important each sprint includes a retrospective that serves the purpose of improving the scrum team's processes.

Scrum comes with three artifacts: The sorted product backlog of all user stories, the sprint backlog and the burndown chart showing the team's progress.

However Scrum is just a framework - it tells you more on the goals, but less on exactly how to reach them. It should serve as a basis to adapt one's processes to the project's needs.

In the usual meetup planning phase we collected potential topics for discussions and ranked them by voting on them in the end. The topics proposed were:

  • Applications of Scrum for non-software-development projects.
  • How to convince teams of Scrum?
  • Awareness of the definition of done.
  • How to integrate testers in a team, extended by discussing the values of cross-functional teams.
  • How to be a tech PO.
  • Adding agile/XP to Scrum.
  • How to keep the team on focus.
  • Decision making in self organising teams.
  • Bonus HR in Scrum.

The topics rated highest were on raising the awareness for the definition of done and on decision making in self organising teams.

Definition of done

To introduce the topic, Felix repeated the goal of Scrum teams to deliver potentially quality - ahem - shipable software. ;) The problem of the guest and his co-workers was described as follows: Teams have definitions that are inconsistant not only across teams but also within teams.

Ultimately the goal of the definition of done is to enable teams to produce shipable software. One option to make the team aware of the need for better quality software might be to make them feel the pain their releases cause. It does not help to dictate a company-wide definition of done: It's up to the team to define it. However to learn more on what shipable means the team must be allowed to make mistakes. They will fail - but learn from that failure as soon as they feel and see what gets influenced by their mistakes. As a resulst, they will refine their definition of done.

As the person to make happy in Scrum iterations is the PO, this could mean that the PO simply does not accept features, after all he is the one to define what shipable means. One factor that is a pre-condition for teams to be able to learn is to keep them stable. Learning needs time - teams need to be allowed to evolve. If yesterday's team does not feel the pain their mistakes caused just because the team does not exist anymore or has been reconfigured - how would people be able to learn at all?

Decision making in self organising teams

The person proposing this question has the problem that in his teams some developers turn into leaders dictating the way software gets implemented. Other team members rarely join into that discussion and close to never take decisions. The result are endless discussions w/o real results.

The first idea that came up was for the Scrum Master to act as moderator. Marion came up with the proposal to use well known mediation techniques. She promised to share the links - would be great to have them published on the Scrumtisch Berlin blog as well. Thilo mentioned there are courses on mediation and moderation that can help him play that role.

As for long discussions: Felix mentioned a few typical patterns (or anti-patterns) that tend to lead to developers discussing endlessly:

  • Fear: fear of punishment for taking the wrong decision usually leads developers to avoid decisions altogether. Fix for that would be establishing and open culture that allows for failure and that enables people to learn from failure.
  • Striving for the 100% solution: Developers are not used to incremental thinking and try to solve all problems at once. Fix would be to teach them they get time for refactoring and are thus not punished for adhering to the YAGNI principle.
  • Personal conflicts in teams can lead to the described situation as well and can only be fixed by double-checking the team configuration, potentially changing it.

There is a very good book by Cohen on "Succeeding with agile" that has a whole chapter on what makes a good Scrum Master. Checking these properties against your chosen Scrum master might help as well.

When discussing this topic we soon discovered one problem with the team configuration as-is: Scrum masters used to be system architects or senior software developers - that is, highly respected, influencial people. Maybe simply re-configuring teams might help already.

Thanks to Marion for organising the evening - and thanks to all attendees for your questions and input on discussion topics. Looking forward to the next edition of the Scrumtisch.

Disclaimer: I usually just take notes on an old-fashioned paper-notebook, typing stuff into the blog after the meeting is over. Only reason I do it the same evening is the goal of keeping the list of draft postings as short as possible.

Apache Dinner DUS

2010-08-17 19:10
the evening after FrOSCon - that is on August 22nd 2010 at 7:30p.m. CEST - a combined "FSFE Fellowship meetup/ Apache dinner*" takes place in Tigges in Düsseldorf (Brunnenstraße 1, at Bilker S-Bahnhof). Given it doesn't rain, we'll be sitting outside.

Would be great to meet you there for tasty food, interesting discussions on Apache in general, as well as projects like Lucene, Hadoop or Tomcat in particular. Anyone interested in either the FSFE or Apache is welcome to join us.

One personal request: Somehow, Rainer (Kersten, FSFE) talked me into preparing a talk on what the ASF is all about - would be really great to have more people around share their experience.

See you in Düsseldorf

Apache Dinner August Berlin recap

2010-08-09 21:54
This evening yet another Apache Dinner took place in Berlin (this time Schöneberg), location booked by Simon Willnauer. As it was announced less then a week ago (see post below) we were expecting no more then some 7 people ... we ended up being a group of 15 attendees: There was Michi Busch from Twitter together with Tanja, Uwe Schindler from Bremen joined us. With Matt and Josh some of our local Hadoop users from Nokia joined our group. We had Sebastian Schelter from Mahout. In addition there were the usual suspects, that is Jan Lehnardt, Simon Willnauer and Torsten Curdt.

Indian food at Yogi Haus was great and very tasty - though we should introduce a sharing algorithm for the various dishes next time around. Speaking of next time: If you would like to be part of the dinner, subscribe to our Apache Dinner mailing list. Best way to make the location suit your needs is to simply send out the next proposal yourself.

As usual the Lucene guys are the last to leave: Currently they are on their way to X-Berg for further drinks, some food and lots of fun. Looking forward to the pictures you promised, Simon ;)

Update: Images added. Thanks for forwarding them.

Apache Dinner Berlin - August 2010

2010-08-04 14:52
Simon (Willnauer) just sent around the following e-mail. If you have some time left next Monday evening, come join us in Yogihaus: For tasty Indian food, geeky discussions and a generally beautiful evening.

Unlike the other dinner mails this one is not a poll, it's an announcement. Some Apache folks are in town next monday (9th of August) so we decided to have a Apache Dinner with a short term notice. If you plan to come please shoot me a quick heads-up and I count you in!

We will meet at on Monday 9th of August at 7:30 pm. I will reserve a table for about 14 people (the average size of the last two meetings while gstein and his gang wasn't counted :D)

Looking forward to meet you there on Monday!!


Looking forward to seeing you on Monday evening next week. Please do not forget to give Simon a quick heads-up if you are coming: Would be nice if our estimated number of guests would at least be close to the real number this time (instead of somewhere at 50% ;) ).

In the unlikely event that you can't make it next Monday, please subscribe to our Apache Dinner Mailinglist to recieve further announcements. If you are not living in Berlin but are still interested in dropping in from time to time: Don't worry we do take into account that schedules of people travelling here are tight and organise meetings accordingly.

Update: Corrected year - must have mixed that up with another conference's kick off meeting that takes place today...

Part 1: Travelling minds

2010-08-03 06:00
In the last post I promised to share some more information on techniques I came across and found useful under an increasing work load. Instead of taking a close look at my professional calendar I decided to use my private one as an example - first because spare time is even more precious then working hours, simply because there is so few of it and secondly because I am free to publicly scrutinize not only the methods for keeping it in good shape but also the entries in it.

I am planning to split the article in four pieces as follows as keeping all information in one article would lead to a text longer then I could possibly expect to be read from beginning to end:

  1. Part 1: Traveling minds - how to stay focussed in an always-on community.
  2. Part 2: Tracking tasks, or: Where the hack did my time go to last week?
  3. Part 3: A polite way to say no - and why there are times when it doesn't work.
  4. Part 4: Constant evaluation and improvement: Finding sources for feedback.
  5. Part 5: A final word on vacation.

Several years ago, I had no problem with tasks like going out reading a book for hours, working on code for hours, answering mails only from time to time, thinking about one particular problem for days. As the number of projects and tasks grew, these tasks became increasingly hard to accomplish: Writing code, my mind would wander off to the mailing list; when reviewing patches my mind would start actually thinking about that one implementation that was still lingering on my hard disk.

There are a few techniques for getting back to that state of thinking about just one thing at a time. One article I found very insightful was an essay by Paul Graham. He gave a pretty good analysis of thoughts that can bind your attention and draw them away from what should actually be the thing you are thinking about. According to his analysis a pretty reliable way to discover ideas that steal your attention is to observe what thoughts your mind wanders to when you are taking a shower (I would add cycling to work here, basically anything that lets your mind free to dream and think): If it is not in line with what you would like to think about, it might be a good time to think about the need to change.

There are a few ways to force your mind to stay "on-topic". Some very easy ones are explained in a recent blog post on attention span (Thanks to Thilo for the link):

  • Organising your virtual desktops such that applications are sorted according to tasks (one for communication, one for coding project x, another one for working on project y) helps to switch off distraction that would otherwise hide in plain sight. Who wants to work on code if TweetDeck is blinking at you next to your editor? In contrast to the original author I would not go so far to switch off multiple monitors: Its great to have your editor, some terminals, documentation in the browser open all at the same time in one workspace. However I do try to keep everything that has do with communication separate from coding etc.
  • Train to work for longer and longer periods of time on one task and one task only: The world does not fall apart, if people have to wait for an answer to your mail for longer than 30min - at least they'll get used to it. You do not need to take your phone to meetings: If anything is starting to melt down there will be people who know where you are and who will drag you out of the meeting room in no time. Anything else can well wait for another 60min.
  • When working with tabbed browsing: Don't open more tabs then you can easily scan. You won't read those interesting blog post you found four weeks ago anyway. In modern browsers it is possible to detach tabs. That way you can follow the first hint of keeping even the web pages sorted on desktops according to activity: You do not need your time tracking application next to your editor. Having only documentation and testing application open there does help.
  • Keep your environment friendly and supportive. Who has ever shared an office (or a lecture at university back when I was a student) with me knows that close to my desk the probability of finding sweets, cookies, drinks and snacks approaches one. Being hungry when trying to fix a bug does not help, believe me.

One additional trick that helps staying just focussed enough for debugging complex problems is to make use of systematic debugging by Andreas Zeller (also explained in Zen and the Art of Motorcycle Maintenance). The trick is to explicitly track you thoughts on paper: Write down your hypothesis of what causes the problem. Then identify an experiment to test the hypothesis - you should know how to use your debugger, when to use print statements, which unit tests to write and when to simply take a very close look at the code and potentially make it simpler for that. Only when your experiment confirms that you have found the cause of the problem you really have identified what you need to fix.

There are a few other techniques for getting things off of your head that are just there to distract you: If you ever have read the book "Getting things done" or seen the Inbox zero presentations you may already have an idea of what I am hinting at.

By now I have a calendar application that works like a charm: It reminds me of meetings ahead of time, it warns me in case of conflicts, it accepts notes, it has an amazing life span of one year and is always available (provided I do not forget it at home):
- got mine here ;) That's for organising meetings, going to conferences, getting articles done in time and not forgetting about family birthdays.

For week to week planning we tend to use Scrum including a scrum board. However that is not only for planning as anyone using Scrum may have expected already.

For my inbox the rule is to filter any mailing list into its own folder. Second rule is to keep the number of messages in my inbox to something that fits into a window with less than 15 lines: Anything I need for further reference (conference instructions, contacts, addresses that did not yet go into my little blue book, phone numbers not yet stored in my mobile phone) goes into its own folder. Anything that needs a reply is not allowed to stay in the inbox for longer than half a week. For larger projects mail gets sorted into their own project folders. Anything else simply goes to an archive: There are search indexes available, even Linux supports desktop search, search is even integrated in most mail clients. Oh and did I mention that I managed to search for one specific mail for an hour just recently, though it was filed into its own perfectly logical folder - simply because I had forgotten which folder it was?

To get rid of things I have to do "some time in the near future but not now" I keep a list in my notebook - just so my mind knows the note is there for me to review and it knows I don't forget about it. So to some extend my notebook is my personal swap space. One thing I learnt at Google was to not use loose paper for these kinds of notes - a bound book is way better in that it keeps all notes in one place. In addition you do not get into danger of throwing notes away too early or mis-place them.

The only thing missing is a real product backlog that keeps track of larger things to do and projects to accomplish - something like "I really do need to find a weekend to drive these >250km north to the eastbaltic sea (Thanks to Astro for pointing out the typo to me - hey, that means there is at least one guy who actually did read that blog post from beginning to end - wow!) and relax" :)

Series: Getting things done

2010-07-30 07:07
Probably not too unusual for people working on free software mostly (though no longer exclusively) in their spare time, the number of items that appear in my private calendar have increased steadily in the past months and years:

  • Every three months I am organising the Apache Hadoop Get Together in Berlin.
  • I have been asked (and accepted the offer) to publish articles on Hadoop and Lucene in magazines.
  • There are various conferences I attend - either as speaker or simply as participant: FOSDEM, Froscon, Apache Con NA, Devoxx, Chemnitzer Linuxtag - to name just a few.
  • For Berlin Buzzwords I did get quite a bit of time for organisation, still some issues leaked over to what others would call free time.
  • I am mentoring one of Mahout's GSoC students which is a lot of fun.
  • At least I try to spend as much time as possible on the Mahout mailing lists keeping up with what is developed and discussed there.

There are various techniques to cope with increased work load and still find enough time to relax. Some of them involve simply remembering what to do at the right time, some involve prioritization, others deal with measuring and planning what to do. In this tiny series I'll explain the techniques I employ - or at least try to - in the hope of getting your feedback, and comments on how to improve the system. After all, the most important task is to constantly improve ones own processes.

Apache Lunch in Portugal

2010-07-15 13:37
Just read on the Apache community mailing list that inspired by our Apache Dinner Berlin people in Porto are organising an Apache Lunch event. As with the dinner here in Berlin, anyone who is interested in Apache is welcome to join - no need to be a committer or even ASF member ;)

If you are living close to Porto, or always wanted to visit the city - after all it's a very beautiful place, there is a beach close by, there are many tasty restaurants - don't hesitate to get in touch with the organisers:

My xmpp is: Feel free to add me.

People interested in coming, let us known your availability during the 2
first weeks of August.

So, if you are interested in Apache head over to Filipe - I'd love to be there, however my summer vacation ended one week ago. Wish you guys a lot of fun.