Part 2: Tracking tasks, or - Where the hack did my time go to last week?

2010-09-03 18:22
After summarising some strategies for not loosing track of tasks, meetings and conferences in the last post, this one is going to focus on the retrospect on achievements. If at some point in time you have asked yourself "Where the hack did time go to?" - maybe after two busy weeks of work this article might have a few techniques for you.

Usually when that happens to me it's either a sign that I've been on vacation (where that is totally fine) or that too many different, sometimes small but numerous tasks have sneaked into my schedule.

Together with Thilo I have found a few techniques helpful in dealing with these kind of problems. The goals in applying them (at least for me) have been:

  • Configure the planned work load to a manageable amount.
  • Make transparent and trackable (to oneself and others) which and how many tasks have been finished.
  • Track over time any changes in number of tasks accomplished per time slot.

After hearing about Scrum and its way of planning tasks I thought about using it not only for software development but for task planning in general. Scrum comprises some techniques that help achieving the goals described above:

  1. In Scrum, development is split into sprints: Iterations of focussed software development that are confined to a fixed length. Each sprint is filled up with tasks. The number of tasks put into one sprint is defined by the so-called velocity of the team.
  2. Tasks are ordered by priority by the product owner. Priority here is influenced by factors like risk (riskier tasks should be attacked earlier than safe ones), ROI (those tasks that promise to increase ROI most should of course be done and launched first) and a few more. After priorisation, tasks are estimated in order - that way those tasks most important to the product owner are guaranteed to have an estimated complexity defined even if there was not enough time to estimate all items.
  3. Complexity here does not mean "amount of time to implement a feature" - it's more like how much time do we need, how much communication overhead is involved, how complex is the feature. A workable way to come up with reasonably sensible numbers is to chose one base item, assing complexity of one to it and estimate all coming items in terms of "is as complex as the base item", "has double the complexity" - and so on - according to the fibonacci series. Fibonacci is well suited for that task as do not increase linearly - similarly humans are better at estimating small things (be it distances or complexities). As soon as items get too big, estimation also tends to be way off the real number.
  4. To come up with a reasonable estimate of what can be done in any week, I usually just look back to past weeks and use that as an estimate. That technique is close enough to the real number to be a working approach.

To track what got done during the past week, we use a whiteboard as Scrum Board. Putting tasks into the known categories of todo, checked out and done. That way when resetting the board after one week and adding tasks for the following week it is pretty obvious which actions ate up most of the time. The amount of work that goes onto the board is restricted to not be larger than what got accomplished during the past week.

So what goes onto the whiteboard? Basically anything that we cannot track as working hours: The Hadoop Get Together can be found just next to doing the laundry. Writing and sending out the long deferred e-mail is put right next to going out for dinner with potential sponsors for free software courses at university.

Now that weekly time tracking is set-up - is there a way to also come up with a nice longer term measure? Turns out, there are actually three:

First and most obviously the whiteboard itself provides an easy measure: By tracking weekly velocity and plotting that against time it is easy to identify weeks with more or less freetime. As a second source of information a quick look into ones calendar quickly shows how many meetings and conferences one attended over the course of a year. Last but not least it helps to track talks given on a separate webpage.

It helps to look back from time to time: To evaluate the benefit of the respective activities, to not loose track of the tasks accomplished, to prioritise and maybe re-order stuff on the ToDo list. Would be great if you'd share some of your techniques of tracking and tracing time and tasks - either in the comments or as a separate blog post.

Berlin Scrumtisch - open discussion

2010-08-25 21:29
This evening the Berlin Scrumtisch took place in Friedrichshain. More than thirty participants followed Marion's invitation for discussions on Scrum, wine and pizza at Vecchia Trattoria.

As there were several new participants, Felix started out with a very brief summary of the very core concepts of Scrum itself: Most important to know is the basic assumption of Scrum, that is planning ahead of time in a very detailed way is impossible. Defining goals and letting those who do the acutal work take the decisions on how to reach that goal is way easier and more promising. The whole process relies on fast feedback loops enabling developers and business people to run experiments on how to improve their work in a controlled environment.

Scrum comes with three roles: The development team responsible for delivering quality software, the product owner responsible for defining development goals that maximise return on investment and the scrum master as the moderator and facilitator who takes care that the roles and rituals are not broken.

Scrum comes with three plus one rituals: The daily standup (about 15min) used by the development team to get everyone up to date on a daily basis on everyone's status, the Scrum Review and the Scrum Planning. In addition very important each sprint includes a retrospective that serves the purpose of improving the scrum team's processes.

Scrum comes with three artifacts: The sorted product backlog of all user stories, the sprint backlog and the burndown chart showing the team's progress.

However Scrum is just a framework - it tells you more on the goals, but less on exactly how to reach them. It should serve as a basis to adapt one's processes to the project's needs.

In the usual meetup planning phase we collected potential topics for discussions and ranked them by voting on them in the end. The topics proposed were:

  • Applications of Scrum for non-software-development projects.
  • How to convince teams of Scrum?
  • Awareness of the definition of done.
  • How to integrate testers in a team, extended by discussing the values of cross-functional teams.
  • How to be a tech PO.
  • Adding agile/XP to Scrum.
  • How to keep the team on focus.
  • Decision making in self organising teams.
  • Bonus HR in Scrum.

The topics rated highest were on raising the awareness for the definition of done and on decision making in self organising teams.

Definition of done

To introduce the topic, Felix repeated the goal of Scrum teams to deliver potentially quality - ahem - shipable software. ;) The problem of the guest and his co-workers was described as follows: Teams have definitions that are inconsistant not only across teams but also within teams.

Ultimately the goal of the definition of done is to enable teams to produce shipable software. One option to make the team aware of the need for better quality software might be to make them feel the pain their releases cause. It does not help to dictate a company-wide definition of done: It's up to the team to define it. However to learn more on what shipable means the team must be allowed to make mistakes. They will fail - but learn from that failure as soon as they feel and see what gets influenced by their mistakes. As a resulst, they will refine their definition of done.

As the person to make happy in Scrum iterations is the PO, this could mean that the PO simply does not accept features, after all he is the one to define what shipable means. One factor that is a pre-condition for teams to be able to learn is to keep them stable. Learning needs time - teams need to be allowed to evolve. If yesterday's team does not feel the pain their mistakes caused just because the team does not exist anymore or has been reconfigured - how would people be able to learn at all?

Decision making in self organising teams

The person proposing this question has the problem that in his teams some developers turn into leaders dictating the way software gets implemented. Other team members rarely join into that discussion and close to never take decisions. The result are endless discussions w/o real results.

The first idea that came up was for the Scrum Master to act as moderator. Marion came up with the proposal to use well known mediation techniques. She promised to share the links - would be great to have them published on the Scrumtisch Berlin blog as well. Thilo mentioned there are courses on mediation and moderation that can help him play that role.

As for long discussions: Felix mentioned a few typical patterns (or anti-patterns) that tend to lead to developers discussing endlessly:

  • Fear: fear of punishment for taking the wrong decision usually leads developers to avoid decisions altogether. Fix for that would be establishing and open culture that allows for failure and that enables people to learn from failure.
  • Striving for the 100% solution: Developers are not used to incremental thinking and try to solve all problems at once. Fix would be to teach them they get time for refactoring and are thus not punished for adhering to the YAGNI principle.
  • Personal conflicts in teams can lead to the described situation as well and can only be fixed by double-checking the team configuration, potentially changing it.

There is a very good book by Cohen on "Succeeding with agile" that has a whole chapter on what makes a good Scrum Master. Checking these properties against your chosen Scrum master might help as well.

When discussing this topic we soon discovered one problem with the team configuration as-is: Scrum masters used to be system architects or senior software developers - that is, highly respected, influencial people. Maybe simply re-configuring teams might help already.

Thanks to Marion for organising the evening - and thanks to all attendees for your questions and input on discussion topics. Looking forward to the next edition of the Scrumtisch.

Disclaimer: I usually just take notes on an old-fashioned paper-notebook, typing stuff into the blog after the meeting is over. Only reason I do it the same evening is the goal of keeping the list of draft postings as short as possible.

Scrumtisch August Berlin

2010-08-14 16:26
Just seen it - the next Scrumtisch Berlin has been scheduled for 25th August 2010 at 18:30 Uhr. So far, no official talk has been scheduled, so please expect two topics on Scrum and its application to be selected for discussion according to Marion's agile topic selection algorithm.

Please talk to Marion Eickmann if you would like to attend the next meetup.

Part 1: Travelling minds

2010-08-03 06:00
In the last post I promised to share some more information on techniques I came across and found useful under an increasing work load. Instead of taking a close look at my professional calendar I decided to use my private one as an example - first because spare time is even more precious then working hours, simply because there is so few of it and secondly because I am free to publicly scrutinize not only the methods for keeping it in good shape but also the entries in it.

I am planning to split the article in four pieces as follows as keeping all information in one article would lead to a text longer then I could possibly expect to be read from beginning to end:

  1. Part 1: Traveling minds - how to stay focussed in an always-on community.
  2. Part 2: Tracking tasks, or: Where the hack did my time go to last week?
  3. Part 3: A polite way to say no - and why there are times when it doesn't work.
  4. Part 4: Constant evaluation and improvement: Finding sources for feedback.
  5. Part 5: A final word on vacation.

Several years ago, I had no problem with tasks like going out reading a book for hours, working on code for hours, answering mails only from time to time, thinking about one particular problem for days. As the number of projects and tasks grew, these tasks became increasingly hard to accomplish: Writing code, my mind would wander off to the mailing list; when reviewing patches my mind would start actually thinking about that one implementation that was still lingering on my hard disk.

There are a few techniques for getting back to that state of thinking about just one thing at a time. One article I found very insightful was an essay by Paul Graham. He gave a pretty good analysis of thoughts that can bind your attention and draw them away from what should actually be the thing you are thinking about. According to his analysis a pretty reliable way to discover ideas that steal your attention is to observe what thoughts your mind wanders to when you are taking a shower (I would add cycling to work here, basically anything that lets your mind free to dream and think): If it is not in line with what you would like to think about, it might be a good time to think about the need to change.

There are a few ways to force your mind to stay "on-topic". Some very easy ones are explained in a recent blog post on attention span (Thanks to Thilo for the link):

  • Organising your virtual desktops such that applications are sorted according to tasks (one for communication, one for coding project x, another one for working on project y) helps to switch off distraction that would otherwise hide in plain sight. Who wants to work on code if TweetDeck is blinking at you next to your editor? In contrast to the original author I would not go so far to switch off multiple monitors: Its great to have your editor, some terminals, documentation in the browser open all at the same time in one workspace. However I do try to keep everything that has do with communication separate from coding etc.
  • Train to work for longer and longer periods of time on one task and one task only: The world does not fall apart, if people have to wait for an answer to your mail for longer than 30min - at least they'll get used to it. You do not need to take your phone to meetings: If anything is starting to melt down there will be people who know where you are and who will drag you out of the meeting room in no time. Anything else can well wait for another 60min.
  • When working with tabbed browsing: Don't open more tabs then you can easily scan. You won't read those interesting blog post you found four weeks ago anyway. In modern browsers it is possible to detach tabs. That way you can follow the first hint of keeping even the web pages sorted on desktops according to activity: You do not need your time tracking application next to your editor. Having only documentation and testing application open there does help.
  • Keep your environment friendly and supportive. Who has ever shared an office (or a lecture at university back when I was a student) with me knows that close to my desk the probability of finding sweets, cookies, drinks and snacks approaches one. Being hungry when trying to fix a bug does not help, believe me.

One additional trick that helps staying just focussed enough for debugging complex problems is to make use of systematic debugging by Andreas Zeller (also explained in Zen and the Art of Motorcycle Maintenance). The trick is to explicitly track you thoughts on paper: Write down your hypothesis of what causes the problem. Then identify an experiment to test the hypothesis - you should know how to use your debugger, when to use print statements, which unit tests to write and when to simply take a very close look at the code and potentially make it simpler for that. Only when your experiment confirms that you have found the cause of the problem you really have identified what you need to fix.

There are a few other techniques for getting things off of your head that are just there to distract you: If you ever have read the book "Getting things done" or seen the Inbox zero presentations you may already have an idea of what I am hinting at.

By now I have a calendar application that works like a charm: It reminds me of meetings ahead of time, it warns me in case of conflicts, it accepts notes, it has an amazing life span of one year and is always available (provided I do not forget it at home):
- got mine here ;) That's for organising meetings, going to conferences, getting articles done in time and not forgetting about family birthdays.

For week to week planning we tend to use Scrum including a scrum board. However that is not only for planning as anyone using Scrum may have expected already.

For my inbox the rule is to filter any mailing list into its own folder. Second rule is to keep the number of messages in my inbox to something that fits into a window with less than 15 lines: Anything I need for further reference (conference instructions, contacts, addresses that did not yet go into my little blue book, phone numbers not yet stored in my mobile phone) goes into its own folder. Anything that needs a reply is not allowed to stay in the inbox for longer than half a week. For larger projects mail gets sorted into their own project folders. Anything else simply goes to an archive: There are search indexes available, even Linux supports desktop search, search is even integrated in most mail clients. Oh and did I mention that I managed to search for one specific mail for an hour just recently, though it was filed into its own perfectly logical folder - simply because I had forgotten which folder it was?

To get rid of things I have to do "some time in the near future but not now" I keep a list in my notebook - just so my mind knows the note is there for me to review and it knows I don't forget about it. So to some extend my notebook is my personal swap space. One thing I learnt at Google was to not use loose paper for these kinds of notes - a bound book is way better in that it keeps all notes in one place. In addition you do not get into danger of throwing notes away too early or mis-place them.

The only thing missing is a real product backlog that keeps track of larger things to do and projects to accomplish - something like "I really do need to find a weekend to drive these >250km north to the eastbaltic sea (Thanks to Astro for pointing out the typo to me - hey, that means there is at least one guy who actually did read that blog post from beginning to end - wow!) and relax" :)

Series: Getting things done

2010-07-30 07:07
Probably not too unusual for people working on free software mostly (though no longer exclusively) in their spare time, the number of items that appear in my private calendar have increased steadily in the past months and years:

  • Every three months I am organising the Apache Hadoop Get Together in Berlin.
  • I have been asked (and accepted the offer) to publish articles on Hadoop and Lucene in magazines.
  • There are various conferences I attend - either as speaker or simply as participant: FOSDEM, Froscon, Apache Con NA, Devoxx, Chemnitzer Linuxtag - to name just a few.
  • For Berlin Buzzwords I did get quite a bit of time for organisation, still some issues leaked over to what others would call free time.
  • I am mentoring one of Mahout's GSoC students which is a lot of fun.
  • At least I try to spend as much time as possible on the Mahout mailing lists keeping up with what is developed and discussed there.

There are various techniques to cope with increased work load and still find enough time to relax. Some of them involve simply remembering what to do at the right time, some involve prioritization, others deal with measuring and planning what to do. In this tiny series I'll explain the techniques I employ - or at least try to - in the hope of getting your feedback, and comments on how to improve the system. After all, the most important task is to constantly improve ones own processes.

Scrum - prepare your meetings

2010-06-27 15:45
"But Scrum involves so many meetings - we already have meetings like all day and don't get to coding anything." - "However we do need transparency and communication to build great software." - "Actually scheduling and re-prioritising items during scrum planning takes so much time." Does that sound familiar to you?

What if you could fit Sprint Review and Planning I for a team of three people doing three week sprints into one hour? Impossible? Well, not quite so. As always there is a very easy trick: Be prepared.

Before the review re-visit all items you have accomplished. Get the application into a state that makes it easy to demonstrate all new features to your product owner. If you are a team working on internal projects with many internal clients - get one of them to test our the new features early on (as in way before review) and get their input onto the table.

Get a list of all items up on a whiteboard. Then with the PO work you way through these items, either demonstrating them or referring to what the "real client" said about the feature. After that compute the velocity of your past sprint.

Then go over to the list of still to be done items. (You did estimate them in separate estimation meetings already, didn't you? You also got the prioritised by your PO beforehand, didn't you?) According to the computed velocity simply check out what you can do in the upcoming sprint.

After that team and PO are done. Guess what: Working with pen, whiteboard and paper did speed up our process considerably. After all, these are meetings of at least four people: Don't want to waste working hours of four people by not using the fastest and most flexible planning method.

So - who's responsible for getting all this information up? And - if using a digital planning tool, who is supposed to get it back into the tool? Trivial: It's the scrum master's job to provide all help and tools to the team to make it a high performing team. It's way better to spend 2 extra hours preparing and "tidying up" the sprint planning than have four people spend 2 hours (total 6 vs. 8 hours) in a sub-optimal meeting.

Together with estimation meetings for me this means that each sprint one to two days go into organisational work. Still this is very low overhead compared to what the team is able to accomplish in that time.

Using Scrum for software development

2010-06-25 15:31
A few months ago I entered a new team of until then two software developers. Being so small and with a rather busy product owner, until then people had followed the rituals of Scrum only loosely. When starting to work on a new component the three of us decided to change a few thing several weeks ago:

  • We would setup infrastructure to be somewhat similar to what we knew from Apache projects: All issues to be accomplished were to be tracked in our issue tracker. We would have a dev list that mirrors whatever goes to JIRA, a commit list to mirror whatever goes into svn and a user list.
  • We would try to follow Scrum rituals more closely: Dailies were re-introduced, we setup estimation meetings and got a cooking timer to stick with 60min per meeting, review and planning was supposed to be prepared and done with paper and pens on a whiteboard. After each planning we would have a planning II as well as a retrospective to improve our processes.

After the first weeks of following these ideas we did notice several improvements that are going to be described in upcoming blog posts.

Lets start with the first (trivial) change we made: Introducing a commit mailing list. With developers sitting all together in one room, communicating openly and regularly this may seem like a huge overkill. However there are a few changes that brought to how we develop:

People would suddenly start to think about what goes into svn: Being very obviously publicly readable, commit messages became much more readable, explaining what's going on. Changes would be checked-in only if they belong to one logical change set. In return others would review what was checked in sort of automatically - spotting problems or questionable code and configuration very early. Changes that got done were made transparent to other team members very easily.

Of course the information is available anyway. However to be honest - I never really closely check each change set that got committed after issuing an svn up. Getting the stuff pushed to my inbox hugely improved the situation while still keeping a faster development cycle than when working with a review-before-commit model. This is especially important for cases where only smaller adjustments are being made. Where larger refactoring steps were necessary we would still get the code reviewed by our colleagues before checking it in.