Learning to Rank Challenge

March 9th, 2010

In one of his recent blog posts, Jeff Dalton published an article on currently running machine learning challenges. Especially interesting for those working on search engines and interested in learning new rankings from data should be the Yahoo! Learning to Rank Challenge to be held in conjunction with this year’s ICML 2010 in Haifa, Israel. The goal is to show that your algorithm does not only scale on real-world data provided by Yahoo!. Tasks are split in two. The first one focusses on traditional learning to rank procedures, the second one on transfer learning. Tracks are open to participants from industry and research.

A second challenge was published by the machine learning theory blog. The challenge is hosted by Yahoo! as well and deals with Key scientific challenges in statistics and machine learning.

Both programs look pretty interesting - would be great to lots of people from the community participating and comparing their systems.

Mahout, Science , ,

Early bird registration for Berlin Buzzwords on June 7th/8th open

March 9th, 2010

Silently registration was opened in the past days for Berlin Buzzwords - a conference on scaling search, data processing and storage taking place on June 7th/8th in Berlin/ Germany. First 100 tickets will be sold for 250 Euros + tax. Registration is possible at later dates as well, however expect prizes to rise shortly before the conference starts.

If you clicked on it earlier this week and were wondering what those strange German terms were all about: We have put online an English version as well, so language shouldn’t be much of a problem anymore.

To avoid any confusion: Conference talks will be in English - no German language skills needed for that. It is perfectly well possible to get around in Berlin w/o speaking German, however knowing a few words as always will make it easier to make friends with people in shops and hotels ;)

Berlin Buzzwords , , ,

Chemnitzer Linuxtage

March 5th, 2010

Title: Chemnitzer Linuxtage
Location: Chemnitz
Link out: Click here
Start Date: 2010-03-13
End Date: 2010-03-14

Next week the Chemnither Linuxtage take place in - well - Chemnitz. It is the second largest Linux event after Linuxtag Berlin. However only obvious for speakers and exhibitors: It is one of those events that are known for its fantastic organisation. Nearly no problems, be it WiFi, admission to the exhibitors area, food or any help in general.

I will be at the event again. You can find me at the FSFE booth, telling people what the FSFE is all about and trying to convince them to become fellows (and yes, since last summer, I am a fellow myself and own one of those really cool green crypto cards).

Events, Free Software , ,

Mahout at Berlin ignite

March 1st, 2010

This evening the first Berlin ignite event took place in the “Festsaal” in Berlin X-Berg. Organiser of the event was Matt Biddulph from Nokia Gate 5. We had eleven fantastic talks (ok, to be more precise: At least ten fantastic ones, my own can only be judged by the audience ;) ).

Topics included things you can learn when starting to collect data, themes from (agile) project management, RepRap machines (see also the Rep Rap FOSDEM 2010 talk), bots and robots. The talks finished with a presentation of a Part time scientist’s vision of getting to the moon - an article on the project is available on heise newsticker.

The room was filled with more then 120 people resulting in a location packed with interested attendees. It was great seeing the talks on such diverse topics. Hope to have more events of this format here in Berlin. Thanks go to Matt, all speakers and everyone involved in generally making the event a big success.

For those who didn’t make it to the event, slides and audio should go online soon. At least the slides on Mahout are available online.

*Camp ,

Preliminary schedule online for ignite Berlin

February 23rd, 2010

Today first talks scheduled for ignite Berlin were published. If you yourself would like to give a talk: Submission seems to still be open.

Events, Freetime, Mahout , ,

Alan Atlas at Scrumtisch Berlin

February 17th, 2010

At the last Berlin Scrumtisch, @AlanAtlas gave a presentation on how he introduced Scrum at Amazon (starting as early as back in 2004). Introducing Scrum at Amazon by that time seemed natural due to a few factors:

  • Amazon was and is always very customer centric. The original methodology of working backwards in time - that is starting with the press release, from there writing the FAQ and manual and finally get to the code - really made people concentrate on the product.
  • Teams were sized according to the two-pizza rule: Each team is supposed to be no larger than can reasonably be fed for lunch with two pizzas. Turns out that is about five to ten people, given regular american pizzas.
  • Management didn’t really care exactly *how* the job of software development was done. They only cared *that* it was done. This proved as both - an advantage as it gave quite a bit of freedom - and a disadvantage - as indifference leads to impediments in the process that cannot be easily resolved.

The goal of Alan’s team was to build an infinitely scalable, zero downtime storage system that had a web service based interface - it was the S3 team. Given the task at hand, it was clear that the task wasn’t going to be anything even close to trivial. Alan’s idea: Try Scrum on that.

He himself came to the idea after attending an Agile conference and hearing about Scrum for the first time there. Alan requested a Scrum training from his manager, who approved it, provided it took place in Seattle - close to Amazon HQ that is. Turns out, the Scrum trainer available in Seattle happened to be Ken Schwaber.

The idea of Scrum basically was spread by word of mouth propaganda: In the hallways, in the smokers lounge, close to the coffee machine. People started adopting it - with some teams doing better, some worse. Allan himself got two days a quarter to give people an introduction to this funny new methodology called Scrum.

18 months later, S3 was shipped - as a side effect, the number of subscribers to the internal scrum mailing list had increased from three to 100, there were 150 CSM graduates, still three teams using XM, reputation of scrum was mostly positive.

In January 2007 others started giving their own introductions to Scrum. Developers generally liked it, there were significant failure cases, lots of resistance and misunderstanding mostly on the management side who sometimes confused it with lean production.

In June 2008 Allen became a fulltime internal Scrum trainer. First thing organised was a Scrum gathering - in an open spaces like setup people were invited to discuss their issues and questions on Scrum.

In January 2009, about 50% Scrum usage was reached, their were 600 subscribers on the Scrum mailing list, 450 CSM graduates, reputation of Scrum was as good (or bad) as months before. However it became more and more clear that further restructuring would only be possible against a lot of resistance.

There were a few lessons learned from introducing Scrum in other companies as well:

  • You cannot introduce Scrum if there is no notion of comparably stable teams (as in for at least six months, not five projects at a time per developer).
  • You need permission and ownership of committments to really get going.
  • You need to know at least the basics of Scrum.

There are a few factors that make introduction easier: Community matters. People need to be able to talk to each other, exchange experiences and learn from one another. Coaching matters. More support, immediate success in lots of cases is the result of getting a coach on the team. Credibility for management increases, Scrum implementation is easier understood that way, skepticism and resistance reduced. Management matters. Middle management must be part of the process. They will mess up scrum if not educated correctly. Going bottom-up works up to a point - but to go the whole way, you do need management.

For Amazon, that is the point where implementation got stuck for Allan: Management was neither interested nor really involved in Scrum. However there are some impediments that cannot be fixed w/o. Still Scrum is working to some extend - teams are still trying to get better and improve the process. So it is not really “Scrum, but”. Even going only part of the way helped a lot already.

As for the questions: There were a few unusual ones - e.g. on what to do with a team that is itself skeptical about Scrum introduction - especially if that was done by higher management. There are a few ways to remedy that: Point out success to the negative team member. Make that member a Scrum master to let him go through the process and really understand what is going on. And above all make the reasons for going that way clear and transparent. Including promises and wishes from that change.

Another question dealt with how to convince management of Scrum. Clearly better performing teams with happy developers (”you cannot buy developers with money only”) are some valid reasons.

Yet another question dealt with convincing customers. Allan’s way is to sneak Scrum into the process: Speak the customers language. If he does not like spring planning, call it a project status meeting.

Asked for cases of developers feeling like they work in a hamster wheel when doing Scrum, the general consensus was that it needs to be made sure that people do not only get more responsibilities but get the benefits from Scrum as well. Otherwise Scrum with all its numbers and measurements can be perfectly abused and turned into an amazing micro management tool.

Asked for other bay area companies using Scrum - yes, apple does so, Microsoft at least tries to. Google certainly uses the ideas, without calling it Scrum. Facebook seems not to use it. Xing (not bay area) uses Scrum. There seem to be about 50% of all software development companies worldwide who are using Scrum.

After the talk there was time to gather and have some pizza and pasta, time for drinks and discussions. I really appreciated the comments and ideas exchanged after the talk. Hope to see you all next time around.

Hadoop, Scrum , ,

Apache brand name - survey

February 16th, 2010

Sally Khudairi (VP, ASF Marketing & Publicity) asked for distributing the following survey to people who might be interested in contributing their views to a study on how the brand name Apache is perceived. Me personally, I would be especially interested in finding out more on whether there are any differences in perception inside the ASF vs. outside…

We have been working with PhD candidate Roland Schroll over the past two years as he’s been compiling information on the value of the Apache brand. His advisor is community-based innovation expert Dr. Johann Füller. This is a joint project of the University of Innsbruck and the Massachusetts Institute of Technology.

If you have 10 minutes to help, it would be much appreciated. The survey is at http://surveys.hyvelive.de/10_apache/p1.php?refGroup=Apache

They would like the surveys to be completed this month (February).

They are seeking at least 300 respondents. As such, if you know others who are interested in Apache from a market perspective, feel free to forward the link to them as well.

Thanks in advance for your interest!

Kind regards,
Sally Khudairi
VP, ASF Marketing & Publicity

Apache ,

MLOSS workshop at ICML accepted

February 15th, 2010

The workshop on machine learning open source software has been accpted. Find further details on the workshop homepage.

Submissions are open until April 10th, Samoa time.

Science , ,

FOSDEM - video recordings online

February 14th, 2010

As published in the FOSDEM blog the video recordings are available online - at least for the main track and the lightning talks. Happy video watching!

Events, Freetime, Hadoop , , ,

FSFE Happy Valentine

February 14th, 2010

Today I got woken up with a friendly hug and roses waiting for me:

I do not really care about presents for sort-of-artificial celebration days like valentines day. However, FSFE had a very nice idea: The proposal was to use valentines day to show your love for free software. The website proposed to e.g. hug a free software developer, to make a gift to a team of free software developers:

I love Free Software!

Happy Valentine!

Uncategorized