Inductive Bias

Last November I innocently asked Grant what it would take to make him to give a talk in Berlin. The only requirement he told me was that I’d have to pay for his flight. About eight months later we had Berlin Buzzwords - a conference all around the topics scalability, data storage and search. With Simon Willnauer, Uwe Schindler, Michael Busch, Robert Muir, Grant Ingersoll, Andrzej Bialecki and many others we had quite a few Lucene people in town.

From the NoSQL community, Peter Neubauer, Rusty Klophaus, Jan Lehnardt, Mathias Meyer, Eric Evans and many others made sure people got their fair share of NoSQL knowledge. With Aaron Kimball, Jay Booth, Doug Judd and Steve Loughran we had several Hadoop and related people at the conference.

The conference also featured two talks on Apache Mahout: An overview from Frank Scholten as well as a more in-depth talk by Sean Owen. It’s great to see the project grow - not only in terms of development community but also in terms of requests from professional Mahout users.

In addition we had a keynote by Pieter Hintjens that concentrated on messaging in general and 0MQ in particular - a scalability topic otherwise highly underrepresented at Berlin Buzzwords.

We got well over 300 attendees that filled Berlin Kosmos - a former cinema. Attendees were a good mixture of Apache and non-Apache people, developers and users. People used the breaks and bar tours after the event to get in touch, exchange ideas. It’s always good to see developers discuss design issues and architectural challenges.

Monday evening was reserved for local people taking out the speakers and interested attendees for Bar Tours to Friedrichshain. Those from Berlin took Berlin Buzzwords people to their favourite restaurants and bars - or to what they considered to be “typical Berlin”. Some spent evenings later that week drinking beer or Berliner Weisse.

The tour for keynote speakers Grant Ingersoll, Pieter Hintjens and friends was organised by Julia and myself. We went over to Kreuzberg - some went to famous Burgermeister for Burgers, the other half went to a nearby Indian restaurant. After that we spent the evening in Club der Visionäre - a club next to the water. Me personally I left at about midnight - several people of the Lucene community moved to the well known Fette Ecke later on.

When asking the audience about repeating the conference next year, all hands went up immediately. Beside lots of praise for the organisation, from the feedback form we put up we got some good ideas on how to improve the conference next year. I’d love to have you guys back here in 2011 - and I’d love to get even more attendees in. Was great fun having you here. Thanks for 5 great days:

Five instead of two days, because:

• Keynote speakers got a special treatment - that is a personal city guide for the weekend before Buzzwords.
• We had the official conference start on Sunday with a Barcamp.
• We had another Apache dinner on Wednesday with those Apache people that live in Berlin. In addition the Aaron and Sarah joined us as they were still in town for the Apache Hadoop trainings. Also Greg Stein had pizza and beer with us - he was in town for the svn conference at the end of the week.

Thanks to all who helped turn this conference into a success: Julia Gemählich for conference management, Ulf and Wetter for WiFi setup, Nils for travel management, Simon and Jan for support ranking talks and reaching out to your communities, all speakers for fantastic talks, those taking pictures of the conference and sharing them on Flickr for showing those who stayed at home how great the conference was, peoplezapping for the videos that will soon be available online, all sponsors for supporting the conference, all attendees for their participation. I’d love to have all of you (and many more) back in Berlin next year. An informal call for presentations has been set up already - submit now and be the one to set the trend instead of just following the Buzzwords!

For those who do not want to wait for another year: We will have another Apache Hadoop Get Together in September 2010 - watch this space for more information. If you’d like to give a talk their and present your Hadoop/ Solr/ Lucene etc. system - please get in touch with me.

Apache Hadoop Get Together Berlin, Berlin Buzzwords, Events Apache, berlinbuzzwords, Hadoop, Lucene, Mahout, NOSQL

A few hours ago, Jan Lehnardt posted a link on How to organise a nerd conference - joking that this is how we planned Berlin Buzzwords. Well, it is not exactly that easy - however the comic actually is not so far from the truth either:

About two years ago, after having started Apache Mahout together with Grant Ingersoll, Karl Wettin and others, several Apache Hadoop user groups, meetups and get togethers started to pop up all around the world. The one closest to me was the Hadoop user group UK. Back in 2008 I was pretty envious to all these user groups - being so distributed, there was no way I could ever attend all of them, though talks were certainly interesting. So the naive thought of a back then naive free software developer was: Let’s have that in Berlin. To have initial talks I called Stefan Groschupf. His answer was very positive: Oh yeah, let’s do this. I am in Germany for another two weeks, so it should be at about that timeframe. We agreed that if no-one showed up, we could still have some pizza together and share insights from our projects.

For the venue I knew from regular meetups of the Free Software Foundation Europe - read FSF*E* - that newthinking store was available for free for meetups for devs of free software. On I went, calling Martin from the store, booked the room. After that some mails went to the usual suspects, mailing lists and such. At the first meetup two years ago, more than 15 attendees - with two more people who had prepared slides. Pizzas obviously had to wait a little.

If you are wondering what that looked like back then - Thanks to Martin for taking the image back then and putting it online.

We (as in all attendees) decided to repeat the exercise three months later*, talks for the next time were proposed during that first session. Noone objected to having it in Berlin again - everyone knew this was the only way to avoid having to do the organization next time.

The meetup grew steadily in size, talks started being proposed three to six months in advance. I ended up creating not only a mailing list for the meetup but also a blog so I could publish news on Jan’s CouchDB talk and Lars George’s HBase talk back then. We got video sponsoring from Cloudera (Thanks Christophe), StudiVZ (Thanks Nils), and Nokia (Thanks Matt). Late last year I did the first European NoSQL meetup together with Jan Lehnardt - 80 attendees, lots of potential for more, the newthinking store obviously a bit too small for that

If you are wondering what NoSQL and Hadoop meetups looked like last time:

During that meetup the idea was born for a larger NoSQL conference in Berlin in 2010. First ideas were tossed around together with Jan and Simon Willnauer during Apache Con US in Oakland. The topic Hadoop got added there. In January 2010 finally Lucene was added to the mix. We contacted newthinking for support - got a very warm welcome.

Now - two years after the first Apache Hadoop Get Together Berlin we are proud to host Berlin Buzzwords - focussed on NoSQL, Apache Hadoop and search as in Apache Lucene.The conference is co-organised by newthinking communications, Simon Willnauer, Jan Lehnardt and myself. A big thanks to neofonie for supporting me by making it possible that I could do most of the organisation during my regular working hours.

The speaker lineup looks fantastic. Registration is going very well - exceeding expectations (did I mention that registration is still open, group and student tickets still available?).

I am really looking forward to an amazing conference on 7th and 8th of June. We will have a NoSQL barcamp in newthinking store Sunday evening before the conference. Keynote speaker packages have been sent out and were well received. Hotel rooms for speakers are booked. We are about to pull together the last loose ends in the coming days. Happy to have so many guys (and a few girls) interested in scalability topics here in town at the beginning of June. Looking forward to seeing you in Berlin.

* The second meetup turned out to be the first and so far only one that took place w/o the organiser - I broke my leg on my way to newthinking by getting hit by a BMW X5… *sigh* Note for other meetup organizers: Always have a backup moderator - in may case that was my neofonie manager Holger Düwiger who happened to attend that meetup for the first time back then.

Apache Hadoop Get Together Berlin, Berlin Buzzwords, Hadoop, Lucene, Mahout Events, Hadoop, Lucene, Meetups, NOSQL, Solr

Uwe and Simon were so kind to take some pictures of the last Hadoop Get Together in Berlin:

Thanks for the pictures.

Apache Hadoop Get Together Berlin Berlin, Get Together, Hadoop, Solr

At the last Hadoop Get Together in Berlin Bob Schulze from eCircle in Munich gave a presentation on “Tips and patterns with HBase”. The talk has been video recorded. The result is now available online:

HBase Bob Schulze from Isabel Drost on Vimeo.

Feel free to share and distribute the video. Thanks to Bob for an awesome talk on eCircle’s usage of HBase - and on providing some background information on how HBase was applied to solve your problems.

Another thanks to Nokia for sponsoring the video taping - and to newthinking for providing the location for free.

Looking forward to Berlin Buzzwords in June. Early registration is open already. Several great talk proposals have been submitted already. If you are a Hadoop Get Together visitor (or even speaker) and would like to have a community ticket, please contact me.

Apache Hadoop Get Together Berlin Berlin, Hadoop, hbase, Video

At the last Hadoop Get Together in Berlin Dragan Milosevic from zanox in Berlin gave a presentation on “Product Search and Reporting powered by Hadoop”. The talk has been video recorded. The result is now available online:

Hadoop Dragan Milosevic from Isabel Drost on Vimeo.

Feel free to share and distribute the video. Thanks to Dragan for a fantastic talk on Zanox’ usage of Hadoop - and on providing some background information on why and how you introduced Hadoop into your systems.

Another thanks to Nokia for sponsoring the video taping - and to newthinking for providing the location for free.

One more video to go. It will be available early next week.

Apache Hadoop Get Together Berlin Berlin, Hadoop, product search, Video

Last week the March 2010 Hadoop Get Together took place in Berlin. Last speaker was Chris Male on spatial search with Lucene and Solr. The video is now available online:

Lucene Chris Male from Isabel Drost on Vimeo.

Feel free to share and distribute the video to anyone who might be interested. Thank you Chris, for traveling over from Amsterdam for an awesome talk on spatial search.

If you want to learn more on what people over at Lucene and Solr are currently working one, head over to Berlin Buzzwords - a conference on scalable search, storage and data analysis. If you yourself have interesting projects - feel free to submit a talk.

Thanks to Nokia for sponsoring the video taping - and again as always thanks to newthinking for providing the location for free.

Apache Hadoop Get Together Berlin Lucene, Solr, Spatial, Video

Slides for the last Hadoop Get Together are available online:

• Spatial Search by Chris Male
• HBase Patterns by Bob Schulze
• Scaling product search with Hadoop and Lucene by Dragan Milosevic
• My own little introduction, just in case you are interested.

Videos will follow as soon as the are ready. Watch this space for further updates.

Apache Hadoop Get Together Berlin Hadoop, hbase, Lucene, Solr, Spatial

Today (or more correctly, yesterday) the March 2010 Hadoop Get Together took place in newthinking store. I arrived rather early to have some time to do some planning for Berlin Buzzwords - got there nearly one hour before the meetup. However it did not take very long until first guests came to the store. So I quickly got my introductory slides in place - Martin from newthinking already had the room setup, camera in place and audio working.

When starting the meetup the room was already packed with some 60 people - we ended up having over 70 people interested in the mix of talks on Hadoop, HBase and Spatial search with Lucene and Solr. Doing the regular “Who are you”-round, we learned that there were people from nurago, Xing, StudiVZ, *lots and lots* of people from Nokia, Zanox, eCircle, nugg.ad and many others.

The meetup was kindly supported by newthinking store (venue for free) and Nokia (sponsored the videos). Steffen Bickel took his chance during the introduction to give a brief overview of Nokia and - guess - explain, that Nokia is a great place to work and yeah - they are hiring!

The first talk was given by Bob Schulze who joined the meetup coming from eCircle in Munich. Given his previous experience with scaling their infrastructure from a regular database/ datawarehouse setup he explained how HBase helped when processing really large amounts of data. Being an e-mail marketing provider, eCircle does have quite a bit of data to process. And yes, eCircle is hiring.

Second talk was by Dragan Milosevic from Zanox on scaling product search and reporting with Hadoop. Just as eCircle, Zanox came from a regular RDMS setup that became too expensive and too complex too scale before switching over to a Hadoop/Lucene stack. He used his chance to make the Lucene developers aware of the fact that there are users who would were actually using Lucene’s compression features. Zanox, as well, is looking for people to hire.

Last talk was by Chris Male from JTeam in Amsterdam on the developments in Lucene and Solr to support for spatial search. There are various development routes being followed: Cartesian tiers as well as numeric range searches. He also explained that most of the features are still under heavy development. He finished his talk with a demo on what can be done with spatial search in Lucene/ Solr. You already guessed so, JTeam is hiring as well

After the talks we went to Cafe Aufsturz for beers, drinks and some food. People enjoyed talking to each other exchanging experiences. A Lucene focussed table quickly formed - main topics: Spatial search, Lucene/Solr merge threads, heavy committing, Mike McCandless (is this guy real or just an alter-ego of the Lucene community?).

At some time around 11p.m. the core of the guests (well - the Lucene part of the meetup, that is Simon, Uwe and the guys from JTeam) moved over to a bar close by next to cinema central for some more beer and drinks. At about 1a.m. it finally was time to head home.

I’d like to say thanks: First of all to the speakers. Without you the meetup would not be possible. Second to newthinking and Nokia for their support. And of course to all attendees for having grown the meetup to its current size.

I had a really nice evening with people from the Hadoop, HBase and Lucene community. Special thanks to you guys from JTeam for traveling 6h to Berlin just for a “little”, though no longer that tiny, Hadoop meetup. Promise stands, to visit one of your next Lucene meetups in Amsterdam and present Mahout there - however I need some help finding affordable accomodation

Hope to see you all in June at Berlin Buzzwords.

Apache Hadoop Get Together Berlin buzzwords, Get Together, hbase, Lucene, Solr, Spatial

Due to conflicts in the schedule of newthinking store, we had to change the time of the Get Together slightly. We will start one hour earlier than announced.

When: March 10th, 4p.m.
Where: newthinking store, Tucholskystr. 48, Berlin Mitte

Looking forward to seeing you there.

Apache Hadoop Get Together Berlin Berlin, GetTogether, Hadoop

Recently I received this mail from Christophe Bisciglia on Cloudera Hadoop trainings. Thought it might be interesting to the Hadoop Berlin community:

Hadoop Fans,

Over the next year, you’ll see new options for Hadoop training and
certification from Cloudera. One of the first things you’ll see will
be live sessions outside the US, tentatively planned for the April /
May time frame.

We’ve seen strong interest in Hadoop on all of our international
trips, so we’d like to ask for community input as we decide exactly
which cities to visit next. For cities we come to, we’ll offer our 3
day developer training + certification, and with sufficient interest,
we may also include a 1 day training + certification program for
system administrators.

If you are interested in attending one or both of these sessions,
please fill out a brief survey (link below). If you’re using Hadoop at
work, and it’s time to train more of your team, you can let us know
how large of a group you have. Survey responses aren’t a commitment to
attend, but we may reach out to respondents before we schedule a
session to get a better understanding of actual attendance.

You can fill out survey here: http://www.surveymonkey.com/s/MKGZHG9

If you have any trouble with the survey, or are interested in a
private training session, please don’t hesitate to reach out directly.

Cheers,
Christophe

Apache Hadoop Get Together Berlin, Hadoop Hadoop, Trainings