Inductive Bias

By now FOSDEM turned into some kind of tradition in our family: Since 2007 every year in February we are travelling to that one comfy B&B in Brussels for a weekend - not to take a closer look at the city but to attend (together with thousands of other geeks and open source hackers) one of the biggest conferences on all things open source.

I love the conference concept for scaling: As it happens on a university campus they only have a limited number of huge rooms, but a fairly large number of mid-sized and small rooms. They use the venue to their advantage: Only the main tracks are organised by the official conference organisers. All other rooms are each filled with a community (usually a handful of people doing the heavy lifting of inviting speakers, putting a cfp out etc.) provided track. As a result even though they attract well over 5000 attendees it's only the most popular topics (e.g. Elasticsearch, GPL license discussions, configuration management, JDK specifics, kdbus, discussions on the relevancy of distribution packaging) tend to fill rooms completely.

When stuck with full rooms there's always some other talk on that probably is similarly interesting. In addition there is a huge exhibition area to visit where you can talk to the core developers behind the individual open source projects (and buy stuff like T-Shirts, grab a few flyers, stickers etc.). In addition there's so many people to talk to you usually won't get to see too many presentations anyway. In addition as of this year the crazy people of the FOSDEM (formerly Debian-only) video team managed to get (on a best effort, "hopefully the technician doesn't sleep in after that many beers yesterday" basis) all 445 talks video-taped with the goal of putting these online Monday after the conference.

On top I had the huge advantage of being able to simply follow to the talks my husband would go to if all of the stuff I am interested in is full: That would give me insight to completely different topics but also make it extremely easy for me to identify the good speakers and interesting presentations based on his experience avoiding the "if I'm unfamiliar with the topic and area I usually bump into the boring, badly given talks" problem.

I spent most of Saturday in a few talks on software patents, Daniel Naber's Language Tool, the Jolla BoF, the FLA. The rest of the time I caught up with the FSFE (thanks again for the bringing the onesies!), Debian, Open Office, Lucene and Elasticsearch people. The day ended at the Jolla community dinner - with a waitress that was completely overwhelmed by the number of geeks wanting food, beer and soft drinks.

Sunday I started with Chris Kühl's talk on Memory Tuning Android for Low Memory Devices. After that I helped at the Elasticsearch booth - FOSDEM definitely is an incredibly busy event. But given their target audience also is an interesting mix of people to talk to: Some had no idea about how text search works but came over because they had heard about Wikipedia using the project for search and wanted to know more. Some had a very clear idea of how they could benefit from Lucene for their NLP projects and wanted to know more on how that fits with Elasticsearch. Others were switching from Solr Cloud and needed some advise on how the systems compare for their particular use case. Others again were using Elasticsearch to analyse log files in a distributed fashion and needed advise on how to implement some feature. There was this one guy from Debian I've known ever since helping at the FSFE booth at Chemnitzer Linuxtage back in 2008 I believe who wanted to know more about Elasticsearch because one of his fellow package maintainers had been volunteered to work on the Elasticsearch RFP (#660826 if you are interested in reading more).

Overall (as every year) a really pleasant experience. What was in particular interesting this year was to meet people I knew only from completely other contexts (CCC events, system administrators, core Apache httpd people). Seems like FOSDEM is not only growing bigger but also more diverse.

The only kind of feedback I would provide is to split some dev rooms by finer grained topics to parallelise and scale even better (the Java, NoSQL and configuration management ones come to my mind first but there probably are others as well - of course again this depends on room availability and actual community members volunteering to submit a dev room related to their project in particular).

I'm glad I'm travelling home by train like last year - not only does that give me time to code and write the blog post you are reading just now, it's also comfortable to get rid of the usual sleep deprivation :)

First of all a disclaimer: I had a little bit of time left during the last few weeks. As a result my blog migrated from dynamic wordpress content to statically hosted pages. If anything looks odd, in case you find any encoding issues, if you miss specific functionality - please do let me know. I'll switch from this beta url back to the old sub-domain in a week or so unless there are major complaints.

Today was my first day in a new office. Some of you may have heard it already: As of today I'm working for Elasticsearch. Apparently the majority of devs here are using Apple hardware so with a little help from Random Tutor and my husband I got my new machine equipped to boot Linux in parallel yesterday.

As a result what was left for today was reading lots of documentation, getting accustomed to the internal tools, attending my first daily standup, forking the repository, starting to take a closer look at the source code, issue tracker and design docs. Now looking forward to both - the elasticsearch training in December and meeting other elasticsearch people over at FOSDEM next year: Find me at their booth there. Thanks for the warm welcome everyone!

Today Retresco hosted the (to my knowledge fourth) Elastic Search User Group Berlin - a group dedicated to using Lucene as part of Elastic Search. With roughly fifteen attendees the meetup attracted a decent crowd - most interestingly many of the people there were already using the software either in production or for closed beta projects.

The fist talk given was by people from ferret-go - a company doing media monitoring for brands focused on the German market. They are pretty new to the search topic, on top they aren't fluent Java developers but do most stuff in Python. Essentially their whole application is built on top of Elastic Search - most features are implemented as more or less complex search queries. In recent weeks they had to deal with typical problems related to growing data set sizes, nodes getting hot in particular when load balancing isn't configured quite right and balancing shards on a per index level instead of doing it globally for all shards (particularly bad in their case as they added an index with smaller shards and one with significantly larger shards into ES).

The second talk gave a really nice overview on things to keep in mind before putting ES to production - as is usually the case, the default configuration makes it easy to get started but most likely is not what you want in your production environment.

Really nice, technically focused event. Thanks to Retresco for hosting the meetup including beer and Club Mate and to ElasticSearch.com for paying for the pizza. Check their meetup page for the next event - most likely to be scheduled in January.

Also if you'd like to learn more on Lucene 4 (talk by Simon Willnauer) - make sure to attend the Apache Hadoop Get Together December 2012 (if you need another ticket, it might make sense to politely ask the organiser, make sure you are registered, otherwise most likely you won't get through security). Other two talks scheduled: Pere Urbon Bayes on NoSQL and Graph, a love story! as well as myself on How to Fail Your Big Data Project Quick and Rapidly.