Devoxx – Day 2 HBase # Devoxx featured several interesting case studies of how HBase and Hadoop can be used to scale data analysis back ends as well as data serving front ends. Twitter Dmitry Ryaboy from Twitter explained how to scale high load and large data systems using Cassandra. Looking at the sheer amount of tweets generated each day it becomes obvious that with a system like MySQL alone this site cannot be run. ...
Devoxx – Day two – Caching # Day two started with a really good talk on caching architectures by Greg Luck. He first motivated why caching works: Even with SSIDs being available now there is still a huge performance gap between RAM access times and having to go to disk. The issue is even worse in systems that are architected in a distributed way making frequent calls to remote systems. ...
Devoxx University – Productive programmer, HBase # The first day at Devoxx featured several tutorials – most interesting to me was the pragramatic programmer. The speaker also is the author of the equally named book at O’Reilly. The book was the result of the observation that developers today are more and more IDE bound, no longer able to use the command line effectively. The result are developers that are unnecessarily slow when creating software. ...
First steps with git # A few weeks ago I started to use git not only for tracking changes in my own private repository but also for Mahout development and for reviewing patches. My setup probably is a bit unusual, so I thought, I’d first describe that before diving deeper into the specifc steps. Workflow to implement With my development I wanted to follow Mahout trunk very closely, integrating and merging any changes as soon as I continue to work on the code. ...
Scientific debugging # Quite some years ago I ready Why programs fail - a systematic guide to debugging - a book written on the art of debugging programs written by Andreas Zeller (Prof. at University Saarbrücken, researcher working on Mining Software Archives, Automated Debugging, Mutation Testing and Mining Models and one of the authors of famous Data Display Debugger). One aspect that I found particularly intriguing about the book was up to then known to me only from the book Zen and the Art of Motorcycle Maintenance: Scientific debugging as a way to find bugs in a piece of software in a structured way with methods usually known from the scientific method. ...
Part 4: Constant evaluation and improvement: Finding sources for feedback. # In recent years demand for shorter feedback cycles especially in software development has increased. Agile development, lean management and even Scrum are all for short feedback cycles: Coming from the dark ages when software projects would last for months or even years before any results could be delivered to customers we are transforming development into a process that integrates the customer in the design and evolution of his own product. ...
A Get Together Checklist # Still on the list of potentially interesting books: The Checklist Manifesto - explaining why checklists can still be valuable - especially for complex problems and tasks. Though not very complex, I chose to come up with a checklist for running a Hadoop Get Together in Berlin as an exercise. I’m trying to stick with advise provided by the Checklist for Checklists. Parties involved Find two to three speakers two months in advance. ...
Are devs contributing to OSS happier? # When talking to fellow developers or meeting with students it happens from time to time that I get the question of why on earth I spent my freetime working on an open source project? Why do I spend weekends at developers’ conferences like FOSDEM? Why do spent afternoons organising meetups? Why is it that I am reviewing and writing code after work for free? ...
Apprenticeship patterns (O’Reilly) # A few days ago I finished reading the book “Apprenticeship Patterns” - Guidance for the Aspiring Software Craftsman, by Dave Hoover, Adewale Oshineye. The book is addressed to readers who have the goal of becoming great software devleopers. One naive question one could ask is why there is a need for such a book at all? Students are trained in computer science at university, then enter some IT departement and simply learn from their peers. ...
Part 3: A polite way to say no - and why there are times when it does not work. # After having shared my thoughts on how to improve focus and how to track tasks eating up time this post will explain how to keep time invested at a more or less constant level. The goal of this exercise is to keep obligations at a reasonable level - be it at work or during ones spare time. ...