Inductive Bias

Last week I had the honour of giving a keynote at Adobe's Open Source Summit EU in Basel. Among many interesting talks they hosted a panel to answer questions around all things open source, strategy, inner source, open development. One of the questions I found intersting is how inner source/ open development and agile are related.

To get everyone on the same page, what do I mean when talking about inner source/ open development? I first encountered the concept at ApacheCon EU Amsterdam in 2008: Bertrand Delacretaz, then Day Software, now Adobe was talking about how to apply open source development principles in a corporate environment. A while ago I ran across the term http://innersourcecommons.org that essentially wants to achieve the same thing. What I find intersting about it is the idea of applying values like transparency, openess to contributions as well as concepts like asynchronous communication and decision making at a wider context.

What does that have to do with Agile? Looking at the mere mechanics of Agile like being co-located, talking face to face frequently, syncing daily in in person meetings this seems to be pretty much opposite of what open source teams do.

Starting with the values and promises of Agile though I think the two can compliment each other quite nicely:

"Individuals and interactions over processes and tools" - there's a saying in many successful popular projects that the most important ingredient to a successful open source project is to integrate people into a team working towards a shared vision. Apache calls this Community over code, ZeroMQ's C4 calls it "People before code", I'm sure there are many other incarnations of the same concept.

Much like in agile, this doesn't mean that tools are neglected alltogether. Git was created because there was nothing quite like this tool before. However at the core of it's design was the desire for ideal support for the way the Linux community works together.

"Working software over comprehensive documentation" - While people strive for good documentation I'd be so bold as to suggest that for the open source projects I'm familiar with the most pressing motivation to provide good documentation is to lower the number of user questions as well as the amount of questions people involved with the project have to answer.

For projects like Apache Lucene or Apache httpd I find the documentation that comes with the project to be exceptionally good. In addition both projects are supported with dead tree documentation even. My guess would be that working on geographically distributed teams who's members work on differing schedules is one trigger for that: Given teams that are geographically distributed over multiple time zones means that most communication will happen in writing anyway. So instead of re-typing the same answers over and over, it's less cost intensive to type them up in a clean way and post them to a place that has a stable URL and is easy to discoveri. So yes, while working software clearly is still in focus, the lack for frequent face to face communication can have a positive influence on the availability of documentation.

"Customer collaboration over contract negotiation" - this is something open source teams take to the extreme: Anybody is invited to participate. Answers to feature requests like "patches welcome" typically are honest requests for support and collaboration, usually if taken up resulting in productive work relationships.

"Responding to change over following a plan" - in my opinion this is another value that's taken to the extreme in open source projects. Without control over your participants, no ability to order people to do certain tasks and no influence over when and how much time someone else can dedicate to a certain task reaciting to changes is core to each open source project cosisting of more than one maintaining entity (be it single human being or simply one company paying for all currently active developers).

One could look further, digging deeper into how the mechanics of specific Agile frameworks like Scrum match against what's being done in at least some open source projects (clearly I cannot speak for each and ever project out there given that I only know a limited set of them), but that's a topic for a another post.

The idea of applying open source development practices within corporations has been hanging in the air for over 15 years, driven by different individuals over time. I think now that people are gathering to collect these ideas in one place and role them out in more corporations discussions around the topic are going to become interesting: Each open source community has their own specifics of working together, even though all seem to follow pretty much the same style when looked at from very high above. I'm curious to what common values and patterns can be derived from that which work across organisations.

This evening Mary Poppendieck gave a presentation on Design Thinking in Berlin. "What does the US Military have in common with Apple?" In her talk Mary went into more detail about how increasing complexity calls for a need to rethink design. Slides are available online. Thanks for sharing them.

Mary started with a question for the audience on who in your company decides what is to be developed: Is it the PO? Is it a "them" or is it "us"? In a world of growing complexities and faster innovation cycles, thinking of them doing the design more and more becomes a huge business risk. It's time to start thinking of development teams as groups of people solving a business need - forgetting about the "that's none of my business" attitude.

Lets first dig deeper into the three steps of design (according to the book The design of design) the three steps of design are:

• Understanding the problem: Actually finding out what is the problem, what is to be designed is the hardest part. It's best done in a small team as opposed to by a single person.
  
• Design the solution: This step involves uncovering specific requirements and identifying alternative solutions - before actually being able to judge whether an alternative is a viable way of solving a given problem, the harder part is identifying possible alternative solutions.
  
• Implementing the design: This step must involve getting users and developers of the resulting system work closely together.
  

To come up with a truly successful design these three steps most likely need to be iterated as what we learn in later steps has to influence earlier steps.

When it comes to design, at traditional corporations there are various job titles for people involved with design. The graph distinguishes teams developing products vs. supporting internal processes. It makes a difference between software and hardware development. In addition there is a difference between people deciding on what to design (denoted in red) and those deciding how to develop (denoted in green). When looking at Scrum all of these roles are combined in the role of the product owner. When taking a closer look this seems to be a large over-simplification - especially when taking into account that the product owner in general is realized not as a role that has to be provided for but as a job description of one single person. It seems like this is too much work and too many hats for one single person.

In contrast Mary advocates a far more hollistic definition of the designers role: In current teams design is mostly being done up-front, in a very detailed way with results in the form of small user stories being handed over to the team afterwards. In contrast to that developers should be integrated in the design process itself - they need to participate, provide their input to come up with really innovative solutions.

Step one: Understanding the problem

The presentation went into more detail on three different approaches to solving the first step in design - that is understanding the problem. The first way was coined as the military approach. In the The operations process a chapter on design was added to cope with complex situations that do not match the expected configuration.



In such situations a combination of collaboration, dialog, critical thinking and creative thinking seems more important than blindly following command. According to the US military's procedures design again is an iterative three step process: From framing the problem, to experimenting and building prototypical solutions, to making sense of the situation and possibly going back to earlier steps based on what we have learned. One important feature of successful design teams is that the are composed of people with diverse backgrounds to not miss details about the situation.

The second approach to design is the Ethnographic approach. Mary showed a video introducing design company IDEO. The interesting take away for me was that again teams need to be diverse to see all details, that designers need not only find statistics on existing products but also go out to the customer using the product and listen to those "experts". The need to work under time constraints, work in a very focused manner, build prototypes - not to deploy them but to learn from them, merge them and throw away what does not work early in the process.

Coming from a data driven, Hadoop-y background, the third approach was the one most familiar to me: The data based approach focuses on finding success metrics and optimising those. When going back to the "Mythical man month" example of IBM building the IBM PCjr - the team spending 2 years developing a product that was a horrible failure, a product that was judged very badly in common it magazines: What could they have changed to do better? Well the solution almost is in the question: If there are people writing articles to judge products, people that have criteria for judging product success - why not optimise for their criteria? Another example would be Toyota developing the Lexus: Early in the process they decided to optimise for a top score on each of the car magazine reviews. Turned out though hard it was, it was not impossible. And lead to huge success on the sales side.

So instead of being caught in our little development bubble maybe it's time to think bigger - to think not only about what makes our project successful but to think about what makes business successful. And while at it - why not measure success or failure of a piece of software by how successful it is monetarily? This leads to a whole new way of thinking about development:

• Instead of a product roadmap we start thinking about a business model canvas: What metric gives us information on the business success of a new feature? How can we measure that? This can be as easy as click conversion rate. It's closely tied to the way you make money.
  
• Instead of a product vision we turn to measuring a product to market fit.
  
• Instead of a release plan we start thinking about the minimal viable product: What is the least you can do to generate more revenue?
  
• Instead of an on-site customer we start having a on-site developer. This is about getting developers out of the building, get them to talk to your users and get this experience back in. Note: To some extend this is even possible by looking over the user's shoulder by analysing your log files. Ever thought of taking a closer look at your search engines queries that return no results? Maybe instead of cursing the users that are too stupid to use your interface it's time to re-think your interface design or add more features to your search engine.
  
• Instead of iterations we have a loop of building - measuring - learning about our product.
  
• Instead of an iteration backlog at the end of each loop we have to decide about whether to preserve the current development direction or pivot it a bit.
  
• Instead of a backlog of things to do we have a list of things to learn about our users.
  
• Instead of detailed tiny user stories we have hypothesis of how our product is used that are validated or falsified by actual data.
  
• Instead of continuous integration you have continuous deployment that is much more interesting.
  
• Instead of acceptance tests we may have split tests, A/B tests or similar.
  
• Instead of a definition of done including only development it suddenly also comprises validating the data - did we actually convert more users?
  
• Instead of costumer feedback we are having a cohort-based metric, we start watching our users to learn - they are rarely willing to give explicit feedback, but their actions are very telling.
  
• Instead of a product owner we have entrepreneurs that have the business as a whole in mind.
  

Design a solution

Good industrial design is made up of ten principles: It's innovative, makes the product useful, understandable, aesthetic, unobtrusive, honest (does not fool users), long lasting, thorough through, environmentally friendly - and most important of all there is as few design in there as possible.

As one example Mary mentioned the flight booking site Hipmunk.com that gives users the chance to sort flights by least agony. It optimises for the best flying experience - not the best buying experience.

In the end it's all about building what the user needs - not selling what your build. It's about watching your users and learning about their needs.

Implementing design

One grave warning: Do not divorce design from development: Ford was a driver and racer himself, making him a great car designer. Wright flew the aircrafts he designed. Tom Edison thought through the whole process. Also in the tech industry the companies that are nowadays most successful were founded by people who are or at least were themselves very actively using the stuff they are building - think Bill Gates, Steve Wozniak and Steve Jobs, Sergey Brin and Larry Page. Takeaway is to do the job you are trying to automate, to have incremental development and delivery.

In the end it all boils down to how fast a learner you are: Not the most intelligent nor the strongest will survive. It's the one that is fastest to adapt to change that will be successful. One commonly used analogy for that is the so-called OODA cycle coming from John Boyd: The tighter your cycle of observing, orienting, deciding and acting is, the likelier you are to be successful among your competitors.

Speaking of release cycles Mary gave a very nice graph of various release cycle lengths.



In her experience teams that have deployment cycles of six months and above are likely to spend one third of their time hardening their software. If you are down to a quarter it's likely that the bottleneck is your business model. Maybe it is weird contracts or business partners expecting support for the exact version you shipped years ago. When you are down to a month per release you likely are running a software as a service model. As clarified later after a question from the audience for Mary this does include setups where customers basically buy a software including all future updates for a set period of time - including an automated roll-out. If you are down to daily or even just weekly releases you are likely running the product yourself. Mose likely then everyone is part of the team.

After her talk Mary answered various questions from the audience: According to her experience Scrum defines the team way too narrow by focusing on development only - with Scrum we are possibly getting perfect at doing the wrong thing very well - forgetting all about design and business.

One very general advise that leads to an interesting conclusion is to not waste your life, to not spend years of your life working on projects that are not needed by anyone. In the end this leads to realizing that failed projects are also caused by the developers not raising concerns and issues.

Mary is a huge proponent of trusting your team: Do not hand them detailed lists of user stories. You have hired great, senior people (or haven't you??) - give them an idea what you want and let them figure out the details by talking to the right people.

When asked for tools to support the lean startup process in a startup Mary emphasized the need for communication. Tools are likely to only slow you down: Get people to communicate, to sit at one table. Have a weekly or bi-weekly meeting to get everyone in sync. Otherwise let developers figure out their way to collaborate.

The talk was kindly hosted by Immobilien Scout, organised by agiliero with some tickets handed over to the Berlin Scrumtisch. Thanks guys.

The February Scrumtisch Berlin featured a talk by Lyssa Adkins, famously known for her publications on Coaching Agile teams. A mixture of fifty developers, scrum masters, coaches and product owner as well as one project manager followed Marion Eikmann's invitation. Thanks for organising the event, as well as thank you to Hypoport for providing the venue.

In her one-hour presentation she mainly focussed on two core topics: On the roles agile coaches have to fullfill as well as on the skill set needed by agile coaches. Being an agile coach (as well as being a scrum master, really), entails four very basic roles:

• Being a bulldozer - that is getting impediments out of the way. That can be as easy (or hard) as getting appropriate equipment for your developers or teaming up with the product owner to communicate with anyone trying to break the sprint.
  
• Being a servant leader - for a coach that means to work for the team, to ask the right questions and enable the team, to listen during dailies - instead of asking for additional reports: Most tools are already within the Scrum toolbox. The hard task is to identify them and use them correctly and efficiently.
  
• Being a shepherd - that may be as easy as getting people to attend the dailies, or as complex as communicating common values, a common goal.
  
• Being the guard of quality and performance - as a coach that means making degrading quality visible - and letting the Scrum team take the decision on how to deal with it.
  

However in reality each team is different, each sprint is different. So coaching really is similar to bing the river guide: Each trip is different. It is your task to make the team succeed and adapt to differing situations. To get to team to high performance - over and over again.

When talking about coaching what people need is a very specific skill set:

• To become a successful agile coach it helps to have coaching skills to be able to understand the client, to see their impediments and help the team become better by listening and asking powerful questions.
  
• Be a facilitator who organises sessions, meetings, conversations and may mediate in meetings.
  
• Have business domain expertise - that is to know process designs, figure out a companies strategy options.
  
• Be a great teacher to help people understand the basic concepts. That entails knowledge on course design, most likely design of exercises.
  
• Have mentoring skills to guide a team in the process.
  
• Know about lean and agile processes and stay up to date.
  
• Have the technical skills on what makes development successful, know the extreme programming techniques to help team excel.
  
• Have transformational skills - that is be familiar with change management, transformation leadership, knowing how to change organisations.
  

However being a coach remember that people are influenced not only by what you teach - but mostly by what you do. Most of what being a good coach means is invisible to the uninitiated outsider. It's about living the process you teach, staying true to the principles you want others to implement.

To get there it helps to get inspired, to talk with others in local meetups (just as with any coding practice: Try to find a common forum to exchange ideas and get fresh input). It may help to keep a value journal - not only to track your accomplishments and be able to prove them to higher management, but also to track where to improve yourself.

The talk provided several interesting starting points for exploring further. However with an added exercise sixty minutes covered only the very basic ideas. Especially for the skill sets needed to successfully coach teams it would be very interesting to learn more on what books to read or what courses to attend to learn more on each topic. Ultimately not only agile coaches benefit from being great teachers, mentors or facilitators.

Yesterday evening the Scrumtisch Berlin hosted a talk by Mary Poppendiek on Lean Development. Mary started the session with a talk on what lean development is all about and why it goes further than Scrum ever did. Some of the core principles she explained:

The first goal of every lean project should be to strive for customer satisfaction. The low hanging fruit is to do exactly what the customer wanted to have. The second step is to give features he took for granted as well - think performance, think extensibility. The ultimate goal should be to fulfill what the customer never knew he even wanted. Customers don't want software - they want their problems solved, the only way they are aware of to solve their problem is through software. Show them how to really solve their problems: Before the iPhone was invented, no one could have predicted what a great smart phone could look like.

Show technical excellence: Testing is not optional. To be a good developer you should always be able to prove that your code is correct. As proving the correctness of software in the mathematical sense is hard, if not impossible today, the recommendation was to use unit testing and integration testing as a poor man's proof for correctness. Without (automated) testing, any agile project is doomed to develop into a big ball of mud that cannot be easily extended.

The third principle is to deliver reliably, to design the system such that it meets its constraints. Constraints here are not only technical requirements - constraints include budget, time, features, stability and scalability.

The fourth observation was that no development process can fix the problems that Junior-level-only teams experience. Every lean organisation must provide for mentoring. The mentor's job is to climb the same path as the mentee and pull him along - instead of pushing. In contrast to many setups, not only should new hires be mentored, but basically every employee should have a mentor. Only that way learning - which is essential in current project (not only it) environments - is made possible.

A very interesting observation came from the audience: Every single advise given above is well known and appreciated by any reasonably good software engineer. Problem is: Why don't we adopt these rules in practice? The guy asking the question gave the answer himself: "The fish starts smelling from its head" (German proverb). So the goal of lean developers in the end would be to talk to their management: Do short iterations of a few weeks. Deliver early, deliver often, deliver in time. Usually that alone is enough to exceed expectations. Choose the best solution (both from business and technical perspective). Your results should speak for themselves to step by step change the way your organisation works.

Slides are online on the Scrumtisch blog.