This series of posts was first published at the Fortnox developer blog during 2014. The articles where collected, refinished and put on display here after Jonas moved on from Fortnox in 2015.

The series consists of five blog posts:

Software craftsmanship - Part 1 - Covers the main points of Agile, the original version.

Software craftsmanship - Part 2 - Cover the rest of the Agile manifesto as well as talking about TDD and testing in general.

Software craftsmanship - Part 3 - Rundown of software craftsmanship.

Software craftsmanship - Part 4 - Spending some time on the SOLID principles.

Software craftsmanship - Part 5 - Refactoring, with actual code and stuff …

There are many trends in software development. Some are new, others old. A few we invented our selves and others we borrowed from other disciplines. In this constantly mutating flora of “best practices” for the day and techniques with slower rises and falls there are a few that have stood the test of time. Techniques that have proven their power over the years, that add real production values to the process but still haven’t reached a level where they are widely taught. Neither in school nor in the workplace.

Intro

This is the first part in a series of blog posts that I’m going to write about Agile, Software craftsmanship and how to write better, maintainable software.

I’m going to talk about what’s called software craftsmanship. A collection of ideals, techniques and processes that help us build better, more maintainable software in less time and that produce a test suite, a more cohesive team and better customer relations as byproducts.

I will try to concentrate on the why’s as much as the how’s. Mostly this is an aspect that is missing, strangely, especially in introductory material such as this.

I hope you will enjoy this series and if you have any thought, questions or comments feel free to email me, I’m easy to find on the internet :) Let’s have at it!

/Jonas Schubert Erlandsson

Domain specific communication

When you went to school or just started working, did you ever get some small task, seemingly well defined. You eagerly, or not, went off to implement the solution and got some working code that you sent in for review. When you got the result back from your teacher or boss it turned out that you had failed!?

Why did this happen? It was a small task. It was limited in scope. You understood the problem and tried your best. Why did you fail? I can’t say for sure but there is a good chance that the specification was written in a language you were not proficient in but that looked like your everyday language.

The impact of domain languages when specifying tasks can be huge. In CS there are several problem domains you come in contact with that all use normal words for domain specific things and where the same word means different things in different domains. The same is true in business domains, words can have a well defined meaning in the domain that doesn’t match their normal definition.

Ultimately this is a knowing what you don’t know problem, you don’t know that you don’t know the right meaning so it’s impossible to correct for. It’s one of many errors that can happen in communication between parties in a development project, and it cuts both ways. How can your customer know if they actually understood what you proposed? They don’t.

Scope

Another limitation we have to contend with is a perceptional one. Humans have evolved in a world where we could get eaten by lions, attacked by another tribe, miss our footing and slip and a million other threats to the survival of our genes.

To combat this in a world full of colour and movement our brain has devoted significant resources to simplify the world for us so we can make decisions fast enough to stay alive. The view of the world we get is a highly filtered one. Anyone who has seen any of the experiments with the man in the gorilla suite in the background knows how it works.

This is another version of a problem where we don’t know what we don’t know, our brain threw it out for us. This ties in to our field when we talk about specifying a problem solution. We know that we always simplify the problem in our head, or our brain does it for us¹. It simply skips a lot of stuff. It tends to focus on the main path, ignoring the failure path, ignoring any special cases and ignoring a lot of lower level details.

All for the noble cause to help us make decisions quickly, so we do. The brain is also very optimistic and tend to ere on the side of “everything will go well”. It’s very difficult to convince yourself that you are wrong so when asked how long something will take we underestimate the complexity and then we give an optimistic estimate of the simplified task.

We give our project manager a best estimate on how long we think that feature will take based on the lies our brain told us. Is there any wonder that we are always late and notorious for not being able to estimate projects.

If you can convince yourself that we, and more specifically you, are this fallible then start multiplying all your estimates by pi. That will adjust for most of your ineptitude. It will also make you look like a third rate developer since all your colleagues will estimate the same task to a third of what you do…

Scale

So imagine that you have to write a spec for an entire system. Knowing what we know now how do we go about making this plan as close to reality as possible?

We can use planning systems, UML, brainstorming maps and whatever else you can think of. But short of actually writing at least pseudo code for the entire project down to method level, we will never get it right and even then there will be omissions and errors².

Setting the complexity aside we also have to do this during the sales phase of the project. The customer wants to know what our bid on the project is. So we don’t have any time to do all the detailed work, that would take person-months of work.

And when we finish our project proposal, full of perceptional blunders, underestimation and domain misunderstandings, we deliver it to the customer and they give us the project.

What are the odds that we will deliver the project on time and on budget? According to recent statistics about 5%, at best. And how likely is it that what we deliver is according to the original spec and that that is what the customer wanted? My guess, 0%.

Solution

So how do you go about solving a problem of this scale? It’s an epidemic of an entire industry so we need some systemic solution. Luckily there has been some proposals on this in the past. As early as the 70′s in fact³.

Product owners

You have certainly heard of the Agile movement and things like XP and Scrum. One of the main goals of this movement was to bring developers closer to the customer. If the customer and developers are in the same room during discussions about what to build you should get better communication, the thought went⁴.

And while this wont solve the domain language problem in itself it will reduce the chain of Chinese whispers up through the customers internals over to the consulting company through sales and project management down to the developers.

The basic principle is simple; the customer elects one person to be their representative and that person is available to the development team during any feature planning. This doesn’t have to be a seriously technical person, but some insight in the technology layer helps.

We call this person the product owner. They are the ones who selects what features should be implemented. They sign off on when things are done and act as a proxy between the requirements of the customer and the implementers of the consultancy.

Short iterations

The product owner idea will get you part of the way to the goal by shortening the information path and making the customer available and engaged throughout the project. But if the spec is nailed down in the beginning it wont help solve the problems of our and the customers limited cognition.

To solve this the agile methodology is to break the monolithic development process into short iterations called sprints⁵. A sprint is essentially a mini project. At the start the product owner chooses a few items from the larger projects backlog, a list of high level requirements for the product, and assigns them priorities.

The development teams then estimate how long they think each item will take and commit to finishing as certain amount of them, the rest will end up back on the backlog.

The sprint then begins and the selected features are implemented. At the end of the sprint the product owner gets a demo of the finished feature. At this point they can change or add whatever they like. The new requirements, changes and additions, end up on the backlog just like every other task in the project. The cycle then begins again.

A sprints length varies but it’s usually recommended to be 2-4 weeks. It’s enough time to finish some real work while not being so much as to allow for gross misestimations. Remember that we are really trying to minimise the cognition errors by developers and customers alike here. Allowing the customer to change their mind is just another way to help them since they rarely know what they really want until they have it or have something to evaluate.

Rounding off

So with two relatively simple techniques we have drastically reduced the risk in larger development projects. The domain language barrier has not gone away but we have reduced the cost of the miscommunications by only building in small increments.

We have also reduced the overall likelihood of project failure since we get a clear indication of where we are heading every 2-4 weeks. The customer gets a lot longer to evaluate the consultancy as a partner and the feasibility of the original brief now that they see how long things take.

As developers we get to work closer with the customer and see their emotional reactions to our work, good and bad. We get a deeper insight into the business values that drive the clients decisions at every sprint planning meeting.

And if worst comes to worst and we can’t finish on time and/or budget the client can always take what they already have and use it. Before agile even a 95% finished project would be of no use to the client since nothing worked. In an agile project the goal is that you should deliver a product at the end of each sprint. Maybe one with only partial functionality, but what functionality is there should work, be tested and documented. It should be done.

Next time

Next time we will look at some problems around design and productivity and lay out some solutions. It will be a dive into why you would want to consider pair programming and test driven development. It might also contain some strong opinions against “just testing” in favour of testing first. Until then, take care and refactor mercilessly.

Image courtesy of Alan Cleaver at Flickr under a Creative Commons - Attribution license.