Of TDD and Unit Testing

Inspired by an amazing talk by Greg Wilson with the topic, What We Actually Know About Software Development, and Why We Believe It's True, I will not accept truths about software development without citation anymore!

Its amazing to realize after doing software professionally soon for 13 years that the things we take as truths in this profession are often not much more than opinions.

So the first thing I decided to ask Google was: Is there proof that unit testing and TDD really results in better software? The most promising result was a research paper published by Boby George and Laurie Williams: An Initial Investigation of Test Driven Development in Industry.

The results seem to back up the claims of TDD (and by reference also unit testing, I'd assume). 18% increase in software quality with only 16% increase in development time compared to the control group (and the control group didn't mostly do any unit tests, even afterward so the increase in development time is probably skewed).

So based on this research I'd have to admit to being wrong about TDD. But...yes, of course I have a but. The researchers admit themselves that the software that was developed was trivial (200 LOC, a bowling game) with static completely known requirements. To me this is a very serious problem with the study. All that can be concluded is that TDD works in optimal situations.

Things that this study ignores are:

1. Granularity of unit tests. In large software there are multiple layers where tests could be written. Are we to assume always the lowest possible level (ie. class)? If not, what is the optimum granularity? 
2. Complexity and lability of requirements. How well would TDD approach work in high complexity work with shifting requirements where fast feedback cycles and ability to change are important (ie. software will never make it into production if correct requirements cannot be harvested via fast feedback loops)?
3. Software size. Unit test code is often overlooked in the size of the overall codebase. In large refactorings test refactoring often (in my personal experience) takes most of the time. Is there a limit after which test code amount starts to hamper development speed enough to matter?

Since I believe in functional, automated black box testing via interfaces or user interface, to me it would be much more interesting to compare software developed with rigorous TDD and unit tests to software developed with tests similar to what the researches in the TDD study themselves used to determine the quality of the solutions. More specifically it would be interesting to find answers to these questions:

1. Which approach yields faster development cycles with a mature code base and shifting requirements?
2. Which approach survives large and small refactorings better and how big difference there is between development effort when refactoring?
3. Is there a difference in overall software quality? 

Also what was a bit striking and surprisingly dogmatic for me was this quote from the paper: "The industry standard for coverage is in the range 80% to 90%,  although ideally the coverage should be 100%" (The paper quoted this from Steve Cornett).

If you've ever worked in a large real-life project of any significance, you'll realize how absurd this is. To get test coverage to 100% with white box unit tests requires enormous amounts of very fragile unit test suites basically casting your software in cement. Refactoring is of course always possible, but I would imagine the psychological barrier for doing a large refactoring in a system like this is very high indeed. 

I know what you're saying now. But isn't that just the point? That the unit tests give enough security to do refactorings? Sure, but don't black box functional tests do just the same? In fact they do much more. They make sure your software works identically from the point of the outside world, which is what matters.

I have zero research to back my hypothesis that a much better overall result could be achieved these ingredients:

• Automated black box functional tests for interfaces and user interface where test code to production code ratio is optimised (Another thing I'm surprised to never see mentioned).
• Unit tests for obvious hot spots in the codebase with difficult algorithms that lend themselves to be easily tested with unit tests.
• Minimising accidental complexity and CDD (Curriculum Driven Development) - this is slightly off topic, but I like to keep reminding.

In fact I find it very surprising that functional black box tests aren't among the first things to be done in a new project. Still I keep finding projects where people write 1000 line unit tests for single Wicket-components that themselves have half the amount of code. Yes, you'll find out if a change in the component breaks something, but you're still utterly clueless as to if your actual user interface works as expected as a whole. 

I'm starting to think there is a background in psychology here as well. Its about responsibility and limting it. "My job is to make this single component. I did it and wrote a unit test to prove it. Now I'm done, I don't need to think of the software as a whole."

That may sound cruel, but I have observed it to be true much more often than I'd like to admit.

I keep coming back to psychology so often these days that I'm starting to think of starting the study of Software Development Psychology :)

Tampere Goes Agile 2012

Honestly I haven't been taking part much in the finnish agile movement in the past years. Yesterday was an exception though as I went with a good fellow to the TGA yearly conference.

The place is at the middle of the old Finlayson factory complex and brought back memories form the time I first started studying in Tampere 13 years ago. The building where the conference was located was exactly the same where I attended business english classes :)

The theme of the conference this year was "building the right thing" which is a great topic that has really been missing from the agile mindset for too long. I've often found myself wondring amidst projects: "Why should we strive for 90% test coverage or perfect Scrum-process if its not granted that the original requirements are sane?".

I'm not going to go through all the talks of the conference, instead I'm just going to pick a few topics and talks that inspired me the most.

Ideas

Lots of ideas were floating around. Lots of talking about how to turn ideas into business. By far the best talk was from Marko Taipale. Marko is a very inspiring talker with loads of real life hands-on experience. Also it was Marko's talk that again verified my own thoughts of how common and bountiful ideas are. 

Everybody has ideas. Another talk given by Ralf Kruse really demonstrated this well. He managed to make the audience sputter out really cool ideas by the dozen during his speech that was more like a hands-on session on how to create the initial product backlog out of literally anything. He just picked up a random guy and demonstrated the daily routines in his life (an university student) and how really cool sounding business ideas can be spawned from just about any aspect of a persons life.

Being part of a small startup myself, I've had a lot of discussions about ideas. The traditional approach, and also some kind of natural instinct is to keep your ideas hidden. The reason for this is of course the fear that someone will steal your great idea. This, as I've come to realize, is nonsense. There are so many ideas everywhere that people and companies are drowning in them. 

Back to Marko's presentation that was basically about explaining how do you iterate from a set of business ideas expressed in a standard format to a viable, scalable business. He laid out a very clear and simple method to follow. I loved the clarity in his thinking. Of course its not new stuff. Most of this has been introduced in the Lean Startup movement, but as with agile all in all, its one thing to know stuff and completely another thing to apply that knowledge successfully in the real world.

To me the lesson I took home from this was that ideas are good, they are the fuel that a startup needs. But you should not get stuck with your ideas. And most importantly, the ideas need to be put out there and tested for viability. My favourite quote from Marko was (sorry, might not be quoted correctly): "The facts are not inside your building." 

As long as you just keep fiddling with stuff in your office, all you have are guesses. Personally I'm not down to coding stuff for months or a year just based on a guess. Not anymore at least.

Software artistry

This is a topic probably as old as software industry itself. I found Sam Aaron to be also an inspiring talker. He has basically (and very literally too) combined art and programming. He was a presentation with the title of "Hyper Agile" which made me chuckle. But basically his talk was about approaching programming from a old fashioned craftsmanship direction and building first your own tools that enable you to create beautiful and artful software for your personal pleasure.

I have mixed feelings about this. I have background in arts (music) and Sam's message appeals to that side of me. My pragmatic self wasn't that thrilled. The question that kept popping in my mind was: "How do you create larger software that requires coordinated efforts by a team or even several teams if every developer has their own "light saber" that is ever so slightly different from everybody else's?"

Still, I'm definitely going to try Sam's pet projects Emacs Live and Overtone he used to give a really cool live music performance with. Basically Emacs Live + Overtone seems to be a programmable sound interface that uses Clojure. 

And that brings us to...

Clojure

Now I know I've already written about Clojure shortly and basically dismissed it. However I'm always willing to change my opinions in the light of new evidence, which was kindly provided by the very talented guys at Metosin. These guys were also sponsoring the event and providing some really fancy coffee (being allergic to caffeine, I can't give a better description, sorry).

The guys from Metosin showed me some production quality Clojure code from a Real Project, which was a first for me. Looking at their stuff briefly I was pleasantly surprised. It looked actually readable and not just clever guys being too clever for their own good (as seems to be the standard with Scala). Very much respect for this.

There's also a video in YouTube from these guys giving a demonstration on how to do stuff with Clojure and Eclipse, its here. 

Soo, as for me, the jury is now back out to judge Clojure again. I installed a Clojure plugin to my IDEA and at least so far its looking rather nice. Still, I have to say that part of the reasoning to jump from Java to Clojure was based on arguments that are not that objective. 

Let me explain. If you state that Clojure is better because it doesn't have a 7-level architecture with relational database, OR-mapping, Spring, Wicket and tons of other frameworks, I would argue that you've just decided to use all this crap with Java, but that's not the fault of Java itself. I have written quite a bit about frameworks vs. tools before and all that applies here.

The problem with Java isn't the language in my opinion. It has always been the culture and the fact that huge corporations like IBM and Oracle and SAP have ruined Java for everybody without a very clear vision on what NOT to use when doing stuff with Java.

I'm in the process of writing a multi-part article on how Java can be used to create just as light and flexible architectures. Stay tuned for those! 

Still, I'm giving Clojure a go as I love to learn new stuff. You can expect a report on my adventures on this blog in the future!

Summary

I had tons of fun at TGA. The organisers were super cool and the speakers were all very good. Thanks for this amazing event!

Also shout outs to guys from Nitor Creations and all other colleagues I stumbled on. Looking forward to chances to work with all of you!

JVM languages and why I'd like Kotlin to succeed

Feeling a bit more technical today I decided to write a bit about JVM-languages, those that are not Java, of course.

Does anyone remember when .Net came out and Java-platform was said to be inferior as it doesn't support multiple VM languages like .Net does? In an odd twist of fate, it seems that JVM actually now is the only real polyglot virtual machine where the non-platform-native languages are actually in widespread use.

For quite some time now I've been in search of a Java better than Java. Without luck so far I have to admit. I doodle for a while with a new language and then eventually return to Java for reasons that are actually quite important for real-life development:

• Java has superior IDE-support, especially with Intellij IDEA
• In Java the language constructs are simple enough to be able to have really intelligent IDE support that makes the tools feel almost like magic sometimes
• The core libraries are, despite some ugly ones, very concise and high quality
• JPA
• Readability of the code
• Simple type system with static typing

There are many more things, but these are the ones that generally drive me back to Java.

Jython

In some ways Python is my favourite language syntax. Its very short, elegant yet still readable (if you write it properly) and very beginner friendly.

I have done some programming with Netbeans back when it had built in Jython/Python support and it was a very pleasant experience. 

Only once you start to get into things like XML-processing, concurrency, persistence and pretty much anything that requires you to do coding on non-core libraries or, god forbid frameworks, things start to get pretty bad pretty fast.  

This is of course mostly related to standalone Python, not Jython. 

At the end of the day what stops me from doing any large development with Jython/Python is lack of static typing. Its fine for scripts, but I'm just not willing to write the unit tests for all the things a compiler (IDE actually does most of the compile time checks these days so you don't even have to compile to get the warnings) will do for free.

Scala

After I gave up on Python/Jython, I found Scala. A statically typed language with actor model concurrency, implicit typing and functional goodies sounded amazing! I bought the green/white book and read it for weeks and I was very impressed.

Scala is a language, I feel, that caters for the computer science laboratory -types among us. It is a very academic language and I might imagine many of its proponents visit regularly on Lambda The Ultimate. 

I still think Scala is a great programming language. I just don't see much value of its added complexity in the development tasks most programmers face daily. The syntax is partly almost mathematical in notation and that way very short and concise, but also almost impossible to understand at a glance, which is a very important quality for any source code that will be read aside from writing.

To give an example, read this and explain within 15 seconds what this piece of code means:

type Out[C[_], B] = C[_ <:B] 

Also it has to be added that tools and frameworks for Scala don't really seem to be made with real life development in mind. My experience was that API's are broken without backwards compatibility without thinking twice and most libraries and frameworks have a two-level versioning meaning they have a separate version per each Scala version.

If I worked in CS lab, I might think Scala is great, but as I'm not, my advice is to steer clear from it.

Clojure

Lisp on JVM! I very much understand the appeal of Lisp and its a very good language to teach how programming languages work over all since its a programmable programming language itself.

But to do something serious with Clojure? No thanks.

Groovy

Groovy gets much less credit than its due in my opinion. Sure, its a dynamically (or should I say optionally) typed language with not that great runtime performance, but I've rarely been as productive as I was when doing a project with Grails.

I really like the simplicity of Groovy's syntax. The closures (lambda-expressions) are also simple to read and understand, although possibly not as complete as in Scala or Clojure.

If I had to pick non-Java JVM language to do my next web-project with, it would be Groovy and Grails. Still, Groovy doesn't satisfy my need of static typing and flawless IDE-support.

Jruby

I remember it was 2005, JAOO-conference and the creator of Ruby On Rails was giving a presentation of his brand spanking new framework himself. Everybody was touting Ruby as the next big language and I suppose it became pretty big.

I never understood why. I understand the idea of RoR, but I think Grails implemented it much better. Ruby as a language isn't much in my humble opinion.

I haven't followed the Ruby-camp for years now to be honest. But when I was looking into it a bit, I saw massive problems with runtime lacking basic scalability and concurrency features we take for granted on JVM. Also the attitude of Ruby-programmers left me perplexed. It was like looking to the backstage of amateur theater rehearsal.  

JRuby might be an ok idea, but not very interesting. I reserve to change my opinion of this given new evidence though.

Kotlin

I have high hopes for this language. For one its being created by JetBrains, the creator of IDEA, the best IDE available for Java (in my opinion). I think they have the correct approach for developing a new general purpose language. 

Freely quoted list of reasons why the language was initiated (from their FAQ):

• Java compatibility
• Complies as fast as Java (Scala for example compiles very slowly and has been reported as a problem in larger projects)
• Safer than Java (NPE's are gone forever),
• More concise than Java with type inference, closures, operator overloading and mixins, extension functions etc (but without going absolutely mad like Scala)
• Simpler than Scala (a huge advantage in my opinion)

With these foundations, I'm expecting a lot of this language and I hope it gets popular and also that they manage to reach the 1.0 milestone.

If there is one JVM language I see that could overtake Java in my daily work, its Kotlin.

Of introversion, Steve Jobs, offices and agile

What have Pixar, Tom DeMarco, Steve Jobs and 37Signals in common?

For one, they are all mentioned in a book I've been listening while driving lately: Quiet: The Power of Introverts in a World That Can't Stop Talking by Susan Cain.

Ok so what does her book have to do with software development, you may be asking. On the surface, not much. The book is about the differences between extroverted and introverted personality types. Introverted persons prefer to observe, think, analyse and generally enjoy solitude more than crowds.

As a sweeping generalisation it could be said that software engineers tend to be introverted, like myself for example. In fact as an even more sweeping generalisation it could be said that many people in occupations that benefit from creativity are populated by introverts.

Let's take me for example. I don't like socialising with people much. I hate large crowds and especially loud places where I have to raise my voice. I never talk just to hear my voice and I think meetings are mostly unnecessary venues for extroverted people to have audience for their monologues. Or arguments if there is more than one dominant person in the meeting. I'm mortally afraid of public speaking and doing things in front of an audience. To get work done optimally, I need peace and quiet and freedom.

Now imagine that someone asked me: "What would be the worst possible place to work you can imagine as an introverted person?".

Off the top of my head I came up with a list of these attributes:

• No privacy
• Having to share my working space with many other people
• No freedom to arrange my working environment
• Lots of obligatory meetings where there's no real discussion of things that matter to my personal work
• Demanded presence, but no freedom to engage in ad hoc conversations
• Noise that I can't avoid
• Interruptions I can't control

Sadly and not surprisingly I just ended up describing the average open office working environment most of us have to work these days. This is bad. Really bad. 

For a while in the beginning of 00s, it seemed like things were getting better in at least one regard: You could escape the horrendous office spaces and work remotely. Then the agile boom swept the industry and oh boy, look at what's being implemented by literally everybody, even the government IT-projects: mandatory 100% on-site presence. All the other fancy stuff of agile is just too complicated or too cumbersome to do, but hey, let's make the bastards all sit in 20 square meter space with absolutely no privacy and we'll reap the benefits of agile!

In all honesty, I'm avoiding high profile agile projects these days for this sole purpose. Or if I'm forced to, I'm quite inclined to simply break the rules and hope I will be forgiven because of how much more productive I am than other people in the team who obediently sit in less space than zoo animals are given and try to desperately get some privacy for actual work by listening to music with insulated headphones.

I would love to see a reform in the agile project culture that would relax the fundamental attitude regarding this. Linux kernel wasn't and isn't done by people sitting in the same room. World wide web wasn't created like that, in fact it was created just for the opposite purpose.

A very interesting subtopic of Cain's book is that creative teamwork, contrary to common belief, isn't more productive than working alone. The times when great results have been made, are most often when the team members have been able to work by themselves and then share and collaborate online through distance!  

And here, of course, come Pixar Headquarters and Steve Jobs into this story. If I had to work 100% of my working time at an office, that is where I would like to work. Pixar headquarters are usually mentioned because of the huge main atrium that provokes unplanned, ad hoc sharing of ideas. But just as important, probably more important, is the fact that besides this atrium where you can go and meet people when you feel like it, the creative employees have private space where they do the actual work. I think for many reasons, this is optimal and the closer you can get to this, the better.

But office space is really expensive! It is. So why not have just the atrium and some space for people who really want to be at the office most of the time? And for the saved renting costs, give a small budget to your employees to decorate their home office, buy them a good chair at home and a good desk. Then have one day a week when most people meet at the office atrium and have a good lunch. Let them share war stories from the projects they are working on, talk about new ideas. Anything, but making them listen and watch through a slideshow by a manager about manager stuff. 

But wasn't the point of being in the same room that you could just lift your head and ask a teammate if you wanted to know something? The funny thing is, my personal observation is that the physical distance of the two people have much less to do with this happening than the willingness of these two people to communicate in the first place. 

Here's where I wholeheartedly agree with 37signals founder Jason Fried. Use good online communication tools that enable passive communication. And then try to create a working atmosphere where this passive communication is encouraged. A good way to do this is to have a team so well spread to different locations that not one person involved feels like she's in a position to expect the others to come to her physically to share information. 

Oh I almost forgot about Tom DeMarco. I mentioned him because I've read his Peopleware -book many times. He's been saying this same thing since 1980s and we've only gotten worse. Working environment is crucial for people who do creative work, especially when they are introverts - like many people in creative arts and crafts are.

Positive effects of project crisis

A while ago I had the pleasure to be the lead developer/lead desginer of a mid-sized software project. When I entered the project, it had been running for about six months already with very high goals and the general attitude of "we're making the perfect thing here".

Time for a confession: I find projects like the above described distressing. Weird huh? Most programmers see these projects as the "dream jobs". Seemingly unlimited budget, finally a place to use all of the cool new frameworks and tricks you've seen all the rockstar developers use. Maybe even do some coding in Clojure, right? And when wearing my programmers hat tight in my head, I agree.

I'm sure there are happy endings for such fairy-tale projects, but my own experience says otherwise. The crash with business reality seems almost unavoidable. At some point the project's budget shows up on the radar of the evil executive who has no understanding of the beauty of Clojure nor how important test coverage of over 90% is. What he is thinking of is: "Ok, we're pouring a ton of money in that project, but the ROI is either unknown or fairly small, either the ROI needs to go up or the expenses go down, preferably both."

And so the budget cutting begins. I've experienced it many times now, but it always feels just as bad. Quite often the whole project is eliminated and possibly code worth months of hard work is thrown out basically meaning you could have spent the last months at home playing Skyrim.

So having been through this myself numerous times during my career (at some point I had a period of 3 years of development without a single line of code gone into production), I've become almost obsessive about avoiding this for the very simple reason that I really hate creating software that is going to end up in the garbage bin.

But back to the project I was leading a while ago. As you can probably guess by now, the budget was cut. Not completely though and with a chance that if the project could produce something of business value with the remaining budget and a really tight time-schedule, it might go into production. What us software engineers want is to get our babies in production so everybody in the team went to a kind of "lifeboat" mode where everything unnecessary was thrown away and all our efforts were focused on producing business value as soon as possible.

This is of course a very crude simplification of the situation to suit my needs for this blog entry. Against odds we succeeded and the project survived and went into production and the story continues, but the interesting bits are on the table now. So let's investigate a bit.

Essentially what happened was that the project's life was threatened, but with a chance to survive if real value could be produced quickly. As a result the project went through a metamorphosis and changed into a very focused ultra slim (yes, yes "lean" is the word to use, I know) creature that could produce business value fast and with a very small budget.

As a joke to a colleague working in the project I once said: "Maybe we should establish a software development method that would deliberately drive projects to the brick wall right in the beginning to get rid of all the fluff from the get-go". The thought was really hilarious at first, but its been bugging me ever since.

So here's what I'm going to do the next time I'm leading a project that's starting either from scratch or starting a new release: I'm going to conduct a workshop with the stakeholders after an initial backlog and roadmap have been produced and have them go through the exercise of "what if the budget is cut in half" and "what if the budget is reduced by 75%". The remaining items on the backlog are the ones that will be done. And with the correct focus from the beginning, its possible to have the desired code quality because there will be less code to begin with.

I suppose you could call the exercise an "emergency evacuation rehearsal" or something like that. In essence its trying to defend from Parkinson's law, which in software development could be expressed as:

Software project always expands so as to fill the budget available for its production

Because let's face it, "Curriculum Driven Development" is a large problem in our industry. And even when that isn't the case, without proper focus even developers with good intentions will produce loads of unnecessary features and code.

The best thing a software project can produce is valuable features, but the second best thing is to avoid producing features that aren't valuable. Let's rephrase this into a "law" of mine that I try to live by:

Best code is unwritten code

Positive Software Development

A few days ago I watched a video by Shawn Achor. Its about positivity and how it can affect our productivity. So that got me thinking: How could I apply this in my work as a software engineer? Wait a minute! I am already!

It's really nice to get scientific support for things you've been doing intuitively until now, isn't it? What I've been doing intuitively while leading software development teams is to try and make the team relaxed and positive. It's not always easy and it doesn't always succeed. But when it does succeed, the result is loads of fun, productivity and friendship.

So how does it work actually? It might help to think of software development as a learning process. Here's a quote from Wikipedia article on Lean software development:

Software development is a continuous learning process with the additional challenge of development teams and end product sizes. The best approach for improving a software development environment is to amplify learning.

And how do we amplify learning? In his speech, Shawn Achor lists the following things:

• 3 gratitudes (3 positive perceptions every day)
• Journaling (going through one positive event of the past day)
• Exercise 
• Meditation
• Random acts of kindness

Ok so many of those are something you as a team leader can't do much about. The last one is really important tool for a team leader though. Giving random, peer-to-peer positive feedback makes people generally feel good about themselves. Your feedback does have to be down to earth and honest though. People can tell - at least highly intelligent people you work with - when you're like a kindergarten teacher. For this to work you need to understand what your team is doing. This is also why praise from a manager usually just makes the team uneasy.

What else can you do? Apparently there's a consensus in the academia that fear of failure is a big obstacle when it comes to learning. See Andrew Martin's work for example. I have an in-built high fear of failure and I'm sensitive to what other people think of me so naturally I always try to do everything I can to alleviate this. 

Software developers are for some reason often unforgiving when it comes to making mistakes. We're doing possibly one of the most complex things us humans are able to do with a staggering amount of variables and forces that can affect the outcome. Yet I often meet attitudes of "I don't make mistakes, ever, and I don't expect anyone else to do them either". What's terrible about this is that its impossible not to make mistakes when developing complex software systems. And in an environment that sneers on people who make mistakes, the mistakes are of course hidden resulting in crippling technical debt and loss of productivity.

In my opinion its almost the duty of a good team lead to set an example of failing fast and failing positively. With this I mean that mistakes and failures should be treated positively as chances to learn together and even more importantly, get rid of the fear of failure. When I work as a team lead, I always bring out my mistakes in the open, make sure everyone understands that I'm not perfect and that I don't expect anyone else to be either. That its ok to make mistakes and show them to others so they can learn from them as well.  My habit is to make jokes of my mistakes because, let's face it, if you're not able to laugh off some of the absurdity of our work, you're going to end up insane.

The funny thing is, I've sometimes been accused of being "unprofessional" when I'm trying to create a relaxed working atmosphere with laughter and permission to make mistakes (as long as you fix them, of course). You can safely ignore this. It's the 21st century and its ok to have fun at work, especially as its now scientifically proven that its more productive that way!