Why should I care about complexity?

While driving to work one morning, I remembered a discussion I had with a colleague maybe a year ago about complexity in software. Our discussion was about Fred Brooks' terms accidental complexity and essential complexity.

Accidental complexity is something we don't want. It's all the "fluff" that creeps into software projects: overly complex architectures, unneeded frameworks, "cool" code written by programmers, the list goes on and on. You'll notice that my definition is different from the original by Brooks'. That's because the original accidental complexity has been mostly solved (we don't write assembly anymore nor do we read core dumps printed on paper). Instead the modern accidental complexity is coming from the bazillion tools, frameworks and platforms all claiming to solve every software problem ever known.

Essential complexity is, well, essential. It is something that has to be solved in order to solve the problem at hand. It cannot be reduced, but it can be tackled. And more, it needs to be clearly visible so people are able to concentrate on solving it with minimum wasted effort.

I have a whole blog entry waiting to be written about how recreating old mainframe-systems these days seems impossible and how it relates to grown accidental complexity in software production, but today I'll address the connection of these two types of complexity and the role of a software architect.

Back to my old French car and my thoughts about our discussion. In a short moment of clarity, I realized what is the most important thing a (good) architect tries to do in every project:


Minimize accidental complexity and maximize the visibility of the essential complexity.


This might sound given or then consult-type jargon so let me explain. One very common reason why software projects fail (aside from plain bad salesmanship and contract negotiation) is that the essential complexity of the problem domain is either hidden completely or overshadowed by accidental complexity. Here's an example: A customer has bought a very expensive <insert your favorite commercial bloatware here> platform that will solve all their business needs - all that needs to be done is some customization, configuration and such. What follows is an often failed project where all of the time that should be spent tackling the essential complexity is spent instead in swimming in the accidental complexity of the given platform.

The tools to make essential complexity visible are well known. The most important is good domain-modeling of the problem domain. Another is projecting use cases on the domain-model to make it response to the concrete usage scenarios better. Nothing new or exciting here.

What's much more challenging is fighting the accidental complexity. This is where the value of an architect is weighed. Do you have the balls to say: "This framework/product is unnecessary and we won't include it in the architecture." Then you need the skills to keep programmers from producing code that is not essential to the problem at hand. This is where it's easiest to see the difference between a junior programmer and a seasoned one too. Seasoned programmers (if they've learned anything) solve the given problem with smaller amount of code - code that is clean, simple and without using unnecessary tools or frameworks.

Keeping accidental complexity at bay is often a matter of life and death for the software being created!

Liferay functional testing

Yesterday and today we've been having our first Fedex Day at Ambientia. The way we did this was to list ideas for it beforehand and then when the day started, we could group around the ideas freely.

I had submitted an idea regarding functional testing on Liferay and also ended up with it. So what to do about such a large topic in 24 hours? Well, first I googled the topic. Everyone including Lifreay itself is using Selenium, but I wanted something lighter, smaller and not dependent on external browser.

Initially I installed Canoo Webtest and started playing with it, but it felt a bit clunky and old fashioned with all the xml-files and rather weird runtime build on top of Ant. I had already decided to use Groovy for the tests since Canoo also supports it, but the documentation just wasn't there and the syntax was too ant-line also.

I've been toying with Grails Functional Testing Plugin recently and I've really enjoyed the simplicity and elegance of it. So I decided, what the hell, I have still almost half of our Fedex Day left so I rolled up my sleeves and started to port the grails plugin over to my Liferay-project.

After a while I had a stripped-down version of the grails plugin with all references to grails removed. Then I proceeded to make a very simple ant-script that compiles the groovy-files (we already have groovy-all.jar in our liferay project template so this was easy to do). Then I set up a directory called "funtional-tests" in the Liferay Plugins-SDK project and created the simplest possible test I could think of:

class SimpleTest extends functionaltestplugin.FunctionalTestCase {
  void testHelloWorld() {
    get('http://localhost:8080/')

    assertContentContains 'Hello World'
  }
}

Naturally it took some figuring out to get rid of weird errors, but soon it was working! Call me a nerd, but I think that's pretty cool.

Future plans

This is of course just a beginning. One big question is how to sandbox the functional test or the environment itself. Luckily I have a colleague here at Ambientia who is working on a small tool that can create pages, portlets and other data based on a simple xml-format. Using that we can setup and sandboxed environment where portlets can be tested functionally without having to ruin the actual portal you're building.

Also I'm thinking of trying to run Lifreay in Jetty embedded. This would make it possible (in theory at least) to control the whole environment while running the functional tests. I will try this anyway and report success or failure.

Another issue is controlling the state of the portlets themselves. What if you have a complex portlet done using Liferay's Service Builder that has it's own database and you want to run functional tests on it? I'm thinking maybe the edit-mode of portlets might help here. You might set up things like "create test data" and "destroy database" as simple buttons in the edit-mode and these wouldn't be too hard to call from the functional test. It could look something like this:

class ComplexTest extends functionaltestplugin.FunctionalTestCase {
  void testEditMode() {
    get('http://localhost:8080/home/portletXTestPage')

    click 'edit portlet x'

    click 'create test data'

    //run the actual test
  }
}

There is still lots to be done, but I'm quite optimistic about this.