Back To The Future

No. This blog entry isn't about the new game about the old movie with car-enabled time travel. Instead try to think back to late 1970's and early 1980's. Ready? Good.

Banks and other businesses with intrinsic need for computers were, at latest, building their information systems. CICS-systems with whopping 64 kilobytes of memory was pretty much the standard fare. Debugging meant printing out the core dump (after all it was ~64 000 characters, not that much). Used language was Cobol with static memory allocation. It's amazing to think it was possible to create large banking systems with that technology, isn't it?

What's more amazing is that many of these systems are still in use. If you use a cash dispenser (at least here in Finland), it's very likely that the transaction is eventually run in a CICS-system. Only companies that have been established after 1990's or so have more modern systems.

The question to ask, of course, is why haven't these antique systems been rewritten with Java, .Net or Haskell even? It's not from lack of trying, I assure you. I've personally witnessed a few very large projects and heard of many others. They were all failures by most standards. There never is one simple answer as to why a software project fails, but one question especially has haunted me for a while now: Why can't we seem to succeed in rewriting software that was originally done with such limited tools?

My theory about this is that back in the past, developers were able to concentrate better on the essential complexity of the software they were creating.

Commercial vendors have clouded us from the essential complexity of any given business-domain with process engines, portals, executable models, frameworks, predefined domains, predefined development processes...on and on. And the problem isn't just commercial vendors. Let's say you're going to use a very popular open source tool Grails to do your project (note, I have nothing againts Grails, I actually like it very much). Here's an incomplete list of techonlogies and frameworks you'll need to master in order to get a large project done:

Groovy, Java, JavaScript, XML, Ant, Gant, GORM, GSP, Spring core, Spring MVC, Hibernate, Log4j, HTML, jQuery, YUI, HQL, SQL, Ivy, Maven (repositories).

I used Grails as an example here because it's easy and productive compared to many other options. Still, the sad fact is that we're drowning in tools and frameworks. And as developers and architects, we've been conditioned into thinking we're helpless without them.

What many don't seem to grasp is that every framework, every library you add to a software stack adds accidental complexity which makes it more difficult to concentrate on the essential complexity. And no, the correct solution is NOT to start rolling your own framework. The solution I'm proposing is to minimize the use of frameworks and especially commercial products in an architecture.

I'll give an example: About a year ago I created a small ERP-solution from scratch. Sounds crazy, right? ERP's are monstrously complex and even starting to use one in a company takes huge amounts of effort. Well, you know what, it's not that complex when you can concentrate on the essential parts of it - the parts that matter to the problem at hand.

I ended up creating a very light (micro?) architecture where the domain was running on JPA-annotated classes in an embedded Jetty-server exposing REST/XML services using a small tool I created for creating XML-messages imperatively. The point is that the following code snippet (a whole service for returning all persons in the database) only requires Java 6 runtime, nothing else. No frameworks to learn or debug, no added complexity.

public void all() {
        ensureLogin();

        Message m = okResult();

        Query q = jpa().createQuery("select p from Person p");
        List persons = q.getResultList();

        for (Person person : persons) {
            m.set("person/@name", person.getWholeName());
            m.set("@id", person.getId());
            m.parent();
        }

        write(m);
    }

I'm not suggesting to stop using frameworks. I'm suggesting to evaluate very carefully whether they're worth the accidental complexity they bring. Do everything possible to minimize accidental complexity while providing developers tools to express the essential complexity!

And learn to make the difference between a tool and a framework. I will blog about tools vs. frameworks at some point. However I think my next blog will be about how (in my opinion) legacy systems could be rewritten successfully.