Software Craft Blog

Global Day of Coderetreat 2014

2014-11-28T00:00:00+00:00

Saturday November 15 set the stage for Global Day of Coderetreat 2014. On this day developers from all over the world joined together to improve their craft.

Together with Stefan Hendriks I facilitated the event in Utrecht, The Netherlands. In this post I sum up the day and provide a few extra links for the participants at out location. Special thanks to Stefan for proofreading this post.

I have to admit, after the Coderetreat we did for 45 people in September this year, having 13 attendees for Global Day is really relaxing and fun! Much less intensive on the facilitators, so we even got to coding ourselves :)

Introduction

At every Coderetreat we start with a short introduction why we facilitate Coderetreats, and briefly discuss the 4 rules of simple design. Also we ask attendees to quickly introduce themselves, what languages they are interested in doing and what languages they are proficient in. This helps people in chosing their pair partners.

Each session is about an hour; we code for 45 minutes, and then have a little retrospective. The kata used in these sessions is Conway’s Game of Life. This is a great fit for the Coderetreat format because everyone can understand it, yet there are a lot of interesting aspects to coding it up. Also it is hard to finish the full kata in only 45 minutes, which helps people to refrain from the pressure of finishing it. Coderetreat is not at all about finishing a kata, it’s about practice.

In every session different constraints are applied to the way people can program. I’ll explain the constraints we used below. Remember that there are a lot more constraints, we don’t use the same ones on every coderetreat. This keeps the sessions interesting, also for coderetreat veterans.

If you’re interested in learning more about the format, please see coderetreat.org. And of course feel free to drop me a question.

Session 1: Warming Up

The first session is to get familiar with the problem domain, and to get everyone’s environment up and running.

On a Coderetreat you don’t need much; your favorite language and a single assert_true function will do. Sometimes I am still amazed at how much effort people want to put into setting up “the perfect” test library.

Session 2: No Conditionals

Programming without using a form of conditionals is tough. This is one of my favorite constraints, especially to start with. It might be mind breaking, and truly opens your mind to find other ways of coding.

When I attended a Coderetreat given by Corey Haines he explained this constrained as being about polymorphism. Conditionals are procedural polymorphism. You can substitute conditionals with either inheritance (in Object Oriented languages) or polymorphic types and function parameters (in Functional languages). Explaining the constraint like so helps people to grow a better understanding and gives them some direction. It turns out I’m not alone on this, Cory Foy wrote a blog post about this on the 8th Light blog.

The common feedback, as usual, was that it is really hard not to use if-statements and to start thinking completely different. Most pairs started finding solutions late in the session.

If you’re truly fanatic about not using if-statement, perhaps the Anti-IF Campaign is for you.

Session 3: Functional Core, Imperative Shell ala Gary Bernhardt

After lunch we first watched Gary Bernhardt’s Boundaries talk. As the topic is relatively unknown. And what’s a better introduction than the original talk anyway?! After watching the talk we shared some thoughts on it and then started the session.

In case you haven’t watched the talk, in a nutshell this constraint comes down to the following. The Functional Core is about keeping all your objects immutable. The imperative shell is a thin layer that manages state (hence is imperative) using the functional core.

The common feedback was that using immutable objects helped understanding and testing the code.

If you’re interested in learning more on this topic, I highly recommend watching Gary’s screencast. As Gary shows in Ruby you don’t have to use a fully functional language to apply some of the concepts. Jessica Kerr’s talk Functional Principles for Object-Oriented Developers explains some functional principles through Java and C# code samples.

Session 4: Mute Pairs with Evil Coder

Another one of my favorite constraints is the mute session. By not allowing pairs to talk and to only communicate through code (not comments!) we simulate pairing with future-you, say in half a year from now. How understandable is the test you write, does it actually express intent the way you think it does?

Add the evil coder role and you’ll see where your tests can be more strict. The goal of the evil coder is to find loop holes in the tests and try to push the implementation in a different direction.

Closing Circle

The overal mood in the closing circle was that we had a great day. I couldn’t agree more. The group got more intimate throughout the day, and got a lot more open. What surprised me was how eager this group was.

Again I met some great people, and had an awesome time. Thank you all for that!

“Software is eating the world”

Throughout the day we mentioned Erik Meijer’s recent talk Software is eating the world. And the One Hacker Way! response of Uncle Bob (Robert C. Martin).

Moved to Github Pages

2014-08-09T00:00:00+00:00

As of this post I moved my blog to Github Pages from Google’s Blogger. As a consequence I built this blog from scratch. As a frequent reader you may notice some changes, hopefully all to your liking.

Github Pages uses Jekyll to serve content. Jekyll generates a static site from your articles (I write these in markdown) and templates. My main goal was to move my blog to Github Pages, with all articles and a similar layout. Of course on the way I came up with a ton more things I could do (now that I entangled myself from the constraints of Blogger). Most didn’t make it because they were not important to move the blog. It’s interesting to be your own customer.

Even comments were not important enough to me. I do much appreciate your comments. It’s just that there are great alternatives such as Reddit, rendering my own comments feature unimportant. As a result, comments from my old blog are not available at this time. Sorry about that. I might support commenting later and move existing comments (if the tools allow me to). Of course you can also contact me directly.

Github Pages

So why Github Pages?

Well, I want to write my blogs in text using a decent editor. Hence I don’t use rich text formatting in an editor. Rich text editors are broken and it highly unlikely for them to be be fixed anytime soon. For me the Blogger editor caused all kinds of issues, from data loss to broken layouts (because the HTML generator changed again). Others (like WordPress) are just as bad.

Then I also wanted to be able to build some decent templates. Believe it or not, my Blogger template counted almost 2400 lines. At this point all my templates and css combined counts about 400 lines. As a result it’s easy to change templates and add new stuff. Also I now have full control over the HTML, which is alright with me.

Starting a new site with Jekyll was easy and writing the initial templates was fun. As I write posts in markdown it caused some trouble to set attribute values on tags (like rel=nofollow on ancors). I ended up using the Kramdown markdown engine, which supports most of the things I need.

But then there comes that point where you have to start migrating content. I could grab the HTML of my articles easily from the atom feed. Then I had to convert that into the markdown I want. That is not fun to do. So rather than converting all of it myself I hacked up a Vim script with a bunch of regex substitutions to do most of the tedious conversion stuff for me.

So far I pretty much enjoyed Jekyll. It has built-in support for things like pagination and code highlighting. It is kind of a do-it-yourself tool, which is a good thing. It keeps things simple. Yet the powerful templates make many things possible (like creating your own atom feed, as Jekyll doesn’t come with one).

A bit of a challenge when moving existing content is to keep the exact same URL structure. Normally I would 301 redirect the old URLs to the new structure. Given the nature of Jekyll this is not possible. The use of 301 redirects (in my case) is mainly important to preserve ranking in a search engine. Luckily Google (and likely others) also treat a meta-refresh with a canonical link like a 301.

All in all this was an interesting experience. For any static site I now would consider Jekyll first, especially if content needs not to be written in a rich text editor. And for many dynamic features of a site you can always use Javascript and cloud service.

Java's Volatile Modifier

2014-07-15T00:00:00+00:00

A while ago I wrote a Java servlet Filter that loads configuration in its init function (based on a parameter from web.xml). The filter’s configuration is cached in a private field. I set the volatile modifier on the field.

When I later checked the company Sonar to see if it found any warnings or issues in the code I was a bit surprised to learn that there was a violation on the use of volatile. The explanation read:

Use of the keyword ‘volatile’ is generally used to fine tune a Java application, and therefore, requires a good expertise of the Java Memory Model. Moreover, its range of action is somewhat misknown. Therefore, the volatile keyword should not be used for maintenance purpose and portability.

I would agree that volatile is misknown by many Java programmers. For some even unknown. Not only because it’s never used much in the first place, but also because it’s definition changed since Java 1.5.

Let me get back to this Sonar violation in a bit and first explain what volatile means in Java 1.5 and up (until Java 1.8 at the time of writing).

What is Volatile?

While the volatile modifier itself comes from C, it has a completely different meaning in Java. This may not help in growing an understanding of it, googling for volatile could lead to different results. Let’s take a quick side step and see what volatile means in C first.

In the C language the compiler ordinarily assumes that variables cannot change value by themselves. While this makes sense as default behavior, sometimes a variable may represent a location that can be changed (like a hardware register). Using a volatile variable instructs the compiler not to apply these optimizations.

Back to Java. The meaning of volatile in C would be useless in Java. The JVM uses native libraries to interact with the OS and hardware. Further more, it is simply impossible to point Java variables to specific addresses, so variables actually won’t change value by themselves.

However, the value of variables on the JVM can be changed by different threads. By default the compiler assumes that variables won’t change in other threads. Hence it can apply optimizations such as reordering memory operations and caching the variable in a CPU register. Using a volatile variable instructs the compiler not to apply these optimizations. This guarantees that a reading thread always reads the variable from memory (or from a shared cache), never from a local cache.

Atomicity

Further more on a 32 bit JVM volatile makes writes to a 64 bit variable atomic (like long or double). To write a variable the JVM instructs the CPU to write an operand to a position in memory. When using the 32 bit instruction set, what if the size of a variable is 64 bits? Obviously the variable must be written with two instructions, 32 bits at a time.

In multi-threaded scenarios another thread may read the variable half way through the write. At that point only first half of the variable is written. This race-condition is prevented by volatile, effectively making writes to 64 bit variables atomic on 32 bit architectures.

Note that above I talked about writes not updates. Using volatile won’t make updates atomic. E.g. ++i when i is volatile would read the value of i from the heap or L3 cache into a local register, inc that register, and write the register back into the shared location of i. In between reading and writing i it might be changed by another thread. Placing a lock around the read and write instructions makes the update atomic. Or better, use non-blocking instructions from the atomic variable classes in the concurrent.atomic package.

Side Effect

A volatile variable also has a side effect in memory visibility. Not just changes to the volatile variable are visible to other threads, but also any side effects of the code that led up to the change are visible when a thread reads a volatile variable. Or more formally, a volatile variable establishes a happens-before relationship with subsequent reads of that variable.

I.e. from the perspective of memory visibility writing a volatile variable effectively is like exiting a synchronized block and reading a volatile variable like entering one.

Choosing Volatile

Back to my use of volatile to initialize a configuration once and cache it in a private field.

Up to now I believe the best way to ensure visibility of this field to all threads is to use volatile. I could have used AtomicReference instead. Since the field is only written once (after construction, hence it cannot be final) atomic variables communicate the wrong intent. I don’t want to make updates atomic, I want to make the cache visible to all threads. And for what it’s worth, the atomic classes use volatile too.

Thoughts on this Sonar Rule

Now that we’ve seen what volatile means in Java, let’s talk a bit more about this Sonar rule.

In my opinion this rule is one of the flaws in configurations of tools like Sonar. Using volatile can be a really good thing to do, if you need shared (mutable) state across threads. Sure thing you must keep this to a minimum. But the consequence of this rule is that people who don’t understand what volatile is follow the recommendation to not use volatile. If they remove the modifier effectively they introduce a race-condition.

I do think it’s a good idea to automatically raise red flags when misknown or dangerous language features are used. But maybe this is only a good idea when there are better alternatives to solve the same line of problems. In this case, volatile has no such alternative.

Note that in no way this is intended as a rant against Sonar. However I do think that people should select a set of rules that they find important to apply, rather than embracing default configurations. I find the idea to use rules that are enabled by default, a bit naive. There’s an extremely high probability that your project is not the one that tool maintainers had in mind when picking their standard configuration.

Furthermore I believe that as you encounter a language feature that you don’t know, you should learn about it. As you learn about it you can decide if there are better alternatives.

Java Concurrency in Practice

The de facto standard book about concurrency in the JVM is Java Concurrency in Practice by Brain Goetz. It explains the various aspects of concurrency in several levels of detail. If you use any form of concurrency in Java (or impure Scala) make sure you at least read the former three chapters of this brilliant book to get a decent high-level understanding of the matter.

Book Review: Understanding the 4 Rules of Simple Design

2014-04-17T00:00:00+00:00

It’s about time that a book is written about the 4 rules of simple design. These rules are possibly the most powerful yet least understood software design practices out there.

TL;DR

If you care about your software being easy to adapt to changing requirements and a continuously evolving environment, you should probably know about the 4 rules of simple design. Understanding the 4 Rules of Simple Design helps you grow a decent understanding of these rules. Or at least enough so that you can practice them yourself (at the next Coderetreat, maybe). So, go buy this wonderful book.

Slightly Longer Review

The 4 rules of simple design were originally codified by Kent Beck in the late 90’s. Kent Beck also writes about these rules in his book Extreme Programming Explained. The rules that will lead you to a simple design are

Tests pass
Express intent
DRY
Small

By themselves they are simple rules. Maybe this is why I find people underestimate them. But following these 4 simple rules does lead to better designs. That’s what makes them so interesting. The rules of simple design aren’t usually taught at trainings. The only training format, that I know of, that practices these rules is Coderetreat.

The author of Understanding the 4 Rules of Simple Design, Corey Haines, is to Coderetreat what Kent Beck is to Extreme Programming. Corey is like the father of Coderetreat. The past 5 years he has traveled the world to give trainings using the Coderetreat format. If someone can write a book about the rules of simple design, no doubt it’s him.

In the introductory part of the book Corey introduces himself, the Coderetreat format, and explains what Coderetreat is about and why it matters. He then discusses the 4 rules and explains what they mean.

These introductions lead to the examples chapter. This is the larger part of the book. The examples are based on the patterns that Corey noticed while facilitating Coderetreat sessions. They include how test names should influence the object’s API, what DRY actually means, how to replace procedural polymorphism, and much more. For each example Corey presents the case and then leads you through the thought process of improving the design.

Of course there are many other design principles that should not be forgotten. Some of these, the SOLID principles and Law of Demeter, are explained in this book as well. Corey mentions how focussing on the 4 rules of simple design naturally leads to satisfying most of the SOLID principles as well. I share this experience.

In only a hundred pages Corey has managed to describe the 4 rules of simple design with some great examples that no doubt show you the true power of these 4 simple rules.

However, simple design is hard and after reading this book everything won’t just fit like magic. Corey has done an excellent job writing this book. Software design is hard and in my experience the way to get better at is through practice. Reading this book is an excellent start,

The style of writing makes this book easy to read, from cover to cover. If you like a preview of some of Corey’s writing you should definitely read his blog post Some thoughts on pair-programming styles.

Completed with a further reading list, this book is most definitely worth it’s money. As it’s published through Leanpub you can set a price yourself. Although you really shouldn’t want to pay less than $20.

I would definitely recommend this book to programmers with all sorts of experience; from beginner to expert, from apprentice to master craftsman.

For a little critical note, I found it a bit disturbing that the first release didn’t include a picture of Zak the cat. It’s awesome that he had fixed that, so this wonderful book now comes with Corey’s trademark! ;-)

Disclaimer: what I’ve written here is my own personal opinion. I do not have any stakes in selling this book. I don’t benefit financially from Corey selling more copies, nor books sold to visitors of this blog.

Method References in Ruby

2014-04-10T00:00:00+00:00

Today I was writing some Ruby code for filtering arrays when I recalled that Ruby has a shorthand notation to pass a method by its name (as a symbol). I’m writing this post to help me remember, while I figure out how it really works.

Say, for example, that we need to select the odd numbers from a sequence. Of course we could pass a block to do that.

(1..9).select { |n| n.odd? } #=> [1, 3, 5, 7, 9]

But we can also pass a symbol, prefixed with an ampersand. While not actually accurate (in a few minutes you’ll see why), this is what I call a method reference.

(1..9).select &:odd? #=> [1, 3, 5, 7, 9]

Personally I prefer the latter form as it’s a better representation of the essence that I want my code to do: from range 1 to 9 select only odd numbers.

How does it work?

When Googling for information on the ampersand unary operator in Ruby, I first found a few blog posts and Stack Overflow answers that told me that & calls to_proc on a given object. That would be convenient, although it implies that this would probably only work for methods that accept both a block or a proc. Let’s worry about that later.

First try and see if this works. If & would call to_proc on the symbol I should be able to pass a proc myself, too.

(1..9).select :foo?.to_proc
ArgumentError: wrong number of arguments (1 for 0)

Whoops! That didn’t work too well. What’s even more interesting is that if I just put the ampersand in front, the code runs.

(1..9).select &:foo?.to_proc #=> [1, 3, 5, 7, 9]

So maybe these first hits weren’t actually correct about things. At this point it seems more likely that ampersand turns a proc into a block. And given any object, it will get a proc by calling to_proc first.

To see how my new theory works out I use a custom object that answers to to_proc. Because Ruby’s blocks are special things that cannot be created or assigned to variables, I cannot simply capture the resulting value of &:foo. Let’s give it a try.

odds = Object.new
def odds.to_proc() ->(n) { n.odd? } end

(1..9).select &odds #=> [1, 3, 5, 7, 9]

Awesome! However, this doesn’t really confirm that the ampersand in fact turns the proc into block, only that actually to_proc is called on the object.

To check that the ampersand turns the proc into a block I use a function that yields.

def yielder(n) yield n end

yielder 1, &odds #=> true
yielder 2, &odds #=> false

While passing a proc obviously won’t work.

yielder 1, odds.to_proc
ArgumentError: wrong number of arguments (2 for 1)

Tentatively my conclusion is that the ampersand unary operator takes a duck-type of a proc (i.e. an object that responds to to_proc) and returns a block. If you see any flaws in this, don’t hesitate to let me know.

Is your Scrum team Agile?

2014-04-03T00:00:00+00:00

Many software development teams use Scrum. Indeed Scrum is a highly popular process. In this post I talk about some of the problems I’ve noticed in some Scrum teams. I don’t believe these problems are related to Scrum, but are a direct consequence of the mindset of these teams.

To stop this from becoming a No True Scotsman story, I’ll talk about the way I’ve seen how some teams have adopted Scrum.

Of the 12 practices the Agile Manifesto is built on, Scrum is mainly about the planning activities. This doesn’t mean the other practices aren’t valuable in Scrum teams, but it’s not the main focus. Most Scrum teams use the planning game for each sprint, do daily stand-ups, use small releases, and have a customer on-site. That should make them agile, right?

Agile Manifesto

Let’s take a look at the Principles behind the Agile Manifesto. The number one priority is “to satisfy the customer through early and continuous delivery of valuable software”. Closely followed by the importance to (even late in development) embrace changing requirements and the notion that working software is the primary measure of progress.

While valuable software obviously is highly important, I think the key is in the early and continuous delivery of software. We add value to software by changing it and extending it. The planning game will help you towards that road. But then what?

This is where the rest of the agile practices pop in. Practices like automation of acceptance tests, TDD, pair programming, simple design and refactoring (not in any specific order). These are practices that actively guide you to more continuous delivery.

Keeping up

That Scrum itself focuses on planning aspects doesn’t mean that these other practices aren’t valuable in Scrum teams. It’s just not the main focus of Scrum. In fact, I wonder how well teams can ever keep up the agile principles when they only use planning. I’ve seen teams that respond to new requirements by demanding some refactor sprint to “prepare” for what’s coming.

I’ve been at such a team. One sprint turned into two sprints. Our biggest problem to me was that we’d only knew if we screwed up when QA told us so. It was seen as the job of QA to tell whether the developers made a mistake. Oh yes, we were a whole team. We had an on-site customer. We had QA inside the team. We did daily stand-ups. We did planning sessions for each sprint. But we didn’t use the other agile practices.

Now I won’t say that it’s impossible for teams to keep up their delivery of valuable software without following these practices. But I’ve never seen it work well. I wonder how it could. I mean, without simple design and constantly keeping the code clean, how well can code be changed, even in a few months from creating it? What raises a flag for a broken feature when lacking stable unit and acceptance test suites?

Scrum

It’s not fair to blame any of this this on Scrum. Scrum works well in a wide variety of teams and organizations.

Scrum mostly embodies the planning and management rules of Extreme Programming (XP). I believe it’s thanks to Scrum that much of the agile planning and management practices have made their way into mainstream today.

It’s just because Scrum doesn’t include the other Agile practices many folks doing Scrum think that those are somehow not valuable. The most successful teams I’ve seen are all doing most, if not all, of the other Agile practices as well. This is also totally possible for a Scrum team.

You’re mileage may vary

Over the past years I’ve been experimenting with other ways to write software. This is especially hard without TDD and simple design. While it’s good to experiment, every time I got back to the agile practices as they work best for me.

You’re mileage may vary, obviously. If you use different practices that even better support the continuous delivery of valuable software I would love to hear about them and give them a try myself.

Lineman.js

2014-03-15T00:00:00+00:00

I think it’s fair to say that JavaScript is no longer that “thing” we used to enhance our otherwise static web apps by adding some dynamic elements to it. Most users use a JavaScript enabled browser and users that lack JavaScript support are simply outnumbered. Without JavaScript you cannot use a large number of online services and this number is growing.

From time to time I’ve been playing around with JavaScript frameworks like Angular and Backbone. When you want to start an app with any of these frameworks - or in fact, any JavaScript app for that matter - you’ll quickly run into tools like Grunt and Bower. If you don’t know what they are, go ahead an see what they’re about. I’ll be here, waiting patiently for your return. On the other hand, you don’t need full knowledge of these tools, so you might as well read on.

The problem with starting an app like this is that it’s pretty harsh to set up. You may need configuration for compiling CoffeeScript and Less, a unit test library, something to run the tests, minimize JavaScript, run JSLint. The list may seem almost infinite.

Enter Lineman

Lineman is a thin wrapper around tools like Grunt and Test’em to take away the initial configuration burden from developers and let them focus on what they love most: building great apps.

Justin Searls, the creator of Lineman did a brilliant talk on what’s been wrong for years with building JavaScript apps. In this talk he gives an overview of some of the major problems you face with JavaScript development and how Lineman helps out. I’d recommend you to watch his talk.

The difference with Yeoman (a competing tool) is that Lineman is not based on code generation. Where Yeoman generates your initial project with a full Grunt configuration and example code (which you don’t need) and some other stuff, Lineman installs itself as your Grunt configuration. The benefit of this is that you can update your tools without breaking the configuration (or manually updating it).

With Lineman all you need to do to start building a new app is tell it to create one.

$ lineman new "app-name"

That’s it. It provides much like a walking skeleton for your application that you can run and test. Immediately you can enter your regular build cycle of changing code and have it automatically built and tested each time you save a file. If that’s not all, it’s blazingly fast. One of the goals the creators of Lineman set is for each incremental build and test run to take no more than a few hundred millis, independent of the size of the codebase. Now if that doesn’t stimulate TDD I don’t know what will.

To start the build cycle all you need to do is to fire up lineman run in one shell and lineman spec in another.

Both of these watch your sources so your app is build and tested whenever you save a change. You need to keep both running because lineman run compiles your sources and lineman spec watches the built files for running your specs. It took me a little help to find this out (much thanks to Justin for helping out).

Framework support

So what about framework support? Lineman creates a vanilla flavored app. This is the walking skeleton, but includes nothing like the framework support you can get from Yeoman.

Personally this is the way I prefer to start an app. Any libraries and frameworks I need I will include myself. Not only gives this full control to what vendor libs I actually use, it also gives me full control to when I want to use them and gives me the opportunity to defer their inclusion if I don’t need them (yet).

If you like to have some popular framework already included that’s okay, Lineman offers that too. All you need to do is to clone one of the Lineman template projects from Github and you’ll get a walking skeleton app with that framework. At the moment templates are provided for Backbone, Ember, and Angular, for building a library or a blog.

What’s next?

At this point I just want to start building things. The lineman website covers a lot of the configuration that’s possible.

Next to build and test configuration one of the first things I need in most apps is to be able to run end-to-end tests. I suspect Lineman can help me out with this too.

I hope I’ll keep enjoying Lineman as much as I do in the first few hours of using it.

Fast Remote Service Tests

2014-02-25T00:00:00+00:00

Testing code that interacts with remote services is often pretty hard. There are a lot of tradeoffs that influence what tests you can write and the amount of tests to write. Most of the times you have zero control over the data you get from the service, which makes assertions tough to say the least.

A while ago I used the VCR library to write some Ruby tests against a remote service. VCR addresses the above problems. It records your test suite’s HTTP interactions to replay them during future runs. The obvious benefits are fast and repeatable tests.

This week I was wondering whether that’s a thing for Java as well. As it turns out there’s Betamax to do that. Actually Betamax is a Groovy port of VCR that can be used with any JVM language.

Betamax installs a proxy in between you and the target host, records each request and response on tape and replays the tape for known requests. It works for any HTTP client that respects Java’s proxy settings, and for a bunch that don’t such as Apache HttpClient and WSLite.

Example

In a JUnit test you can use Betamax as a method-level TestRule. On each test-method that should record and replay you put an @Betamax recorder and set a tape.

Consider the following example where I use the Spotify Metadata API to get the popularity of an artist. In this example I use the Apache HttpClient library and configure it for Betamax.

public class SpotifyTest {
  @Rule public final Recorder recorder = new Recorder();

  private final DefaultHttpClient http = new DefaultHttpClient();

  @Betamax(tape = "fixtures/popularity")
  @Test
  public void get_popularity() throws Exception {
    Spotify spotify = new Spotify(http);
    assertThat(spotify.popularity("The Beatles"), is(.55f));
  }

  @Before
  public void setUp() throws Exception {
    BetamaxRoutePlanner.configure(http);
  }
}

At the moment of writing this code the popularity of The Beatles is .55 but as this number is based on user opinion it is highly likely to change. Using a Betamax tape gets the same response (as long as the request does not change) and allows to assert .55 for popularity.

HTTPS

As I’ve shown you Betamax properly records and replays any HTTP communication using either a proxy or a wrapper class (as in the example). HTTPS is also supported but may be a bit more interesting as you use Betamax in a proxy-based setup. Using a wrapper will work just fine.

The problem with HTTPS and a proxy-based setup obviously is that the proxy cannot intercept data on standard HTTPS communication. This is why we trust HTTPS.

Betamax has its way around this. You can enable sslSupport on the Betamax Recorder. When your client code is okay with a broken SSL certificate chain you can make this work.

Again this is only really a problem as you use a proxy-based setup. Using a client wrapper enables Betamax directly on API calls easing HTTPS communication.

Try it yourself

Betamax can help you to write fast and repeatable unit tests for clients of remote services. The most beneficial to me is that the tests are really fast because remote communication is eliminated. Asserting on specific values can be helpful although personally I like a property-based style for these tests (e.g. popularity must be a number >= 0 and <= 5).

Give Betamax a try the next time you interact with a remote service.

Code Katas

2014-02-10T00:00:00+00:00

In this post I want to talk about code katas. Most of you have heard of code katas before, many of you probably have done some katas. A while ago code katas were getting a fair bit of attention. Including of course some skepticism.

Just in case you haven’t heard of code kata yet, let me explain briefly. The term code kata is coined by Dave Thomas, co-author of The Pragmatic Programmer. It’s a bow to the kata concept in martial arts. Kata is a Japanese word and literally translates to form. Originally kata was a training method to help a student master a specific move or technique.

The intention of code katas is to help programmers to do just that: master a specific move or technique. To practice these we use simple algorithms that we can dream up at any moment. Think of sorting and prime factor calculation.

So what’s effective about these code katas? I mean coding up the same algorithm time after time just doesn’t seem like you practice any technique. Except maybe your typing skills. This is why skepticists will tell you that katas are not effective and no more than a waste of time. Solving the same set of programming problems doesn’t make you a master programmer.

And they are spot on. Solving the same set of problems with (almost) the exact same code time after time doesn’t help a bit towards mastery.

Yet I totally believe that katas can be a very effective way to practice. Just not by coding up a solution to a problem in the same way over and over again. On the path towards mastery you will have to find many ways of solving problems, not just one. A kata is not about the solution to the problem. It’s about the path of how you get to the solution.

How many ways can you think of to calculate a Fibonacci sequence? In what forms can you code a bubble sort? And a quick sort? There’s no need to learn these algorithms, you probably can dream them already. Instead you focus on the code you use to get you there. You’ll walk along a certain path until you master it. You make little tweaks to the way you code up the solution along the way. You carefully consider every step you take.

To me this is what katas are really about. I often use katas to practice and learn. Again, not about the solution, but about the path. My path. The way I code. Katas help me to challenge each of my steps along the way. Is this next step the best I can think of? Why this step? Why not another? By challenging every step on my way it’s almost like pairing with myself. As a navigator I can pick up important feedback.

Then I find code katas also particularly useful for learning a new language. The famous Hello World asserts that my installation works and I can write a main. Next I want to feel the language. By doing a few katas I can get a decent grasp on its syntax. Then I want to explore some of the language’s unique features. Because I’ve seen many solutions to the katas in various languages I can get a feeling of what the language has to offer.

That said, katas are not the only way to practice and to learn. Many programming languages now have a set of koans available, to help you learn the language. I find these are very useful, too. And most katas are useless for learning more about a particular library of framework. Which is not the point of katas anyway.

As you can read, in my experience code katas help me practice and learn. Yet I completely agree that doing katas by coding up the same solution over and over again, is a waste of energy. I want to reiterate that it’s not about the solution here, it’s about the path. Most of the time I delete my code directly after I finish a kata. The lessons I learn from walking the path remain in my head and I won’t ever look back at the solution. The exception here is the first few katas I do in a completely new language. Before I have the syntax in my fingertips, sometimes I find it useful to look back at the previous exercises.

If you’re interested there are a lot different katas out there. Google is your friend. To get started you can use this Kata Catalog.

Acceptance Test Driven Development

2014-01-21T00:00:00+00:00

In this short series about acceptance testing I previously wrote about collaboration between testers and developers and how acceptance tests help to define clear requirements of a system.

Where the previous article were on the theoretical side, let’s now take a look at using acceptance testing as a practice in our development process. I prefer using Acceptance Test Driven Development, or ATDD.

In a Nutshell

With ATDD you use acceptance tests to guide the implementation path of a feature. You write a failing test scenario and the required glue code. When the scenario fails with the correct message (i.e. the test tool can run the acceptance test and the message correctly describes the missing feature) you start implementing the feature.

Like in TDD use the simplest way possible to make the scenario pass, but on a feature level. In TDD you would pass a test in the simplest possible way and then improve from there. In ATDD you just implement the feature in the simplest way possible to pass the scenario. Simply stated, if you’ve got the right set of scenarios the feature is implemented when the last one passes.

[Note that I deliberately chose not to use the word done, as after implementing the feature often we need to fiddle with the UI a bit more.]

Example Problem

For an example we’re going to build a shopping cart for Harry Potter book sale kata using ATDD. Customers get a discount based on the number of books in the series they buy. This exercise is taken from a list of kata from the Coding Dojo website. For a full description you can read: http://codingdojo.org/cgi-bin/wiki.pl?KataPotter.

The first 5 books from the Harry Potter series are on sale. One copy of any of the books costs $8. If you buy two different books from the series you get a 5% discount, if you buy three different books you get a 10% discount, if you buy 4 different books your discount is 20%, and if you buy all 5 you get a 25% discount.

Let’s pair up and build this feature together.

Discuss and Distill

The cycle for building a new feature goes through a few phases. Before we think about writing production code we have to get ourselves some scenarios. Remember how we talked about the team meeting with a business expert in the previous article? These activities form the first phases in the development cycle: discuss and distill.

Our team arranges a meeting with a business expert to discuss the shopping cart. We have prepared some questions, so this meeting can be very effective. Our team knows better than to discuss technical details in this meeting. We do not want to waste any of the business expert’s valuable time. At the end of the meeting we have taken note of examples describing the behavior of the shopping cart. For clarity we decide to list the books using roman numerals.

I	II	III	IV	V	Formula	$
1					1 * 8	8.00
1	1				2 * 8 * 0.95	15.20
1	1	1			3 * 8 * 0.9	21.60
1	1	1	1		4 * 8 * 0.8	25.60
1	1	1	1	1	5 * 8 * 0.75	30.00
2					2 * 8	16.00
2	1				2 * 8 * 0.95 + 1 * 8	23.20
2	2	2	1	1	4 * 8 * 0.8 + 4 * 8 * 0.8	51.20

Most of these lines are pretty much self-explanatory. The last line shows that there are 2 sets of 4 books. This might be surprising because you’d expect a set of 5 books plus a set of 3 books. The reason for this is that the discount must be optimized for the customer. The best possible discount in this scenario is to purchase 2 sets of 4 books as that is $0.40 cheaper than a set of 5 books and a set of 3 books.

After the meeting we distill acceptance tests from these examples and save them as Cucumber scenarios. The scenario for the first example reads:

Scenario: buy single book without discount

  When I buy 1 copy of "Harry Potter I"
  Then I must pay $8

The scenario for the second example turns out to be much alike the first one:

Scenario: buy two distinct books

  When I buy 1 copy of "Harry Potter II"
  Then I must pay $15.20

Obviously the remaining scenarios can be added in the same way. But a pattern is emerging. It would be pretty tedious and repetitive to have to copy and paste the exact same scenarios only to use different values. Besides, it would be nice if the example table we took from meeting with the business expert could be used in the acceptance test as well. That would make our scenarios more useful as documentation.

Cucumber provides a Scenario Outline feature to help out. Scenario Outlines more concisely express these examples through the use of a template with placeholders and an example table. We try a scenario outline for the former two scenarios and then decide to add the rest of the scenarios using the outline.

Scenario Outline: buy discounted Harry Potter books

When I buy <I> copies of "Harry Potter I"
  And I buy <II> copies of "Harry Potter II"
  And I buy <III> copies of "Harry Potter III"
  And I buy <IV> copies of "Harry Potter IV"
  And I buy <V> copies of "Harry Potter V"
  Then I must pay $<Total>

Examples:
  | I | II | III | IV | V | Total |
  | 1 |  0 |   0 |  0 | 0 |  8.00 |
  | 1 |  1 |   0 |  0 | 0 | 15.20 |
  | 1 |  1 |   1 |  0 | 0 | 21.60 |
  | 1 |  1 |   1 |  1 | 0 | 25.60 |
  | 1 |  1 |   1 |  1 | 1 | 30.00 |
  | 2 |  0 |   0 |  0 | 0 | 16.00 |
  | 2 |  1 |   0 |  0 | 0 | 23.20 |
  | 2 |  2 |   2 |  1 | 1 | 51.20 |

That’s more like it. Using the outline our examples read just like the table we draw earlier.

Obviously we’re not done at this stage, although we have successfully discussed the feature and distilled acceptance scenarios that discussion. We now have a complete set of acceptance scenarios. Our scenarios are a complete representation of the problem according to the business expert. The notation format of the scenarios is simple enough for the business expert to review them.

Now our team is ready for the implementation phase.

Implement

First thing we need is the glue code so that examples can be run and fail with a reasonable error message. Luckily Cucumber helps us creating the necessary glue functions by generating stubs. After a first run without glue code we can copy and paste to get to pending steps.

An example of such a generated function for Java is listed below. This is the glue code for a single step in a Cucumber scenario, also known as a step definition.

When("^I buy (\\d+) copies of \"([^\"]*)\"$")
public void I_buy_copies_of(int arg1, String arg2) throws Throwable {
    // Express the Regexp above with the code you wish you had
    throw new PendingException();
}

Now that all steps are pending we start implementing the steps to make the first scenario fail with the appropriate messages. To get these steps to compile we need to add some skeleton production code, which we’ll later enhance with the implementation. This is interesting because at this time the API we wish we’d have flows rather natural from the acceptance tests.

When the first scenario fails with a message that describes the missing feature, we implement the feature. Then we move on to the next, and so on, until all features pass. We keep our code clean by using TDD.

The actual implementation of the step definitions and the code isn’t that interesting for the sake of this article. As I don’t wish to withold these details as well I did the exercise and put the code on Github.

Review

The last phase of building a new feature is to review it. I’m not talking about code reviews here, these take place in the implementation phase.

To wrap up our new feature we do some exploratory testing. Exploratory testing is finding out how the system actually works by using it freestyle. I.e. there are not test scripts to follow. Instead test cases are invented on the fly and immediately executed against the system. If you wish to learn more about exploratory testing I can recommend you to read Elisabeth Hendrickson her fantastic book Explore It!

In our review phase we write down any inconsistencies we find through exploratory testing.

To finish our feature we demo it to the business experts and collect their feedback. Combined with any results from exploratory testing, this is important input for business experts. Only they can decide how the system should behave.

More Examples

Personally I find ATDD a very pleasant way for the entire team to work. While having the benefits of clear requirements you enjoy the fact that you grow an automated regression suite that you can (and should) run every build. This regression suite gains the team a lot of confidence not to be afraid to make changes anywhere. As soon as something breaks, a test will fail, and the team knows what to fix.

If you like to see more examples of ATDD you should read ATDD by Example, in which Markus Gärtner pairs up with you to practice ATDD while working example problems.

Acceptance Tests as Specifications

2013-12-18T00:00:00+00:00

In my previous article about acceptance testing I wrote about how tools like FitNesse and Cucumber focus on collaboration between developers and testers. Of course that’s not all these tools help us with. In this post we’ll take a deeper look at acceptance testing as a practice and see what the tools gain us.

Acceptance Testing is an agile practice that refers to functional testing a user story. Sometimes it is referred to as Agile Acceptance Testing or Specification by Example. Acceptance testing addresses one of the most significant problems in software development: the requirements.

Requirements

Often requirements don’t come across to the team members the way business people intended them to. There are many reasons for this to happen. Often these reasons can be reduced down to the team having a different context than the business people. Because the team often has only a basic understanding of the business world, the team easily misinterprets the ambiguous requirements.

This problem isn’t limited to agile methodologies. The problem is more persistent, for a few decades already. Remember those classical waterfall projects? For sure the requirements weren’t communicated briefly like user stories. They were however written from the perspective of business people and often got misinterpreted further down the production chain. By the time business people could see the system and verify that it didn’t match what they had in mind, it was too late. Many of these projects were canceled as a result, with lots of wasted money as a result. Some of these projects even caused bankruptcy.

The agile planning game (today known as Scrum) helped reducing the impact of this problem. Its short feedback loops ensure that business people in an early stage will become aware of discrepancies between their world and the system as it gets build. In case the team got it wrong, only one sprint of work is wasted. Now that’s still ridiculously expensive. Nobody wants to throw away an entire sprint’s work only because the requirements were poorly communicated. But on the plus side, most projects will survive one such a sprint.

Another approach to fixing the requirements problem is to reduce the level of ambiguity in the requirements. We could use a formal language to write a specification of the requirements. Before you tell me that won’t work or is simply not practical, let me explain. We don’t need math precision. We need to reduce the level of ambiguity just enough so that humans will understand. And conveniently enough on most projects such a formal specification is already created. It’s called the test plan.

Test Plan

The test plan defines the inputs to the system, the actions on the system, and the expected outputs in a rather formal manner. Thereby reducing the level of ambiguity enough so that humans will understand.

By executing the test plan against the system QA gets feedback about any gaps between the behavior of the system (the interpretation of the developers) and their own interpretations. Agile reduces the amount of differences by putting developers and testers in the same team, but still QA writes the test plan while developers implement the feature. At best the team finds certain differences in interpretation while implementing.

Compared to business people finding that the team didn’t implement ‘the right thing’ after a sprint, having the team to find this herself within the sprint is much, much better. But it causes rework nonetheless and that takes up some of our valuable time.

What if we could learn these gaps before implementing a feature? That could reduce the rework even further and get the feature even closer to right from the start. Note that I am definitely not claiming that the team shouldn’t make any mistakes, they will. But given an opportunity to prevent certain mistakes by early learning, we should take it.

Specification by Example

So business people write a bunch of stories. The team does some form of estimation of each story’s size and takes some stories in sprint. At this point the team doesn’t know all the details yet. To find out they can organize a meeting with a business expert and ask her any questions to learn these details. This is where the team discusses for example a complex business algorithm or details about the UI. Together with the business expert the team creates a set of examples and acceptance criteria.

[Note that it is helpful if all team members attends these meetings to gain a shared understanding of the feature. If not, make sure to form a delegation of at least one developer and one tester.]

The team records the examples and acceptance criteria as test scenarios. Preferably the business expert reviews the scenarios afterwards, to assert the outcome of their meeting. This is the point where the tools pop back in. Tools like FitNesse and Cucumber use a simple yet formal language for defining test scenarios and example tables. At this stage the benefit of using a tool would be that you don’t have to invent such a language yourself.

The entire team now can read the what the feature is supposed to do and is on the same page. The first thing the developers (or sometimes the testers) do after the scenarios are written is to create some glue code. This code binds (glues) the scenarios to the system to allow the test tool to run the scenarios against the system and verify the outputs.

Power of Tools

This makes these test tools really powerful. The test scenarios can be executed any time we wish, at high speed. Writing test scenarios and glue code are typically tasks for humans, but humans are terrible at consistently repeating the same tasks, making them extremely inefficient test execution platforms. Computers on the other hand are very effective at consistently repeating tasks, which makes them well-suited as an execution platform for our test scenarios.

Not only the test scenarios help building a shared understanding of a feature before the implementation. Every team member can (and should) read the scenarios, as can the business stakeholders. The team is not done implementing until all scenarios pass.

Because the scenarios can be executed by computers, they make for an excellent regression suite. We can have the computer run our scenarios every build, or at least a few times a day. As the team gets feedback from the test suite quickly after they change something, they can catch regression bugs in an early stage.

This may all sounds just too theoretical. Therefore I will continue on acceptance testing with a more practical post about using the practices in our development process.

Acceptance Testing: Blaming the Tools

2013-11-29T00:00:00+00:00

About 5 years ago I was on a project to build a system for collateral management. The system was connected to a large financial network, and got its instructions through standardized financial messages.

This project was run in waterfall style, with lots of restrictions on collaboration between disciplines. Without to digress in details on this, I do still think it’s interesting that those project managers, even the experienced ones, believe that collaboration is bad. Anyhow, a team of business analysts wrote down the detailed specifications, then handed them over to the developers. The developers implemented the system according to the specifications and then delivered the software to the team of testers.

Testing the system

Because this particular system was operated through all these financial messages, it was much harder to test than a system with a regular user interface. The developers created a testing tool to interact with the system using a command line interface. By the nature of its command line interface the tool was also fit for scripting tests for entire business flows. The tool was low-level, just as the developers needed it to be.

The test team also needed a tool to interact with the system. The tool used by the developers was too low-level for the testers. One of the testers knew a tool he had used before to describe tests in a language that is more friendly to most people, and then having that tool link the friendly language to code. This tool was FitNesse.

To be able to interact with the main system from FitNesse, an adapter was added to the test tool the developers used. This was a fixture to get FitNesse to run the test tool. Some additional features were added to the test tool as well, to allow the testers to create template messages and then set specific values using xpath expressions in their FitNesse test tables.

Making changes

This approach went well for about 2 months. Within this time the test team had added a FitNesse plugin to connect to the system’s database to verify some of the system’s intermediate state, and had added a few layers of indirection to the FitNesse tests so it wasn’t really clear what operations were performed on the system from the ‘friendly’ specifications in the test tables.

And then the type of financial messages to support changed. This influenced all the message templates and all of the xpaths that were used so far. You can imagine that this touched almost every test, causing tons of rework on the test team. Because they couldn’t get it done fast enough, they got help from the development team.

A short while after the tests were all ‘migrated’ to use the new message structure, the database schema changed. And it didn’t change subtle either. Again most of the acceptance tests failed and needed lots of rework.

Blaming the tool

Boy, we really started to hate this FitNesse tool. Every time the system evolved all the tests started to fail. And to fix them, developers were to help the testers rework those tests. We could think of a nicer task or two.

It was not until a few weeks after I left the project that I realized that FitNesse was not to blame for the problems we faced. We were to blame ourselves! It wasn’t the tool that was wrong, it were the people who used it wrong. It was on the Advanced Test-Driven Development masterclass by Uncle Bob (Robert Martin) where I learned that the purpose of FitNesse isn’t to provide a textual abstraction to running your acceptance tests.

Collaboration

Don’t get me wrong, that textual abstraction is really nice and all. But it shouldn’t be the primary goal.

FitNesse is a tool for automated acceptance testing, with the focus being collaboration between business people, developers and testers. Through collaboration a team of developers and testers can together learn all about the features, each bringing their own skills and qualities to the discussion. Together they register their learnings as a test table or brief sentences.

As I mentioned earlier collaboration wasn’t one of the focal points on this project. Instead it was something you ‘do not waste your time upon’.

Where in this particular project the problems appeared to be around FitNesse, I believe it’s the same thing for many of the Cucumber stories I read. On the internet there are a lot of opinions about how Cucumber (and likewise tools) add no value to a project. I do believe that this is true in many projects, but for different reasons than projected in most of those articles.

Tools like Cucumber and FitNesse are not developer tools. If no-one but the developers read and write these scripts, using such tools is no more than a waste of time. Developers can faster automate acceptance tests using their regular test framework (e.g. rspec or junit).

But these tools neither are QA tools. Only when working together, testers and developers can get most out of these tools and use them for real good. Both Cucumber and FitNesse are collaboration tools. They belong in the toolbelt of agile teams.

To be continued

Of course collaboration is only the beginning of automated acceptance testing. In the next weeks I will get back to other aspects of automated acceptance testing.

Wishful Programming

2013-11-20T00:00:00+00:00

Wishful Programming (or: Programming by Wishful Thinking) is something I heard about for the first time a couple of years ago. It is a very powerful programming practice, but I found many programmers are unaware of it.

Wishful thinking is a general technique to help you think in abstractions. As far as I know it was first mentioned in the context of programming in Structure and Interpretation of Computer Programs. This brilliant book suggests to defer the decision about how to represent your data by imagining you have any function you wish you had in order to solve your problem in the simplest way possible.

In other words, you write some code that uses a component, before you implement the component. This helps to discover what functions and data you need, and following this discovery will lead you to more simple and useful APIs.

I find that this technique is especially valuable with an outside-in approach for implementing a feature. To start with, it helps writing acceptance tests in terms of business functions. Details about how the system implements these functions are hidden from the context of the acceptance tests and implemented in a fixture.

Once the acceptance test reads pretty much like the high-level steps you would take to execute the feature and fails with a decent error message it’s time to implement the feature. I drive out the design using TDD, starting at the point where the acceptance test fixture integrates with the production code. This allows me to design any collaborators right from the tests in the form I want them to collaborate, without worrying about creating them first.

In other words, I write acceptance tests using a fixture I wish I had, and then implement the feature using collaborators I wish I had.

That is wishful programming. I’ve seen how this helped my code to evolve and be more structured in terms of the solution domain, rather than a sequence of technical steps. I hope being aware of this technique will help you in the same way.

Lambdas for Fluent and Stable APIs

2013-11-19T00:00:00+00:00

A few weeks ago I wrote an introduction on Java 8 lambdas. In this introduction I explained what a lambda is and how to use them in conjunction with the new Stream API, that is also introduced in Java 8.

The Stream API provides a more functional interface to collections. This interface heavily depends on lambdas. But there’s much more to lambdas than improved collection processing

Lambdas offer you a chance to build much more fluent APIs. To show this, as an example I like to use a UserStore which facilitates fetching and saving users using a database. Its public API commonly looks like the following.

public interface UserStore {
  User find(Long id);
  List<User> findByLastname(String lastname);
  List<User> findByCompany(String company);
  ..
}

The list of findBy methods often is longer than just the two I’ve included here. As the system grows it’s likely that there will be others. While that works, the thing is, all these methods basically do the same thing. They return all the users with a property that matches a specific value.

Some frameworks offer work-arounds for this mess. If you’ve used Hibernate you’re probably aware that they provide a work-around to this with findByExample where you provide a User as an example object that provides the properties and values to query by. Any value that is set in this example object is used to query by, while any field that is null is excluded from the query. You can tweak this behavior a little, but there are a number of problems with this approach. Think of default values, required fields (i.e. fields that cannot be made null), and immutability. iBatis, MyBatis, and also Spring Data, use code generation to save you time implementing all these methods, leaving the API bloated with a list of findBy methods.

These work-arounds might come a long way, but they do leave their own specific problems behind. A different approach would be to use lambdas.

Lambdas can help us to decouple the query part from the filter specification. Let’s change the findBy functions to a single function that accepts a lambda.

public interface UserStore {
  User find(Long id);
  List<User> findBy(Predicate<User> p);
}

That’s a much better API. Obviously it’s a bit naive as the predicate checks a User object; you generally want to filter using your database query. Nonetheless it serves the purpose of this example pretty well, and you can experiment with your own lambdas to filter using database queries. [Note: Predicate comes with Java 8 and is located in the java.util.function package.]

Before you might want to get mad because at least with the previous API the predicates were bundled in one place, we can still bundle (common) predicates. For example, by creating a utility class UserPredicates that contains them.

public final class UserPredicates {
  public static Predicate<User> lastname(String matcher) {
    return candidate -> matcher.equals(candidate.getLastname());
  }
}

Using the new UserStore API has become pretty simple.

static import UserPredicates.lastname;
..
userStore.findBy(lastname("<lastname>");

There one thing left in UserStore that really bothers me, though. The find(id) function returns a user. But what if there’s no such user?

Optional

To improve on this we can (and should) look at another new feature of Java 8, Optional. This is sort of the Java implementation of the maybe monad. It looks a lot like Scala’s Option.

With Optional we can better express that a function may return a value but not necessarily, and prevent using null. In our find(id) example returning Optional clearly expresses that we might find a user with the requested ID, but maybe no such user exists.

public interface UserStore {
  Optional<User> find(Long id);
  List<User> findBy(Predicate p);
}

Not only does the API now document the fact that you might get a user, it also never returns null. I consider an API that never returns null much safer. One day a new programmer might not realize that find can return null and a null-pointer exception is the result. Just hopefully the team catches it before production. Null-pointer exceptions are really easy to prevent, as long as you never use null.

We can use functions on Optional to get a value from user if available, or to get a default value otherwise. For example, to safely get the user’s lastname we write the following.

Optional<User> user = userStore.find(id);
String lastname = user.map(User::getLastname).orElse("");

This code is pretty expressive and doesn’t require a lot of explaination. If there’s a user, get it’s lastname. Otherwise, get an empty string.

What if we need to send the user a password-reset email, if it is found?

Optional<User> user = userStore.find(id);
user.ifPresent(passwordReset::send);

The password-reset is sent if a user is found, otherwise nothing happens.

Since Java does not support destructuring like many other languages that provide maybe (e.g. Haskell, Clojure, and Scala), we are limited to the functions of Optional. This makes Optional weaker than its equivalent in any of these languages.

Builder

Of course not only the APIs of repositories can benefit from lambdas. Optional is also a good example of an API that benefits from lambdas. Personally I also find lambdas particulary useful to replace passing builders around. Rather than passing a specific builder to a function, often it improves decoupling by yielding a builder from the function. Let me show you an example for sending email to clarify this idea.

public interface Mailer {
  void sendTextMessage(TextMessageBuilder message);
  void sendMimeMessage(MimeMessageBuilder message);
}

To use the Mailer we need to pass a specific builder into it. These builders have a common interface but are different in the kind of message they build. And the Mailer has separate methods because it must add different information based on the type being used. Because of this any client-code is tightly coupled to pass the correct builder.

As you probably suspect, this is where lambdas are helpful. Rather than requiring the client to create a builder and pass it in, the Mailer functions can create the builder they need and yield it to a lambda.

public interface Mailer {
  void sendTextMessage(MessageConfigurator configurator);
  void sendMimeMessage(MessageConfigurator configurator);

  @FunctionalInterface
  interface MessageConfigurator {
    MessageBuilder configure(MessageBuilder message);
  }
}

To use the Mailer all we need to do is provide a lambda to build the message.

mailer.sendTextMessage(message ->
  message.from(sender).to(recipients)
      .subject("APIs")
      .body("Lambdas can make for more fluent and stable APIs")
);

The API now is much more stable. Client-code is decoupled from any changes in specific builders, and won’t break as long as functions on the builder remain compatible.

As the examples helped me show, lambdas can help you build more fluent and stable APIs that are more intention revealing. These APIs require not much documentation for other programmers to get going because it’s actually hard for them to get it wrong. As a general guideline I prefer clear and intention revealing code over documentation. Fix, don’t document.

Of course I’ve only shown a few examples in this article. Lambdas are much more widely applicable than only with the examples here. I hope this article provided you with some new insights in what lambdas can help you with, and that you can think of ways how they can improve your code.

Introduction to Java Lambdas

2013-10-26T00:00:00+00:00

The main theme of Java 8 is lambdas. I have noticed that for many Java programmers lambdas are pretty tough material. So let’s try to get a basic understanding of them.

First of all, what exactly is a lambda? A lambda is an anonymous function that is, unlike a regular function, not bound to an identifier (i.e. it has no name). These functions can be passed as arguments to other functions (known as higher-order functions).

Suppose our application must write to a bunch of files from different places of the system. We don’t want to handle the checked exceptions every time [see Exceptions: checked and unchecked for more about checked exceptions]. So we decide to write a low-level writeToFile function that opens a FileWriter and passes it to a function that can then safely write to the file.

Using this low-level function we write the following code.

writeToFile("todo.txt", new FileWriteFunction() {
    @Override
    public void apply(Writer file) throws IOException {
        file.write("learn about lambdas\n");
        file.write("learn stream API\n");
    }
})

The object we pass into writeToFile is an anonymous implementation of FileWriteFunction [anonymous because we didn’t name it as a class]. It has a single function making this effectively passing an anonymous function. In the world of Java these are sometimes referred to as callbacks. Probably you’ve used this at least a few times before, maybe without taking notice.

This anonymous object is effectively a lambda. But obviously this doesn’t look like passing a function. The syntax is very unwieldy. And this is exactly what changes in Java 8.

With the syntactic support for lambdas in Java 8 the code reads like we pass a function. Using a Java 8 lambda we rewrite the above code as follows.

writeToFile("todo.txt", file -> {
    file.write("learn about lambdas\n");
    file.write("learn stream API\n");
})

That’s better. It emphasizes the code that matters and hides most of the unwieldy parts.

Closures

Often lambdas are used interchangeably with lexical functions (closures). While they are both anonymous functions, the definition of a closure is that it is a function containing bound variables. I.e. the closure includes a referencing table that contains references to local variables.

For example, if we accept a parameter data that we want to write to a file, we use a closure.

void save(String data) {
    writeToFile("file.db", file -> file.write(data) );
}

While anonymous inner-classes restrict access to final variables, closures provide access to any variable. However, the variable is effectively final to the closure, so it cannot be reassigned.

Compilation

What about compilation of lambdas? Does Java 8 only provide a spoonful of syntactic sugars upon anonymous inner-classes with only one method?

Not really, no. It’s true that it allows the lambda syntax for any single-method anonymous inner-class. But lambdas are not compiled into inner-classes. Instead the compiler outputs lambda$ methods in the defining class and uses invokedynamic to dispatch the call.

So now you know how to use a lambda in Java 8. While by themselves lambdas are pretty useful, they are even more so when applying them to collections.

Stream API

The new Stream API provides an alternative to iterators by offering a more functional API to collections: java.util.stream.Stream. The most noteworthy functions on a Stream are: collect, filter, map, and reduce.

To start with a simple example, here’s how to sum all the numbers in a list.

asList(1,2,3,4,5).stream()
    .reduce(0, (acc, value) -&gt; acc + value) // => 15

This reduces the sequence by adding each value to an accumulator, starting at zero. For comparison, classically you’d write a loop.

int acc = 0;
for (int n : asList(1,2,3,4,5))
    acc += n;
acc // => 15

Moving on to sum only odd numbers. First we filter odd numbers, then we reduce.

asList(1,2,3,4,5).stream()
    .filter(Predicates::odd)
    .reduce(0, (acc, n) -> acc + n) // => 9

The argument to filter is a function reference to a static odd function in a Predicates class I used. It is a boolean function that, as the name suggests, tests if a number is odd.

So far, so good. Now suppose we want to transform a list of centimeter sizes to their equivalent size in inches. We use map for this.

List<Inch> inches = centimeters.stream()
    .map(Centimeter::toInches)
    .collect(Collectors.toList())

The centimeters are mapped to inches by applying the toInches function to every item in the centimeters collection.

By its nature, Stream is continuous. It is used to describe the operations to apply on a collection. But to acquire the results the data must be collected. This is what the collect function is for. It reduces the elements of the stream to a mutable container [e.g. a list].

Using the Stream API and lambdas can greatly simplify the code you use to work with collections, and make the code much more expressive. Preferring using non-destructive operations (e.g. map, filter) over destructive operations (e.g. forEach) makes the code more easy to reason about.

That’s it! These are the basics you need to know about lambdas (and closures) in Java 8. Of course there’s much more to write about lambdas, but that’s something for another post.

End of the Journey

2013-10-09T00:00:00+00:00

Wow! Those were seven exciting weeks following a wonderful journey through seven fantastic languages, each with their own specialties. To three of the languages I was completely blank, with the other four I was (a bit) familiar.

The journey started with Ruby, a language that adopts today’s mainstream programming paradigm of object orientation, and with so much syntactic sugar that every programmer is able to understand what’s going on.

Through the Io language that uses the prototype based programming style (also the object oriented paradigm) the book went on with logic programming using Prolog. This is completely different from the first two languages, and lays a firm foundation for some of the languages to come. It also drags you into recursion.

From Prolog the journey took a turn for a more functional style of programming. First stop was Scala, the object oriented and functional hybrid. From thereon the destinations were Erlang, Clojure, and finally the purely functional Haskell.

The author, Bruce Tate, kicks off every language with an introduction and some background information on the language. By the order of the languages you can notice that Bruce did his best with each language to provide a foundation for the next one. I am not sure if Bruce intended it this way, but for me it felt like everything came together in Haskell.

I really enjoyed this vast journey through these seven languages. It’s very interesting how they all have their own strengths and weaknesses, each of them being great at what it was created for. Some of them are applicable to more general purpose programming, others solve a very specific set of problems.

In the book Bruce sometimes speculates a bit about the future of languages and whether or not languages can become mainstream. Below are some of my thoughts about this.

There is no longer any doubt that functional programming techniques become increasingly important as our hardware gets more concurrent. Using threads to do concurrent programming just won’t do if programmers must be productive. Moore’s law these day’s no longer applies to the speed or our hardware, but it does seem to nearly apply to parallelism. Our hardware doesn’t get twice as fast every year, but it gets twice as parallel every few years.

That said, I think that the best candidate to become a mainstream language is Scala, simply because it offers a smooth transition from object orientation to functional programming. Personally I doubt whether this is actually helpful. It’s much like C++ transitioned many programmers from procedural programming in C to the same way of thinking with objects that is still much applied today in mainstream languages as Java and C#. [Of course this wasn’t all C++ did and no offense towards C++ is meant in any way.]

At the moment I believe Ruby is becoming a mainstream language, but without many programmers who use it actually knowing it. Tools like Puppet or Chef offer a configuration DSL written in Ruby, where programmers can fallback on Ruby for doing more complicated and exotic configurations. As a scripting language that is available on all major platforms it is great for building system administration tools and configurations. It is rather ironic how offensive many Java programmers are against Ruby (or all dynamic languages for that matter) and at the same time telling how much they like the configurations of Puppet.

To anyone who is passioned about programming I would strongly suggest reading this well-crafted piece of work. Especially if you haven’t learned a new language in a while, let me remind you Andy Hunt’s advice in The Pragmatic Programmer to learn a new language every year.

As I mentioned in the articles, Ruby and Clojure belong to my favorite languages today. And for Erlang and Haskell there’s much to learn, which I intend to pick up next year.

Thank you for joining me on my journey. I had a great time, I hope you did too.

Week 7: Haskell

2013-10-02T00:00:00+00:00

Haskell is the seventh language in the book. It is a pure functional language, so functions have no side effects. This makes it easier to reason about the code.

In contrast to most other languages Haskell is not created by a single person. When in the late 1980s a large number of people were designing and implementing functional languages, a committee formed to work together so they’d be stronger. The original goals of the language were to form a foundation for research that was suitable for teaching and industrial usages.

Haskell exploits a static type system, like Scala. It has strong support to infer types. Its type system is build to clearly express the intent of the code. It can prevent much error handling and argument validation. Its type system is much more powerful than Java’s (and so Scala’s).

Bruce compares Haskell with Spock from Star Trek. Like Spock, Haskell is completely predictable (because of its pure functions), and always logical.

This week of Haskell was my first experience with the language. It took me a little while to get started. On day 2 things got better because the exercises got more difficult. I liked the exercise for generating a lazy list with prime numbers. I used a simple algorithm (Siege of Eratosthenes) that is not really fast but easier to implement than many other algorithms.

On day 3 Bruce introduced monads. In short, using a monad we can compose functions that have certain properties. This is not an accurate explanation of monad, and if you wish to really understand them you will have to invest a bit of time reading through example monads and the explanations. Monads are not specific to Haskell, lots of (partly) functional languages support them. Every other functional language in the book (Scala, Erlang, and Clojure) does.

I enjoyed the exercise on day 3 of representing a maze using Haskell’s type system and solving it using the List monad. The exercise of implementing the List monad in a non-functional language was interesting. I chose Ruby and Io.

My solutions to the most interesting exercises in the book are available in the 7-languages-in-7-weeks repository on my Github profile.

One of the major strengths of Haskell is of course its type system. Personally I prefer duck typing over static typing, but that’s mostly because all major static type systems (C, Java, Scala) force so much redundancy. Haskell changes this by understanding that static typing is a tool that is very useful in some situations while not so much in others. Like I said earlier, the compiler can infer types if you don’t provide them.

Other great strengths of Haskell include its lazy computation strategies, the expressiveness of the language, and the purity of its programming model. Also Haskell is the primary language for teaching functional programming techniques.

A definite weakness of Haskell is the very steep learning curve. To get the most out of the language and fully embrace its strengths you have to understand many functional concepts (monads is one of them). As Haskell is a pure functional language, there’s no cheating in using a few non-pure functions.

Personally I find Haskell a particularly interesting language, one that is important to get a good grasp on. As I am not yet comfortable doing some real work with Haskell yet, I definitely want to learn more about it. I believe that getting more proficient in Haskell will strengthen my understanding of functional techniques.

Finishing this week of Haskell only leaves me to wrap up my final thoughts about the book and learning new programming languages. I’ll leave that for next week.

Week 6: Clojure

2013-09-23T00:00:00+00:00

Clojure is one of my favorite languages of the moment. Although I haven’t used it for real work yet, I enjoy the simplicity of the language and syntax. So with a week full of Clojure exercises I am in for a joyful week!

Clojure is a dialect of Lisp. It’s a functional language but not a pure functional one. One of the most interesting aspects of Lisp (in my opinion) is that the language is homoiconic. Simply stated this means that both code and data share the same internal representation. This makes the language ideal for metaprogramming.

Rich Hickey created Clojure because he wanted a functional, dynamic language for the industry dominant platforms of the JVM (Java’s virtual machine) and CLR (common language runtime, a virtual machine for the .NET platform).

Clojure is the first Lisp that is designed to be a tool for general-purpose programming. Its syntax is a little easier on the parentheses than other Lisps, so that less programmers are put off by them. Moving from foo(bar, baz) to (foo bar baz) should not be difficult for programmers.

Bruce chose Yoda as a character for Clojure. His communication style is often inverted and in the early beginning hard to understand. Especially if you’ve never read Lisps before, this is true for Clojure. Where the Jedi master has an inner power that others can’t seem to master, Lisp has macros and higher-order constructs that most no other language is offering.

Like I said I haven’t used Clojure for real work yet but my previous experience was enough to get me through the first days quick. On day 3 the sleeping barber exercise (by Edsger Dijkstra in 1965) was interesting and fun. I got to the average of ~475 haircuts in 10 seconds, whereas the theoretical maximum is 500.

My solutions to the exercises in the book are available in the 7-languages-in-7-weeks repository on my Github profile.

Clojure has a lot to offer. It is Lisp and therefore is extremely flexible and very powerful. Being on the JVM it can take advantage of the Java ecosystem and its libraries. Lazy evaluation features can simplify many problems by delaying execution until it is actually needed.

Clojure has an interesting approach to concurrency. STM (shared transactional memory) protects you from many race conditions by forcing state mutations to happen within appropriately protected functions. This really pays off for solving problems like the sleeping barber.

One of the greatest weaknesses of Clojure comes from being on the JVM. Optimization for tail recursion is limited to loop/recur. We can only hope that the JVM will fix this some day.

I like Clojure a lot. Once you get past the awkward feeling of the syntax it actually starts making sense. It is highly expressive and its functional nature allows for much more clear and beautiful abstractions that Java ever can. I believe that better software can be built faster with Clojure.

Next week is the last language of this book: Haskell. See you in a week.

Week 5: Erlang

2013-09-14T00:00:00+00:00

Erlang! The language of concurrency, and much more. It’s the first actual functional language from the book, although not purely functional.

Erlang is acronym for Ericsson Language and was originally developed by Joe Armstrong in 1986. What started as finding a better way of writing control software for a telephone exchange, is now the foundation of many highly reliable systems like CouchDB and Facebook’s chat.

Bruce characterizes Erlang as Agent Smith, the artificial intelligence program in the Matrix. It has the ability to take any form and bend the rules of reality to be in many places at once. He was unavoidable.

Erlang is the second language in the book that is completely new to me (the other was Prolog). After a week of Prolog, Erlang isn’t as much of a shock. They both heavily use pattern matching (for the good) and their syntax is much alike. Although a big advantage in Erlang is the ability to use return values. It might be because Prolog was still fresh in my memory, but I had not much trouble with Erlang’s syntax.

The first exercises would be rather trivial with Erlang’s core libraries, but of course that would spoil the exercise. Therefore I didn’t use any of the core library functions and wrote any functions I needed, myself. This helped getting a better grip on the language.

On day 3 the exercises were all about concurrency. To explore the concurrency features introduced on day 3 I went a bit beyond the exercise of monitoring a translation process and restart it in case is dies. The doctor I wrote can monitor and restart any process, and I attached translation services for two languages.

Further exercises on day 3 were to set up two-way process monitoring, so that both processes are automatically restarted in case of failure. Erlang makes this pretty easy.

My solutions to the exercises in the book are available in the 7-languages-in-7-weeks repository on my Github profile.

All in all I liked Erlang a lot. Once you get used to the syntax (which might take a little while, but again it is definitely simpler than Prolog) you have access to an amazingly powerful language. I believe that the most power in Erlang comes from its unique approach to concurrency. It uses very lightweight processes to handle messages. It’s extremely easy to monitor processes and restart them might they fail. This complements Erlang’s let it fail strategy, which makes error handling so much easier.

What’s more is that the Open Telecom Platform (OTP) provides standard ways for creating and monitoring processes. With only a few lines of code you build reliable concurrent processes. Erlang provides what many other languages cannot: scalable concurrency and reliability.

Compare that to Java’s heavy threading model! Java might be the enterprise technology of today, but Erlang has industrial strength ready for adoption.

I definitely want to come back to Erlang to release its true power and added Programming Erlang to my reading list. For now this is where I leave Erlang and move on to next week’s language: Clojure.

Week 4: Scala

2013-09-07T00:00:00+00:00

It’s a while ago since I last wrote some Scala code so this week was fun picking it back up.

Scala is not a full object-oriented language nor a functional language, but it supports both paradigms. In fact, the intension of Scala’s creator, Martin Odersky, was to form a bridge between the object-oriented and functional paradigms, much like C++ was a bridge between the procedural and object-oriented paradigms.

Scala gets the character of Edward Scissorhands: often awkward, sometimes amazing, but always a unique expression. This is absolutely true for Scala, with it’s academic-like syntax.

Scala fits the role of being a bridge like a glove. It has a strong affinity with Java; it runs on the Java virtual machine, it allows you to call Java libraries directly, and as mentioned before, it supports the ideas of object orientation.

Personally I find its static typing still pretty intrusive and still found myself adding redundant type information just to satisfy the compiler. Also the Scala language looks much more complicated because of its static typing. But it definitely eases the static typing burden of the Java language. Because the syntax is simpler and the code is less polluted with type information, I also found it easier to express ideas in Scala than in Java.

On the up-side, Scala’s actor model is a welcome improvement in concurrent programming. Combined with immutable (shared) state it makes it much more easy to reason about concurrency. Scala is also very powerful for creating DSLs, and its built-in XML support is a great example of that.

My solutions to the more interesting exercises in the book (a Tic-tac-toe game and a text censor trait) are available in the 7-languages-in-7-weeks repository on my Github profile. For day 3 I didn’t include the exercise as it was rather trivial.

I do believe that there’s a future for Scala. Its support for object-orientation makes it easier to get started for programmers who don’t know functional programming. And for programmers who do understand a bit about functional programming, Scala has excellent support for that too - especially with the Scalaz extension.

I would like to get back to Scala and definitely prefer it over Java. For now this is where I leave Scala and focus on the next language: Erlang.

Week 3: Prolog

2013-08-31T00:00:00+00:00

Prolog is the first language that completely gets me out of my comfort zone and brought me back to noob. Prolog is a pretty amazing language that at first makes you wonder how it knows so much.

Bruce picked the Rain Man character for Prolog. I can see this resemblance. They both are a great source of knowledge, if you get your questions right. Many times you’re left wondering ‘how did he know that?!’.

In the beginning I was in a continuous struggle to get to my questions right. The first hour or so I had to get used to thinking only in terms of rules and recursion. My brain seems adjusted for being able to return values. At first this made it more difficult to reason about the code and the problems at hand.

After some fooling around and getting not much done without some help, I took a different approach. I started evolving the code in really small increments, trying each increment in the prolog repl. This really helped and got me gain a better understanding of the language.

Simple algorithms like fibonacci and bubblesort that were hard at first (writing them in a single attempt) were suddenly easier to get done.

The fibonacci algorithm gave me a little deeper insight into Prolog. I implemented a tail recursive calculation of the fibonacci sequence that gave me the fibonacci number at a certain point in the index (e.g. the fibonacci of index 6 is 8) and I wondered why Prolog couldn’t reverse the rules and give me the index of a provided fibonacci number (e.g. the index of 8 is 6).

That’s when I saw the difference between functions and rules. The fibonacci algorithm is a function and in fact there is no way for Prolog to revert the algorithm like it can with rules. After some googling if fibonacci can be done with rules, it seems to be solvable by applying the Constraint Logic Programming library (or CPL).

The Sudoku solver was surprisingly simple. I took the solution from the book a bit further by generalizing it for mutiple valid Sudoku puzzle sizes (4x4, 6x6, 9x9). As I choose to stick with GNU prolog I had to write some utility functions to make for a more general solver (like partition, transpose, take, first, etc).

I wrote these utility functions using actual TDD [by which I mean here having a set of tests I can run every time I want to, rather than using the repl]. This made it a lot easier. Interestingly, I ended up taking somewhat larger increments than I would normally use.

My solutions to the more interesting exercises in the book (like the Sudoku solver) are available in the 7-languages-in-7-weeks repository on my Github profile.

Prolog is a very interesting language that makes solving the right problem a breeze. Basically it provided all-or-nothing solutions. If it cannot find a solution to the proposed problem it just prints ‘no’. This is where you have to adjust your questioning and retry.

After a bit of a puzzling start I had great fun using Prolog. It has an interesting way of thinking about the problem and describing the problem domain itself rather than a recipe for the solution. If I again come across a problem that is well suited for Prolog, I will take this advantage.

For now this is where I leave Prolog and move to my next destination: Scala. See you in a week!

Week 2: Io

2013-08-24T00:00:00+00:00

After I had some fun with the Ruby exercises, week 2 brings a language that’s new to me. Or at least it was new to me as I started reading about the Io language about half a year ago. Like I wrote in the introduction post I spent some more time with Io and came to write iospec, a small BDD-style test library that is inspired by Ruby’s RSpec.

Io is a prototype language, like Lua or JavaScript. Hence every thing is an object, and every object is a clone of another object. You interact with an object through messages. Io is an interesting language that provides a lot of flexibility. It’s syntax is simple enough so that you can learn it in about 15 minutes.

The character Bruce chose to represent the Io language is Ferris Bueler. Let him take you for a day and you’re in for a real treat, and end up seeing things you’ve never considered before. Io uses normal order evaluation, meaning that it delays evaluation of method arguments until they are required (i.e. lazy evaluation).

Being a homoiconic language (meaning that program structure is, like data, represented in a primitive data structure of the language) Io offers lots of flexibility, making Io a great language for implementing DSLs. It’s this flexibility that made it possible to create the iospec test library. Io is almost like an object-oriented Lisp.

Io is relatively new with a small community. Development started in 2002 by Steve Dekorte, to help him better understand how interpreters work. It is inspired by Smalltalk, Self and Lisp. Io is used in different environments, like router configuration language and a scripting language for video games. Its small footprint, rich concurrency (it supports the actor model) and simple integration with C makes it highly suitable for embedded systems.

I loved spending time with Io, doing the exercises of the book and creating iospec. Unlike Ruby there are not much syntactic sugars, but because of its awesome flexibility adding some yourself is fairly simple.

My solutions to the more interesting exercises in the book are available in the 7-languages-in-7-weeks repository on my Github profile. These exercises include a Fibonacci sequence (both recursive and destructive), a two-dimensional array including transpose, and an XML builder DSL.

Io is an amazing language. Maybe it will have a lasting impact, maybe not. Learning Io gained me many insights in how JavaScript actually works.

Now it’s time to move on with the next language: Prolog.

Week 1: Ruby

2013-08-17T00:00:00+00:00

My journey through these seven different programming languages starts with a language I am already familiar with: Ruby. Even so the chapter was an interesting read and provided some new background information.

The character that Bruce chose to represent Ruby is Mary Poppins, the British nanny. With a spoonful of sugar she made the household more efficient by making it fun. Ruby does the same thing to programming, with more than just a spoon of sugar. Syntactic sugar. These little language features break the basic rules of the language to make the code a little easier to understand.

The creator of Ruby, Matz (Yukihiro Matsumoto), doesn’t worry much about the runtime efficiency, he optimizes Ruby for programmer efficiency. He makes the language simpler to read and to program in by adding more complexity on the parsing and interpretation side.

Ruby is a strongly typed dynamic language. Dynamic type systems rely on duck typing, where the features of an object are based on what it can support rather than its class hierarchy. Duck typing, in general, will gain you cleaner abstractions with more concise, readable code. The cost of duck typing is less comprehensive tool support (e.g. the IDEs for static languages). The fact that Ruby is a very dynamic language is proven by its extensive support for meta-programming.

The book provides some nice exercises to get hands on with Ruby, like writing a number-guess game, a basic grep implementation, and an interesting API to the information from a CSV file using meta-programming. I put my solutions to these exercises in the 7-languages-in-7-weeks repository on my Gibhub profile. The rest of the exercises from the book where much smaller and more trivial. I did these in the Repl [Read-eval-print loop].

The strengths of Ruby lie in its pure object orientation. Ruby is a great language for web development. Because the language is so productive it is awesome for bringing a viable product to marked quickly. Ruby is also ideal for scripting since it is present on most major operating systems.

The main weakness of Ruby is its runtime performance. Some major performance improvements have been done in Ruby 1.9 and there are some attempts at bringing JIT compilation to Rubinius. And as with any object oriented language the programming model also is a weakness. It depends on the idea of wrapping behavior around (usually mutable) state. This model often leads to serious concurrency problems, that are almost impossible to test and debug.

As I mentioned at the beginning of this article, Ruby and I have met before. In fact, Ruby is one of my favorite languages at the moment. After the long ruling of languages from the C family it’s a breath of fresh air. It’s syntactic sugars help you to write very clean and readable code.

Time to continue my travel to the destination for next week: Io.

Back to Noob

2013-08-13T00:00:00+00:00

A while ago I bought the book Seven Languages in Seven Weeks by Bruce Tate.

The goal of this book is to take you (the reader) on an enlightening journey that will challenge the way you look at programming. Using this book alone you won’t become an expert on any of the languages. But the book will teach you a lot more than just “Hello, world”.

In the introduction Bruce explains the method he uses to learn a new programming language and how he needs a deep, fast dive into a language to taste its true flavor. In the book he guides you through seven such experiences.

The languages that are covered in the book are: Ruby, Io, Prolog, Scala, Erlang, Clojure, and Haskell. Of each of these languages the book helps you to study, amongst other things:

the type model - i.e. strong or weak, static of dynamic
the programming model - i.e. object oriented, functional, procedural, logic, or hybrid
the core features that make the language unique
and more, that I won’t spoil for you

The introduction comes with the warning that the book won’t be your installation guide. Personally I welcome this as I strongly dislike such instructions in a book. They’ll become out of date and there are at least a number of platforms irrelevant to your situation. There’s plenty of online documentation to cover installation details and there’s Google to find it.

Throughout the book each language is assigned its very own character that matches its distinct personality. As an example of such characterization, Bruce compares Java with having a rich lawyer for a brother. He was fun when he was younger, but now he’s a black hole that sucks away all the joy.

The book is in my collection for about half a year now. The moment I received my copy I started reading the first chapters about Ruby and Io. I played a while with Io and got to write a small BDD-style test library for it.

Last week I picked up the book again, from the start. In addition to working through the languages by the exercises from the book, I’ll write an article on this blog about how I experienced the language. Also I’ll try to stick with book’s schedule of doing 7 languages in 7 weeks.

Some of the languages I’m already familiar with, others are completely new to me. I am excited to get cracking at the languages and experience how it is again to be a noob, and learn all about them.

I will see you in a week with a post about the first language, Ruby.

Mockito

2013-07-26T00:00:00+00:00

Yesterday I send out a tweet in which I claimed that Mockito and clean tests seem not to go hand in hand. It triggered some reactions on which I’ll elaborate in this post a bit more than in the Twitter conversations.

First of all, in no way I intend to say that your tests are a mess by definition if you use Mockito. I am saying that Mockito doesn’t really help keeping your tests clean. Rather it puts that burden on you.

The responses were from @sdudzin and @jeroenrosenberg, for which I like to thank them. Their reactions included that it’s kind of a bold statement, what is my definition of clean tests, why Mockito breaks that, Mockito provides feedback about design, and that verifying that an invocation did not happen is over specification.

So, yes, my tweet was a bit of a bold statement. It even might be exaggerated a bit. But that doesn’t invalidate it - it is my opinion on Mockito.

The context in which I tweeted this is that I was implementing a dispatch function. Based on some input, a single method must be invoked on a collaborator. As I TDD-ed my way in soon I refactored to a switch statement. I’m not sure why but in one of the branches I forgot to break. The result is a bug - which luckily I caught while writing the next test.

In an after-method I added a call to verifyNoMoreInteractions causing these types of bugs to show up for any test, polluting the tearDown method with this incoherent statement.

Which brings me to my definition of clean tests. In my perception, a clean test simply stated is a test that clearly communicates its intent. It’s name clearly states what is tested, and the implementation clearly describes how it’s tested.

In my opinion adding these statements adds noise to the tests. These statements are incoherent with the tests and are only necessary to work around strange behavior of a test library. Because of this you have to work hard to keep your tests clean.

I don’t really agree on the statement that Mockito gives you feedback on the design. Test libraries don’t provide feedback. Tests do that. Test libraries only provide some convenience. Therefore I like test libraries to be as simple as possible, so the focus always lies on the test itself. Messy tests often tell you the design is messy. However, if a test library makes your tests just a little messier, it takes away this focus making it easier to ignore this feedback.

Obviously I do agree that having many mocks in one test indicates a design problem. In fact, if you need more than one mock in a test, you probably need to refactor. Also I completely agree that verifying for an interaction not to take place is over specification and should not be necessary. This actually leads to fragile tests.

That still leaves the fact that I don’t want my test doubles to ignore calls that didn’t specify myself. As Mockito’s ignorance cannot be switched off, the verification for no other interactions sadly is necessary. It’s exactly this kind of behavior that I don’t like about Mockito. Other examples include that I find setting up doubles with Mockito gets complicated much earlier than with a library like JMock.

This article is no way meant as a rant against Mockito. If you happen to like Mockito, that’s okay. If you don’t know about other libraries, it might be time to investigate some. Personally, at this moment I prefer JMock for test doubles in Java. I find JMock to enable me to clearly express the role of collaborators, while Mockito just makes it possible to setup test doubles. JMock also is much more consistent with other libraries, like RSpec, Jasmine and Sinon.

To wrap up I send my thanks to Siarhei and Jeroen for the conversations we had.

Partial Functions in JavaScript

2013-05-18T00:00:00+00:00

JavaScript (both the language itself and libraries such as Underscore) provides many functions that use closures. A closure is a block of code that is passed as an argument to a function or returned from a function. In JavaScript you pass in that code in the form of a function.

Consider the following example for calculating the 20% tip of an array of checks.

[120, 90, 180].map(function tip(check) {
  return check * 0.2
})
> [24,18,36]

The map function accepts a closure that in the example returns a 20% tip.

Rather than using an inline tip function it makes sense to pass a reference to a defined function, so we can reuse the tip function.

function tip(check) { return check * 0.2 }
tip(100)
> 20
[120, 90, 180].map(tip)
> [24,18,36]

Probably you don’t always want to tip 20%. Sometimes you want to tip a little more, other times you may want to tip a little less. Therefore you want to pass in the percentage to tip as an argument rather than tipping 20% all the time.

function tip(percentage, check) {
  return check * percentage
}

Adding the percentage as argument is of course simple. But now we can no longer pass the tip function into map since it needs an extra argument that map ain’t going to provide. Rather than using an inline function that calls tip with 20%, let’s explore another possibility.

Partial Functions

Partial functions is a functional programming concept. A partial function takes a function and fewer arguments than normal. It returns a function that takes the remaining arguments. When called the returned function call the original function with both sets of arguments.

As JavaScript is a dynamic language that can create functions on the fly, it should be possible to implement this feature.

We need a function partial that takes a function and a variable number of arguments for its input. That’s pretty easy considering that any function in JavaScript takes a variable number of arguments. These arguments can be accessed from the arguments array.

[arguments is not an actual array but an object that supports array lookups. It lacks most array functions (like map), but we can call the functions from the array prototype using arguments as their context.]

Then partial must return a function that takes the remaining arguments and, as invoked, calls the original function with the combined argument arrays. Since partial can only access arguments in form of an array we cannot call the original function (i.e. using parentheses or call) but must apply it instead. This way the array of arguments we pass to apply is mapped to arguments of the original function.

function partial(f) {
  var args = Array.prototype.slice.call(arguments, 1)
  return function() {
    var remainingArgs = Array.prototype.slice.call(arguments)
    return f.apply(null, args.concat(remainingArgs))
  }
}

Using partial the general tip function can be passed into map as a 20%-tip function like this.

[120, 90, 180].map(partial(tip, 0.2))
> [24,18,36]

This partial function only works for pure functions and not for a function that gets needs some state of the object it is in. This is due to the function being applied without a context.

Partial functions can offer a great deal of flexibility and can prevent a lot of adapter functions. Pay attention to the order of your arguments as JavaScript binds arguments by the order they are passed in. The example with the tip function obviously wouldn’t have worked out if the percentage argument was last.

JavaScript: Prototypal Inheritance

2013-05-13T00:00:00+00:00

In my previous article about JavaScript: Objects, Prototypes and Inheritance I wrote using objects and constructor functions to create object hierarchies in JavaScript. I also over-simplified things a little, to make the whole article a bit more comprehensible.

In the article I stated that JavaScript’s behavior for copying objects is different from other prototype-based programming languages, like Self and Io. And it is. But the article is constrained to JavaScript’s classical object inheritance using the new keyword.

Prototypal Inheritance

JavaScript also supports prototypal inheritance. By this I mean inheritance by copying (or cloning) or objects that is typical for prototype-based languages. In JavaScript this practically means inheriting from any object rather than limiting to Function derivatives.

Furthermore, for prototypal inheritance you won’t need the strange (class-like) prototype of a function. You inherit from any object, much like the first example of inheritance I used in my previous article. To refresh your memory, here it is again.

var Person = {
  introduce: function() {
    return 'Hi, ' + this.name + ' here';
  }
}
var bart = {
  name: 'Bart'
}
bart.__proto__ = Person
bart.introduce()
> 'Hi, Bart here'

By changing the object’s internal prototype (i.e. its parent) to Person it inherits all of its attributes. This is prototypal inheritance.

From its early beginning JavaScript supported classical inheritance (i.e. inheriting from a prototype using the new operator). But as a prototype-based programming language it is somewhat awkward to lack prototypal inheritance. Back in 2006 Douglas Crockford wrote an article about this.

His article manifested itself into the core language as the Object.create function. Taking the above example you can make a copy of Person using this function.

var bart = Object.create(Person)
bart.name = 'Bart'
bart.introduce()

The object is identical to the one in the former example and nearly identical to an object created using a constructor.

Indeed, nearly identical. One difference is that JavaScript keeps track of the constructor function of an object. An object created by classical inheritance has its constructor point to the function that created it, while an object created by prototypal inheritance has its constructor point to the constructor of its parent object.

I.e. the following statements would be true given that Person is a constructor function and that Person is the object from the first example respectively.

new Person().constructor === Person
Object.create(Person).constructor === Object

Browser Support

Browser support for prototypal inheritance is pretty good, meaning that Internet Explorer is last of the major browsers to support it (as of version 10). The Object.create function is part of version 5 of the ECMAScript specification and implemented as of JavaScript 1.8.5.

And if you need to support a browser that runs an earlier JavaScript version you can always monkey patch Object, which obviously is one of the major advantages of using a dynamic language.

Performance

But what about performance? JavaScript has been optimized for years on using the new keyword and constructor functions. How does prototypal inheritance relate to that? There one way really to find that out and that would be by testing it.

So I’ve prepared some performance testing and was somewhat surprised by the (very early) results. Not that overall prototypal inheritance is slower than classical inheritance. Like I said, JavaScript has been optimizing the new keyword for years. What surprised me was that this assumption failed for Firefox. It’s also interesting to see how different browsers relate to others, performance wise.

To get a decent overall view it is of course important to get a wide range of different browsers and operating systems. So please run the performance tests on all your browsers, it only takes a few seconds.

The tests and intermediate results are available at: http://jsperf.com/javascript-classical-versus-prototypal-inheritance

So now what?

I believe that both classical and prototypal inheritance have their place in JavaScript development. If you need the speed (i.e. you create many, many objects) you should probably choose using the new keyword. For most other cases I would at least seriously consider prototypal inheritance.

I also believe that most JavaScript code is written in a strict class-based approach. If you find yourself choosing for classical inheritance just because ‘you are used to that’, start practicing with prototypal inheritance only to learn its benefits.

JavaScript has dynamic types, weak types, and no classes. No matter the inheritance approach you choose this does not change. Don’t attempt to change these rules in your code for you will fail. Value duck typing and prototype objects over type checking and class-like object definitions.

JavaScript: Objects, Prototypes and Inheritance

2013-05-09T00:00:00+00:00

JavaScript is a prototype-based object-oriented programming language. The prototype-based programming paradigm characterizes itself by the absence of classes. Reuse of behavior is performed by cloning existing objects, known as prototypes.

Examples of other prototype-based programming languages include Self and Io, and of course other ECMAScript implementations. [As JavaScript is an implementation of ECMAScript most things in this article are true for other ECMAScript implementations too.]

Because there are no classes you get an object by simply creating one. For example, the following creates an object that has a single method.

var bart = {
  introduce: function() {
    return 'Hi, Bart here';
  }
}

This allows me to introduce myself:

bart.introduce()
> 'Hi, Bart here'

Of course this is nice and all but not of too much help. If I want to create yet another person who can introduce herself I have to duplicate the introduce function. Like class-based programming languages group behavior in classes, we want to group the behavior of a person in an object prototype.

The prototype would look something like this:

var Person = {
  introduce: function() {
    return 'Hi, ' + this.name + ' here';
  }
}

To reuse the behavior of this prototype in JavaScript we must manually set its __proto__. [This is not recommended and I’ll get to that in a second.]

var bart = {
  name: 'Bart'
}
bart.__proto__ = Person
bart.introduce()
> 'Hi, Bart here'

Like I said, this is not recommended. I included this example because I want to show that there’s not much magic going on, it’s just all about the objects. The reason for this not being recommended is that __proto__ is the internal prototype used for implementing JavaScript’s inheritance.

In fact, __proto__ had the value of Object before manually changing it to Person. [This is not exactly true, but consider it the truth for now. I’ll reveal the lie towards the end of this article.] JavaScript tracks the internal prototype on object construction.

To get JavaScript to automatically track the prototype, the Person object must be rewritten as a constructor object.

function Person() {}
Person.prototype.introduce = function() {
  return 'Hi, ' + this.name + ' here';
}
var bart = new Person()
bart.name = 'Bart'
bart.introduce()

[Normally I would pass in name as a constructor argument on Person, but that discussion is beyond the scope of how objects work in JavaScript.]

Beside defining Person as a function now, something else changed here. Something important. The introduce function is no longer an attribute of Person but instead of its prototype. This is necessary because when copying from an object, JavaScript limits access to the object’s prototype, not to its own attributes.

The way to copy from objects is by using keyword new on a function. This implies that you can only copy from objects that are actually a constructor and thus objects that have Function in their prototype-chain (and most of the time are direct descendants of it). Because Function is an object itself you can use it like any other object and put attributes on it. But these attributes are put on the constructor and are not inherited by descendant objects. Copies only get access to the prototype. That is due to how JavaScript manages the object’s internal prototype. In the above example, the following is true.

bart.__proto__ === Person.prototype // => true

This might seem a bit strange, but that is only because it is a bit strange. JavaScript’s behavior for copying objects is different from other prototype-based programming languages (like Self and Io). In Self an Io objects have constructors themselves and you can copy or clone (respectively) an object.

I started this article by showing how to manually set up the prototype-chain for an object and that there’s not much special to it. While JavaScript’s management of the internal prototype-chain may seem much like the work of a sorcerer, it’s basically all objects (and some internal state).

In the beginning I also told you a lie. It should be clear by now that the internal prototype of the object was not Object but rather Object.prototype. This is also where the prototype-chain ends.

Explaining JavaScript objects this detailed helped me to really understand how it works. I hope that writing this down helps you getting a better understanding of JavaScript objects, too.

Exceptions: Checked and Unchecked

2013-03-28T00:00:00+00:00

In the Java community there’s a still this disagreement about whether to use statically checked exceptions versus their unchecked opponent. In this article I provide some facts about exceptions and my personal view on using them.

Exceptions

An exception is an event that disrupts the normal flow of execution in a program.

Nowadays almost any major programming language support the raising and handling of exceptions. These languages allow for registering an exception handler (i.e. a catch/rescue block). If an exception is raised the first exception handler up the call stack is invoked.

Because handling an exception always was optional, OCaml introduced exception checkers. This was an external tool that could be used during compilation to fail when no handler for a particular exception was registered.

Growing on this idea compile-time checks upon exception handling were added to CLU and Modula-3. These languages check for each exception that is listed in the throws declaration of a function whether a handler is registered up the call stack. Any exception not listed was not checked at compilation time. C++ has a likewise feature known as exception specifications.

Checked exceptions

Then checked exceptions as we know them today were added in the first version of Java. The difference with earlier attempts is that Java decides to check an exception based on its class hierarchy, while others decide based on function signatures. Therefore a checked exception is an exception that is checked on compile time. A function that might raise it must declare it in its signature. A calling function must either register a handler or declare the exception it its signature as well.

At the time of introduction the strictness of checked exceptions seemed like a good idea. By having to list all the exceptions that are possibly raised in a functions signature, you could never lack a handler.

The problem with checked exception wasn’t visible until late after a project entered maintenance. (Remember, at the time agile wasn’t a big thing and most projects were done waterfall style.) As it turned out any change to a function that might raise a checked exception forces its callers to change too (possibly multiple layers of them). And that’s a straight violation of the Open-Closed Principle [OCP]. Micheal Feathers provides more details on this in an excellent chapter on error handling in Clean Code.

Because of this checked exceptions are a bad idea. After Java no major language adopted the concept (not even C#). And a few years ago C++ deprecated exception specifications in its recent C++11 standard.

Express intent

As unchecked exceptions don’t violate the OCP, does that mean they should always be used over checked exceptions? In Effective Java Joshua Bloch says that if you can reasonably expect your client to recover from the exception, you should consider using a checked exception. Hence any client is forced to deal with the exception.

Although this might be considered a good reason to use checked exceptions (especially at the time this great book was written), personally I see their use more as a design smell.

As Bertrand Meyer writes in his book Object-Oriented Software Construction there are only two meaningful ways that a routine can react when an exception occurs: by failure or by retry.

Using unchecked exceptions a function automatically handles a failure. The failure then is normally passed to some more general handler up in the call stack. Practically this handler logs the failure and display a friendly message to the user.

If a function should opt a retry (in which case you might doubt to use a checked exception) it would help if this was expressed somehow different from expecting a failure reaction. An exception (checked or unchecked) won’t express this intention quite well.

In such case I want to communicate the event through the API. Adding a dependency to the target object (whose member function must communicate the error) helps me reason about what the error means in term of the caller. By doing this I can descriptively name the relationship between the caller and the target, and make it explicit that someone must handle the error.

I hope this might be of help to you and provided you some new insights.

Signing Git Tags

2013-03-14T00:00:00+00:00

One of Git’s interesting features is definitely the ability to sign tags. Using your GPG key-pair you can create a signed tag. Anyone with your public key can verify that you are the true creator of that tag.

Why you should sign your tags

As Git is a distributed version control system (DVCS) you need a to verify the source of a release. Just like you verify checksums of releases you download to make sure it’s the release you expect it to be (you do verify your downloads, don’t you?!), you want to prove that the tag is created by the expected trusted source.

While signing tags is more important in open-source projects, it’s generally also a good idea for your private projects. See the signing as one of the core team members placing stamp in your repository, as they release a version of their software.

Also signing tags is dead easy as I’ll show you next.

Create a signed tag

To create a signed tag all you need is a Git repository and a GPG private key you want to use to sign the tag. Don’t worry if you don’t have GPG set up yet, I’ll cover this a little later.

To make things really easy set your GPG signing key as a global configuration setting. Obviously you don’t need to do this - especially if you have more than one account (e.g. work and private) you want multiple signing keys so set them up per repository (discard the global option).

$ git config --global user.signingkey [gpg-key-id]

Now all you need to do is create a tag and sign it.

$ git tag --sign [signed-tag-name]

Verify a tag

Verifying tags is as easy as creating one.

$ git tag --verify [signed-tag-name]

However, you need the signer’s public key to be able to verify the commit. Most maintainers put their public key in a special object in the repository - an object that can only be reached from a specific tag (or of course its own SHA).

You can make your public key available in a tag as follows.

$ gpg -a --export [gpg-key-id] | git hash-object -w --stdin
[object SHA]

$ git tag -a [object SHA] maintainer-pgp-pub

To import a maintainer’s public key into your keyring you can show the tag.

$ git show maintainer-pgp-pub | gpg --import

Generally it is also a good idea to leave instructions on importing your public key and verifying tags in the tag message.

Create a GPG key

If you don’t have a GPG key pair yet it’s about time to create one. If you’re in doubt most likely you don’t have any but it’s always a good idea to check your list of GPG keys.

$ gpg --list-keys

If you need to, generate a key. Choose the RSA algorithm if you can as DSA only supports up to 1024 bits.

$ gpg --gen-key

As you see using GPG and signing Git tags is really simple, so there’s actually no reason not to sign your tags.

So go and sign and verify!

Coderetreat: Global Day 2012

2012-12-09T00:00:00+00:00

Yesterday, Saturday December 8th, the Global Day of Coderetreat 2012 was finally there. As facilitator of the event in Haarlem (The Netherlands) I was quite excited about this event, and think we it was great and much fun.

Even though we had an event with only 5 people, we really had an awesome day. Throughout the day we did 5 implementations of Conway’s Game of Life, each with different constraints. The constraint for all session was that the universe is infinite (so two-dimensional arrays are probably not the best option).

The constraints in a coderetreat are extreme. This is the purpose of a coderetreat. By taking practices into the extreme you’ll learn when they can be useful and when to avoid them. On each coderetreat people are strongly suggested not to write code like that on the job.

Each session is 45 minutes. When a session ends the code must be deleted. After each session there’s a short retrospective.

Warming up

To get everyone started the warming up didn’t have much constraints. The Game of Life must be implemented using clean code and TDD - like usual. This allowed people to get set and into the mood for the rest of the day.

After 40 minutes I communicated a little change in the requirements. The universe must be 3 dimensional rather than 2d. The remaining few minutes in the session were for each pair to figure out and discuss in how many places they would need to make a change.

Baby Steps

The second session was constrained with taking baby steps. Each pair had to set a timer on 2 minutes, write a failing test and implement it (make it green) before the timer rings. Then they commit, reset the timer, and refactor their code before the timer rings. If they didn’t finish within the timer they had to roll back their changes and start over - taking a smaller step.

To be able to commit changes but still delete the code after the session ends each pair set up a local git repository. As it turned out the pairs struggled a little with git as they weren’t familiar with it. Next time I’ll give more introduction into git, if needed.

Silent Ping-Pong

After a nice lunch, the third session focused on how you document your code using tests. Future-you may not fully understand your test while current-you think it’s fine.

As pairing with future-you is rather hard - or probably impossible if you don’t own a Delorean (and none of the attendees did) - we used the ping-pong pairing technique. One writes a test (tester) and the other implements it (implementer). As you cannot communicate with future-you other than through code, the pairs may only communicate through code in this session. To make the test suite a little more interesting the implementer had the role of evil coder (try to find loop-holes in the tests).

It’s interesting how the pairs had the same experience I had when I did this session on a coderetreat. You get really far without talking to each other. You won’t be discussing design. You just code. One drives the design, the other implements and refactors.

No return values

The fourth session constrains the use of return values: it’s lambda’s all the way down.

People were using Java all day long, which is really quite awful at this exercise because of Java’s lack of lambda’s. For those programming only in Java this really is a great exercise as it helps them understand what callbacks are about and when they can be useful.

On a side note: if you only know how program in Java, learn another language.

Cooling Down

For the cooling down we did a fish-bowl session. One pair was programming on the big screen, the others were watching. Each 2 minutes one of the pair was replaced. No constraints were applied in this session.

This session was a really fun and interesting way to finish the day.

Closing Circle

At the end of the day we all joined together in the closing circle to each tell briefly about their view on the day. For me it was really interesting to hear how people experienced the day, what they learned, what surprised them and what they’ll take with them.

Next year

Global Day 2012 was an awesome day that takes place in an amazing community. Special thanks goes out to Corey Haines and Jim Hurne for all their hard work on raising the bar for coderetreat events and making sure every location in the world could host an awesome event!

The most important thing I’ll take with me to next year is to get more people to attend. Even though coderetreat scales down well it is even better with more people.

That said, yes, I plan to facilitate Global Day 2013 too. Hope to see you next year!

Coderetreat

2012-12-02T00:00:00+00:00

The Sunday after the SCNA conference a code retreat was held at 3 locations in Chicago. I attended the one at the 8th Light office, hosted by Corey Haines.

Coderetreats are really awesome! With global day being in a week from now I like to tell a little about coderetreats and why you want to join us on global day.

So what is a coderetreat?

Besides being just really awesome, a coderetreat is a day of intensive practice, focusing on fundamentals of programming and software design.

The essence is in practice. A coderetreat offers you a safe environment to practice a day long together with other programmers, without having to worry about getting things done.

By doing the same kata multiple times with different constraints, you’ll learn when these constraints can be useful and when to avoid them. Examples of constraints are to not use if-statements, use at most 2 lines of code in a function, or not to use return values.

Why we need practice

In most professions regular practice is mandatory. For example, would you want to see a band that only plays together when they perform live on stage? You would probably only go see them once, and find them rather disappointing.

Programmers also need constant practice to improve their skills. Programming is hard. Becoming good at it takes a lot of practice. And as part of becoming better, we’ll learn about even more things we want to practice. Like a band we also need a safe environment to practice. We cannot experiment in our live performance -our every day job- as it would hurt our customers.

A coderetreat session provides you this environment. Besides practice being important, practicing in a coderetreat is really fun too; program all day with other people, without the urge to deliver!

Update obviously there’s no way you’ll make it to Global Day of Coderetreat 2012, so I deleted the registration information. If you like programming, follow the annual Global Day of Coderetreat event and attend a location near to you.

SCNA 2012

2012-11-25T00:00:00+00:00

The Software Craftsmanship North America (SCNA) 2012 conference was held at November 9 and 10 on the top floor of the AON Center in Chicago, IL.

Proud to be one of the few Europeans to attend this great conference, in this post I write about my personal highlights.

As this post is becoming pretty large I just have to skip over some talks. As the conference was amazing with great speakers, the talks I skip I still found very interesting.

Opening

Micah Martin -co-founder of 8th Light- opens the conference with a brief chat about software craftsmanship and a short introduction to the conference.

"This conference is not about specific technologies or programming languages. This conference is about attitude." Micah Martin

Cory Foy

When Code Cries: Listening to What Your Code is Saying

The honor of having the first talk of the conference was for Cory Foy. In this great talk he discussed code quality and communication.

A really interesting point that he made is that the quality of code is not something static. Code quality happens at the interaction point of the code base: when we try to understand it, modify it, or when the code is used by people. I.e. quality code is code that actually gets used and is a joy to work with. Code cries when it is not understood at the interaction point.

Cory made a comparison between the course of the Mississippi river and the course of design. We tend to feel the need to control things. We want to control the course of the Mississippi river even though it is natural for a river to change course over time. We just happen to like how it flows next to Orleans and we do anything we can to prevent it to naturally flow another course. We fear the natural change of the river.

Much in the same way we tend to control software design. The first time you use TDD to drive the design is quite fearful. After you get used to it and allow letting design flow more naturally, you learn that tests drive towards a design that is fully applicable to the situation on hand.

Further Cory explained the Foy-Z technique, a model that captures the way people learn to program in a simple yet elegant way. The model moves from cognitively undemanding (i.e. without clients, dead lines, etc) with context embedded to a cognitively demanding with context reduced, in a few steps.

With a few quotes from Christopher Alexander’s classic The Timeless Way of Building, this fantastic talk by Cory belongs to the best of the conference in my opinion.

Sarah Allen

Crafting Software Products

This talk is about product development. Sarah Allen told the story of one of her customers asking for a mobile app. Rather than coding up the MVP (Minimum Viable Product) she took a much different approach.

Together with her customer she examined the target audience of the mobile app and what they would expect of the app. Using ads to draw attention to prototypes (mainly static images and web pages) and various measurement techniques help to learn the target audience and their expectations of the app.

"Potential users should not be the target audience of an app, but those who will love and crave using the product" Sarah Allen

She referred to this as test-first product development: to start with a hypothesis, then test it, analyze the test results and learn from it. Then adapt to what you learned and reiterate, leading from a belief (the hypothesis) to knowledge (learned from the tests).

Adewale Oshineye

What Happens When Software Craftsmanship Meets User Experience?

Adewale Oshineye talked about usability of the tools and APIs developers use. Even though most tools are well designed, well made, very useful, they are not very usable.

Many tools developers use on a daily basis -like Git, Vim, Emacs- suffer usability. We use them anyway because they are the least worst tools out there.

Using the example of a glass door with the text ‘push’ on the one side and the text ‘pull’ on the other he makes the case to fix usability rather than document how to use it.

"A poor workman blames his tools" Adewale Oshineye

It’s very interesting because if anyone should be able to fix these problems in software it should be the people creating these products. Yet most APIs (and also tools) are designed from the perspective of its creator, not of its user.

Robert Martin

The Reasonable Expectations of Your CTO

Uncle Bob goes over the attitude of professionals. Simply stated, professional programmers:

don’t ship shit
know when to say no
don’t lie about estimates
are eager to learn
invest time to improve their skills and work on their craft

The most reasonable thing for a CTO to expect of programmers is for them to behave professional.

In addition, a CTO can reasonably expect work to be done at a stable speed. Don’t start very fast and slow down along the way by making a mess. Also be fearless of the code you create; fearfulness is unacceptable. This means you can make changes to code easily without breaking it, which implies good design, clean code, and a healthy safety net. The reason we call this SOFTware is that it is easy to change.

Another thing is that QA should not find anything. And when they do the team should find out how that happened, learn from it, and prevent it to happen the next time.

"QA is not your debugger" Uncle Bob

Having read his book on professionalism - The Clean Coder - the attitude that can be expected of professionals was not new to me. Still talks from Robert Martin are always great. He’s a very experienced speaker, and you’ll learn something more about science every time.

Jen Myers

Developers Can’t Understand Design*
* and other completely mistaken design myths

Jen Myers tried to clear many of the common misconceptions about the work of a designer. Using a lot of quotes and great stories she made clear that designers do not just make things pretty, they make things work.

Using their own set of tools (like fonts, color theories, ui patterns, user testing, etc) designers make a product work for the intended audience. Therefore it is important for designers and developers to work together and have much interaction, unlike the common task handed over to a designer by a developer: “it works, please make it look pretty”.

She introduces the concept digital-native designer, who knows how to do visual and user experience design, and can implement it. For developers who want to learn about design, David Kadavy’s book Design for Hackers: Reverse Engineering Beauty could be good start.

Michael Feathers

Testing Revisited

In this wonderful talk Michael Feathers revisits testing. He starts with some sweetness.

Tests are sweet. They help us to think, precisely detect change, serve as documentation, and to get to done. Tests give us a hard line, 0 or 1.

Tests that serve more than one purpose may become harder to change. As with all code, a test should focus on one thing. This is where the SRP (Single Responsibility Principle) is applied in tests too.

He steps through some common problems with larger test suites.

Some teams phase that they are held hostage by their tests. Instead of supporting them to change the system these teams suffer from many failing tests if they try to make a change. This makes modifications to the system extremely hard.

Also tests can cause systems to increase heavily in build time. At first this does not seem like a big deal but as the system grows it becomes more and more time consuming to run all tests, to end with not running the tests anymore as it would take too long.

To the build time problem Michael proposes an interesting solution: to set a build budget. This forces the improvement of tests to not run out of the budget. It also makes a good case to do this to your customer.

Further he talks about deleting tests that are no longer of use or not really necessary. And to option of maintaining multiple suites and run only some of them.

Gary Bernhardt

Boundaries

Gary Bernhardt kicked off the second day of SCNA with his talk about boundaries and isolation.

Code is often tested in isolation by using stubs and mocks. While these serve isolation, they might be different from the real world. Using tools to check test doubles against a contract (such as rspec-fire, static typing, or integration tests) allows for writing tests that help catch these differences.

In this talk Gary shows a natural isolation of code: through value boundaries. In his presentation he uses examples from a Twitter client he wrote and talked about in a few DAS episodes. If you haven’t seen his screencasts, make sure to watch them soon, they are really awesome!

He separated the program into a functional core that contains the intelligence, surrounded by an imperative shell that brings in the dependencies.

After going through the following table of known paradigms, he introduces the name for this approach that combined functional and OO: Faux-O.

	mutation	data & code
procedural	yes	separated
OO	yes	combined
functional	no	separated
faux-O	no	combined

Functional code is much easier to test than imperative code as it has no side effects. You can test a function by its values, which are the natural boundaries of a function. Values also afford shifting process boundaries, of which the actor model is an example.

Sarah Gray

Visualizing Enumerable: Own Abstract Concepts Through Physicalization

Sarah talked about beginners, and how they require different kinds of help based on their background. She shared her stories of mentoring different beginners, and how often she’s reminded of the difficulties beginners face.

What help for example is it to point a beginner to the Ruby Docs? She used the example of the Enumerable select method. In this definition of select is so much context implied, that it would probably blows a beginners mind.

Sarah’s solution to this was to visualize these concepts by drawing pictures of what the lines of code actually do. She showed a picture explaining select with her select-cat, selecting balls by color and dropping everything else in the ocean.

Even though select-cat is not really friendly to the environment by dropping stuff in the ocean, she really uses a great way to visualize concepts for beginners.

Sarah also talked about how people learn in different ways, and that some require more visualization than others. I found her talk one of the most inspiring of the conference.

Kata Battles

On each day a kata battle was organized. This is a contest where both contestants would perform the Coin Changer kata live in front of all conference attendees. The winner was determined by the audience based on speed, style, creativity, etc.

The battles were a lot of fun to spectate, and I can only imagine also fun to perform (although under the huge pressure of the watching audience).

Fish bowl

On day one during lunch a fish bowl was organized.

Programmers rotate in as pair working on a software exercise. The exercise was the Clojure version of the Gilded Rose kata. The audience can observe, heckle, or help solve the problem.

Fish bowls are both fun and instructive.

Jeopardy

On day two during lunch a Jeopardy of software trivia was organized. The contestants were Cory Foy, Corey Haines, and Gary Bernhardt.

In conclusion

When I went to Chicago to attend SCNA I had high expectations of the conference. The schedule showed really experienced speakers, and some great entertainment. And SCNA managed to completely beat my expectations.

I find this an amazing conference, with very experienced speakers, great entertainment - I didn’t even mention the brewery trip, which was lots of fun! - and very nice, like-minded people.

I’ll try very hard to attend next year, so hope to see you there!

Continuous Delivery Workshop

2012-09-28T00:00:00+00:00

On Saturday September 15, I attended a free workshop about continuous delivery by Robert Chatley, in Amsterdam. This workshop was organized by Ugly Duckling.

Continuous delivery is most about getting feedback much faster than with traditional (i.e. conservative) delivery strategies. Traditionally releasing is error-prone and time consuming, and often quite stressful. Teams and project managers want to release as few times as possible because it is so hard. The agile mindset teaches that if something is this hard you should it more often (practice it) so that you get better at it. Continuous delivery adopts this mindset and is about removing the bottlenecks from the delivery process, so that you can deliver in minutes rather than hours (or sometimes even days), at any time.

The workshop was really fun and introduced the concepts of continuous delivery in a playful manner. We were given a small Java application, containing a web server, HTML interface and API interface (which I’ll come back to a little later). As a good workshop befits we worked in pairs.

About the first thing you always need is some sort of source code management system. We used GitHub, created an empty repository and pushed the provided source code as first commit.

Then we set up continuous integration using TeamCity, on a provided virtual server. I hadn’t seen much of TeamCity before this workshop and it’s ease of use really quite surprised me. Much more so than Jenkins ever did. TeamCity lives up to the high standards of its creator JetBrains (known from tools like IntelliJ and RubyMine). To make it ever better TeamCity comes with a free professional license, allowing up to 20 build configurations per server.

To deploy the application we used Heroku. With Heroku you can easily deploy a Ruby, Clojure, Python, Scala, Java, or Node.js application. To deploy something you push your code to Heroku using Git. I.e. you add your Heroku deployment as a remote and git push to it. If you don’t know Git that well, see git-remote(1).

To make deployment even more easy (and to allow for automatic deploys later on), we set up a deployment build in TeamCity. We cloned the GitHub repository on the continuous integration server, added the Heroku remote and set up a TeamCity build configuration to run git pull and git push heroku master.

Now this may seem like a lot of work, it actually is quite trivial to set up and use. But continuous delivery is about doing this continuous and the way to do something continuous is to automate it.

To automate deployment our application when something is changed (i.e. changes are pushed to GitHub) we used TeamCity to trigger the deployment build after a successful continuous integration build. This continuous integration build was already set up to trigger on SCM changes.

Now only a few minutes after pushing changes to the GitHub repository the changes were deployed in production.

When you have a deployment chain like this it is a good idea to have the possibility to rollback a deployment. With Heroku you can perform rollback from the command line (heroku rollback). To make rollbacks more accessible a TeamCity build configuration can be created to perform the rollback.

After we did all this, it was about time to see the continuous delivery of added value in action. The organizers used a nice way to show this. The application we were handed earlier had an API that allowed to register the application as player in a game. Once the game started your application got lots of different questions to answer. You scored on each right answer, lost points on each wrong answer, and neither scored nor lost when your app didn’t know (yet).

To play the game we needed to implement the answering of certain questions. To get the questions we watched the application log from Heroku and read the requested URLs from it (also some logging could be added to the application by printing to stdout, but we didn’t need that).

We choose to push changes after each implemented answer, so that we gained points (i.e. increased our business value) quickly. We also used TDD to implement the answers. We wanted to go fast, really fast, to score as much as we could, so we couldn’t afford not to use TDD. As we went, we wrote a failing test, fixed the test, refactored, and committed, and repeated this this cycle until we got a feature fully implemented. Then we pushed the changes causing a production deployment within a few minutes. At the end of the game we were ranked second.

After discussing the different approaches to the game it was nice to see that almost everybody used some form of tests to implement the features, and that most of those who didn’t write tests were ranked very low.

While playing this game we used the deployment pipeline to go all the way into production. This is useful to get as much feedback as possible, but not a deal breaker for using continuous delivery. Once you’ve automated the deployment pipeline, you control where you want automatic delivery (on changes) to stop. This should be an environment from which you get feedback (i.e. no development environment) but depending on the project might be some test environment for your customer rather than production. This can even put the control of production deployments at your customer. If he or she approves of the changes he or she can push a button to instantly deploy the changes to production. And if the changes are not well-received, you’ll get that valuable feedback.

The workshop made me really enthusiast on continuous delivery. To learn more I bought the book Continuous Deployment by Jez Humble and David Farley.

As a last note here I like to thank to Ugly Duckling for doing this great workshop. They were very well-prepared, including providing a virtual server for each team. Thank you for doing this free workshop it was a free Saturday well spent!

Professionalism

2012-07-12T00:00:00+00:00

Last night I read a tweet from Alan Cooper, the father of Visual Basic. His tweet read:

IMHO, the problem is that people (managers) keep looking for shortcuts. THERE ARE NO SHORTCUTS IN SOFTWARE!!

Very good point on the shortcuts. Shortcuts in software always turn out to be detours. By taking a shortcut -no matter how small- you always delay your team and almost always in the very near future. You don’t want to be responsible for your team’s delay, do you?!

But this tweet should not only mention managers as the people looking for shortcuts. Isn’t the programmer the one that takes the actual shortcut? Sure thing it’s more convenient to blame someone else. But the programmers are the ones taking the shortcuts, not the managers.

Though managers can be quite pushy, it’s not fair to only blame them for any taken shortcuts. When a manager tries to push for a shortcut, the programmers have to say no. And they must stick with it, as a team.

A professional programmer knows not to take shortcuts. He or she would try to find out the reason why the manager is pushing for early delivery and what he want it for. And a professional manager would openly discuss his needs with the programmers. Together they can work out a solution that is mutually agreeable.

If you want to learn more on becoming a professional programmer -and you should- make sure to read The Clean Coder, an excellent work by Robert C. Martin.

Building Web Services with Haste

2012-06-12T00:00:00+00:00

A few weeks ago a customer needed two web services. One of them needed to return an image, the other some data in XML document form. The contract of these services didn’t really matter. And of course, haste was required. They had other companies who wanted to start their integration with these services.

Haste

It’s quite common that haste is required when adding value to software. Most of the time customers are in a hurry, meaning that they want to add the value as soon as possible.

This shouldn’t be thought of as annoying. In fact it can be rather helpful to know that haste is required. It means that there is something to focus on first.

You must never deal with haste by rushing into an implementation. Take the time to interact with your customer and end users. That allows you to do what’s necessary, and doing it right the first time.

We started on these web services with an attempt to define their contracts. These contracts are a great start for communication with the end users, the parties waiting to integrate with the services.

Then we left the web service contract part for what is was, and implemented the use cases. We didn’t bother to look at the web service view once. This put focus on the use cases, while we kept all options open for feedback on the contract.

Feedback-First

After a few days the use cases were implemented. So we could continue with the view layer. In these few days we got the first feedback on the contract. We adjusted the contract and implemented the web services view on top of the use case implementations. We implemented these web services feedback-first.

Sure, I know the style we used to build the services is called contract-first. But defining a contract first, is all about communication and early feedback. It well-supports the lean principle “everybody, all together, from early on”. To emphasize this I like feedback-first in this post.

By the way, you should use contract-first development style for any service you build. I don’t elaborate on all the benefits here - use your Google skills to learn about this style.

Deferring the view

Now it’s easy to say that you should first define a web service contract, then leave it to generate feedback, and later on implement it on top of a web service agnostic application, after you got the feedback. But in order to do this, your architecture must allow you to defer the view. That means that your view must not be an integral part of the application, but instead, a plugin.

If you defer the view until you know how to build it, you can focus on getting the functionality done. And after you have sufficient feedback on the view, you implement it and plug it into the application.

In fact, deferring the view is only the technical consequence of deferring the implementation details for a particular stakeholder. And to be able to do that, you must make sure that the various stakeholders in your system are separated from each other. E.g. the use cases are driven by a different stakeholder than the web service view, so they must be decoupled.

Architecture

So the architecture must partition the system so that different stakeholders are separated. Taking the image service as example here, on forehand I saw three stakeholders: our customer, who drives the use case, the parties integrating with the services, who drive the view, a virtual data stakeholder Hence they served the primary partitioning of the system. For the sake of this post, the virtual data stakeholder is left out - in the actual implementation it is decoupled by an image entity gateway.

The next picture gives you an idea of the partitioning.

The ImageWebService does all the translation of the web request into a vanilla data structure and passes it to the control object, Viewer. The Viewer performs all the use case interactions, and displays the image on the interface object, Display, which in fact is implemented by the ImageWebService (or some class close to it).

The partitioning allows to defer the implementation of the view and the data layer. It allows you to focus on use cases first, without bothering about the data or the view. Plus any part of the system whose implementation can be deferred, can also be changed or even replaced without affecting the rest of the system.

Learn more

Uncle Bob did a Clean Coders episode on architecture; watch it if you haven’t. The architecture he talks about is much inspired by the work of Ivar Jacobson, which he wrote about in his book Object Oriented Software Engineering: A Use Case Driven Approach.

About Code Coverage

2012-06-05T00:00:00+00:00

The discussion about the number of code coverage you must have, seems to gain more and more attention lately. On the one side there are the people who write their tests after the fact and their managers, claiming numbers that vary more or less in the range of 60-80%. On the other side there are the people who do TDD, who wonder how you can get anything less than about 100%.

Sure thing I am from the TDD camp. But I’m not going to write another post that claims that you must have 100% code coverage. Well, I might just a bit, but I’ll provide good reasoning. Moreover I want to make the point that putting down any number of coverage that is sufficient, does not improve the actual coverage.

A project I recently worked on, the goal was set, of course after a lot of discussion, to 80%. Project managers interpreted anything less as lack of quality, and anything more as waste (or as they called it, ‘overkill’). As a result I was asked why my code was about 95% covered by tests. In their eyes I was ‘wasting time by writing more tests than necessary’. I am anything but unwilling to cut waste, and I asked what 20% of the features didn’t matter, so I could cut even more waste by skipping the tests and the production code.

As you might expect, the project manager didn’t like any bit of that question. ‘Of course the features matter, it’s just that testing them is overkill’, he said.

So why do I think discussing a target number of code coverage (goal) for a project, or worse, in general, is nonsense? First of all, the goal is not something to discuss about. The only possible goal is 100%! Sure, 60% is a milestone, and so is 80%, as is 90%. But the only goal you can ever set is 100%.

And how could it be anything else? Setting the goal to anything lower is an obvious statement that a part of your code doesn’t matter. A vicious statement that for a part of the customer’s or end users’ problems, it doesn’t matter if they get solved. If that’d be true, why waste time on solving these problems in the first place? Everything you build adds value to the system and therefore the goal is to test it, or you shouldn’t be building it in the first place.

You probably won’t ever reach 100% code coverage, though. On a class level you will, but not for an entire project. In fact, I find projects with 100% overall coverage quite suspicious. Such a project smells like the focus was on gaining test coverage rather than using tests to drive the system.

I use TDD for a while now. And I use code coverage to look for problems in my tests. Yes, that’s right, I don’t care about the number. Normally my coverage is about 95-99%, but again, I don’t care. What I care about is the feasible coverage, and make sure that I covered everything feasible. And I use code coverage tools to look for any weak spots. But that I’ll discuss in another post.

With feasible coverage I refer to the code that you can possibly test. There will be some lines of code in any project that you cannot get under test. An example is I/O in java. Any interaction with I/O requires you to handle IOException. You handle it and throw a properly named runtime exception instead of the checked one (and of course include its cause). You cannot test any of the possible I/O failures, and you should not. Oh sure, there are some ways to throw these exceptions; you can use one of the nifty mocking frameworks to apply all the reflection necessary to be able to throw that exception. But again, you should not.

It doesn’t matter that you cannot get every line of code under test. What matters is that you maintain a constant drive to improve the tests and the production code. Therefore the only sensible goal of code coverage is 100%, and again, we’ll never reach that goal.

And this is why I find the discussion about a level of coverage to be wrong. The behavior you stimulate by setting a minimum amount of code coverage, is to reach that level no matter what. This leads to writing bad tests - e.g. tests without asserts. These tests will only get in the way when you make changes to the system rather than helping you to make changes. In an earlier post I wrote about the art of testing and how the primary purpose of tests is to help you refactor. As soon as tests stop helping you modify the system, you should rethink your tests!

Life's not about Frameworks

2012-05-24T00:00:00+00:00

Last week I came across a post on Java Code Geeks about how “ORM haters don’t get it”. The author rambles on about how the arguments of the so-called haters are wrong, and he explains his understanding of ORM and how he claims that without using ORM from the start, every project eventually ends up with a home-grown ORM-like framework.

I don’t place myself in a group of ORM haters, and frankly I have to agree with the author that a lot of bashing against ORM is going on lately. Partly this is because other, more simple, techniques became available to interact with the database, such as the Active Record. Another reason is the trend of using NoSQL over relational databases.

Whether or not you use ORM, Active Record, NoSQL, or any other way to serialize data, is up to you. This post is not about data access specifically. What I want to talk about is the fanatic embracing of frameworks. We tend to put frameworks and other tools (such as the database) at the center of our application. Uncle Bob blogged about how the database is just a detail, last week. I want to include frameworks in this discussion.

Frameworks - most of them anyway - ease the implementation of application components and features. Sticking to the ORM example, ORM helps us to interact with the database by communicating with it in type-safe data structures. Another example is Google Guice that, though not in all cases, eases applying the Dependency Inversion Principle. These frameworks are indeed of great help to us, programmers. If you need ORM you should use an existing framework; it makes no sense to implement ORM yourself.

But that doesn’t mean these frameworks should be at the center of our application. An application is always about its use cases. Neither the customer nor its end users care about frameworks. Frameworks are implementation details, and use cases are at the center of our application.

How many times have you told a customer or end user, that the changes they want are not feasible because the architecture or a framework does not support it? The answer, most likely, is too many! Customers and end users do not care about implementation details, and they shouldn’t. They care about their own needs. The end users care about their processes and how to ease them, while the customer cares about its market.

So, then how must we work with frameworks? The answer to that is simple; do not embrace them! We should not trust them with our lives, nor with the life of the application! We must keep all frameworks at arms-length, keep them as details to the implementation.

If we keep those frameworks at arms-length, we can replace them when they no longer support our customer and end users. And I mean all frameworks, no matter how unlikely to change. If the customer or end user wants changes that the framework is not capable of, we replace the implementation with one that supports those changes. We should never have to tell the customer that something is not possible because of the frameworks we chose. It’s our responsibility to keep the options open, support and embrace future change, by deferring decision as long as we can. Any decision that you can defer, is a decision that you can change!

I strongly suggest you to watch Clean Coders episode 7, about Architecture, Use Cases, and High Level Design. In this episode, Uncle Bob explains why a good architecture maximizes the number of decisions not made.

From now on, be very careful with the frameworks you use, don’t trust them with your life, and make sure you can replace them whenever necessary!

The Art of Testing

2012-04-30T00:00:00+00:00

In a few previous posts I talked about the importance of unit testing and Test-Driven Development (TDD).

TDD helps us to ensure we’ve covered everything in our system by tests. Sure thing if you never write more production code than to satisfy a failing test, you’ve always covered all your production code by tests.

But how do we write a good suite of tests? How do we ensure that our tests help us maintaining the system? And remember, in agile development maintenance starts after the first successful compilation.

Purpose of tests

To know how to create a good suite of tests, we first must understand our expectations of these tests. What is the primary reason we need a good suite of tests?

The primary reason is not to prove that our code works. There are many ways to prove this, including an army of QA staff. Sure, it’s unprofessional to not know if all your code works, but it’s not the primary reason for our suite of tests. Why do we need this suite of tests? What do we expect from it?

The primary reason for a suite of tests is to allows us to refactor! The suite of tests eliminates the fear of changing the code. With a good suite of tests we can refactor the code while having continuous feedback if we broke something.

Good tests

A good unit test is Fast, Independent, Repeatable, Self-Validating, and Timely. For details on this F.I.R.S.T. acronym, read Clean Code by Uncle Bob, Robert C. Martin. Seriously, if you haven’t read it yet, order a copy now and come back after to finish reading this post.

Tests that follow this acronym, support refactoring the production code quite well. But there’s more to create a sustainable suite of good tests. Not only is a unit test independent of other unit tests, it also validates something that no other test validates. In order words, a single feature of the system (under test) is unit tested once.

Furthermore, tests must be stable, meaning that things that things that commonly change should not be asserted. E.g. asserting text on a screen leads to fragile tests; whenever the text is changed, the test fails. It’s better not to assert this text, but stub it or only assert that some text is printed.

Design feedback

Tests not given feedback about the behavior of the system, they also provide feedback about its design. Let me clarify this with an example.

@Test public void shouldGetWeekOfYear() {
  int weekOfYear = Calendar.getInstance().get(Calendar.WEEK_OF_YEAR);
  assertThat(getWeekOfYear(), equalTo(weekOfYear));
}

As you can see the test determines the current week of the year, and then asserts the getWeekOfYear(..) function returning the same value. Most likely this function determines the week of the year pretty much the same way.

Quite often I see constructions like this; tests that repeat the production code to assert its behavior. Most of these are written after the fact, but either way this is a clear symptom of bad design.

Understand design feedback

To find a better way to test this function, we must understand the feedback from this test. We can hear it scream bad design at us, but what’s wrong with the design for this function? Isn’t the function too trivial to be burdened by bad design?

Remember the Single Responsibility Principle? It teaches us that each function in a system must do one thing, and one thing only. This function getWeekOfYear() in fact has two responsibilities; it determines the week of the year, of the current system date. Let’s see what the test would look like when we extract this second responsibility.

@Test public void shouldGetWeekOfYear() throws Exception {
  assertThat(getWeekOfYear(date(2012, APRIL, 30)), equalTo(18));
}

This is much better. We fixed the function’s design by extracting the extra responsibility from the function, and the test now is much more expressive. Note that I used a function date(..) to get a specific date, to improve expressiveness somewhat more.

Don’t fix the symptom

But wait a minute. It’s nice and all that the test now is more expressive, and that this specific function is left with only one responsibility. But now it became responsibility of the caller to determine the current date. How’s that better than having a low-level function to do two things?

Indeed. Our test helped finding a much more serious design problem: the concept of date/time is not properly separated in this particular system. Now that we’ve got this design flaw clear, we can fix it.

By the way, I’m not suggesting that you create your own date and time abstractions, but access to the current date/time is a separate concern that must not be spread throughout the system. We can introduce a Clock that gets the current date/time, and use this clock to pass the current date into the getWeekOfYear(..) function.

Conclusion

When you have trouble creating the scenario for a function to be tested, listen to this feedback from your test, take the time to understand what exactly is wrong, and fix it. This way tests help you from early on to separate concerns by constantly providing feedback about the system design.

Because tests given you loads of design feedback, writing tests after the fact is not really useful. If you get this feedback after you wrote the production code, you either have to rework the production code, or worse, ignore the design feedback. Writing tests after the fact therefore makes tests more likely to fail even on minor changes to the system, and thus they support but our primary purpose of tests: the ability to refactor!

Legacy Code: Treat it with Silk Gloves

2012-02-12T00:00:00+00:00

Legacy code is tough to deal with. It may be the toughest thing we have to deal with in our everyday jobs.

Before we start, let’s align our definition of what legacy code is. Legacy code is untested code. Legacy code may be old, but not necessarily. Unfortunately some projects are actively producing legacy code.

Most programmers have to work with legacy code frequently. Even worse, they often must make changes in the legacy code. They must fix bugs in the legacy code, or change it to implement new features. At the end of last October I started working on a project that at the time, fully consisted of legacy code. I won’t share any specific details about that client and their projects, but I like to share how we work with the legacy code.

First of all, as a general rule, we try hard not to change the legacy code. If a feature can be implemented without changing the legacy code, implement it with completely new code. And of course, write it using TDD.

Only if this first rule does not apply - i.e. when we must change the legacy code - we continue to think about making some changes. And before making any changes we bring the legacy code under test - or at least the parts where the changes will be. This is the hardest part. I won’t go into any details about how to bring it under test here, but I strongly advise reading Michael Feathers’ excellent book Working Effectively with Legacy Code. In his book Michael presents a large set of strategies that will help you to get your legacy system under test.

Testing the legacy code mostly is time consuming. Still it must be done, there’s no alternative. We can’t fiddle in some changes and only make things worse, we’re professionals. And besides, it wouldn’t help at all. I’ve seen lots of changes fiddled into untested code, almost all with the same result: the introduction of some or even many defects. But like I said, we’re professionals, we don’t fiddle around in untested code.

When the legacy code is under test and we have a sufficient amount of coverage, we can refactor. Because the code is under test we are confident that when we break something, the tests tell us so we can undo our last change. We are no longer afraid of changing the code.

After the code is refactored, the changes can be implemented using TDD. Focusing on not changing the legacy code, doesn’t mean that it’s better to duplicate code or features. It means that legacy code should not be changed if you can avoid it. If you must reuse some feature already implemented in legacy code, you bring it under test and refactor to reusable components.

Never ever try to fix unreported issues in legacy code, even if the change seems so trivial. Sometimes code can be disguised like a bug and even smell like one, but in the end it may turn out to be a feature of the legacy code.

A good example of such a feature in disguise is one I heard in a chat with a colleague last week. He told me that on his project they face a lot of nasty legacy code, and how one of his coworkers changed a string comparison by reference (==) to compare the strings using equals. The change made it to production and, as you may expect by this point, ended up to cause a production defect. The comparison by reference, no matter how much of a code smell, was a feature in disguise and shows how dangerous it is to change any untested code based on assumptions.

To fix the production defect, they reverted the change and put a comment in the code to tell others not to change the comparison. By doing this, they left the code in worse shape then they found it. They added a comment to express their fear of changing it.

Instead, by the time the defect was reported, they should have started writing tests. Clearly they were changing code they didn’t understand. Whenever you must change code you don’t understand, you must refactor first. No matter how trivial the change! And to refactor legacy code, you must get it under test first.

This is how I would like everybody to treat legacy code: with extreme care. I hope this post will be of help to you, and remember: never be the one that creates the legacy code! There is no reason not to test your code, ever.