Hidden Benefits from Commenting Your Code

In his famous book Refactoring: Improving the Design of Existing Code Martin Fowler mentions comments as a possible code smell: if you need comments to explain what your code does, try to refactor it so that the comments become superfluous. Uncle Bob also mentions this same principle in his Clean Code book: most comments are bad comments, typically redundant or misleading.

I largely follow this reasoning. However, I've also seen an interesting benefit from trying to comment your code, especially with higher level comments, for instance JavaDoc comments on classes. Writing these kinds of comments forces you to explain things like the purpose and responsibility of the class. Personally, I've had several occasions where explaining things in plain English highlighted problems. If it's hard to explain what the class's purpose is, or what the reasoning behind the class or method names is, this typically means you have code problems. Maybe the class has unclear or too many responsibilities? Maybe the class name doesn't communicate its purpose well? Maybe the class's role in the system is confusing, and so on.

Nowadays, I try to keep comments in internal code to a minimum, but I do try to write a brief JavaDoc style comment for each class, forcing myself to go through that explaining exercise I just mentioned. I do the same thing for methods that are part of a public API: they also get brief JavaDoc style comments.

Page Flow Challenge Clarification

The Page Flow Challenge in my previous post warrants a bit of clarification. A few readers had questions about the usefulness or sanity of certain steps in the test scenario. I'll go through the scenario and highlight what is being tested in each of the steps.

1. Start the page flow
2. On Page 1, enter "foo" and click Next
   You should end up on Page 2 where "foo" is displayed as input from page 1
3. On Page 2, enter "baz" and click Next
   You should end up on Page 3 where "foo" and "baz" are displayed as input from the previous pages
4. Click the browser back button
   Without any browser warnings you should end up on Page 2, where "foo" is still displayed as input from page 1

The first part of the scenario sets up a basic set of data associated with the page flow: "foo" and "baz" at this point. At step 4 we're using the browser back button to backtrack. This tests POST-REDIRECT-GET behaviour: if the application does not use POST-REDIRECT-GET, the browser will show a warning before displaying Page 2 again. This is so because Page 2 is the result of posting the HTML form on Page 1, and modern browsers never automatically resubmit post data. Essentially we're protecting the user from accidental resubmits. Not having the browser show ugly warnings about resubmitting also makes the application feel stable and trustworthy.

5. Right click the Back link on the page and do Open Link in New Window
   The new window should show Page 1 where "foo" is displayed in the input field
6. In the new window, enter "kit" (replacing "foo") and click Next
   You should end up on Page 2 where "kit" is displayed as input from page 1
7. In the new window, on Page 2 enter "kat" and click Next
   You should end up on Page 3 where "kit" and "kat" are displayed as input from the previous pages
8. In the original window, click the browser refresh button
   Without any browser warnings you should see Page 2 again with "foo" as input from page 1

Step 5, admittedly a contrived step, simulates the user branching off the page flow into two separate windows. The point of doing this is testing whether or not the two branches are isolated. In steps 6 and 7 new data is entered ("kit" and "kat") that potentially interferes with the previously entered data ("foo" and "baz"). Step 8 verifies that no interference happened: the page flow in the original window still has "foo" as data.

9. In the original window, enter "bar" and click Next
   You should end up on Page 3 with "foo" and "bar" as input from previous pages
10. In the original window, click Finish
    You should end up on the start page of the application: the page flow has finished
11. In the original window, click the browser back button
    You should either receive an error immediately, or when you try to resubmit by clicking Finish again

In step 9 and 10, we complete the page flow and finally submit values "foo" and "bar". At this point the page flow has finished. The state and navigation management of the framework should have invalidated the page flow at this point, making it impossible to resubmit. Step 11 verifies this: if you go back using the browser back button, you should not be allowed to resubmit. This part of the scenario tests intentional double submit protection (classically solved with something like a synchronizer token).

12. (For extra bonus points) In the new window, (where you still are on Page 3) click Finish
    You should receive an error of some sort (potentially a automatic page flow restart)

The final step of the scenario tests whether or not the framework tracks the two branches of the page flow as being part of the same original conversation. This is an advanced form of double submit protection. In most cases this is a beneficial extra protection. In some cases this might not be what you want.

To make all of this a bit more tangible, let's suppose this is an airline ticket booking flow. The scenario would go something like this:
In step 2, on Page 1, the users fills in basic traveller information (name, age, seating class) and advances forward. In step 3, on Page 2, he selects a seating preference. Page 3 in step 4 shows an overview of the captured information and asks for confirmation. The user notices that he made a mistake and backs up.
Steps 5, 6 and 7 has the user thinking he might want to try a seat in a different seating class and compare the prices. Page 3 in step 7 displays an overview of the price with the new seating class.
In step 8 and 9 the user completes the page flow with the original seating class. On step 10 he compares the prices of the two alternatives and decides to submit the first of the two alternatives.
Steps 11 and 12 verify that the user cannot accidentally book two tickets, either by resubmitting the first alternative or submitting the second alternative. If he really wants another ticket, he'll have to start the ticket booking process (the page flow) again.

I hope this clarifies things a bit and shows that the test scenario is actually somewhat realistic.

A Page Flow Challenge

In this post I setup a page flow challenge to find out if the new or up-and-coming web frameworks handle page flows well.

Why am I doing this? Over the last several years I've seen quite a few blogs and discussions where people talk about how web framework X handles page flows, typically comparing the page flow functionality in framework X with Spring Web Flow. The problem with most of these comparisons is that they focus on just two parts of the page flow puzzle: expressing page flows through some kind of DSL, and reusing page flows as modules throughout the application. There is an important third piece: managing navigation and the associated state. The reason why this is often overlooked is probably because the navigation and state management is completely automatic in Spring Web Flow, so many people don't even realize that it's there.

While attending a session on the Lift framework last week at Devoxx09, this issue came up again. Somebody (not me :) asked whether "Lift has page flow functionality like Spring Web Flow?". The answer was as expected: the Lift developers have a page flow DSL in the works but seem to have missed the navigation and state management part (update: this appears to be incorrect, Lift does have navigation and state management.).

It's not enough to be able to define a page flow, the framework also needs to manage it! To help users and framework developers realize this, I've setup a page flow challenge. The idea is that you develop a simple 3-page page flow in your framework of choice and test whether it can correctly run a somewhat contrived scenario that tests things like POST-REDIRECT-GET to avoid accidental resubmission, double submit protection to avoid intentional resubmission, and correct state management. In essence the page flow should behave correctly whatever the user does and protect the user from potential mistakes.

Let's look at the page flow:

Easy peasy. This could for instance be a flow to submit a payment in an electronic banking application, or a flow to handle checkout in a web shop.

The page flow challenge scenario is as follows (note: I've clarified this scenario in a follow-up post):

1. Start the page flow
2. On Page 1, enter "foo" and click Next
   You should end up on Page 2 where "foo" is displayed as input from page 1
3. On Page 2, enter "baz" and click Next
   You should end up on Page 3 where "foo" and "baz" are displayed as input from the previous pages
4. Click the browser back button
   Without any browser warnings you should end up on Page 2, where "foo" is still displayed as input from page 1
5. Right click the Back link on the page and do Open Link in New Window
   The new window should show Page 1 where "foo" is displayed in the input field
6. In the new window, enter "kit" (replacing "foo") and click Next
   You should end up on Page 2 where "kit" is displayed as input from page 1
7. In the new window, on Page 2 enter "kat" and click Next
   You should end up on Page 3 where "kit" and "kat" are displayed as input from the previous pages
8. In the original window, click the browser refresh button
   Without any browser warnings you should see Page 2 again with "foo" as input from page 1
9. In the original window, enter "bar" and click Next
   You should end up on Page 3 with "foo" and "bar" as input from previous pages
10. In the original window, click Finish
    You should end up on the start page of the application: the page flow has finished
11. In the original window, click the browser back button
    You should either receive an error immediately, or when you try to resubmit by clicking Finish again
12. (For extra bonus points) In the new window, (where you still are on Page 3) click Finish
    You should receive an error of some sort (potentially a automatic page flow restart)

UPDATE: I've introduced a new step 11 in the above scenario, and what used to be step 11 is now step 12.

The only extra rule to keep in mind is that the HTML forms should use POST, to follow proper web design rules.

I've made reference implementations of the page flow challenge in Spring Web Flow 1 and 2. The source code is available in Subversion:

Spring Web Flow 1: https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf1/
Spring Web Flow 2: https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf2/

If you have Subversion and Maven installed, getting things up and running is as simple as:

$ svn export https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf2/ pageflowchallenge-swf2
$ cd pageflowchallenge-swf2
$ mvn jetty:run

You can now point your browser to http://localhost:8080/pageflowchallenge-swf2/ to test things out.

This challenge is solely focussed on page flow functionality, so it's not meant to reflect on the quality of a particular framework in other areas. For instance, SWF1 and SWF2 handle this scenario flawlessly, but probably have a harder time dealing with some other use-cases.

If you implemented the challenge in your web framework of choice, please let me know in the comments! I'm especially interested to see how Wicket, Seam, Lift and GWT fare. If you think this is irrelevant for your web framework because it handles things differently, I'd love to learn why you think this is the case, and how a use-case like this would then be implemented.

The Role of Entities in DDD

In his famous book Domain-Driven Design, Eric Evans defines entities as follows:

An object defined primarily by its identity is called an ENTITY.

This implies that the primary responsibility of an entity is maintaining this thread of continuity, this identity.
Implementation wise, this is typically accomplished by using an identity field, a 'primary key' field as it were. Here is an example using JPA annotations:

@Entity
@Table(name = "PERS")
public class Person {

 @Id
 @Column(name = "ID")
 @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PERS_ID_SEQ")
 @SequenceGenerator(name = "PERS_ID_SEQ", sequenceName = "PERS_ID_SEQ", allocationSize = 20)
 private Long id;

 @Column(name = "NAME")
 private String name;

 @Column(name = "DOB")
 private Date dateOfBirth;

 public Long getId() {
  return this.id;
 }

 @SuppressWarnings("unused")
 private void setId(Long id) {
  this.id = id;
 }

 public String getName() {
  return this.name;
 }

 public void setName(String name) {
  this.name = name;
 }

 public Date getDateOfBirth() {
  return this.dateOfBirth;
 }

 public void setDateOfBirth(Date dateOfBirth) {
  this.dateOfBirth = dateOfBirth;
 }

 public String toString() {
  return this.name;
 }
}

Pretty straightforward code. Still, there's a lot of it and it will baloon quickly when you add more fields, constructors and serializability to the mix.

Say that we need to determine in some part of the application whether or not a person is a minor. Naively, we could put an isMinor() method on the Person class. However, the fact of the matter is that the core responsibility of the Person class is maintaining identity, and determining whether or not a person is a minor is not part of that responsibility. Futhermore, the size of the Person class indicates that it's already plenty busy with it's core responsibility.

Eric Evans also hints at this in his book:

Rather than focusing on the attributes or even the behaviour, strip the ENTITY object's definition down to the most intrinsic characteristics, particularly those that identify it or are commonly used to find or match it.

It's better to factor the new behaviour into a separate class. One way of doing this that I particularly like is using interpretation wrappers. Here's an example:

public class InterpretedPerson() {

 private Person person;

 public InterpretedPerson(Person person) {
  this.person = person;
 }

 public Person getPerson() {
  return this.person;
 }

 public int getAge() {
  // calculate the age based on person.getDateOfBirth()
  // ...
 }

 public boolean isMinor() {
  return getAge() < 18;
 }
}

This class simply interprets the data available in a Person entity, for instance using the dateOfBirth property to determine whether or not the person is a minor. Seperating data interpretation from the underlying entities brings several advantages:

• The entity classes remain focussed on identity and the associated attributes.
• You can have several interpretation wrappers, doing different kinds of interpretation (for instance, another part of the application might use another definition of what it means to be a minor).
  
• Responsibilities are properly factored.

The interpretation wrapper is a useful little pattern that I haven't seen described elsewhere, so here you go!

On Testing and Refactoring

The last few days there was quite a bit of commotion in the blogosphere around unit testing, TDD and testing in general. (InfoQ has some more background.) I don't want to get into that whole debate -- I'm a big fan of testing but I don't practice TDD religiously -- but I do want to touch on an interesting subject that came up: the way testing and refactoring relate.

There is a bit of a dual relationship between testing and refactoring. On one hand, good test coverage allows you to refactor with confidence: the tests tell you whether or not you broke things. On the other hand, if the refactoring you want to execute has a large impact on the test suite, you're held back in two ways: the extra work required to rework the tests, and the possibility of introducing bugs in the tests while reworking them, which reduces your refactoring confidence.

On the project I'm currently working on we've tackled this in an interesting way (nothing novel but worth pointing out nonetheless): next to the normal unit tests, we also have a large suite of what we call scenario tests. These scenario tests operate at a somewhat higher level and run against stable abstractions. Unlike unit tests they don't test isolated units of the application. Instead they test the system as an integrated whole and run real-life scenarios through it. However, the scenario tests are not really integration tests: all environmental dependencies are mocked out making it possible to run these scenario tests as part of the normal test suite with every build.

To make this a bit more tangible, lets consider a fictional event processing component:

public class CalculatingEventProcessor implements EventProcessor {

  private SomeCalculator calculator;
  private SomeRepository repository;

  ...

  @Override
  public void process(Event event) {
    if (weNeedToCalculateSomethingFor(event)) {
      SomeObject input = extractSomeObjectFrom(event);
      SomeOtherObject output = calculator.calculate(input);
      repository.store(output);
    }
  }

  ...
}

Instead of simply having unit tests for the SomeCalculator and SomeRepository classes, we build scenario tests at the level of the EventProcessor interface to verify that the correct things are calculated in the expected situations.

public interface EventProcessor {
  void process(Event event);
}

Such a scenario test would look something like this:

public class MyCalculationScenarioTest extends AbstractEventProcessorScenarioTest {

  @Override
  public EventProcessor setupEventProcessor() {
    return ...; // instantiate, or pull from a Spring app context, or ...
  }

  @Override
  public void runScenario() {
    // setup data as required by the scenario
    ...
    
    // create a relevant event
    Event event = ...;

    publishEvent(event);

    // the test infrastructure will run the event through the event processor

    // assert the necessary calculations have been done
    ...
  }
}

Note that the above test sets up all required data, typically in a throw-away database like HSQLDB. This is necessary to make sure that the scenario test can be run as a normal unit test: independent of any other tests or its environment.

It takes a bit of effort to setup the required scenario test infrastructure for your project, but the pay-off is big. Once you've got a sizeable set of scenario tests, you can refactor large parts of the internal design of your application without any impact on the scenario tests, which you can run at any time to see if things are moving in the right direction. The key here is the fact that the scenario tests run against a stable abstraction such as the EventProcessor interface in the example, which is unlikely to change.

UPDATE: It's worth pointing out that the scenario tests described above follow the basic form of a scenario in BDD (Behaviour-Driven Development):

Given some initial context (the givens),
When an event occurs,
then ensure some outcomes.

In BDD scenarios serve as an executable specification: a set of executable acceptance criteria that become proper end-to-end functional tests. The scenario tests described here serve the exact same purpose.

Event Driven Architecture for Systems that Never Stop

InfoQ posted an excellent talk by Joe Armstrong (of Erlang fame) titled Systems that Never Stop (and Erlang). Joe presents 6 laws that he sees as crucial to systems that never stop (i.e. systems with X nines of reliability): Isolation, Concurrency, Failure Detection, Fault Identification, Live Code Upgrade, and Stable Storage.

The Erlang language and runtime were designed for exactly this kind of environment so it comes as no surpise that it shines in all areas. However, it occurred to me that an Event Driven Architecture goes a long way in building a system that never stops with more mainstream tools like Java and a relational database.

• Isolation: Crashing of one process should not influence any other process, effectively isolating different processes. The big boon here is that this can really improve the reliability of your system: the probability of two independent processes failing is only half that of a single process failing.
  In an event driven world events are typically handled independently of one-another. Failure to process a particular event does not affect the processing of any other event in the system.
• Concurrency: A runtime environment should not put any undue constraints on how far you can take concurrency, i.e. the number of threads or processes in your system. Designing a system with concurrency in mind allows you to take advantage of current day hardware (multi-cores and the like), but also goes hand in hand with the first law: independent, isolated processes can run concurrently without much problem.
  In an event driver system, multiple independent event processors can run concurrently on a single machine, taking advantage of all CPU cores available on the machine. Furthermore, event driven systems scale horizontally, allowing you to easily add additional machines with even more event processors.
• Failure Detection: You must be able to detect when things go wrong, for instance by having a nanny like process that can monitor the system and can take appropriate action when failures occur. Furthermore, you should fail early, limiting the potential impact of the failure.
  In an event driven system you typically have failover handling that holds on to events that failed processing for later analysis or retry. One system property that Joe does not mention in his talk but is very relevant in this area is idempotence: being able to retry things without affecting the final result. With some effort, idempotence comes naturally in an event driven system.
• Fault Identification: If something has gone wrong, and you've detected it as per the previous law, you need to be able to identify what went wrong.
  Again, in event driven systems this comes naturally: the event that failed processing is normally maintained by the failover system, along with error messages and the like, and can easily be used to simulate what happened.
• Life Code Upgrade: You shouldn't need to bring down your system for an upgrade. Whether or not this is important to you is of course very dependent of the application that you're writing. Still, if you have a cluster of event processors, you can upgrade them one by one, keeping the system live at all time. On a single machine you could use things like OSGi to attain similar results.
• Stable Storage: This essentially boils down to having transactional (ACID) storage available. In your average enterprise application this storage will be provided by an RDBMS (Oracle, ...). Most current-day MOM solutions (MQ, ...) are also completely transactional, and even XA aware. So there should be no need to ever loose any information in an event driven system build on top of these technologies.

So in all, an event driver architecture goes a long way in bringing you a system that never stops. This of course in stark contrast with batch systems, that have a hard time living up to a number of these laws.

Welcome!

I've been thinking about this for quite a while but I've finally decided to start blogging! Hopefully this will prove interesting, useful or entertaining to at least a few people.
Welcome and enjoy!