Erwin Vervaet

Spring bean definition profiles and web.xml

2012-01-04T10:26:00.000+01:00

Spring 3.1 was released a few weeks ago. One of the interesting new features is the first-class support for bean definition profiles: you can define beans as part of a profile and than activate that profile if required. Bean definitions that are part of a profile that has not been activated are simply skipped.

Last week, I spent a few days upgrading our application to Spring 3.1 and updating it to make use of bean definition profiles. As most other sizable Spring based apps, we had devised our own configuration switching system based on a combination of Maven resource filtering and XML <import/> elements containing ${placeholders}. This works well enough but it would obviously be better to directly use the bean definition profile feature of Spring 3.1, if only because it's better documented.

By default, the list of active bean definition profiles will be determined by looking at a spring.profiles.active property. Out-of-the-box, a Spring WebApplicationContext will setup a PropertySources list containing the servlet config parameters, servlet context parameters, JNDI properties, system properties and environment variables, in that order. This implies that you can define the spring.profiles.active property in any of those locations, the most obvious one being servlet context parameters in web.xml:

<context-param>
 <param-name>spring.profiles.active</param-name>
 <param-value>foo-profile, bar-profile</param-value>
</context-param>

We wanted to use Maven profiles to switch between Spring bean definition profiles, so the initial idea was to simply do Maven resource filtering on web.xml:

<context-param>
 <param-name>spring.profiles.active</param-name>
 <param-value>${spring.profiles.active}</param-value>
</context-param>

It turns out doing resource filtering on web.xml leads you to a world of pain:

The Jetty plugin for Maven assumes the web.xml is static and simply uses the one in the source folder. You can explicitly tell it to use the web.xml from the target folder (using the <webXml> config element), but that results in other problem: by default resource filtering does not process the src/main/webapp directory. Of course you can specify filtering for that directory in your POM, but that again given problems: the Maven war plugin will overwrite the filtered web.xml with a copy from the source folder while packaging the web application. You can fix this by adding a <webResources> definition to your POM for the war plugin. Clearly this situation is far from ideal: it's complex and requires an explicit mvn package just to be able to do mvn jetty:run.
Idem ditto for Tomcat.
Eclipse WTP also directly uses the web.xml from the sources folder. Maybe there are ways around this, but I have no idea how that would work.

In the end we decided to forgo Maven resource filtering on web.xml and simply use an application specific properties file:

spring.profiles.active=${spring.profiles.active}

To get Spring to pick up this additional properties file, we simply add a property source to the application context using an ApplicationContextInitializer:

public class MyAppCtxInitializer
  implements ApplicationContextInitializer<ConfigurableApplicationContext> {
 public void initialize(ConfigurableApplicationContext applicationContext) {
  try {
   applicationContext.getEnvironment().getPropertySources()
     .addLast(new ResourcePropertySource("classpath:/myapp.properties"));
  } catch (IOException e) {
   // properties file cannot be found: ignore and just continue without these properties
   logger.info("Unable to load fallback properties: " + e.getMessage());
  }
 }
}

This approach avoids all the downsides of doing filtering on web.xml (web.xml is now completely static and mvn package no longer required). The only negative aspect is that it introduced an application specific properties file instead of simply relying on standard Spring conventions.

Good code and good wine

2011-12-25T09:08:00.000+01:00

I've discussed the "What is good code?" question before. This is such a broad topic that people come up with all kinds of interesting analogies to describe beneficial traits of good code. For instance, Uncle Bob often compares good code to a good book: it's important to have a good story (design/architecture), introduce key characters (concepts) at appropriate times and so on. Another interesting analogy that I've heard is that good code is like a leather glove: over time it becomes more flexible in areas where it needs to be flexible and more rigid in places that need more rigidity, exactly the way good code should behave and evolve (I'm not sure where I first heard this analogy and who I should credit for it).

Another analogy I (? -- I've never heard it before so I'll be so bold as to take credit myself) came up with recently is comparing good code with good wine. An important characteristic of good wine, maybe even the most important one, is that it tastes good. Likewise, good code needs to function properly and deliver the required functionality. However, as wine aficionados will tell you, tasting good is not the only important trait of a good wine: great wine also has things like an interesting texture, a complex taste palette and so on (not being a wine connaisseur myself, I won't dwell on it). Similarly, having a functional program doesn't imply good source code. Good code needs to exhibit extra characteristics such as flexibility, testability, adhering to the principle of least astonishment and so on.

I'd love to hear about more analogies used to describe good code!

JDK8 and the future of JVM based languages

2011-11-24T21:31:00.004+01:00

Last week at Devoxx 2011 I attended a number of sessions covering new JVM based programming languages such as Kotlin and Ceylon. Of course there were also a number of sessions talking about the usual suspects: Groovy, Scala and JRuby.

There were also a considerable amount of talks covering JDK8 and how that's shaping up. I got the strong impression that Oracle certainly doesn't want to see JDK8 turn into another multi-year debacle. Progress on JDK8 (vitual extension functions, closures, ...) seems impressive and I got a real sense of direction and focus from those involved.

I've been following the whole other languages on the JVM movement that has been going on the last couple of years with interest, but mostly as a by-stander: I'm not a hard-core, opinionated programming language expert like some. However, if I look at the input I gathered at Devoxx I can't help but make a few observations:

Although some of the new JVM languages have interesting ideas and are definately a step up from Java, they didn't trigger that aha-erlebnis I had when I first started using Java 1.02 after having used C.
The infectuation with dynamic typing seems to be largely over: everybody is back in agreement that strong static typing is the way to go.
Scale can't get any love and seems destined to remain an interesting language experiment.
Oracle is on somewhat of a mission to deliver JDK8 sooner rather than later.

All of this makes me wonder if Java's successor will actually be Java itself, but in it's 8th incarnation, and whether or not Java 8 will effectively blow all new contenders out of the water? I'm looking forward to seeing how this unfolds!

Java puzzler

2011-10-28T14:43:00.000+02:00

Take a look at the following piece of code:

package test;

import java.util.Date;

public class Test {

 public static String format(Date date) {
  return String.valueOf(date);
 }

 public static String format(long millis) {
  return format(new Date(millis));
 }

 public static String format(Instant instant) {
  return format(instant == null ? null : instant.getMillis());
 }

 public static class Instant {

  private long millis;

  public Instant(long millis) {
   this.millis = millis;
  }

  public long getMillis() {
   return millis;
  }
 }

 public static void main(String[] args) {
  Instant instant = null;
  format(instant);
 }
}

Any idea what the output will be? Turns out this throws a NullPointerException:

Exception in thread "main" java.lang.NullPointerException
 at test.Test.format(Test.java:16)
 at test.Test.main(Test.java:34)

Interesting. Line 16 is clearly fishy: The idea is that if the instant is null, the format(Date) method should be called, while format(long) should be called otherwise. It turns out things are much more contrived at the bytecode level (use javap -c):

public static java.lang.String format(test.Test$Instant);
  Code:
   0: aload_0
   1: ifnonnull 8
   4: aconst_null
   5: goto 15
   8: aload_0
   9: invokevirtual #35; //Method test/Test$Instant.getMillis:()J
   12: invokestatic #41; //Method java/lang/Long.valueOf:(J)Ljava/lang/Long;
   15: invokevirtual #46; //Method java/lang/Long.longValue:()J
   18: invokestatic #49; //Method format:(J)Ljava/lang/String;
   21: areturn

This actually translates to the following Java code, which clearly explains the NullPointerException:

public static String format(Instant instant) {
 return format((instant != null ? Long.valueOf(instant.getMillis()) : null).longValue());
}

Sometimes autoboxing can really rear its ugly head!

Surprise of the day

2011-09-04T08:17:00.000+02:00

Well what do you know! It turns out Microsoft was a bigger contributor to Linux kernel version 3.0 than Canonical, as LWN reports. Probably a bit of a fluke but an interesting statistic nonetheless!

Q_rsqrt

2011-07-23T11:55:00.002+02:00

I still remember encountering the following piece of code while browsing through the Quake 3 source code back in 2005:

float Q_rsqrt( float number )
{
  long i;
  float x2, y;
  const float threehalfs = 1.5F;

  x2 = number * 0.5F;
  y  = number;
  i  = * ( long * ) &y;  // evil floating point bit level hacking
  i  = 0x5f3759df - ( i >> 1 ); // what the fuck?
  y  = * ( float * ) &i;
  y  = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration
  // y  = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed

  #ifndef Q3_VM
  #ifdef __linux__
    assert( !isnan(y) ); // bk010122 - FPE?
  #endif
  #endif
  return y;
}

That comment on line 10 pretty much sums it up: WTF!?!?

I again ran across this piece of code while browsing an amuzing list of the best comments in source code you've encountered, over on Stackoverflow. ~~It turns out the original author of this illustrious piece of code is still unknown~~ (it's not John Carmack as you might have assumed). A nice write-up of the likely history of this hack can be found on Beyond3D. Interesting reading for sure!
UPDATE: The original author is actually known and it is Greg Walsh. Wikipedia also has a lot of background.

Talking about funny comments in code, my personal favorite from the Stackoverflow list must be the following:

stop(); // Hammertime!

Not so much the comment, but the german version really cracked me up!

Java SE 7

2011-07-19T19:03:00.000+02:00

It was a long time in the making but all JSRs constituting Java SE 7 have crossed the finish line! The actual Java SE 7 software release is just around the corner (July 28th). Exciting stuff!

Tech support interference

2011-07-02T07:43:00.000+02:00

It's a well known fact that developers and tech support people typically don't get along. The developers want freedom to do as they like with their machines, while the tech support guys want to close everything down, mainly for security reasons.

Sometimes this can get quite ridiculous. I recently heard a story about a development team building a web app that required Firefox compatibility, yet they were not allowed to install Firefox on their machines! Another typical example which recently came up at TSS, is developers going to great lengths to disable virus scanning software because it makes their IDEs unbearably slow.

As always, solving this conflict requires a middle-of-the-road solution. I agree that giving developers unlimited privileges on machines that are connected to production networks is an unwanted security violation. Similarly it's unacceptable to bog down a development machine to the point where the developer can no longer properly do his job. One of the better solutions I have encountered is a separate development network. This development network is an isolated sandbox where the developers reign supreme without putting other parts of the network or organization at risk.

Slow Java platform evolution

2011-05-27T15:01:00.000+02:00

It's fascinating to read the Java Module-System Requirements document that Mark Reinhold has drafted to see just why a platform of Java's size and complexity (both code and community wise) is hard to evolve. The requirements document puts the bar quite high for what a Java Module-System needs to be able to do. You also clearly see the influence of systems like Maven and OSGi, and how the experiences with those systems has impacted the requirements.

The end result of all of this is that a Java Module-System can't start off small and simple and grow over time. Instead, the designers are faced with a complex and extended set of requirements right from the start. Slow progress is inevitable. Nevertheless, I'm hoping Java SE 8 will include this Module-System because it would bring a lot of interesting new possibilities to the Java platform!

Properties file troubles

2011-04-22T20:25:00.000+02:00

I stumbled over a stupid properties file gotcha the other day trying to get a Spring META-INF/spring.schemas file working. Spring uses this file to translate an XML Schema schemaLocation into the name of a classpath resource. This allows Spring to load the schema directly from the classpath instead of downloading it from a remote server which might not be reachable from your deployment environment (e.g. because of firewalls). So for instance, suppose you have the following in your XML:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:demo="http://www.ervacon.com/schema/demo"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.ervacon.com/schema/demo http://www.ervacon.com/schema/demo/demo.xsd">

If you combine this with the following META-INF/spring.schemas file, Spring will load demo.xsd directly from the classpath without even trying to access http://www.ervacon.com/schema/demo/demo.xsd:

http\://www.ervacon.com/schema/demo/demo.xsd=demo.xsd

The key thing to notice in the properties file is the escaping backslash before the colon! This is necessary because a colon is a valid delimiter in a Java properties file:

public static void main(String[] args) throws IOException {
 StringReader propsIn = new StringReader("http://my.server.com/foo.xsd=foo.xsd");
 
 Properties props = new Properties();
 props.load(propsIn);

 for (String propName : props.stringPropertyNames()) {
  System.out.println(propName + " --> " + props.getProperty(propName));
  // prints: http --> //my.server.com/foo.xsd=foo.xsd
 }
}

Keep in mind that all colons need to be escaped! So don't forget the colon in front of the port number:

http\://my.server.com\:8080/foo.xsd=foo.xsd

Doh!

Software and Mud

2011-04-07T13:39:00.001+02:00

There seems to be a strange relationship between software and mud. Many will be familiar with the Big Ball of Mud architectural style and there are countless examples of code that resembles mud more than anything else.

A while ago a colleague of mine used an analogy between a software related problem he was facing and wading through mud (or as we say in Dutch: "door de moose ploegen"). I'm amazed at how poignant this analogy actually is. Think of a situation where the first thing you notice is that you have mud on the soles of your shoes. As you trot on you quickly end up knee deep in mud. At this point you can either back out and retreat to solid ground, or persevere, in which case you often end up stuck!

Of course you might also make it through, as John Carmack once noted:

Yes, you can make windows do anything you want to if you have enough time to beat on it, but you can come out of it feeling like you just walked through a sewer. [.plan]

Many software related situations come to mind that closely follow this story:

Merging exercises gone wrong
Refactoring plans that start simple but spiral out of control
Anything to do with character encoding
...

Feel free to add this mud analogy to your software vocabulary!

(As an aside: some people can wade through mud more elegantly than others, as Kate Moss illustrates.)

Hibernate hurdles

2011-02-05T17:49:00.000+01:00

I've been using Hibernate for the better half of 10 years now (since Hibernate 2), and while I really like Hibernate it still surprises me from time to time how complex things can get. This post highlights a few of these hurdles. I'm pretty sure other ORMs have the same or similar issues, but I'll use Hibernate here because that's what I know best. To demonstrate things I will use the classic Order/LineItem example:

@Entity
@Table(name = "T_ORDER")
public class Order {

 @Id
 @GeneratedValue
 private Long id;

 @ElementCollection
 @CollectionTable(name = "T_LINE_ITEM")
 private Set<LineItem> lineItems = new HashSet<LineItem>();

 public Long getId() {
  return id;
 }

 public Set<LineItem> getLineItems() {
  return lineItems;
 }
 
 public int getTotalPrice() {
  int total = 0;
  for (LineItem lineItem : lineItems) {
   total += lineItem.getPrice();
  }
  return total;
 }
}

@Embeddable
public class LineItem {

 @ManyToOne(fetch = FetchType.LAZY)
 private Product product;
 
 private int price;
 
 @SuppressWarnings("unused")
 private LineItem() {
 }

 public LineItem(Product product, int price) {
  this.product = product;
  this.price = price;
 }
 
 public Product getProduct() {
  return product;
 }

 public int getPrice() {
  return price;
 }
}

@Entity
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@Table(name = "T_PRODUCT")
public abstract class Product {

 @Id
 @GeneratedValue
 private Long id;
 
 private String name;
 
 protected Product() {
 }
 
 protected Product(String name) {
  this.name = name;
 }
 
 public Long getId() {
  return id;
 }

 public String getName() {
  return name;
 }
}

@Entity
public class PcProduct extends Product {
 
 @SuppressWarnings("unused")
 private PcProduct() {
 }

 public PcProduct(String name) {
  super(name);
 }
}

@Entity
public class MacProduct extends Product {
 
 @SuppressWarnings("unused")
 private MacProduct() {
 }

 public MacProduct(String name) {
  super(name);
 }
}

That's pretty straightforward stuff. Assume we have an order stored in the database:

Product pc = new PcProduct("Dell XPS M1330");
session.save(pc);
Product mac = new MacProduct("Apple MacBook Pro");
session.save(mac);

Order order = new Order();
order.getLineItems().add(new LineItem(pc, 1000));
order.getLineItems().add(new LineItem(mac, 2000));
session.save(order);

Now let's look at some surprising pieces of code.

Beware of fetch joins

Suppose you have a business transaction processing orders involving Dell XPS M1330s:

List<Order> results = session.createQuery(
  "from Order as o join fetch o.lineItems as li where li.product.name = 'Dell XPS M1330'").list();
for (Order result : results) {
 System.out.println(result.getTotalPrice());
}

What do you think this prints out for our sample order containing both a Dell XPS M1330 and an Apple MacBook Pro? If you guessed 1000 you would be right... oopsie! The fetch in the query combined with criteria on the joined relation causes the lineItems collection of the resulting Order entities to be incomplete! Unfortunately, there is no way to tell that those Order objects are crippled. Worse still, they will end up in the Hibernate session so you might inadvertently end up using them later on in the same transaction, unaware that they are damaged goods. This can lead to very hard to diagnose bugs, because the cause of the problem (the join fetch) and the effects (incorrect results) might be far apart in the code. It's interesting to note that normal joins (so not using fetch) do not have this adverse side effect. So my advice is:

Be very wary of fetch joins. Only use them if you are sure the resulting objects will not be reused later on in the same session for different purposes.

Beware of inheritance mapping

Let's look at another order processing transaction:

long orderId = ...;
Order order = (Order) session.get(Order.class, orderId);
Product product = order.getLineItems().iterator().next().getProduct();

PcProduct pcProduct = (PcProduct) session.get(PcProduct.class, product.getId());

System.out.println(pcProduct.getId().equals(product.getId()));
System.out.println(pcProduct == product);

If you run this piece of code, Hibernate will spit out a warning:

1463 [main] WARN org.hibernate.engine.StatefulPersistenceContext.ProxyWarnLog
 Narrowing proxy to class com.ervacon.order.PcProduct - this operation breaks ==

And indeed, the output is true / false, confirming that the session can now no longer guarantee that primary key equality is equivalent with reference equality (==). This is a dangarous situation that could again lead to hard to diagnose bugs, especially if some parts of the code (e.g. the lineItem.getProduct() call) are only executed is certain situations. There are other problems with mapping inheritance hierarchies (e.g. the fact that instanceof in unreliable), so my advice here is:

When using Hibernate, avoid mapping inheritance hierarchies. There are often better ways to factor your code that do not require mapping an inheritance hierarchy to the database, for instance by introducing an common interface to facilitate polymorphism.

There are other non-obvious head-scratchers you run into when using an object-relational mapper like Hibernate. If you know about an interesting one, I'd love to hear about it!

To reuse or not to reuse, that's the question

2010-11-11T07:49:00.003+01:00

It's clear that code duplication is a very bad code smell. As a result you use things like extract method refactoring to keep your code DRY. This brings up the question if this can be taken too far? Should all code duplication be avoided at all times?

Let me illustrate. Assume you have a bit of code like this:

public static void notNull(Object obj) throws IllegalArgumentException {
 if (obj == null) {
  throw new IllegalArgumentException("Argument cannot be null");
 }
}

This method is clearly designed for reuse. It's a small building block of code that can be used thoughout a piece of software to avoid duplicating null-checking code in several places. In this case there are also no real downsides to consider.

Now consider a somewhat higher level piece of (admittedly extremely contrived) code:

public void doSomeProcessing(String str) {
 checkPreconditions(str);
 ...
}

private void checkPreconditions(String str) {
 Check.notNull(str);
 if (!"A".equals(str) && !"B".equals(str)) {
  throw new IllegalArgumentException("Argument can only be A or B");
 }
}

As it stands right now the private checkPreconditions() method is not designed for reuse. If another component in the system happens to require the exact same preconditions, should you try to reuse it? In some situations the answer to this question is clearly: yes! If the new component intrinsically requires the same preconditions, it's natural to try to reuse the method and highlight the fact that the two components have some functional relation. However, in other situations the fact that the two components do the same pre-condition checks might be purely accidental. In other words: there is no functional relation between the two components. In this case refactoring the code to be able to reuse the checkPreconditions() method in both cases comes with a few downsides you should consider:

By reusing the method, the refactored code now potentially communicates a relationship between the two components, where there really is none.
If the preconditions of the two components are in reality unrelated, it would not be uncommon for the preconditions
of one of the components to change, while the other component's preconditions remain the same. When making this change in the refactored code, you need to realise you can't just change the common checkPreconditions() method. Instead, a more complex code change will be required.

In my experience DRY code is certainly something to strive for. However, avoiding code duplication at all costs also comes at a cost!

Installing ING Home'Bank and Belgian eID on Ubuntu

2010-10-30T17:34:00.000+02:00

I spend some time today upgrading my laptop from Ubuntu 9.04 (Jaunty Jackalope) to 10.04 LTS (Lucid Lynx) since 9.04 reached it's end of life earlier this month. Overall this was a very smooth process except for two specific apps that I require: ING Home'Bank and Belgian eID.

In an attempt to help people who also need to install one of these apps on a recent Ubuntu version, here is how I did it:

Home'Bank
ING is phasing out it's Home'Bank security module so the download is ages old and references a bunch of old libraries.

Start by downloading the HomeBank333.deb and simply install it as you would any other .deb. Next thing to do is pin the Home'Bank version in Synaptec to avoid this bug: 380511. To do that, start Synaptec, find the homebank package and select Package>Lock Version.

If you try to run /opt/HomeBank/HBSecurity, it will complain about 3 missing libraries:

libtiff.so.3 is missing -- simply sym-link it:

cd /usr/lib
sudo ln -s libtiff.so.4 libtiff.so.3

libexpat.so.0 is missing -- install and sym-link it:

sudo apt-get install libexpat1
cd /usr/lib
sudo ln -s /lib/libexpat.so.1 libexpat.so.0

libstdc++libc6.2-2.so.3 is missing -- this is an old Dapper Drake package that you can still download and install

Once these steps have been completed you should be able to run /opt/HomeBank/HBSecurity and start using Home'Bank.

Belgian eID
This is actually straightforward. A .deb package is provided for recent Ubuntu versions that installs and runs without any problems. The only problem on my machine was that my smart card reader was not being recognized (an Alcor Micro Corp. EMV Certified Smart Card Reader in my case -- do a lsusb in a terminal to find out what type of smart card reader you have). To rectify that, I had to install the libccid package:

sudo apt-get install libccid

Once that is done you can simply follow the install instructions provided in a PDF to configure your Firefox to use the new certificates and you should be up-and-running!

ThreadLocal naming conventions

2010-10-07T08:59:00.002+02:00

Since Sonar is now part of our normal build platform on the project I'm currently doing, we're keeping a close eye on the violations it identifies. One rule we violate in a couple of places is the CheckStyle constant naming rule. This rule simply says that the names of all static final variables should be ALL_UPPERCASE_WITH_UNDERSCORES. This is of course a well known Java coding convention. Still, it feels a little unnatural if you apply it to ThreadLocals, which are technically static final variables, but are neither immutable nor do they have a global scope like typical constants. This makes code using ThreadLocals look a bit weird if you use normal Java constant naming, e.g. compare the following:

private static final ThreadLocal<Date> TIME_FRAME = new ThreadLocal<Date>();
...
TIME_FRAME.set(myDate);

private static final ThreadLocal<Date> timeFrame = new ThreadLocal<Date>();
...
timeFrame.set(myDate);

For me using normal variable naming conventions for ThreadLocals seems to better communicate their role and intented usage in the code. Does anybody have an idea what the official Java naming conventions for ThreadLocals are?

Killer tool: Sonar

2010-08-27T19:38:00.001+02:00

Today a colleague at work showed Sonar to me, and I must say that I was really impressed! Sonar is an open source code quality analysis tool that uses a number of popular Java code analyzers like PMD, CheckStyle, FindBugs and Cobertura under the hood, and presents the metrics calculated by all of these in a consistent and integrated way. It also keeps track of all the metrics data in a database so you can see how your code quality evolves.

One part of Sonar is a Web front-end for the metrics database, and the other part is a Maven plugin that runs all code analyzers and pumps the collected data into that database. (As an aside: imagine how much harder it would be to develop a tool like this if there wasn't a dominant build solution in the Java space like Maven.)

As a package Sonar just ticks all the right boxes. It's really easy to get up and running (starting its DB/Web server and running mvn clean install sonar:sonar in your Maven project is all there is to it), feels absolutely solid, has a friendly UI and looks very polished overall.

Warmly recommended!

How much damage can a Java Error do?

2010-07-21T07:25:00.001+02:00

In Java there is a clear distinction between recoverable Exceptions and fatal Errors. It's common for applications to have a catch-all block so that they can continue running even if unexpected problems occur:

while (true) {
 try {
  // execute application logic
 } catch (Throwable t) {
  // log exception and/or display to the user
 }
}

This will catch both Exceptions and Errors. Although it's fine to catch an Exception and have the application carry on, this is typically not a good idea when an Error occurs!

I recently saw a bug in an application that caused a StackOverflowError. However, because of a catch-all block the application tried to recover from the problem and carry on. As it turns out this caused all kinds of instability and seemingly unrelated problems: connection leaks, ThreadLocal corruption and other such joys.

If you think about it this makes sense. A StackOverflowError indicates that the JVM ran out of stack space to properly invoke methods. Of course this has implications for things like finally blocks, and certainly for method invocations in those finally blocks! If you can no longer rely on the proper execution of your finally blocks you end up in a world of pain with things like messed up ThreadLocals. There are similar considerations for OutOfMemoryError and certainly for the other VirtualMachineErrors.

In summary: Errors can do a lot of damage! Don't keep the application running if you encounter an Error. Instead, try to log the problem and die. Only recover in case of Exceptions.

while (true) {
 try {
  // execute application logic
 } catch (Exception e)
  // log error and/or display to the user
  // => recover
 } catch (Error e) {
  // log error and/or display to the user
  // => exit JVM
 }
}

Bitemporality in Python

2010-06-01T20:23:00.001+02:00

A few years ago I published a Java framework to deal with bitemporal issues in rich domain models. The code is freely available under a BSD style license.

As a first piece of non-trivial Python programming I coded up an equivalent framework in Python 2.5, available here. The framework allows you to set up a class with bitemporal properties that track the history and evolution of the value assigned to those properties. Here's a small example that should give you a feel for what you can do:

from bitemporal import *

class Person:

 def __init__(self, name):
  self.name = name
  self.address = BitemporalProperty()

p = Person("John Doe");
p.address.assign("Smallville", Interval(date(1975, 4, 3), date.max))
p.address.assign("Bigtown", Interval(date(1994, 8, 26), date.max))

print p.address.now() # Bigtown
print p.address.on(date(1980, 1, 1)) # Smallville

I'm looking for feedback on this code, which right now is only about 300 lines long. Is this code pythonic? Or is this Java-style Python code? If so, what areas need to be improved? Personally, I have doubts about at least a few things:

The type checks in the TimeFrame class (and a few other classes) feel iffy:

@classmethod
def set_reference(cls, value):
 if type(value) == date:
  cls.reference = datetime.combine(value, time())
 else:
  cls.reference = value

But how else do I ensure that a timeframe always holds a datetime?

The BitemporalWrapper class feels Java-like. It decorates another object with bitemporal information and functionality. Is there a better way to do this in a dynamically typed language?
I considered using @property for the assign() method on the BitemporalProperty class, but the problem is that you often want to pass in a validity interval, which rules out use of "=".

Does anybody have any tips how this code can be improved?

Python default argument madness

2010-05-13T08:02:00.001+02:00

I've been learning python recently. I'm just getting started so I'm running into all the stupid beginner problems, for instance the insane behaviour of default argument values. Take this example:

from datetime import datetime
import time

class Weird:

 def __init__(self, dt = datetime.now()):
  self.dt = dt

w1 = Weird()
time.sleep(1)
w2 = Weird()
w3 = Weird(datetime.now())

print w1.dt == w2.dt # True
print w1.dt == w3.dt # False

The datetime in the w1 and w2 objects will actually be the same since the default argument value in the method definition is only evaluated once! What!?!? This essentially renders default arguments useless since you always end up writing the following:

class Weird:

 def __init__(self, dt = None):
  self.dt = dt or datetime.now()

I don't understand why you would design a default argument values features in your language like this. Doing it this way certainly violates the principle of least astonishment!

What is good code?

2010-05-01T07:24:00.001+02:00

As a project tech lead I do a lot of code review, so the "What is good code?" question comes up a lot. Of course there a many things that factor into the good code equation such as efficiency, correctness and elegance. For me the most important property of good code is that it should adhere to the principle of least astonishment. In other words: good code is code that does what you expect it to do in the way you expect it to be done.

Following this principle has important benefits when it comes to reading, understanding and maintaining a piece of code. It also typically improves the correctness of the code: you tend to get code that obviously has no deficiencies, in stead of no obvious deficiencies (as Tony Hoare said).

I was happy to learn that I'm in good company when it comes to attributing the principle of least astonishment to good code. Both Uncle Bob (in his excellent Clean Code book) and Peter Seibel (in his very interesting Coders at Work) ask several famous programmers about good code. A lot of the interviewees directly or indirectly mention the principle of least astonishment (for instance Ward Cunningham, Joe Armstrong and Simon Peyton Jones).

TreeSet Gotcha

2010-04-02T18:52:00.001+02:00

Sometimes API design mistakes can make you go d'oh! In the back of my mind I knew about the weird workings of Java's TreeSet, but today I was bitten by the consequences anyway.

The basic problem is that TreeSet implements a Set in terms of compare() or compareTo(), instead of the usual equals() method (as documented in the TreeSet JavaDoc). In strict terms this makes TreeSet violate the Set interface with regard to the Liskov substitution principle.

A quick example to illustrate the problem. Suppose you have a class holding two pieces of information. You want the objects of the class to have a natural order based on just a single piece of information, so you implement Comparable like this:

public class Foo implements Comparable<Foo> {
 
 private Object data;
 private int num;

 public Foo(Object data, int num) {
  this.data = data;
  this.num = num;
 }
 
 @Override
 public boolean equals(Object obj) {
  if (obj instanceof Foo) {
   Foo other = (Foo) obj;
   return this.data.equals(other.data) && this.num == other.num;
  }
  return false;
 }
 
 @Override
 public int hashCode() {
  return data.hashCode() + num;
 }

 @Override
 public int compareTo(Foo other) {
  return this.num - other.num;
 }
}

It's natural to assume that compareTo() only influences the order of the objects, but does not influence their equality. However, TreeSet uses compareTo() to test for object equality, which means the following test fails unexpectedly:

@Test
public void usageInASetWorksAsExpected() {
 Set<Foo> set = new TreeSet<Foo>();
 set.add(new Foo("bar", 1));
 set.add(new Foo("baz", 1));
 set.add(new Foo("bar", 1));
 set.add(new Foo("bar", 2));
 Assert.assertEquals(3, set.size()); // actual set size will be 2!
}

D'oh!

Mac Fanboys in Java Land

2010-03-30T20:26:00.002+02:00

Could somebody explain to me why lots of Java (and Ruby, Phyton, ...) programmers seem to think Macs are the best thing since sliced bread? I'll be the first to admit that Apple hardware is excellent and that they have a UI flair that is unrivaled by any other company in the IT industry today.

However, as a programmer I don't care too much about dumbing it all down to a level where my grandmother can use it. I tried the Mac thing and sure it's great for some casual usage, but a number of things just annoy me when I try to get real work done:

First of all: why does Apple treat Java as a second grade citizen?
Why can't I resize my windows on the left side?
Why can't Apple just use a standard PC AZERTY keyboard layout (used in Belgium where I live)? If they did, I would at least be able to find my curly braces and square brackets, and pipe symbol for that matter!
Why does OS X need to deviate from pretty much every common Unix convention like the FHS, making things like the hosts file end up in /private/etc/hosts.
Why is it that I need to copy the entire Eclipse installation just to be able to run two workspaces at the same time?
Why is Apple incapable of putting a standard DVI or VGA adaptor on a MacBook?

And the list goes on! (For those wondering: I currently run Ubuntu on a Dell XPS laptop: no installation or configuration hassles, and usage freedom to boot!)

MQ's MQRFH2 Header and JMS

2010-03-05T19:00:00.000+01:00

I've known about the MQRFH2 header used by WebSphere MQ for a long time. I was also aware of the problems this header can cause when exchanging messages between JMS and non-JMS based clients, and how to suppress use of the header by using targetClient=1 (MQJMS_CLIENT_NONJMS_MQ).

What I did not know before today however, is that if a JMS application receives a message not containing the MQRFH2 header, some basic JMS message properties like JMSDestination will not be available! This makes some sense since WebSphere MQ uses the MQRFH2 header to carry JMS specific data associated with the message. If you omit MQRFH2, only the properties that fit in the default MQMD header will be transmitted, the rest will be lost. (The WebSphere MQ documentation covers this in some detail.)

Still, for something basic like JMSDestination this came as a bit of a surprise since this property is set by the JMS service provider when sending the message and you expect it to be available when receiving that message, for instance to be able to do failover processing and send the message back to its original destination at some later time.

Source Code Management Friendly Design

2010-02-13T09:06:00.007+01:00

Good software design, especially proper modularization, brings many benefits such as single points of change, encapsulation and a reduced event horizon, just to name a few. This is clearly a Good Thing at the source code level, but it also has important source code management (SCM) implications. Let's look at an example.

Suppose your code needs to process some data in several different ways. When I say processing think about things like manipulating the data, reacting to it, and so on. The simplest thing that could possibly work would be something like this:

public class DataProcessingEngine {

 public void process(Data data) {
  processOneWay(data);
  processAnotherWay(data);
 }
 
 private void processOneWay(Data data) {
  // ...
 }
 
 private void processAnotherWay(Data data) {
  // ...
 }
}

This code has many issues. Purely at the source code level we have a class that just has too many responsibilities. At the SCM level, this class could become a merging nightmare. Imagine multiple development branches each adding new ways to process the data. Merging these branches back onto the trunk will almost certainly result in merging conflicts.

Of course we can do a lot better. Let's factor out the different ways to process the data into seperate DataProcessors:

public interface DataProcessor {
 void process(Data data);
}

Each different way of processing the data would have its own DataProcessor implementation. The DataProcessingEngine now becomes:

public class DataProcessingEngine {
 
 private List<DataProcessor> dataProcessors;
 
 public DataProcessingEngine(List<DataProcessor> dataProcessors) {
  this.dataProcessors = dataProcessors;
 }

 public void process(Data data) {
  for (DataProcessor dataProcessor : dataProcessors) {
   dataProcessor.process(data);
  }
 }
}

Of course we still need to configure the DataProcessingEngine with the appropriate DataProcessors somewhere:

public class DataProcessingEngineFactory {

 public static DataProcessingEngine create() {
  List<DataProcessor> dataProcessors = new ArrayList<DataProcessor>();
  dataProcessors.add(new OneWayDataProcessor());
  dataProcessors.add(new AnotherWayDataProcessor());
  return new DataProcessingEngine(dataProcessors);
 }
}

You could also use something like Spring to do this for you. In this case you would end up with bean definitions equivalent to the code above. At the code level this refactoring has pretty much solved the problem. We now have a few small classes, each with its own responsibility. However, at the SCM level, the DataProcessingEngineFactory class (or equivalent alternative configuration) still sits in a single file causing merge conflicts.

To solve this problem, we have to make the system a bit more dynamic. If the DataProcessingEngineFactory could automagically detect all available DataProcessor implementations, adding a new way of processing the data would be as simple as adding a new DataProcessor to the classpath. As a result, there would be no need to change the DataProcessingEngineFactory every time, an no more merge conflicts!

In Java you have quite a few options to implement such a dynamic discovery mechanism:

The service provider framework can automatically detect service implementations defined in META-INF/services.
Spring's ResourcePatternResolver can be used for this kind of thing.
Maybe you can even use the annotation processing tool to do something like this?

Using the Java 6 ServiceLoader, the DataProcessingEngineFactory would end up looking something like this:

public class DataProcessingEngineFactory {

 public static DataProcessingEngine create() {
  List<DataProcessor> dataProcessors = new ArrayList<DataProcessor>();
  for (DataProcessor dataProcessor : ServiceLoader.load(DataProcessor.class)) {
   dataProcessors.add(dataProcessor);
  }
  return new DataProcessingEngine(dataProcessors);
 }
}

It's interesting to note that annotation based configuration systems, which are all the rage, typically don't hold all configuration information in a single location, which helps ease your source code management as I've shown. XML based configuration is more problematic in this respect.

OSGI Dependency Resolution is NP-Complete

2010-02-02T15:41:00.000+01:00

Apparently, the resolution problem in OSGI is NP-Complete, as are similar dependency resolution problems like apt package installation on Debian Linux systems. If you think about it it makes sense, but who would have thought!

I still remember learning about P and NP problems, and the famous P = NP question at university, and being really intrigued by all of it. Lance Fortnow provides an excellent overview of the P = NP problem in Communications of the ACM. It turns out that this is no longer a computer science problem, but one of the fundamental questions in all of science. Fascinating stuff! If only I could find a simple proof :-).

REST Reservations

2010-01-01T13:36:00.002+01:00

I'm currently reading RESTful Web Services by Leonard Richardson and Sam Ruby, and happened to come across an interesting QCon talk by Mark Nottingham on the status of HTTP. To me, this talk carries an interesting critical undertone on REST.

I've followed the whole REST and RESTful Web Services hype with some interest the last few years, and genuinely like several aspects of REST:

You can't go wrong with a KISS approach to software development.
Most everybody is familiar with how the Web works, so exploiting that familiarity makes a lot of sense.
The HTTP underpinnings of REST make it very well supported by a multitude of tools.
By leveraging HTTP, REST has excellent support for intermediaries (i.e. proxies and caches) and all the cool things they can bring to the table.

On the other hand, I also have some reservations about the whole REST thing, and Mark Nottingham touched on several of these topics in his talk:

As always, there is a lot of dogma to go around, and everybody has his own flavour. Is using this or that HTTP method in a particular way RESTful? Mark, who's obviously intimately familiar with HTTP, seems to have a much more pragmatic view on how you can or should use HTTP (i.e. listen to his comments on POST at the very end of the talk).
HTTP is deceptively simple. The spec is BIG, and full of ambiguities. The entire first part of Mark's talk deals with this, and I would argue that most people (me included) only have a superficial familiarity with HTTP.
Another thing I dislike about REST is that it's not generally applicable. For instance, suppose you're developing a search service, like Google (a typical text-book RESTful Web Service example). Most everybody in the REST community agrees that you should use HTTP GET for the search and encode the query in the URI: http://www.google.be/search?q=jellyfish. Imagine a similar search service that allows you to search for images that look like a given image. Logically this is completely in-line with the simple search service, but because of technical limitations of HTTP GET you're forced to implement this in a different way (WS-* is more consistently applicable this way).

For me, it's all about sane software engineering. If you have a well designed service, using established standards and conventions where relevant, people are going to be able to use your service without much problems. Things like REST and WS-* all have there place as tools in a developers toolbox, and it's up to us developers to make informed decisions on when to use what to build the best possible software we can.

Concurrency Hides Latency

2009-12-13T08:06:00.000+01:00

To paraphrase Brian Goetz's excellent The Concurrency Revolution: The Hardware Story talk at Devoxx09: "concurrency hides latency". In other words: use concurrency to combat latency. This really seems to be the common theme in current-day software development.

In his talk, Brian explained how CPU speeds have increased at a much faster pace than memory speeds the last few decades. This has resulted in a situation where going to main memory is now extremely expensive in terms of CPU time. It could take several hundred clock cycles to fetch some data from main memory. Brian captured the problem in anther very quotable statement: "memory is the new disk". Hardware designers have been trying to minimize the impact of memory latency on computer performance by clever tricks like speculative execution and extensive caching. However, we've now come to a point where these techniques are breaking down because the gap between CPU and memory speed is just too big. As a result we're seeing more and more multi-core CPUs: slower in absolute terms but designed for concurrency so latency is less of a problem.

In web applications, a round-trip to the server involves a lot of latency, so web applications are doing more things on the client in JavaScript and hide this latency by concurrently doing other things: think AJAX. In a typical back-end business processing system, a round-trip to the database also involves a lot of latency, so we use techniques like event driven architectures to process several transactions concurrently, again hiding the latency cost.

More and more performance is about data, not code (another quote from Brian's talk)! I've seen several situations where the efficiency of a particular piece of software is completely determined by it's data access strategy. Relatively speaking, fetching data from the database is so expensive that it doesn't matter that you process that data in a naive or unoptimized way. All of this of course also implies that a key design question is the data access patterns and data structures used by the application. If you want a fast application, data is your key concern, and you can fight the latency of getting to the data by using concurrency.

Hidden Benefits from Commenting Your Code

2009-11-28T11:51:00.001+01:00

In his famous book Refactoring: Improving the Design of Existing Code Martin Fowler mentions comments as a possible code smell: if you need comments to explain what your code does, try to refactor it so that the comments become superfluous. Uncle Bob also mentions this same principle in his Clean Code book: most comments are bad comments, typically redundant or misleading.

I largely follow this reasoning. However, I've also seen an interesting benefit from trying to comment your code, especially with higher level comments, for instance JavaDoc comments on classes. Writing these kinds of comments forces you to explain things like the purpose and responsibility of the class. Personally, I've had several occasions where explaining things in plain English highlighted problems. If it's hard to explain what the class's purpose is, or what the reasoning behind the class or method names is, this typically means you have code problems. Maybe the class has unclear or too many responsibilities? Maybe the class name doesn't communicate its purpose well? Maybe the class's role in the system is confusing, and so on.

Nowadays, I try to keep comments in internal code to a minimum, but I do try to write a brief JavaDoc style comment for each class, forcing myself to go through that explaining exercise I just mentioned. I do the same thing for methods that are part of a public API: they also get brief JavaDoc style comments.

Page Flow Challenge Clarification

2009-11-24T17:32:00.001+01:00

The Page Flow Challenge in my previous post warrants a bit of clarification. A few readers had questions about the usefulness or sanity of certain steps in the test scenario. I'll go through the scenario and highlight what is being tested in each of the steps.

Start the page flow
On Page 1, enter "foo" and click Next
You should end up on Page 2 where "foo" is displayed as input from page 1
On Page 2, enter "baz" and click Next
You should end up on Page 3 where "foo" and "baz" are displayed as input from the previous pages
Click the browser back button
Without any browser warnings you should end up on Page 2, where "foo" is still displayed as input from page 1

The first part of the scenario sets up a basic set of data associated with the page flow: "foo" and "baz" at this point. At step 4 we're using the browser back button to backtrack. This tests POST-REDIRECT-GET behaviour: if the application does not use POST-REDIRECT-GET, the browser will show a warning before displaying Page 2 again. This is so because Page 2 is the result of posting the HTML form on Page 1, and modern browsers never automatically resubmit post data. Essentially we're protecting the user from accidental resubmits. Not having the browser show ugly warnings about resubmitting also makes the application feel stable and trustworthy.

Right click the Back link on the page and do Open Link in New Window
The new window should show Page 1 where "foo" is displayed in the input field
In the new window, enter "kit" (replacing "foo") and click Next
You should end up on Page 2 where "kit" is displayed as input from page 1
In the new window, on Page 2 enter "kat" and click Next
You should end up on Page 3 where "kit" and "kat" are displayed as input from the previous pages
In the original window, click the browser refresh button
Without any browser warnings you should see Page 2 again with "foo" as input from page 1

Step 5, admittedly a contrived step, simulates the user branching off the page flow into two separate windows. The point of doing this is testing whether or not the two branches are isolated. In steps 6 and 7 new data is entered ("kit" and "kat") that potentially interferes with the previously entered data ("foo" and "baz"). Step 8 verifies that no interference happened: the page flow in the original window still has "foo" as data.

In the original window, enter "bar" and click Next
You should end up on Page 3 with "foo" and "bar" as input from previous pages
In the original window, click Finish
You should end up on the start page of the application: the page flow has finished
In the original window, click the browser back button
You should either receive an error immediately, or when you try to resubmit by clicking Finish again

In step 9 and 10, we complete the page flow and finally submit values "foo" and "bar". At this point the page flow has finished. The state and navigation management of the framework should have invalidated the page flow at this point, making it impossible to resubmit. Step 11 verifies this: if you go back using the browser back button, you should not be allowed to resubmit. This part of the scenario tests intentional double submit protection (classically solved with something like a synchronizer token).

(For extra bonus points) In the new window, (where you still are on Page 3) click Finish
You should receive an error of some sort (potentially a automatic page flow restart)

The final step of the scenario tests whether or not the framework tracks the two branches of the page flow as being part of the same original conversation. This is an advanced form of double submit protection. In most cases this is a beneficial extra protection. In some cases this might not be what you want.

To make all of this a bit more tangible, let's suppose this is an airline ticket booking flow. The scenario would go something like this:
In step 2, on Page 1, the users fills in basic traveller information (name, age, seating class) and advances forward. In step 3, on Page 2, he selects a seating preference. Page 3 in step 4 shows an overview of the captured information and asks for confirmation. The user notices that he made a mistake and backs up.
Steps 5, 6 and 7 has the user thinking he might want to try a seat in a different seating class and compare the prices. Page 3 in step 7 displays an overview of the price with the new seating class.
In step 8 and 9 the user completes the page flow with the original seating class. On step 10 he compares the prices of the two alternatives and decides to submit the first of the two alternatives.
Steps 11 and 12 verify that the user cannot accidentally book two tickets, either by resubmitting the first alternative or submitting the second alternative. If he really wants another ticket, he'll have to start the ticket booking process (the page flow) again.

I hope this clarifies things a bit and shows that the test scenario is actually somewhat realistic.

A Page Flow Challenge

2009-11-22T11:17:00.009+01:00

In this post I setup a page flow challenge to find out if the new or up-and-coming web frameworks handle page flows well.

Why am I doing this? Over the last several years I've seen quite a few blogs and discussions where people talk about how web framework X handles page flows, typically comparing the page flow functionality in framework X with Spring Web Flow. The problem with most of these comparisons is that they focus on just two parts of the page flow puzzle: expressing page flows through some kind of DSL, and reusing page flows as modules throughout the application. There is an important third piece: managing navigation and the associated state. The reason why this is often overlooked is probably because the navigation and state management is completely automatic in Spring Web Flow, so many people don't even realize that it's there.

While attending a session on the Lift framework last week at Devoxx09, this issue came up again. Somebody (not me :) asked whether "Lift has page flow functionality like Spring Web Flow?". The answer was as expected: the Lift developers have a page flow DSL in the works ~~but seem to have missed the navigation and state management part~~ (update: this appears to be incorrect, Lift does have navigation and state management.).

It's not enough to be able to define a page flow, the framework also needs to manage it! To help users and framework developers realize this, I've setup a page flow challenge. The idea is that you develop a simple 3-page page flow in your framework of choice and test whether it can correctly run a somewhat contrived scenario that tests things like POST-REDIRECT-GET to avoid accidental resubmission, double submit protection to avoid intentional resubmission, and correct state management. In essence the page flow should behave correctly whatever the user does and protect the user from potential mistakes.

Let's look at the page flow:

Easy peasy. This could for instance be a flow to submit a payment in an electronic banking application, or a flow to handle checkout in a web shop.

The page flow challenge scenario is as follows (note: I've clarified this scenario in a follow-up post):

Start the page flow
On Page 1, enter "foo" and click Next
You should end up on Page 2 where "foo" is displayed as input from page 1
On Page 2, enter "baz" and click Next
You should end up on Page 3 where "foo" and "baz" are displayed as input from the previous pages
Click the browser back button
Without any browser warnings you should end up on Page 2, where "foo" is still displayed as input from page 1
Right click the Back link on the page and do Open Link in New Window
The new window should show Page 1 where "foo" is displayed in the input field
In the new window, enter "kit" (replacing "foo") and click Next
You should end up on Page 2 where "kit" is displayed as input from page 1
In the new window, on Page 2 enter "kat" and click Next
You should end up on Page 3 where "kit" and "kat" are displayed as input from the previous pages
In the original window, click the browser refresh button
Without any browser warnings you should see Page 2 again with "foo" as input from page 1
In the original window, enter "bar" and click Next
You should end up on Page 3 with "foo" and "bar" as input from previous pages
In the original window, click Finish
You should end up on the start page of the application: the page flow has finished
In the original window, click the browser back button
You should either receive an error immediately, or when you try to resubmit by clicking Finish again
(For extra bonus points) In the new window, (where you still are on Page 3) click Finish
You should receive an error of some sort (potentially a automatic page flow restart)

UPDATE: I've introduced a new step 11 in the above scenario, and what used to be step 11 is now step 12.

The only extra rule to keep in mind is that the HTML forms should use POST, to follow proper web design rules.

I've made reference implementations of the page flow challenge in Spring Web Flow 1 and 2. The source code is available in Subversion:

Spring Web Flow 1: https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf1/
Spring Web Flow 2: https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf2/

If you have Subversion and Maven installed, getting things up and running is as simple as:

$ svn export https://svn.ervacon.com/public/spring/samples/trunk/pageflowchallenge-swf2/ pageflowchallenge-swf2
$ cd pageflowchallenge-swf2
$ mvn jetty:run

You can now point your browser to http://localhost:8080/pageflowchallenge-swf2/ to test things out.

This challenge is solely focussed on page flow functionality, so it's not meant to reflect on the quality of a particular framework in other areas. For instance, SWF1 and SWF2 handle this scenario flawlessly, but probably have a harder time dealing with some other use-cases.

If you implemented the challenge in your web framework of choice, please let me know in the comments! I'm especially interested to see how Wicket, Seam, Lift and GWT fare. If you think this is irrelevant for your web framework because it handles things differently, I'd love to learn why you think this is the case, and how a use-case like this would then be implemented.

The Role of Entities in DDD

2009-11-14T10:24:00.002+01:00

In his famous book Domain-Driven Design, Eric Evans defines entities as follows:

An object defined primarily by its identity is called an ENTITY.

This implies that the primary responsibility of an entity is maintaining this thread of continuity, this identity.
Implementation wise, this is typically accomplished by using an identity field, a 'primary key' field as it were. Here is an example using JPA annotations:

@Entity
@Table(name = "PERS")
public class Person {

 @Id
 @Column(name = "ID")
 @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PERS_ID_SEQ")
 @SequenceGenerator(name = "PERS_ID_SEQ", sequenceName = "PERS_ID_SEQ", allocationSize = 20)
 private Long id;

 @Column(name = "NAME")
 private String name;

 @Column(name = "DOB")
 private Date dateOfBirth;

 public Long getId() {
  return this.id;
 }

 @SuppressWarnings("unused")
 private void setId(Long id) {
  this.id = id;
 }

 public String getName() {
  return this.name;
 }

 public void setName(String name) {
  this.name = name;
 }

 public Date getDateOfBirth() {
  return this.dateOfBirth;
 }

 public void setDateOfBirth(Date dateOfBirth) {
  this.dateOfBirth = dateOfBirth;
 }

 public String toString() {
  return this.name;
 }
}

Pretty straightforward code. Still, there's a lot of it and it will baloon quickly when you add more fields, constructors and serializability to the mix.

Say that we need to determine in some part of the application whether or not a person is a minor. Naively, we could put an isMinor() method on the Person class. However, the fact of the matter is that the core responsibility of the Person class is maintaining identity, and determining whether or not a person is a minor is not part of that responsibility. Futhermore, the size of the Person class indicates that it's already plenty busy with it's core responsibility.

Eric Evans also hints at this in his book:

Rather than focusing on the attributes or even the behaviour, strip the ENTITY object's definition down to the most intrinsic characteristics, particularly those that identify it or are commonly used to find or match it.

It's better to factor the new behaviour into a separate class. One way of doing this that I particularly like is using interpretation wrappers. Here's an example:

public class InterpretedPerson() {

 private Person person;

 public InterpretedPerson(Person person) {
  this.person = person;
 }

 public Person getPerson() {
  return this.person;
 }

 public int getAge() {
  // calculate the age based on person.getDateOfBirth()
  // ...
 }

 public boolean isMinor() {
  return getAge() < 18;
 }
}

This class simply interprets the data available in a Person entity, for instance using the dateOfBirth property to determine whether or not the person is a minor.Seperating data interpretation from the underlying entities brings several advantages:

The entity classes remain focussed on identity and the associated attributes.
You can have several interpretation wrappers, doing different kinds of interpretation (for instance, another part of the application might use another definition of what it means to be a minor).
Responsibilities are properly factored.

The interpretation wrapper is a useful little pattern that I haven't seen described elsewhere, so here you go!

On Testing and Refactoring

2009-11-07T21:10:00.007+01:00

The last few days there was quite a bit of commotion in the blogosphere around unit testing, TDD and testing in general. (InfoQ has some more background.) I don't want to get into that whole debate -- I'm a big fan of testing but I don't practice TDD religiously -- but I do want to touch on an interesting subject that came up: the way testing and refactoring relate.

There is a bit of a dual relationship between testing and refactoring. On one hand, good test coverage allows you to refactor with confidence: the tests tell you whether or not you broke things. On the other hand, if the refactoring you want to execute has a large impact on the test suite, you're held back in two ways: the extra work required to rework the tests, and the possibility of introducing bugs in the tests while reworking them, which reduces your refactoring confidence.

On the project I'm currently working on we've tackled this in an interesting way (nothing novel but worth pointing out nonetheless): next to the normal unit tests, we also have a large suite of what we call scenario tests. These scenario tests operate at a somewhat higher level and run against stable abstractions. Unlike unit tests they don't test isolated units of the application. Instead they test the system as an integrated whole and run real-life scenarios through it. However, the scenario tests are not really integration tests: all environmental dependencies are mocked out making it possible to run these scenario tests as part of the normal test suite with every build.

To make this a bit more tangible, lets consider a fictional event processing component:

public class CalculatingEventProcessor implements EventProcessor {

  private SomeCalculator calculator;
  private SomeRepository repository;

  ...

  @Override
  public void process(Event event) {
    if (weNeedToCalculateSomethingFor(event)) {
      SomeObject input = extractSomeObjectFrom(event);
      SomeOtherObject output = calculator.calculate(input);
      repository.store(output);
    }
  }

  ...
}

Instead of simply having unit tests for the SomeCalculator and SomeRepository classes, we build scenario tests at the level of the EventProcessor interface to verify that the correct things are calculated in the expected situations.

public interface EventProcessor {
  void process(Event event);
}

Such a scenario test would look something like this:

public class MyCalculationScenarioTest extends AbstractEventProcessorScenarioTest {

  @Override
  public EventProcessor setupEventProcessor() {
    return ...; // instantiate, or pull from a Spring app context, or ...
  }

  @Override
  public void runScenario() {
    // setup data as required by the scenario
    ...
    
    // create a relevant event
    Event event = ...;

    publishEvent(event);

    // the test infrastructure will run the event through the event processor

    // assert the necessary calculations have been done
    ...
  }
}

Note that the above test sets up all required data, typically in a throw-away database like HSQLDB. This is necessary to make sure that the scenario test can be run as a normal unit test: independent of any other tests or its environment.

It takes a bit of effort to setup the required scenario test infrastructure for your project, but the pay-off is big. Once you've got a sizeable set of scenario tests, you can refactor large parts of the internal design of your application without any impact on the scenario tests, which you can run at any time to see if things are moving in the right direction. The key here is the fact that the scenario tests run against a stable abstraction such as the EventProcessor interface in the example, which is unlikely to change.

UPDATE: It's worth pointing out that the scenario tests described above follow the basic form of a scenario in BDD (Behaviour-Driven Development):

Given some initial context (the givens),
When an event occurs,
then ensure some outcomes.

In BDD scenarios serve as an executable specification: a set of executable acceptance criteria that become proper end-to-end functional tests. The scenario tests described here serve the exact same purpose.

Event Driven Architecture for Systems that Never Stop

2009-11-02T16:05:00.005+01:00

InfoQ posted an excellent talk by Joe Armstrong (of Erlang fame) titled Systems that Never Stop (and Erlang). Joe presents 6 laws that he sees as crucial to systems that never stop (i.e. systems with X nines of reliability): Isolation, Concurrency, Failure Detection, Fault Identification, Live Code Upgrade, and Stable Storage.

The Erlang language and runtime were designed for exactly this kind of environment so it comes as no surpise that it shines in all areas. However, it occurred to me that an Event Driven Architecture goes a long way in building a system that never stops with more mainstream tools like Java and a relational database.

Isolation: Crashing of one process should not influence any other process, effectively isolating different processes. The big boon here is that this can really improve the reliability of your system: the probability of two independent processes failing is only half that of a single process failing.
In an event driven world events are typically handled independently of one-another. Failure to process a particular event does not affect the processing of any other event in the system.
Concurrency: A runtime environment should not put any undue constraints on how far you can take concurrency, i.e. the number of threads or processes in your system. Designing a system with concurrency in mind allows you to take advantage of current day hardware (multi-cores and the like), but also goes hand in hand with the first law: independent, isolated processes can run concurrently without much problem.
In an event driver system, multiple independent event processors can run concurrently on a single machine, taking advantage of all CPU cores available on the machine. Furthermore, event driven systems scale horizontally, allowing you to easily add additional machines with even more event processors.
Failure Detection: You must be able to detect when things go wrong, for instance by having a nanny like process that can monitor the system and can take appropriate action when failures occur. Furthermore, you should fail early, limiting the potential impact of the failure.
In an event driven system you typically have failover handling that holds on to events that failed processing for later analysis or retry. One system property that Joe does not mention in his talk but is very relevant in this area is idempotence: being able to retry things without affecting the final result. With some effort, idempotence comes naturally in an event driven system.
Fault Identification: If something has gone wrong, and you've detected it as per the previous law, you need to be able to identify what went wrong.
Again, in event driven systems this comes naturally: the event that failed processing is normally maintained by the failover system, along with error messages and the like, and can easily be used to simulate what happened.
Life Code Upgrade: You shouldn't need to bring down your system for an upgrade. Whether or not this is important to you is of course very dependent of the application that you're writing. Still, if you have a cluster of event processors, you can upgrade them one by one, keeping the system live at all time. On a single machine you could use things like OSGi to attain similar results.
Stable Storage: This essentially boils down to having transactional (ACID) storage available. In your average enterprise application this storage will be provided by an RDBMS (Oracle, ...). Most current-day MOM solutions (MQ, ...) are also completely transactional, and even XA aware. So there should be no need to ever loose any information in an event driven system build on top of these technologies.

So in all, an event driver architecture goes a long way in bringing you a system that never stops. This of course in stark contrast with batch systems, that have a hard time living up to a number of these laws.

Welcome!

2009-11-01T08:12:00.000+01:00

I've been thinking about this for quite a while but I've finally decided to start blogging! Hopefully this will prove interesting, useful or entertaining to at least a few people.
Welcome and enjoy!