Check caching of HTTP resources

Setting up HTTP caching can be a bit of pain. Essentially the HTTP response needs to contain appropriate cache control headers, either an Expires header or a Cache-Control max-age directive. Here's a quick Java program using the HttpClient and HttpClient Cache (both part of the Apache HttpComponents) to test client-side caching of HTTP responses.

public class TestHttpCaching {

 public static void main(String[] args) throws Exception {
  CacheConfig config = new CacheConfig();
  config.setMaxObjectSize(Long.MAX_VALUE);
  HttpClient httpClient = new CachingHttpClient(config);

  doGetRequest(httpClient, args[0]);
  doGetRequest(httpClient, args[0]);
 }

 private static void doGetRequest(HttpClient httpClient, String url) throws Exception {
  HttpGet httpGet = new HttpGet(url);
  HttpContext httpContext = new BasicHttpContext();
  try {
   System.out.print("Getting... ");
   System.out.print(httpClient.execute(httpGet, httpContext).getStatusLine());
   System.out.println(": " + httpContext.getAttribute(CachingHttpClient.CACHE_RESPONSE_STATUS));
  } finally {
   httpGet.releaseConnection();
  }
 }
}

The standard DefaultHttpClient does not do any caching. Instead you have to use the CachingHttpClient which wraps a DefaultHttpClient and adds caching functionality. Also notice how the cache is configured with a very large maximum object size. This avoids resources not being cached because they are too large for the standard in-memory cache (BasicHttpCacheStorage).

HTTP basic authentication in Grails with Spring Security

Setting up HTTP basic authentication in Grails using Spring Security is pretty straightforward. Here's a quick how-to:

1. grails create-app basicauthdemo
2. cd basicauthdemo
3. grails install-plugin spring-security-core
4. grails s2-quickstart basicauthdemo User Role
5. Edit grails-app/conf/Config.groovy and add two lines telling the Spring Security plugin to use HTTP basic authentication:

   grails.plugins.springsecurity.useBasicAuth = true
   grails.plugins.springsecurity.basic.realmName = "HTTP Basic Auth Demo"
   

6. Edit grails-app/conf/BootStrap.groovy to setup a user and role:

   import basicauthdemo.*
   
   class BootStrap {
   
       def init = { servletContext ->
           def userRole = Role.findByAuthority("ROLE_USER") ?: new Role(authority: "ROLE_USER").save(flush: true)
           def user = User.findByUsername("tst") ?: new User(username: "tst", password: "foo", enabled: true).save(flush: true)
           UserRole.create(user, userRole, true)
       }
       def destroy = {
       }
   }
   

7. grails create-controller hello
8. Edit grails-app/controllers/basicauthdemo/HelloController.groovy and add a security annotation:

   package basicauthdemo
   
   import grails.plugins.springsecurity.Secured
   
   class HelloController {
   
       @Secured(['ROLE_USER'])
       def index() {
           render "Hello World!"
       }
   }
   

9. grails run-app
10. Open http://localhost:8080/basicauthdemo/hello

Presto!

Watching digital HD TV without a Telenet decoder

And now for something completely different. If you live in Flanders and use Telenet as cable TV provider, it might be interesting to know that you can watch quite a few digital HD channels without using a Telenet decoder (either an HD Digibox or HD Digicorder; Telenet also has a FAQ article documenting this). All you need is:

• A Telenet cable TV subscription
• A television set with DVB-C tuner

Since every somewhat recent HD TV has a built-in DVB-C tuner, these requirements aren't very demanding. Just have your TV do a full scan of the digital spectrum. That takes quite a while and will find more than 200 channels. Of all those channels, you can currently watch the following unencrypted in their full digital HD 720p magnificence:

• één HD
• Canvas HD
• Ketnet/OP12
• France 3
• Arte Belgique
• TV5 Monde
• Actua TV
• CNBC Europe
• BBC World
• CNN

Next to these TV channels, you'll also be able to listen to a number of digital radio stations (STU BRU, ...). Of course there is one caveat: you won't be able to use interactive features such as Net Gemist.

Getting started with Spring Shell

I thought I'd write up a quick getting-started guide for Spring Shell. Spring Shell is the new Spring portfolio project that helps you in building easy to use command line interfaces for whatever commands you provide. Commands can really be anything. For instance, on most projects the developers end up writing a bunch of tools and utilities automating tedious tasks such as setting up a database schema, scanning through log files or doing some code generation. All of these would be perfect examples for what a Spring Shell command can be. Using Spring Shell to house all of your tooling and utility commands in a coherent shell makes them self documenting and easier to use for other developers on the team.

Let's setup a trivial Spring Shell application to get started. In follow-up posts I'll cover more advanced Spring Shell functionality. I'll be using Maven in this example because I think that's what most people are familiar with (Spring Shell itself is built with Gradle).

Here's the POM for the spring-shell-demo project (available on GitHub):

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.ervacon</groupId>
 <artifactId>spring-shell-demo</artifactId>
 <packaging>jar</packaging>
 <version>1.0-SNAPSHOT</version>
 <name>Spring Shell Demo</name>

 <properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 </properties>

 <repositories>
  <!-- see https://jira.springsource.org/browse/SHL-52 -->
  <repository>
   <id>ext-release-local</id>
   <url>http://repo.springsource.org/simple/ext-release-local/</url>
  </repository>
 </repositories>

 <dependencyManagement>
  <dependencies>
   <dependency>
    <groupId>org.springframework.shell</groupId>
    <artifactId>spring-shell</artifactId>
    <version>1.0.0.RELEASE</version>
   </dependency>
  </dependencies>
 </dependencyManagement>
 
 <dependencies>
  <dependency>
   <groupId>org.springframework.shell</groupId>
   <artifactId>spring-shell</artifactId>
  </dependency>
 </dependencies>
 
 <build>
  <plugins>
   <!-- copy all dependencies into a lib/ directory -->
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>2.1</version>
    <executions>
     <execution>
      <id>copy-dependencies</id>
      <phase>prepare-package</phase>
      <goals>
       <goal>copy-dependencies</goal>
      </goals>
      <configuration>
       <outputDirectory>${project.build.directory}/lib</outputDirectory>
       <overWriteReleases>true</overWriteReleases>
       <overWriteSnapshots>true</overWriteSnapshots>
       <overWriteIfNewer>true</overWriteIfNewer>
      </configuration>
     </execution>
    </executions>
   </plugin>
   
   <!-- make the jar executable by adding a Main-Class and Class-Path to the manifest -->
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <version>2.3.1</version>
    <configuration>
     <archive>
      <manifest>
       <addClasspath>true</addClasspath>
       <classpathPrefix>lib/</classpathPrefix>
       <mainClass>org.springframework.shell.Bootstrap</mainClass>
      </manifest>
     </archive>
    </configuration>
   </plugin>
  </plugins>
 </build>

</project>

There's quite a bit going on in this POM. First thing to note is the declaration of the ext-release-local SpringSource repository. This is needed to be able to resolve one of Spring Shell's dependencies: JLine (check SHL-52 for more background info). Hopefully this will no longer be necessary in future versions of Spring Shell. Next there's a dependency on Spring Shell itself. No surprise there. Finally, the POM customizes the build by copying all dependencies into a lib/ directory and adding a Main-Class and Class-Path property to the manifest of the generated jar file. Doing this produces the following directory structure in your target folder:

target/
 spring-shell-demo-1.0-SNAPSHOT.jar
 lib/
  all dependencies

You can simply package this up and distribute your shell. It will be fully self-contained and launching it is trivial:

java -jar spring-shell-demo-1.0-SNAPSHOT.jar

Hold on there, we're getting ahead of ourselves. Before launching the shell let's first add an echo command that just prints its input text back out to the console. Here's the code, which lives in the com.ervacon.ssd package:

@Component
public class DemoCommands implements CommandMarker {

 @CliCommand(value = "echo", help = "Echo a message")
 public String echo(
   @CliOption(key = { "", "msg" }, mandatory = true, help= "The message to echo") String msg) {
  return msg;
 }
}

As you can see, a Spring Shell command is just a @CliCommand annotated method on a Java class tagged with the CommandMarker interface. Our echo method takes a single argument which will be a mandatory @CliOption for the command. By using both the empty string and msg as keys for the option, you'll be able to invoke the command both as echo test and echo --msg test. The method simply returns the message: Spring Shell will make sure it gets printed to the console.

Alright, we've got our command implemented! We still have to tell Spring Shell about it. Since Spring Shell is Spring based, adding a command simply means defining a Spring bean. On start up, Spring Shell will automatically load the application context defined in classpath:/META-INF/spring/spring-shell-plugin.xml. You typically setup component scanning and simply mark your command classes as @Components, making development of new commands trivial since the shell will automatically detect them. Here's the application context definition:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:context="http://www.springframework.org/schema/context"
 xsi:schemaLocation="
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
  http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd">

 <context:component-scan base-package="com.ervacon.ssd" />

</beans>

And that's it! We're ready to build and launch our shell:

mvn package
java -jar target/spring-shell-demo-1.0-SNAPSHOT.jar

More mud: SimpleDateFormat and the Gregorian calendar

I spent another few hours wading through mud at work the other day, SimpleDateFormat related mud this time around. You would assume "yyyy/MM/dd HH:mm z" to be a pretty solid date format for representing a timestamp with minute precision:

• yyyy/MM/dd: the date, e.g. "2012/10/23"
• HH:mm: the time in 24-hour format, e.g. "18:32"
• z: the timezone, e.g. "CET"

This date format can be used to store dates in a text file (or any other textual format for that matter, e.g. XML). Let's go one step further and always use UTC as the time zone for the dates stored in our text file. That avoids all confusion when parsing the file again: all dates are expressed in UTC. Here's a bit of code that does what we need: it takes an input Date object and formats it as a UTC date string, it then parses that UTC date again and verifies that the input and output are the same.

Date input = new Date(-62135773200000L); // "0001/01/01 00:00 CET"

SimpleDateFormat utc = new SimpleDateFormat("yyyy/MM/dd HH:mm z");
utc.setTimeZone(TimeZone.getTimeZone("UTC"));

String str = utc.format(input);

Date output = utc.parse(str);

if (input.equals(output)) {
 System.out.println("Equal!");
} else {
 System.out.println(input + " != " + output);
}

For most dates this would print Equal!. However, I used a special date in the code above: midnight on January 1st of the year 1 expressed in CET. The output on my machine is:

Sat Jan 01 00:00:00 CET 1 != Sun Jan 01 00:00:00 CET 2

What!? Year 1 became 2? Let's look at the string produced by the date formatting:

0001/12/31 23:00 UTC

The time becomes 23:00 UTC because CET is one hour ahaid of UTC. To understand why the day jumped to December 31st of the year 1, you have to realize that the Gregorian calendar, on which UTC and CET are based, does not have a year 0. This means the timeline looks like this (BC is Before Christ, AD is Anno Domini, the era indicator in SimpleDateFormat terms):

..., 2 BC, 1 BC, 1 AD, 2 AD, ...

The date formatting assumes January first of the year 1 to be AD. To move from midnight CET to 23:00 UTC, we end up in the previous day: December 31st of the year 1 BC. However, our date format does not encode the era (BC / AD), so when the date is parsed, year 1 is again assumed to be AD, which leads to the rather surprising result that year 1 AD becomes year 2 AD after the UTC to CET time adjustment.

Luckily fixing the problem is easier than understanding it! You simply need to add the era indicator to the date format pattern: "yyyy/MM/dd HH:mm z G", and the above code will work as expected.

PS: This problem popped up during an XStream version upgrade. Check XSTR-556 and XSTR-711 in the XStream JIRA for more information.

Spring bean definition profiles and web.xml

Spring 3.1 was released a few weeks ago. One of the interesting new features is the first-class support for bean definition profiles: you can define beans as part of a profile and than activate that profile if required. Bean definitions that are part of a profile that has not been activated are simply skipped.

Last week, I spent a few days upgrading our application to Spring 3.1 and updating it to make use of bean definition profiles. As most other sizable Spring based apps, we had devised our own configuration switching system based on a combination of Maven resource filtering and XML <import/> elements containing ${placeholders}. This works well enough but it would obviously be better to directly use the bean definition profile feature of Spring 3.1, if only because it's better documented.

By default, the list of active bean definition profiles will be determined by looking at a spring.profiles.active property. Out-of-the-box, a Spring WebApplicationContext will setup a PropertySources list containing the servlet config parameters, servlet context parameters, JNDI properties, system properties and environment variables, in that order. This implies that you can define the spring.profiles.active property in any of those locations, the most obvious one being servlet context parameters in web.xml:

<context-param>
 <param-name>spring.profiles.active</param-name>
 <param-value>foo-profile, bar-profile</param-value>
</context-param>

We wanted to use Maven profiles to switch between Spring bean definition profiles, so the initial idea was to simply do Maven resource filtering on web.xml:

<context-param>
 <param-name>spring.profiles.active</param-name>
 <param-value>${spring.profiles.active}</param-value>
</context-param>

It turns out doing resource filtering on web.xml leads you to a world of pain:

• The Jetty plugin for Maven assumes the web.xml is static and simply uses the one in the source folder. You can explicitly tell it to use the web.xml from the target folder (using the <webXml> config element), but that results in other problems: by default resource filtering does not process the src/main/webapp directory. Of course you can specify filtering for that directory in your POM, but that again given problems: the Maven war plugin will overwrite the filtered web.xml with a copy from the source folder while packaging the web application. You can fix this by adding a <webResources> definition to your POM for the war plugin. Clearly this situation is far from ideal: it's complex and requires an explicit mvn package just to be able to do mvn jetty:run.
• Idem ditto for Tomcat.
• Eclipse WTP also directly uses the web.xml from the sources folder. Maybe there are ways around this, but I have no idea how that would work.

In the end we decided to forgo Maven resource filtering on web.xml and simply use an application specific properties file:

spring.profiles.active=${spring.profiles.active}

To get Spring to pick up this additional properties file, we simply add a property source to the application context using an ApplicationContextInitializer:

public class MyAppCtxInitializer
  implements ApplicationContextInitializer<ConfigurableApplicationContext> {
 public void initialize(ConfigurableApplicationContext applicationContext) {
  try {
   applicationContext.getEnvironment().getPropertySources()
     .addLast(new ResourcePropertySource("classpath:/myapp.properties"));
  } catch (IOException e) {
   // properties file cannot be found: ignore and just continue without these properties
   logger.info("Unable to load fallback properties: " + e.getMessage());
  }
 }
}

This approach avoids all the downsides of doing filtering on web.xml (web.xml is now completely static and mvn package no longer required). The only negative aspect is that it introduced an application specific properties file instead of simply relying on standard Spring conventions.

Good code and good wine

I've discussed the "What is good code?" question before. This is such a broad topic that people come up with all kinds of interesting analogies to describe beneficial traits of good code. For instance, Uncle Bob often compares good code to a good book: it's important to have a good story (design/architecture), introduce key characters (concepts) at appropriate times and so on. Another interesting analogy that I've heard is that good code is like a leather glove: over time it becomes more flexible in areas where it needs to be flexible and more rigid in places that need more rigidity, exactly the way good code should behave and evolve (I'm not sure where I first heard this analogy and who I should credit for it).

Another analogy I (? -- I've never heard it before so I'll be so bold as to take credit myself) came up with recently is comparing good code with good wine. An important characteristic of good wine, maybe even the most important one, is that it tastes good. Likewise, good code needs to function properly and deliver the required functionality. However, as wine aficionados will tell you, tasting good is not the only important trait of a good wine: great wine also has things like an interesting texture, a complex taste palette and so on (not being a wine connaisseur myself, I won't dwell on it). Similarly, having a functional program doesn't imply good source code. Good code needs to exhibit extra characteristics such as flexibility, testability, adhering to the principle of least astonishment and so on.

I'd love to hear about more analogies used to describe good code!