Wednesday, August 23, 2023

Green-bar-shippable software

I can't remember where exactly I heard about the concept of "green-bar-shippable software", maybe it was in a conference talk or in some book, but somehow the Internet forgot and doesn't bring up anything useful. Lets rectify that and note it down for posterity's sake!

As the name suggests, green-bar-shippable software is software that can be shipped when running the test suite presents a "green bar" indicating all tests have succeeded.

Put differently, geen-bar-shippable software is software where the developers have such a high level of confidence in their automated tests that they consider a successful build sufficient sign off to release the software to customers.

I'm a big fan of this principle and I think it's the gold standard to strive for. Every bug that testers find or is discovered in a production deployment, essentially highlights a shortcoming in the test suite. Test driven development of course also leads in this direction.

Saturday, February 13, 2021

HTML over the wire

Yes please!

DHH is on to something with his recent post on HTML over the wire. It's high time that we start tearing down this gigantic mount of complexity that we've built the last two decades and get back to application development that developers can actually understand and excell with!

Friday, June 12, 2020

YAML vs XML

Remember how everybody used to complain about "programming in XML", for instance in Spring bean definition files? Luckily we've moved beyond that and are now in the brave new world of "programming in YAML", where we can spend (waste) hours in the cloud writing Kubernetes YAML manifests, Terraform YAML templates, and so on.
  • Even less readable!
  • No more syntax checks with proper schemas!
  • Go crazy fixing stupid indentation mistakes!
  • ...
Oh the joys! Sigh...

Thursday, January 30, 2020

The Modular Monolith

Having been a micro-services sceptic from the start, and a proponent of the Monolith First approach, I've found Kamil Grzybek's Modular Monolith article series quite insightful:

Starting with a well designed and internally modularised monolith does seem to be the way to go. When architectural drivers change, like different modules evolving at different rates, it comes natural to split up the monolith into a few modules that are life-cycled and deployed independently, essentially creating micro-services. Scalability concerns in a module is another example of an architectural driver that often leads to such a split up.

You've got several tools at your disposal to keep your monolith properly modularised.

  • Designing proper module interfaces used by other parts of the system to interact with the module.
  • Use of Java packages to encapsulate modules, while keeping an eye on cyclomatic complexity.
  • The Java 9 module system. (Although I haven't used this myself yet.)
  • Smart use of multi-module builds in Maven. If you get this right, extracting a module to become its own micro-service might be a relatively trivial matter.
My basic advice would be: focussing on creating properly designed, modularised and high quality software first will make sure micro-services come easy later on.

Friday, August 2, 2019

Adding a JDBC driver to Spring Cloud Data Flow server

A quick tech note on how you can add a JDBC driver dependency (jar) to the classpath when launching the spring-cloud-dataflow-server.

The Spring Cloud Data Flow documentation suggests rebuilding (!?!) when you need to add a custom JDBC driver. That seems a bit crazy to me so after a bit of Googling it turns out you can do it using the Spring Boot PropertiesLauncher:

java -cp spring-cloud-dataflow-server-2.2.0.RELEASE.jar -Dloader.path=lib \
   org.springframework.boot.loader.PropertiesLauncher \
   --spring.datasource.url=jdbc:mysql://localhost:3306/y9a?useSSL=false \
   --spring.datasource.username=root \
   --spring.datasource.password=password \
   --spring.datasource.driverClassName=com.mysql.jdbc.Driver
And you simply have to put the JDBC driver jar file in the lib/ directory:
spring-cloud-dataflow-server-2.2.0.RELEASE.jar
lib/
   mysql-connector-java-8.0.17.jar
Disco!

Tuesday, January 8, 2019

Building a WebSocket application with Spring Boot

This article shows you how to setup a very basic WebSocket application powered by Spring Boot. It uses the Java API for WebSocket (JSR-356) to define a WebSocket server endpoint and includes a very simple JavaScript WebSocket client.

As usual for a Spring Boot application, it all starts with the Maven POM:

  <?xml version="1.0" encoding="UTF-8"?>
  <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

      <modelVersion>4.0.0</modelVersion>
      <groupId>com.ervacon.sb</groupId>
      <artifactId>websocket</artifactId>
      <version>0.0.1-SNAPSHOT</version>

      <!-- Inherit defaults from Spring Boot -->
      <parent>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-parent</artifactId>
          <version>2.1.1.RELEASE</version>
      </parent>

      <!-- Add typical dependencies for a websocket application -->
      <dependencies>
          <dependency>
              <groupId>org.springframework.boot</groupId>
              <artifactId>spring-boot-starter-websocket</artifactId>
          </dependency>
      </dependencies>

      <!-- Package as an executable jar -->
      <build>
          <plugins>
              <plugin>
                  <groupId>org.springframework.boot</groupId>
                  <artifactId>spring-boot-maven-plugin</artifactId>
              </plugin>
          </plugins>
      </build>

  </project>
Notice how we pull in the spring-boot-starter-websocket dependency. This will make sure the Java API for WebSocket (javax.websocket) is on the classpath, together with Spring's support for WebSocket. When Spring Boot starts it will detect these dependencies and auto-configure the embedded Servlet engine (Tomcat by default) to handle the WebSocket protocol.

Next, we define our @SpringBootApplication class:

package com.ervacon.sb.websocket;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.web.socket.server.standard.ServerEndpointExporter;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    public ServerEndpointExporter serverEndpointExporter() {
        return new ServerEndpointExporter();
    }
}
The only special thing here is the fact that we register the ServerEndpointExporter, which will make sure our @ServerEndpoint annotated classes get registered with the WebSocket runtime.

We can now define our server side WebSocket endpoint:

package com.ervacon.sb.websocket;

import org.springframework.stereotype.Component;

import javax.websocket.OnMessage;
import javax.websocket.server.ServerEndpoint;

@Component
@ServerEndpoint("/socket")
public class WebSocketEndpoint {

    @OnMessage
    public String echo(String message) {
        return "Echo: " + message;
    }
}
Nothing to surprising here: we're just using the Java API for WebSocket (JSR-356) to define a method that will handle incoming text messages by simply echoing them back to the client. That client is implemented using some basic JavaScript code in a simple HTML page (add this as index.html to src/main/resources/static in your Spring Boot application):
<html>
  <head>
      <title>WebSocket Client</title>
      <!-- based on https://www.nexmo.com/blog/2018/10/08/create-websocket-server-spring-boot-dr/ -->
  </head>
  <body>
  <div class="container">
      <div id="messages" style="border: 1px solid gray; padding: 1em;"></div>
      <div class="input-fields">
          <p>Type your message:</p>
          <input id="message"/>
          <button id="send">Send</button>
      </div>
  </div>
  </body>
  <script>
      const messageWindow = document.getElementById("messages");

      const sendButton = document.getElementById("send");
      const messageInput = document.getElementById("message");

      const socket = new WebSocket("ws://localhost:8080/socket");

      socket.onopen = function (event) {
          addMessageToWindow("Connected");
      };

      socket.onmessage = function (event) {
          addMessageToWindow(`Got Message: ${event.data}`);
      };

      sendButton.onclick = function (event) {
          sendMessage(messageInput.value);
          messageInput.value = "";
      };

      function sendMessage(message) {
          socket.send(message);
          addMessageToWindow("Sent Message: " + message);
      }

      function addMessageToWindow(message) {
          messageWindow.innerHTML += `<div>${message}</div>`
      }
  </script>
</html>
We can now launch our Spring Boot application using a simple mvn spring-boot:run and test it by accessing http://localhost:8080/.

Monday, September 17, 2018

Software maturity

While reading a news article on Domain Driven Design over at InfoQ, I came across an interesting metaphor attributed to Eric Evans (i.e. the father of DDD):
A new metaphor Evans introduced was comparing a large software system to a community garden. Looking past the obvious bounded contexts of people sharing space in the garden, he sees positive analogies to legacy systems, looking at the "abundance of maturity." Gardens are most valuable in late summer, when they are most productive. However, that is long past the stage when you can easily make changes to the garden, in early spring. Similarly, the most malleable phase for software is not when it is the most productive.
This puts an interesting perspective on something I've noticed in my own career. Just like all developers, I love getting started on a new greenfield project: things are new and exciting and together with the team you're constantly inventing new ways of tackling problems and (re)factoring the system accordingly.

Some time after the first go-live, which typically comes with a host of teething problems, the application stabilizes and slowly evolves into a mature system over the next several releases. You can no longer reinvent the wheel at this time: you're building on the foundations you established earlier.
I've found that I also get a great deal of satisfaction from this phase: shepherding a system to maturity, seeing it come to fruition and hopefully realizing it's full potential! It as this time users are most happy with what has been delivered: the software is well understood and quality is still high, making changes predictable and quick.

Over time quality slowly deteriorates and accidental complexity grows. Changes become more difficult and risky, with more regressions slipping in. Ultimately the system is replaced with something new and better, bringing everything full circle, just like in a garden.

Wednesday, August 29, 2018

Consistent Error Handling in a Spring Boot REST Service

Setting up a REST service using Spring Boot is simple. However, setting up consistent error handling for that service can be a bit more daunting. You'll typically want your service to use one particular error structure (JSON) for every error generated by the application. The question is: how do you do that in the simplest possible way with all the options you have available?
  • Option 1 - Servlet API Error Handling - The Servlet API itself contains an error handling system configured via web.xml. You can use the <error-page> element to specify the response handler (a JSP, Servlet, ...) for certain exceptions and HTTP status codes. In practice this means that if a Servlet or JSP generates an exception or calls HttpServletResponse.sendError(), the corresponding <error-page> page will be sent back to the client.
  • Option 2 - Spring MVC Error Handling - Spring MVC's DispatcherServlet comes with it's own error handling system, implemented using a HandlerExceptionResolver. In modern Spring applications this is typically setup using @ExceptionHandler @ControllerAdvice. When a request handler throws and exception, an appropriate exception handler is selected and tasked with resolving the error by generating an error response. Note that if an exception handler resolves the error, the exception never propagates up to the Servlet engine, meaning the defined Servlet error pages are not considered.
  • Option 3 - Spring Boot Error Handling - Spring Boot comes with a default error handling setup built on top of the Servlet API and Spring MVC error handling systems. The default setup (see ErrorMvcAutoConfiguration) does three things:
    1. It configures a Servlet <error-page> directed at /error for all exceptions and response codes.
    2. It defines an ErrorController handling /error requests, forwarded to it because of the configured Servlet error page. The default ErrorController implementation is the BasicErrorController, which will try to resolve an "error" view used to render a model prepared by an ErrorAttributes implementation. If the client is requesting text/html, the default whitelabel error page (see below) will be used, otherwise the model will be rendered directly as response body (typically JSON).
    3. The whitelabel error page serves as a simplistic HTML "error" view detailing the error that occurred by rendering the model prepared by the ErrorAttributes implementation.

That's a lot to take in! Customizing the default error structure (see DefaultErrorAttributes) involves several steps:

  1. Define a class for your own error structure, say ErrorInfo.
  2. Implement approprate @ExceptionHandlers returning ResponseEntity<ErrorInfo> objects, typically in a ResponseEntityExceptionHandler subclass.
  3. Implement an ErrorAttributes bean to return a Map (ugh!) corresponding with your ErrorInfo structure.
  4. Replace the whitelabel error page with one that can handle your ErrorInfo structure.
Although steps 1 and 2 make sense, steps 3 and 4 feel clumsy and unelegant.

Luckily there is a simpler way to do this if you're implementing a pure REST service that's only concerned with returning JSON responses, while still leveraging part of the default setup done by Spring Boot (ErrorMvcAutoConfiguration, specifically the /error Servlet error page).

  1. Of course you still need to define your own ErrorInfo structure.
  2. And you'll also still need to implement appropriate @ExceptionHandlers returning ResponseEntity<ErrorInfo> objects.
  3. Now implement an ErrorController that handles /error requests, returning an appropriate ResponseEntity<ErrorInfo>.
  4. Finally, disable the whitelabel error page by setting the server.error.whitelabel.enabled property to false.
That feels quite a bit better: all code uses your own ErrorInfo class to render an error and you don't need to spend time implementing an HTML error view for an application that should never output HTML anyway.

Friday, August 17, 2018

Git and Command Line Productivity for Windows

If you're stuck on Windows (which I have to admit is not that bad nowadays), and want an easy and productive way to work with Git on the command line, I highly recommend Cmder!

Cmder doesn't need administrative privileges to be installed: you can simply unzip it and of you go! This makes it ideal for use on highly locked down machines you might find at some clients, typically large companies with strict computer policies. If you download the "full" version, it even comes with Git for Windows built in, along with a host of Unix utilities (grep, find, ...) giving you a very powerful and configurable tabbed command line experience.

Really an excellent piece of software that I highly recommend!

Friday, March 2, 2018

Nested Enums

Little known fact that I also only just found out about: nested enum types are implicitly static in Java (see the JLS). However, it is permitted for the declaration of a nested enum type to redundantly specify the static modifier. In other words, the following two enum declarations are effectively equivalent:
public class Foo {

   public static enum Bar {
   }
}


public class Foo {

   public enum Bar {
   }
}

Thursday, May 18, 2017

File.listFiles Gotcha

I had another WTF!? moment trying to get to the bottom of a NullPointerException we've been seeing. It turns out File.listFiles returns null when you're trying to list something that's not actually a directory, or when an I/O error occurs!
/**
 * Returns an array of abstract pathnames denoting the files in the
 * directory denoted by this abstract pathname.

...

 * @return  An array of abstract pathnames denoting the files and
 *          directories in the directory denoted by this abstract pathname.
 *          The array will be empty if the directory is empty.  Returns
 *          {@code null} if this abstract pathname does not denote a
 *          directory, or if an I/O error occurs.
 *
 * @throws  SecurityException
 *          If a security manager exists and its {@link
 *          SecurityManager#checkRead(String)} method denies read access to
 *          the directory
 *
 * @since  1.2
 */
public File[] listFiles() {
Good luck trying to find out which I/O error occurred! I'm guessing just throwing an IOException would have been too easy. Luckily Java 8 provides Files.list as an alternative which does throw an IOException if needed.

Friday, November 25, 2016

Java 8 Update of Bitemporal Framework

Just to push this out to the world a bit more: I've updated the "com.ervacon.bitemporal" framework to bring it up-to-date with current-day Java:
  • Use Java 8's new time classes instead of JodaTime
  • Leverage Java 8's goodies where relevant: streams, diamond operator, ...
  • Switched to Hibernate 5
  • Use JUnit 4
Just as before, "com.ervacon.bitemporal" provides a good starting point for those that need to tackle temporal issues in their applications. For more information, check the GitHub page: https://github.com/klr8/bitemporal.

Monday, May 23, 2016

Java Bean Validation Gotcha

A well known trick for doing cross field validation using Java Bean Validation (JSR-303) is to simply put an @AssertTrue annotation on a method that actually does the validation, like so:
public class Document {

   @NotNull
   private Status status;   
   private Date signingDate;

   @AssertTrue(message = "A signed document should have a signing date")
   public boolean isASignedDocumentHasASigningDate() {
      return status != SIGNED || signingDate != null;
   }
}
The fact that the assertion method actually returns a boolean also makes unit testing easy. So far so good!

However, there is a little known gotcha related to this: the annotated validation method actually has to follow the naming conventions of a boolean bean property getter, i.e. getBla() or isBla(). In other words the following would not work:

   @AssertTrue(message = "A signed document should have a signing date")
   public boolean aSignedDocumentHasASigningDate() {
      return status != SIGNED || signingDate != null;
   }
That's quite unfortunate, and certainly doesn't follow the principle of least astonishment: we're explicitly annotating the method so there is no reason why the bean validator should not pick it up! Even worse, unless you're thoroughly testing all you bean validation annotations, it's easy to miss the fact that the validation is not actually happening. Something to keep in mind!

Friday, December 11, 2015

The Benefits of Building on Multiple Platforms

Organizational policy at a client of mine recently saw me switching my development setup from an Ubuntu Linux based machine to Windows 7 Enterprise. Before this switch everything was Linux based: development was done in Linux, continuous integration ran on Linux, and we were ultimately also deploying to Linux, albeit another distribution.

Making the switch to Windows unearthed a few subtle coding errors, mainly related to resource management. It's no secret Windows is a lot stricter when it comes to file manipulation. For instance, consider the following:

File f = new File("test.tmp");
try (FileOutputStream fout = new FileOutputStream(f)) {
 fout.write("foo".getBytes());
 fout.flush();
 Files.move(f.toPath(), new File("test.txt").toPath());
}
Using Java 8, this runs fine on Linux. However, on Windows you get an error:
java.nio.file.FileSystemException: test.tmp -> test.txt: The process cannot access the file because it is being used by another process.
Looking at the code again you can see that the file is being moved inside the try-with-resources block. In other words, we're attempting to move the file before we've closed the output stream!

In general I would advise different developers in your development team to use different platforms. That will highlight these kind of subtle errors early and will generally improve the quality of your system. It of course comes as little surprise that Juergen Hoeller (of Spring fame) told me on several occasions that he's still developing on Windows for exactly this reason! :-)

Friday, November 20, 2015

Bizar replaceAll tricks

Image you want to replace all asterisk (*) characters in an input string with \*. In other words you want to escape them. One way of doing this in Java is using the String.replaceAll() method:
"foo*bar".replaceAll("\\*", "\\\\*");
To understand what's going on here, let's remind ourselves of what replaceAll() actually does:
/**
 * Replaces each substring of this string that matches the given regular expression
 * with the given replacement.
 * ...
 */
public String replaceAll(String regex, String replacement) {
So that already explains why the first argument to replaceAll() is "\\*": for it to be a valid regular expression we need to escape the asterisk (which of course means zero or more times in a regular expression) using a backslash, and we all know that a backslash character in a Java String needs to be escaped.

But what about the second argument? Shouldn't that just be "\\*" also: a backslash followed by an asterisk? It turns out replaceAll() doesn't treat the replacement as a simple string literal. The Javadoc states the following:

* Note that backslashes (\) and dollar signs ($) in the
* replacement string may cause the results to be different than if it were
* being treated as a literal replacement string
So just having "\\*" as the replacement string would mean we have a backslash in there which we again need to escape! Hence the "\\\\*". It's interesting to note that in a funny twist of fate this actually makes the code less bizar. If the replacement string would have been a simple literal the code would have been "foo*bar".replaceAll("\\*", "\\*");. Imagine coming across that gem when trying to maintain some old piece of code... :-)

Friday, September 4, 2015

Java Exception Fun

Look at this:
public class Test {

  public void f() {
  }

  public void g() {
    try {
      f();
    } catch (Exception e) {
      throw e;
    }
  }
}
Does this compile? Notice that method g() is throwing an exception object declared to be of type Exception but does not actually mention this in its throws clause. In Java 6 this indeed does not compile:
Test.java:10: unreported exception java.lang.Exception; must be caught or declared to be thrown
      throw e;
      ^
1 error
But it came as somewhat of a surprise to me that this compiles just fine in both Java 7 and Java 8! Thinking back this was of course part of the exception improvements made in Java 7. Goes to show that you can still come across little fun nuggets like this after all these years :-).

Saturday, April 25, 2015

Ubuntu 15.04 on Lenovo X1 Carbon 3rd Generation

Following up on my previous post, where I talked about installing Ubuntu 14.10 on my Lenovo X1 Carbon 3rd generation, I've now gone ahead and reinstalled the machine from scratch using the recently released Ubuntu 15.04.

I'm happy to tell you installation is now a complete breeze:

  • No more USB startup disk problems.
  • The brightness function keys work out-of-the-box.
  • The trackpoint buttons work out-of-the-box.
  • No more graphics glitches (mainly text rendering) in Unity.

It even seems that the default font sizes are better suited for a WQHD display, although I can't actually confirm anything has changed here compared to 14.10. In the end, the only things I did was setting the "Scale for menu and title bars" to 1,25 in the display settings and changing the "Page zoom" to 125% in Google Chrome.

In summary: Ubuntu 15.10 just works on the 3rd generation Lenovo X1 Carbon. No tweaking required really. I highly recommend this combination for those looking for a new Linux based laptop!

Saturday, February 28, 2015

Ubuntu 14.10 on Lenovo X1 Carbon 3rd Generation

I just switched my main laptop from an Asus Zenbook Prime UX31A to a Lenovo X1 Carbon 3rd generation (the 2015 model). Since I always run Ubuntu, I got started wiping the hard disk and installing Ubuntu 14.10 as the main operating system.

Although installing Linux on brand new hardware like the Lenovo X1 Carbon can be a hairy adventure, it turned out to be a pretty smooth ride. I thought I'd share what I had to do to get to an (almost) fully functional system.

  • The first snag I hit was trying to boot the Ubuntu 14.10 startup USB disk. I was greeted with a somewhat unnerving "gfxboot.c32: not a COM32R image" message and a "boot:" prompt. A quick search brought me to Ask Ubuntu: simply type "live" at the prompt and hit enter. You'll boot into the Live CD where you can start the installation.
    I haven't seen other people running Ubuntu 14.10 on their X1 Carbon complain about this, so I'm not sure why I ran into this problem.
  • The next issue I faced was the brightness function keys not working. Some more web searching revealed an Arch Linux thread compiling a number of issues people had encountered trying to run Linux on the 3rd generation X1 Carbon. Fixing the function key problem was again easy: just force the thinkpad_acpi module to load:
    echo thinkpad_acpi > /etc/modules-load.d/thinkpad_acpi.conf
    echo "options thinkpad_acpi force_load=1" > /etc/modprobe.d/thinkpad_acpi.conf 
    
  • My laptop sports a fancy 2560 x 1440 WQHD display ("1440p"). That's great and all but with the default Ubuntu fonts text gets really tiny. Working around that involved installing the unity-tweak-tool and setting the Text scaling factor to "1,20" (explained here). I also set the default "Page zoom" to 125% in Google Chrome.
And that's about it as far as I'm concerned. There are a few more things not working properly, like the track-point buttons or fingerprint reader, but these are things I never use so I didn't bother fixing those.

So far the system has been really sweet! The hardware seems up-to-par with my previous ThinkPad which I owned many years ago (a real IBM ThinkPad X40): really rugged feel and a superb keyboard. Combine this with good Linux support and you've got a winning combination!

Wednesday, October 1, 2014

SWF Humor

Funny little blast from the (Spring Web Flow) past. Back in 2006 while working on Spring Web Flow 1.0 we needed an exception class signaling a failure to parse a flow execution key. Keith Donald and myself jokingly considered calling it the FuckedUpFlowExecutionKeyException but ultimately settled for BadlyFormattedFlowExecutionKeyException. Indeed, it would be quite painful if a user was presented with a FuckedUpFlowExecutionKeyException stack trace after having accidentally corrupted the key in the URL of his browser. :-)

The joke continued for a little bit with the following piece of JavaDoc that you can find in the Spring Web Flow 1.0 source code:

package org.springframework.webflow.execution.repository;

/**
 * Thrown when an encoded flow execution key is badly formatted and could not be
 * parsed. We debated calling this the FuckedUpFlowExecutionKeyException.
 * 
 * @author Keith Donald
 * @author Erwin Vervaet
 */
public class BadlyFormattedFlowExecutionKeyException extends FlowExecutionRepositoryException {
This was not enough however, a particularly sensitive person actually complained about Spring Web Flow using offensive language! In the end this bit of nostalgia was kicked out in the 1.0.1 release. Still makes for a fun anecdote however! ;-)

Monday, June 16, 2014

Puzzling loop

Another fun Java puzzler that you would think you would never encounter in the wild but I happened to run across something similar a few days ago:
public class Puzzler {

 public static void main(String[] args) throws Exception {
  PuzzleFuction f = new PuzzleFuction();
  for (int i = 0; i < f.invoke(); i++) {
   System.out.println(i);
  }
 }

 public static class PuzzleFuction {

  private int invocationCount;

  public int invoke() {
   invocationCount++;
   return invocationCount < 2 ? 2 : 5;
  }
 }
}
What does this print? In other words: does the loop boundary (f.invoke()) get evaluated again every iteration? Of course it does! :-)
01234