Showing posts with label design. Show all posts
Showing posts with label design. Show all posts

Thursday, January 30, 2020

The Modular Monolith

Having been a micro-services sceptic from the start, and a proponent of the Monolith First approach, I've found Kamil Grzybek's Modular Monolith article series quite insightful:

Starting with a well designed and internally modularised monolith does seem to be the way to go. When architectural drivers change, like different modules evolving at different rates, it comes natural to split up the monolith into a few modules that are life-cycled and deployed independently, essentially creating micro-services. Scalability concerns in a module is another example of an architectural driver that often leads to such a split up.

You've got several tools at your disposal to keep your monolith properly modularised.

  • Designing proper module interfaces used by other parts of the system to interact with the module.
  • Use of Java packages to encapsulate modules, while keeping an eye on cyclomatic complexity.
  • The Java 9 module system. (Although I haven't used this myself yet.)
  • Smart use of multi-module builds in Maven. If you get this right, extracting a module to become its own micro-service might be a relatively trivial matter.
My basic advice would be: focussing on creating properly designed, modularised and high quality software first will make sure micro-services come easy later on.

Monday, September 17, 2018

Software maturity

While reading a news article on Domain Driven Design over at InfoQ, I came across an interesting metaphor attributed to Eric Evans (i.e. the father of DDD):
A new metaphor Evans introduced was comparing a large software system to a community garden. Looking past the obvious bounded contexts of people sharing space in the garden, he sees positive analogies to legacy systems, looking at the "abundance of maturity." Gardens are most valuable in late summer, when they are most productive. However, that is long past the stage when you can easily make changes to the garden, in early spring. Similarly, the most malleable phase for software is not when it is the most productive.
This puts an interesting perspective on something I've noticed in my own career. Just like all developers, I love getting started on a new greenfield project: things are new and exciting and together with the team you're constantly inventing new ways of tackling problems and (re)factoring the system accordingly.

Some time after the first go-live, which typically comes with a host of teething problems, the application stabilizes and slowly evolves into a mature system over the next several releases. You can no longer reinvent the wheel at this time: you're building on the foundations you established earlier.
I've found that I also get a great deal of satisfaction from this phase: shepherding a system to maturity, seeing it come to fruition and hopefully realizing it's full potential! It as this time users are most happy with what has been delivered: the software is well understood and quality is still high, making changes predictable and quick.

Over time quality slowly deteriorates and accidental complexity grows. Changes become more difficult and risky, with more regressions slipping in. Ultimately the system is replaced with something new and better, bringing everything full circle, just like in a garden.

Wednesday, August 29, 2018

Consistent Error Handling in a Spring Boot REST Service

Setting up a REST service using Spring Boot is simple. However, setting up consistent error handling for that service can be a bit more daunting. You'll typically want your service to use one particular error structure (JSON) for every error generated by the application. The question is: how do you do that in the simplest possible way with all the options you have available?
  • Option 1 - Servlet API Error Handling - The Servlet API itself contains an error handling system configured via web.xml. You can use the <error-page> element to specify the response handler (a JSP, Servlet, ...) for certain exceptions and HTTP status codes. In practice this means that if a Servlet or JSP generates an exception or calls HttpServletResponse.sendError(), the corresponding <error-page> page will be sent back to the client.
  • Option 2 - Spring MVC Error Handling - Spring MVC's DispatcherServlet comes with it's own error handling system, implemented using a HandlerExceptionResolver. In modern Spring applications this is typically setup using @ExceptionHandler @ControllerAdvice. When a request handler throws and exception, an appropriate exception handler is selected and tasked with resolving the error by generating an error response. Note that if an exception handler resolves the error, the exception never propagates up to the Servlet engine, meaning the defined Servlet error pages are not considered.
  • Option 3 - Spring Boot Error Handling - Spring Boot comes with a default error handling setup built on top of the Servlet API and Spring MVC error handling systems. The default setup (see ErrorMvcAutoConfiguration) does three things:
    1. It configures a Servlet <error-page> directed at /error for all exceptions and response codes.
    2. It defines an ErrorController handling /error requests, forwarded to it because of the configured Servlet error page. The default ErrorController implementation is the BasicErrorController, which will try to resolve an "error" view used to render a model prepared by an ErrorAttributes implementation. If the client is requesting text/html, the default whitelabel error page (see below) will be used, otherwise the model will be rendered directly as response body (typically JSON).
    3. The whitelabel error page serves as a simplistic HTML "error" view detailing the error that occurred by rendering the model prepared by the ErrorAttributes implementation.

That's a lot to take in! Customizing the default error structure (see DefaultErrorAttributes) involves several steps:

  1. Define a class for your own error structure, say ErrorInfo.
  2. Implement approprate @ExceptionHandlers returning ResponseEntity<ErrorInfo> objects, typically in a ResponseEntityExceptionHandler subclass.
  3. Implement an ErrorAttributes bean to return a Map (ugh!) corresponding with your ErrorInfo structure.
  4. Replace the whitelabel error page with one that can handle your ErrorInfo structure.
Although steps 1 and 2 make sense, steps 3 and 4 feel clumsy and unelegant.

Luckily there is a simpler way to do this if you're implementing a pure REST service that's only concerned with returning JSON responses, while still leveraging part of the default setup done by Spring Boot (ErrorMvcAutoConfiguration, specifically the /error Servlet error page).

  1. Of course you still need to define your own ErrorInfo structure.
  2. And you'll also still need to implement appropriate @ExceptionHandlers returning ResponseEntity<ErrorInfo> objects.
  3. Now implement an ErrorController that handles /error requests, returning an appropriate ResponseEntity<ErrorInfo>.
  4. Finally, disable the whitelabel error page by setting the server.error.whitelabel.enabled property to false.
That feels quite a bit better: all code uses your own ErrorInfo class to render an error and you don't need to spend time implementing an HTML error view for an application that should never output HTML anyway.

Sunday, December 25, 2011

Good code and good wine

I've discussed the "What is good code?" question before. This is such a broad topic that people come up with all kinds of interesting analogies to describe beneficial traits of good code. For instance, Uncle Bob often compares good code to a good book: it's important to have a good story (design/architecture), introduce key characters (concepts) at appropriate times and so on. Another interesting analogy that I've heard is that good code is like a leather glove: over time it becomes more flexible in areas where it needs to be flexible and more rigid in places that need more rigidity, exactly the way good code should behave and evolve (I'm not sure where I first heard this analogy and who I should credit for it).

Another analogy I (? -- I've never heard it before so I'll be so bold as to take credit myself) came up with recently is comparing good code with good wine. An important characteristic of good wine, maybe even the most important one, is that it tastes good. Likewise, good code needs to function properly and deliver the required functionality. However, as wine aficionados will tell you, tasting good is not the only important trait of a good wine: great wine also has things like an interesting texture, a complex taste palette and so on (not being a wine connaisseur myself, I won't dwell on it). Similarly, having a functional program doesn't imply good source code. Good code needs to exhibit extra characteristics such as flexibility, testability, adhering to the principle of least astonishment and so on.

I'd love to hear about more analogies used to describe good code!

Thursday, April 7, 2011

Software and Mud

There seems to be a strange relationship between software and mud. Many will be familiar with the Big Ball of Mud architectural style and there are countless examples of code that resembles mud more than anything else.

A while ago a colleague of mine used an analogy between a software related problem he was facing and wading through mud (or as we say in Dutch: "door de moose ploegen"). I'm amazed at how poignant this analogy actually is. Think of a situation where the first thing you notice is that you have mud on the soles of your shoes. As you trot on you quickly end up knee deep in mud. At this point you can either back out and retreat to solid ground, or persevere, in which case you often end up stuck!

Of course you might also make it through, as John Carmack once noted:
Yes, you can make windows do anything you want to if you have enough time to beat on it, but you can come out of it feeling like you just walked through a sewer. [.plan]

Many software related situations come to mind that closely follow this story:
  • Merging exercises gone wrong
  • Refactoring plans that start simple but spiral out of control
  • Anything to do with character encoding
  • ...
Feel free to add this mud analogy to your software vocabulary!

(As an aside: some people can wade through mud more elegantly than others, as Kate Moss illustrates.)

Thursday, November 11, 2010

To reuse or not to reuse, that's the question

It's clear that code duplication is a very bad code smell. As a result you use things like extract method refactoring to keep your code DRY. This brings up the question if this can be taken too far? Should all code duplication be avoided at all times?

Let me illustrate. Assume you have a bit of code like this:
public static void notNull(Object obj) throws IllegalArgumentException {
 if (obj == null) {
  throw new IllegalArgumentException("Argument cannot be null");
 }
}
This method is clearly designed for reuse. It's a small building block of code that can be used thoughout a piece of software to avoid duplicating null-checking code in several places. In this case there are also no real downsides to consider.

Now consider a somewhat higher level piece of (admittedly extremely contrived) code:
public void doSomeProcessing(String str) {
 checkPreconditions(str);
 ...
}

private void checkPreconditions(String str) {
 Check.notNull(str);
 if (!"A".equals(str) && !"B".equals(str)) {
  throw new IllegalArgumentException("Argument can only be A or B");
 }
}
As it stands right now the private checkPreconditions() method is not designed for reuse. If another component in the system happens to require the exact same preconditions, should you try to reuse it? In some situations the answer to this question is clearly: yes! If the new component intrinsically requires the same preconditions, it's natural to try to reuse the method and highlight the fact that the two components have some functional relation. However, in other situations the fact that the two components do the same pre-condition checks might be purely accidental. In other words: there is no functional relation between the two components. In this case refactoring the code to be able to reuse the checkPreconditions() method in both cases comes with a few downsides you should consider:
  • By reusing the method, the refactored code now potentially communicates a relationship between the two components, where there really is none.
  • If the preconditions of the two components are in reality unrelated, it would not be uncommon for the preconditions
    of one of the components to change, while the other component's preconditions remain the same. When making this change in the refactored code, you need to realise you can't just change the common checkPreconditions() method. Instead, a more complex code change will be required.
In my experience DRY code is certainly something to strive for. However, avoiding code duplication at all costs also comes at a cost!

Thursday, October 7, 2010

ThreadLocal naming conventions

Since Sonar is now part of our normal build platform on the project I'm currently doing, we're keeping a close eye on the violations it identifies. One rule we violate in a couple of places is the CheckStyle constant naming rule. This rule simply says that the names of all static final variables should be ALL_UPPERCASE_WITH_UNDERSCORES. This is of course a well known Java coding convention. Still, it feels a little unnatural if you apply it to ThreadLocals, which are technically static final variables, but are neither immutable nor do they have a global scope like typical constants. This makes code using ThreadLocals look a bit weird if you use normal Java constant naming, e.g. compare the following:
private static final ThreadLocal<Date> TIME_FRAME = new ThreadLocal<Date>();
...
TIME_FRAME.set(myDate);
private static final ThreadLocal<Date> timeFrame = new ThreadLocal<Date>();
...
timeFrame.set(myDate);
For me using normal variable naming conventions for ThreadLocals seems to better communicate their role and intented usage in the code. Does anybody have an idea what the official Java naming conventions for ThreadLocals are?

Friday, August 27, 2010

Killer tool: Sonar

Today a colleague at work showed Sonar to me, and I must say that I was really impressed! Sonar is an open source code quality analysis tool that uses a number of popular Java code analyzers like PMD, CheckStyle, FindBugs and Cobertura under the hood, and presents the metrics calculated by all of these in a consistent and integrated way. It also keeps track of all the metrics data in a database so you can see how your code quality evolves.

One part of Sonar is a Web front-end for the metrics database, and the other part is a Maven plugin that runs all code analyzers and pumps the collected data into that database. (As an aside: imagine how much harder it would be to develop a tool like this if there wasn't a dominant build solution in the Java space like Maven.)


As a package Sonar just ticks all the right boxes. It's really easy to get up and running (starting its DB/Web server and running mvn clean install sonar:sonar in your Maven project is all there is to it), feels absolutely solid, has a friendly UI and looks very polished overall.

Warmly recommended!

Saturday, May 1, 2010

What is good code?

As a project tech lead I do a lot of code review, so the "What is good code?" question comes up a lot. Of course there are many things that factor into the "good code" equation such as efficiency, correctness and elegance. For me the most important property of good code is that it should adhere to the principle of least astonishment. In other words: good code is code that does what you expect it to do in the way you expect it to be done.

Following this principle has important benefits when it comes to reading, understanding and maintaining a piece of code. It also typically improves the correctness of the code: you tend to get code that obviously has no deficiencies, in stead of no obvious deficiencies (as Tony Hoare said).

I was happy to learn that I'm in good company when it comes to attributing the principle of least astonishment to good code. Both Uncle Bob (in his excellent Clean Code book) and Peter Seibel (in his very interesting Coders at Work) ask several famous programmers about good code. A lot of the interviewees directly or indirectly mention the principle of least astonishment (for instance Ward Cunningham, Joe Armstrong and Simon Peyton Jones).

Saturday, February 13, 2010

Source Code Management Friendly Design

Good software design, especially proper modularization, brings many benefits such as single points of change, encapsulation and a reduced event horizon, just to name a few. This is clearly a Good Thing at the source code level, but it also has important source code management (SCM) implications. Let's look at an example.

Suppose your code needs to process some data in several different ways. When I say processing think about things like manipulating the data, reacting to it, and so on. The simplest thing that could possibly work would be something like this:
public class DataProcessingEngine {

 public void process(Data data) {
  processOneWay(data);
  processAnotherWay(data);
 }
 
 private void processOneWay(Data data) {
  // ...
 }
 
 private void processAnotherWay(Data data) {
  // ...
 }
}
This code has many issues. Purely at the source code level we have a class that just has too many responsibilities. At the SCM level, this class could become a merging nightmare. Imagine multiple development branches each adding new ways to process the data. Merging these branches back onto the trunk will almost certainly result in merging conflicts.

Of course we can do a lot better. Let's factor out the different ways to process the data into seperate DataProcessors:
public interface DataProcessor {
 void process(Data data);
}
Each different way of processing the data would have its own DataProcessor implementation. The DataProcessingEngine now becomes:
public class DataProcessingEngine {
 
 private List<DataProcessor> dataProcessors;
 
 public DataProcessingEngine(List<DataProcessor> dataProcessors) {
  this.dataProcessors = dataProcessors;
 }

 public void process(Data data) {
  for (DataProcessor dataProcessor : dataProcessors) {
   dataProcessor.process(data);
  }
 }
}
Of course we still need to configure the DataProcessingEngine with the appropriate DataProcessors somewhere:
public class DataProcessingEngineFactory {

 public static DataProcessingEngine create() {
  List<DataProcessor> dataProcessors = new ArrayList<DataProcessor>();
  dataProcessors.add(new OneWayDataProcessor());
  dataProcessors.add(new AnotherWayDataProcessor());
  return new DataProcessingEngine(dataProcessors);
 }
}
You could also use something like Spring to do this for you. In this case you would end up with bean definitions equivalent to the code above. At the code level this refactoring has pretty much solved the problem. We now have a few small classes, each with its own responsibility. However, at the SCM level, the DataProcessingEngineFactory class (or equivalent alternative configuration) still sits in a single file causing merge conflicts.

To solve this problem, we have to make the system a bit more dynamic. If the DataProcessingEngineFactory could automagically detect all available DataProcessor implementations, adding a new way of processing the data would be as simple as adding a new DataProcessor to the classpath. As a result, there would be no need to change the DataProcessingEngineFactory every time, an no more merge conflicts!

In Java you have quite a few options to implement such a dynamic discovery mechanism:

Using the Java 6 ServiceLoader, the DataProcessingEngineFactory would end up looking something like this:
public class DataProcessingEngineFactory {

 public static DataProcessingEngine create() {
  List<DataProcessor> dataProcessors = new ArrayList<DataProcessor>();
  for (DataProcessor dataProcessor : ServiceLoader.load(DataProcessor.class)) {
   dataProcessors.add(dataProcessor);
  }
  return new DataProcessingEngine(dataProcessors);
 }
}

It's interesting to note that annotation based configuration systems, which are all the rage, typically don't hold all configuration information in a single location, which helps ease your source code management as I've shown. XML based configuration is more problematic in this respect.

Friday, January 1, 2010

REST Reservations

I'm currently reading RESTful Web Services by Leonard Richardson and Sam Ruby, and happened to come across an interesting QCon talk by Mark Nottingham on the status of HTTP. To me, this talk carries an interesting critical undertone on REST.

I've followed the whole REST and RESTful Web Services hype with some interest the last few years, and genuinely like several aspects of REST:
  • You can't go wrong with a KISS approach to software development.
  • Most everybody is familiar with how the Web works, so exploiting that familiarity makes a lot of sense.
  • The HTTP underpinnings of REST make it very well supported by a multitude of tools.
  • By leveraging HTTP, REST has excellent support for intermediaries (i.e. proxies and caches) and all the cool things they can bring to the table.

On the other hand, I also have some reservations about the whole REST thing, and Mark Nottingham touched on several of these topics in his talk:
  • As always, there is a lot of dogma to go around, and everybody has his own flavour. Is using this or that HTTP method in a particular way RESTful? Mark, who's obviously intimately familiar with HTTP, seems to have a much more pragmatic view on how you can or should use HTTP (i.e. listen to his comments on POST at the very end of the talk).
  • HTTP is deceptively simple. The spec is BIG, and full of ambiguities. The entire first part of Mark's talk deals with this, and I would argue that most people (me included) only have a superficial familiarity with HTTP.
  • Another thing I dislike about REST is that it's not generally applicable. For instance, suppose you're developing a search service, like Google (a typical text-book RESTful Web Service example). Most everybody in the REST community agrees that you should use HTTP GET for the search and encode the query in the URI: http://www.google.be/search?q=jellyfish. Imagine a similar search service that allows you to search for images that look like a given image. Logically this is completely in-line with the simple search service, but because of technical limitations of HTTP GET you're forced to implement this in a different way (WS-* is more consistently applicable this way).

For me, it's all about sane software engineering. If you have a well designed service, using established standards and conventions where relevant, people are going to be able to use your service without much problems. Things like REST and WS-* all have there place as tools in a developers toolbox, and it's up to us developers to make informed decisions on when to use what to build the best possible software we can.

Sunday, December 13, 2009

Concurrency Hides Latency

To paraphrase Brian Goetz's excellent The Concurrency Revolution: The Hardware Story talk at Devoxx09: "concurrency hides latency". In other words: use concurrency to combat latency. This really seems to be the common theme in current-day software development.

In his talk, Brian explained how CPU speeds have increased at a much faster pace than memory speeds the last few decades. This has resulted in a situation where going to main memory is now extremely expensive in terms of CPU time. It could take several hundred clock cycles to fetch some data from main memory. Brian captured the problem in anther very quotable statement: "memory is the new disk". Hardware designers have been trying to minimize the impact of memory latency on computer performance by clever tricks like speculative execution and extensive caching. However, we've now come to a point where these techniques are breaking down because the gap between CPU and memory speed is just too big. As a result we're seeing more and more multi-core CPUs: slower in absolute terms but designed for concurrency so latency is less of a problem.

In web applications, a round-trip to the server involves a lot of latency, so web applications are doing more things on the client in JavaScript and hide this latency by concurrently doing other things: think AJAX. In a typical back-end business processing system, a round-trip to the database also involves a lot of latency, so we use techniques like event driven architectures to process several transactions concurrently, again hiding the latency cost.

More and more performance is about data, not code (another quote from Brian's talk)! I've seen several situations where the efficiency of a particular piece of software is completely determined by it's data access strategy. Relatively speaking, fetching data from the database is so expensive that it doesn't matter that you process that data in a naive or unoptimized way. All of this of course also implies that a key design question is the data access patterns and data structures used by the application. If you want a fast application, data is your key concern, and you can fight the latency of getting to the data by using concurrency.

Saturday, November 28, 2009

Hidden Benefits from Commenting Your Code

In his famous book Refactoring: Improving the Design of Existing Code Martin Fowler mentions comments as a possible code smell: if you need comments to explain what your code does, try to refactor it so that the comments become superfluous. Uncle Bob also mentions this same principle in his Clean Code book: most comments are bad comments, typically redundant or misleading.

I largely follow this reasoning. However, I've also seen an interesting benefit from trying to comment your code, especially with higher level comments, for instance JavaDoc comments on classes. Writing these kinds of comments forces you to explain things like the purpose and responsibility of the class. Personally, I've had several occasions where explaining things in plain English highlighted problems. If it's hard to explain what the class's purpose is, or what the reasoning behind the class or method names is, this typically means you have code problems. Maybe the class has unclear or too many responsibilities? Maybe the class name doesn't communicate its purpose well? Maybe the class's role in the system is confusing, and so on.

Nowadays, I try to keep comments in internal code to a minimum, but I do try to write a brief JavaDoc style comment for each class, forcing myself to go through that explaining exercise I just mentioned. I do the same thing for methods that are part of a public API: they also get brief JavaDoc style comments.

Saturday, November 14, 2009

The Role of Entities in DDD

In his famous book Domain-Driven Design, Eric Evans defines entities as follows:
An object defined primarily by its identity is called an ENTITY.
This implies that the primary responsibility of an entity is maintaining this thread of continuity, this identity.
Implementation wise, this is typically accomplished by using an identity field, a 'primary key' field as it were. Here is an example using JPA annotations:
@Entity
@Table(name = "PERS")
public class Person {

 @Id
 @Column(name = "ID")
 @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PERS_ID_SEQ")
 @SequenceGenerator(name = "PERS_ID_SEQ", sequenceName = "PERS_ID_SEQ", allocationSize = 20)
 private Long id;

 @Column(name = "NAME")
 private String name;

 @Column(name = "DOB")
 private Date dateOfBirth;

 public Long getId() {
  return this.id;
 }

 @SuppressWarnings("unused")
 private void setId(Long id) {
  this.id = id;
 }

 public String getName() {
  return this.name;
 }

 public void setName(String name) {
  this.name = name;
 }

 public Date getDateOfBirth() {
  return this.dateOfBirth;
 }

 public void setDateOfBirth(Date dateOfBirth) {
  this.dateOfBirth = dateOfBirth;
 }

 public String toString() {
  return this.name;
 }
}
Pretty straightforward code. Still, there's a lot of it and it will baloon quickly when you add more fields, constructors and serializability to the mix.

Say that we need to determine in some part of the application whether or not a person is a minor. Naively, we could put an isMinor() method on the Person class. However, the fact of the matter is that the core responsibility of the Person class is maintaining identity, and determining whether or not a person is a minor is not part of that responsibility. Futhermore, the size of the Person class indicates that it's already plenty busy with it's core responsibility.

Eric Evans also hints at this in his book:
Rather than focusing on the attributes or even the behaviour, strip the ENTITY object's definition down to the most intrinsic characteristics, particularly those that identify it or are commonly used to find or match it.
It's better to factor the new behaviour into a separate class. One way of doing this that I particularly like is using interpretation wrappers. Here's an example:
public class InterpretedPerson() {

 private Person person;

 public InterpretedPerson(Person person) {
  this.person = person;
 }

 public Person getPerson() {
  return this.person;
 }

 public int getAge() {
  // calculate the age based on person.getDateOfBirth()
  // ...
 }

 public boolean isMinor() {
  return getAge() < 18;
 }
}
This class simply interprets the data available in a Person entity, for instance using the dateOfBirth property to determine whether or not the person is a minor. Seperating data interpretation from the underlying entities brings several advantages:
  • The entity classes remain focussed on identity and the associated attributes.
  • You can have several interpretation wrappers, doing different kinds of interpretation (for instance, another part of the application might use another definition of what it means to be a minor).
  • Responsibilities are properly factored.
The interpretation wrapper is a useful little pattern that I haven't seen described elsewhere, so here you go!