A Very Peculiar, but Possibly Cunning Kotlin Language Feature

This has caught me by surprise. After studying the Kotlin language to learn about how to best leverage this interesting new language for jOOQ, I stumbled upon this puzzler. What do you think the following program will print?

fun main(args: Array) {
    (1..5).forEach {
        if (it == 3)
            return
        print(it)
    }

    print("done")
}

Well… You might have guessed wrong. The above will print:

12

It will NOT print what most people might expect:

1245done

Note to those of you who are not surprised:

The above is peculiar for someone used to working with Java 8, where the following code will indeed print 1245done:

public static void main(String[] args) {
    IntStream.rangeClosed(1, 5).forEach(it -> {
        if (it == 3)
            return;

        System.out.print(it);
    });

    System.out.print("done");
}

The syntactical reason is explained in this section of the Kotlin manual:
https://kotlinlang.org/docs/reference/returns.html

In lambdas / closures, the return statement will not (necessarily) return from the lambda / closure, but from the immediate enclosing scope of the lambda / closure. The rationale has been kindly given to me by Dmitry Jemerov from JetBrains in two tweets:

Cunningly, the Kotlin language has removed language-based support for Java constructs like try-with-resources, or the synchronized statement. That’s very reasonable, because these language constructs don’t necessarily belong in the language (as we’ve previously claimed in another blog post), but could be moved to libraries instead. For example:

// try-with-resources is emulated using an
// extension function "use"
OutputStreamWriter(r.getOutputStream()).use {
    it.write('a')
}

(criticism here)

Or:

// Synchronized is a function!
val x = synchronized (lock, { computation() })

See also:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/synchronized.html

After all, even in Java, the language feature only works because the language depends on library types, like Iterable (foreach), AutoCloseable (try-with-resources), or JVM features (monitor on each reference for synchronized)

So, what’s the deal with return?

Along the lines of the above rationale, when language designers want to avoid language constructs for things that can be implemented with libraries, but still want you to feel like these were language constructs, then the only reasonable meaning of return inside of such a “construct-ish” lambda / closure is to return from the outer scope. So, when you write something like:

fun main(args : Array) {
    val lock = Object()
    val x = synchronized(lock, {
        if (1 == 1)
            return

        "1"
    })

    print(x)
}

The real intention is for this to be the equivalent of the following Java code:

public static void main(String[] args) {
    Object lock = new Object();
    String x;

    synchronized (lock) {
        if (1 == 1)
            return;

        x = "1";
    }

    System.out.println(x);
}

In the Java case, obviously, the return statement exits the main() method, because there is no other reasonable stack frame to return from. Unlike in Kotlin, where one might argue the lambda / closure would produce its own stack frame.

But it really doesn’t. The reason for this is the inline modifier on the synchronized function:

public inline fun <R> synchronized(lock: Any, block: () -> R): R {
    monitorEnter(lock)
    try {
        return block()
    }
    finally {
        monitorExit(lock)
    }
}

See also:
https://kotlinlang.org/docs/reference/inline-functions.html

Which means that the block closure passed as an argument isn’t really a pure lambda expression, but just syntactic sugar embedded in the call-site’s scope.

Weird. Cunning. Clever. But a bit unexpected.

Is this a good idea? Or will the language designers regret this, later on? Are all lambdas / closures potentially “language construct-ish”, where such a return statement is expected to leave the outer scope? Or are there clear cases where this inline behaviour just makes total sense?

We’ll see. In any case, it is very interesting for a language to have chosen this path.

Liked this article?

Stay tuned for a follow-up about the exciting Kotlin language. In the meantime, read

Java 8 Friday Goodies: SQL ResultSet Streams

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem. We have blogged a couple of times about some nice Java 8 goodies, and now we feel it’s time to start a new blog series, the…

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Java 8 Goodie: SQL ResultSet Streams

Yes, the SQL subject must be dealt with again. Even if last week, we promised an article on concurrency, there is one very important aspect of Java 8 lambdas and interoperability with “legacy” APIs that we need to talk about, first.

Checked Exceptions

Yes. Unfortunately, those beasts from the past still haunt us, more than ever when we’re using Java 8’s lambda expressions. Already before Java 8’s release, there are a couple of Stack Overflow questions related to the subject.

Let’s remember how the IOExceptions caused issues when traversing the file system. Unless you write your own utility, you’ll have to resort to this beauty:

Arrays.stream(dir.listFiles()).forEach(file -> {
    try {
        System.out.println(file.getCanonicalPath());
    }
    catch (IOException e) {
        throw new RuntimeException(e);
    }

    // Ouch, my fingers hurt! All this typing!
});

We think it is safe to say:

Java 8 and checked exceptions don’t match.tweet this

A workaround is to write your own CheckedConsumer that wraps the checked exception. Such a consumer will be highly reusable, but… Did you think of all the other FunctionalInterfaces? There are quite a few of them in the java.util.function package:

Some of the many types in java.util.function
Some of the many types in java.util.function

jOOλ – Fixing lambda in Java 8

jool-logo-blackWhile writing this Java 8 blog series, we’ve constantly run into the need to wrap checked exceptions inside lambda expressions. And what do we geeks do when we frequently run into a problem? We fix it! And we have created jOOλ (also jOOL, jOO-Lambda), ASL 2.0 licensed, where we have duplicated pretty much every FunctionalInterface that is available from the JDK to support checked exceptions. Here’s how you would use jOOλ in the above example:

Arrays.stream(dir.listFiles()).forEach(
    Unchecked.consumer(file -> {
        // Throw all sorts of checked exceptions
        // here, we don't care...
        System.out.println(file.getCanonicalPath());
    })
);

The above example shows how you can simply ignore and pass checked exceptions as RuntimeExceptions. If you actually want to handle them, you can pass an exception handler lambda:

Arrays.stream(dir.listFiles())
      .forEach(Unchecked.consumer(

    file -> {
        System.out.println(file.getCanonicalPath());
    },
    e -> {
        log.info("Log stuff here", e);
        throw new MyRuntimeException(e);
    }
);

The second example now seems equally verbose, but don’t worry. You will probably reuse that exception handler and fall back to this:

Arrays.stream(dir.listFiles())
      .forEach(Unchecked.consumer(
    file -> {
        System.out.println(file.getCanonicalPath());
    },
    myExceptionHandler
);

jOOλ – Providing JDBC ResultSet Streams

Unfortunately, most efforts in the Java 8 Streams API were made in the area of correctly implementing parallelisable streams. While this is very useful for those of us actually doing parallel computing, for most others better integration with legacy APIs would have been better. One API that seriously deserves some lifting is JDBC, and we’ve blogged about this before. With jOOλ, you can now generate Streams directly from ResultSets or even from PreparedStatements. Here’s how you prepare:

Class.forName("org.h2.Driver");
try (Connection c = getConnection()) {
    String sql = "select schema_name, is_default " +
                 "from information_schema.schemata " +
                 "order by schema_name";

    try (PreparedStatement stmt = c.prepareStatement(sql)) {
        // code here
    }
}

Now, all you have to do when using jOOλ is stream your PreparedStatements as such:

SQL.stream(stmt, Unchecked.function(rs ->
    new SQLGoodies.Schema(
        rs.getString("SCHEMA_NAME"),
        rs.getBoolean("IS_DEFAULT")
    )
))
.forEach(System.out::println);

Where SQLGoodies.Schema is just an ordinary POJO. Some of the stream() method’s signatures are these ones:

public static <T> Stream<T> stream(
    PreparedStatement stmt,
    Function<ResultSet, T> rowFunction
);

public static <T> Stream<T> stream(
    PreparedStatement stmt,
    Function<ResultSet, T> rowFunction,
    Consumer<? super SQLException> exceptionHandler
);

Others are available as well.

That is awesome, isn’t it?

JDBC ResultSets should be Java 8 Streams.tweet this

Too bad, the above code didn’t make it into the JDK 8, as this would have been a chance to finally greatly improve on the JDBC API. Another, similar attempt at improving things has been done here by Julian Exenberger.

Java 8 alternatives of writing SQL

We’ve also published a couple of alternatives to jOOλ, using Java 8 with SQL here:

http://www.jooq.org/java-8-and-sql

Conclusion

While Java 8’s lambda expressions are awesome, the new Streams API is pretty incomplete. When implementing the above, we had to implement our own ResultSetIterator, and write all this mess to wrap the iterator in a Stream:

StreamSupport.stream(
    Spliterators.spliteratorUnknownSize(
        new ResultSetIterator<>(
            supplier, 
            rowFunction, 
            exceptionTranslator
        ), 0
    ), false
);

And it shouldn’t be necessary to write an Iterator in the first place, if only we were able to generate finite streams:

// Unfortunately, this method doesn't exist
Stream.generate(
    // Supplier, generating new POJOs
    () -> { 
        rs.next(); 
        return new SQLGoodies.Schema(
            rs.getString("SCHEMA_NAME"),
            rs.getBoolean("IS_DEFAULT")
        );
    },

    // Predicate, terminating the Stream
    () -> { !rs.isLast(); }
);

While jOOλ is an acceptable intermediate solution, and the Guava guys are probably already working out how to fix their library, it is really too bad, that Java 8 is lacking such utility functionality.

But we’re complaining on a high level. Next week, as promised, we’ll see a couple of examples related to concurrency, so stay tuned!

More on Java 8

In the mean time, have a look at Eugen Paraschiv’s awesome Java 8 resources page

Java 8 Friday Goodies: Map Enhancements

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem. We have blogged a couple of times about some nice Java 8 goodies, and now we feel it’s time to start a new blog series, the…

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub. tweet this

Java 8 Goodie: Map Enhancements

In previous posts, we’ve already dealt with a couple of new Streams features, for instance when sorting. Most API improvements are indeed part of the new Streams API. But a few nice methods were also added to java.util.List and most importantly, to java.util.Map. If you want a quick overview of feature additions, go to the JDK8 Javadoc and click on the new “Default Methods” tab:

java.util.Map default methods
java.util.Map default methods

For backwards-compatibility reasons, all new methods added to Java interfaces are in fact default methods. So we have a couple of exciting new additions!

compute() methods

Often, we fetch a value from a map, make some calculations on it and put it back into the map. This can be verbose and hard to get right if concurrency is involved. With Java 8, we can pass a BiFunction to the new compute(), computeIfAbsent(), or computeIfPresent() methods and have the Map implementation handle the semantics of replacing a value.

The following example shows how this works:

// We'll be using this simple map
// Unfortunately, still no map literals in Java 8..
Map<String, Integer> map = new HashMap<>();
map.put("A", 1);
map.put("B", 2);
map.put("C", 3);

// Compute a new value for the existing key
System.out.println(map.compute("A", 
    (k, v) -> v == null ? 42 : v + 41));
System.out.println(map);

// This will add a new (key, value) pair
System.out.println(map.compute("X", 
    (k, v) -> v == null ? 42 : v + 41));
System.out.println(map);

The output of the above program is this:

42
{A=42, B=2, C=3}
42
{A=42, B=2, C=3, X=42}

This is really useful for ConcurrentHashMap, which ships with the following guarantee:

The entire method invocation is performed atomically. Some attempted update operations on this map by other threads may be blocked while computation is in progress, so the computation should be short and simple, and must not attempt to update any other mappings of this Map.

forEach() method

This is a really nice goodie which lets you pass a method reference or a lambda to receive (key, value) pairs one by one. A trivial example would be this:

map.forEach((k, v) -> 
    System.out.println(k + "=" + v));

Its output being:

A=1
B=2
C=3

merge() method

Now this one is really not so easy to understand. The Javadoc uses this example here:

map.merge(key, msg, String::concat)

Given the following contract:

If the specified key is not already associated with a value or is associated with null, associates it with the given value. Otherwise, replaces the value with the results of the given remapping function, or removes if the result is null.

So, the above code translates to the following atomic operation:

String value = map.get(key);
if (value == null)
    map.put(key, msg);
else
    map.put(key, value.concat(msg));

This is certainly not an everyday functionality and might just have leaked from an implementation to the top-level API. Additionally, if the map already contains null (so, null values are OK), and your remappingFunction returns null, then the entry is removed. That’s quite unexpected. Consider the following program:

map.put("X", null);
System.out.println(map.merge(
    "X", null, (v1, v2) -> null));
System.out.println(map);

Its output is:

null
{A=1, B=2, C=3}

Update: I first wrote the above code first with JDK 8 build 116. With build 129, things have changed completely again. First off, the value passed to merge() is not allowed to be null. Secondly. nullvalues are treated by merge() just like absent values. To produce the same output, we’ll write:

map.put("X", 1);
System.out.println(map.merge(
    "X", 1, (v1, v2) -> null));
System.out.println(map);

This merge() operation has thus removed a value from the map. That’s probably OK because the semantics of “merge” is often a combination of INSERT, UPDATE, and DELETE if we’re using SQL-speak. And a somewhat reasonable way to indicate that a value should be removed is to return null from such a function.

But the map is allowed to contain null values, which can never be inserted into the map using merge(). tweet this

getOrDefault()

This is a no-brainer. Right? Right! Wrong!

Unfortunately, there are two types of Maps. Those supporting null keys and/or values and those who don’t support nulls. While the previous merge() method didn’t distinguish between a map not containing a key and a map containing a key with a null value, this new getOrDefault() only returns the default when the key is not contained. It won’t protect you from a NullPointerException:

map.put("X", null);
try {
  System.out.println(map.getOrDefault("X", 21) + 21);
}
catch (NullPointerException nope) {
  nope.printStackTrace();
}

That’s quite a bummer. In general, it can be said the Map API has become even more complex with respect to nulls. tweet this

Trivial additions

There are a few more methods, like putIfAbsent() (pulled up from ConcurrentHashMap, remove() (with key and value arguments), replace().

Conclusion

All in all, it can be said that a lot of atomic operations have made it to the top-level Map API, which is good. But then again, the pre-existing confusion related to the semantics of null in maps has deepened. The terminologies “present” vs. “absent”, “contains”, “default” don’t necessarily help clarifying these things, which is surprisingly against the rules of keeping an API consistent and most importantly, regular. Thus as a consumer of this API, ideally, you should keep null out of maps, both as keys and as values!

Next week in this blog series, we’re going to look at how Java 8 will allow you to define local transactional scope very easily, so stay tuned!

More on Java 8

In the mean time, have a look at Eugen Paraschiv’s awesome Java 8 resources page

A Lesser-Known Java 8 Feature: Generalized Target-Type Inference

Going through the list of Java 8 features, Generalized Target-Type Inference struck me as a particularly interesting, lesser-known gem. It looks as though the Java language designers will ease some of the pain that we’ve been having with generics in the past (Java 5-7). Let’s have a look at their example:

class List<E> {
  static <Z> List<Z> nil() {..}
  static <Z> List<Z> cons(Z head, List<Z> tail) {..}
  E head() {..}
}

Given the above example, the JEP 101 feature claims that it would be nice to be able to write:

// This:
List.cons(42, List.nil());
String s = List.nil().head();

// ... instead of this:
List.cons(42, List.<Integer>nil());
String s = List.<String>nil().head();

Being a fluent API designer myself, I was thrilled to see that such an improvement is on the roadmap, particularly the latter. What’s so exciting about these changes? Let me comment on that more in detail:

// In addition to inferring generic types from
// assignments
List<String> l = List.nil();

// ... it would be nice for the compiler to be able
// to infer types from method argument types
List.cons(42, List.nil());

// ... or from "subsequent" method calls
String s = List.nil().head();

So in the last example where methods are chained, the type inference would be delayed until the whole assignment expression has been evaluated. From the left-hand side of the assignment, the compiler could infer that <Z> binds to String on the head() call. This information could then be used again to infer that <Z> binds again to String on the nil() call.

Sounds like a lot of trickery to me, as the nil() call’s AST evaluations would need to be delayed until a “dependent” sub-AST is evaluated. Is that a good idea?

Yes, this is so awesome!

… you may think. Because a fluent API like jOOQ or the Streams API could be designed in a much much more fluent style, delaying type inference until the end of the call chain.

So I downloaded the latest evaluation distribution of the JDK 8 to test this with the following program:

public class InferenceTest {
    public static void main(String[] args) {
        List<String> ls = List.nil();
        List.cons(42, List.nil());
        String s = List.nil().head();
    }
}

I compiled this and I got:

C:\Users\Lukas\java8>javac InferenceTest.java
InferenceTest.java:5: error: incompatible types: 
    Object cannot be converted to String
        String s = List.nil().head();
                                  ^
1 error

So, the type inference based on the method argument type is implemented (and thus, compiles), but not the type inference for chained method calls. I searched the internet for an explanation and found this Stack Overflow question linking to this interesting thread on the lambda-dev mailing list.

It appears that the Java type system has become quite complex. Too complex to implement such crazy type inference stuff. But still, a slight improvement that will be greatly valued when writing every day Java 8 code.

And maybe, in Java 9, we’ll get val and var, like everyone else ;-)

Does Java 8 Still Need LINQ? Or is it Better than LINQ?

Photo by Ade Oshineye
Erik Meijer, Tye Dye Expert. Photo by Ade Oshineye. Licensed under CC-BY-SA

LINQ was one of the best things that happened to the .NET software engineering ecosystem in a long time. With its introduction of lambda expressions and monads in Visual Studio 2008, it had catapulted the C# language way ahead of Java, which was at version 6 at the time, still discussing the pros and cons of generic type erasure. This achievement was mostly due to and accredited to Erik Meijer, a Dutch computer scientist and tye-dye expert who is now off to entirely other projects.

Where is Java now?

With the imminent release of Java 8 and JSR-355, do we still need LINQ? Many attempts of bringing LINQ goodness to Java have been made since the middle of the last decade. At the time, Quaere and Lambdaj seemed to be a promising implementation on the library level (not the language level). In fact, a huge amount of popular Stack Overflow questions hints at how many Java folks were (and still are!) actually looking for something equivalent:

Interestingly, “LINQ” has even made it into EL 3.0!

But do we really need LINQ?

LINQ has one major flaw, which is advertised as a feature, but in our opinion, will inevitably lead to the “next big impedance mismatch”. LINQ is inspired by SQL and this is not at all a good thing. LINQ is most popular for LINQ-to-Objects, which is a fine way of querying collections in .NET. The success of Haskell or Scala, however, has shown that the true functional nature of “collection querying” tends to employ entirely other terms than SELECT, WHERE, GROUP BY, or HAVING.  They use terms like “fold”, “map”, “flatMap”, “reduce”, and many many more. LINQ, on the other hand, employs a mixture of GROUP BY and terms like “skip”, “take” (instead of OFFSET and FETCH).

In fact, nothing could be further from the functional truth than a good old SQL partitioned outer join, grouping set, or framed window function. These constructs are mere declarations of what a SQL developer would like to see as a result. They’re not self-contained functions, which actually contain the logic to be executed in any given context. Moreover, window functions can be used only in SELECT and ORDER BY clauses, which is obvious when thinking in a declarative way, but which is also very weird if you don’t have the SQL context. Specifically, a window function in a SELECT clause influences the whole execution plan and the way indexes are employed to pre-fetch the right data.

Conversely, true functional programming can do so much more to in-memory collections than SQL ever can. Using a SQLesque API for collection querying was a cunning decision to trick “traditional” folks into adopting functional programming. But the hopes that collection and SQL table querying could be interfused were disappointed, as such constructs will not produce the wanted SQL execution plans.

But what if I am doing SQL?

It’s simple. When you do SQL, you have two essential choices.

  • Do it “top-down”, putting most focus on your Java domain model. In that case, use Hibernate / JPA for querying and transform Hibernate results using the Java 8 Streams API.
  • Do it “bottom-up”, putting most focus on your SQL / relational domain model. In that case, use JDBC or jOOQ and again, transform your results using the Java 8 Streams API.

This is illustrated more in detail here: http://www.hibernate-alternative.com

Don’t look back. Embrace the future!

While .NET was “ahead” of Java for a while, this was not due to LINQ itself. This was mainly due to the introduction of lambda expressions and the impact lambdas had on *ALL* APIs. LINQ is just one example of how such APIs could be constructed, although LINQ got most of the credit.

But I’m much more excited about Java 8’s new Streams API, and how it will embrace some functional programming in the Java ecosystem. A very good blog post by Informatech illustrates, how common LINQ expressions translate to Java 8 Streams API expressions.

So, don’t look back. Stop envying .NET developers. With Java 8, we will not need LINQ or any API that tries to imitate LINQ on the grounds of “unified querying”, which is a better sounding name for what is really the “query target impedance mismatch”. We need true SQL for relational database querying, and we need the Java 8 Streams API for functional transformations of in-memory collections. That’s it. Go Java 8!

Brian Goetz’s Final State of the Lambda Documentation

This week, Brian Goetz has published the final State of the Lambda documentation, which can be seen here:

These are exciting times in Java, as Java 8’s most impactful project has finally stabilised and can be downloaded here:

Fast File System Operations with Xtend, Lambdas, and ThreadPools

Recently, I’ve blogged about 10 Subtle Best Practices when Coding Java, and I have mentioned that you should start writing SAMs (Single Abstract Method) now, in order to be prepared for Java 8. But there’s another language gem out there, which comes in handy every once in a while, and that’s Eclipse Xtend. Xtend is a “dialect” of the Java language, compiling into Java source code, which then compiles into byte code.

Here’s a quickie showing how easily recursive file system operations can be done with Xtend, Lambdas, and ThreadPools.

class Transform {

  // This is the thread pool performing
  // all the "hard" work
  static ExecutorService ex;

  def static void main(String[] args) {
    // Initialise the thread pool with
    // something meaningful
    ex = Executors::newFixedThreadPool(4);

    // Pass the root directory to the
    // transform method
    val in = new File(...);

    // Recurse into the file transformation
    transform(in);
  }

  def static transform(File in) {

    // Calculate the target file name
    val out = new File(...);

    // Recurse into directories
    if (in.directory) {

      // Pass a FileFilter in the form of an
      // Xtend lambda expression
      for (file : in.listFiles[path |
             !path.name.endsWith(".class")
          && !path.name.endsWith(".zip")
          && !path.name.endsWith(".jar")
        ]) {
        transform(file);
      }
    }
    else {
      // Pass an Xtend lambda expression to
      // the ExecutorService
      ex.submit[ |
        // Read and write could be implemented
        // in Apache Commons IO
        write(out, transform(read(in)));
      ];
    }
  }

  def static transform(String content) {
    // Do the actual string transformation
  }
}

Granted, with Java 8, we’ll get lambdas as well, and that’s awesome. But Xtend has a couple of other nice features that can be seen above:

  • Passing lambdas to a couple of JDK methods, such as File.listFiles() or ExecutorService.submit()
  • Local variable type inference using valvar, or for
  • Method return type inference using def
  • Ability to omit parentheses when passing a lambda to a method
  • Calling getters and setters by convention, e.g. path.name, instead of path.getName(), or in.directory, instead of in.isDirectory()
  • You could also omit semi-colons, although I don’t personally think that’s a good idea.

Xtend is Java 8 already now, which can be very useful for scripts like the above

10 Subtle Best Practices when Coding Java

This is a list of 10 best practices that are more subtle than your average Josh Bloch Effective Java rule. While Josh Bloch’s list is very easy to learn and concerns everyday situations, this list here contains less common situations involving API / SPI design that may have a big effect nontheless.

I have encountered these things while writing and maintaining jOOQ, an internal DSL modelling SQL in Java. Being an internal DSL, jOOQ challenges Java compilers and generics to the max, combining generics, varargs and overloading in a way that Josh Bloch probably wouldn’t recommend for the “average API”.

jOOQ is the best way to write SQL in Java

Let me share with you 10 Subtle Best Practices When Coding Java:

1. Remember C++ destructors

Remember C++ destructors? No? Then you might be lucky as you never had to debug through any code leaving memory leaks due to allocated memory not having been freed after an object was removed. Thanks Sun/Oracle for implementing garbage collection!

But nonetheless, destructors have an interesting trait to them. It often makes sense to free memory in the inverse order of allocation. Keep this in mind in Java as well, when you’re operating with destructor-like semantics:

  • When using @Before and @After JUnit annotations
  • When allocating, freeing JDBC resources
  • When calling super methods

There are various other use cases. Here’s a concrete example showing how you might implement some event listener SPI:

@Override
public void beforeEvent(EventContext e) {
    super.beforeEvent(e);
    // Super code before my code
}

@Override
public void afterEvent(EventContext e) {
    // Super code after my code
    super.afterEvent(e);
}

Another good example showing why this can be important is the infamous Dining Philosophers problem. More info about the dining philosophers can be seen in this awesome post:
http://adit.io/posts/2013-05-11-The-Dining-Philosophers-Problem-With-Ron-Swanson.html

The Rule: Whenever you implement logic using before/after, allocate/free, take/return semantics, think about whether the after/free/return operation should perform stuff in the inverse order. tweet this

2. Don’t trust your early SPI evolution judgement

Providing an SPI to your consumers is an easy way to allow them to inject custom behaviour into your library / code. Beware, though, that your SPI evolution judgement may trick you into thinking that you’re (not) going to need that additional parameter. True, no functionality should be added early. But once you’ve published your SPI and once you’ve decided following semantic versioning, you’ll really regret having added a silly, one-argument method to your SPI when you realise that you might need another argument in some cases:

interface EventListener {
    // Bad
    void message(String message);
}

What if you also need a message ID and a message source? API evolution will prevent you from adding that parameter easily, to the above type. Granted, with Java 8, you could add a defender method, to “defend” you bad early design decision:

interface EventListener {
    // Bad
    default void message(String message) {
        message(message, null, null);
    }
    // Better?
    void message(
        String message,
        Integer id,
        MessageSource source
    );
}

Note, unfortunately, the defender method cannot be made final.

But much better than polluting your SPI with dozens of methods, use a context object (or argument object) just for this purpose.

interface MessageContext {
    String message();
    Integer id();
    MessageSource source();
}

interface EventListener {
    // Awesome!
    void message(MessageContext context);
}

You can evolve the MessageContext API much more easily than the EventListener SPI as fewer users will have implemented it.

The Rule: Whenever you specify an SPI, consider using context / parameter objects instead of writing methods with a fixed amount of parameters. tweet this

Remark: It is often a good idea to also communicate results through a dedicated MessageResult type, which can be constructed through a builder API. This will add even more SPI evolution flexibility to your SPI.

3. Avoid returning anonymous, local, or inner classes

Swing programmers probably have a couple of keyboard shortcuts to generate the code for their hundreds of anonymous classes. In many cases, creating them is nice as you can locally adhere to an interface, without going through the “hassle” of thinking about a full SPI subtype lifecycle.

But you should not use anonymous, local, or inner classes too often for a simple reason: They keep a reference to the outer instance. And they will drag that outer instance to wherevery they go, e.g. to some scope outside of your local class if you’re not careful. This can be a major source for memory leaks, as your whole object graph will suddenly entangle in subtle ways.

The Rule: Whenever you write an anonymous, local or inner class, check if you can make it static or even a regular top-level class. Avoid returning anonymous, local or inner class instances from methods to the outside scope. tweet this

Remark: There has been some clever practice around double-curly braces for simple object instantiation:

new HashMap<String, String>() {{
    put("1", "a");
    put("2", "b");
}}

This leverages Java’s instance initializer as specified by the JLS §8.6. Looks nice (maybe a bit weird), but is really a bad idea. What would otherwise be a completely independent HashMap instance now keeps a reference to the outer instance, whatever that just happens to be. Besides, you’ll create an additional class for the class loader to manage.

4. Start writing SAMs now!

Java 8 is knocking on the door. And with Java 8 come lambdas, whether you like them or not. Your API consumers may like them, though, and you better make sure that they can make use of them as often as possible. Hence, unless your API accepts simple “scalar” types such as int, long, String, Date, let your API accept SAMs as often as possible.

What’s a SAM? A SAM is a Single Abstract Method [Type]. Also known as a functional interface, soon to be annotated with the @FunctionalInterface annotation. This goes well with rule number 2, where EventListener is in fact a SAM. The best SAMs are those with single arguments, as they will further simplify writing of a lambda. Imagine writing

listeners.add(c -> System.out.println(c.message()));

Instead of

listeners.add(new EventListener() {
    @Override
    public void message(MessageContext c) {
        System.out.println(c.message()));
    }
});

Imagine XML processing through jOOX, which features a couple of SAMs:

$(document)
    // Find elements with an ID
    .find(c -> $(c).id() != null)
    // Find their  child elements
    .children(c -> $(c).tag().equals("order"))
    // Print all matches
    .each(c -> System.out.println($(c)))

The Rule: Be nice with your API consumers and write SAMs / Functional interfaces already now. tweet this

Remarks: A couple of interesting blog posts about Java 8 Lambdas and improved Collections API can be seen here:

5. Avoid returning null from API methods

I’ve blogged about Java’s NULLs once or twice. I’ve also blogged about Java 8’s introduction of Optional. These are interesting topics both from an academic and from a practical point of view.

While NULLs and NullPointerExceptions will probably stay a major pain in Java for a while, you can still design your API in a way that users will not run into any issues. Try to avoid returning null from API methods whenever possible. Your API consumers should be able to chain methods whenever applicable:

initialise(someArgument).calculate(data).dispatch();

In the above snippet, none of the methods should ever return null. In fact, using null’s semantics (the absence of a value) should be rather exceptional in general. In libraries like jQuery (or jOOX, a Java port thereof), nulls are completely avoided as you’re always operating on iterable objects. Whether you match something or not is irrelevant to the next method call.

Nulls often arise also because of lazy initialisation. In many cases, lazy initialisation can be avoided too, without any significant performance impact. In fact, lazy initialisation should be used carefully, only. If large data structures are involved.

The Rule: Avoid returning nulls from methods whenever possible. Use null only for the “uninitialised” or “absent” semantics. tweet this

6. Never return null arrays or lists from API methods

While there are some cases when returning nulls from methods is OK, there is absolutely no use case of returning null arrays or null collections! Let’s consider the hideous java.io.File.list() method. It returns:

An array of strings naming the files and directories in the directory denoted by this abstract pathname. The array will be empty if the directory is empty. Returns null if this abstract pathname does not denote a directory, or if an I/O error occurs.

Hence, the correct way to deal with this method is

File directory = // ...

if (directory.isDirectory()) {
    String[] list = directory.list();

    if (list != null) {
        for (String file : list) {
            // ...
        }
    }
}

Was that null check really necessary? Most I/O operations produce IOExceptions, but this one returns null. Null cannot hold any error message indicating why the I/O error occurred. So this is wrong in three ways:

  • Null does not help in finding the error
  • Null does not allow to distinguish I/O errors from the File instance not being a directory
  • Everyone will keep forgetting about null, here

In collection contexts, the notion of “absence” is best implemented by empty arrays or collections. Having an “absent” array or collection is hardly ever useful, except again, for lazy initialisation.

The Rule: Arrays or Collections should never be null. tweet this

7. Avoid state, be functional

What’s nice about HTTP is the fact that it is stateless. All relevant state is transferred in each request and in each response. This is essential to the naming of REST: Representational State Transfer. This is awesome when done in Java as well. Think of it in terms of rule number 2 when methods receive stateful parameter objects. Things can be so much simpler if state is transferred in such objects, rather than manipulated from the outside. Take JDBC, for instance. The following example fetches a cursor from a stored procedure:

CallableStatement s =
  connection.prepareCall("{ ? = ... }");

// Verbose manipulation of statement state:
s.registerOutParameter(1, cursor);
s.setString(2, "abc");
s.execute();
ResultSet rs = s.getObject(1);

// Verbose manipulation of result set state:
rs.next();
rs.next();

These are the things that make JDBC such an awkward API to deal with. Each object is incredibly stateful and hard to manipulate. Concretely, there are two major issues:

  • It is very hard to correctly deal with stateful APIs in multi-threaded environments
  • It is very hard to make stateful resources globally available, as the state is not documented
State is like a box of chocolates
Theatrical poster for Forrest Gump, Copyright © 1994 by Paramount Pictures. All Rights Reserved. It is believed that the above usage fulfils what is known as Fair Use

The Rule: Implement more of a functional style. Pass state through method arguments. Manipulate less object state. tweet this

8. Short-circuit equals()

This is a low-hanging fruit. In large object graphs, you can gain significantly in terms of performance, if all your objects’ equals() methods dirt-cheaply compare for identity first:

@Override
public boolean equals(Object other) {
    if (this == other) return true;

    // Rest of equality logic...
}

Note, other short-circuit checks may involve null checks, which should be there as well:

@Override
public boolean equals(Object other) {
    if (this == other) return true;
    if (other == null) return false;

    // Rest of equality logic...
}

The Rule: Short-circuit all your equals() methods to gain performance. tweet this

9. Try to make methods final by default

Some will disagree on this, as making things final by default is quite the opposite of what Java developers are used to. But if you’re in full control of all source code, there’s absolutely nothing wrong with making methods final by default, because:

  • If you do need to override a method (do you really?), you can still remove the final keyword
  • You will never accidentally override any method anymore

This specifically applies for static methods, where “overriding” (actually, shadowing) hardly ever makes sense. I’ve come across a very bad example of shadowing static methods in Apache Tika, recently. Consider:

TikaInputStream extends TaggedInputStream and shadows its static get() method with quite a different implementation.

Unlike regular methods, static methods don’t override each other, as the call-site binds a static method invocation at compile-time. If you’re unlucky, you might just get the wrong method accidentally.

The Rule: If you’re in full control of your API, try making as many methods as possible final by default. tweet this

10. Avoid the method(T…) signature

There’s nothing wrong with the occasional “accept-all” varargs method that accepts an Object... argument:

void acceptAll(Object... all);

Writing such a method brings a little JavaScript feeling to the Java ecosystem. Of course, you probably want to restrict the actual type to something more confined in a real-world situation, e.g. String.... And because you don’t want to confine too much, you might think it is a good idea to replace Object by a generic T:

void acceptAll(T... all);

But it’s not. T can always be inferred to Object. In fact, you might as well just not use generics with the above methods. More importantly, you may think that you can overload the above method, but you cannot:

void acceptAll(T... all);
void acceptAll(String message, T... all);

This looks as though you could optionally pass a String message to the method. But what happens to this call here?

acceptAll("Message", 123, "abc");

The compiler will infer <? extends Serializable & Comparable<?>> for T, which makes the call ambiguous!

So, whenever you have an “accept-all” signature (even if it is generic), you will never again be able to typesafely overload it. API consumers may just be lucky enough to “accidentally” have the compiler chose the “right” most specific method. But they may as well be tricked into using the “accept-all” method or they may not be able to call any method at all.

The Rule: Avoid “accept-all” signatures if you can. And if you cannot, never overload such a method. tweet this

Conclusion

Java is a beast. Unlike other, fancier languages, it has evolved slowly to what it is today. And that’s probably a good thing, because already at the speed of development of Java, there are hundreds of caveats, which can only be mastered through years of experience.

jOOQ is the best way to write SQL in Java

Stay tuned for more top 10 lists on the subject!

JDK 8: State of the Collections

Here’s the latest publication by Brian Goetz, Oracle’s project lead for JSR 335, a.k.a. Project Lambda. Here’s a nice example showing new collection features, such as “Streams” using method references:

List<String> strings = ...
int sumOfLengths = strings.stream()
                          .map(String::length)
                          .reduce(0, Integer::plus);

Another nice example showing the use of lambda expressions:

int sum = shapes.stream()
                .filter(s -> s.getColor() == BLUE)
                .map(s -> s.getWeight())
                .sum();

See more, here:
http://cr.openjdk.java.net/~briangoetz/lambda/sotc3.html

Exciting ideas in Java 8: Streams

Brian Goetz’s recent post on the State of the Lambda reveils exciting new ideas that are prone to be included in Java 8. One of them is the concept of “Streams” as opposed to “Collections”. Using the new Java 8 extension methods, the Iterable interface can be extended compatibly with a lot of “lazy” and “eager” methods for streaming behaviour:

public interface Iterable<T> {
    // Abstract methods
    Iterator<T> iterator();

    // Lazy operations
    Iterable<T> filter(Predicate<? super T> predicate) default ...
    <U> Iterable<U> map(Mapper<? super T, ? extends U> mapper) default ...
    <U> Iterable<U> flatMap(
        Mapper<? super T, ? extends Iterable<U>> mapper) default ...
    Iterable<T> cumulate(BinaryOperator<T> op) default ...

    Iterable<T> sorted(
        Comparator<? super T> comparator) default ...
    <U extends Comparable<? super U>> Iterable<T> sortedBy(
        Mapper<? super T, U> extractor) default ...
    Iterable<T> uniqueElements() default ...

    <U> Iterable<U> pipeline(
        Mapper<Iterable<T>, ? extends Iterable<U>> mapper) default ...
    <U> BiStream<T, U> mapped(
        Mapper<? super T, ? extends U> mapper) default ...
    <U> BiStream<U, Iterable<T>> groupBy(
        Mapper<? super T, ? extends U> mapper) default ...
    <U> BiStream<U, Iterable<T>> groupByMulti(
        Mapper<? super T, ? extends Iterable<U>> mapper) default ...

    // Eager operations
    boolean isEmpty() default ...;
    long count() default ...
    T getFirst() default ...
    T getOnly() default ...
    T getAny() default ...

    void forEach(Block<? super T> block) default ...
    T reduce(T base, BinaryOperator<T> reducer) default ...
    <A extends Fillable<? super T>> A into(A target) default ...

    boolean anyMatch(Predicate<? super T> filter) default ...
    boolean noneMatch(Predicate<? super T> filter) default ...
    boolean allMatch(Predicate<? super T> filter) default ...

    <U extends Comparable<? super U>> T maxBy(
        Mapper<? super T, U> extractor) default ...
    <U extends Comparable<? super U>> T minBy(
        Mapper<? super T, U> extractor) default ...
}

The above are rough and incomplete API ideas, or as Brian Goetz puts it: A “strawman writeup of the key concepts”.

The goal of this early publication is for the community to “TRY OUT THE CODE. *The single most valuable thing that the community can do to help move this effort forward, and improve the quality of the result, is to try out the code early and often.*”

Read the full article here:

http://cr.openjdk.java.net/~briangoetz/lambda/collections-overview.html