Java 8 Friday: The Dark Side of Java 8

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

The dark side of Java 8

So far, we’ve been showing the thrilling parts of this new major release. But there are also caveats. Lots of them. Things that

  • … are confusing
  • … are wrong
  • … are omitted (for now)
  • … are omitted (for long)

There are always two sides to Java major releases. On the bright side, we get lots of new functionality that most people would say was overdue. Other languages, platforms have had generics long before Java 5. Other languages, platforms have had lambdas long before Java 8. But now, we finally have these features. In the usual quirky Java-way.

Lambda expressions were introduced quite elegantly. The idea of being able to write every anonymous SAM instance as a lambda expression is very compelling from a backwards-compatiblity point of view. So what are the dark sides to Java 8?

Overloading gets even worse

Overloading, generics, and varargs aren’t friends. We’ve explained this in a previous article, and also in this Stack Overflow question. These might not be every day problems in your odd application, but they’re very important problems for API designers and maintainers.

With lambda expressions, things get “worse”. So you think you can provide some convenience API, overloading your existing run() method that accepts a Callable to also accept the new Supplier type:

static <T> T run(Callable<T> c) throws Exception {
    return c.call();
}

static <T> T run(Supplier<T> s) throws Exception {
    return s.get();
}

What looks like perfectly useful Java 7 code is a major pain in Java 8, now. Because you cannot just simply call these methods with a lambda argument:

public static void main(String[] args)
throws Exception {
    run(() -> null);
    //  ^^^^^^^^^^ ambiguous method call
}

Tough luck. You’ll have to resort to either of these “classic” solutions:

    run((Callable<Object>) (() -> null));
    run(new Callable<Object>() {
        @Override
        public Object call() throws Exception {
            return null;
        }
    });

So, while there’s always a workaround, these workarounds always “suck”. That’s quite a bummer, even if things don’t break from a backwards-compatibility perspective.

Not all keywords are supported on default methods

Default methods are a nice addition. Some may claim that Java finally has traits. Others clearly dissociate themselves from the term, e.g. Brian Goetz:

The key goal of adding default methods to Java was “interface evolution”, not “poor man’s traits.”

As found on the lambda-dev mailing list.

Fact is, default methods are quite a bit of an orthogonal and irregular feature to anything else in Java. Here are a couple of critiques:

They cannot be made final

Given that default methods can also be used as convenience methods in API:

public interface NoTrait {

    // Run the Runnable exactly once
    default final void run(Runnable r) {
        //  ^^^^^ modifier final not allowed
        run(r, 1);
    }

    // Run the Runnable "times" times
    default void run(Runnable r, int times) {
        for (int i = 0; i < times; i++)
            r.run();
    }
}

Unfortunately, the above is not possible, and so the first overloaded convenience method could be overridden in subtypes, even if that makes no sense to the API designer.

They cannot be made synchronized

Bummer! Would that have been difficult to implement in the language?

public interface NoTrait {
    default synchronized void noSynchronized() {
        //  ^^^^^^^^^^^^ modifier synchronized
        //  not allowed
        System.out.println("noSynchronized");
    }
}

Yes, synchronized is used rarely, just like final. But when you have that use-case, why not just allow it? What makes interface method bodies so special?

The default keyword

This is maybe the weirdest and most irregular of all features. The default keyword itself. Let’s compare interfaces and abstract classes:


// Interfaces are always abstract
public /* abstract */ interface NoTrait {

    // Abstract methods have no bodies
    // The abstract keyword is optional
    /* abstract */ void run1();

    // Concrete methods have bodies
    // The default keyword is mandatory
    default void run2() {}
}

// Classes can optionally be abstract
public abstract class NoInterface {

    // Abstract methods have no bodies
    // The abstract keyword is mandatory
    abstract void run1();

    // Concrete methods have bodies
    // The default keyword mustn't be used
    void run2() {}
}

If the language were re-designed from scratch, it would probably do without any of abstract or default keywords. Both are unnecessary. The mere fact that there is or is not a body is sufficient information for the compiler to assess whether a method is abstract. I.e, how things should be:

public interface NoTrait {
    void run1();
    void run2() {}
}

public abstract class NoInterface {
    void run1();
    void run2() {}
}

The above would be much leaner and more regular. It’s a pity that the usefulness of default was never really debated by the EG. Well, it was debated but the EG never wanted to accept this as an option. I’ve tried my luck, with this response:

I don’t think #3 is an option because interfaces with method bodies are unnatural to begin with. At least specifying the “default” keyword gives the reader some context why the language allows a method body. Personally, I wish interfaces would remain as pure contracts (without implementation), but I don’t know of a better option to evolve interfaces.

Again, this is a clear commitment by the EG not to commit to the vision of “traits” in Java. Default methods were a pure necessary means to implement 1-2 other features. They weren’t well-designed from the beginning.

Other modifiers

Luckily, the static modifier made it into the specs, late in the project. It is thus possible to specifiy static methods in interfaces now. For some reason, though, these methods do not need (nor allow!) the default keyword, which must’ve been a totally random decision by the EG, just like you apparently cannot define static final methods in interfaces.

While visibility modifiers were discussed on the lambda-dev mailing list, but were out of scope for this release. Maybe, we can get them in a future release.

Few default methods were actually implemented

Some methods would have sensible default implementations on interface – one might guess. Intuitively, the collections interfaces, like List or Set would have them on their equals() and hashCode() methods, because the contract for these methods is well-defined on the interfaces. It is also implemented in AbstractList, using listIterator(), which is a reasonable default implementation for most tailor-made lists.

It would’ve been great if these API were retrofitted to make implementing custom collections easier with Java 8. I could make all my business objects implement List for instance, without wasting the single base-class inheritance on AbstractList.

Probably, though, there has been a compelling reason related to backwards-compatibility that prevented the Java 8 team at Oracle from implementing these default methods. Whoever sends us the reason why this was omitted will get a free jOOQ sticker 🙂

The wasn’t invented here – mentality

This, too, was criticised a couple of times on the lambda-dev EG mailing list. And while writing this blog series, I can only confirm that the new functional interfaces are very confusing to remember. They’re confusing for these reasons:

Some primitive types are more equal than others

The int, long, double primitive types are preferred compared to all the others, in that they have a functional interface in the java.util.function package, and in the whole Streams API. boolean is a second-class citizen, as it still made it into the package in the form of a BooleanSupplier or a Predicate, or worse: IntPredicate.

All the other primitive types don’t really exist in this area. I.e. there are no special types for byte, short, float, and char. While the argument of meeting deadlines is certainly a valid one, this quirky status-quo will make the language even harder to learn for newbies.

The types aren’t just called Function

Let’s be frank. All of these types are simply “functions”. No one really cares about the implicit difference between a Consumer, a Predicate, a UnaryOperator, etc.

In fact, when you’re looking for a type with a non-void return value and two arguments, what would you probably be calling it? Function2? Well, you were wrong. It is called a BiFunction.

Here’s a decision tree to know how the type you’re looking for is called:

  • Does your function return void? It’s called a Consumer
  • Does your function return boolean? It’s called a Predicate
  • Does your function return an int, long, double? It’s called XXToIntYY, XXToLongYY, XXToDoubleYY something
  • Does your function take no arguments? It’s called a Supplier
  • Does your function take a single int, long, double argument? It’s called an IntXX, LongXX, DoubleXX something
  • Does your function take two arguments? It’s called BiXX
  • Does your function take two arguments of the same type? It’s called BinaryOperator
  • Does your function return the same type as it takes as a single argument? It’s called UnaryOperator
  • Does your function take two arguments of which the first is a reference type and the second is a primitive type? It’s called ObjXXConsumer (only consumers exist with that configuration)
  • Else: It’s called Function

Good lord! We should certainly go over to Oracle Education to check if the price for Oracle Certified Java Programmer courses have drastically increased, recently… Thankfully, with Lambda expressions, we hardly ever have to remember all these types!

More on Java 8

Java 5 generics have brought a lot of great new features to the Java language. But there were also quite a few caveats related to type erasure. Java 8’s default methods, Streams API and lambda expressions will again bring a lot of great new features to the Java language and platform. But we’re sure that Stack Overflow will soon burst with questions by confused programmers that are getting lost in the Java 8 jungle.

Learning all the new features won’t be easy, but the new features (and caveats) are here to stay. If you’re a Java developer, you better start practicing now, when you get the chance. Because we have a long way to go.

Nonetheless, thigns are exciting, so stay tuned for more exciting Java 8 stuff published in this blog series.

Are you in for another critique about Java 8? Read “New Parallelism APIs in Java 8: Behind the Glitz and Glamour” by the guys over

Java 8 Friday: Optional Will Remain an Option in Java

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Optional: A new Option in Java

So far, we’ve been pretty thrilled with all the additions to Java 8. All in all, this is a revolution more than anything before. But there are also one or two sore spots. One of them is how Java will never really get rid of

Null: The billion dollar mistake tweet this

In a previous blog post, we have explained the merits of NULL handling in the Ceylon language, which has found one of the best solutions to tackle this issue – at least on the JVM which is doomed to support the null pointer forever. In Ceylon, nullability is a flag that can be added to every type by appending a question mark to the type name. An example:

void hello() {
    String? name = process.arguments.first;
    String greeting;
    if (exists name) {
        greeting = "Hello, ``name``!";
    }
    else {
        greeting = "Hello, World!";
    }
    print(greeting);
}

That’s pretty slick. Combined with flow-sensitive typing, you will never run into the dreaded NullPointerException again:

Recently in the Operating Room. By Geek and Poke

Recently in the Operating Room. By Geek and Poke

Other languages have introduced the Option type. Most prominently: Scala. Java 8 now also introduced the Optional type (as well as the OptionalInt, OptionalLong, OptionalDouble types – more about those later on)

How does Optional work?

The main point behind Optional is to wrap an Object and to provide convenience API to handle nullability in a fluent manner. This goes well with Java 8 lambda expressions, which allow for lazy execution of operations. An example:

Optional<String> stringOrNot = Optional.of("123");

// This String reference will never be null
String alwaysAString =
    stringOrNot.orElse("");

// This Integer reference will be wrapped again
Optional<Integer> integerOrNot = 
    stringOrNot.map(Integer::parseInt);

// This int reference will never be null
int alwaysAnInt = stringOrNot
        .map(s -> Integer.parseInt(s))
        .orElse(0);

There are certain merits to the above in fluent APIs, specifically in the new Java 8 Streams API, which makes extensive use of Optional. For example:

Arrays.asList(1, 2, 3)
      .stream()
      .findAny()
      .ifPresent(System.out::println);

The above piece of code will print any number from the Stream onto the console, but only if such a number exists.

Old API is not retrofitted

For obvious backwards-compatibility reasons, the “old API” is not retrofitted. In other words, unlike Scala, Java 8 doesn’t use Optional all over the JDK. In fact, the only place where Optional is used is in the Streams API. As you can see in the Javadoc, usage is very scarce:

http://docs.oracle.com/javase/8/docs/api/java/util/class-use/Optional.html

This makes Optional a bit difficult to use. We’ve already blogged about this topic before. Concretely, the absence of an Optional type in the API is no guarantee of non-nullability. This is particularly nasty if you convert Streams into collections and collections into streams.

The Java 8 Optional type is treacherous tweet this

Parametric polymorphism

The worst implication of Optional on its “infected” API is parametric polymorphism, or simply: generics. When you reason about types, you will quickly understand that:

// This is a reference to a simple type:
Number s;

// This is a reference to a collection of
// the above simple type:
Collection<Number> c;

Generics are often used for what is generally accepted as composition. We have a Collection of String. With Optional, this compositional semantics is slightly abused (both in Scala and Java) to “wrap” a potentially nullable value. We now have:

// This is a reference to a nullable simple type:
Optional<Number> s;

// This is a reference to a collection of 
// possibly nullable simple types
Collection<Optional<Number>> c;

So far so good. We can substitute types to get the following:

// This is a reference to a simple type:
T s;

// This is a reference to a collection of
// the above simple type:
Collection<T> c;

But now enter wildcards and use-site variance. We can write

// No variance can be applied to simple types:
T s;

// Variance can be applied to collections of
// simple types:
Collection<? extends T> source;
Collection<? super T> target;

What do the above types mean in the context of Optional? Intuitively, we would like this to be about things like Optional<? extends Number> or Optional<? super Number>. In the above example we can write:

// Read a T-value from the source
T s = source.iterator().next();

// ... and put it into the target
target.add(s);

But this doesn’t work any longer with Optional

Collection<Optional<? extends T>> source;
Collection<Optional<? super T>> target;

// Read a value from the source
Optional<? extends T> s = source.iterator().next();

// ... cannot put it into the target
target.add(s); // Nope

… and there is no other way to reason about use-site variance when we have Optional and subtly more complex API.

If you add generic type erasure to the discussion, things get even worse. We no longer erase the component type of the above Collection, we also erase the type of virtually any reference. From a runtime / reflection perspective, this is almost like using Object all over the place!

Generic type systems are incredibly complex even for simple use-cases. Optional makes things only worse. It is quite hard to blend Optional with traditional collections API or other APIs. Compared to the ease of use of Ceylon’s flow-sensitive typing, or even Groovy’s elvis operator, Optional is like a sledge-hammer in your face.

Be careful when you apply it to your API!

Primitive types

One of the main reasons why Optional is still a very useful addition is the fact that the “object-stream” and the “primitive streams” have a “unified API” by the fact that we also have OptionalInt, OptionalLong, OptionalDouble types.

In other words, if you’re operating on primitive types, you can just switch the stream construction and reuse the rest of your stream API usage source code, in almost the same way. Compare these two chains:

// Stream and Optional
Optional<Integer> anyInteger = 
Arrays.asList(1, 2, 3)
      .stream()
      .filter(i -> i % 2 == 0)
      .findAny();
anyInteger.ifPresent(System.out::println);

// IntStream and OptionalInt
OptionalInt anyInt =
Arrays.stream(new int[] {1, 2, 3})
      .filter(i -> i % 2 == 0)
      .findAny();
anyInt.ifPresent(System.out::println);

In other words, given the scarce usage of these new types in JDK API, the dubious usefulness of such a type in general (if retrofitted into a very backwards-compatible environment) and the implications generics erasure have on Optional we dare say that

The only reason why this type was really added is to provide a more unified Streams API for both reference and primitive types tweet this

That’s tough. And makes us wonder, if we should finally get rid of primitive types altogether.

Oh, and…

Optional isn’t Serializable.

Nope. Not Serializable. Unlike ArrayList, for instance. For the usual reason:

Making something in the JDK serializable makes a dramatic increase in our maintenance costs, because it means that the representation is frozen for all time. This constrains our ability to evolve implementations in the future, and the number of cases where we are unable to easily fix a bug or provide an enhancement, which would otherwise be simple, is enormous. So, while it may look like a simple matter of “implements Serializable” to you, it is more than that. The amount of effort consumed by working around an earlier choice to make something serializable is staggering.

Citing Brian Goetz, from:

http://mail.openjdk.java.net/pipermail/jdk8-dev/2013-September/003276.html

Want to discuss Optional? Read these threads on reddit:

Stay tuned for more exciting Java 8 stuff published in this blog series.

More on Java 8

In the mean time, have a look at Eugen Paraschiv’s awesome Java 8 resources page

Java Collections API Quirks

So we tend to think we’ve seen it all, when it comes to the Java Collections API. We know our ways around Lists, Sets, Maps, Iterables, Iterators. We’re ready for Java 8’s Collections API enhancements.

But then, every once in a while, we stumble upon one of these weird quirks that originate from the depths of the JDK and its long history of backwards-compatibility. Let’s have a look at unmodifiable collections

Unmodifiable Collections

Whether a collection is modifiable or not is not reflected by the Collections API. There is no immutable List, Set or Collection base type, which mutable subtypes could extend. So, the following API doesn’t exist in the JDK:

// Immutable part of the Collection API
public interface Collection {
  boolean  contains(Object o);
  boolean  containsAll(Collection<?> c);
  boolean  isEmpty();
  int      size();
  Object[] toArray();
  <T> T[]  toArray(T[] array);
}

// Mutable part of the Collection API
public interface MutableCollection 
extends Collection {
  boolean  add(E e);
  boolean  addAll(Collection<? extends E> c);
  void     clear();
  boolean  remove(Object o);
  boolean  removeAll(Collection<?> c);
  boolean  retainAll(Collection<?> c);
}

Now, there are probably reasons, why things hadn’t been implemented this way in the early days of Java. Most likely, mutability wasn’t seen as a feature worthy of occupying its own type in the type hierarchy. So, along came the Collections helper class, with useful methods such as unmodifiableList(), unmodifiableSet(), unmodifiableCollection(), and others. But beware when using unmodifiable collections! There is a very strange thing mentioned in the Javadoc:

The returned collection does not pass the hashCode and equals operations through to the backing collection, but relies on Object’s equals and hashCode methods. This is necessary to preserve the contracts of these operations in the case that the backing collection is a set or a list.

“To preserve the contracts of these operations”. That’s quite vague. What’s the reasoning behind it? A nice explanation is given in this Stack Overflow answer here:

An UnmodifiableList is an UnmodifiableCollection, but the same is not true in reverse — an UnmodifiableCollection that wraps a List is not an UnmodifiableList. So if you compare an UnmodifiableCollection that wraps a List a with an UnmodifiableList that wraps the same List a, the two wrappers should not be equal. If you just passed through to the wrapped list, they would be equal.

While this reasoning is correct, the implications may be rather unexpected.

The bottom line

The bottom line is that you cannot rely on Collection.equals(). While List.equals() and Set.equals() are well-defined, don’t trust Collection.equals(). It may not behave meaningfully. Keep this in mind, when accepting a Collection in a method signature:

public class MyClass {
  public void doStuff(Collection<?> collection) {
    // Don't rely on collection.equals() here!
  }
}