Java 8 Friday: Optional Will Remain an Option in Java

At Data Geekery, we love Java. And as we’re really into jOOQ’s fluent API and query DSL, we’re absolutely thrilled about what Java 8 will bring to our ecosystem.

Java 8 Friday

Every Friday, we’re showing you a couple of nice new tutorial-style Java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. You’ll find the source code on GitHub.

Optional: A new Option in Java

So far, we’ve been pretty thrilled with all the additions to Java 8. All in all, this is a revolution more than anything before. But there are also one or two sore spots. One of them is how Java will never really get rid of

Null: The billion dollar mistake

In a previous blog post, we have explained the merits of NULL handling in the Ceylon language, which has found one of the best solutions to tackle this issue – at least on the JVM which is doomed to support the null pointer forever. In Ceylon, nullability is a flag that can be added to every type by appending a question mark to the type name. An example:


void hello() {
    String? name = process.arguments.first;
    String greeting;
    if (exists name) {
        greeting = "Hello, ``name``!";
    }
    else {
        greeting = "Hello, World!";
    }
    print(greeting);
}

That’s pretty slick. Combined with flow-sensitive typing, you will never run into the dreaded NullPointerException again:

Recently in the Operating Room. By Geek and Poke

Other languages have introduced the Option type. Most prominently: Scala. Java 8 now also introduced the Optional type (as well as the OptionalInt, OptionalLong, OptionalDouble types – more about those later on)

How does Optional work?

The main point behind Optional is to wrap an Object and to provide convenience API to handle nullability in a fluent manner. This goes well with Java 8 lambda expressions, which allow for lazy execution of operations. An example:


Optional<String> stringOrNot = Optional.of("123");

// This String reference will never be null
String alwaysAString =
    stringOrNot.orElse("");

// This Integer reference will be wrapped again
Optional<Integer> integerOrNot = 
    stringOrNot.map(Integer::parseInt);

// This int reference will never be null
int alwaysAnInt = stringOrNot
        .map(s -> Integer.parseInt(s))
        .orElse(0);

There are certain merits to the above in fluent APIs, specifically in the new Java 8 Streams API, which makes extensive use of Optional. For example:


Arrays.asList(1, 2, 3)
      .stream()
      .findAny()
      .ifPresent(System.out::println);

The above piece of code will print any number from the Stream onto the console, but only if such a number exists.

Old API is not retrofitted

For obvious backwards-compatibility reasons, the “old API” is not retrofitted. In other words, unlike Scala, Java 8 doesn’t use Optional all over the JDK. In fact, the only place where Optional is used is in the Streams API. As you can see in the Javadoc, usage is very scarce: http://docs.oracle.com/javase/8/docs/api/java/util/class-use/Optional.html This makes Optional a bit difficult to use. We’ve already blogged about this topic before. Concretely, the absence of an Optional type in the API is no guarantee of non-nullability. This is particularly nasty if you convert Streams into collections and collections into streams.

The Java 8 Optional type is treacherous

Parametric polymorphism

The worst implication of Optional on its “infected” API is parametric polymorphism, or simply: generics. When you reason about types, you will quickly understand that:


// This is a reference to a simple type:
Number s;

// This is a reference to a collection of
// the above simple type:
Collection<Number> c;

Generics are often used for what is generally accepted as composition. We have a Collection of String. With Optional, this compositional semantics is slightly abused (both in Scala and Java) to “wrap” a potentially nullable value. We now have:


// This is a reference to a nullable simple type:
Optional<Number> s;

// This is a reference to a collection of 
// possibly nullable simple types
Collection<Optional<Number>> c;

So far so good. We can substitute types to get the following:


// This is a reference to a simple type:
T s;

// This is a reference to a collection of
// the above simple type:
Collection<T> c;

But now enter wildcards and use-site variance. We can write


// No variance can be applied to simple types:
T s;

// Variance can be applied to collections of
// simple types:
Collection<? extends T> source;
Collection<? super T> target;

What do the above types mean in the context of Optional? Intuitively, we would like this to be about things like Optional<? extends Number> or Optional<? super Number>. In the above example we can write:


// Read a T-value from the source
T s = source.iterator().next();

// ... and put it into the target
target.add(s);

But this doesn’t work any longer with Optional


Collection<Optional<? extends T>> source;
Collection<Optional<? super T>> target;

// Read a value from the source
Optional<? extends T> s = source.iterator().next();

// ... cannot put it into the target
target.add(s); // Nope

… and there is no other way to reason about use-site variance when we have Optional and subtly more complex API. If you add generic type erasure to the discussion, things get even worse. We no longer erase the component type of the above Collection, we also erase the type of virtually any reference. From a runtime / reflection perspective, this is almost like using Object all over the place! Generic type systems are incredibly complex even for simple use-cases. Optional makes things only worse. It is quite hard to blend Optional with traditional collections API or other APIs. Compared to the ease of use of Ceylon’s flow-sensitive typing, or even Groovy’s elvis operator, Optional is like a sledge-hammer in your face. Be careful when you apply it to your API!

Primitive types

One of the main reasons why Optional is still a very useful addition is the fact that the “object-stream” and the “primitive streams” have a “unified API” by the fact that we also have OptionalInt, OptionalLong, OptionalDouble types. In other words, if you’re operating on primitive types, you can just switch the stream construction and reuse the rest of your stream API usage source code, in almost the same way. Compare these two chains:


// Stream and Optional
Optional<Integer> anyInteger = 
Arrays.asList(1, 2, 3)
      .stream()
      .filter(i -> i % 2 == 0)
      .findAny();
anyInteger.ifPresent(System.out::println);

// IntStream and OptionalInt
OptionalInt anyInt =
Arrays.stream(new int[] {1, 2, 3})
      .filter(i -> i % 2 == 0)
      .findAny();
anyInt.ifPresent(System.out::println);

In other words, given the scarce usage of these new types in JDK API, the dubious usefulness of such a type in general (if retrofitted into a very backwards-compatible environment) and the implications generics erasure have on Optional we dare say that

The only reason why this type was really added is to provide a more unified Streams API for both reference and primitive types

That’s tough. And makes us wonder, if we should finally get rid of primitive types altogether.

Oh, and…

… Optional isn’t Serializable. Nope. Not Serializable. Unlike ArrayList, for instance. For the usual reason:

Making something in the JDK serializable makes a dramatic increase in our maintenance costs, because it means that the representation is frozen for all time. This constrains our ability to evolve implementations in the future, and the number of cases where we are unable to easily fix a bug or provide an enhancement, which would otherwise be simple, is enormous. So, while it may look like a simple matter of “implements Serializable” to you, it is more than that. The amount of effort consumed by working around an earlier choice to make something serializable is staggering.

Citing Brian Goetz, from: http://mail.openjdk.java.net/pipermail/jdk8-dev/2013-September/003276.html Want to discuss Optional? Read these threads on reddit:

Stay tuned for more exciting Java 8 stuff published in this blog series.

More on Java 8

In the mean time, have a look at Eugen Paraschiv’s awesome Java 8 resources page

Optional is a final class, so using use-site variance on Optional itself doesn’t really make any sense (i.e. there are no sensible super types or subtypes). Anyway, the source / target semantics isn’t really about Optional anyway, it is about the T type, which is wrapped by Optional. In other words, I’d like to model a “source of T“ and a “target of T“. This really cannot be modelled easily without additional helper classes when Optional is involved.

use-site variance is just hard

It is indeed, and Ross Tate (from Kotlin) has written a nice paper about that subject. Things do have a lot to do with Optional, though, because when people will start adopting Optional, we’ll find lots and lots of questions on Stack Overflow, etc. and people ranting about generics, without realising that Optional and its use of generics is simply wrong. In Scala, this is shadowed by the fact that Scala has:

A lot of additional language tools (like match) to untangle Option and its wrapped type
Declaration-site variance, which is also a very expressive way to circumvent some issues at hand
Some and None types (unlike Java)
No direct usage of primitive types and null (thanks to None)

Nonetheless, I still believe that even in Scala, Option is conceptually wrong and broken and will forever break the way we perceive generics.

4 thoughts on “Java 8 Friday: Optional Will Remain an Option in Java”

arnaud says:

March 28, 2014 at 19:16

@Nullable String getName()
or
Optional getName()

? :)

Loading...

1. lukaseder says:
  
  March 28, 2014 at 19:39
  
  A new religious war!
  
  Loading...
  
blaisorblade says:

March 29, 2014 at 18:55

> Collection<Optional> source;
> Collection<Optional> target;

> [… stuff breaks…]

Don’t you “just” need something like

Collection<? extends Optional> source;
Collection<? super Optional> target;

// Read a value from the source
Optional s = source.iterator().next();

target.add(s); // should work

? Granted, figuring this out is not trivial, but the problem has nothing to do with Optional — use-site variance is just hard. (In fact, everything you can do with declaration-site variance, like in Scala or IIRC C#, is also expressible with use-site variance with the right type annotations).

Loading...

1. lukaseder says:
  
  March 31, 2014 at 10:32
  Optional is a final class, so using use-site variance on Optional itself doesn’t really make any sense (i.e. there are no sensible super types or subtypes). Anyway, the source / target semantics isn’t really about Optional anyway, it is about the T type, which is wrapped by Optional. In other words, I’d like to model a “source of T“ and a “target of T“. This really cannot be modelled easily without additional helper classes when Optional is involved.
  
  use-site variance is just hard
  
  It is indeed, and Ross Tate (from Kotlin) has written a nice paper about that subject. Things do have a lot to do with Optional, though, because when people will start adopting Optional, we’ll find lots and lots of questions on Stack Overflow, etc. and people ranting about generics, without realising that Optional and its use of generics is simply wrong. In Scala, this is shadowed by the fact that Scala has:
  - A lot of additional language tools (like match) to untangle Option and its wrapped type
  - Declaration-site variance, which is also a very expressive way to circumvent some issues at hand
  - Some and None types (unlike Java)
  - No direct usage of primitive types and null (thanks to None)
  Nonetheless, I still believe that even in Scala, Option is conceptually wrong and broken and will forever break the way we perceive generics.
  
  Loading...