Ceylon Might Just be the Only (JVM) Language that Got Nulls Right

Posted on by lukaseder
Here we go again. THAT TOPIC. But hang on. The approach discussed here (and in the Ceylon language) is not something you see every day. At the same time, it is very cunning.

Nulls are baked into the language

… or so it may seem. Indeed, in Ceylon, like in Kotlin (and possibly many other languages), there is a special type “annotation” that you can postfix to any reference type in order to make it nullable. For instance: 

String firstName = "Homer";
String? middleName = "J";
String lastName = "Simpson";
In the above example, both firstName and lastName are mandatory values that can never be null, whereas middleName is optional. Most languages that support the above then ship with special operators to access the optional value, e.g. ?. in Ceylon and also in Kotlin. 

// Another optional value:
Integer? length = middleName?.length;

// A non-optional value:
Integer length = middleName?.length else 0;

So, what is it about Ceylon that works so smoothly?

The thing that Ceylon got very right is the fact that all of the above is just syntactic sugar that is:
  • Easy to use
  • Maps well to our mindset, where null still is a thing
  • Can interoperate with Java
  • Doesn’t introduce cognitive friction
For us Java folks, we can still pretend that null is an OK-ish, hard to avoid thing (as we’ve claimed before on this blog). But what is null really? Is it the absent value? The unknown value? The uninitialised value? Java only has one null thingy, and it is (ab-)used for all of the previous things, and more, when in theory, it is only really the uninitialised value, nothing more. On the other hand, when working with JDBC (and thus, SQL), it implicitly means the unknown value (with all the related caveats). In Ceylon, however, Null is a special type, similar to Void in Java. The only value that can be assigned to the Null type is null: 

// Ceylon
Null x = null;

// Java
Void x = null;
But the big difference is, null cannot be assigned to any other type! Wait. Couldn’t we assign null to String? … ? Of course, the following is possible in Ceylon: 

String? x = null;
But why is this possible? Because String? is just syntax sugar for String|Null, a union type, i.e. a type that is either the String type or the Null type.

Huh, what are union types?

Let’s look at this more closely. When in the jOOQ API you want to work with SQL functions and expressions, there is always a great set of overloads that provide you with a standard version, and a convenience version where you can pass a bind variable. Take the equals operator, for instance: 

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
}
The above overloads allow you for writing things like the following, without needing to think about the distinction between a SQL expression and a Java bind variable (which is ultimately also a SQL expression): 

// Comparing a column with bind variable
.where(BOOK.ID.eq(1))

// Comparing a column with another column expression
.and(BOOK.AUTHOR_ID.eq(AUTHOR.ID))
In fact, there are even more overloads, because the right hand side of a comparison operation can have other expressions as well, for instance: 

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
    Condition eq(Select<? extends Record1<T>> query);
    Condition eq(QuantifiedSelect<? extends Record1<T>> query);
}
Now, the same set of overloads needs to be repeated for not equals, greater than, greater or equal, etc. Wouldn’t it be nice to be able to express this “right-hand-side” thingy as a single, reusable type? I.e. a union type of all of the above types? 

interface Field<T> {
    Condition eq(
        Field<T>
      | T
      | Select<? extends Record1<T>>
      | QuantifiedSelect<? extends Record1<T>> thingy
    );
}
Or even 

// This is called a type alias. Another awesome
// Ceylon language feature (pseudo syntax)
alias Thingy => 
    Field<T>
  | T
  | Select<? extends Record1<T>>
  | QuantifiedSelect<? extends Record1<T>>;

interface Field<T> {
    Condition eq(Thingy thingy);
}
After all, that’s also how the SQL language is defined. Heck, that’s how any BNF notation defines syntactic elements. For instance: 
<predicate> ::=
    <comparison predicate>
  | <between predicate>
  | <in predicate>
  | <like predicate>
  | <null predicate>
  | <quantified comparison predicate>
  | <exists predicate>
  | <unique predicate>
  | <match predicate>
  | <overlaps predicate>
OK, granted, a syntactic element is not strictly the same thing as a type, but the intuitive perception is the same.

Oh, and Java has union types, too!

In a brief flash of revelation, the Java 7 expert groups added support for union types in exception handling. You can write things like: 

try {
    ...
}
catch (IOException | SQLException e) {
    // e can be any of the above!
}
And you can emulate union types with generics, which don’t support union types but intersection types in Java.

Back to Ceylon and NULL

Ceylon has gotten Null right. Because, historically, a nullable type is a type that can be the “real” type or the “null” value. We want that. We Java developers crave that. We cannot live without the soothing option of this kind of optional. But the excellent thing about this approach is that it is extendable. What if I really need to distinguish between “unknown”, “uninitialised”, “undefined”, “42”? I can. Using types. Here’s a String that can model all of the aforementioned “special values”: 

String|Unknown|Uninitialised|Undefined|FortyTwo
And if that’s too verbose, I just assign a name to it 

interface TheStringToRuleThemAll
  => String|Unknown|Uninitialised|Undefined|FortyTwo;
But it cannot be Null. Because I don’t want it to be that value, that is everything and nothing. Are you convinced? I bet you are. From now on:
Don’t trust any language that pretends that the Option(al) monad is a decent approach at modelling null. It isn’t. ― me. Just now
Why? Let me illustrate. Kotlin/Ceylon/Groovy style syntax sugar using the elvis operator (regardless of the backing null semantics): 

String name = bob?.department?.head?.name
Same thing with Optional monads: 

Optional<String> name = bob
    .flatMap(Person::getDepartment)
    .map(Department::getHead)
    .flatMap(Person::getName);
AND POOR YOU IF YOU MIX UP map() WITH flatMap() JUST ONCE!!
Some people claim
Using union types is like driving around in a brand new Ferrari with your mother-in-law in the passenger seat. ― by Elvira
Sure. But I claim: Well done, Ceylon. Let’s hope we’ll get union types in Java, too, outside of catch blocks!

Further reading

Liked this article? How about:

Published by lukaseder

I made jOOQ

27 thoughts on “Ceylon Might Just be the Only (JVM) Language that Got Nulls Right”

  1. Interesting article. How does this compare with the Haskell Maybe type? You mention “Optional monads”, which implies you’ve heard of it. But that last example in Haskell would be

    let bossName = name bob >>= department >>= head

    Which is a bit hairier than your version, but OTOH it is using a general-purpose syntax rather than special-case syntactic sugar. In particular, the above code will work if you are using an Error monad instead of Maybe, and if you defined your TheStringToRuleThemAll type as a monad then it would work with that too.

    In practice I find that stacking Maybes like this is actually quite rare. Sometimes I get to write one >>= to link two Maybe functions, but it almost never goes to a third.

    Reply

    1. Well, first class sum types are also a general-purpose syntax, which can be used in entirely different contexts as well.

      A monad puts a value in a computational context. I personally don’t see nullability as a sufficiently important computational context. But I can’t argue against using a monad outside of subjective terms. I just don’t like to wrap and unwrap a value all the time.

      Reply

      1. But you need to do that either way! Namely, if you really need a String but only have a String?

        Reply

        1. Yeah, but a String? and a String?? are the same thing. Unlike Optional<String>, which isn’t the same as Optional<Optional<String>>

          Reply

  2. First of all, great article and great conclusion!

    Second of all, slight correction: Sum types and union types are not the same thing. A union type can encompass any other unrelated type. A sum type is a series of related types that otherwise would not exist. For instance, you can’t have a sum type with String and Integer. An algebraic data type is a sum type, which Ceylon supports as well.

    Reply

    1. Hmm, interesting. Some months ago, I was educated that I was using the term “union types” too much, when I should really use “sum types”. Now it’s the inverse. What’s in a name, eh? :)

      So, essentially, a sum type is a (tagged) union type, but a union type is not necessarily a sum type? (As in https://en.wikipedia.org/wiki/Tagged_union)

      Reply

  3. Your type alias in the current form doesn’t seem like valid Ceylon syntax. Also, it likely needs to have `T` as a type argument (or be defined inside some context where `T` is already given). Here how it could look like:

    // This is called a type alias. Another awesome
// Ceylon language feature
alias Thingy => 
    Field
  | T
  | Select<out Record1>
  | QuantifiedSelect<out Record1>;

    (This assumes `Record1` is invariant – you could declare that as covariant and omit the `out` here.)

    Reply

        1. No, this was about the fact that things between smaller-than and greater-than signs (i.e. everything which looks like a HTML tag, but thus also ceylon type arguments in pointy brackets) are filtered out by the commenting engine. In my example, there are some <T> missing (I hope it works this way?).

          Reply

    2. Thanks for the feedback. Indeed, your suggestion would be more accurate for Ceylon syntax. The article meant to sketch a possible Java syntax… Currently, Java doesn’t support declaration site variance, which is why I left the use-site variance in place.

      Reply

  4. In scala you would write 

    val deptHeadName = for {
  department <- bob.getDepartment
  head <- department.getHead
  name <- head.getName
} yield name

    This is a bit more verbose than both elvis operator and haskell, but it doesn't use any special language features for null.

    How would you write the same function as you suggested in Ceylon if getDepartment, getHead, and getName all returned String|Unknown|Uninitialized|Undefined ?

    Reply

    1. Indeed, for comprehensions are nice for this kind of use case, but still you have to do one unwrapping at a time, rather than in a one liner.

      How would you write the same function as you suggested in Ceylon if getDepartment, getHead, and getName all returned String|Unknown|Uninitialized|Undefined ?

      I think that once you have so many special states, verbosity is a good thing, because it forces you to really think about these states every time you want to unwrap the string. That seems to be a bit different from ordinary null handling. We’ll quickly get into a similar discussion like whether checked exceptions are good.

      Asked differently, how would you deal with such a type in Scala? :-)

      Reply

    2. Well, in that contrived case, one could write:

      value deptHeadName = if (is Department d = bob.department, is Person h = d.head, is String name = h.name) then name else null;

      Or something like that. Actually there are no Unknown, Uninitialized, or Undefined classes to worry about in Ceylon, so in practice it looks like this:

      value deptHeadName = if (exists d = bob.department, exists h = d.head, exists name = h.name) then name else null;

      Since “exists” is a more convenient syntax sugar for “is Object” or “!is Null”.

      And, sure, most people prefer to write it like this:

      value deptHeadName = bob?.department?.head?.name;

      But that’s just further syntax sugar.

      Reply

  5. Nice article as always!

    I would love to start working with Ceylon, a very nice lang! But the Intellij plugin is still in development… (and there’s no way to get myself back to Eclipse!)

    If I remember well C# 6 has elvis-operator too!

    Optional monad I think it’s a nice idea in order to get rid of some null-checks, it’s a little bit less natural than elvis-operator (and much more verbose for sure), but both have the problem that if you got a wanna-be-developer in your team you have to check his null-check conditions anyway :\ («oh com’on! it’s clearer to read if (bob != null) var department = bob.departement instead of var department = bob?departement!!»)

    Reply

  6. Hi

    To me the notion of Option[al] is more that a simple way to map/flatmap. I use extensively filter, ifPresent, orElse, orElseGet, not only map/flatMap.

    Furthermore, most of my calls aren’t just “walk through the fields path”, but more something with logic it it or using functions.

    Actually, to me, something like bob?.department?.head?.name is almost a smell, per the law of demeter (https://en.wikipedia.org/wiki/Law_of_Demeter), so I would actually think twice before writing it.

    So ? with only the ability to transparently do flapMap feels a bit weak to me. Actually I even suspect it would end up being annoying, because it provides very few. bob?.department?.head?.name doesn’t return the name? Where is it null? Is it bob, the departement, the head? Who knows ? And how easy is to know ?

    So, while the union type looks the way to go, having just ? meaning flatmap doesn’t feel like doing the trick IMHO

    but then I’m most likely wrong, so feel free to enlighten me :)

    Reply

    1. I see your point. Those methods are expressive, indeed.

      There’s always the question of API vs. language feature. It is much easier to maintain a reasonably expressive API than a reasonably expressive language. Of course, there could be language operators for all of “ifPresent”, “orElse”, “orElseGet”, but that seems a bit excessive.

      Oh wait. Here you go :)

      ifPresent

      if (x != null) ...

      orElse

      if (x == null) ...

      orElseGet

      if (x == null) ...

      I guess it only boils down to a matter of stylistic preference, and the fact that you’re willing to trade in one inlinable stack frame for some “convenience”, in case you’re not relying on JIT method inlining…

      Reply

      1. Well, your main point for kotlin ‘?’ was, as far as I got it, about the reduction of verbosity when chaining calls.

        But this is only one use case around null handling. And IMHO this one isn’t even properly handled: ‘?’ doesn’t address the times where you need to know which reference was null. So even on chaining calls ‘?’ isn’t a silver bullet.

        Anyway, chaining map/flatmap/filter/orElse and the like is another use case . And in this one, the verbosity of ‘?’ is increased tremendously, forcing you back to good old procedural code. And this is bad. No more stateless functions all over the place, but plenty of nested if and local state management.

        So, to me, operators like ‘?’ look nice from far, but don’t stand closer inspection. Furthermore, if going for a brand new language, why bothering with null in the first place? Other did avoid it!

        ++

        Reply

  7. I find it a little amusing that this catch (IOException | SQLException e) construction finally happened in Java 7 because I believe it originated with me when I suggested this to Josh Bloch (who worked a couple doors down from me) in 1997. He thought it was a good idea so he put it on his list of things to do, but wow it sure took a while!

    BTW, I really liked this article. Thanks for daring to have an opinion… I have also been wondering if monads are the best way, let alone The Only Way, to do optionality right and this type union idea is pretty elegant. I wrote about this some years ago where I called my less developed idea “type arithmetic”. My friend Eelco who swears by JOOQ thinks I should ask you questions about languages (I’m building one… *slaps forehead*) and this blog post seems to indicate he’s right.

    Reply

    1. Oh wow, excellent. Thank you very much for suggesting this to Josh, then, and nice meeting you, secret inventor of union types :)

      Indeed, I blog about the pragmatic side of languages from time to time. Being the designer of an internal domain-specific language (jOOQ) I get a very fresh, not every day angle at a variety of languages, thus my interest.

      I wish more language designers blogged as well. You should certainly follow Gavin King’s posts on http://ceylon-lang.org/blog. While I don’t think that Ceylon is a good language for most people (it feels weird and esoteric), it is one of the most sophisticated and advanced languages, where all other languages should (and sometimes also do, e.g. TypeScript) learn from.

      Another very interesting but completely unknown language (outside of academia and language design) is the Whiley language: http://whiley.org by Dave Pearce (https://twitter.com/whileydave). It (probably) invented sophisticated flow sensitive typing, which I’m sure we’ll see in many future languages. E.g. after IOException | SQLException e, we do instanceof checks on e, and within that check, the type changes to whatever we type checked it to be, so e now is of type IOException only, for instance. Ceylon copied or re-invented this feature from Whiley.

      Looking forward to learn about your language! Do you have a website already?

      Reply

  8. Just wanted to mention that Dart has null aware operators, so you write the “Object?.Object” change there too. It also gets expanded to the nested “if (obj!=null)” checks. Dart doesn’t have null aware types.

    Reply

    1. Thanks for your comment. Yes, null aware operators are nice syntax sugar, but they’re just that. Solving the problem with the type system seems to be the most robust solution, though.

      Reply

Leave a Reply