Ceylon Might Just be the Only (JVM) Language that Got Nulls Right

Here we go again. THAT TOPIC.

But hang on. The approach discussed here (and in the Ceylon language) is not something you see every day. At the same time, it is very cunning.

Nulls are baked into the language

… or so it may seem. Indeed, in Ceylon, like in Kotlin (and possibly many other languages), there is a special type “annotation” that you can postfix to any reference type in order to make it nullable. For instance:

String firstName = "Homer";
String? middleName = "J";
String lastName = "Simpson";

In the above example, both firstName and lastName are mandatory values that can never be null, whereas middleName is optional. Most languages that support the above then ship with special operators to access the optional value, e.g. ?. in Ceylon and also in Kotlin.

// Another optional value:
Integer? length = middleName?.length;

// A non-optional value:
Integer length = middleName?.length else 0;

So, what is it about Ceylon that works so smoothly?

The thing that Ceylon got very right is the fact that all of the above is just syntactic sugar that is:

  • Easy to use
  • Maps well to our mindset, where null still is a thing
  • Can interoperate with Java
  • Doesn’t introduce cognitive friction

For us Java folks, we can still pretend that null is an OK-ish, hard to avoid thing (as we’ve claimed before on this blog). But what is null really? Is it the absent value? The unknown value? The uninitialised value?

Java only has one null thingy, and it is (ab-)used for all of the previous things, and more, when in theory, it is only really the uninitialised value, nothing more. On the other hand, when working with JDBC (and thus, SQL), it implicitly means the unknown value (with all the related caveats).

In Ceylon, however, Null is a special type, similar to Void in Java. The only value that can be assigned to the Null type is null:

// Ceylon
Null x = null;

// Java
Void x = null;

But the big difference is, null cannot be assigned to any other type! Wait. Couldn’t we assign null to String? … ? Of course, the following is possible in Ceylon:

String? x = null;

But why is this possible? Because String? is just syntax sugar for String|Null, a union type, i.e. a type that is either the String type or the Null type.

Huh, what are union types?

Let’s look at this more closely. When in the jOOQ API you want to work with SQL functions and expressions, there is always a great set of overloads that provide you with a standard version, and a convenience version where you can pass a bind variable. Take the equals operator, for instance:

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
}

The above overloads allow you for writing things like the following, without needing to think about the distinction between a SQL expression and a Java bind variable (which is ultimately also a SQL expression):

// Comparing a column with bind variable
.where(BOOK.ID.eq(1))

// Comparing a column with another column expression
.and(BOOK.AUTHOR_ID.eq(AUTHOR.ID))

In fact, there are even more overloads, because the right hand side of a comparison operation can have other expressions as well, for instance:

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
    Condition eq(Select<? extends Record1<T>> query);
    Condition eq(QuantifiedSelect<? extends Record1<T>> query);
}

Now, the same set of overloads needs to be repeated for not equals, greater than, greater or equal, etc. Wouldn’t it be nice to be able to express this “right-hand-side” thingy as a single, reusable type? I.e. a union type of all of the above types?

interface Field<T> {
    Condition eq(
        Field<T>
      | T
      | Select<? extends Record1<T>>
      | QuantifiedSelect<? extends Record1<T>> thingy
    );
}

Or even

// This is called a type alias. Another awesome
// Ceylon language feature (pseudo syntax)
alias Thingy => 
    Field<T>
  | T
  | Select<? extends Record1<T>>
  | QuantifiedSelect<? extends Record1<T>>;

interface Field<T> {
    Condition eq(Thingy thingy);
}

After all, that’s also how the SQL language is defined. Heck, that’s how any BNF notation defines syntactic elements. For instance:

<predicate> ::=
    <comparison predicate>
  | <between predicate>
  | <in predicate>
  | <like predicate>
  | <null predicate>
  | <quantified comparison predicate>
  | <exists predicate>
  | <unique predicate>
  | <match predicate>
  | <overlaps predicate>

OK, granted, a syntactic element is not strictly the same thing as a type, but the intuitive perception is the same.

Oh, and Java has union types, too!

In a brief flash of revelation, the Java 7 expert groups added support for union types in exception handling. You can write things like:

try {
    ...
}
catch (IOException | SQLException e) {
    // e can be any of the above!
}

And you can emulate union types with generics, which don’t support union types but intersection types in Java.

Back to Ceylon and NULL

Ceylon has gotten Null right. Because, historically, a nullable type is a type that can be the “real” type or the “null” value. We want that. We Java developers crave that. We cannot live without the soothing option of this kind of optional.

But the excellent thing about this approach is that it is extendable. What if I really need to distinguish between “unknown”, “uninitialised”, “undefined”, “42”? I can. Using types. Here’s a String that can model all of the aforementioned “special values”:

String|Unknown|Uninitialised|Undefined|FortyTwo

And if that’s too verbose, I just assign a name to it

interface TheStringToRuleThemAll
  => String|Unknown|Uninitialised|Undefined|FortyTwo;

But it cannot be Null. Because I don’t want it to be that value, that is everything and nothing. Are you convinced? I bet you are. From now on:

Don’t trust any language that pretends that the Option(al) monad is a decent approach at modelling null. It isn’t.

― me. Just now

Why? Let me illustrate. Kotlin/Ceylon/Groovy style syntax sugar using the elvis operator (regardless of the backing null semantics):

String name = bob?.department?.head?.name

Same thing with Optional monads:

Optional<String> name = bob
    .flatMap(Person::getDepartment)
    .map(Department::getHead)
    .flatMap(Person::getName);

AND POOR YOU IF YOU MIX UP map() WITH flatMap() JUST ONCE!!

Some people claim

Using union types is like driving around in a brand new Ferrari with your mother-in-law in the passenger seat.

by Elvira

Sure. But I claim: Well done, Ceylon. Let’s hope we’ll get union types in Java, too, outside of catch blocks!

Further reading

Liked this article? How about:

10 Reasons not to Choose a Particular Open Source software

We’re all Software Engineers of one type or another. Most of us have one thing in common, though: We’re lazy. And we know that someone else was less lazy and has already solved that tedious problem that we’re on. And because we’re not only lazy but also stingy, we search for Free Open Source software.

But the problem with Open Source software is: There are millions of options for about every problem domain. Just look at web development with “modern” JavaScript. Which tool to choose? Which one will still be there tomorrow? Will it work? Will I get maintenance? New features? Plugins from the community?

While it is not so easy to find the right tool among the good ones (commons or guava? mockito or jmock? Hibernate or jOOQ or MyBatis?), it is certainly easier to rule out the bad ones.

Here are some things to look out for, when evaluating an Open Source software (in no particular order)

1. NullPointerExceptions, ClassCastExceptions

This is one of my favourites. It is very easy to google. No one is completely safe from these annoyances. But when you find stack traces, bug reports, investigate them closely.

  • Do they appear often?
  • Do they appear in similar contexts?
  • Do they appear in places where they could’ve been omitted?

It’s a matter of good design to be able to avoid NullPointerExceptions and ClassCastExceptions. It happens to everyone. But no NullPointerException should be thrown from a place that can be statically discovered by the Java compiler or with FindBugs.

Needless to say that the list of no-go exceptions thrown from a database library, for instance, can be extended with SQLExceptions due to syntax errors produced by that library.

2. Community Discussing Bugs instead of Features, Strategies, Visions

Every Open Source software has users and with Google Groups and GitHub, it has become fairly easy to interact with an OSS community.

For larger projects, the community also extends to Stack Overflow, Reddit, Twitter, etc. These next steps are a sign of popularity of an Open Source software, but not necessarily a sign that you should use them. Also, don’t be blinded by users saying “hey this is so cool”, “it just made my day”, “best software ever”. They say that to everyone who’s supporting them out of their misery (or laziness, see the intro of this post).

What you should be looking out for is whether the community is discussing visions, strategies, features, truly awesome ideas that can be implemented next year, in the next major release. It’s a true sign that not only the software will probably stick around, it will also become much better.

The converse to this is a community that mainly discusses bugs (see NullPointerException, ClassCastException). Unlike a “visionary” community, a “buggy” community will just create work, not inspiration to the vendor. But which one’s the chicken, which one’s the egg?

Another converse to this is a community that is disappointed by the false promises given by the visions of the vendor. I often have a feeling that Scala’s SLICK might qualify for that as it introduces an insurmountable language-mapping impedance mismatch between its own, LINQ-inspired querying DSL and SQL.

3. Poor Manual, Poor Javadoc

That’s easy to discover. Do you really want that? The best and most authoritative information should come from the software vendor, not some weirdo forum on the web that you’ve googled.

A good example are PostgreSQL’s Manuals.

A rant about bad examples can be seen here:
http://www.cforcoding.com/2009/08/its-time-we-stopped-rewarding-projects.html

Don’t be deceived by the idea that it might get better eventually. Poorly documented software will be poor in many other aspects.  And it’s such an easy thing to discover!

Of course, the “right” amount of documentation is an entirely other story…

4. No Semantic Versioning

Search for release notes and see if you’ll find something that roughly corresponds to semver.org. You will want patch releases when your Open Source software that you’re using in mission-critical software fails. When you get a patch release, you don’t want 50 new features (with new NullPointerExceptions, ClassCastExceptions).

5. Unorganised Appearance

Again, we’re in times of GitHub. The good old CVS times are over, where HTML was still used to share cooking recipes. Check if your Open Source software uses those tools. If they show that they’re using them. It will help you ascertain that the software will still be good in a couple of years if the vendor isn’t crushed by the mess they’ve gotten themselves in.

6. Vendor Side-Project evolving into an Offspring Product

Now that is a sign not everyone may agree upon, I guess. But after the experience I’ve made in previous jobs, I strongly believe that software that has evolved out of necessity before making it a product really suffers from its legacy. It wasn’t a product from the beginning and it has strong ties to the vendor’s original requirements, which doesn’t bother the vendor, but it will bother you. And because the vendor still has very strong ties to their offspring, they won’t be ready to make fundamental changes in both code and vision!

Specifically, in the database field, there are a couple of these software, e.g.

Note, I don’t know any of the above tools, so they may as well be awesome. But be warned. They weren’t designed as products. They were designed for a very narrow purpose originating from a pre-Apache context.

7. Generics are Poorly (or Overly) Adopted

Generics were introduced in 2004 with Java 5. Now that the heated debates about generic type erasure are over, generics are well adopted. Or aren’t they? The latest stable release 3.2.1 of Apache Commons Collections is still not generified! That must’ve been the number 1 reason why people had started shifting to Google Guava (or its predecessors) instead. There’s not much making for a lousier day than having raw types (or eels) slapped around your face.

The other thing that you should look out for, though, is over-generification. Generics can become really hard, even for top-shot Java architects. A common blunder is to strongly correlate subtype polymorphism with generic polymorphism without being aware of the effects. Having too many generics in an API is a good sign for an architecture astronaut. (or a design astronaut in this case). We’ll see further down how that may correlate with the person behind the design decisions.

8. Vendor Cannot Handle Objective Criticism or Competition

Here’s how to find out, who’s behind the Open Source software. While this isn’t important for a small, geeky tool, you should be very interested in the vendor as a person when looking for a strategic OSS addition, especially if you’re dealing with a benevolent dictator. The vendor should be:

  • Aware of competition, i.e. they’re doing marketing, learning from them. Improving to compete. This means that they are interested in being truly better, not just “convinced that they’re better”.
  • Open minded with their competition, with you as a customer, and ready to discuss various points of view.
  • Interested in new ideas, possibly putting them on a roadmap right away (but without losing focus for his main strategic roadmap).

Even if this is Open Source, there’s no point in being arrogant or conceited. The vendor should treat you like a customer (as long as you’re not trolling). Open-mindedness will eventually lead to the better product in the long run.

9. Vendor has no Commercial or Marketing Interests at All

Now, (Free) Open Source is nice for many reasons. As a vendor, you get:

  • Feedback more quickly
  • Feedback more often
  • Community (with pull requests, feature additions, etc.)
  • The feeling that you’re doing something good

True? Yes. But that’s true for commercial software as well. So what’s the real reason for doing Open Source? It depends. Adobe for instance has started opening up a lot, recently, since their acquisition of Day Software. All of JCR, JackRabbit, the upcoming JackRabbit Oak, Sling and Felix are still at Apache with the original committers still on board. But one can certainly not say that Adobe has no commercial interests.

OSS vendors should think economically and build products. Eventually, they may start selling stuff around their core products, or separate community and commercial licenses. And unlike they get too greedy (see Oracle and MySQL, vs RedHat and MariaDB), that can make commercial Open Source a very interesting business, also for the customer who will then get the good parts of Open Source (partially free, open, with a vibrant community) along with the good parts of commercial software (premium support, warranties, etc.)

In other words, don’t choose overly geeky stuff. But you might have recognised those tools before (poor documentation, no semantic versioning, poor tooling).

10. No Traction Anymore

To wrap this up, here’s an obvious last one. Many Open Source products don’t show any traction by the vendor. That goes along well with the previous point, where the vendor has no commercial interest. Without commercial long-term interest, they’ll also lose all other interest. And you’re stuck with maintaining a pile of third-party code yourself (fixing its many many ClassCastExceptions, NullPointerExceptions).

TL;DR : Conclusion

You should chose Open Source just like commercial software. Economically.

  • Open Source is not an excuse for bad quality.
  • Open Source is not an excuse for lack of support.
  • Open Source is not an excuse for non-professionalism.

If Open Source fails you on any of the above, the joke will be on you, the customer. You’ll get a bad product, and you’ll pay the price with exaggerated maintenance on your side, which you thought you’d avoid by chosing something free. Nothing is free. Not even Free Open Source. Ask the Grumpy Nerd

On Java 8’s introduction of Optional

I had recently discovered the JDK 8’s addition of the Optional type. The Optional type is a way to avoid NullPointerException, as API consumers that get Optional return values from methods are “forced” to perform “presence” checks in order to consume their actual return value. More details can be seen in the Javadoc.

A very interesting further read can be seen here in this blog post, which compares the general notion of null and how null is handled in Java, SML, and Ceylon:
http://blog.informatech.cr/2013/04/10/java-optional-objects

“blank” and “initial” states were already known to Turing . One could also argue that the “neutral” or “zero” state was required in the Babbage Engine, which dates back to Ada of Lovelace in the 1800′s.

On the other hand, mathematicians also prefer to distinguish “nothing” from “the empty set”, which is “a set with nothing inside”. This compares well with “NONE” and “SOME”, as illustrated by the aforementioned Informatech blog post, and as implemented by Scala, for instance.

Anyway, I’ve given Java’s Optional some thought. I’m really not sure if I’m going to like it, even if Java 9 would eventually add some syntactic sugar to the JLS, which would resemble that of Ceylon to leverage Optional on a language level. Since Java is so incredibly backwards-compatible, none of the existing APIs will be retrofitted to return Optional, e.g, the following isn’t going to surface the JDK 8:

public interface List<E> {
    Optional<E> get(int index);
    [...]
}

Not only can we assign null to an Optional variable, but the absence of “Optional” doesn’t guarantee the semantics of “SOME”, as lists will still return “naked” null values. When we mix the two ways of thinking, we will wind up with two checks, instead of one

Optional<T> optional = // [...]
T nonOptional = list.get(index);

// If we're paranoid, we'll double-check!
if (optional != null && optional.isPresent()) {
    // do stuff
}

// Here we probably can't trust the value
if (nonOptional != null) {
    // do stuff
}

Hence…

-1 from me to Java’s solution

Further reading

Of course, this has been discussed millions of times before. So here are a couple of links: