Using Kotlin’s Apply Function for Dynamic SQL with jOOQ

It was hard to limit ourselves to 10 Nice Examples of Writing SQL in Kotlin With jOOQ, recently, because the Kotlin language has many nice little features that really help a lot when working with Java libraries. We’ve talked about the nice with() stdlib function, which allows to “import” a namespace for a local scope or closure:

with (AUTHOR) {
    ctx.select(FIRST_NAME, LAST_NAME)
       .from(AUTHOR)
       .where(ID.lt(5))
       .orderBy(ID)
       .fetch {
           println("${it[FIRST_NAME]} ${it[LAST_NAME]}")
       }
}

In the above example, the AUTHOR table is made available as the this reference in the closure following the with function, which works exactly like JavaScript’s with(). Everything in AUTHOR is available, without dereferencing it from AUTHOR.

Apply is very similar

A very similar feature is made available through apply(), although with different syntactic implications. Check out this Stack Overflow question for some details about with() vs. apply() in Kotlin.

When using jOOQ, apply() is most useful for dynamic SQL. Imagine you have local variables indicating whether some parts of a query should be added to the query:

val filtering = true;
val joining = true;

These boolean variables would be evaluated dynamically, of course. filtering specifies whether a dynamic filter / where clause is needed, whereas joining specifies whether an additional JOIN is required.

So, the following query will select authors, and:

  • if “filtering”, we’re selecting only author ID = 1
  • if “joining”, we’ll join the books table and count the number of books per author

Both of these predicates are independent. Enter the game: apply():

ctx.select(
      a.FIRST_NAME, 
      a.LAST_NAME, 
      if (joining) count() else value(""))
   .from(a)
   .apply { if (filtering) where(a.ID.eq(1)) }
   .apply { if (joining) join(b).on(a.ID.eq(b.AUTHOR_ID)) }
   .apply { if (joining) groupBy(a.FIRST_NAME, a.LAST_NAME) }
   .orderBy(a.ID)
   .fetch {
       println(it[a.FIRST_NAME] + " " + 
               it[a.LAST_NAME] +
               (if (joining) " " + it[count()] else ""))
   }

That’s neat! See, the jOOQ API doesn’t specify any apply() method / function, yet you can chain the apply() function to the jOOQ API as if it were natively supported.

Like with(), apply() makes a reference available to a closure as this, so it doesn’t have to be referenced explicitly anymore. Which means, we can write neat things like

   .apply { if (filtering) where(a.ID.eq(1)) }

Where a where() clause is added only if we’re filtering!

Of course, jOOQ (or any other query builder) lends itself to this kind of dynamic SQL, and it can be done in Java too:
https://www.jooq.org/doc/latest/manual/sql-building/dynamic-sql

But the Kotlin-specific fluent integration using apply() is exceptionally neat. Well done, Kotlin!

Side-note

This only works because the jOOQ DSL API of jOOQ 3.x is mutable and every operation returns the same this reference as was kindly pointed out by Ilya Ryzhenkov

In the future (e.g. version 4.0), we’re planning on making the jOOQ API more immutable – mutability is a historic legacy (although, often, it’s the desired behaviour for a query builder).

More nice Kotlin/jOOQ tricks in this article here.

10 Nice Examples of Writing SQL in Kotlin With jOOQ

Kotlin is the next big thing. With Google announcing official support for Kotlin on Android, we’ll see a lot more traction for this lovely language.

We’ve already blogged about the Kotlin language recently: 10 Features I Wish Java Would Steal From the Kotlin Language.

Should you migrate your application to Kotlin? Perhaps – there are many questions. The most important question is (of course): What are you going to do with your awesome SQL code?

The answer is: Use jOOQ! Need convincing? Here are 10 nice examples of writing SQL in Kotlin with jOOQ:

(Note: All the code is available on GitHub)

1. Using Kotlin’s Map Access Literals With jOOQ Records

This is a free feature that you’re getting from Kotlin thanks to their smart move of supporting certain operators on custom types by convention. Check out how easy it is to access a value from a record using a column reference:

val a = AUTHOR
val b = BOOK

ctx.select(a.FIRST_NAME, a.LAST_NAME, b.TITLE)
   .from(a)
   .join(b).on(a.ID.eq(b.AUTHOR_ID))
   .orderBy(1, 2, 3)
   .forEach {
       println("${it[b.TITLE]} by "
             + "${it[a.FIRST_NAME]} ${it[a.LAST_NAME]}")
   }

The output from our sample code is this:

1984 by George Orwell
Animal Farm by George Orwell
Brida by Paulo Coelho
O Alquimista by Paulo Coelho

We’re already using a couple of useful language features here. First off:

val for local variable type inference

We’re renaming AUTHOR to a and BOOK to b in our code to shorten table names a little bit. Using val we don’t even need to explicitly refer to the type of those tables.

As it looks right now, we’re going to get that feature in the Java language as well, so stay tuned.

(More about val in item #3)

More info about this feature here.

String interpolation

As you can see, inside of the String literal, we’re using the ${dollar.notation} to access variables from the context.

More info about this feature here.

(More about string interpolation in item #5)

Implicit lambda parameter “it”

When writing single-parameter lambda expressions, which is typical for loops, mapping, and many other standard library features, we don’t need to actually name that parameter in Kotlin. The default name is “it”. Groovy developers will rejoice, as they can profit from the same feature.

More info about this feature here.

Finally: Operator overloading

Kotlin supports a nice and pragmatic approach to operator overloading. We’re making use of the fact that the following two things mean exactly the same in Kotlin:

val x1 = anything[key];
val x2 = anything.get(key);

anything[key] = value;
anything.set(key, value);

Yeah, why not? In jOOQ, we’ve already seen this coming in version 3.9, and we’ve thus added get() and set() methods on our ubiquitous Record type, so you can access record values by:

  • Field reference (with type safety)
  • Column name (no type safety)
  • Index (no type safety)

Note that operator overloading can be used as well with a couple of arithmetic operators:

ctx.select(TRANSACTION.AMOUNT + 10) // Translates to Field.plus(10)
   .from(TRANSACTION)
   .fetch();

More info about this feature here.

2. Loop Over Record Contents

Another really nice feature of the language is destructuring of “tuples” into several local variables. Quite a few languages support this now, and so does Kotlin, even without first-level language support for real tuples. But often, we don’t actually need tuples, we just need their syntax. This can be done with any Map, for instance. Consider this code:

for (r in ctx.fetch(b))
    for ((k, v) in r.intoMap())
        println("${r[b.ID]}: ${k.padEnd(20)} = $v")

The output from our sample code is this:

1: ID                   = 1
1: AUTHOR_ID            = 1
1: TITLE                = 1984
1: PUBLISHED_IN         = 1948
2: ID                   = 2
2: AUTHOR_ID            = 1
2: TITLE                = Animal Farm
2: PUBLISHED_IN         = 1945
3: ID                   = 3
3: AUTHOR_ID            = 2
3: TITLE                = O Alquimista
3: PUBLISHED_IN         = 1988
4: ID                   = 4
4: AUTHOR_ID            = 2
4: TITLE                = Brida
4: PUBLISHED_IN         = 1990

This is really nice! Every jOOQ record can be represented as a Map using Record.intoMap(), which can be destructured automatically in loops as can be seen above.

More info about this feature here.

3. Local Variable Type Inference

An API that makes use of generics as heavily as jOOQ can leave quite some burden on the user occasionally. Type safety is great for the DSL, e.g. in jOOQ, the Java / Kotlin compilers can type-check things like:

// Type checks: Both sides are of type string
AUTHOR.FIRST_NAME.in("Alice", "Bob");

// Type checks: 
// - Both sides are of type string
// - Both sides are of degree one
AUTHOR.FIRST_NAME.in(
  select(CUSTOMER.FIRST_NAME).from(CUSTOMER)
);

// Doesn't type check: Wrong degree
AUTHOR.FIRST_NAME.in(
  select(CUSTOMER.FIRST_NAME, CUSTOMER.LAST_NAME).from(CUSTOMER)
)

This works through fancy types like:

Select<Record2<String, String>> subselect =
  select(CUSTOMER.FIRST_NAME, CUSTOMER.LAST_NAME).from(CUSTOMER);

These types can get quite hairy as they go up to 22 (higher degrees are supported without type safety). So, in Kotlin, we can simply write:

val subselect =
  select(CUSTOMER.FIRST_NAME, CUSTOMER.LAST_NAME).from(CUSTOMER);

// Still doesn't type check:
AUTHOR.FIRST_NAME.in(subselect)

// This works:
row(AUTHOR.FIRST_NAME, AUTHOR.LAST_NAME).in(subselect)

More info about this feature here.

4. Inline Functions for New jOOQ Features

Are you missing a feature in jOOQ, for instance the PostgreSQL ilike operator? Well, it is supported through the Field.likeIgnoreCase() method, but you might prefer the more native looking ilike operator. No problem with Kotlin. Just write an inline function (which works similar to an extension function but they’re not exactly the same thing):

inline fun <F: Field<String>> F.ilike(field: String): Condition {
    return condition("{0} ilike {1}", this, field)
}

These inline functions make use of jOOQ’s plain SQL API which is heavily used by Java developers as well, but in this case, the keyword inline on the above functions indicates that the function works like an inline function. We can now write a query like this:

println("${ctx.select(b.TITLE)
              .from(b)
              .where(b.TITLE.ilike("%animal%"))
              .fetchOne(b.TITLE)}")

The result being:

Animal Farm

Notice how we can simply write b.TITLE.ilike("..") even if the jOOQ API doesn’t have such a method!

More info about this feature here

5. String Interpolation and Multiline Strings

How many times have we wished for multiline strings in Java? So many other languages have it, even SQL and PL/SQL. When working with string-based embedded SQL, this is just a killer feature. Good for Kotlin! Not only does Kotlin support multiline strings, but also string interpolation.

Here’s how to use the jOOQ plain SQL API with Kotlin:

ctx.resultQuery("""
    SELECT *
    FROM (
      VALUES (1, 'a'),
             (2, 'b')
    ) AS t
    """)
    .fetch()
    .forEach {
        println("${it.intoMap()}")
    }

Just copy-paste any arbitrary SQL statement right from your SQL console (in this case: H2 syntax) and you’re done. The output of the above is:

{C1=1, C2=a}
{C1=2, C2=b}

Bonus feature if you’re using the upcoming jOOQ 3.10 parser: You can standardise on SQL features that are not available from your database:

val colX = field("x")
val colY = field("y")
ctx.parser()
   .parseResultQuery("""
    SELECT *
    FROM (
      VALUES (1, 'a'),
             (2, 'b')
      -- This feature (derived column lists) 
      -- isn't really available in H2 yet it works!
    ) AS t(${colX.name}, ${colY.name})
    """)
   .fetch()
   .forEach {
       println("${it[colX]}, ${it[colY]}")
   }

The above statement makes use of derived column lists (renaming tables and columns in one go). Unfortunately, H2 doesn’t support this feature but jOOQ can emulate it, transparently. Note that we’re again using string interpolation, this time in order to reuse column names.

More info about the feature here.

6. Null Safe Dereferencing

Null is the mother of all bikesheds. Of course, Kotlin is opinionated about null as well

You’re not sure if your query will produce a result? No problem with Kotlin’s various null-related operator:

println("${ctx.fetchOne(b, b.ID.eq(5))?.title ?: "not found"}")

The above will print:

not found

Why? Because fetchOne() won’t return a record from our database, it will return null. We cannot dereference ?.title from null, but the ?. operator won’t throw an exception, it will just return null again. Finally, the Elvis operator will provide a default value if the expression on the left of it is null. Perfect!

More info about the feature here.

7. Getters and setters feel like properties

Remember the last 20 years when we emulated properties in Java using the obnoxious JavaBeans convention? Tired of writing getters and setters? Well, at least, with Kotlin, it pays off. We can syntactically pretend that we have a property where there are only getters and setters. Check this out:

val author1 = ctx.selectFrom(a).where(a.ID.eq(1)).fetchOne();
println("${author1.firstName} ${author1.lastName}")

val author2 = ctx.newRecord(a);
author2.firstName = "Alice"
author2.lastName = "Doe"
author2.store()
println("${author2.firstName} ${author2.lastName}")

Isn’t that awesome? we can just dereference author.firstName, which simply translates to author.getFirstName(). Likewise, we can assign a value to author2.lastName = "Doe", which simply translates to author2.setLastName("Doe")

It seems so obvious, no?

More info about the feature here

8. With to Abbreviate Setter Calls

We can take this feature one step further and use the nice with() inline function (not a keyword!). In the above example, when storing a new author, we have to repeat the author reference all the time. This is no longer true with… with!

val author3 = ctx.newRecord(a);
with (author3) {
    firstName = "Bob"
    lastName = "Doe"
    store()
}

All the code that is inside of the block supplied to the with() function is going to operate on the with() function’s first argument. We already know this interesting feature from JavaScript, but in Kotlin, it’s typesafe!

And notice, the store() call is also affected!

More info about this feature here

9. With to Locally Import Tables

We can use with() also with queries! In jOOQ, columns are instance members inside of tables, and as such, must be dereferenced from a concrete table. That can be tedious (SQL doesn’t have this requirement), especially when selecting only from a single table. You can turn on static member generation in jOOQ’s code generator, but then table aliasing is much less easy.

Or you use Kotlin again! The with() function can help:

with (AUTHOR) {
    ctx.select(FIRST_NAME, LAST_NAME)
       .from(AUTHOR)
       .where(ID.lt(5))
       .orderBy(ID)
       .fetch {
           println("${it[FIRST_NAME]} ${it[LAST_NAME]}")
       }
}

Resulting in

George Orwell
Paulo Coelho

Isn’t that nice? Now, all the FIRST_NAME, LAST_NAME, ID column references are dereferenced from AUTHOR in a much more lenient way.

More info about this feature here

10. Destructuring jOOQ records into sets of local variables

This is so great, we’re going to add more native support for it in jOOQ 3.10 right away.

Kotlin’s type destructuring feature is implemented following convention. This means that any type that has methods with the name component[N] can be destructured. We currently don’t have those methods, but they can be added really easily using inline operators:

operator fun <T, R : Record3<T, *, *>> R.component1() : T {
    return this.value1();
}

operator fun <T, R : Record3<*, T, *>> R.component2() : T {
    return this.value2();
}

operator fun <T, R : Record3<*, *, T>> R.component3() : T {
    return this.value3();
}

These inline functions are added to the R type, which is any subtype of Record3, and then they can be used as such:

for ((first, last, title) in ctx
   .select(a.FIRST_NAME, a.LAST_NAME, b.TITLE)
   .from(a)
   .join(b).on(a.ID.eq(b.AUTHOR_ID))
   .orderBy(1, 2, 3))
       println("$title by $first $last")

The result is again:

1984 by George Orwell
Animal Farm by George Orwell
Brida by Paulo Coelho
O Alquimista by Paulo Coelho

As can be seen, our query returns the type Record3<String, String, String>. Our inline functions will add component1(), component2(), and component3() methods to the type, which are used by the Kotlin language to destructure the record into three local loop variables: (first, last, title).

This is incredibly useful!

(note also that we don’t have to execute the query explicitly in the loop because any jOOQ ResultQuery is Iterable)

More info about this feature here

Conclusion

Kotlin is getting increasingly popular. It contains a lot of smart yet pragmatic features that make daily work with programming on the JVM a lot easier. Kotlin’s high focus on Java interoperability makes it really really powerful, and working with Kotlin and jOOQ is a really productive way of writing SQL on the JVM.

Do give it a shot. The sources are here:

https://github.com/jOOQ/jOOQ/tree/master/jOOQ-examples/jOOQ-kotlin-example

10 Features I Wish Java Would Steal From the Kotlin Language

This article is overdue. After the hype around the release of Kotlin 1.0 has settled, let’s have a serious look at some Kotlin language features that we should have in Java as well.

In this article, I’m not going to wish for unicorns. But there are some low hanging fruit (as far as I naively can see), which could be introduced into the Java language without great risk. While you’re reading this article, be sure to copy paste examples to http://try.kotlinlang.org, an online REPL for Kotlin

1. Data class

Language designers hardly ever agree on the necessity and the feature scope of what a class is. In Java, curiously, every class always has identity a concept that is not really needed in 80% – 90% of all real world Java classes. Likewise, a Java class always has a monitor on which you can synchronize.

In most cases, when you write a class, you really just want to group values, like Strings, ints, doubles. For instance:

public class Person {
    final String firstName;
    final String lastName;
    public JavaPerson(...) {
        ...
    }
    // Getters
    ...

    // Hashcode / equals
    ...

    // Tostring
    ...

    // Egh...
}

By the time you’ve finished typing all of the above, your fingers will no longer be. Java developers have implemented ugly workarounds for the above, like IDE code generation, or lombok, which is the biggest of all hacks. In a better Java, nothing in Lombok would really be needed.

As, for instance, if Java had Kotlin’s data classes:

data class Person(
  val firstName: String,
  val lastName: String
)

The above is all we need to declare the equivalent of the previous Java code. Because a data class is used to store data (duh), i.e. values, the implementation of things like hashCode(), equals(), toString() is obvious and can be provided by default. Furthermore, data classes are first class tuples, so they can be used as such, e.g. to destructure them again in individual references:

val jon = Person("Jon", "Doe") 
val (firstName, lastName) = jon

In this case, we may hope. Valhalla / Java 10 is being designed and with it, value types. We’ll see how many features will be provided on the JVM directly, and in the Java language. This will certainly be an exciting addition.

Notice how val is possible in Kotlin: Local variable type inference. This is being discussed for a future Java version right now.

2. Defaulted parameters

How many times do you overload an API like the following:

interface Stream<T> {
    Stream<T> sorted();
    Stream<T> sorted(Comparator<? super T> comparator);
}

The above are exactly the same JDK Stream operations. The first one simply applies Comparator.naturalOrder() to the second one. So we could write the following, in Kotlin:

fun sorted(comparator : Comparator<T> 
         = Comparator.naturalOrder()) : Stream<T>

The advantage of this isn’t immediately visible, when there is only one defaulted parameter. But imagine a function with tons of optional parameters:

fun reformat(str: String,
             normalizeCase: Boolean = true,
             upperCaseFirstLetter: Boolean = true,
             divideByCamelHumps: Boolean = false,
             wordSeparator: Char = ' ') {
...
}

Which can be called in any of the following ways:

reformat(str)
reformat(str, true, true, false, '_')
reformat(str,
  normalizeCase = true,
  upperCaseFirstLetter = true,
  divideByCamelHumps = false,
  wordSeparator = '_'
)

The power of defaulted parameters is that they are especially useful when passing arguments by name, rather than by index. This is currently not supported in the JVM, which until Java 8, doesn’t retain the parameter name at all (in Java 8, you can turn on a JVM flag for this, but with all of Java’s legacy, you shouldn’t rely on this yet).

Heck, this feature is something I’m using in PL/SQL every day. Of course, in Java, you can work around this limitation by passing a parameter object.

3. Simplified instanceof checks

If you will, this is really an instanceof switch. Some people may claim that this stuff is evil, bad OO design. Nja nja. I say, this happens every now and then. And apparently, in Java 7, string switches were considered sufficiently common to modify the language to allow them. Why not instanceof switches?

val hasPrefix = when(x) {
  is String -> x.startsWith("prefix")
  else -> false
}

Not only is this doing an instanceof switch, it is doing it in the form of an assignable expression. Kotlin’s version of this when expression is powerful. You can mix any sort of predicate expressions, similar to SQL’s CASE expression. For instance, this is possible as well:

when (x) {
  in 1..10 -> print("x is in the range")
  in validNumbers -> print("x is valid")
  !in 10..20 -> print("x is outside the range")
  else -> print("none of the above")
}

Compare to SQL (not implemented in all dialects):

CASE x
  WHEN BETWEEN 1 AND 10 THEN 'x is in the range'
  WHEN IN (SELECT * FROM validNumbers) THEN 'x is valid'
  WHEN NOT BETWEEN 10 AND 20 'x is outside the range'
  ELSE 'none of the above'
END

As you can see, only SQL is more powerful than Kotlin.

4. Map key / value traversal

Now this could really be done very easily only with syntax sugar. Granted, having local variable type inference would already be a plus, but check this out

val map: Map<String, Int> = ...

And now, you can do:

for ((k, v) in map) {
    ...
}

After all, most of the time when traversing a map, it’ll be by Map.entrySet(). Map could have been enhanced to extend Iterable<Entry<K, V>> in Java 5, but hasn’t. That’s really a pity. After all, it has been enhanced in Java 8 to allow for internal iteration over the entry set in Java 8 via Map.forEach():

map.forEach((k, v) -> {
    ...
});

It’s not too late, JDK gods. You can still let Map<K, V> extend Iterable<Entry<K, V>>

5. Map access literals

This one is something that would add tons and tons of value to the Java language. We have arrays, like most other languages. And like most other languages, we can access array elements by using square brackets:

int[] array = { 1, 2, 3 };
int value = array[0];

Note also the fact that we have array initialiser literals in Java, which is great. So, why not also allow for accessing map elements with the same syntax?

val map = hashMapOf<String, Int>()
map.put("a", 1)
println(map["a"])

In fact, x[y] is just syntax sugar for a method call backed by x.get(y). This is so great, we have immediately proceeded with renaming our Record.getValue() methods in jOOQ to Record.get() (leaving the old ones as synonyms, of course), such that you can now dereference your database record values as such, in Kotlin

ctx.select(a.FIRST_NAME, a.LAST_NAME, b.TITLE)
   .from(a)
   .join(b).on(a.ID.eq(b.AUTHOR_ID))
   .orderBy(1, 2, 3)
   .forEach {
       println("""${it[b.TITLE]} 
               by ${it[a.FIRST_NAME]} ${it[a.LAST_NAME]}""")
   }

Since jOOQ holds all column type information on individual record columns, you can actually know in advance that it[b.TITLE] is a String expression. Great, huh? So, not only can this syntax be used with JDK maps, it can be used with any library that exposes the basic get() and set() methods.

Stay tuned for more jOOQ and Kotlin examples here:
https://github.com/jOOQ/jOOQ/blob/master/jOOQ-examples/jOOQ-kotlin-example/src/main/kotlin/org/jooq/example/kotlin/FunWithKotlinAndJOOQ.kt

6. Extension functions

This one is a controversial topic, and I can perfectly understand when language designers stay clear of it. But every now and then, extension functions are very useful. The Kotlin syntax here is actually just for a function to pretend to be part of the receiver type:

fun MutableList<Int>.swap(index1: Int, index2: Int) {
  val tmp = this[index1] // 'this' corresponds to the list
  this[index1] = this[index2]
  this[index2] = tmp
}

This will now allow for swapping elements in a list:

val l = mutableListOf(1, 2, 3)
l.swap(0, 2)

This would be very useful for libraries like jOOλ, which extends the Java 8 Stream API by wrapping it in a jOOλ type (another such library is StreamEx, with a slightly different focus). The jOOλ Seq wrapper type is not really important, as it pretends to be a Stream on steroids. It would be great, if jOOλ methods could be put onto Stream artificially, just by importing them:

list.stream()
    .zipWithIndex()
    .forEach(System.out::println);

The zipWithIndex() method isn’t really there. The above would just translate to the following, less readable code:

seq(list.stream())
    .zipWithIndex()
    .forEach(System.out::println);

In fact, extension methods would even allow to bypass wrapping everything explicitly in a stream(). For instance, you could then do:

list.zipWithIndex()
    .forEach(System.out::println);

As all of jOOλ’s method could be designed to also be applied to Iterable.

Again, this is a controversial topic. For instance, because

While giving the illusion of being virtual, extension functions really are just sugared static methods. It’s a significant risk for object oriented application design to engage in that trickery, which is why this feature probably won’t make it into Java.

7. Safe-call operator (and also: Elvis operator)

Optional is meh. It’s understandable that an Optional type needed to be introduced in order to abstract over the absence of primitive type values, which cannot be null. We now have things like OptionalInt, e.g. to model things like:

OptionalInt result =
IntStream.of(1, 2, 3)
         .filter(i -> i > 3)
         .findFirst();

// Agressive programming ahead
result.orElse(OR_ELSE);

Optional is a monad

Yes. It allows you to flatMap() the absent value.

o_O

Sure, if you want to do sophisticated functional programming, you’ll start typing map() and flatMap() everywhere. Like today, when we’re typing getters and setters. Along will come lombok generating flatmapping calls, and Spring will add some @AliasFor style annotation for flatmapping. And only the enlightened will be able to decipher your code.

When all we needed was just a simple null safety operator before getting back to daily business. Like:

String name = bob?.department?.head?.name

I really like this type of pragmatism in Kotlin. Or do you prefer (flat)mapping?

Optional<String> name = bob
    .flatMap(Person::getDepartment)
    .map(Department::getHead)
    .flatMap(Person::getName);

Can you read this? I cannot. Neither can I write this. If you get this wrong, you’ll get boxoxed.

Of course, Ceylon is the only language that got nulls right. But Ceylon has tons of features that Java will not get before version 42, and I’m not wishing for unicorns. I’m wishing for the safe-call operator (and also the elvis operator, which is slightly different), which could be implemented in Java too. The above expression is just syntax sugar for:

String name = null;
if (bob != null) {
    Department d = bob.department
    if (d != null) {
        Person h = d.head;
        if (h != null)
            name = h.name;
    }
}

What can possibly be wrong with that simplification?

8. Everything is an expression

Now this might just be a unicorn. I don’t know if there is a JLS / parser limitation that will forever keep us in the misery of prehistoric distinction between statement and expression.

At some point in time, people have started using statements for things that yield side-effects, and expressions for more functional-ish things. It is thus not surprising, that all String methods are really expressions, operating on an immutable string, returning a new string all the time.

This doesn’t seem to go well with, for instance, if-else in Java, which is expected to contain blocks and statements, each possibly yielding side-effects.

But is that really a requirement? Can’t we write something like this in Java as well?

val max = if (a > b) a else b

OK, we have this weird conditional expression using ?:. But what about Kotlin’s when (i.e. Java’s switch)?

val hasPrefix = when(x) {
  is String -> x.startsWith("prefix")
  else -> false
}

Isn’t that much more useful than the following equivalent?

boolean hasPrefix;

if (x instanceof String)
    hasPrefix = x.startsWith("prefix");
else
    hasPrefix = false;

(yes, I know about ?:. I just find if-else easier to read, and I don’t see why that should be a statement, not an expression. Heck, in Kotlin, even try is an expression, not a statement:

val result = try {
    count()
} catch (e: ArithmeticException) {
    throw IllegalStateException(e)
}

Beautiful!

9. Single expression functions

Now this. This would save so much time reading and writing simple glue code. And in fact, we already have the syntax in annotations. Check out Spring’s magical @AliasFor annotation, for instance. It yields:

public @interface AliasFor {
    @AliasFor("attribute")
    String value() default "";
    @AliasFor("value")
    String attribute() default "";
}

Now, if you squint really hard, these are just methods yielding constant values, because annotations are just interfaces with generated byte code for their implementations. We can discuss syntax. Of course, this irregular usage of default is weird, given that it was not re-used in Java 8 for default methods, but I guess Java always needs the extra syntax so developers feel alive as they can better feel their typing fingers. That’s OK. We can live with that. But then again, why do we have to? Why not just converge to the following?

public @interface AliasFor {
    String value() = "";
    String attribute() = "";
}

And the same also for class / interface default methods?

// Stop pretending this isn't an interface
public interface AliasFor {
    String value() = "";
    String attribute() = "";
}

Now that would look nice. But given Java’s existing syntax, this might just be a unicorn, so let’s move on to…

10. Flow-sensitive typing

Now this. THIS!

We’ve blogged about sum types before. Java has sum types with exceptions since Java 7:

try {
    ...
}
catch (IOException | SQLException e) {
    // e can be of type IOException and/or SQLException
    // within this scope
}

But Java, unfortunately, doesn’t have flow-sensitive typing. Flow-sensitive typing is of the essence in a language that supports sum types, but it is also useful otherwise. For instance, in Kotlin:

when (x) {
    is String -> println(x.length)
}

We don’t need to cast, obviously, because we already checked that x is String. Conversely, in Java:

if (x instanceof String)
    System.out.println(((String) x).length());

Aaagh, all this typing. IDE autocompletion is smart enough to offer a contextual type’s methods already and then generate the unnecessary cast for you. But it would be great if this was never needed, every time we explicitly narrow a type using control flow structures.

For more info, see this wikipedia entry about flow sensitive typing. A feature that could absolutely be added to the Java language. After all, we already got flow-sensitive final local variables since Java 8.

11. (Bonus) Declaration site variance

Last but not least, better generics via declaration site variance. Many other languages know this, for instance also C#’s IEnumerable:

public interface IEnumerable<out T> : IEnumerable

The keyword out here means that the generic type T is produced from the type IEnumerable (as opposed to in, which stands for consumption). In C#, Scala, Ceylon, Kotlin, and many other languages, we can declare this on the type declaration, rather than on its usage (although, many languages allow for both). In this case, we say that IEnumerable is covariant with its type T, which means again that IEnumerable<Integer> is a subtype of IEnumerable<Object>

In Java, this isn’t possible, which is why we have a bazillion question by Java newbies on Stack Overflow. Why can’t I…

Iterable<String> strings = Arrays.asList("abc");
Iterable<Object> objects = strings; // boom

In languages like Kotlin, the above would be possible. After all, why shouldn’t it? A thing that can produce strings can also produce objects, and we can even use it in this way in Java:

Iterable<String> strings = Arrays.asList("abc");
for (Object o : strings) {
    // Works!
}

The lack of declaration site variance has made a lot of APIs very intelligible. Consider Stream:

<R> Stream<R> flatMap(Function<? super T, ? extends Stream<? extends R>> mapper);

This is just noise. A function is contravariant with its argument type and covariant with its result type by nature a better definition of Function or Stream would be:

interface Function<in T, out R> {}
interface Stream<out T> {}

If this were possible, all that ? super and ? extends garbage could be removed without losing any functionality.

In case you’re wondering what I’m even talking about? 🙂

The great news is, this is being discussed for a (near) future version of Java:
http://openjdk.java.net/jeps/8043488

Conclusion

Kotlin is a promising language, even if it is very late to a game that already seems to have been decided, not in favour of alternative languages on the JVM. Nonetheless, it is a very interesting language to learn from, and with a lot of very good decisions made about some simple things.

Some of these decisions will hopefully be picked up by the Java language gods and integrated into Java. This list here shows some features that might be “easy” to add.

More info about Kotlin idioms:
https://kotlinlang.org/docs/reference/idioms.html

Liked this article?

Read on here:

Ceylon Might Just be the Only (JVM) Language that Got Nulls Right

Here we go again. THAT TOPIC.

But hang on. The approach discussed here (and in the Ceylon language) is not something you see every day. At the same time, it is very cunning.

Nulls are baked into the language

… or so it may seem. Indeed, in Ceylon, like in Kotlin (and possibly many other languages), there is a special type “annotation” that you can postfix to any reference type in order to make it nullable. For instance:

String firstName = "Homer";
String? middleName = "J";
String lastName = "Simpson";

In the above example, both firstName and lastName are mandatory values that can never be null, whereas middleName is optional. Most languages that support the above then ship with special operators to access the optional value, e.g. ?. in Ceylon and also in Kotlin.

// Another optional value:
Integer? length = middleName?.length;

// A non-optional value:
Integer length = middleName?.length else 0;

So, what is it about Ceylon that works so smoothly?

The thing that Ceylon got very right is the fact that all of the above is just syntactic sugar that is:

  • Easy to use
  • Maps well to our mindset, where null still is a thing
  • Can interoperate with Java
  • Doesn’t introduce cognitive friction

For us Java folks, we can still pretend that null is an OK-ish, hard to avoid thing (as we’ve claimed before on this blog). But what is null really? Is it the absent value? The unknown value? The uninitialised value?

Java only has one null thingy, and it is (ab-)used for all of the previous things, and more, when in theory, it is only really the uninitialised value, nothing more. On the other hand, when working with JDBC (and thus, SQL), it implicitly means the unknown value (with all the related caveats).

In Ceylon, however, Null is a special type, similar to Void in Java. The only value that can be assigned to the Null type is null:

// Ceylon
Null x = null;

// Java
Void x = null;

But the big difference is, null cannot be assigned to any other type! Wait. Couldn’t we assign null to String? … ? Of course, the following is possible in Ceylon:

String? x = null;

But why is this possible? Because String? is just syntax sugar for String|Null, a union type, i.e. a type that is either the String type or the Null type.

Huh, what are union types?

Let’s look at this more closely. When in the jOOQ API you want to work with SQL functions and expressions, there is always a great set of overloads that provide you with a standard version, and a convenience version where you can pass a bind variable. Take the equals operator, for instance:

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
}

The above overloads allow you for writing things like the following, without needing to think about the distinction between a SQL expression and a Java bind variable (which is ultimately also a SQL expression):

// Comparing a column with bind variable
.where(BOOK.ID.eq(1))

// Comparing a column with another column expression
.and(BOOK.AUTHOR_ID.eq(AUTHOR.ID))

In fact, there are even more overloads, because the right hand side of a comparison operation can have other expressions as well, for instance:

interface Field<T> {
    Condition eq(Field<T> field);
    Condition eq(T value);
    Condition eq(Select<? extends Record1<T>> query);
    Condition eq(QuantifiedSelect<? extends Record1<T>> query);
}

Now, the same set of overloads needs to be repeated for not equals, greater than, greater or equal, etc. Wouldn’t it be nice to be able to express this “right-hand-side” thingy as a single, reusable type? I.e. a union type of all of the above types?

interface Field<T> {
    Condition eq(
        Field<T>
      | T
      | Select<? extends Record1<T>>
      | QuantifiedSelect<? extends Record1<T>> thingy
    );
}

Or even

// This is called a type alias. Another awesome
// Ceylon language feature (pseudo syntax)
alias Thingy => 
    Field<T>
  | T
  | Select<? extends Record1<T>>
  | QuantifiedSelect<? extends Record1<T>>;

interface Field<T> {
    Condition eq(Thingy thingy);
}

After all, that’s also how the SQL language is defined. Heck, that’s how any BNF notation defines syntactic elements. For instance:

<predicate> ::=
    <comparison predicate>
  | <between predicate>
  | <in predicate>
  | <like predicate>
  | <null predicate>
  | <quantified comparison predicate>
  | <exists predicate>
  | <unique predicate>
  | <match predicate>
  | <overlaps predicate>

OK, granted, a syntactic element is not strictly the same thing as a type, but the intuitive perception is the same.

Oh, and Java has union types, too!

In a brief flash of revelation, the Java 7 expert groups added support for union types in exception handling. You can write things like:

try {
    ...
}
catch (IOException | SQLException e) {
    // e can be any of the above!
}

And you can emulate union types with generics, which don’t support union types but intersection types in Java.

Back to Ceylon and NULL

Ceylon has gotten Null right. Because, historically, a nullable type is a type that can be the “real” type or the “null” value. We want that. We Java developers crave that. We cannot live without the soothing option of this kind of optional.

But the excellent thing about this approach is that it is extendable. What if I really need to distinguish between “unknown”, “uninitialised”, “undefined”, “42”? I can. Using types. Here’s a String that can model all of the aforementioned “special values”:

String|Unknown|Uninitialised|Undefined|FortyTwo

And if that’s too verbose, I just assign a name to it

interface TheStringToRuleThemAll
  => String|Unknown|Uninitialised|Undefined|FortyTwo;

But it cannot be Null. Because I don’t want it to be that value, that is everything and nothing. Are you convinced? I bet you are. From now on:

Don’t trust any language that pretends that the Option(al) monad is a decent approach at modelling null. It isn’t.

― me. Just now

Why? Let me illustrate. Kotlin/Ceylon/Groovy style syntax sugar using the elvis operator (regardless of the backing null semantics):

String name = bob?.department?.head?.name

Same thing with Optional monads:

Optional<String> name = bob
    .flatMap(Person::getDepartment)
    .map(Department::getHead)
    .flatMap(Person::getName);

AND POOR YOU IF YOU MIX UP map() WITH flatMap() JUST ONCE!!

Some people claim

Using union types is like driving around in a brand new Ferrari with your mother-in-law in the passenger seat.

by Elvira

Sure. But I claim: Well done, Ceylon. Let’s hope we’ll get union types in Java, too, outside of catch blocks!

Further reading

Liked this article? How about:

A Very Peculiar, but Possibly Cunning Kotlin Language Feature

This has caught me by surprise. After studying the Kotlin language to learn about how to best leverage this interesting new language for jOOQ, I stumbled upon this puzzler. What do you think the following program will print?

fun main(args: Array) {
    (1..5).forEach {
        if (it == 3)
            return
        print(it)
    }

    print("done")
}

Well… You might have guessed wrong. The above will print:

12

It will NOT print what most people might expect:

1245done

Note to those of you who are not surprised:

The above is peculiar for someone used to working with Java 8, where the following code will indeed print 1245done:

public static void main(String[] args) {
    IntStream.rangeClosed(1, 5).forEach(it -> {
        if (it == 3)
            return;

        System.out.print(it);
    });

    System.out.print("done");
}

The syntactical reason is explained in this section of the Kotlin manual:
https://kotlinlang.org/docs/reference/returns.html

In lambdas / closures, the return statement will not (necessarily) return from the lambda / closure, but from the immediate enclosing scope of the lambda / closure. The rationale has been kindly given to me by Dmitry Jemerov from JetBrains in two tweets:

Cunningly, the Kotlin language has removed language-based support for Java constructs like try-with-resources, or the synchronized statement. That’s very reasonable, because these language constructs don’t necessarily belong in the language (as we’ve previously claimed in another blog post), but could be moved to libraries instead. For example:

// try-with-resources is emulated using an
// extension function "use"
OutputStreamWriter(r.getOutputStream()).use {
    it.write('a')
}

(criticism here)

Or:

// Synchronized is a function!
val x = synchronized (lock, { computation() })

See also:
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/synchronized.html

After all, even in Java, the language feature only works because the language depends on library types, like Iterable (foreach), AutoCloseable (try-with-resources), or JVM features (monitor on each reference for synchronized)

So, what’s the deal with return?

Along the lines of the above rationale, when language designers want to avoid language constructs for things that can be implemented with libraries, but still want you to feel like these were language constructs, then the only reasonable meaning of return inside of such a “construct-ish” lambda / closure is to return from the outer scope. So, when you write something like:

fun main(args : Array) {
    val lock = Object()
    val x = synchronized(lock, {
        if (1 == 1)
            return

        "1"
    })

    print(x)
}

The real intention is for this to be the equivalent of the following Java code:

public static void main(String[] args) {
    Object lock = new Object();
    String x;

    synchronized (lock) {
        if (1 == 1)
            return;

        x = "1";
    }

    System.out.println(x);
}

In the Java case, obviously, the return statement exits the main() method, because there is no other reasonable stack frame to return from. Unlike in Kotlin, where one might argue the lambda / closure would produce its own stack frame.

But it really doesn’t. The reason for this is the inline modifier on the synchronized function:

public inline fun <R> synchronized(lock: Any, block: () -> R): R {
    monitorEnter(lock)
    try {
        return block()
    }
    finally {
        monitorExit(lock)
    }
}

See also:
https://kotlinlang.org/docs/reference/inline-functions.html

Which means that the block closure passed as an argument isn’t really a pure lambda expression, but just syntactic sugar embedded in the call-site’s scope.

Weird. Cunning. Clever. But a bit unexpected.

Is this a good idea? Or will the language designers regret this, later on? Are all lambdas / closures potentially “language construct-ish”, where such a return statement is expected to leave the outer scope? Or are there clear cases where this inline behaviour just makes total sense?

We’ll see. In any case, it is very interesting for a language to have chosen this path.

Liked this article?

Stay tuned for a follow-up about the exciting Kotlin language. In the meantime, read